Large model comprehensive evaluation system based on artificial intelligence
By introducing adaptive vector set center adjustment and multi-dimensional evaluation mechanism, the problems of incomplete and unstable evaluation in traditional large model evaluation methods are solved, and efficient and accurate comprehensive quality evaluation is achieved.
Patent Information
- Application Number
- CN202511652143.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-12
- Publication Date
- 2026-02-10
AI Technical Summary
Traditional large model evaluation methods rely on a single metric, which makes it difficult to fully reflect model performance. The evaluation scenarios are limited and the manual cost is high. In addition, there are problems such as unstable vector set center, difficulty in automatically optimizing the number of sets, inaccurate data vector allocation, and evaluation results being easily affected by initial conditions.
An adaptive vector set center adjustment mechanism, dynamic optimization of set size, fusion similarity measurement method, and adaptive termination criterion are introduced. Combined with a multi-dimensional evaluation mechanism of feature importance, distribution rationality, and structural stability, a dynamic weighted fusion strategy is used to comprehensively score data vectors, and a deep screening mechanism is used to identify excellent data vectors.
It significantly improved the accuracy and stability of the evaluation process, enhanced the precision and reliability of the comprehensive quality assessment of large models, and optimized the accuracy and hierarchy of the evaluation results.
Smart Images
Figure CN121502255A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application relates to the technical field of large model evaluation, in particular to a large model comprehensive evaluation system based on artificial intelligence. BACKGROUND
[0002] With the rapid development of large models, the parameter quantity has reached the order of 100 billion, and the application field is continuously expanding, but the traditional evaluation method has the problems of relying on a single index, being difficult to comprehensively reflect the model performance, and having limited evaluation scenes and high labor costs; with the increasing demand of the industry for model selection, optimization and compliance, based on the automatic evaluation process and multi-dimensional evaluation framework, the key technology of constructing a comprehensive evaluation system covering performance, efficiency and security has become a technology development trend; the large model comprehensive evaluation system based on artificial intelligence solves the technical challenges of evaluation index system design and representative dataset construction, and provides scientific and efficient evaluation support for the research and application of artificial intelligence models. SUMMARY
[0003] In view of the above situation, in order to overcome the defects of the prior art, the application provides a large model comprehensive evaluation system based on artificial intelligence, which is aimed at the problems of unstable vector set center, difficult automatic optimization of set quantity, inaccurate data vector allocation and evaluation results easily affected by initial conditions in the traditional model evaluation method, and the scheme introduces an adaptive vector set center adjustment mechanism, dynamically optimizes the set quantity, fuses the similarity measurement method and the adaptive termination criterion, and significantly improves the accuracy and stability of the evaluation process; in view of the problems of incomplete feature evaluation, unreasonable distribution and difficult quantification of structural stability in the traditional deep evaluation method, the scheme introduces a multi-dimensional evaluation mechanism of feature importance, distribution rationality and structural stability, and performs comprehensive scoring on the data vector through a dynamic weighting fusion strategy, improves the comprehensive evaluation ability of the data vector quality, at the same time, through the deep screening mechanism, the excellent data vector is effectively identified, the accuracy and hierarchy of the evaluation result are optimized, and finally the comprehensive quality evaluation precision and reliability of the large model are improved.
[0004] The technical scheme adopted by the application is as follows: the large model comprehensive evaluation system based on artificial intelligence provided by the application comprises a dataset construction module, a preliminary evaluation model construction module, a deep evaluation model construction module and a comprehensive evaluation module;
[0005] The dataset construction module comprises a model metadata acquisition unit, a model performance monitoring unit and a dataset generation unit;
[0006] The preliminary evaluation model construction module comprises a vector set center adjustment unit, a set quantity updating unit, a similarity measurement unit, a data vector allocation unit, an adaptive termination unit, an iteration unit and a preliminary screening unit;
[0007] The module for constructing the in-depth evaluation model includes: a feature importance evaluation unit, a distribution rationality evaluation unit, a structural stability evaluation unit, a comprehensive scoring unit, and an in-depth screening unit;
[0008] The comprehensive evaluation module uses a preliminary evaluation model and a deep evaluation model to conduct a comprehensive evaluation of the large model.
[0009] Furthermore, the dataset construction module specifically includes the following units:
[0010] The model metadata acquisition unit collects model architecture and model parameters; the model architecture includes the model's hierarchical structure, the number of nodes in each layer, and the connection method; the model parameters include the learning rate, optimizer type, batch size, and the model's final loss value.
[0011] The model performance monitoring unit records the inference time from input data to output results, and obtains the CPU and GPU utilization during the inference process; it also obtains the throughput by calculating the ratio of the number of samples to the inference time.
[0012] The dataset generation unit uses data acquired by the model metadata acquisition unit and the model performance monitoring unit to create data vectors. The feature dimensions of the data vectors include: hierarchical structure, number of nodes in each layer, connection method, learning rate, optimizer type, batch size, CPU utilization, GPU utilization, inference time, model final loss value, and throughput. The unit creates a large model evaluation dataset by adding all the created data vectors to the large model evaluation dataset.
[0013] Furthermore, the module for constructing the preliminary evaluation model specifically includes the following units:
[0014] The vector set center adjustment unit calculates the weighted average of all data points, with data points closer to the current set center having higher weights.
[0015] The set quantity update unit introduces a mechanism for dynamically adjusting the number of centers, automatically updating the set quantity based on the data structure;
[0016] The similarity measurement unit first calculates the Euclidean distance between the two data vectors that are furthest apart. ,Will The result of dividing by the total dimension of the data vector is set as the neighborhood distance. Data vectors within the neighborhood distance range are set as neighborhood data vectors. The fusion similarity between the data vector and the center of the vector set is further calculated.
[0017] The data vector allocation unit assigns data vectors to the set of vectors with the highest fusion similarity among the data vectors.
[0018] An adaptive termination unit is introduced to calculate the set loss change value, and determine whether to stop according to the stability of the vector set center and the quality of the overall vector set;
[0019] An iteration unit is configured to set a loss threshold, first randomly select three data vectors as initial vector set centers, then adjust the vector set through the vector set center adjustment unit, update the number of sets after the adjustment, calculate the difference between the number of sets after the update and the number of sets before the update, when the difference is less than zero, randomly select set center points in the absolute value of the difference, and restore the selected set center points to data vectors; when the difference is greater than zero, randomly select data vectors in the difference, and set the selected data vectors as new set center points; then perform data vector allocation; when the set loss change value is less than the loss threshold, the iteration ends; otherwise, repeat the iteration process.
[0020] A preliminary screening unit is configured to obtain all vector sets after the iteration ends, and calculate the mean of the model final loss value and the throughput of all vectors in each set; next, the mean of the model final loss value and the mean of the throughput are sorted from high to low, respectively; then, the half of the vector sets with the lowest mean of the model final loss value and the half of the vector sets with the highest mean of the throughput are screened out; the vector sets that appear in both parts are regarded as qualified vector sets, and the remaining sets are regarded as unqualified vector sets.
[0021] Further, the deep evaluation model construction module specifically includes the following contents:
[0022] A feature importance evaluation unit is configured to assign a feature importance score to each data vector based on the model final loss value and the throughput of the data vector.
[0023] A distribution rationality evaluation unit is configured to calculate the distance between the data vector and the set mean and the difference between the feature dimension to obtain a distribution rationality score.
[0024] A structure stability evaluation unit is configured to calculate the sum of squares of distances between the data vector and its neighbors, and combine the feature variance of the data vector to evaluate its stability in the local structure.
[0025] A comprehensive scoring unit is configured to generate a comprehensive quality score of each data vector according to the scores of the data vector in feature importance, distribution rationality and structure stability through a dynamic weighting fusion strategy.
[0026] The deep screening unit takes the qualified vector set screened by the preliminary evaluation model as input data of deep screening, sets a comprehensive score threshold, calculates the comprehensive score of the data vector in all qualified vector sets, and sets the data vector as an excellent data vector when the comprehensive score of the data vector is greater than the comprehensive score threshold; otherwise, the data vector is set as a good data vector.
[0027] Further, the comprehensive evaluation module collects the hierarchical structure, the number of nodes of each layer, the connection mode, the learning rate, the optimizer type, the batch size, the CPU utilization, the GPU utilization, the inference time, the model final loss value and the throughput data of the large model to form a data vector; the data vector is first input into the preliminary evaluation model, and when the output result of the preliminary evaluation model is an unqualified vector set, it indicates that the comprehensive quality of the large model is not up to standard; when the output result of the preliminary evaluation model is a qualified vector set, the data vector needs to be input into the deep evaluation model again, and when the output result of the deep evaluation model is a good data vector, it indicates that the comprehensive quality of the large model is up to standard but does not reach the excellent level, and when the output result of the deep evaluation model is an excellent data vector, it indicates that the comprehensive quality of the large model is excellent.
[0028] The above-mentioned scheme has the following beneficial effects:
[0029] (1) In view of the problems of unstable vector set center, difficult automatic optimization of set quantity, inaccurate data vector allocation and evaluation result easily affected by initial conditions in the traditional model evaluation method, the scheme introduces an adaptive vector set center adjustment mechanism, dynamically optimizes the set quantity, fuses the similarity measurement method and the adaptive termination criterion, significantly improves the accuracy and stability of the evaluation process, and further, the set loss change value control in the iteration process and the preliminary screening mechanism ensure the reliability and efficiency of the final evaluation result.
[0030] (2) In view of the problems of incomplete feature evaluation, unreasonable distribution and difficult quantification of structural stability in the traditional deep evaluation method, the scheme introduces a multi-dimensional evaluation mechanism of feature importance, distribution rationality and structural stability, and calculates the comprehensive score of the data vector through a dynamic weighted fusion strategy, improves the comprehensive evaluation ability of the data vector quality, and further, effectively identifies excellent data vectors through the deep screening mechanism, optimizes the accuracy and hierarchy of the evaluation result, and finally improves the comprehensive quality evaluation precision and reliability of the large model. BRIEF DESCRIPTION OF DRAWINGS
[0031] Figure 1 A schematic diagram of the large model comprehensive evaluation system based on artificial intelligence provided by the present application;
[0032] Figure 2 A schematic diagram of the preliminary evaluation model module;
[0033] Figure 3 A schematic diagram for constructing a deep evaluation model module.
[0034] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof. Detailed Implementation
[0035] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.
[0036] In the description of this invention, it should be understood that the terms "upper", "lower", "front", "rear", "left", "right", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.
[0037] Example 1, see Figure 1 The large-scale model comprehensive evaluation system based on artificial intelligence provided by this invention includes a dataset construction module, a preliminary evaluation model construction module, a deep evaluation model construction module, and a comprehensive evaluation module.
[0038] The dataset construction module includes: a model metadata acquisition unit, a model performance monitoring unit, and a dataset generation unit;
[0039] The module for constructing the preliminary evaluation model includes: a vector set center adjustment unit, a set quantity update unit, a similarity measurement unit, a data vector allocation unit, an adaptive termination unit, an iteration unit, and a preliminary screening unit;
[0040] The module for constructing the in-depth evaluation model includes: a feature importance evaluation unit, a distribution rationality evaluation unit, a structural stability evaluation unit, a comprehensive scoring unit, and an in-depth screening unit;
[0041] The comprehensive evaluation module uses a preliminary evaluation model and a deep evaluation model to conduct a comprehensive evaluation of the large model.
[0042] Example 2, see Figure 1 This embodiment is based on the above embodiment, and the dataset construction module specifically includes the following units:
[0043] The model metadata acquisition unit collects model architecture and model parameters; the model architecture includes the model's hierarchical structure, the number of nodes in each layer, and the connection method; the model parameters include the learning rate, optimizer type, batch size, and the model's final loss value.
[0044] The model performance monitoring unit records the inference time from input data to output results, and obtains the CPU and GPU utilization during the inference process; it also obtains the throughput by calculating the ratio of the number of samples to the inference time.
[0045] The dataset generation unit uses data acquired by the model metadata acquisition unit and the model performance monitoring unit to create data vectors. The feature dimensions of the data vectors include: hierarchical structure, number of nodes in each layer, connection method, learning rate, optimizer type, batch size, CPU utilization, GPU utilization, inference time, model final loss value, and throughput. The unit creates a large model evaluation dataset by adding all the created data vectors to the large model evaluation dataset.
[0046] Example 3, see Figure 1 and Figure 2 This embodiment is based on the above embodiment, and the module for constructing the preliminary evaluation model specifically includes the following units:
[0047] The vector set center adjustment unit calculates the weighted average of all data points, with data points closer to the current set center having higher weights, as shown below:
[0048] ;
[0049] Where t represents the number of center adjustments, and k represents the index of the vector set. This represents the vector center of the k-th vector set at the (t+1)-th center adjustment. Indicates the index of the data vector. This represents the total number of data vectors in the k-th vector set. This represents the center adjustment weight of the i-th data vector during the t-th center adjustment. Let represent the i-th data vector, j represent the index of the center of the data vector, and K represent the total number of vectors in the set. This represents the center of the j-th vector set during the t-th center adjustment. This represents the center of the vector set to which the i-th data vector belongs at the t-th center adjustment. Indicates modulo, Represents an exponential function with the natural constant as its base;
[0050] The set quantity update unit introduces a mechanism for dynamically adjusting the number of centers, automatically updating the set quantity based on the data structure, as shown below:
[0051] ;
[0052] in, This represents the total number of vector sets at the (t+1)th center adjustment. This indicates taking the maximum value. This indicates taking the absolute value. Let N represent the total number of vector sets at the t-th center adjustment, and let N represent the total number of data vectors.
[0053] The similarity measurement unit first calculates the Euclidean distance between the two data vectors that are furthest apart. ,Will The result of dividing by the total dimension of the data vector is set as the neighborhood distance. Data vectors within the neighborhood distance range are set as neighborhood data vectors. The fusion similarity between the data vectors and the center of the vector set is further calculated, as follows:
[0054] ;
[0055] in, Let represent the fusion similarity between the center of the i-th data vector and the center of the j-th vector set, l represent the feature dimension index of the data vector, and L represent the total number of feature dimensions of the data vector. This represents the metric weight of the l-th feature dimension. This represents the feature value of the l-th dimension of the i-th data point. This represents the eigenvalue of the l-th dimension representing the center of the j-th vector set. This represents the Euclidean distance between the centers of the i-th data vector and the j-th vector set. This represents the Euclidean distance between the i-th data vector and the center of the farthest vector set;
[0056] The data vector allocation unit assigns data vectors to the set of vectors with the highest fusion similarity among the data vectors.
[0057] The adaptive termination unit introduces an adaptive termination criterion, calculates the change in set loss, and determines whether to stop based on the stability of the vector set center and the overall quality of the vector set, as shown below:
[0058] ;
[0059] in, This represents the set loss during the t-th center adjustment. This represents the set loss during the (t+1)th center adjustment. This represents the change in set loss during the t-th center adjustment;
[0060] The iterative unit sets a loss threshold. First, three data vectors are randomly selected as the initial vector set centers. Then, the vector set is adjusted using a vector set center adjustment unit. After adjustment, the number of sets is updated, and the difference between the updated and unupdated number of sets is calculated. If the difference is less than zero, a set center point with an absolute value equal to the difference is randomly selected, and the selected set center point is restored to a data vector. If the difference is greater than zero, a data vector with an absolute value equal to the difference is randomly selected, and the selected data vector is set as the new set center point. Data vector allocation then occurs. The iteration ends when the change in set loss is less than the loss threshold; otherwise, the iteration process is repeated.
[0061] After initial screening of units and iteration, all vector sets are obtained, and the mean of the model's final loss value and throughput for all vectors in each set is calculated. Next, the mean of the model's final loss value and the mean of the throughput are sorted from high to low. Then, the half of the vector sets with the lowest mean of the model's final loss value and the half of the vector sets with the highest mean of the throughput are selected. Vector sets that appear in both of these sets are considered qualified vector sets, while the remaining sets are considered unqualified vector sets.
[0062] By performing the above operations, this scheme addresses the problems of unstable vector set centers, difficulty in automatically optimizing the number of sets, inaccurate data vector allocation, and susceptibility of evaluation results to initial conditions in traditional model evaluation methods. It significantly improves the accuracy and stability of the evaluation process by introducing an adaptive vector set center adjustment mechanism, dynamically optimizing the number of sets, fusing similarity measurement methods, and an adaptive termination criterion. In addition, the set loss change value control and preliminary screening mechanism during the iteration process ensure the reliability and efficiency of the final evaluation results.
[0063] Example 4, see Figure 1 and Figure 3 This embodiment is based on the above embodiment, and the module for constructing the deep evaluation model specifically includes the following:
[0064] The feature importance evaluation unit assigns a feature importance score to each data vector based on the model's final loss value and throughput, as shown below:
[0065] ;
[0066] in, Let represent the feature importance score of the i-th data vector, and g represent the index of the data vector. This represents the Euclidean distance between the i-th data vector and the center of its nearest data vector. This represents the Euclidean distance between the g-th data vector and the center of its nearest data vector. This represents the ratio of the throughput of the i-th data vector to the final loss value of the model. This represents the ratio of the throughput of the g-th data vector to the final loss value of the model;
[0067] The distribution rationality assessment unit calculates the distribution rationality score by measuring the distance between the data vector and the set mean, as well as the differences between feature dimensions, and is represented as follows:
[0068] ;
[0069] in, This represents the distribution rationality score of the i-th data vector. Let represent the mean vector of all vectors in the vector set to which the i-th data vector belongs. Indicates the transpose symbol. Represents a data vector The value of the l-th dimension;
[0070] The structural stability assessment unit calculates the sum of squared distances between a data vector and its nearest neighbors, and combines this with the eigenvariance of the data vector to evaluate its stability within the local structure, as shown below:
[0071] ;
[0072] in, This represents the structural stability score of the i-th data vector. This represents the total number of data vectors in the vector set to which the i-th data vector belongs. The index of the data vector in the vector set to which the i-th data vector belongs. This represents the p-th data vector. This represents the feature variance of the i-th data vector;
[0073] The comprehensive scoring unit generates a comprehensive quality score for each data vector based on its scores in feature importance, distribution rationality, and structural stability using a dynamic weighted fusion strategy, as shown below:
[0074] ;
[0075] in, This represents the overall score of the i-th data vector. , and Indicates the weighting of the scores;
[0076] The deep screening unit uses the set of qualified vectors selected by the preliminary evaluation model as input data for deep screening. It sets a comprehensive score threshold and calculates the comprehensive score of the data vectors in all qualified vector sets. If the comprehensive score of a data vector is greater than the comprehensive score threshold, the data vector is set as an excellent data vector; otherwise, the data vector is set as a good data vector.
[0077] By performing the above operations, this solution addresses the problems of incomplete feature evaluation, unreasonable distribution, and difficulty in quantifying structural stability in traditional deep evaluation methods. It introduces a multi-dimensional evaluation mechanism based on feature importance, reasonable distribution, and structural stability, and uses a dynamic weighted fusion strategy to comprehensively score data vectors, thereby improving the ability to comprehensively evaluate the quality of data vectors. At the same time, the deep screening mechanism effectively identifies excellent data vectors, optimizes the accuracy and hierarchy of the evaluation results, and ultimately improves the accuracy and reliability of the comprehensive quality evaluation of large models.
[0078] Example 5, see Figure 1 This embodiment is based on the above embodiment. The comprehensive evaluation module collects data on the hierarchical structure, number of nodes in each layer, connection method, learning rate, optimizer type, batch size, CPU utilization, GPU utilization, inference time, final loss value, and throughput of the large model to form a data vector. The data vector is first input into the preliminary evaluation model. When the output result of the preliminary evaluation model is a set of unqualified vectors, it indicates that the overall quality of the large model is not up to standard. When the output result of the preliminary evaluation model is a set of qualified vectors, the data vector needs to be input into the deep evaluation model. When the output result of the deep evaluation model is a good data vector, it indicates that the overall quality of the large model meets the standard but has not reached the excellent level. When the output result of the deep evaluation model is an excellent data vector, it indicates that the overall quality of the large model is at the excellent level.
[0079] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.
[0080] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention.
[0081] The present invention and its embodiments have been described above. This description is not restrictive, and the accompanying drawings are only one embodiment of the present invention; the actual structure is not limited thereto. In conclusion, if those skilled in the art are inspired by this description and design similar structures and embodiments without departing from the spirit of the invention, such designs should fall within the protection scope of the present invention.
Claims
1. A large-scale model comprehensive evaluation system based on artificial intelligence, characterized by: It includes a dataset building module, a preliminary evaluation model building module, a deep evaluation model building module, and a comprehensive evaluation module; The dataset construction module includes: a model metadata acquisition unit, a model performance monitoring unit, and a dataset generation unit; The module for constructing the preliminary evaluation model includes: a vector set center adjustment unit, a set quantity update unit, a similarity measurement unit, a data vector allocation unit, an adaptive termination unit, an iteration unit, and a preliminary screening unit; The module for constructing the in-depth evaluation model includes: a feature importance evaluation unit, a distribution rationality evaluation unit, a structural stability evaluation unit, a comprehensive scoring unit, and an in-depth screening unit; The comprehensive evaluation module uses a preliminary evaluation model and a deep evaluation model to conduct a comprehensive evaluation of the large model.
2. The large-scale model comprehensive evaluation system based on artificial intelligence according to claim 1, characterized in that: The dataset construction module specifically includes the following units: The model metadata acquisition unit collects model architecture and model parameters; the model architecture includes the model's hierarchical structure, the number of nodes in each layer, and the connection method; the model parameters include the learning rate, optimizer type, batch size, and the model's final loss value. The model performance monitoring unit records the inference time of the model from input data to output results, and obtains the CPU utilization and GPU utilization during the inference process; Throughput is obtained by calculating the ratio of the number of samples in the inference process to the inference time; The dataset generation unit uses data acquired by the model metadata acquisition unit and the model performance monitoring unit to create data vectors. The feature dimensions of the data vectors include: hierarchical structure, number of nodes in each layer, connection method, learning rate, optimizer type, batch size, CPU utilization, GPU utilization, inference time, model final loss value, and throughput. The unit creates a large model evaluation dataset by adding all the created data vectors to the large model evaluation dataset.
3. The large-scale model comprehensive evaluation system based on artificial intelligence according to claim 1, characterized in that: The module for constructing the preliminary evaluation model specifically includes the following units: The vector set center adjustment unit calculates the weighted average of all data points, with data points closer to the current set center having higher weights. The set quantity update unit introduces a mechanism for dynamically adjusting the number of centers, automatically updating the set quantity based on the data structure; The similarity measurement unit first calculates the Euclidean distance between the two data vectors that are furthest apart. ,Will The result of dividing by the total dimension of the data vector is set as the neighborhood distance. Data vectors within the neighborhood distance range are set as neighborhood data vectors. The fusion similarity between the data vector and the center of the vector set is further calculated. The data vector allocation unit assigns data vectors to the set of vectors with the highest fusion similarity among the data vectors. The adaptive termination unit introduces an adaptive termination criterion, calculates the change in set loss, and determines whether to stop based on the stability of the vector set center and the overall quality of the vector set.
4. The large-scale model comprehensive evaluation system based on artificial intelligence according to claim 1, characterized in that: The module for constructing the preliminary evaluation model specifically includes the following units: The iterative unit sets a loss threshold. First, three data vectors are randomly selected as the initial vector set centers. Then, the vector set is adjusted using a vector set center adjustment unit. After adjustment, the number of sets is updated, and the difference between the updated and unupdated number of sets is calculated. If the difference is less than zero, a set center point with an absolute value equal to the difference is randomly selected, and the selected set center point is restored to a data vector. If the difference is greater than zero, a data vector with an absolute value equal to the difference is randomly selected, and the selected data vector is set as the new set center point. Data vector allocation then occurs. The iteration ends when the change in set loss is less than the loss threshold; otherwise, the iteration process is repeated. After initial screening of units and iteration, all vector sets are obtained, and the mean of the model's final loss value and throughput for all vectors in each set is calculated. Next, the mean of the model's final loss value and the mean of the throughput are sorted from high to low. Then, the half of the vector sets with the lowest mean of the model's final loss value and the half of the vector sets with the highest mean of the throughput are selected. Vector sets that appear in both of these sets are considered qualified vector sets, while the remaining sets are considered unqualified vector sets.
5. The large-scale model comprehensive evaluation system based on artificial intelligence according to claim 1, characterized in that: The module for constructing a deep evaluation model specifically... Includes the following: The feature importance evaluation unit assigns a feature importance score to each data vector based on the model's final loss value and throughput. The distribution rationality assessment unit calculates the distribution rationality score by measuring the distance between the data vector and the set mean, as well as the differences between feature dimensions. The structural stability assessment unit calculates the sum of squared distances between a data vector and its nearest neighbors, and combines the characteristic variance of the data vector to assess its stability in the local structure. The comprehensive scoring unit generates a comprehensive quality score for each data vector based on its scores in feature importance, distribution rationality, and structural stability through a dynamic weighted fusion strategy.
6. The large-scale model comprehensive evaluation system based on artificial intelligence according to claim 1, characterized in that: The module for constructing a deep evaluation model specifically includes the following: The deep screening unit uses the set of qualified vectors selected by the preliminary evaluation model as input data for deep screening. It sets a comprehensive score threshold and calculates the comprehensive score of the data vectors in all qualified vector sets. If the comprehensive score of a data vector is greater than the comprehensive score threshold, the data vector is set as an excellent data vector; otherwise, the data vector is set as a good data vector.
7. The large-scale model comprehensive evaluation system based on artificial intelligence according to claim 1, characterized in that: The comprehensive evaluation module collects data on the hierarchical structure, number of nodes in each layer, connection method, learning rate, optimizer type, batch size, CPU utilization, GPU utilization, inference time, final loss value, and throughput of the large model, forming a data vector. This data vector is first input into the preliminary evaluation model. If the preliminary evaluation model outputs a set of unqualified vectors, it indicates that the overall quality of the large model is substandard. If the preliminary evaluation model outputs a set of qualified vectors, the data vector is then input into the deep evaluation model. If the deep evaluation model outputs good data vectors, it indicates that the overall quality of the large model meets the standard but has not reached the excellent level. If the deep evaluation model outputs excellent data vectors, it indicates that the overall quality of the large model is excellent.