AI algorithm model training management system

Abstract

The invention relates to the technical field of model training, in particular to an AI algorithm model training management system, which comprises the steps of analyzing a data value association matching degree in an AI algorithm model training process based on training data information and model deployment environment information of an AI algorithm model in the model training process; in combination with training configuration information and model architecture information of the AI algorithm model in the model training process, the effective utilization degree of computing resources in the AI algorithm model training process is analyzed; evaluating the training effect of the AI algorithm model based on the data value association matching degree and the effective utilization degree of the computing resources; according to the training effect evaluation result, the training scheme of the AI algorithm model is adjusted, the reliability and efficiency of the training process of the AI algorithm model can be remarkably improved, and the high performance and high robustness of the AI algorithm model in a real scene are ensured.

Description

Technical Field

[0001] This invention relates to the field of model training technology, and in particular to an AI algorithm model training and management system. Background Technology

[0002] As the core of intelligent decision-making in research fields such as intelligent manufacturing, smart healthcare, and autonomous driving, the scientific nature and reliability of AI algorithm models during training directly determine the performance and stability of artificial intelligence systems in practical applications. With the continuous expansion of deep learning model scale and the increasing complexity of application scenarios, model training has transformed from early experimental attempts into a systems engineering project requiring precise management. Throughout the entire training cycle, models are not only affected by the quality of training data, gradually introducing distribution bias and noise interference, leading to defects in model representation capabilities; moreover, improper allocation of training resources can cause the model optimization process to progress from a rapid decline in loss in the initial stage, to gradient oscillation and fluctuation in the middle stage, and finally to degradation in generalization performance in the later stage. Traditional AI model training management relies heavily on manual experience and static monitoring, which is not only slow to respond but also difficult to achieve comprehensive coverage and precise control when facing high-dimensional data distributions, complex model architectures, and dynamic resource environments.

[0003] Meanwhile, existing technologies for managing AI model training fail to correlate the statistical characteristics of training data with the real-world dynamic environment in which the model will be deployed. For example, when there are distributional differences between training data and the current production environment, without considering the coverage of edge cases and the representativeness of sparse regions in the feature space, it is difficult to accurately assess the model's generalization ability and robustness in real-world key scenarios, leading to overly optimistic estimates of model performance. Furthermore, existing technologies lack correlation analysis between training resource allocation and the theoretical performance ceiling of the model. This makes it difficult for resource allocation schemes to accurately support the optimization needs of the model architecture and to provide a comprehensive basis for adjusting training strategies, rendering existing training management methods inadequate for handling complex and ever-changing real-world application needs.

[0004] To address these issues, this application designs an AI algorithm model training and management system. Summary of the Invention

[0005] The purpose of this invention is to provide an AI algorithm model training and management system. By constructing a data value correlation matching degree analysis model and a computing resource effective utilization degree analysis model, the system quantitatively evaluates the environmental adaptability of training data and the rationality of resource allocation. Based on this, a comprehensive evaluation and dynamic adjustment mechanism for training effect is established to achieve synergistic optimization of data value and computing resources, thereby ensuring the reliability of the model training process and its high performance in real-world scenarios.

[0006] This invention is implemented as follows: In a first aspect, this invention provides an AI algorithm model training and management system, including a multi-source information acquisition module, a training data analysis module, a resource utilization analysis module, a training effect evaluation module, and a training scheme adjustment module. The multi-source information acquisition module is used to acquire training data information, training configuration information, and real-time system monitoring information of the AI ​​algorithm model during model training, while also acquiring model architecture information and model deployment environment information of the AI ​​algorithm model. The training data analysis module is used to construct a data value association matching degree analysis model based on the training data information and model deployment environment information of the AI ​​algorithm model during model training, and to analyze the data value association matching degree during the AI ​​algorithm model training process. The system comprises four modules: a resource utilization analysis module, a training configuration module, and a training effect evaluation module. The resource utilization analysis module combines the training configuration and architecture information of the AI ​​algorithm model during training to construct a computational resource utilization effectiveness analysis model, which analyzes the effective utilization of computational resources during the AI ​​algorithm model training process. The training effect evaluation module, based on the data value association matching degree analysis results and the computational resource utilization effectiveness analysis results during the AI ​​algorithm model training process, constructs an AI algorithm model training effect evaluation model, which evaluates the training effect during the AI ​​algorithm model training process. The training scheme adjustment module adjusts the training scheme of the AI ​​algorithm model based on the training effect evaluation results during the AI ​​algorithm model training process.

[0007] As a preferred embodiment of the present invention, a data value association matching degree analysis model is constructed to analyze the data value association matching degree during the training process of the AI ​​algorithm model, including the following specific steps: S21. Extract training data information and model deployment environment information of the AI ​​algorithm model during model training; S22. Import the training data information into the data value density analysis strategy to analyze the training data value density of the AI ​​algorithm model during the model training process. The training data value density is quantified by calculating the sparse coverage of the training data in the feature space; specifically, the analysis steps of the data value density analysis strategy are as follows: S221. For each sample in the training dataset, calculate the average Euclidean distance to its k nearest neighbors, where the value of k is the square root of the total number of samples in the training dataset and rounded down. S222. Calculate the mean and standard deviation of the distances from the mean to all samples, and identify samples whose distances from the mean are greater than the sum of the mean and a standard deviation as high-value sparse samples. S223. The ratio of the number of high-value sparse samples to the total number of samples in the training dataset is used as the training data value density of the AI ​​algorithm model during model training. S23. Import the training data information and model deployment environment information into the environment evolution and adaptation analysis strategy to analyze the environment evolution and adaptation of the AI ​​algorithm model during the model training process. The environmental evolution adaptation is quantified by evaluating the distribution matching degree between training data and deployment environment data; specifically, the specific analysis steps of the environmental evolution adaptation analysis strategy are as follows: S231. For each feature dimension, the feature value range of each feature dimension is divided into ten equally wide intervals. The proportion of data points in each interval of the training data and the deployment environment data to the total data points is calculated as the frequency value of the training data and the deployment environment data in each interval. S232. For each equal-width interval, take the minimum value between the frequency value of the training data in each equal-width interval and the frequency value of the deployment environment data in each equal-width interval, and add the minimum values ​​of these ten equal-width intervals to obtain the overlap of each feature dimension. S233. Take the average value of the overlap of all feature dimensions as the environmental evolution and adaptation of the AI ​​algorithm model during model training. S24. The weighted summation of the training data value density and environmental evolution adaptation of the AI ​​algorithm model during the model training process is performed to obtain the degree of data value correlation and matching during the AI ​​algorithm model training process.

[0008] As a preferred embodiment of the present invention, a computing resource utilization efficiency analysis model is constructed to analyze the utilization efficiency of computing resources during the training process of AI algorithm models, including the following specific steps: S31. Extract the training configuration information, real-time system monitoring information, and model architecture information of the AI ​​algorithm model during the model training process; S32. Import the training configuration information and model architecture information into the model accuracy upper limit level analysis strategy to analyze the model accuracy upper limit level during the AI ​​algorithm model training process; The upper limit of model accuracy is predicted by comparing model complexity with available computing resources; specifically, the analysis steps of the upper limit of model accuracy analysis strategy are as follows: S321. Calculate the minimum memory estimate required for model training, wherein the minimum memory estimate is obtained by adding the product of the batch size and the forward propagation activation memory estimate per sample to four times the number of model parameters; S322. When the available GPU memory is less than the minimum memory estimate required for model training, the upper limit of model accuracy during AI algorithm model training is zero. Otherwise, calculate the minimum training time estimate required for model training. The minimum training time estimate is obtained by dividing the product of the training dataset size and the number of model parameters by the estimated throughput per unit time. S323. When the available training time is less than the estimated minimum training time required for model training, the ratio of the available training time to the estimated minimum training time required for model training shall be used as the upper limit of model accuracy during the AI ​​algorithm model training process; otherwise, the upper limit of model accuracy shall be 1. S33. Import the real-time monitoring information of the system into the training elasticity state analysis strategy to analyze the training elasticity state during the training process of the AI ​​algorithm model. The training elasticity state is assessed by monitoring resource utilization in real-time information from the monitoring system; specifically, the analysis content of the training elasticity state analysis strategy is as follows: The GPU memory utilization and CPU utilization are obtained in real time during the current training process. The average value of GPU memory utilization and CPU utilization is subtracted from the value of 1 to obtain the training elasticity state during the AI ​​algorithm model training process. S34. The upper limit of model accuracy and the training elasticity state during the AI ​​algorithm model training process are weighted and summed to obtain the effective utilization of computing resources during the AI ​​algorithm model training process.

[0009] As a preferred embodiment of the present invention, an AI algorithm model training effect evaluation model is constructed to evaluate the training effect during the AI ​​algorithm model training process, including the following specific steps: S41. Extract the degree of data value correlation and matching and the degree of effective utilization of computing resources during the training process of AI algorithm models; S42. The weighted sum of the data value association matching degree and the effective utilization of computing resources is used to obtain the training effect in the AI ​​algorithm model training process.

[0010] In a preferred embodiment of the present invention, the training scheme of the AI ​​algorithm model is adjusted based on the training effect evaluation results during the AI ​​algorithm model training process, including the following specific contents: The system acquires the training effect during the AI ​​algorithm model training process and presets a training effect threshold. When the training effect during the AI ​​algorithm model training process is greater than or equal to the training effect threshold, the current training scheme of the AI ​​algorithm model is deemed to be up to standard. When the training effect during the AI ​​algorithm model training process is less than the training effect threshold, the training scheme adjustment process is triggered.

[0011] Secondly, this invention provides an AI algorithm model training management method, including the following specific steps: Acquire training data, training configuration information, and real-time system monitoring information of the AI ​​algorithm model during the model training process, and at the same time acquire model architecture information and model deployment environment information of the AI ​​algorithm model; Based on the training data and deployment environment information of the AI ​​algorithm model during the model training process, a data value association matching degree analysis model is constructed to analyze the data value association matching degree during the AI ​​algorithm model training process; By combining the training configuration information and model architecture information of the AI ​​algorithm model during the model training process, a model for analyzing the effective utilization of computing resources is constructed to analyze the effective utilization of computing resources during the AI ​​algorithm model training process. Based on the analysis results of the data value correlation and matching degree and the analysis results of the effective utilization of computing resources during the training process of AI algorithm model, an AI algorithm model training effect evaluation model is constructed to evaluate the training effect during the training process of AI algorithm model; Based on the evaluation results of the training effect during the AI ​​algorithm model training process, the training scheme of the AI ​​algorithm model is adjusted.

[0012] Thirdly, the present invention provides an electronic device, comprising: a processor and a memory, wherein the memory stores a computer program that can be called by the processor, and the processor executes an AI algorithm model training management method by calling the computer program stored in the memory.

[0013] Compared with the prior art, the present invention has the following advantages and beneficial effects: 1. This invention, by quantitatively analyzing the degree of correlation and matching between training data and the real value of the target domain, can effectively identify high-value data samples that make a decisive contribution to the prediction task, ensure the high adaptability of training data distribution to dynamic deployment environment, reduce the model generalization failure caused by data environment drift or insufficient coverage of edge cases, and improve the decision reliability of AI algorithms in actual key scenarios. 2. By conducting a detailed analysis of the effective utilization of computing resources during the training process, this invention can accurately predict the theoretical upper limit of model accuracy and the elasticity of the training process under a given resource configuration, thereby ensuring the optimal match between computing resource input and model performance output, reducing the risk of training interruption or model underoptimization caused by resource bottlenecks, and significantly improving the input-output efficiency of large-scale model training. 3. This invention constructs a comprehensive evaluation system for data value and resource utilization, enabling real-time monitoring and diagnosis of the training process. It can promptly detect and warn of training deviations and resource mismatches, reducing model performance defects caused by uncontrolled training processes. Attached Figure Description

[0014] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1This is a schematic diagram of the overall process of the AI ​​algorithm model training management method of the present invention; Figure 2 This is a schematic diagram of the AI ​​algorithm model training management system of the present invention; Figure 3 This is an analysis flowchart of step S2 in the AI ​​algorithm model training management method of the present invention; Figure 4 This is an analysis flowchart of step S3 of the AI ​​algorithm model training management method of the present invention. Detailed Implementation

[0015] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations thereof. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.

[0016] Example 1

[0017] like Figure 1 As shown, this embodiment provides an AI algorithm model training management method, including the following specific steps: S1. Acquire training data, training configuration information, and real-time system monitoring information of the AI ​​algorithm model during model training. Simultaneously, acquire model architecture information and model deployment environment information of the AI ​​algorithm model. Specifically, acquire the feature vectors and label distribution of the training dataset in real time through the data acquisition interface, including feature dimensions, number of samples, and data distribution statistics as training data information; extract batch size, learning rate, optimizer type, and number of training iterations from the training logs as training configuration information; parse the number of model layers, number of parameters, activation function type, and connection topology from the model definition file as model architecture information; and acquire the data flow statistics, hardware resource specifications, and real-time load status of the production environment from the deployment monitoring system as model deployment environment information.

[0018] S2. Based on the training data information and model deployment environment information of the AI ​​algorithm model during the model training process, construct a data value association matching degree analysis model to analyze the data value association matching degree during the AI ​​algorithm model training process; S3. Combining the training configuration information and model architecture information of the AI ​​algorithm model during the model training process, construct a model to analyze the effective utilization of computing resources and analyze the effective utilization of computing resources during the AI ​​algorithm model training process. S4. Based on the analysis results of the data value association matching degree and the analysis results of the effective utilization of computing resources during the AI ​​algorithm model training process, construct an AI algorithm model training effect evaluation model to evaluate the training effect during the AI ​​algorithm model training process; S5. Adjust the training scheme of the AI ​​algorithm model based on the evaluation results of the training effect during the training process.

[0019] In this embodiment, as Figure 3 As shown, step S2 involves constructing a data value association matching degree analysis model to analyze the data value association matching degree during the AI ​​algorithm model training process. This includes the following specific steps: S21. Extract training data information and model deployment environment information of the AI ​​algorithm model during model training. It should be noted that in this embodiment, the extraction of training data information needs to be achieved in real-time through a dedicated data acquisition interface. Specifically, feature vectors need to be extracted from each sample in the training dataset, covering the specific values ​​of the sample in each feature dimension. For example, in image recognition model training, feature vectors may include pixel values, edge features, texture features, etc. Label distribution requires statistical analysis of the proportion of different category labels in the dataset, such as the number of samples of each category and their proportion of the total number of samples in a classification task. Obtaining feature dimensions requires traversing all feature fields of the training dataset, clarifying the specific meaning and data type of each feature, such as numerical or categorical. The number of samples is obtained by counting all valid samples in the training dataset. Data distribution statistics include the mean, variance, maximum, and minimum values ​​of each feature, obtained through statistical calculations of the feature data. These statistics reflect the overall distribution trend of the features. In this embodiment, the extraction of model deployment environment information relies on the deployment monitoring system. Data flow statistics require the collection of indicators such as data transmission volume per unit time, data processing latency, and data format conversion frequency in the production environment, for example, the amount of request data processed per hour and the average latency from data generation to input into the model. Hardware resource specifications require the acquisition of hardware parameters such as the CPU model, number of cores, memory capacity, GPU model, video memory size, and number of computing cores of the servers in the deployment environment. Real-time load status needs to be sampled at fixed time intervals (such as every 10 seconds), including indicators such as CPU utilization, memory usage, GPU video memory utilization, and GPU computing utilization, to reflect the real-time usage status of hardware resources.

[0020] S22. Import the training data information into the data value density analysis strategy to analyze the training data value density of the AI ​​algorithm model during the model training process. The training data value density is quantified by calculating the sparse coverage of the training data in the feature space; specifically, the analysis steps of the data value density analysis strategy are as follows: S221. For each sample in the training dataset, calculate the average Euclidean distance to its k nearest neighbors, where the value of k is the square root of the total number of samples in the training dataset and rounded down. S222. Calculate the mean and standard deviation of the distances from the mean to all samples, and identify samples whose distances from the mean are greater than the sum of the mean and a standard deviation as high-value sparse samples. S223. The ratio of the number of high-value sparse samples to the total number of samples in the training dataset is used as the training data value density of the AI ​​algorithm model during model training. It should be noted that in this embodiment, the training data information is imported into the data value density analysis strategy to analyze the training data value density of the AI ​​algorithm model during the model training process. This density is quantified by the sparse coverage of the training data in the feature space to accurately identify high-value training data. First, when calculating the average Euclidean distance from each sample to its k nearest neighbors, the process of obtaining the value of k is as follows: First, the total number of samples is counted from the metadata of the training dataset or through a data loading tool. For example, after reading the dataset through the pandas library in Python, the shape[0] function is used to obtain the number of samples. Then, the square root of the total number of samples is taken, and finally, the k value is determined by rounding down through a mathematical function (such as the floor function of numpy). The core reason for using this method to determine the k value in this embodiment is that the square root can dynamically balance the number of nearest neighbor samples when the number of samples changes, avoiding the situation where the k value is too large, resulting in too much similar data in the neighbor samples and the inability to effectively distinguish sparse samples, or the k value is too small, resulting in the instability of the distance calculation results due to the influence of individual abnormal samples. For example, when the total number of samples is 100, the square root is approximately 10, rounded down to 10. At this point, the number of nearest neighbors for each sample is sufficient to cover enough reference samples without introducing excessive redundant information. If the total number of samples is 1000, the square root is approximately 31.62, rounded down to 31, maintaining the same fit between the number of neighbors and the sample size. Next, the mean and standard deviation of the distances to the average distances of all samples are calculated: first, the average Euclidean distances of all samples are summed, and the arithmetic mean of this set is calculated using statistical tools to reflect the overall level of distances across all samples; then, the standard deviation is calculated to measure the dispersion of the average distance. Samples whose average distance is greater than the sum of the mean and a standard deviation are identified as high-value sparse samples. This effectively filters out samples that deviate from the overall data distribution. These samples are located in relatively sparse regions in the feature space and often contain unique feature information, which is crucial for improving the model's generalization ability. Therefore, using only the mean as a standard can misclassify many ordinary samples in dense regions as high-value samples, while adding the standard deviation allows for more accurate location of sparse samples. Finally, the ratio of the number of high-value sparse samples to the total number of samples in the training dataset is used as the training data value density. This provides a direct quantification of the proportion of high-value samples in the training data, facilitating subsequent analysis in conjunction with other indicators. This embodiment uses a scientific quantitative method to screen high-value samples from massive training data, avoiding inefficient training or insufficient model generalization due to the use of low-value or redundant data during model training. Simultaneously, a clear data value density indicator provides a basis for data screening and supplementation. For example, when the value density is too low, high-value sparse samples can be selectively added to improve the quality of the training data, thereby providing a reliable core indicator for subsequent analysis of data value association matching.

[0021] S23. Import the training data information and model deployment environment information into the environment evolution and adaptation analysis strategy to analyze the environment evolution and adaptation of the AI ​​algorithm model during the model training process. The environmental evolution adaptation is quantified by evaluating the distribution matching degree between training data and deployment environment data; specifically, the specific analysis steps of the environmental evolution adaptation analysis strategy are as follows: S231. For each feature dimension, the feature value range of each feature dimension is divided into ten equally wide intervals. The proportion of data points in each interval of the training data and the deployment environment data to the total data points is calculated as the frequency value of the training data and the deployment environment data in each interval. S232. For each equal-width interval, take the minimum value between the frequency value of the training data in each equal-width interval and the frequency value of the deployment environment data in each equal-width interval, and add the minimum values ​​of these ten equal-width intervals to obtain the overlap of each feature dimension. S233. Take the average value of the overlap of all feature dimensions as the environmental evolution and adaptation of the AI ​​algorithm model during model training. It should be noted that this embodiment imports training data information and model deployment environment information into the environment evolution adaptation analysis strategy. By evaluating the distribution matching degree between training data and deployment environment data, the environmental evolution adaptation of the AI ​​algorithm model during model training is quantified. First, equal-width intervals are divided for each feature dimension: the maximum and minimum values ​​of the feature dimension are extracted from the training data information, the difference between the two is calculated to determine the feature value range, and then this range is evenly divided into ten equal-width intervals. This embodiment divides each feature dimension into ten intervals. On the one hand, it can achieve a fine characterization of the feature value distribution and avoid the loss of distribution details due to too few intervals (for example, dividing into only 2 intervals cannot reflect the distribution differences in the middle area). On the other hand, it can prevent the data volume of some intervals from being too small due to too many intervals, resulting in large errors when calculating the frequency values ​​(for example, dividing into 100 intervals may result in a large number of empty intervals), ensuring the accuracy and reliability of subsequent frequency calculations. Furthermore, this embodiment counts the number of data points falling into each equal-width interval in both the training data and the deployment environment data. This number is then divided by the total number of data points in the corresponding dataset to obtain the frequency value for each interval. This frequency value reflects the density of data distribution within the corresponding interval. This embodiment uses the minimum value for summation to obtain the overlap of each feature dimension. It should be noted that the minimum value truly reflects the degree of common coverage between the two in that interval. If the training data has a high frequency in a certain interval but the deployment environment data has a low frequency, it indicates that the training data for that interval cannot match the deployment environment requirements. Taking the minimum value avoids misjudgment of adaptation due to high frequency in one aspect. Summing the minimum values ​​of the ten intervals comprehensively reflects the overall overlap of the distribution of the two under that feature dimension, avoiding the one-sidedness of single-interval analysis. In this embodiment, since different feature dimensions have equally important effects on model deployment performance, taking the average value comprehensively reflects the adaptation information of all feature dimensions, ensuring the comprehensiveness of the analysis results. If the adaptation degree of one feature dimension is low but other dimensions are high, the average value balances the influence of each dimension, avoiding errors in overall adaptation judgment due to deviations in individual dimensions. By conducting refined distribution matching analysis in step S23, the distribution differences between training data and deployment environment data are identified in advance. This embodiment can avoid the model's prediction accuracy from decreasing or its operation becoming unstable in the production environment due to data distribution mismatch. For example, if the deployment environment data has a high frequency in a certain feature interval while the training data has a low frequency in that interval, the model's prediction ability in that interval will be weak. This step can promptly identify and supplement the training data in the corresponding interval, improve the model's adaptability to the deployment environment, and provide key environmental indicators for subsequent data value association matching degree analysis.

[0022] S24. The weighted summation of the training data value density and environmental evolution adaptation of the AI ​​algorithm model during the model training process is performed to obtain the degree of data value correlation and matching during the AI ​​algorithm model training process.

[0023] It should be noted that step S24 of this embodiment comprehensively evaluates the synergistic effect of training data value and environmental adaptability. First, the weight coefficients in this embodiment are determined through a combination of historical case analysis and expert experience. For example, historical training data of similar models are collected, and the correlation between the degree of data value association matching and the final performance of the model under different weights is analyzed. The weight combination with the highest correlation is selected, thus ensuring the rationality of the weight setting. Next, the training data value density is multiplied by its corresponding weight, and the environmental evolution adaptability is multiplied by its corresponding weight to obtain a specific value of the data value association matching degree. The higher the value, the better the synergistic effect of training data value and environmental adaptability. In this embodiment, the impact of training data value density and environmental evolution adaptability on data value association is not equally important. A single indicator cannot fully reflect the matching degree between data and model deployment requirements. If only data value density is considered, deployment problems caused by insufficient environmental adaptability may be ignored; while if only environmental adaptability is considered, low-value data may be used, leading to poor model performance. This embodiment uses a weighted summation to balance the influence of both, achieving a more accurate comprehensive evaluation. This provides core data support for evaluating the training effect of subsequent AI algorithm models. By quantifying the synergistic effect of data value and environment adaptation, it helps R&D personnel quickly determine whether the current training data can support the effective operation of the model in the deployment environment. For example, when the correlation between data value and matching is low, it can be determined whether the data value is insufficient or the environment adaptability is poor. Then, targeted measures such as supplementing high-value data or adjusting the deployment environment adaptation strategy can be taken to avoid blindly adjusting the training plan and wasting resources, thereby improving the relevance and efficiency of model training.

[0024] In this embodiment, as Figure 4 As shown, step S3 involves constructing a computational resource utilization analysis model to analyze the effective utilization of computational resources during the AI ​​algorithm model training process. This includes the following specific steps: S31. Extract the training configuration information, real-time system monitoring information, and model architecture information of the AI ​​algorithm model during the model training process; It should be noted that in this embodiment, the training configuration information needs to be extracted from the training logs. Training logs typically record the entire training process data. Core configuration parameters such as batch size, learning rate, optimizer type, and number of training iterations are extracted using log parsing tools (e.g., the ELK log analysis system). The batch size is read from the "batch_size" field in the log, the learning rate is obtained from the "learning_rate" field, the optimizer type is determined by identifying the optimizer initialization statement (e.g., "optimizer=Adam()") in the log, and the number of training iterations is counted from the "epoch" field. In this embodiment, the extraction of real-time monitoring information relies on multi-dimensional monitoring tools. GPU memory utilization can be collected in real-time using GPU monitoring tools, sampled every 5 seconds and averaged to avoid the impact of instantaneous fluctuations; CPU utilization can be collected using CPU monitoring tools, also using periodic sampling and averaging. Extracting model architecture information requires parsing the model definition file. Specifically, from model definition files generated by frameworks such as PyTorch and TensorFlow, the number of model layers, the number of model parameters, activation function types, and connection topology are parsed using the framework's built-in model parsing interface. The number of model layers is determined by counting the number of network layer definition statements, and the number of model parameters is calculated by traversing the parameters of each layer and counting the total number of elements. In this embodiment, step S31 ensures that subsequent analysis of the effective utilization of computing resources combines the model's configuration requirements, system resource status, and architectural characteristics, avoiding judgment biases caused by single-dimensional analysis. For example, if only training configuration information is extracted while ignoring real-time system monitoring information, it is impossible to determine whether the current resources can support the operation of the configured parameters; if model architecture information is missing, it is impossible to assess the matching between model complexity and resource requirements. At the same time, automated information extraction methods can reduce human error, improve data acquisition efficiency, and provide a reliable data foundation for subsequent analysis of the upper limit of model accuracy and training elasticity, ensuring the accuracy and practicality of the entire analysis of the effective utilization of computing resources.

[0025] S32. Import the training configuration information and model architecture information into the model accuracy upper limit level analysis strategy to analyze the model accuracy upper limit level during the AI ​​algorithm model training process; The upper limit of model accuracy is predicted by comparing model complexity with available computing resources; specifically, the analysis steps of the upper limit of model accuracy analysis strategy are as follows: S321. Calculate the minimum memory estimate required for model training, wherein the minimum memory estimate is obtained by adding the product of the batch size and the forward propagation activation memory estimate per sample to four times the number of model parameters; S322. When the available GPU memory is less than the minimum memory estimate required for model training, the upper limit of model accuracy during AI algorithm model training is zero. Otherwise, calculate the minimum training time estimate required for model training. The minimum training time estimate is obtained by dividing the product of the training dataset size and the number of model parameters by the estimated throughput per unit time. S323. When the available training time is less than the estimated minimum training time required for model training, the ratio of the available training time to the estimated minimum training time required for model training shall be used as the upper limit of model accuracy during the AI ​​algorithm model training process; otherwise, the upper limit of model accuracy shall be 1. It should be noted that this embodiment predicts the upper limit of model accuracy during AI algorithm model training by comparing model complexity with available computing resources. Specifically, this embodiment calculates the minimum memory estimate required for model training. The forward propagation activation memory estimate per sample is determined based on the characteristics of the model architecture. For example, convolutional neural networks (CNNs) need to consider the feature map size and number of channels, while recurrent neural networks (RNNs) need to consider the sequence length and hidden layer dimension. This can be obtained statistically during small-scale test training using the framework's built-in memory analysis tools (such as PyTorch's torch.cuda.memory_summary()). The number of model parameters is directly obtained from the model architecture information extracted from S31. Model parameters are usually stored in 32-bit floating-point format, with each floating-point number occupying 4 bytes of memory. This embodiment uses 4 times the number of model parameters to accurately estimate the minimum memory required for parameter storage, avoiding memory overflow errors during training due to insufficient memory estimation. Furthermore, in this embodiment, the available GPU memory is obtained through the real-time system monitoring information extracted in S31. If the available GPU memory is less than the minimum memory estimate, it indicates that the current hardware resources cannot meet the basic memory requirements for model training, and the model cannot run normally. Therefore, the upper limit of model accuracy is set to 0. If the available GPU memory meets the requirements, the minimum training time estimate required for model training is calculated. In this embodiment, the training dataset size is obtained from the training data information extracted in S21. The estimated throughput per unit time is obtained based on hardware performance and model type testing. For example, small-scale training of the model is run on the target GPU, and the number of parameter calculations completed per unit time (e.g., floating-point operations per second, FLOPS) is statistically analyzed. In this embodiment, training time is positively correlated with the amount of data and the amount of parameter calculations. The larger the amount of data and the more parameters, the more calculations are required. Training time is negatively correlated with the computational throughput. The higher the throughput, the greater the computational load per unit time. This embodiment uses the above calculation formula to reasonably estimate the minimum time required to complete model training, avoiding training interruption due to time estimation deviation. In this embodiment, the available training time is preset according to the project progress and resource scheduling plan. If the available training time is less than the minimum estimated training time, it means that the complete training cannot be completed within the specified time. In this case, the ratio of the available training time to the minimum estimated training time is used as the upper limit of the model accuracy, because the shorter the training time, the lower the convergence of the model, and the upper limit of accuracy is positively correlated with the time ratio. If the available training time meets the requirements, it means that the complete training can be completed and the model can reach the theoretical maximum accuracy. Therefore, the upper limit of accuracy is set to 1.This embodiment uses scientific resource and time matching analysis to objectively predict the highest accuracy the model can achieve, avoiding researchers from blindly pursuing excessively high accuracy when resources are insufficient or time is limited, which could lead to training failure or resource waste. For example, when available GPU memory is insufficient, the model architecture can be adjusted in advance (e.g., reducing the number of parameters) or the batch size can be reduced to avoid frequent memory overflows during training. When available time is insufficient, a reasonable accuracy target can be set to prioritize training completion, providing key indicators of accuracy for subsequent analysis of the effective utilization of computing resources.

[0026] S33. Import the real-time monitoring information of the system into the training elasticity state analysis strategy to analyze the training elasticity state during the training process of the AI ​​algorithm model. The training elasticity state is assessed by monitoring resource utilization in real-time information from the monitoring system; specifically, the analysis content of the training elasticity state analysis strategy is as follows: The GPU memory utilization and CPU utilization during the current training process are obtained in real time. The average value of GPU memory utilization and CPU utilization is subtracted from the value of 1 to obtain the training elasticity state during the AI ​​algorithm model training process. First, in this embodiment, the GPU memory utilization and CPU utilization during the current training process are obtained in real time through system real-time monitoring information: GPU memory utilization is collected periodically through the NVIDIA-SMI tool. When collecting, the GPU memory occupied by other processes in the system needs to be excluded, and only the proportion of memory occupied by the model training process is counted. CPU utilization is collected through the system task manager or the top command. Similarly, only the CPU usage ratio of the model training-related processes (in this embodiment, related processes include but are not limited to training script processes and data loading processes) is counted to avoid other irrelevant processes (such as system background processes) from affecting the accuracy of utilization statistics. Since GPU and CPU are the core hardware resources for model training, their utilization rates together reflect the overall resource usage of the system. This embodiment uses an average value to balance the influence of both, avoiding misjudgments of elasticity due to excessively high or low utilization of a single resource. For example, if GPU utilization is as high as 90% but CPU utilization is only 30%, looking only at GPU utilization might lead to the conclusion of poor elasticity, but the average value (60%) more objectively reflects the overall resource usage. Furthermore, in this embodiment, the higher the average utilization rate, the less remaining system resources there is, and the smaller the adjustable elasticity during training; conversely, the lower the average value, the more remaining space there is, and the greater the elasticity. Subtracting the average value from 1 transforms resource usage into an intuitive elasticity indicator; the higher the value, the better the elasticity. For example, when the average value is 70%, the elasticity is 30%, indicating that 30% of the resources are still available to cope with resource fluctuations during training (such as a sudden increase in data volume). This embodiment helps developers adjust training strategies in a timely manner by monitoring the elasticity of system resources in real time, avoiding waste due to low resource utilization or system lag due to high resource utilization. For example, when the training elasticity is too low, the batch size can be reduced to decrease GPU memory usage, or the data loading process can be optimized to reduce CPU utilization, releasing some resources to improve elasticity and ensure the stability of the training process. When the elasticity is too high, the batch size can be appropriately increased or multi-process data loading can be enabled to improve resource utilization, avoid resource waste, and provide key indicators of the elasticity dimension for subsequent analysis of the effective utilization of computing resources.

[0027] S34. The upper limit of model accuracy and the training elasticity state during the AI ​​algorithm model training process are weighted and summed to obtain the effective utilization of computing resources during the AI ​​algorithm model training process.

[0028] It should be noted that this embodiment integrates the upper limit of model accuracy with the training elasticity state through weighted summation to obtain the effective utilization of computing resources. This allows for a comprehensive evaluation of the resources' ability to support model accuracy and their adjustability. In step S34 of this embodiment, the weight coefficients need to be set according to the resource priority of model training. Specifically, in this embodiment, the weight coefficients are determined through a resource utilization effect verification experiment: multiple different weight combinations are selected and trained under the same model and data conditions. The correlation between the effective utilization of computing resources in each group and the actual resource utilization rate and model accuracy is analyzed. The weight combination with the highest correlation is selected to ensure that the weight setting conforms to the actual resource utilization law. In this embodiment, a higher specific value of the effective utilization of computing resources indicates a better resource utilization effect, which can both support the model to achieve high accuracy and have sufficient elasticity to cope with resource fluctuations. It's important to note that the upper limit of model accuracy reflects the resource support capacity for model performance, while training flexibility reflects the ability to flexibly allocate resources; both are indispensable. Focusing solely on the upper limit of accuracy may overlook training stability issues caused by insufficient resource flexibility; conversely, focusing solely on the flexibility status may fail to guarantee that model accuracy will meet targets. This embodiment employs a weighted summation method to combine the advantages of both, achieving a comprehensive evaluation of resource utilization. This provides a core basis for evaluating the training effect of subsequent AI algorithm models from a resource perspective, helping developers quickly determine whether the current resource allocation is reasonable. For example, when the effective utilization of computing resources is low, if the upper limit of accuracy is low, hardware resources need to be optimized or the model architecture adjusted; if the flexibility status is low, resource scheduling strategies need to be optimized. This targeted approach improves resource utilization, avoids blindly increasing hardware investment or optimizing ineffective processes, and reduces the cost and time consumption of model training.

[0029] In this embodiment, step S4 involves constructing an AI algorithm model training effect evaluation model to assess the training effect during the AI ​​algorithm model training process. This includes the following specific steps: S41. Extract the degree of data value correlation and matching and the degree of effective utilization of computing resources during the training process of AI algorithm models; S42. The weighted sum of the data value association matching degree and the effective utilization of computing resources is used to obtain the training effect in the AI ​​algorithm model training process.

[0030] It should be noted that in this embodiment, the determination of weight coefficients needs to be combined with historical training cases and business objective analysis. For example, this embodiment can collect historical training data of similar models, statistically analyze the correlation between training effect values ​​under different weights and the final deployment performance of the model (e.g., prediction accuracy, running efficiency), and select the weight combination that best reflects the actual deployment performance to ensure that the weight setting fits the business needs. Furthermore, in this embodiment, a higher specific training effect indicates a better overall effect of the model training process, which can both ensure the value of training data and environmental adaptability, and achieve efficient utilization of computing resources. In this embodiment, different business scenarios have different levels of emphasis on data and resources, and a single indicator cannot meet diverse evaluation needs. Therefore, weighted summation can flexibly adapt to different scenarios, highlighting core needs by adjusting weights, making the training effect evaluation results more practically instructive. For example, in cost-sensitive scenarios, if the effective utilization of computing resources is high but the correlation and matching degree of data value is slightly low, the training effect value can still be within a reasonable range by adjusting the weights, avoiding negating the value of overall efficient resource utilization due to slight deficiencies in data indicators. In this embodiment, step S42 provides researchers with clear evaluation criteria for the training process through quantified training effect indicators, avoiding evaluation bias caused by subjective judgment. For example, when the training effect value is high, it indicates that the current training plan is reasonable and can continue to be implemented; when the value is low, the problem dimension can be quickly located, the training plan can be adjusted in a targeted manner, and the closed-loop optimization efficiency of model training can be improved.

[0031] In this embodiment, step S5 adjusts the training scheme of the AI ​​algorithm model based on the training effect evaluation results during the AI ​​algorithm model training process, including the following specific contents: The training effect of the AI ​​algorithm model during training is obtained, and a training effect threshold is preset. When the training effect of the AI ​​algorithm model during training is greater than or equal to the training effect threshold, the current training scheme of the AI ​​algorithm model is deemed to have met the standard. When the training effect of the AI ​​algorithm model during training is less than the training effect threshold, the training scheme adjustment process is triggered. Specifically, in this embodiment, the training effect threshold needs to be determined by statistical analysis of the standard-reaching effects of similar historical models and business performance requirements. For example, the standard-reaching training effect values ​​of 10 similar models in the past are collected, and their training effect distribution range is statistically analyzed. The threshold is set in combination with the business performance requirements of the current model. For example, when the business performance requirement of the current model is a prediction accuracy of 95%, the threshold can be set to the maximum value of the distribution range. If the business performance requirement is low (for example, only simple classification is performed by the model), the threshold can be the minimum value of the distribution range (such as 0.7). If the business performance requirement is medium, the median of the distribution range can be used as the training effect threshold. Furthermore, this embodiment executes an adjustment strategy based on the comparison between the training effect and the threshold: when the training effect is greater than or equal to the threshold, the current training scheme is deemed satisfactory, and training continues according to the original scheme, while recording the current training parameters and effect data as a reference for subsequent similar models; when the training effect is less than the threshold, the training scheme adjustment process is triggered, which is as follows: when the data value association matching degree is lower than the average data value association matching degree of historical satisfactory models, high-value sparse samples are added or the distribution matching degree between training data and deployment environment is optimized; if the effective utilization of computing resources is lower than the average effective utilization of computing resources of historical satisfactory models, the training configuration is optimized (e.g., adjusting batch size, optimizer type) or hardware resources are upgraded (e.g., increasing GPU memory); if both are lower than the average of historical satisfactory models, the data and resource strategies are adjusted simultaneously. After adjusting the new training scheme, the S1-S4 process is re-executed to calculate the new training effect until the training effect reaches the threshold. In this embodiment, step S5 achieves closed-loop optimization of model training, avoiding resource waste and substandard performance caused by continuously using inefficient training schemes. By using clear threshold standards and targeted adjustment strategies, the blindness of adjustments is reduced, and the efficiency of training scheme optimization is improved. For example, when the training effect is low due to insufficient data value, high-value samples can be directly added without adjusting resource configuration, reducing optimization costs. At the same time, the recording and reuse of compliant schemes can provide experience reference for subsequent model training, reduce repeated trial and error, improve the overall model development efficiency, and ensure that the final trained model can not only meet business performance requirements but also achieve efficient resource utilization. Example 2

[0032] like Figure 2As shown, this embodiment provides an AI algorithm model training and management system, including a multi-source information acquisition module, a training data analysis module, a resource utilization analysis module, a training effect evaluation module, and a training scheme adjustment module; Among them, the multi-source information acquisition module is used to acquire training data information, training configuration information and real-time system monitoring information of AI algorithm model during model training, and at the same time acquire model architecture information and model deployment environment information of AI algorithm model; The training data analysis module is used to construct a data value correlation matching degree analysis model based on the training data information and model deployment environment information of the AI ​​algorithm model during the model training process, and to analyze the data value correlation matching degree during the AI ​​algorithm model training process; The resource utilization analysis module is used to combine the training configuration information and model architecture information of the AI ​​algorithm model during the model training process to build a model for analyzing the effective utilization of computing resources and to analyze the effective utilization of computing resources during the AI ​​algorithm model training process. The training effect evaluation module is used to construct an AI algorithm model training effect evaluation model based on the analysis results of the data value correlation matching degree and the analysis results of the effective utilization of computing resources during the AI ​​algorithm model training process, and to evaluate the training effect of the AI ​​algorithm model training process. The training scheme adjustment module is used to adjust the training scheme of the AI ​​algorithm model based on the evaluation results of the training effect during the AI ​​algorithm model training process.

[0033] The steps for implementing the corresponding functions of each parameter and each unit module in the AI ​​algorithm model training management system of the present invention can be referred to the parameters and steps in the embodiments of the AI ​​algorithm model training management method above, and will not be repeated here.

[0034] Example 3

[0035] An electronic device according to an embodiment of the present invention includes a processor and a memory, wherein the memory stores a computer program that can be called by the processor, and the processor executes an AI algorithm model training management method by calling the computer program stored in the memory. It should be noted that all computer programs for the AI ​​algorithm model training management method are implemented using C language, and the multi-source information acquisition module, training data analysis module, resource utilization analysis module, training effect evaluation module, and training scheme adjustment module are all controlled by a remote server.

[0036] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0037] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. An AI algorithm model training management system, characterized in that, The system comprises four modules: a multi-source information acquisition module, a training data analysis module, a resource utilization analysis module, a training effect evaluation module, and a training scheme adjustment module. The multi-source information acquisition module acquires training data, training configuration information, and real-time system monitoring information of the AI ​​algorithm model during training, as well as model architecture and deployment environment information. The training data analysis module constructs a data value correlation matching analysis model based on the training data and deployment environment information to analyze the degree of data value correlation matching during AI algorithm model training. The resource utilization analysis module combines the training configuration and architecture information of the AI ​​algorithm model to construct a computational resource effective utilization analysis model to analyze the effective utilization of computational resources during AI algorithm model training. The training effect evaluation module constructs an AI algorithm model training effect evaluation model based on the data value correlation matching and computational resource effective utilization analysis results to evaluate the training effect during AI algorithm model training. The training scheme adjustment module adjusts the training scheme of the AI ​​algorithm model based on the training effect evaluation results.

2. The AI ​​algorithm model training and management system according to claim 1, characterized in that, The construction of the data value association matching degree analysis model analyzes the degree of data value association matching during the training process of AI algorithm model, including the following specific steps: S21. Extract training data information and model deployment environment information of the AI ​​algorithm model during model training; S22. Import the training data information into the data value density analysis strategy to analyze the training data value density of the AI ​​algorithm model during the model training process. S23. Import the training data information and model deployment environment information into the environment evolution and adaptation analysis strategy to analyze the environment evolution and adaptation of the AI ​​algorithm model during the model training process. S24. The weighted summation of the training data value density and environmental evolution adaptation of the AI ​​algorithm model during the model training process is performed to obtain the degree of data value correlation and matching during the AI ​​algorithm model training process.

3. The AI ​​algorithm model training and management system according to claim 2, characterized in that, The specific analysis steps of the data value density analysis strategy are as follows: S221. For each sample in the training dataset, calculate the average Euclidean distance from each sample to its k nearest neighbors, where the value of k is the square root of the total number of samples in the training dataset and rounded down. S222. Calculate the mean and standard deviation of the distances from the mean to all samples, and identify samples whose distances from the mean are greater than the sum of the mean and a standard deviation as high-value sparse samples. S223. The ratio of the number of high-value sparse samples to the total number of samples in the training dataset is used as the training data value density of the AI ​​algorithm model during model training.

4. The AI ​​algorithm model training and management system according to claim 3, characterized in that, The construction of the computing resource effective utilization analysis model analyzes the effective utilization of computing resources during the AI ​​algorithm model training process, including the following specific steps: S31. Extract the training configuration information, real-time system monitoring information, and model architecture information of the AI ​​algorithm model during the model training process; S32. Import the training configuration information and model architecture information into the model accuracy upper limit level analysis strategy to analyze the model accuracy upper limit level during the AI ​​algorithm model training process; S33. Import the real-time monitoring information of the system into the training elasticity state analysis strategy to analyze the training elasticity state during the training process of the AI ​​algorithm model.

5. The AI ​​algorithm model training and management system according to claim 4, characterized in that, The specific analysis steps of the model accuracy upper limit level analysis strategy are as follows: S321. Calculate the minimum memory estimate required for model training, wherein the minimum memory estimate is obtained by adding the product of the batch size and the forward propagation activation memory estimate per sample to four times the number of model parameters; S322. When the available GPU memory is less than the minimum memory estimate required for model training, the upper limit of model accuracy during AI algorithm model training is zero. Otherwise, calculate the minimum training time estimate required for model training. The minimum training time estimate is obtained by dividing the product of the training dataset size and the number of model parameters by the estimated throughput per unit time. S323. When the available training time is less than the estimated minimum training time required for model training, the ratio of the available training time to the estimated minimum training time required for model training shall be used as the upper limit of model accuracy during the AI ​​algorithm model training process; otherwise, the upper limit of model accuracy shall be 1.

6. The AI ​​algorithm model training and management system according to claim 5, characterized in that, The construction of the AI ​​algorithm model training effect evaluation model, which evaluates the training effect during the AI ​​algorithm model training process, includes the following specific steps: S41. Extract the degree of data value correlation and matching and the degree of effective utilization of computing resources during the training process of AI algorithm models; S42. The weighted sum of the data value association matching degree and the effective utilization of computing resources is used to obtain the training effect in the AI ​​algorithm model training process.

7. The AI ​​algorithm model training and management system according to claim 6, characterized in that, The adjustment of the AI ​​algorithm model training scheme based on the training effect evaluation results during the AI ​​algorithm model training process includes the following specific contents: The system acquires the training effect during the AI ​​algorithm model training process and presets a training effect threshold. When the training effect during the AI ​​algorithm model training process is greater than or equal to the training effect threshold, the current training scheme of the AI ​​algorithm model is deemed to be up to standard. When the training effect during the AI ​​algorithm model training process is less than the training effect threshold, the training scheme adjustment process is triggered.

8. An AI algorithm model training management method, implemented based on the AI ​​algorithm model training management system according to any one of claims 1-7, characterized in that, The specific steps include the following: Acquire training data, training configuration information, and real-time system monitoring information of the AI ​​algorithm model during the model training process, and at the same time acquire model architecture information and model deployment environment information of the AI ​​algorithm model; Based on the training data and deployment environment information of the AI ​​algorithm model during the model training process, a data value association matching degree analysis model is constructed to analyze the data value association matching degree during the AI ​​algorithm model training process; By combining the training configuration information and model architecture information of the AI ​​algorithm model during the model training process, a model for analyzing the effective utilization of computing resources is constructed to analyze the effective utilization of computing resources during the AI ​​algorithm model training process. Based on the analysis results of the data value correlation and matching degree and the analysis results of the effective utilization of computing resources during the training process of AI algorithm model, an AI algorithm model training effect evaluation model is constructed to evaluate the training effect during the training process of AI algorithm model; Based on the evaluation results of the training effect during the AI ​​algorithm model training process, the training scheme of the AI ​​algorithm model is adjusted.

9. An electronic device, comprising: A processor and a memory, wherein the memory stores a computer program that can be called by the processor; characterized in that the processor executes the AI ​​algorithm model training management method as described in any one of claims 8 by calling the computer program stored in the memory.

Images