Mine AI model fine adjustment and optimization management method based on incremental learning

Abstract

The invention relates to the technical field of AI model management, in particular to a mine AI model fine adjustment and optimization management method based on incremental learning, which comprises the following steps: calling a current version model and a previous version model for each heading machine state data sample in an incremental data stream, calculating the KL divergence of the output distribution of the two models, and calculating the KL divergence of the output distribution of the two models; and meanwhile, the prediction entropy of the current version model on the heading machine state data sample is calculated. According to the method, the KL divergence and the prediction entropy in the heading machine state data sample are subjected to two-dimensional calculation, and the sample is mapped to the two-dimensional space to identify the Pareto frontier set, so that screening of high-potential samples with coexistence of information value and model uncertainty can be realized, invalid training of redundant samples is avoided, the data utilization rate is improved, and the method is suitable for large-scale popularization and application. And an incremental training key sample set is uniformly constructed after the set boundary samples are manually labeled, so that a high-quality information source can be continuously introduced in model updating, and targeted convergence of a training target is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of AI model management, and particularly relates to a mine AI model fine-tuning and optimization management method based on incremental learning. BACKGROUND

[0002] AI model management includes model training data management, model version control, model performance evaluation, online and offline model switching mechanism. The prior art in AI model management usually only includes static management of model training data, version control, performance evaluation and model switching mechanism, and the processing of incremental data flow often depends on additional training, causing frequent model parameter updates but insignificant results, in addition, ignoring the influence of model internal structure complexity on inference efficiency and generalization ability, which easily leads to model structure expansion and increased computing resource consumption. Therefore, improvement is needed. SUMMARY

[0003] The purpose of the present application is to solve the shortcomings in the prior art and to provide a mine AI model fine-tuning and optimization management method based on incremental learning.

[0004] In order to achieve the above-mentioned purpose, the present application adopts the following technical scheme, a mine AI model fine-tuning and optimization management method based on incremental learning, comprising the following steps: For each tunneling machine state data sample in the incremental data stream, call the current version model and the last version model, calculate the KL divergence of the output distribution of the two models, and calculate the prediction entropy of the current version model for the tunneling machine state data sample, integrate the KL divergence and the prediction entropy into a two-dimensional coordinate point, generate a two-dimensional evaluation value for a single sample, and identify a selected sample Pareto frontier set; According to the selected sample Pareto frontier set, all tunneling machine state data samples located on the boundary of the set are screened to form a high-value sample index, the corresponding original tunneling machine state data is extracted from the incremental data stream according to the high-value sample index, and artificial labeling is performed, the labeled tunneling machine state data is combined to establish an incremental training key sample set; According to the incremental training key sample set, the parameters of the current model are updated to obtain an updated model, the classification accuracy of the updated model is measured on a preset ore grade image verification set to obtain a model performance value, the weight matrix inside the updated model is extracted, the stable rank is calculated, and the model effective complexity measure is obtained. According to the model performance value and the model effective complexity measure, a current knowledge density value is calculated and generated. According to the sequence of the current knowledge density values obtained in a plurality of continuous update cycles, it is determined whether the sequence presents a continuous downward trend and whether the latest current knowledge density value is lower than a preset threshold value, and if the determination condition is met, a model reconstruction trigger signal is generated, a student model of a random parameter is initialized according to the model reconstruction trigger signal, and knowledge distillation is performed on the support pressure monitoring value summary data using the current model as a teacher model to generate a knowledge-compact new baseline model.

[0005] Preferably, the obtaining step of the candidate sample Pareto frontier set is: According to each tunneling machine state data sample in the incremental data stream, the current version output distribution and the last version output distribution are read, the difference is calculated item by item according to the probability vector index and is summarized, the prediction entropy value of the current version output distribution for the tunneling machine state data sample is counted, and a two-dimensional evaluation value of a single sample is generated; According to the two-dimensional evaluation value of the single sample, all newly added tunneling machine state data samples are traversed, the two components of the two-dimensional evaluation value of each single sample are extracted as the abscissa and the ordinate, and the coordinate point list is written in the sample order and the corresponding sample index is recorded to form a two-dimensional evaluation coordinate set; According to the two-dimensional evaluation coordinate set, each pair of components of the coordinate points is compared, the coordinate points that are not simultaneously superior to any other coordinate points in the two components are marked, the tunneling machine state data sample indexes corresponding to the marked coordinate points are extracted and the original order is kept, and a candidate sample Pareto frontier set is generated.

[0006] Preferably, the obtaining step of the high-value sample index is: According to the candidate sample Pareto frontier set, the corresponding tunneling machine state data samples are marked one by one according to whether the coordinate points are located on the set boundary, the unique indexes of the marked tunneling machine state data samples are extracted and stored in the original order, and a high-value sample index is generated.

[0007] Preferably, the obtaining step of the incremental training key sample set is: According to the high-value sample index, the corresponding original tunneling machine state data is read from the incremental data stream according to the index position, the timestamp and sensor identifier of each original tunneling machine state data are recorded, and manual labeling is submitted, the correspondence between the manual labeling result and the original tunneling machine state data is summarized, and the labeled tunneling machine state data is generated. According to the labeled tunneling machine state data, the sample order is organized according to the sample timestamp and the grouping strategy of the equipment identifier, the fixed batch capacity is set and filled in order to generate the data batch for model fine-tuning, all data batches for model fine-tuning are integrated and the index mapping is kept, and the incremental training key sample set is formed.

[0008] Preferably, the step of obtaining the current knowledge density value is: According to the incremental training key sample set, a sample batch is loaded into the current model in chronological order, the predicted probability is compared with the real label in batches, and the parameters are updated, the parameter state after each batch update is recorded, and the sample range covered is recorded, and an updated model is generated; According to the updated model, the prediction category is generated for each image in the ore grade image verification set, the correct prediction number and the total prediction number are accumulated to convert the classification accuracy, which is recorded as the model performance value, and the weight matrix of each layer is extracted to calculate the stable rank according to the matrix norm, and the stable ranks of each layer are summed to generate the effective complexity measure of the model; According to the model performance value and the effective complexity measure of the model, the current knowledge density value is calculated.

[0009] Preferably, the step of obtaining the model reconstruction trigger signal is: According to the sequence of the current knowledge density value, the difference value symbol of adjacent items is calculated in chronological order, and the number of items with negative end is accumulated, and the accumulated number of items is compared with the continuous decline determination standard, and the latest current knowledge density value is compared with the preset threshold value, and the identification is set if the two conditions are met, and the model reconstruction trigger signal is generated.

[0010] Preferably, the step of obtaining the knowledge compactification new baseline model is: According to the model reconstruction trigger signal, the parameter tensor memory of the student model is allocated according to the layer list, and the random number generator state is set to complete random initialization, the current model is specified as the teacher model, the time field and the statistical field of the support pressure monitoring value summary data are analyzed to generate the input batch queue, and the student model initialization state is formed.

[0011] Preferably, the step of obtaining the knowledge compactification new baseline model further comprises: according to the student model initialization state, generating the current model soft target on the support pressure monitoring value summary data according to the input batch queue, and synchronously generating the student model output, calculating the soft target difference and updating the parameters, until the difference reduction amplitude of the continuous round is lower than the stop threshold value, the parameter state with the optimal performance is reserved as the result, and the knowledge compactification new baseline model is formed.

[0012] Compared with the prior art, the application has the advantages and positive effects that: The application can realize the screening of high-potential samples with coexisting information value and model uncertainty by calculating the two-dimensional KL divergence and prediction entropy of the heading machine state data sample and mapping the sample to a two-dimensional space to identify the Pareto frontier set, avoid the invalid training of redundant samples, improve data utilization, and uniformly construct an incremental training key sample set after manually labeling the set boundary samples, continuously introduce high-quality information sources in model updating, realize targeted convergence of the training target, analyze the sequence trend in the continuous period, generate a reconstruction signal with threshold judgment, so that the model updating process can adaptively perceive the capability bottleneck, timely trigger the re-initialization of the structure layer, and reconstruct on the support pressure monitoring value summary data through knowledge distillation, compress redundant parameters while retaining important knowledge representation. BRIEF DESCRIPTION OF DRAWINGS

[0013] Figure 1 The figure is a schematic diagram of the steps of the application. DETAILED DESCRIPTION

[0014] In order to make the purpose, technical scheme and advantages of the application clearer, the application will be further described in detail below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the application and do not limit the application.

[0015] Please refer to Figure 1 The application provides a technical scheme, a mine AI model fine-tuning and optimization management method based on incremental learning, including the following steps: For each heading machine state data sample in the incremental data stream, the current version model and the last version model are called, the KL divergence of the output distribution of the two models is calculated, and the prediction entropy of the current version model for the heading machine state data sample is calculated, the KL divergence and the prediction entropy are integrated into two-dimensional coordinate points, the double-dimensional evaluation value of a single sample is generated, and the Pareto frontier set of the selected samples is identified; According to the Pareto frontier set of the selected samples, all heading machine state data samples located on the set boundary are screened to form a high-value sample index, the corresponding original heading machine state data is extracted from the incremental data stream according to the high-value sample index, and is manually labeled, the labeled heading machine state data is combined to establish an incremental training key sample set; According to the incremental training key sample set, the parameters of the current model are updated, the updated model is obtained, the classification accuracy of the updated model is measured on the preset ore grade image verification set, the model performance value is obtained, the weight matrix inside the updated model is extracted, the stable rank is calculated, the model effective complexity measure is obtained, and the current knowledge density value is calculated and generated according to the model performance value and the model effective complexity measure; According to the sequence of the current knowledge density values obtained in a plurality of continuous update cycles, it is determined whether the sequence presents a continuous downward trend and whether the latest current knowledge density value is lower than a preset threshold value. If the determination condition is met, a model reconstruction trigger signal is generated. According to the model reconstruction trigger signal, a student model of a random parameter is initialized, and the current model is used as a teacher model to perform knowledge distillation on the support pressure monitoring value summary data, and a knowledge-compact new baseline model is generated.

[0016] The obtaining step of the candidate sample Pareto frontier set is: According to each heading machine state data sample in the incremental data stream, the current version output distribution and the last version output distribution are read, the difference is calculated item by item according to the probability vector index and is summarized, the prediction entropy value of the current version output distribution for the heading machine state data sample is counted, and the two-dimensional evaluation value of a single sample is generated; According to the two-dimensional evaluation value of a single sample, all new heading machine state data samples are traversed, the two components of the two-dimensional evaluation value of each single sample are extracted as the abscissa and the ordinate, and the coordinate point list is written in the sample order and the corresponding sample index is recorded, forming a two-dimensional evaluation coordinate set; According to the two-dimensional evaluation coordinate set, each pair of components of the coordinate points is compared, the coordinate points that are not simultaneously superior to any other coordinate points in the two components are marked, the heading machine state data sample index corresponding to the marked coordinate points is extracted and the original order is kept, and a candidate sample Pareto frontier set is generated.

[0017] Specifically, according to each heading machine state data sample in the incremental data stream, first, the heading machine state classification probability distribution output by the current version model is read, denoted as The distribution is a vector, and the elements correspond to the prediction probability of each preset state (such as normal operation, tool wear, and hydraulic abnormality). At the same time, the probability distribution output by the last version model for the same heading machine state data sample is read, denoted as Then, the Kullback-Leibler divergence between the two probability distribution vectors is calculated as a measure of model difference. The specific calculation process is to divide each element in the probability distribution output by the current version model by the element of the probability distribution output by the last version model at the corresponding index position, then take the natural logarithm, then multiply by the corresponding element of the probability distribution output by the current version model, and finally sum the calculation results at all index positions. The calculation expression is Wherein, represents the category index of the heading machine state, is a very small positive number, for example , which is used to prevent overflow caused by a zero denominator, then the probability distribution , which is used to measure the degree of uncertainty of the model's prediction for the current sample, and its calculation method is to take the natural logarithm of the probability value of each element in the probability distribution, then multiply it by the probability value itself, and finally sum up all the index positions and take the negative, the calculation expression is , where , also represents the category index of the roadheader state, finally, the calculated KL divergence value is taken as the first dimension and the prediction entropy value is taken as the second dimension to form a two-dimensional coordinate point, and a two-dimensional evaluation value of a single sample is generated.

[0018] According to the two-dimensional evaluation value of a single sample generated for each new roadheader state data sample according to the foregoing steps, the traversal processing of all new roadheader state data samples in the current batch is started, and a data record is created for each sample in the traversal process, which contains three core parts, namely the unique index of the sample in the incremental data stream, the KL divergence value as the abscissa and the prediction entropy value as the ordinate, for example, for the sample with index 101, its KL divergence value is 0.85 and the prediction entropy is 1.23, then a record (101, 0.85, 1.23) is generated for it, and all such data records of samples in the current processing batch are added to a temporary coordinate point list in the original order of their appearance in the incremental data stream, the structure of this list is designed as a tuple containing the sample index and the corresponding two-dimensional coordinate, for example [(index1, (KL value1, entropy value1)), (index2, (KL value2, entropy value2)),...], in this way, not only the two-dimensional coordinate points for evaluation are saved, but also the explicit mapping relationship between each coordinate point and its source data sample is maintained, after completing the traversal and data record writing of all new roadheader state data samples in the current batch, the temporary coordinate point list constitutes a complete two-dimensional evaluation coordinate set.

[0019] According to the two-dimensional evaluation coordinate set constructed, the Pareto front identification algorithm is executed, the core of which is to compare pairs to find all non-dominated solutions, in specific implementation, first initialize a Boolean flag array as long as the number of coordinate points, all elements are initially set to "non-dominated", then use double loop to traverse all coordinate points in the two-dimensional evaluation coordinate set, in the outer loop, take out each coordinate point , whose coordinates are , , in the inner loop, take out all other coordinate points , whose coordinates are , , , in each inner loop comparison, judge whether there is any domination , the condition of domination is defined as and , and at least one of the inequalities is strict, i.e. or If any one of the points in the inner loop is found to satisfy the domination condition , the point is immediately marked as "dominated" in the Boolean flag array, and the inner loop comparison for is terminated, and the next point in the outer loop is processed. When the outer loop has traversed all coordinate points, the Boolean flag array is again traversed, and all coordinate points whose flags are still "non-dominated" are extracted, and the original tunneling machine state data sample indices corresponding to these points are also extracted. To ensure the determinacy and traceability of the results, these extracted indices are sorted in ascending order according to their numerical values. The final ordered index list is the selected sample Pareto front set.

[0020] The steps for obtaining the high-value sample indices are as follows: According to the selected sample Pareto front set, mark the corresponding tunneling machine state data samples one by one according to whether the coordinate points are located on the boundary of the set, extract the unique indices of the marked tunneling machine state data samples, and store them according to the original order to generate high-value sample indices.

[0021] Specifically, according to the set of to-be-selected sample Pareto frontiers, a geometric analysis algorithm is first applied to identify coordinate points located on the boundary of the set, and a Graham Scan algorithm is specifically used to construct a convex hull of the coordinate points. First, the coordinate point with the smallest y-coordinate value in the set of to-be-selected sample Pareto frontiers is found as a starting reference point. If there are multiple points with the same y-coordinate value, the one with the smallest x-coordinate value is selected. Then, taking the reference point as the origin, the polar angle of all other coordinate points relative to the reference point is calculated, and all coordinate points are sorted in ascending order of polar angle. If the polar angles are the same, the points are sorted in descending order of Euclidean distance from the reference point. Next, a stack structure is initialized, and the first two sorted coordinate points are pushed into the stack. Starting from the third sorted coordinate point, all remaining coordinate points are iterated. For the currently iterated coordinate point, the direction formed by the top two coordinate points in the stack is checked. If the direction is a right turn or the three points are collinear, it indicates that the top coordinate point in the stack is an in-polygon point and does not belong to the convex hull boundary, so it is popped out of the stack. This pop operation is repeated until the newly formed direction is a left turn. Then, the currently iterated coordinate point is pushed into the stack. After the iteration is completed, all remaining coordinate points in the stack constitute the convex hull vertices of the Pareto frontier set. These vertices are marked as coordinate points located on the boundary of the set. Finally, the unique indexes of the original roadheader state data samples corresponding to the marked coordinate points are extracted and rearranged in ascending order of index value to generate high-value sample indexes.

[0022] The obtaining step of the incremental training key sample set is: According to the high-value sample indexes, the corresponding original roadheader state data is read from the incremental data stream according to the index position, the timestamp and sensor identifier of each original roadheader state data are recorded, and the artificial labeling is submitted. The correspondence between the artificial labeling result and the original roadheader state data is summarized, and the labeled roadheader state data is generated; According to the labeled roadheader state data, the sample order is organized according to the sample timestamp and the grouping strategy of the equipment identifier, the fixed batch capacity is set and filled in order to generate data batches for model fine-tuning, all data batches for model fine-tuning are integrated and the index mapping is retained, and the incremental training key sample set is formed.

[0023] Specifically, according to the high-value sample index, each unique index in the index list is traversed, and the corresponding original tunneling machine state data is accurately located and read from the stored incremental data stream according to the index value. Each piece of original tunneling machine state data is recorded as a vector containing multiple sensor readings. For example, one record can contain the cutting head motor current (e.g., 350 A), the hydraulic system main pump pressure (e.g., 25 MPa), the rocker arm cylinder temperature (e.g., 65°C), and the X, Y, Z axis acceleration of a three-component vibration sensor (e.g., 1.2g, 0.8g, 2.5g, respectively). While reading the data, the associated meta-information of each piece of data is parsed and recorded, including the time stamp accurate to milliseconds (e.g., '2023-10-27 10:30:15.123') and the unique device identification of the tunneling machine (e.g., 'TBM-007'). Subsequently, each piece of data containing original sensor data, time stamp, and device identification is packaged into an independent labeling task, and these tasks are pushed to a human-computer interaction labeling platform designed for experts in the mining field. On this platform, the labeling tasks are presented in a visual form, such as drawing each sensor data into a time series curve graph. Experts make judgments on the state of the tunneling machine at that moment based on their domain knowledge and operating experience, and select the most appropriate label from a predefined label set. The label set includes 'normal tunneling','severe cutter wear', 'hydraulic system pressure anomaly', 'electrical system failure','surrounding rock change', etc. After labeling is completed, all expert labeling results are summarized, and each original data record is associated with its corresponding expert labeling label, labeling person information, and labeling time to form a structured data pair. The collection of these data pairs finally generates the completed labeled tunneling machine state data.

[0024] According to the completed labeled heading machine state data, first, a double sorting operation is performed to organize the final order of the samples, the first layer sorting is grouped according to the equipment identifier, and all samples from the same heading machine (for example, 'TBM-007') are aggregated together, and the second layer sorting is arranged in ascending order according to the sample timestamp within each equipment group, and the samples are organized into a time sequence reflecting the real running condition of the equipment, then, a fixed batch capacity is set, which is based on the hardware resources of the current training environment, mainly the memory size of the graphics processing unit (GPU), and the stability requirements of model training, for example, on a GPU with 32GB memory, in order to balance the calculation efficiency and the smoothness of gradient update, the batch capacity is set to 64, after setting the batch capacity, the data batches are filled in sequence from the beginning according to the sorted sample sequence, that is, the first 64 samples in the sequence form the first batch, the 65th to 128th samples form the second batch, and so on, until all samples are allocated to the batches, if the total number of samples is not an integer multiple of the batch capacity, the last batch will contain all the remaining samples, during the generation of data batches, an index mapping table is created simultaneously, which records the correspondence between each batch and the unique index of the original heading machine state data sample it contains, for example, the mapping table will record that batch 1 corresponds to index [101, 105,..., 230], and batch 2 corresponds to index [231,..., 350], finally, all generated data batches and this complete index mapping table are integrated together to form the incremental training key sample set.

[0025] The acquisition step of the current knowledge density value is: According to the incremental training key sample set, load the sample batch into the current model according to the timestamp order, compare the prediction probability with the true label batch by batch and update the parameters, record the parameter state after each batch update and the sample range covered, generate the updated model; According to the updated model, generate the prediction category for each image in the ore grade image verification set, accumulate the correct prediction number and the total prediction number to calculate the classification accuracy, which is denoted as the model performance value, at the same time, extract each layer weight matrix to calculate the stable rank according to the matrix norm, and sum up the stable ranks of each layer to generate the model effective complexity measure; According to the model performance value and the model effective complexity measure, the current knowledge density value is calculated, the expression is: ; Where D is the current knowledge density value, a is the model performance value, g is the model effective complexity measure, and ε is a small positive number used for smoothing to avoid a zero denominator.

[0026] Specifically, according to the incremental training key sample set, the index mapping table and the timestamp sorting information contained therein are sorted, and each data batch for model fine-tuning is sequentially loaded. The tunneling machine state data in the batch, which is a multi-dimensional sensor time series, such as a sample containing 60 seconds of continuous cutting motor current, hydraulic pump pressure and machine body vibration data, is input into the input layer of the current version model. The input layer of the model is a one-dimensional convolution layer used to extract time series features. Subsequently, the data flows through two long short-term memory network (LSTM) layers, each containing 128 neurons. Finally, a fully connected layer and a Softmax activation function are used to output a probability distribution vector corresponding to the preset tunneling machine state category. After obtaining the prediction probability, it is compared with the true label corresponding to each sample in the batch. The difference between the two is calculated using the cross-entropy loss function, which can measure the distance between the predicted distribution and the true distribution. Then, the Adam optimizer is used to update the model parameters. The learning rate of the optimizer is set to a small value, for example This value is determined by analyzing historical fine-tuning tasks to select the largest learning rate that allows the model to learn new knowledge without destroying existing knowledge. At the same time, the of the optimizer is set to 0.9, to 0.999 to control the mean and variance of gradient movement. After the parameter update process is completed on all samples in the current batch, the complete parameter state of all weight matrices and bias vectors of the current model is recorded, and the unique index range of the original tunneling machine state data samples covered by the batch is also recorded, for example, batch 1 covers indexes 101 to 164. After all data batches in the incremental training key sample set are traversed and processed, the final parameter state is the updated model.

[0027] According to the updated model, first load the preset ore grade image verification set, which contains 2000 expert annotated rock mass images collected from the excavation face, covering four categories of high grade, medium grade, low grade and waste rock, 500 images for each category, before processing each image, first adjust its size to 224x224 pixels, and normalize the pixel value, that is, subtract the mean value of the data set from each pixel point RGB value and divide by the standard deviation, then input the preprocessed image into the input end of the updated model, the model performs a forward propagation calculation, and finally generates a four-dimensional probability vector through the output layer, the class corresponding to the maximum probability value in the vector is the predicted class of the model, compare the predicted class with the true label of the image, if they are consistent, add one to a preset correct prediction counter, regardless of whether the prediction is correct or not, add one to the total prediction counter, after traversing all 2000 images in the verification set, divide the final value of the correct prediction counter by the value of the total prediction counter (i.e. 2000) to get the classification accuracy, and record this accuracy value as the model performance value, at the same time, access the parameter structure inside the updated model, extract the weight matrix of all convolution layers and fully connected layers, for each weight matrix , first calculate its largest singular value by singular value decomposition (SVD), get its spectral norm, then calculate the square root of the sum of squares of all elements, get its Frobenius norm, then divide the square of the Frobenius norm by the square of the spectral norm, calculate the stable rank of this layer, finally sum all the stable rank values of the weight matrix in the model to get the effective complexity measure of the model.

[0028] Formula: , the numerator represents the performance of the model, the higher the performance, the higher the basis of the knowledge density value, the denominator part is a penalty term, which increases with the increase of the proportion of model complexity relative to performance , thereby reducing the knowledge density value, specifically, This term reflects the model complexity consumed per unit performance, the larger the ratio, the more "bloated" the model, the lower the efficiency, and the natural logarithm function is used to smooth this ratio to avoid the complexity having too much impact on the final result, making the index less sensitive to changes in complexity, the entire denominator structure ensures that the knowledge density value is always less than the performance value , and when the complexity is zero, the knowledge density is equal to the performance, this formula encourages the model to maintain a low effective complexity while maintaining high accuracy, quantifying the "compactness" of the model knowledge.

[0029] Model performance value, representing the classification accuracy of the updated model on the standardized ore grade image validation set, with a value ranging from 0 to 1. The parameter is obtained by inputting a validation set containing , for example, 2000 images, into the updated model and counting the number of images correctly classified by the model . The calculation formula of the model performance value is: For example, after verifying 2000 images, the model correctly identified 1870 of them, so the model performance value .

[0030] Model effective complexity measure, a quantitative evaluation of the redundancy of the model's internal parameters, obtained by calculating the sum of the stable ranks of all weight matrices in the model. A weight matrix The stable rank calculation formula is , where is the Frobenius norm, is the spectral norm (maximum singular value). The acquisition steps are as follows: first, traverse all layers containing weight parameters in the model (such as convolutional layers and fully connected layers), calculate the stable rank of each layer's weight matrix respectively, and the final model effective complexity measure is the sum of the stable ranks of all layers: , where is the number of layers containing weight matrices in the model. For example, a model contains three layers, and the calculated stable ranks of each layer are 62.5, 85.3 and 17.2, respectively, so the model effective complexity measure .

[0031] Extreme positive number, set to .

[0032] According to the parameters, the calculation is as follows: Substitute the parameter values obtained in the previous steps into the formula: model performance value , model effective complexity measure , and extreme positive number .

[0033] ; ; ; ; ; ; ; The result shows that in the current update cycle, the current knowledge density value of the model is 0.1513, compared with the knowledge density value of the historical update cycle, a time series is formed, by analyzing the trend of the sequence, the evolution state of the model can be judged, if the subsequent knowledge density value continues to decline, for example, 0.1450, 0.1380, 0.1310 are obtained in three consecutive cycles respectively, and the latest value 0.1310 is lower than the preset model health state threshold (for example, 0.14), it indicates that the model may have knowledge degradation or structure bloating problem, and deeper optimization such as model reconstruction needs to be triggered, otherwise, if the knowledge density value remains stable or rises, it means that the current incremental fine-tuning strategy is effective.

[0034] The acquisition step of the model reconstruction trigger signal is: According to the sequence of the current knowledge density value, the adjacent term difference sign is calculated in time sequence and the number of consecutive negative terms at the end is accumulated, the accumulated term number is compared whether it reaches the continuous decline judgment standard, at the same time, the latest current knowledge density value is compared whether it is less than the preset threshold, if the two conditions are met, the identification is set, and the model reconstruction trigger signal is generated.

[0035] Specifically, according to the sequence of the current knowledge density value, for example, a list containing the values of the last 10 update cycles [0.165, 0.166, 0.162, 0.159, 0.161, 0.155, 0.151, 0.148, 0.145, 0.141], first traverse the sequence from back to front, calculate the difference value of adjacent items, that is, subtract the previous item from the next item, and determine the sign of the difference value. For the last two items 0.145 and 0.141 of the sequence, the difference value is -0.004 and the sign is negative. For 0.148 and 0.145, the difference value is -0.003 and the sign is negative. For 0.151 and 0.148, the difference value is -0.003 and the sign is negative. For 0.155 and 0.151, the difference value is -0.004 and the sign is negative. For 0.161 and 0.155, the difference value is -0.006 and the sign is negative. For 0.159 and 0.161, the difference value is +0.002 and the sign is positive. At this time, stop backtracking, and the number of items with consecutive negative difference values starting from the end of the sequence is 5. Then, compare this cumulative number 5 with a preset sustained decline criterion, which is set to 4. This criterion is set based on statistical analysis of historical model life cycle data. It is found that before the irreversible decline of model performance, there is usually a continuous decline in knowledge density for at least 4 cycles. Therefore, setting the criterion to 4 can issue an early warning when the model state deteriorates. Since the cumulative number 5 is greater than or equal to 4, the first condition is met. Then, extract the latest current knowledge density value in the sequence, which is 0.141, and compare it with a preset threshold value. The preset threshold value is calculated according to the minimum requirement of model reliability for mine production. The specific method is to determine that the model performance value below 0.85 will pose an unacceptable risk to production decisions, and then calculate that the knowledge density value corresponding to this performance value is about 0.145, combined with the historical average model complexity. Therefore, the preset threshold value is set to 0.145. Since the latest value 0.141 is less than 0.145, the second condition is also met. When both conditions are met, immediately set a state identifier position to 1 and generate a model reconstruction trigger signal.

[0036] The acquisition step of the knowledge-compact new baseline model is: According to the model reconstruction trigger signal, allocate parameter tensor memory to the student model according to the layer list and set the random number generator state to complete random initialization, specify the current model as the teacher model, parse the time field and statistical field of the support pressure monitoring value summary data to generate an input batch queue, and form a student model initialization state. According to the student model initialization state, the current model soft target is generated on the support pressure monitoring value summary data in the input batch queue, and the student model output is generated synchronously, the soft target difference is calculated and the parameter is updated until the continuous round difference reduction amplitude is lower than the stop threshold, the student model parameter state with the optimal performance is reserved as the result, and a knowledge compact new baseline model is formed.

[0037] Specifically, according to the model reconstruction trigger signal, when it is detected that the signal is set to 1, the construction process of the student model is started. First, the network structure of the student model is defined. The structure is configured according to a preset layer list. The list creates a network that is more lightweight than the current model. For example, the current model (teacher model) contains two LSTM layers with 128 units each, while the layer list of the student model is defined as an input layer, followed by a one-dimensional convolutional layer with 64 filters and a convolution kernel size of 3, then a gated recurrent unit (GRU) layer containing 64 units, and finally a fully connected layer with the same output dimension as the number of task categories (for example, 5 categories) and a Softmax activation function. According to this layer list, a memory tensor of the corresponding size is allocated for each layer parameter (weight matrix and bias vector) of the student model. Then, the state of the random number generator is set, i.e. a seed value is fixed, for example 42, to ensure the reproducibility of each initialization process. The He normal distribution initialization method is used to assign initial values to the weight parameters of all convolutional layers and fully connected layers, and the biases are all initialized to zero. After initialization, the current model with performance degradation is specified as the teacher model, and its parameter state is frozen and no longer updated. Next, the support pressure monitoring value summary data is loaded and parsed. The data is summarized by the roadheader hydraulic support sensor network every minute. Each record contains five fields: timestamp, average pressure, maximum pressure, minimum pressure, and pressure standard deviation. When parsing, the last four statistical fields are extracted as model input features, and these feature vectors are organized into fixed-size data batches, for example, each batch contains 256 records, forming an input batch queue for knowledge distillation training. Finally, the initialized student model and the input batch queue are combined to form the student model initialization state.

[0038] According to the student model initialization state, the knowledge distillation training loop is started, and the data batches are taken from the input batch queue in order. For each batch, first input it into the teacher model for a complete forward propagation, get the logical values (logits) of its output before the Softmax layer of the teacher model, and divide these logical values by a preset temperature coefficient, for example Then apply the Softmax function to generate the soft target probability distribution of the teacher model. The temperature coefficient The setting is determined by experiment, at which temperature, the entropy value of the probability distribution output by the teacher model is higher, which can reveal the similarity information between categories, at the same time, the same data batch is input into the student model, the output logic value is also obtained, and the same temperature coefficient is used to process and generate the soft output of the student model, then the KL divergence between the teacher model soft target and the student model soft output is calculated as the distillation loss, which measures the difference between the two probability distributions, then the Adam optimizer is used to update the parameters of the student model according to the calculated distillation loss, the learning rate is set to , 0.9, 0.999, this process is repeated on all batches to form an epoch, after each epoch, the distillation loss is calculated on an independent validation set, and the validation loss is continuously monitored for 5 consecutive epochs, when the average loss reduction of the last 5 epochs is less than the preset stop threshold 0.05% for 3 consecutive times, the training process is terminated, the setting of the stop threshold is to prevent overfitting caused by continuing ineffective training when the loss is no longer substantially improved, during the entire training process, the student model parameter state of the epoch in which each validation loss sets a new low is saved, after the training is completed, the parameter state with the lowest validation loss is loaded as the final result, forming a knowledge compact new baseline model.

[0039] The above is only the preferred embodiment of the present application, and does not limit the present application in other forms, any skilled person in the art can use the above disclosed technical content to make changes or modifications as equivalent embodiments applied to other fields, but any simple modification, equivalent change and modification made according to the technical essence of the present application without departing from the technical solution content of the present application, still belongs to the protection scope of the technical solution of the present application.

Claims

1. A method for managing fine-tuning and optimization of mine AI models based on incremental learning, characterized in that, The method comprises the following steps: According to each tunneling machine state data sample in the incremental data stream, read the current version output distribution and the last version output distribution, calculate the difference degree item by item according to the probability vector index and summarize, count the prediction entropy value of the current version output distribution for the tunneling machine state data sample, and generate a two-dimensional evaluation value of a single sample; According to the two-dimensional evaluation value of the single sample, traverse all new tunneling machine state data samples, extract two components of the two-dimensional evaluation value of each single sample as the horizontal and vertical coordinates, write the coordinate point list according to the sample order and record the corresponding sample index, and form a two-dimensional evaluation coordinate set; According to the two-dimensional evaluation coordinate set, compare each pair of coordinates points according to the two components, mark the coordinate points that are not superior to any other coordinate points in the two components at the same time, extract the tunneling machine state data sample index corresponding to the marked coordinate points and keep the original order, and generate a selected sample Pareto frontier set. The acquisition step of the high-value sample index is:

2. The incremental learning based mine AI model fine-tuning and optimization management method according to claim 1, characterized in that, According to the selected sample Pareto frontier set, mark the corresponding tunneling machine state data sample according to whether the coordinate point is located on the boundary of the set, extract the unique index of the marked tunneling machine state data sample and store it according to the original order, and generate a high-value sample index. The acquisition step of the incremental training key sample set is: ​ ​ 3. The incremental learning based mine AI model fine-tuning and optimization management method according to claim 1, characterized in that, ​ ​ 4. The incremental learning based mine AI model fine-tuning and optimization management method according to claim 1, characterized in that, ​ According to the high-value sample index, corresponding original heading machine state data is read from the incremental data stream according to the index position, the timestamp and sensor identifier of each piece of original heading machine state data are recorded, manual annotation is submitted after recording, the manual annotation result and the corresponding relationship of the original heading machine state data are summarized, and the heading machine state data with completed annotation is generated; According to the heading machine state data with completed annotation, the sample order is organized according to the order of sample timestamps and the grouping strategy of device identifiers, the fixed batch capacity is set and filled in sequence to generate data batches for model fine-tuning, all data batches for model fine-tuning are integrated and index mapping is retained, and an incremental training key sample set is formed.

5. The incremental learning based mine AI model fine-tuning and optimization management method according to claim 1, characterized in that, The acquisition step of the current knowledge density value is: According to the incremental training key sample set, sample batches are loaded into the current model according to the timestamp order, the prediction probability and the true label are compared batch by batch, and the parameter state after each batch update and the covered sample range are recorded, and an updated model is generated; According to the updated model, the prediction category is generated for each image in the ore grade image verification set, the correct prediction number and the total prediction number are accumulated to calculate the classification accuracy, which is denoted as the model performance value, and the stable rank of each layer weight matrix is calculated according to the matrix norm, and the sum of the stable ranks of each layer is calculated to generate the effective complexity measure of the model; According to the model performance value and the effective complexity measure of the model, the current knowledge density value is calculated.

6. The incremental learning based mine AI model fine-tuning and optimization management method according to claim 1, characterized in that, The acquisition step of the model reconstruction trigger signal is: According to the sequence of the current knowledge density value, the adjacent item difference value sign is calculated in time sequence and the number of items with negative end is accumulated, the accumulated item number is compared with the continuous decline determination standard, and the latest current knowledge density value is compared with the preset threshold value, if both conditions are met, the identification is set, and the model reconstruction trigger signal is generated.

7. The incremental learning based mine AI model fine-tuning and optimization management method according to claim 1, characterized in that, The acquisition step of the knowledge compactification new baseline model is: According to the model reconstruction trigger signal, the parameter tensor memory of the student model is allocated according to the layer list and the random number generator state is set to complete random initialization, the current model is specified as the teacher model, the time field and the statistical field of the support pressure monitoring value summary data are analyzed to generate the input batch queue, and the student model initialization state is formed.

8. The incremental learning based mine AI model fine-tuning and optimization management method according to claim 1, characterized in that, The acquisition step of the knowledge compactification new baseline model further includes: according to the student model initialization state, the current model soft target is generated on the support pressure monitoring value summary data according to the input batch queue, and the student model output is generated synchronously, the soft target difference is calculated and the parameter is updated, until the difference reduction amplitude of the continuous round is lower than the stop threshold value, the parameter state of the student model with the optimal performance is retained as the result, and the knowledge compactification new baseline model is formed.

Images