A ore classification method combining transfer learning and model pruning

Abstract

The present invention discloses an ore classification method integrating transfer learning and model pruning, comprising the following steps: obtaining a pre-trained model in a data model; re-initializing the shallow part of the pre-trained model according to the amplitude or similarity index to obtain a partially initialized model; and re-fine-tuning the model after the shallow part is initialized on a target data set. The present invention introduces transfer learning to solve the problem of insufficient ore data sets when applying deep learning models, so as to better learn the features on the target data set while using the features learned by the model on the pre-trained data set; the present invention uses amplitude and similarity metrics, two widely used indicators, to measure the similarity of convolution kernels, and uses the idea of ​​model pruning to partially initialize the shallow layer of the model to release redundant parameters; the present invention fine-tunes the partially initialized model on the antimony ore data set to obtain a model that is more suitable for ore characteristics.

Description

Technical Field

[0001] The present invention relates to the technical field of ore sorting processes, and in particular to an ore classification method integrating transfer learning and model pruning. Background Art

[0002] Although traditional mineral processing methods, such as hand selection, gravity selection, and flotation, can achieve ore sorting to a certain extent, they each have some limitations and shortcomings, such as low recognition accuracy, large space occupation, high investment costs, etc., and there are also environmental and safety challenges. With the development of computer technology and image processing technology, machine vision technology has become an important application direction in the field of mineral sorting. Among them, ore sorting technology based on dual-energy X-rays has attracted widespread attention due to its high efficiency, green environmental protection, and economy. This technology uses the differences in the interaction between X-rays and matter at different energies. By analyzing the image after the X-rays penetrate the mineral, different types of minerals can be effectively distinguished.

[0003] At present, the classification of X-ray mineral images is mainly based on the mechanism model classification method represented by the R value method. Under ideal conditions, the R value of the ratio of the mass attenuation coefficients of the high energy and low energy levels in the monochromatic energy spectrum is calculated as follows:

[0004]

[0005] In the formula are the initial intensity of X-rays at high energy and low energy, μ m (E high ), μ m (E low ) is the mass absorption coefficient of the substance at high and low energy, σx(E high ),σ x (E low) is the total cross section of the material at high and low energy. In the case of dual-energy single spectrum of X-ray, the mass thickness can be simply cancelled. At this time, the calculation of R value is not affected by factors such as mass and thickness. Therefore, R value reflects the average atomic number of the material, so materials with different atomic numbers can be distinguished. However, in actual use, the R value method is susceptible to beam hardening effect, thickness effect, afterglow effect, etc. In actual practice, an X-ray tube with a continuous spectrum is generally selected as the X-ray light source. At this time, it does not meet the above-mentioned dual-energy spectrum conditions. When the continuous spectrum passes through an object, low-energy rays are easily absorbed and high-energy rays are easy to pass. Therefore, during the propagation of rays, the average energy will become higher and the rays will gradually become harder. This phenomenon is called beam hardening effect. In addition, in the case of continuous spectrum, the influence of the mass thickness of the object cannot be ignored, and the calculation of R value will be affected. When the ore transmitted on the belt passes through the X-ray transmission area, the signal intensity on the X sensor does not suddenly increase or decrease, but changes slowly. This phenomenon of delayed change of X-ray signal is called the afterglow effect of X-ray detector. The afterglow effect will cause the grayscale information of the object's image edge to deviate, which will result in the inability to accurately extract the edge contour of the object during segmentation. In addition, it will further affect the effect of the adaptive R value method in judging the material category.

[0006] In recent years, with the increasing application of deep learning, there are many researchers who have applied visual technology to mines to improve the efficiency of ore sorting. Based on the Inception-v3 network, the automatic classification of three types of rocks, granite, phyllite and breccia, was realized. The LeNet-5 convolutional neural network was optimized to complete the binary classification task of coal and gangue. The improved PSO-Faster R-CNN algorithm was used to identify the ore, and the speed and accuracy were improved to a certain extent. The pre-trained MobilenetV2 model was used to detect material accumulation through transfer learning, and EfficientNet was used to extract features from ore images to achieve the classification of different ores. However, when using deep learning for ore classification, it often faces the problem of insufficient data sets, and transfer learning can alleviate this problem. Based on the idea of ​​transfer learning, the VGG16 shallow model was frozen, and then the fully connected layer was customized to increase the accuracy of gangue recognition to 82.5%. The migrated deep learning model was used to classify wolframite ore and surrounding rock, achieving a recognition rate of 97.51%. Summary of the invention

[0007] In order to solve the problem that the current X-ray ore classification is mainly based on the mechanism model represented by the R value method, the ore classification accuracy is not high and the R value threshold correction is complicated due to the single feature used in X-ray ore classification, which is easily affected by the thickness effect, beam hardening effect and afterglow effect. The present invention provides an ore classification method that integrates transfer learning and model pruning. In order to achieve the above technical purpose, the present invention adopts the following technical solutions:

[0008] A method for ore classification integrating transfer learning and model pruning comprises the following steps:

[0009] S1: Get the pre-trained model in the data model;

[0010] S2: reinitialize the shallow part of the pre-trained model according to the amplitude or similarity index to obtain a partially initialized model;

[0011] S3: Fine-tune the model after the shallow part initialization on the target dataset.

[0012] Preferably, step S1 is specifically as follows:

[0013] S11: Perform transfer learning on multiple experimental data sets, and select data sets with positive transfer effects based on the results of transfer learning;

[0014] S12: Select a pre-trained model, analyze the network structure of the pre-trained model, and define the target shallow area of ​​the corresponding model.

[0015] Preferably, step S2 is specifically as follows:

[0016] S21: Treat the shallow part of the pre-trained model as a whole, identify the mutually dependent parameter groups and process them as a whole, that is, the dependent group {G1, G2, …G i};

[0017] S22: For a neural network with L layers, N i and N i+1 denote the number of input and output channels of the i-th layer respectively, then the j-th convolution kernel of the i-th layer with a size of K×K is expressed as Therefore, for the i-th convolutional layer, the set of convolution kernels contained in the network is represented as F i,j , where 1≤j≤N i+1 , whose weight value is expressed as The i-th convolution kernel F i,j The weight value is represented as a one-dimensional vector This means that the i-th convolutional layer contains N i+1 convolution kernel vectors, each of which has a size of M i =N i×K×K; introduce amplitude clipping ratio w mag As a decision factor, the clipping ratio P is introduced to determine when to use the amplitude or similarity index;

[0018] Select the convolution kernel according to the amplitude index of the dependent group; use the norm of the convolution kernel as a way to measure the importance of the convolution kernel; convolve the j-th layer x j Expressed as Then its p-norm can be calculated as follows:

[0019]

[0020] After calculating the amplitude of the convolution kernel, it is necessary to sort them by size and calculate the target cropping rate P×w. mag Determine the dependency group to be pruned;

[0021] S23: Reinitialize until the amplitude clipping ratio is exceeded, then select the similarity index of the dependent group convolution kernel, sort the similarity, and reinitialize again; the cosine similarity shown in the following formula is used as the evaluation index, where x and y represent the submodules in the dependent group:

[0022]

[0023] After calculating the pairwise similarity of the convolution kernels and obtaining the similarity matrix, in order to determine the dependent groups to be pruned, it is necessary to sort them according to the similarity size and calculate the target pruned rate P×(1-w mag ) Determine the number of sub-modules to be pruned, and select one of the two convolution kernels in the convolution kernel pair as the sub-module to be pruned W i ;

[0024] S24: Execute S22 and S23, iteratively process unimportant convolution kernels, obtain their corresponding dependency groups, and partially reinitialize all sub-modules in the dependency groups. After calculating the latest reinitialization rate, compare it with the target initialization rate. If it is not satisfied, continue to execute S22 and S23 until the initialization ratio exceeds the target initialization ratio.

[0025] Preferably, step S3 is specifically as follows:

[0026] S31: After obtaining the reinitialized model M with some shallow parameters released r Afterwards, the shallow parameters are assigned one by one to the fully reinitialized model M n After that, the target model M can be obtained by initializing the shallow layer and the deep layer. n' ;

[0027] S32: Model M n' In the target dataset Dt Train on it until the model converges, and then you can get the final fine-tuned model M d .

[0028] The present invention proposes an ore classification method that integrates transfer learning and model pruning. It uses a pre-trained deep learning model to reduce the dependence on a large amount of labeled data through transfer learning, and releases the shallow redundant parameters of the model by partially reinitializing the shallow layer of the model, so that the shallow part of the fine-tuned model can better reflect the characteristics of the target data set.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] 1. The present invention introduces transfer learning to solve the problem of insufficient ore data sets when applying deep learning models, so as to better learn the features on the target data set while using the features learned by the model on the pre-training data set.

[0031] 2. The present invention uses two widely used indicators, amplitude and similarity measurement, to measure the similarity of convolution kernels, and utilizes the idea of ​​model pruning to partially initialize the shallow layer of the model to release redundant parameters.

[0032] 3. The present invention fine-tunes the partially initialized model on the antimony ore dataset to obtain a model that is more suitable for the ore characteristics. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present invention or the technical solutions of the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0034] Figure 1 Flowchart of the ore classification method that combines transfer learning and model pruning;

[0035] Figure 2 Initialize the flow chart for the shallow part of the model that fuses the amplitude and similarity indicators;

[0036] Figure 3 w mag Impact on ore sorting accuracy;

[0037] Figure 4 The accuracy of the model on the antimony ore dataset changes with the clipping rate;

[0038] Figure 5 This is the trend of the model's accuracy in DogVs Cat changing with the cropping rate. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] Example

[0041] The present invention provides an ore classification method integrating transfer learning and model pruning. All simulation experiments are run on a computer equipped with an Intel-i9-12900kx 24CPU and a GenForce RTX3090 GPU. Figure 1 As shown, the following steps are included:

[0042] S1: Get the pre-trained model in the data model; the details are as follows:

[0043] S11: Perform transfer learning of multiple experimental data sets, and select data sets with positive transfer effects for use according to the effect of transfer learning; the accuracy of the ore classification model of the present invention when pre-training is performed using four data sets of Mineral, Chest X-ray, Dog VsCat, and ImageNet without transfer learning;

[0044] The training set and test set division of the fine tailings data set are shown in Table 1:

[0045] Table 1. Concentrate tailings data set

[0046]

[0047] The experimental results obtained by training tailings on the antimony ore dataset without using the pre-training dataset and using the Mineral, Chest X-Ray, DogVs Cat, and ImageNet datasets are shown in Table 2:

[0048] Table 2 Model accuracy under different pre-training datasets

[0049]

[0050] From the experimental results, we can see that although the target task of this method is the classification of antimony ore by X-ray, the Mineral and ChestX-ray datasets, which are highly correlated with the target dataset, have a negative effect on the improvement of antimony ore classification accuracy, while the DogVs Cat and ImageNet datasets promote antimony ore classification. This shows that the scale and number of datasets have a greater impact on migration, which helps the model learn a wider range of features.

[0051] S12: Select a pre-trained model, analyze the pre-trained model network structure, and define the target shallow area of ​​the corresponding model: The present invention uses VGG16 and ResNet18 models for deep learning models for classification; the VGG16 model structure contains 16 weight layers, namely 13 convolutional layers {L c1 ,L c2 ,…,L c3} and 3 fully connected layers {L l1 ,L l2 ,L l3}; The convolutional layer is divided into multiple repeated units, each unit contains two or three convolutional blocks using small 3×3 convolutional kernels, followed by a maximum pooling layer; after the convolutional layer, there are three fully connected layers, the first two of which have 4096 neurons each, and the number of neurons in the last fully connected layer depends on the number of categories of the classification task; on the VGG16 model, the first three convolutional layers are used as the target shallow area L for partial reinitialization of this method shallow ={L c1 ,L c2 ,L c3}; The ResNet18 model contains 18 weight layers and 17 convolutional layers {L c1 ,L c2 ,…,L c17} and a fully connected layer L l1 ; The convolution layer is divided into multiple repeated units, which are mixed with pooling layers, 3×3 convolution kernels, and 7×7 convolution kernels. The first three convolution layers are taken as the shallow part, that is, L shallow ={L c1 ,L c2 ,L c3}.

[0052] S2: Reinitialize the shallow part of the pre-trained model according to the amplitude or similarity index to obtain a partially initialized model, as follows:

[0053] S21: Treat the shallow part of the pre-trained model as a whole, identify the mutually dependent parameter groups and process them as a whole, that is, the dependent group {G1, G2, …G i};

[0054] S22: For a neural network with L layers, N i and N i+1 denote the number of input and output channels of the i-th layer respectively, then the j-th convolution kernel of the i-th layer with a size of K×K is expressed as Therefore, for the i-th convolutional layer, the set of convolution kernels contained in the network is represented as F i,j , where 1≤j≤N i+1 , whose weight value is expressed as The i-th convolution kernel F i,j The weight value is represented as a one-dimensional vector This means that the i-th convolutional layer contains N i+1 convolution kernel vectors, each of which has a size of M i =N i ×K×K; introduce amplitude clipping ratio w mag As a decision factor, the clipping ratio P is introduced to determine when to use the amplitude or similarity index;

[0055] Select the convolution kernel according to the amplitude index of the dependent group; use the norm of the convolution kernel as a way to measure the importance of the convolution kernel; convolve the j-th layer x j Expressed as Then its p-norm can be calculated as follows:

[0056]

[0057] After calculating the amplitude of the convolution kernel, it is necessary to sort them by size and calculate the target cropping rate P×w. mag Determine the dependency group to be pruned;

[0058] S23: Reinitialize until the amplitude clipping ratio is exceeded, then select the similarity index of the dependent group convolution kernel, sort the similarity, and reinitialize again; the cosine similarity shown in the following formula is used as the evaluation index, where x and y represent the submodules in the dependent group:

[0059]

[0060] After calculating the pairwise similarity of the convolution kernels and obtaining the similarity matrix, in order to determine the dependent groups to be pruned, it is necessary to sort them according to the similarity size and calculate the target pruned rate P×(1-w mag ) Determine the number of sub-modules to be pruned, and select one of the two convolution kernels in the convolution kernel pair as the sub-module to be pruned W i ; The magnitude importance criterion and similarity importance criterion are used to select the submodules that need to be partially initialized. These two submodule pruning criteria are used by using the magnitude pruning ratio w magto determine when to use the magnitude criterion and when to use the similarity criterion, and w mag It is necessary to dynamically adjust the model for different models and data sets to optimize the overall performance of the model. mag Regarding the impact on model accuracy, taking the recognition accuracy of the VGG16 and ResNet18 models pre-trained on ImageNet after migration on the antimony ore dataset as an example, different w mag The influence of the selection on the precision identification of antimony ore is shown in Table 3. The data in the table are plotted as a dot-line graph as shown in Figure 3 As shown, P vm , P vs They are the mean and variance on the VGG16 model, P rm and P rs are the mean and variance of the ResNet18 model. The table shows w mag The average and standard deviation of the accuracy of antimony ore identification obtained after 5 experiments with different values. It can be seen from the table that the amplitude index and the similarity index alone cannot obtain the best recognition accuracy, so the best parameters should be selected experimentally for specific models and data sets.

[0061] Table 3w mag Impact on ore sorting accuracy

[0062]

[0063] S24: Execute S22 and S23, iteratively process unimportant convolution kernels, obtain their corresponding dependency groups, and partially reinitialize all submodules in the dependency groups. After calculating the latest reinitialization rate, compare it with the target initialization rate. If it is not satisfied, continue to execute S22 and S23 until the initialization ratio exceeds the target initialization ratio. Figure 2 shown.

[0064] In order to better illustrate the effectiveness of the method of the present invention, the method of the present invention is applied to the VGG16 and ResNe18 models and its accuracy is evaluated on the antimony ore dataset. mag When the crop rate P is set to 0.5, the table of different precision data obtained after 5 experiments with different precisions is shown in Table 4. The data in the table are plotted as a dot-line graph as shown in Figure 4 shown.

[0065] Table 4 Accuracy of ore sorting model

[0066]

[0067] Although a given cropping rate P is specified, since the idea of ​​structured cropping is used, the convolution kernels with dependencies in the computational graph will be reinitialized, so the proportion of reinitialized parameters is P r will be greater than the given clipping rate P; by observing Figure 4 The data presented show an overall trend about the relationship between model performance and cropping rate; in this trend, the classification performance of the model experiences a process of first decreasing, then increasing, and then decreasing again as the cropping rate increases; specifically, when the cropping rate reaches a certain value, the classification performance of the model reaches the optimal state; in this experiment, when the specified cropping rate P = 0.3 and the actual cropping rate P r When P = 0.52, the VGG16 model performs best on the antimony ore dataset, achieving a classification accuracy of 86.76%. In terms of precision, the VGG16 model also has the smallest variance at this time, which means that the classification performance of the model after multiple training is more stable than that at other clipping rates. This performance improvement shows a significant improvement compared with the classification model that only adopts the transfer learning strategy (P = 0). The ResNet18 model also has a similar trend in accuracy. This shows that the present invention can effectively optimize the structure of the model, thereby obtaining better performance on specific tasks.

[0068] S3: Fine-tune the model after the shallow part initialization on the target dataset as follows:

[0069] S31: After obtaining the reinitialized model M with some shallow parameters released r Afterwards, the shallow parameters are assigned one by one to the fully reinitialized model M n After that, the target model M can be obtained by initializing the shallow layer and the deep layer. n' ;

[0070] S32: Model M n' In the target dataset D t Train on it until the model converges, and then you can get the final fine-tuned model M d .

[0071] The present invention tests the classification effect of the DogVsCat dataset model on the VGG16 and ResNet18 pre-trained models at different cropping rates. The results are shown in Table 5 and Figure 5 shown.

[0072] Table 5 Classification effect of pre-trained model on DogVs Cat dataset

[0073]

[0074]

[0075] from Figure 5 It can be seen that under different cropping rates, the classification accuracy of the model shows a trend of first increasing and then decreasing, which is similar to the observations on the antimony ore dataset. This trend shows that there is an optimal cropping rate that allows the model to maximize the adaptation to the characteristics of the target dataset while maintaining the original pre-training knowledge. In the experiment, when the target initialization rate P is 0.1, the model has the highest average accuracy, which means that at this initialization rate, the model successfully balances the knowledge transfer between the source and target datasets and effectively learns the characteristics of the DogVs Cat dataset.

[0076] This result further confirms the versatility of the proposed method. It is not only applicable to the classification of ore images, but can also be extended to other types of image datasets and classification tasks. This shows that by reasonably selecting the cropping rate and applying the strategy of transfer learning and initializing the shallow part of the model, the classification effect of the model on various datasets can be effectively improved. In addition, this versatility also means that the proposed method can be used as an effective tool to quickly adapt to new classification tasks in scenarios where the amount of data is limited or the model needs to be deployed quickly.

Claims

1. A method for ore classification combining transfer learning and model pruning, characterized in that: The steps include: S1: Obtain a pre-trained model in the data model; including performing transfer learning of multiple sets of experimental data sets, and selecting a data set that has a positive transfer effect on ore classification based on the effect of transfer learning; S2: Reinitialize the shallow part of the pre-trained model according to the amplitude or similarity index to obtain a partially initialized model; step S2 is as follows: S21: Treat the shallow part of the pre-trained model as a whole, identify the mutually dependent parameter groups and process them as a whole, i.e., the dependent group ; S22: For layer of neural network, and Respectively represent The number of channels of the input and output of the layer, then Layer Has The convolution kernel size is expressed as , so for the The convolution layer of the layer, the set of convolution kernels contained in the network is expressed as ,in , whose weight value is expressed as ;No. Layer convolution kernel The weight value is represented as a one-dimensional vector , which means that in The convolutional layers contain convolution kernel vectors, each of which has a size of ; Introduce amplitude clipping ratio As a decision factor, the cropping rate is introduced to determine when to use magnitude or similarity metrics; Select the convolution kernel according to the amplitude index of the dependent group; use the norm of the convolution kernel as a way to measure the importance of the convolution kernel; Layer Convolution Expressed as , then its The norm can be calculated as follows: After calculating the amplitude of the convolution kernel, you need to sort them by size and according to the target cropping rate Determine the dependency group to be pruned; S23: Reinitialize until the amplitude clipping ratio is exceeded, then select the similarity index of the dependent group convolution kernel, sort the similarity, and reinitialize again; the cosine similarity shown in the following formula is used as the evaluation index, where , Represented as submodules in the dependency group: After calculating the pairwise similarity of the convolution kernels and obtaining the similarity matrix, in order to determine the dependent groups to be pruned, it is necessary to sort them according to the similarity size and calculate the target pruned rate. Determine the number of submodules to be pruned, and select one of the two convolution kernels in the convolution kernel pair as the submodule to be pruned ; S24: Execute S22 and S23, iteratively process unimportant convolution kernels, obtain their corresponding dependency groups, and partially reinitialize all submodules in the dependency groups. After calculating the latest reinitialization rate, compare it with the target initialization rate. If it is not satisfied, continue to execute S22 and S23 until the initialization ratio exceeds the target initialization ratio. S3: The model initialized with the shallow part is re-fine-tuned on the antimony ore dataset.

2. The ore classification method combining transfer learning and model pruning according to claim 1 is characterized in that: Step S1 is specifically as follows: S11: Perform transfer learning on multiple experimental data sets, and select data sets with positive transfer effects based on the results of transfer learning; S12: Select a pre-trained model, analyze the network structure of the pre-trained model, and define the target shallow area of ​​the corresponding model.

3. The ore classification method combining transfer learning and model pruning according to claim 1 is characterized in that: Step S3 is as follows: S31: After obtaining the reinitialized model with some shallow parameters released After that, the shallow parameters are assigned one by one to the fully reinitialized model After that, the target model with shallow partial initialization and deep full initialization can be obtained. ; S32: The model In the target dataset Train on it until the model converges, and you can get the final fine-tuned model .

Images