Lead flotation froth image classification method and system based on deep autonomous learning

Abstract

The invention relates to a computer vision technology, in particular to a lead flotation froth image classification method and system based on deep autonomous learning, and the method comprises the steps: constructing an image classification network fused by an OfficientNetB6 and Transform network structure, obtaining image data with classification labels and image data without labels, and carrying out the preprocessing of an image; sampling m samples from the image data with the classification labels to carry out initialization training on the image classification network; selecting n pieces of data with the maximum information amount from the unlabeled image data through autonomous learning, and carrying out manual labeling; and further training the initialized training data by using the selected data to obtain a trained image classification network. According to the method, the training speed of the model and the classification capability of the complex foam image are remarkably improved, and the number of samples required by labeling is reduced, so that the manpower and time cost is reduced.

Description

Technical Field

[0001] The present invention relates to computer vision technology, and in particular to a lead flotation foam image classification method and system based on deep autonomous learning. Background Art

[0002] Flotation technology is a separation technology commonly used in mineral processing. It is a mature method for effectively separating minerals in water slurry. This process relies on the surface characteristics of mineral particles. During the flotation process, the target mineral particles will selectively adhere to the rising bubbles after adding specific flotation reagents, and float to the surface with them to form a mineral-rich foam layer. This layer of foam not only carries the concentrate rich in useful minerals, but also serves as a window for evaluating flotation performance. By observing its color, stability, flow rate and bubble size, immediate feedback on the flotation process status can be obtained. The concentrate grade and tailings grade, as well as the final recovery rate, are key indicators that determine the economic benefits of the flotation process. They are directly related to the profit and loss balance of mining companies. Therefore, the detailed execution of the flotation process and the scientific and precise monitoring and control have become technical problems that engineers continue to tackle. In traditional operations, experienced operators rely on their intuitive judgment of the foam state to make operational decisions in the flotation tank. However, the individual differences and subjective decision-making uncertainties brought about by this method cause fluctuations in flotation effects, often leading to many problems such as insufficient raw material utilization, excessive reagent consumption, and low resource recovery efficiency.

[0003] The introduction of machine vision technology has shown great potential in the extraction of flotation foam surface features, opening up new ways to accurately identify the working conditions of minerals. With the continuous evolution of technology, especially the continuous breakthroughs in deep learning, deep convolutional neural networks have shown excellent performance in image recognition, classification, and segmentation. Models based on deep convolutional neural networks not only improve the accuracy of image processing, but also can quickly identify different types of foam images, thereby effectively guiding the flotation process. However, a severe challenge is that deep learning models usually require large labeled data sets for training. The labels of these data usually rely on the rich experience and knowledge of domain experts to accurately mark them, and this process requires a lot of manpower and time costs. Summary of the invention

[0004] In order to improve the accuracy of image classification and significantly reduce the overall cost and labor input of labeling, the present invention proposes a lead flotation foam image classification method based on deep autonomous learning, constructs an image classification network fused with EfficientNetB6 and Transformer network structures, and uses the trained image classification network to perform image classification. The training process of the image classification network includes:

[0005] Obtain image data with classification labels and unlabeled image data, and preprocess the images;

[0006] Sample m samples from image data with classification labels to initialize the image classification network for training;

[0007] Through autonomous learning, n data with the largest amount of information are selected from unlabeled image data for manual annotation;

[0008] The selected data is used to further train the initialized training data to obtain a trained image classification network.

[0009] Furthermore, the image classification network consists of cascaded convolutional layers, EfficientNetB6, Transformer networks, and linear layers, where EfficientNetB6 consists of seven cascaded convolutional modules, and each convolutional module consists of one or more convolutional units.

[0010] Furthermore, the seven convolution modules that cascade to form EfficientNetB6 are respectively composed of one convolution unit, two convolution units cascaded, two convolution units cascaded, three convolution units cascaded, three convolution units cascaded, four convolution units cascaded, and one convolution unit.

[0011] Furthermore, the data processing process of the convolution unit includes: the input feature map is processed through a cascaded convolution layer, a depth-separable convolution layer, a self-attention module, a convolution layer, and a dropout layer to obtain a feature map, and the feature map is added and fused with the input feature map through a jump connection layer as the output of the convolution module.

[0012] Furthermore, the data processing process of the self-attention module includes: the input feature map is processed by a cascaded global average pooling layer and two fully connected layers, and the feature map obtained is multiplied and fused with the input feature map through a skip connection layer as the output of the self-attention module.

[0013] Furthermore, the Transformer network structure consists of four cascaded sub-layers with the same structure, each sub-layer includes a multi-head attention layer and a position-based feedforward network, and each sub-layer adopts a residual connection.

[0014] Furthermore, the linear layer processes the data by sequentially processing the feature map output by the Transformer network structure through a global average pooling layer, a fully connected layer, and a softmax classifier.

[0015] The present invention also proposes a lead flotation foam image classification system based on deep autonomous learning, which is used to implement a lead flotation foam image classification method based on deep autonomous learning, including using a trained image classification network to perform a lead flotation foam image classification task, and the training process of the image classification network includes:

[0016] Obtain image data with classification labels and image data without labels, and preprocess the images;

[0017] Sample m samples from image data with classification labels to initialize the image classification network for training;

[0018] Use the initialized trained image classification network to classify and predict the unlabeled image data, and select n samples with the lowest classification prediction confidence for manual labeling;

[0019] The manually annotated image data is added to the image data with classification labels, and used to train the image classification network again.

[0020] Compared with the prior art, the technical solution of the present invention has the following significant advantages:

[0021] The present invention proposes a novel deep autonomous learning algorithm and system for lead flotation foam image classification, which is particularly suitable for processing flotation foam images captured by industrial field cameras, and through a new loss function designed to solve the problem of class imbalance, a deep autonomous learning solution framework is innovatively constructed. The framework adjusts and integrates the characteristics of EfficientNetB6 and Transformer, and creates a fusion convolutional neural network model as the power engine for the foam image classification problem. The model uses its convolutional neural network part to deeply learn the unique characteristics of flotation foam, while the autonomous learning part focuses on the effective labeling of unlabeled data. The samples with the richest information are intelligently selected and integrated into the labeled data set for model training and updating. This method greatly reduces the burden of manual labeling, while achieving accurate capture of information. The combination of deep learning and autonomous learning methods significantly improves the model's training speed and the ability to classify complex foam images, reduces the number of samples required for labeling, and thus reduces labor and time costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a comprehensive flow chart of a novel deep autonomous learning lead flotation froth image classification algorithm in one embodiment of the present invention;

[0023] Figure 2 1 is a flotation froth image under five different working conditions of the froth flotation process in one embodiment of the present invention;

[0024] Figure 3 It is a structural diagram of the novel fusion convolutional neural network model proposed in the present invention;

[0025] Figure 4 It is a framework diagram of the deep autonomous learning proposed in this invention. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] The present invention proposes a lead flotation foam image classification method based on deep autonomous learning, constructs an image classification network fused with EfficientNetB6 and Transformer network structures, and uses the trained image classification network to perform image classification. The training process of the image classification network includes:

[0028] Obtain image data with classification labels and unlabeled image data, and preprocess the images;

[0029] Sample m samples from image data with classification labels to initialize the image classification network for training;

[0030] Through autonomous learning, n data with the largest amount of information are selected from unlabeled image data for manual annotation;

[0031] The selected data is used to further train the initialized training data to obtain a trained image classification network.

[0032] The core objective of the present invention is to introduce a new lead ore flotation foam image classification algorithm and corresponding system. By accurately classifying flotation foam images, it is helpful to conduct real-time analysis of the slurry working conditions, so that the flotation operation variables can be accurately controlled, and the continuous optimization and real-time management of the entire flotation process can be achieved. In the face of the problems of category imbalance and high labeling workload in the current foam image classification, the present invention proposes a method that combines the powerful feature extraction capabilities of deep learning and autonomous learning to reduce the need for manual labeling. This method not only greatly saves the labeling workload by intelligently selecting samples that have the greatest contribution to improving model performance for labeling, but also effectively improves the accuracy of image classification and the efficiency of the training process, providing strong technical support for the efficient processing and resource recovery of lead ore. Figure 1 The present invention performs data enhancement on the collected data and then trains the model, and uses the trained model for prediction of unlabeled samples. The method specifically includes the following steps:

[0033] Step 1: Dataset construction;

[0034] In the lead fast selection process, foam images are collected to construct a foam image dataset, which consists of two parts: one is a sample set with clear labels (X L , Y L ), and the other part is the unlabeled sample set X to be labeled U ;

[0035] Step 2: Data preprocessing and enhancement;

[0036] The collected foam images are subjected to a series of preprocessing measures, including flipping, rotation, brightness processing and other operations, to achieve data enhancement and expand and enrich the sample image set, and then all foam images are divided into a training set and a test set in proportion. In this embodiment, the data of the training set is used for model training, and the data of the test set is used to verify the effectiveness of the present invention;

[0037] Step 3: Build a deep network architecture;

[0038] The deep network architecture is based on the fusion of EfficientNetB6 and Transformer network models, which includes 1 3×3 convolutional layer, 7 MBConv modules, and 1 Transformer encoder block. The Transformer encoder block is set after the 7 MBConv modules to better handle complex patterns and relationships in the image.

[0039] Step 4: Perform initial training of the network model;

[0040] From the labeled sample set (X L , Y L ) randomly select m samples to form the initial training set L=(x1,x2,......,x m ), and then input these samples into the network model to be trained to start the initial training of the model;

[0041] Step 5: Sample screening and model updating;

[0042] By autonomous learning from an unlabeled sample set X U The most informative samples are selected for manual annotation, and then these newly annotated samples are used to further train and adjust the current model;

[0043] Step 6: Obtain the fusion model and perform image classification;

[0044] After completing all the training adjustments, we get an efficient convolutional neural network model that combines the Transformer and EfficientNetB6 architectures. The foam image data to be identified is input into this fusion model for classification and identification, and the final classification decision of the foam image is produced.

[0045] As an optional implementation, this embodiment divides the images in the foam image dataset into five different categories according to their characteristics, such as Figure 2 , marked as Class I, Class II, Class III, Class IV, and Class V, respectively, representing abnormal, qualified, medium, good, and excellent conditions, with the grade value ranges being (-, 43], (43, 44], (44, 45], (45, 46], and (46, -).

[0046] like Figure 3 In this embodiment, the image classification network fused with the EfficientNetB6 and Transformer network structures includes the following structure:

[0047] The convolution (Conv) layer receives the 512×512 pixel foam image from the data layer, and convolves the input data with a 3×3 convolution kernel with a step size of 2. After batch normalization, ReLU activation function, and maximum pooling layer, 32 feature maps of 224×224 are obtained and passed to the first convolution (MBConv1) module;

[0048] The multiplication factor in the MBConv1 module is 1. The MBConv1 module consists of one convolution unit (MBConv). From front to back, one MBConv includes one 1×1 Conv layer, one Depwise Conv layer (3×3 convolution), one SE module, one 1×1 Conv layer, one Dropout layer, and a shortcut connection. The SE module consists of a global average pooling and two FC layers. Finally, 16 feature maps of size 112×112 are extracted and passed to the second convolution (MBConv2) module.

[0049] The multiplication factor in the MBConv2 module is 6. The MBConv2 module consists of two MBConv modules. The convolution kernel size used by DepthwiseConv is 3×3. Finally, 24 feature maps of size 112×112 are extracted and passed to the third convolution (MBConv3) module.

[0050] The multiplication factor in the MBConv3 module is 6. The MBConv3 module consists of two MBConv modules. The convolution kernel size used by DepthwiseConv is 5×5. Finally, 40 feature maps of size 224×224 are extracted. The fourth convolution (MBConv4) module;

[0051] The multiplication factor in the MBConv4 module is 6. The MBConv4 module consists of 3 MBConv modules. The convolution kernel size used by DepthwiseConv is 3×3. Finally, 80 feature maps of size 112×112 are extracted and passed to the fifth convolution (MBConv5) module.

[0052] The multiplication factor in the MBConv5 module is 6. The MBConv5 module consists of 3 MBConv modules. The convolution kernel size used by DepthwiseConv is 5×5. Finally, 112 feature maps of size 64×64 are extracted and passed to the sixth convolution (MBConv6) module.

[0053] The multiplication factor in the MBConv6 module is 6. The MBConv6 module consists of 4 MBConv modules. The convolution kernel size used by DepthwiseConv is 5×5. Finally, 192 feature maps of size 32×32 are extracted and passed to the seventh convolution (MBConv7) module;

[0054] The multiplication factor in the MBConv7 module is 6. The MBConv7 module consists of 1 MBConv module. The convolution kernel size used by DepthwiseConv is 3×3. Finally, 320 feature maps of size 32×32 are extracted and passed to the Transformer encoder module.

[0055] The Transformer encoder module is composed of 4 identical layers stacked together, each of which has two sublayers. The first sublayer is multi-head self-attention aggregation; the second sublayer is a position-based feedforward network;

[0056] When calculating the encoder’s self-attention, the query, key, and value all come from the output of the previous encoder layer, and each sublayer uses residual connections. Finally, 1280 feature maps of size 32×32 are extracted. The output of the Transformer layer first passes through a global average pooling layer, which is responsible for averaging the feature maps to reduce the dimensionality, thereby producing a one-dimensional feature vector.

[0057] This vector is then sent to a fully connected layer, where the Dropout technique is used to randomly discard a certain proportion of neuron outputs in order to prevent the model from overfitting due to too much training data.

[0058] After being processed by the fully connected layer, the final output is sent to the softmax classifier, which is responsible for outputting the final category result.

[0059] In the present invention, the process of selecting n data with the largest amount of information from unlabeled image data through autonomous learning for manual labeling includes the following steps:

[0060] First, the unlabeled sample set X is evaluated using the autonomous learning strategy U The amount of information of each unlabeled sample in , and the samples are sorted from high to low according to their information amount;

[0061] Next, the top K samples with the most information are selected for manual annotation to generate new sample label pairs (x * ,y * );

[0062] Incorporate these newly labeled sample pairs into the labeled sample set (X L , Y L ) as a new data set to optimize and adjust the training model;

[0063] By iteratively executing this process, the model continues to learn and adapt until the predetermined performance standard is achieved or all unlabeled sample sets have been labeled; finally, the fine-tuned convolutional neural network model is saved.

[0064] like Figure 4 ,The autonomous learning strategy adopts an autonomous learning mechanism with built-in loss prediction function, which adds a loss prediction module to the deep learning model to estimate the possible loss value of unlabeled samples, so as to quantify the amount of information of each sample in the unlabeled sample pool. In this way, the model can identify and select the top K samples with the highest predicted loss value, then annotate these samples and add them to the training set for further model training.

[0065] During the model training phase, a weighted loss function that integrates category weights and loss prediction functions is adopted. The design of this loss function fully considers the weight differences between different categories and combines the output of the loss prediction module to form a comprehensive loss function expression. The definition is as follows:

[0066]

[0067] The first part of the formula is the weight loss of different importance between categories, and the second part of the loss is the prediction loss; N represents the number of samples in a small batch, C represents the total number of categories, and w c represents the weight of the cth category, y c represents the true category value, represents the predicted category value; α is the weight, Indicates l c and The prediction loss between c represents the target loss of the c-th category sample (the loss is obtained by calculating the difference between the true label and the predicted result), Represents the loss prediction value of the c-th category sample (in the training process of this embodiment, the feature vector used for prediction in the present invention is fitted with the distribution of the actual loss through an auxiliary network (such as a shallow neural network), and the loss value fitted by the auxiliary network is used as the loss prediction value).

[0068]

[0069]

[0070] in, represents the target loss of a sample x in the cth category The target loss with another sample y in category c The difference between represents the loss prediction value of a sample x in the cth category, represents the loss prediction value of a sample y in the cth category; δ represents a pre-set positive boundary value.

[0071] Table 1 Simulation results

[0072]

[0073] In order to verify the effectiveness of the present invention, the classification method of the present invention is compared with other image classification methods in the prior art. The accuracy (Accuracy) of the classification results, that is, the proportion of the number of samples predicted correctly by the model to the total number of samples, the precision (Precision), that is, the proportion of samples predicted by the model to be positive categories that are truly positive categories, the recall rate (Recall) and the F1 score are used to evaluate the classification results of lead flotation foam images by each classification model. From the simulation results given in Table 1, it can be seen that the accuracy, precision, recall rate and F1 score indicators of the present invention are significantly better than other existing image classification models in the prior art.

[0074] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A lead flotation foam image classification method based on deep autonomous learning, characterized in that: Construct an image classification network that integrates the EfficientNetB6 and Transformer network structures, and use the trained image classification network to perform image classification. The training process of the image classification network includes: Obtain image data with classification labels and unlabeled image data, and preprocess the images; Sample m samples from image data with classification labels to initialize the image classification network for training; Through autonomous learning, n data with the largest amount of information are selected from unlabeled image data for manual annotation; Update the manually annotated data with image data with classification labels, and use the updated image dataset to further train the initialized trained image classification network; The further trained image classification network is used to perform the lead flotation foam image classification task.

2. A lead flotation foam image classification method based on deep autonomous learning according to claim 1, characterized in that: The image classification network consists of cascaded convolutional layers, EfficientNetB6, Transformer network and linear layers, where EfficientNetB6 consists of seven cascaded convolutional modules, each of which consists of one or more convolutional units.

3. A lead flotation foam image classification method based on deep autonomous learning according to claim 1, characterized in that: The seven convolution modules that cascade to form EfficientNetB6 are composed of one convolution unit, two convolution units cascaded, two convolution units cascaded, three convolution units cascaded, three convolution units cascaded, four convolution units cascaded, and one convolution unit.

4. A lead flotation foam image classification method based on deep autonomous learning according to claim 2 or 3, characterized in that: The data processing process of the convolution unit includes: the input feature map is processed through the cascaded convolution layer, depth-separable convolution layer, self-attention module, convolution layer, and Dropout layer to obtain a feature map, and the feature map is added and fused with the input feature map through the jump connection layer as the output of the convolution module.

5. A lead flotation foam image classification method based on deep autonomous learning according to claim 4, characterized in that: The data processing process of the self-attention module includes: the input feature map is processed by a cascaded global average pooling layer and two fully connected layers, and the feature map obtained is multiplied and fused with the input feature map through a skip connection layer as the output of the self-attention module.

6. A lead flotation foam image classification method based on deep autonomous learning according to claim 2, characterized in that: The Transformer network structure consists of four cascaded sub-layers with the same structure. Each sub-layer includes a multi-head attention layer and a position-based feedforward network, and each sub-layer uses a residual connection.

7. A lead flotation foam image classification method based on deep autonomous learning according to claim 2, characterized in that: The data processing process of the linear layer includes: processing the feature map output by the Transformer network structure through the global average pooling layer, the fully connected layer, and the softmax classifier in sequence.

8. A lead flotation foam image classification method based on deep autonomous learning according to claim 2, characterized in that: The loss function of the image classification network is expressed as: Where N is the number of samples in a mini-batch, C is the total number of categories, and w c represents the weight of the cth category, y c represents the true category value of the c-th category sample, represents the category value predicted by the c-th category sample; α is the weight, l c represents the target loss of the c-th category sample, represents the loss prediction value of the c-th category sample, Indicates l c and The prediction loss between represents the target loss of a sample x in the cth category The target loss with another sample y in category c The difference between represents the loss prediction value of a sample x in the cth category, represents the loss prediction value of a sample y in the cth category; δ represents a pre-set positive boundary value.

9. A lead flotation foam image classification system based on deep autonomous learning, characterized in that: The method for classifying lead flotation foam images based on deep autonomous learning according to claim 1 comprises using a trained image classification network to perform a lead flotation foam image classification task, wherein the training process of the image classification network comprises: Obtain image data with classification labels and image data without labels, and preprocess the images; Sample m samples from image data with classification labels to initialize the image classification network for training; Use the initialized trained image classification network to classify and predict the unlabeled image data, and select n samples with the lowest classification prediction confidence for manual labeling; The manually annotated image data is added to the image data with classification labels, and used to train the image classification network again.

Images