An intelligent mineral sorting method based on convolutional neural network and self-attention mechanism

Abstract

The present invention discloses an intelligent mineral sorting method based on convolutional neural networks and self-attention mechanisms, which realizes intelligent mineral sorting with high precision, high efficiency, low volume and low computational complexity. This architecture combines the inductive bias of locality and translation invariance in convolution operations with the globality and long-distance dependence of the self-attention mechanism to establish a mineral image classification model with stronger recognition ability and better feature capture ability. Compared with a single network based on convolutional neural networks or self-attention mechanisms, the architecture proposed by the present invention has higher classification accuracy and lower computational complexity. Secondly, the architecture provided by the present invention is suitable for fine-grained classification tasks of multiple types and categories of mineral images, and has excellent discrimination ability for mineral images with small differences in apparent features.

Description

Technical Field

[0001] The present invention relates to an intelligent mineral sorting method based on convolutional neural networks and self-attention mechanism, named OreFormer, and belongs to the field of mineral sorting and computer vision technology. Background Art

[0002] Mineral resources are the backbone of modern industrial production, making significant contributions to economic development, infrastructure, national defense, and security. Rational, efficient, and sustainable mineral resource allocation is a top priority for the current development of the mining industry. With the accelerating pace of development, mineral reserves are showing a significant downward trend, and mineral resources are characterized by being poor, fine, mixed, and difficult to select. Furthermore, the dramatic increase in surrounding rock volume during mineral mining leads to high energy consumption for transportation, crushing, and grinding. Therefore, rapid mineral identification and sorting after mining and before washing are urgently needed to achieve pre-emptive waste removal, reduce costs, and increase efficiency.

[0003] In recent years, intelligent mineral sorting technology based on radiation sensors has attracted much attention. It can achieve high-efficiency, batch sorting of two products (ore gangue) or even multiple products. At the same time, with the rapid development of sensor equipment, the application prospects of radiation sorting equipment based on gamma rays, Roman spectroscopy, X-ray diffraction (XRD) analysis, and X-ray fluorescence (XRF) analysis in the field of intelligent mineral sorting are becoming increasingly broad. It has the advantages of high precision, simple operation, and user-friendliness, and is particularly effective in sorting large minerals. However, this type of intelligent mineral sorting equipment is still subject to problems such as high price, high equipment cost, high radiation hazards, and large equipment size. Therefore, it is not suitable for use as a general mineral sorting technology in practical operation scenarios.

[0004] The rapid development of imaging and computer vision technologies has provided a new solution to these problems. These technologies utilize optical instruments (such as industrial cameras, hyperspectral imagers, microscopic imaging systems, and scanning electron microscopes) to capture mineral images and employ image processing and analysis methods to analyze surface features such as color, luster, streaks, grooves, and texture, effectively replacing X-ray sensor-based mineral sorting equipment. Early vision-based mineral sorting equipment was primarily based on traditional machine learning algorithms, such as decision trees, naive Bayes, K-nearest neighbors, artificial neural networks, principal component analysis, and support vector machines. These algorithms offered advantages such as fast recognition speed, high classification accuracy, zero radiation, no pollution, low cost, and easy deployment. However, shortcomings such as high image pixel requirements, limited feature extraction, slow inference speed, and the inability to fully utilize large amounts of image data continue to limit the further development of this technology.

[0005] The emergence of deep learning technology has provided a new technological development direction for the cross-integration of computer vision and mineral processing. As a data-driven technology (i.e., greater data volumes or more robust data analysis yield higher accuracy), it can fully leverage large volumes of mineral image data collected in static or dynamic scenarios to extract representative, general, and distinctive features to accomplish mineral sorting tasks. Mineral sorting methods based on deep learning inherit the advantages of traditional machine learning methods, such as high precision, high efficiency, low cost, and low complexity. Furthermore, they effectively reduce the dependence on input image resolution and limitations in inference speed. These factors collectively promote the development potential of deep learning technology in the field of intelligent mineral sorting. Currently, mineral sorting researchers have extensively explored model selection and construction, multi-scale feature extraction and optimization, model deployment and lightweighting, and the application of deep learning technology in complex and challenging scenarios, laying the foundation for subsequent technological innovation. However, existing research has largely relied on a single convolutional neural network architecture. While these networks introduce local and translational invariance biases into the feature extraction process through convolution operations, they still fail to fully utilize the global information of mineral images, hindering further improvements in sorting accuracy. Furthermore, this lack of global information creates a bottleneck in the convolutional neural network architecture for representing mineral image features. These issues need to be addressed at the architectural design level to effectively improve the industrial application performance of intelligent mineral sorting technology. Summary of the Invention

[0006] In view of this, the present invention provides an intelligent mineral sorting method based on convolutional neural networks and self-attention mechanism, named OreFormer. After model training, this architecture can be directly deployed at the mineral sorting site to achieve high-precision and rapid sorting of minerals.

[0007] An intelligent mineral sorting method based on convolutional neural network and self-attention mechanism includes the following steps:

[0008] Step 1: Obtain the mineral image data to be processed and divide it into training set and validation set in a ratio of 8:2;

[0009] Step 2: annotating the mineral image data obtained in step 1 by manual annotation to obtain a label for the mineral image data;

[0010] Step 3: Build a mineral image classification model based on convolutional neural network and self-attention mechanism for the mineral image data obtained in step 1 and the mineral image data labels obtained in step 2;

[0011] Step 4: Train the mineral image classification model based on convolutional neural network and self-attention mechanism built in step 3;

[0012] Step 5: terminating the training process of the mineral image classification model to obtain the trained mineral image classification model based on convolutional neural network and self-attention mechanism;

[0013] Step 6: The mineral image classification model based on convolutional neural network and self-attention mechanism obtained in step 5 is used to predict the newly collected mineral image data to complete the mineral sorting task.

[0014] In step 1, when obtaining mineral image data, a mineral image data acquisition platform is built; the mineral image data is taken by an industrial camera directly above the conveyor belt, and the training set and the validation set are divided into a ratio of 8:2;

[0015] In step 3, the mineral image classification model based on convolutional neural network and self-attention mechanism includes a convolution neck, a downsampling layer, a self-attention encoder, a mean pooling layer, a projection layer and a classifier;

[0016] Among them, the convolution neck consists of three convolutional layers, which is responsible for introducing the inductive bias of the convolution operation;

[0017] The downsampling layer consists of one convolutional layer, which is responsible for reducing the dimension of the feature image and generating feature data for the self-attention encoder feature operation;

[0018] The self-attention encoder consists of 1 convolutional layer, 2 layer normalization layers, 1 convolutional multi-head self-attention module (including 1 deep convolutional layer, 3 linear projection layers and 1 multi-head self-attention layer), 1 inverted bottleneck feedforward neural network (including 3 convolutional layers) and 3 residual connections, which is responsible for calculating the attention weights of each region in the input image;

[0019] The projection layer consists of a convolutional layer, which is responsible for integrating the mineral image feature information extracted by the previous network and transmitting the output to the classifier;

[0020] The classifier consists of one fully connected layer, which is responsible for establishing the relationship between feature information and categories, and thus completing the mineral image classification task.

[0021] The convolution neck contains one convolution layer with a stride of 2 and a convolution kernel size of 3×3, and two convolution layers with a stride of 1 and a convolution kernel size of 3×3;

[0022] The downsampling layer consists of a convolutional layer with a stride of 2 and a convolution kernel size of 2×2;

[0023] The self-attention encoder consists of a deep convolutional layer with a stride of 1 and a convolution kernel size of 3×3, as well as residual connections before and after the deep convolutional layer; a layer normalization layer and a convolutional multi-head self-attention module, as well as residual connections before and after the layer normalization layer; a layer normalization layer and an inverted bottleneck feedforward neural network, as well as residual connections before and after the layer normalization layer.

[0024] Each convolution kernel in the depth convolution layer is responsible for performing convolution operations within one channel of the input image, and the number of convolution kernels is equal to the number of channels in the input image.

[0025] The layer normalization layer normalizes the input feature values ​​according to the hierarchical structure;

[0026] The convolutional multi-head self-attention structure consists of a deep convolutional layer with a stride of k and a convolution kernel size of k×k, a linear projection layer, and a multi-head self-attention layer, which is responsible for calculating the attention weights of each region in the input image;

[0027] The inverted bottleneck feedforward neural network consists of a convolutional layer with a stride of 1 and a convolution kernel size of 1×1, a depthwise convolutional layer with a stride of 1 and a convolution kernel size of 3×3, and a convolutional layer with a stride of 1 and a convolution kernel size of 1×1, which is responsible for interacting and fusing the input feature values.

[0028] The mean pooling layer performs average pooling operations on the input feature values, which is responsible for gathering important mineral image feature information;

[0029] The projection layer consists of a convolutional layer with a stride of 1 and a convolution kernel size of 1×1. It is responsible for integrating the mineral image feature information extracted by the previous network and transmitting the output to the classifier.

[0030] The classifier consists of a fully connected layer, which is responsible for establishing the relationship between feature information and category information, and thus completing the mineral image classification task.

[0031] The convolutional multi-head self-attention module calculates the query matrix using linear projection, calculates the key matrix and value matrix using a deep convolution layer with a step size of k and a convolution kernel size of k×k and linear projection, and calculates the attention weight according to the following formula:

[0032]

[0033] Where Q is the Query matrix after linear projection calculation, K′ and V′ are the depth convolution layer with a step size of k and a convolution kernel size of k×k, and the Key matrix and Value matrix calculated by linear projection, respectively. kis the dimension of the Key matrix, Softmax() is the activation function, and B is an additional relative position matrix that is initially a random value and can be trained and adjusted along with the network.

[0034] In step 4, the training process includes:

[0035] Determine the loss of the mineral image classification model based on convolutional neural network and self-attention mechanism:

[0036]

[0037] Where N is the total number of samples, M is the number of categories in the classification task, i = 1, 2, 3..., N, c = 1, 2, ..., M, y ic is the indicator function, which is 1 when i belongs to category c and 0 otherwise. ic is the probability that the i-th sample belongs to category c;

[0038] In step 5, the loss value of the mineral image classification model based on the convolutional neural network and the self-attention mechanism is determined according to the determined classification loss Loss, and the learning rate can be obtained during the training process. When the training is terminated, the termination time is determined according to the loss value and the threshold of the learning rate to obtain the trained mineral image classification model based on the convolutional neural network and the self-attention mechanism.

[0039] The present invention has the following beneficial effects:

[0040] (1) The architecture provided by the present invention combines the locality and translation invariance of convolution operations with the globality and long-range dependence of the self-attention mechanism to establish a mineral image classification model with stronger recognition and better feature capture capabilities. Compared with a single convolutional neural network or a network based on the self-attention mechanism, the architecture proposed by the present invention has higher classification accuracy and lower computational complexity. Therefore, the present invention is more convenient for implementing the task of intelligent mineral sorting.

[0041] (2) The architecture provided by the present invention is suitable for fine-grained classification tasks of multiple types and categories of mineral images, and has excellent discrimination capabilities for mineral images with small differences in apparent features. At the same time, by dynamically increasing the depth and width of the network, the mineral image feature representation capability of the proposed architecture can be further improved, which effectively solves the problem of the existing technology's weak recognition and classification performance in mineral images with small differences between groups.

[0042] (3) The framework provided by this invention reduces the number of model parameters and computational complexity by using deep convolution and an inverted bottleneck structure, achieving an excellent balance between recognition performance and model complexity. Therefore, this invention has greater industrial practical significance and meets the deployment requirements of intelligent mineral processing equipment.

[0043] (4) The framework provided by the present invention has better feature extraction capabilities, can quickly capture and aggregate important information in mineral images that contributes more to category determination, and the trained model is more uniform in the distribution of classification weights, and the classification weights are more moderate.

[0044] (5) The present invention discloses an intelligent mineral sorting method based on convolutional neural networks and self-attention mechanisms, which realizes intelligent mineral sorting with high precision, high efficiency, low volume and low computational complexity. This architecture combines the inductive bias of locality and translation invariance in convolution operations with the globality and long-distance dependence of the self-attention mechanism to establish a mineral image classification model with stronger recognition ability and better feature capture ability. Compared with a single network based on convolutional neural networks or self-attention mechanisms, the architecture proposed by the present invention has higher classification accuracy and lower computational complexity. Secondly, the architecture provided by the present invention is suitable for fine-grained classification tasks of multiple types and categories of mineral images, and has excellent discrimination ability for mineral images with small differences in apparent features. At the same time, the framework provided by the present invention reduces the number of model parameters and computational complexity by using deep convolution and inverted bottleneck structures, and obtains an excellent balance between recognition performance and model complexity. In addition, the framework provided by the present invention has better feature extraction capabilities and can quickly capture and aggregate important information in mineral images that contributes more to category determination. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A flow chart of the architecture provided by the present invention;

[0046] Figure 2 Schematic diagram of the mineral image data acquisition platform of the present invention;

[0047] Figure 3 Schematic diagram of the mineral image classification architecture based on convolutional neural network and self-attention mechanism of the present invention;

[0048] Figure 4 Schematic diagram of setting detailed parameters of each level of the present invention;

[0049] Figure 5 This is an example of a mineral image collected by the present invention;

[0050] Figure 6 This is an example diagram of the confusion matrix for the discrimination ability test of the present invention;

[0051] Figure 7 This is a visualization example diagram of feature extraction of the present invention. DETAILED DESCRIPTION

[0052] The present invention will be described in detail below with reference to the accompanying drawings.

[0053] like Figure 1 As shown, the present invention provides a mineral image classification architecture based on convolutional neural network and self-attention mechanism, which specifically includes the following steps:

[0054] Step 1: Build Figure 2 The mineral image data acquisition platform shown in the figure primarily consists of six components: a vibrating screening system, a dust removal system, a mineral conveyor system, an image acquisition system, a computer control system, and an electromagnetic injection system. During the acquisition process, the vibrating screening system vibrates and screens the raw mineral material, evenly distributing material smaller than the screening aperture onto the head of the conveyor. Simultaneously, the dust removal system uses high-frequency airflow to remove dust from the mineral surface, minimizing the impact of surface dust on subsequent image quality. The image acquisition system utilizes a 4K color 3CCD linear array industrial camera and accompanying image processing software, enabling continuous capture and storage of sample images at a constant speed. The linear light strip's illumination intensity can be adjusted according to actual needs. While transported via the mineral conveyor system, mineral samples are sequentially imaged by the image acquisition system, and the captured data is uploaded in real time to the computer control system for mineral classification determination. At the rear of the image acquisition platform, the electromagnetic injection system sorts the mineral material into the appropriate areas based on information returned by the control terminal.

[0055] Step 2: Use a manual labeling method to label the mineral image data obtained in step 1 to obtain a label for the mineral image data.

[0056] Step 3: Build Figure 3 The mineral image classification architecture based on convolutional neural network and self-attention mechanism shown in FIG, the mineral image classification architecture based on convolutional neural network and self-attention mechanism includes convolution neck, downsampling layer, self-attention encoder, mean pooling layer, projection layer and classifier. Figure 3As shown in the figure, in this architecture, the convolutional neck consists of one convolutional layer with a stride of 2 and a kernel size of 3×3, and two convolutional layers with a stride of 1 and a kernel size of 3×3. The downsampling layer consists of a convolutional layer with a stride of 2 and a kernel size of 2×2. The self-attention encoder consists of three parts: one depthwise convolutional layer with a stride of 1 and a kernel size of 3×3, and residual connections before and after the depthwise convolutional layer; one layer normalization layer and one convolutional multi-head self-attention module, and residual connections before and after the layer normalization layer; and one layer normalization layer and one inverted bottleneck feedforward neural network, and residual connections before and after the layer normalization layer. Each convolution kernel in the deep convolutional layer performs a convolution operation within a channel of the input image, and the number of convolution kernels is equal to the number of channels in the input image. The layer normalization layer normalizes the input feature values ​​according to the hierarchical structure. The convolutional multi-head self-attention structure includes a deep convolution layer with a stride of k and a kernel size of k×k, a linear projection layer, and a multi-head self-attention layer, which is responsible for calculating the attention weights for each region in the input image. The inverted bottleneck feedforward neural network includes one convolution layer with a stride of 1 and a kernel size of 1×1, one deep convolution layer with a stride of 1 and a kernel size of 3×3, and one convolution layer with a stride of 1 and a kernel size of 1×1, which is responsible for interacting and fusing the input feature values. The mean pooling layer performs an average pooling operation on the input feature values, which is responsible for aggregating important mineral image feature information. The projection layer consists of a convolutional layer with a stride of 1 and a kernel size of 1×1. It is responsible for integrating the mineral image feature information extracted by the previous network and transmitting the output to the classifier. The classifier consists of a fully connected layer, which is responsible for establishing the relationship between feature information and category information, thereby completing the mineral image classification task.

[0057] In step 3, the convolutional multi-head self-attention module calculates the Query matrix using linear projection, calculates the Key matrix and the Value matrix using a deep convolution layer with a step size of k and a convolution kernel size of k×k and linear projection, and calculates the attention weight according to the following formula:

[0058]

[0059] Where Q is the Query matrix after linear projection calculation, K′ and V′ are the depth convolution layer with a step size of k and a convolution kernel size of k×k, and the Key matrix and Value matrix calculated by linear projection, respectively. k is the dimension of the Key matrix, Softmax() is the activation function, and B is an additional relative position matrix that is initially a random value and can be trained and adjusted along with the network.

[0060] The detailed parameters of each level of the mineral image classification architecture based on convolutional neural network and self-attention mechanism are as follows Figure 4As shown in this parameter diagram, H i , (i=1,2,3,4) represents the number of computing heads in the convolutional multi-head self-attention module, k i , (i=1,2,3,4) represents the size and stride of the deep convolution kernel in the convolutional multi-head self-attention module, R i ,(i=1,2,3,4) represents the expansion multiple of the channel dimension in the inverted bottleneck feedforward neural network, the size of the convolution kernel in the 3×3, 2×2, and 1×1 convolution layers, 38, 76, 152, 304, 608, 1280, N represents the channel dimension of the current layer, and ×2, ×4, and ×20 represent the number of repetitions of the current module. For a given input mineral image of size 224×224, the dimension of the feature map after the convolution neck is 112×112×38, the dimension of the feature map after downsampling 1 and self-attention encoder 1 (repeated 4 times) is 56×56×76, the dimension of the feature map after downsampling 2 and self-attention encoder 2 (repeated 4 times) is 28×28×152, the dimension of the feature map after downsampling 3 and self-attention encoder 3 (repeated 20 times) is 14×14×304, the dimension of the feature map after downsampling 4 and self-attention encoder 4 (repeated 4 times) is 7×7×608, and the dimension of the feature map after the projection layer is 1×1×1280. The classifier will use 1 fully connected layer to establish the connection between the 1280-dimensional features and N categories, thereby completing the final classification task.

[0061] Step 4: Based on the mineral image classification model's loss function (Equation 2), the mineral image data and their labels are used to train the convolutional neural network (CNN)-based mineral image classification model. During training, the loss value is fed back to each layer of the network in real time to optimize the network weights. Finally, model training is terminated based on whether the training reaches a preset number of rounds, the loss value, or the learning rate meets a preset threshold, resulting in a trained CNN-based mineral image classification model.

[0062]

[0063] Step 5: Use the trained mineral image classification model based on convolutional neural network and self-attention mechanism to predict the newly collected mineral image data and complete the mineral sorting task.

[0064] Example

[0065] This example uses anthracite from the coal fields of Anhui and Ningxia in China as the mineral sample. The samples are manually screened to select the final experimental samples of 13-50mm and are classified according to the density level of <1.4g / cm 3 , 1.4-1.6g / cm 3 , 1.6g-1.8g / cm3 and >1.8 g / cm 3 The mass percentage and ash content of the four types of mineral samples can be seen in Table 1, and the results of microscopic component analysis can be seen in Table 2.

[0066] Table 1 Mass percentage and ash content of mineral samples in each density group (unit: percentage)

[0067]

[0068] Table 2 Microscopic component analysis of mineral samples of each density group (unit: percentage)

[0069]

[0070] The mineral image data acquisition platform described in step 1 was constructed. The embodiment selected a 4K color 3CCD linear array camera (JAI 3CCD Datasheet_LT-400CL) as an industrial camera, which enables continuous capture and storage of sample images under constant speed operation. A built-in sensor is embedded in the conveyor shaft, which can convert the conveyor belt speed into a digital signal for adaptive adjustment of the industrial camera. The embodiment selected a linear light strip with a length of 500 mm, uniform illumination, and a color temperature of 5800-7000K. The surface brightness of the belt during the shooting process was approximately 250klux. Since the brightness of the external ambient light was only 130lx, the influence of ambient light could be completely shielded. In the experiment, the conveyor belt speed was 1m / s, and the coal image size collected was 4096×3000pixel with an accuracy of 0.098mm / pixel. The mineral image data acquisition platform was used to cyclically collect multiple batches of mineral image data. A total of 20,000 mineral images (5,000 for each category) were collected on the mineral image data acquisition platform. The training set and validation set were divided into two sets according to the ratio of 8:2, and 2,000 images (500 for each category) were collected as the test set. The data volume details of each data set can be seen in Table 3.

[0071] Table 3 Mineral image dataset of the embodiment (unit: sheet)

[0072]

[0073] Use the annotation method described in step 2 to annotate the mineral image and obtain the label of the mineral image data. The example of mineral images of each category is shown in the figure below. Figure 5 shown.

[0074] Build a structure like Figure 3 The mineral image classification architecture based on convolutional neural networks and self-attention mechanism is shown in Figure 1, named OreFormer. The architecture includes convolution neck, downsampling layer, self-attention encoder, mean pooling layer, projection layer and classifier. The specific setting parameters of each layer in the architecture are as follows: Figure 4 As shown, the output dimension N of the classifier is 4 in the current experiment.

[0075] A loss function is determined for the mineral image classification model based on a convolutional neural network and a self-attention mechanism. The model is then trained using the collected mineral image data and its labels based on the determined loss function. During training, the model automatically calculates and provides feedback on the loss and automatically adjusts weights. Finally, model training is terminated based on whether a preset number of training rounds has been reached or whether the loss value or learning rate meets a preset threshold, resulting in a trained mineral image classification model based on a convolutional neural network and a self-attention mechanism.

[0076] The embodiment built four additional common mineral image classification models based on a single convolutional neural network or a single self-attention mechanism for detection performance comparison tests, including ResNet 152, EfficientNet-B6, ViT and Swin Transformer-B (Swin-B). The embodiment uses Accuracy (Acc), Precision and Recall to evaluate the model classification accuracy, and uses #Params and #FLOPs to evaluate the model volume and computational complexity. After training in the same environment, the performance comparison of the mineral image classification model based on convolutional neural network and self-attention mechanism proposed in the present invention and the four common image classification models in the anthracite image classification task can be seen in Table 4. The performance comparison results of the embodiment show that the proposed mineral image classification architecture based on convolutional neural network and self-attention mechanism (OreFormer) has higher Acc, Precision and Recall than other common models, reaching 93.3%, 93.5% and 93.3% respectively. Comparisons of model size and computational complexity in the implementation examples show that the proposed architecture has a lower computational complexity of only 9.3 GB compared to other comparison models. Its model size is only slightly higher than the EfficientNet-B6 model, which has the fewest parameters, by 2.9 MB. This results in a 0.7% improvement in classification accuracy. These experimental results demonstrate the superior performance of the proposed architecture in multi-category fine-grained mineral image classification tasks.

[0077] Table 4 Performance comparison of the proposed lightweight object detection model for mineral images and 10 common object detection models in the anthracite detection task (unit: mAP (percentage), #Params (million), Resolution (pixels), #FLOPs (billion floating-point operations))

[0078]

[0079] In order to test the discrimination ability of the proposed mineral image classification model based on convolutional neural network and self-attention mechanism in multi-category mineral image classification tasks, the embodiment predicts the anthracite test set images and draws the confusion matrix, as shown in the following example: Figure 6 The experimental results show that the proposed architecture can accurately determine the category of each image in the anthracite test set, and the 3 and >1.8 g / cm 3 The discrimination effect of the two types of anthracite is relatively better than 1.4-1.6g / cm 3 and 1.6-1.8g / cm 3 Therefore, the test results effectively demonstrate the application performance of the proposed architecture in the multi-category mineral image classification task.

[0080] In addition, in order to clearly demonstrate the representation ability and feature extraction effect of the proposed architecture on mineral image features, the classification weights of the model are displayed in the form of heat map gradient class activation mapping, such as Figure 7 The visualization results show that the proposed architecture can quickly capture and aggregate important information in mineral images that contributes more to category determination, and the trained model has a relatively uniform distribution of classification weights, with moderate classification weights.

[0081] In summary, the present invention provides a mineral sorting architecture based on convolutional neural networks and self-attention mechanisms, which can be applied to most mineral sorting tasks. The main process includes the construction of a mineral image data acquisition platform, mineral image acquisition and annotation, the construction and training of a mineral image classification model based on convolutional neural networks and self-attention mechanisms, and the predictive application of the trained mineral image classification model. The classification performance comparison experiment shows that the proposed architecture achieves higher classification accuracy and lower computational complexity by combining the inductive bias of locality and translation invariance in convolution operations with the globality and long-distance dependence of the self-attention mechanism. At the same time, the confusion matrix results show that the proposed architecture also has good discrimination ability for mineral images with small differences between groups, and can accurately discriminate different types of ores. In addition, the classification weight visualization results show that the proposed architecture has a good effect in mineral image feature extraction and characterization, which proves the effectiveness of the proposed architecture.

[0082] It should be understood that the above are only preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An intelligent mineral sorting method based on convolutional neural network and self-attention mechanism, characterized in that The following steps are involved: Step 1: Obtain the mineral image data to be processed, and divide the obtained mineral image data into a training set and a validation set; Step 2: Annotate the mineral image data obtained in step 1 to obtain the label of the mineral image data; Step 3: Based on the mineral image data obtained in step 1 and the mineral image data labels obtained in step 2, a mineral image classification model based on convolutional neural network and self-attention mechanism is constructed; Step 4: Use the training set and validation set divided in step 1 to train the mineral image classification model based on convolutional neural network and self-attention mechanism constructed in step 3; Step 5: terminating the training process of the mineral image classification model to obtain the trained mineral image classification model based on convolutional neural network and self-attention mechanism; Step 6: Use the mineral image classification model based on convolutional neural network and self-attention mechanism obtained in step 5 to predict the newly collected mineral image data and complete the mineral sorting; In step 1, when obtaining mineral image data, a mineral image data acquisition platform is built; the mineral image data is taken by an industrial camera directly above the conveyor belt, and the training set and the validation set are divided into a ratio of 8:2; In step 3, the mineral image classification model based on convolutional neural network and self-attention mechanism includes a convolution neck, a downsampling layer, a self-attention encoder, a mean pooling layer, a projection layer and a classifier; The convolution neck consists of three convolutional layers, which are responsible for introducing the inductive bias of the convolution operation. One of the convolutional layers has a stride of 2 and a convolution kernel size of 3×3, while the other two have a stride of 1 and a convolution kernel size of 3×3. The downsampling layer consists of a convolutional layer, which is responsible for reducing the feature image dimension and generating feature data for the self-attention encoder feature operation, where the stride of the convolutional layer is 2 and the convolution kernel size is 2×2; The self-attention encoder consists of 1 convolutional layer, 2 layer normalization layers, 1 convolutional multi-head self-attention module, 1 inverted bottleneck feedforward neural network and 3 residual connections, which is responsible for calculating the attention weights of each region in the input image; The mean pooling layer performs average pooling operation on the input feature values, which is responsible for gathering important mineral image feature information; The projection layer consists of a convolutional layer, which is responsible for integrating the mineral image feature information extracted by the previous network and transmitting the output to the classifier. The step size of the convolutional layer is 1 and the convolution kernel size is 1×1. The classifier consists of one fully connected layer, which is responsible for establishing the relationship between feature information and category information.

2. The intelligent mineral separation method based on convolutional neural network and self-attention mechanism according to claim 1, characterized in that: The convolutional multi-head self-attention module in the self-attention encoder contains one deep convolutional layer, three linear projection layers, and one multi-head self-attention layer; the convolutional multi-head self-attention module is responsible for calculating the attention weights of each region in the input image; The three residual connections in the self-attention encoder are the residual connections before and after the deep convolutional layer, the residual connection before the layer normalization layer to after the convolutional multi-head self-attention layer, and the residual connection before the layer normalization layer to after the inverted bottleneck feedforward neural network. The layer normalization layer in the self-attention encoder is used to normalize the input feature values ​​according to the hierarchical structure.

3. The intelligent mineral separation method based on convolutional neural network and self-attention mechanism according to claim 2 is characterized in that: The step size of the depth convolution layer is 1 and the convolution kernel size is 3×3. Each convolution kernel in the depth convolution layer is responsible for performing convolution operation in one channel of the input image and the number of convolution kernels is equal to the number of channels of the input image.

4. The intelligent mineral separation method based on convolutional neural network and self-attention mechanism according to claim 3 is characterized by: The inverted bottleneck feedforward neural network contains three convolutional layers, one of which is a deep convolutional layer with a stride of 1 and a convolution kernel size of 3×3, and the other two convolutional layers have a stride of 1 and a convolution kernel size of 1×1. The inverted bottleneck feedforward neural network is responsible for interacting and fusing the input feature values.

5. The intelligent mineral separation method based on convolutional neural network and self-attention mechanism according to claim 4 is characterized in that: The convolutional multi-head self-attention module calculates the Query matrix using linear projection, calculates the Key matrix and the Value matrix using a deep convolution layer with a step size of k and a convolution kernel size of k×k and linear projection, and calculates the attention weight according to the following formula: Where Q is the Query matrix after linear projection calculation, K′ and V′ are the depth convolution layer with a step size of k and a convolution kernel size of k×k, and the Key matrix and Value matrix calculated by linear projection, respectively. k is the dimension of the Key matrix, Softmax() is the activation function, and B is an additional relative position matrix that is initially a random value and can be trained and adjusted along with the network.

6. The intelligent mineral separation method based on convolutional neural network and self-attention mechanism according to claim 5, characterized in that: In step 4, the training process includes: Determine the loss of the mineral image classification model based on convolutional neural network and self-attention mechanism: Where N is the total number of samples, M is the number of categories in the classification task, i = 1, 2, 3..., N, c = 1, 2,..., M, y ic is the indicator function, which is 1 when i belongs to category c and 0 otherwise. ic is the probability that the i-th sample belongs to category c.

7. The intelligent mineral separation method based on convolutional neural network and self-attention mechanism according to claim 6, characterized in that: In step 5, the loss value of the mineral image classification model based on the convolutional neural network and the self-attention mechanism is determined according to the determined classification loss Loss, and the learning rate can be obtained during the training process. When the training is terminated, the termination time is determined according to the loss value and the threshold of the learning rate to obtain the trained mineral image classification model based on the convolutional neural network and the self-attention mechanism.

Images