Copper ore preselection method based on generative adversarial network and low-rank linear pooling

Abstract

The invention discloses a copper ore preselection method based on a generative adversarial network and low-rank linear pooling. The copper ore preselection method comprises the following steps that a high-energy X-ray transmission image and a low-energy X-ray transmission image of copper ore are obtained; fusing the high-energy X-ray transmission image and the low-energy X-ray transmission image by adopting an improved generative adversarial network to obtain a dual-energy X-ray fusion image; constructing a copper ore pre-selection model; inputting the high-energy, low-energy and dual-energy X-ray fusion images of the copper ore into the model for feature extraction, fusing each extracted feature vector by adopting low-rank trilinear pooling, and then classifying to obtain a pre-selection result of the copper ore; according to the copper ore pre-selection model designed by the invention, through improving low-rank bilinear pooling and using a low-rank trilinear pooling feature fusion strategy, combined representation solution of three image features is realized, multi-modal data input requirements can be met, and the classification performance of copper ore target ore and waste rock is further improved.

Description

Technical Field

[0001] The present invention relates to the technical field of copper ore preselection, and in particular to a copper ore preselection method based on generative adversarial networks and low-rank linear pooling. Background Art

[0002] Copper metal has the characteristics of electrical conductivity, thermal conductivity, strong ductility, and easy alloying. It has a wide range of use value in modern industry and is one of my country's important strategic resources. However, my country's copper ore resources are mostly poor ores, and there are fewer rich ores. During the mining process, a large amount of surrounding rock and waste rock will be mixed in, which increases the difficulty and cost of subsequent flotation and refining processes. For this reason, it is urgent to study ore sorting methods that can improve the efficiency of copper ore waste sorting. In industry, ore pre-selection is often added between ore mining and post-processing. After pre-sorting by ore sorting methods, the grade of selected ore for subsequent processing can be improved, the amount of selected waste rock can be reduced, and ore pre-enrichment can be achieved, which effectively improves the utilization rate of ore resources and reduces production costs. Common ore sorting methods have problems such as narrow application range, destruction of ore morphology, and large associated pollution. In recent years, intelligent photoelectric beneficiation methods based on artificial intelligence have been widely used in ore pre-selection. Intelligent photoelectric beneficiation methods are based on image color sorting technology and X-ray transmission technology to non-destructively collect ore information, and apply artificial intelligence algorithms such as deep learning models to quickly and accurately classify target ores and waste rocks. Among them, the performance of the ore classification algorithm directly determines the ore classification efficiency. Therefore, the study of copper ore pre-selection models based on deep learning has become a must for the efficient utilization of copper ore resources.

[0003] In recent years, image recognition methods based on deep learning have shown strong performance in classification tasks. Dual-energy X-ray transmission technology is based on the different attenuation effects of X-ray transmission of different materials, and identifies the material category by analyzing the intensity of the transmitted rays. This technology does not destroy the material morphology, is not affected by the appearance size and magnetic differences of the material, and is widely used in the field of photoelectric mineral processing. Obtaining high- and low-energy X-ray transmission images of copper ore through dual-energy X-ray imaging equipment, and using deep learning models to process ore images to achieve the discrimination of target ore and waste rock is a new solution to improve ore sorting efficiency. The ore sorting model based on deep learning uses the characteristics of deep neural networks for automatic feature learning, and has achieved good recognition results on copper ore high- and low-energy X-ray transmission image data sets. However, deep neural network image feature processing is usually regarded as a black box model and lacks interpretability. At present, the related work of applying deep learning models to ore sorting is relatively simple in the processing method of simultaneously learning high- and low-energy image data features, and lacks in-depth research. Therefore, it cannot be guaranteed that the model can effectively learn and utilize the features of high- and low-energy image data. Summary of the invention

[0004] In view of the shortcomings of the prior art, the present invention provides a copper ore preselection method based on generative adversarial networks and low-rank linear pooling, which aims to solve the problems in the background technology.

[0005] To achieve the above object, the present invention provides the following technical solution: a copper ore preselection method based on generative adversarial network and low-rank linear pooling, comprising the following steps: Step S1: obtaining a high-energy X-ray transmission image and a low-energy X-ray transmission image of the copper ore based on a dual-energy X-ray imaging device; Step S2: using an improved generative adversarial network to perform modal fusion on the high-energy X-ray transmission image and the low-energy X-ray transmission image of the copper ore to obtain a dual-energy X-ray fusion image of the copper ore; The improved generative adversarial network adds a discriminator on the basis of the original generative adversarial network; Step S3: constructing a copper ore pre-selection model, which includes a feature extraction coding layer, a feature fusion coding layer and a classification decoding layer; Step S4: inputting the high-energy X-ray transmission image, the low-energy X-ray transmission image and the dual-energy X-ray fusion image of the copper ore into a feature extraction coding layer to obtain high-energy, low-energy and dual-energy feature vectors; Step S5: inputting the high-energy, low-energy and dual-energy feature vectors into a feature fusion coding layer, and the feature fusion coding layer fuses the high-energy, low-energy and dual-energy feature vectors using low-rank trilinear pooling to obtain a fused feature vector; Step S6: Input the fused feature vector into the classification decoding layer for classification to obtain the pre-selection result of the copper ore.

[0006] Furthermore, the specific process of step S5 is as follows: Assume that the high energy, low energy and dual energy eigenvectors are , and , solved by low-rank trilinear pooling , and The joint feature representation of is: (1); In the formula, is the first elements; is the weight parameter for trilinear pooling, controlling , , The interaction of corresponding elements in; for The elements; for The elements; for The elements; For the Bias terms; For the A weight tensor; , , They are , , Dimensions; , , , , is the set of real numbers; The weight tensor Decompose into the product of three 2D matrices: , Indicates that From the dimension Reduce to dimension Matrix of express The transpose of Indicates that From the dimension Reduce to dimension Matrix of express The transpose of Indicates that From the dimension Reduce to dimension Matrix of The decomposed weight tensor Substituting into formula (1) to simplify formula (1): (2); In the formula, express The transpose of express The transpose of express The transpose of represents the Hadamard product; express The transpose of represents a vector of all 1s; Reuse Replace (2) ,definition , and , obtain the joint feature representation, that is, the fused feature vector : (3); In the formula, , and Respectively , and The transpose of , and Respectively , , The dimension reduction matrix of Represents the bias vector, with dimension ; express The transpose of Represents the weight matrix used for the linear transformation.

[0007] Further, the improved generative adversarial network consists of a discriminator , Discriminator and generator G; first, the high-energy X-ray transmission image and the low-energy X-ray transmission image of the copper ore are superimposed together in channel dimensions as the input of the generator G. After processing, the generator outputs a fused image. and the discriminator The high-energy X-ray transmission image and the low-energy X-ray transmission image of the copper ore are received respectively, and the high-energy X-ray transmission image and the low-energy X-ray transmission image of the copper ore are discriminated with the fusion image output by the generator. and the discriminator The discrimination result is input into the generator G, which adjusts the fused image and finally outputs the dual-energy X-ray fusion image of the copper ore.

[0008] Furthermore, the generator G is composed of a five-layer convolutional neural network, and the first layer, second layer, third layer, fourth layer and fifth layer of the generator G are all convolutional layers; the first layer, second layer, third layer and fourth layer of the convolutional layers of the generator G are respectively connected to a batch normalization layer and a Leaky ReLU activation layer, and the fifth layer of the convolutional layer is connected to a Tanh activation layer; wherein, the convolutional layers of the first and second layers of the generator G use a 5*5 convolutional kernel, the convolutional layers of the third and fourth layers use a 3*3 convolutional kernel, and the convolutional layer of the fifth layer uses a 1*1 convolutional kernel; the convolutional layer step size of each layer in the generator G is set to 1, and no padding operation is performed in the convolution.

[0009] Furthermore, the discriminator and the discriminator The structure is the same, also composed of a five-layer convolutional neural network, the discriminator and the discriminator The first, second, third and fourth layers are convolutional layers, and the fifth layer is a linear layer; the discriminator and the discriminator The first, second, third, and fourth convolutional layers are connected in sequence with a batch normalization layer and a Leaky ReLU activation layer; the discriminator and the discriminator The first, second, third and fourth convolutional layers use 3*3 convolution kernels and the stride is set to 2.

[0010] Furthermore, the feature extraction coding layer adopts three parallel feature extraction encoders DenseNet with the same structure to extract features from the high-energy X-ray transmission image, low-energy X-ray transmission image and dual-energy X-ray fusion image of the copper ore respectively to obtain high-energy, low-energy and dual-energy feature vectors.

[0011] Further, improve the generator in the generative adversarial network The loss function It is expressed as: (4); In the formula, It means fighting against loss; Indicates content loss; represents a hyperparameter used to adjust the content loss in The weight in It is expressed as: (5); In the formula, Indicates expected value; Represents dual-energy X-ray fusion image.

[0012] Further, improve the discriminator in the generative adversarial network The loss function and the discriminator The loss function It is expressed as: (6); (7); In the formula, Represents high energy X-ray transmission image; Represents a low-energy X-ray transmission image.

[0013] Compared with the existing technology, the present invention has the following beneficial effects:

[0014] (1) The present invention improves the collaborative design of generative adversarial networks and low-rank trilinear pooling, adopts multimodal adaptive adversarial training to balance multi-source information in pixel-level fusion, and applies low-rank trilinear pooling to efficiently establish high-order interactions of the three image features of high-energy, low-energy, and fused images through tensor decomposition, thereby enhancing the joint characterization capability. This combined strategy further improves the classification accuracy of the model compared to a single method and significantly enhances the model sorting performance.

[0015] (2) The copper ore pre-selection model designed by the present invention improves low-rank bilinear pooling and adopts a low-rank trilinear pooling feature fusion strategy to realize the joint representation solution of the three image features of high-energy, low-energy X-ray transmission images and dual-energy X-ray fusion images extracted by the encoder, which can adapt to the multi-modal data input requirements and further improve the classification performance of copper ore target ore and waste rock; the improved generative adversarial network designed by the present invention adapts to the input of high-energy and low-energy transmission images of copper ore by adopting two discriminators, so that the generator simultaneously learns the probability distribution of the two image data, realizes image fusion at the pixel level, and obtains a fused image with stronger representation ability. The input data of the classification model is optimized in the early fusion stage, and the performance of the copper ore classification model is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 The present invention is a flow chart of the method.

[0017] Figure 2 This is a schematic diagram of the improved generative adversarial network structure of the present invention.

[0018] Figure 3 It is a schematic diagram of the structure of the copper ore preselection model of the present invention. DETAILED DESCRIPTION

[0019] like Figure 1 As shown, the present invention provides a technical solution: a copper ore preselection method based on a generative adversarial network and low-rank linear pooling, comprising the following steps:

[0020] Step S1: obtaining a high-energy X-ray transmission image and a low-energy X-ray transmission image of the copper ore based on a dual-energy X-ray imaging device.

[0021] Step S2: using an improved generative adversarial network to perform modal fusion on the high-energy X-ray transmission image and the low-energy X-ray transmission image of the copper ore to obtain a dual-energy X-ray fusion image of the copper ore.

[0022] like Figure 2 As shown in the figure, the improved generative adversarial network adds a discriminator on the basis of the original generative adversarial network (GAN); the improved generative adversarial network consists of the discriminator , Discriminator and generator G; first, the high-energy X-ray transmission image and the low-energy X-ray transmission image of the copper ore are superimposed together in channel dimensions as the input of the generator G. After processing, the generator outputs a fused image. and the discriminator The high-energy X-ray transmission image and the low-energy X-ray transmission image of the copper ore are received respectively, and the high-energy X-ray transmission image and the low-energy X-ray transmission image of the copper ore are discriminated with the fusion image output by the generator. and the discriminator The discrimination result is input into the generator G, which adjusts the fused image and finally outputs the dual-energy X-ray fusion image of the copper ore.

[0023] Among them, the generator G is composed of a five-layer convolutional neural network, and the first, second, third, fourth and fifth layers of the generator G are all convolutional layers; the first, second, third and fourth convolutional layers of the generator G are connected with a batch normalization layer and a Leaky ReLU activation layer in sequence, and the fifth convolutional layer is connected with a Tanh activation layer; the first and second convolutional layers of the generator G use a 5*5 convolution kernel, the third and fourth convolutional layers use a 3*3 convolution kernel, and the fifth convolutional layer uses a 1*1 convolution kernel; the convolutional layer step size of each layer in the generator G is set to 1, and no padding operation is performed in the convolution.

[0024] Among them, the discriminator and the discriminator The structure is the same, also composed of a five-layer convolutional neural network, the discriminator and the discriminator The first, second, third and fourth layers are convolutional layers. At the same time, since the role of the discriminator is to determine whether the generated image is a real image, the fifth layer is a linear layer for classification; the discriminator and the discriminator The first, second, third, and fourth convolutional layers are connected in sequence with a batch normalization layer and a Leaky ReLU activation layer; the discriminator and the discriminator The first, second, third and fourth convolutional layers use 3*3 convolution kernels and the stride is set to 2.

[0025] Step S3: Construct a copper ore pre-selection model (LRTP-PTENet), such as Figure 3 As shown, the copper ore pre-selection model includes a feature extraction coding layer, a feature fusion coding layer and a classification decoding layer.

[0026] Step S4: input the high-energy X-ray transmission image, the low-energy X-ray transmission image and the dual-energy X-ray fusion image of the copper ore into a feature extraction coding layer to obtain high-energy, low-energy and dual-energy feature vectors.

[0027] Among them, the feature extraction coding layer adopts three parallel feature extraction encoders (DenseNet) with the same structure to extract features from the high-energy X-ray transmission image, low-energy X-ray transmission image and dual-energy X-ray fusion image of copper ore respectively, and obtain high-energy, low-energy and dual-energy feature vectors.

[0028] Step S5: input the high-energy, low-energy and dual-energy feature vectors into the feature fusion coding layer, and the feature fusion coding layer uses low-rank trilinear pooling to fuse the high-energy, low-energy and dual-energy feature vectors to obtain a fused feature vector.

[0029] The fully bilinear pooling method will produce high-dimensional features that are expanded twice, which is prone to the problem of too high dimensionality of fused features. This will consume a lot of computing resources and become a bottleneck limiting model performance.

[0030] Low-Rank Bilinear pooling (LRBP) decomposes a three-dimensional weight tensor required by full bilinear pooling into three two-dimensional weight matrices, thereby reducing the rank of the weight tensor and effectively reducing the amount of calculation parameters.

[0031] In view of the multimodal fusion of copper ore, the low-rank bilinear pooling method is extended, and the high-energy, low-energy and dual-energy feature vectors are set as , and , solved by low-rank trilinear pooling , and The joint feature representation of is: (1); In the formula, is the first elements; is the weight parameter for trilinear pooling, controlling , , The interaction of corresponding elements in; for The elements; for The elements; for The elements; For the A bias term used to adjust the joint feature representation; For the A weight tensor, used for fusion of feature vectors; , , They are , , Dimensions; , , , , is a set of real numbers; the dimension of the fused feature vector can be set to ,but , .

[0032] Since the weight tensor is directly calculated is extremely difficult and time-consuming, so the weight tensor Decompose into the product of three 2D matrices: , Indicates that From the dimension Reduce to dimension Matrix of express The transpose of Indicates that From the dimension Reduce to dimension Matrix of express The transpose of Indicates that From the dimension Reduce to dimension The matrix of .

[0033] The decomposed weight tensor Substituting into formula (1) to simplify formula (1): (2); In the formula, express The transpose of express The transpose of express The transpose of represents the Hadamard product (element-wise product); express The transpose of Represents a vector of all 1s.

[0034] Reuse Replace (2) ,definition , and , obtain the joint feature representation, that is, the fused feature vector : (3); In the formula, , and Respectively , and The transpose of , and Respectively , , The dimension reduction matrix of Represents the bias vector, with dimension ; express The transpose of Represents the weight matrix used for the linear transformation.

[0035] Step S6: Input the fused feature vector into the classification decoding layer for classification to obtain the pre-selection result of the copper ore.

[0036] Among them, the generator in the improved generative adversarial network The loss function It is expressed as: (4); In the formula, Represents adversarial loss, which is used to measure the difference between samples generated by the generator and real samples in the eyes of the discriminator; adversarial loss encourages the samples generated by the generator to deceive the discriminator, making it difficult to distinguish between generated samples and real samples; represents the content loss, which is used to measure the similarity between the generated samples and the target samples in terms of content. The content loss ensures that the generated samples are not only visually similar to the real samples, but also consistent in content; represents a hyperparameter used to adjust the content loss in , which can help strike a balance between generation quality and diversity.

[0037] It is expressed as: (5); In the formula, Indicates expected value; Represents dual-energy X-ray fusion image.

[0038] Improving the discriminator in generative adversarial networks The loss function and the discriminator The loss function It is expressed as: (6); (7); In the formula, Represents high energy X-ray transmission image; Represents a low-energy X-ray transmission image.

[0039] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A copper ore preselection method based on generative adversarial networks and low-rank linear pooling, characterized in that: The steps include: Step S1: obtaining a high-energy X-ray transmission image and a low-energy X-ray transmission image of the copper ore based on a dual-energy X-ray imaging device; Step S2: using an improved generative adversarial network to perform modal fusion on the high-energy X-ray transmission image and the low-energy X-ray transmission image of the copper ore to obtain a dual-energy X-ray fusion image of the copper ore; The improved generative adversarial network adds a discriminator on the basis of the original generative adversarial network; Step S3: constructing a copper ore pre-selection model, which includes a feature extraction coding layer, a feature fusion coding layer and a classification decoding layer; Step S4: inputting the high-energy X-ray transmission image, the low-energy X-ray transmission image and the dual-energy X-ray fusion image of the copper ore into a feature extraction coding layer to obtain high-energy, low-energy and dual-energy feature vectors; Step S5: inputting the high-energy, low-energy and dual-energy feature vectors into a feature fusion coding layer, and the feature fusion coding layer fuses the high-energy, low-energy and dual-energy feature vectors using low-rank trilinear pooling to obtain a fused feature vector; Step S6: Input the fused feature vector into the classification decoding layer for classification to obtain the pre-selection result of the copper ore.

2. The copper ore preselection method based on generative adversarial network and low-rank linear pooling according to claim 1 is characterized in that: The specific process of step S5 is: Assume that the high energy, low energy and dual energy eigenvectors are , and , solved by low-rank trilinear pooling , and The joint feature representation of is: （1）； In the formula, is the first elements; is the weight parameter for trilinear pooling, controlling , , The interaction of corresponding elements in; for The elements; for The elements; for The elements; For the Bias terms; For the A weight tensor; , , They are , , Dimensions; , , , , is the set of real numbers; The weight tensor Decompose into the product of three 2D matrices: , Indicates that From the dimension Reduce to dimension Matrix of express The transpose of Indicates that From the dimension Reduce to dimension Matrix of express The transpose of Indicates that From the dimension Reduce to dimension Matrix of The decomposed weight tensor Substituting into formula (1) to simplify formula (1): （2）； In the formula, express The transpose of express The transpose of express The transpose of represents the Hadamard product; express The transpose of represents a vector of all 1s; Reuse Replace (2) ,definition , and , obtain the joint feature representation, that is, the fused feature vector : （3）； In the formula, , and Respectively , and The transpose of , and Respectively , , The dimension reduction matrix of Represents the bias vector, with dimension ; express The transpose of Represents the weight matrix used for the linear transformation.

3. The copper ore preselection method based on generative adversarial network and low-rank linear pooling according to claim 2 is characterized in that: Improved Generative Adversarial Network by Discriminator , Discriminator and generator G; first, the high-energy X-ray transmission image and the low-energy X-ray transmission image of the copper ore are superimposed together in channel dimensions as the input of the generator G. After processing, the generator outputs a fused image. and the discriminator The high-energy X-ray transmission image and the low-energy X-ray transmission image of the copper ore are received respectively, and the high-energy X-ray transmission image and the low-energy X-ray transmission image of the copper ore are discriminated with the fusion image output by the generator. and the discriminator The discrimination result is input into the generator G, which adjusts the fused image and finally outputs the dual-energy X-ray fusion image of the copper ore.

4. The copper ore preselection method based on generative adversarial network and low-rank linear pooling according to claim 3 is characterized in that: The generator G is composed of a five-layer convolutional neural network, and the first, second, third, fourth and fifth layers of the generator G are all convolutional layers; the first, second, third and fourth convolutional layers of the generator G are respectively connected to a batch normalization layer and a Leaky ReLU activation layer, and the fifth convolutional layer is connected to a Tanh activation layer; wherein, the first and second convolutional layers of the generator G use a 5*5 convolution kernel, the third and fourth convolutional layers use a 3*3 convolution kernel, and the fifth convolutional layer uses a 1*1 convolution kernel; the convolutional layer step size of each layer in the generator G is set to 1, and no padding operation is performed in the convolution.

5. The copper ore preselection method based on generative adversarial network and low-rank linear pooling according to claim 4 is characterized in that: The discriminator and the discriminator The structure is the same, also composed of a five-layer convolutional neural network, the discriminator and the discriminator The first, second, third and fourth layers are convolutional layers, and the fifth layer is a linear layer; the discriminator and the discriminator The first, second, third, and fourth convolutional layers are connected in sequence with a batch normalization layer and a Leaky ReLU activation layer; the discriminator and the discriminator The first, second, third and fourth convolutional layers use 3*3 convolution kernels and the stride is set to 2.

6. The copper ore preselection method based on generative adversarial network and low-rank linear pooling according to claim 5, characterized in that: The feature extraction encoding layer uses three parallel feature extraction encoders DenseNet with the same structure to extract features from the high-energy X-ray transmission image, low-energy X-ray transmission image and dual-energy X-ray fusion image of copper ore respectively to obtain high-energy, low-energy and dual-energy feature vectors.

7. The copper ore preselection method based on generative adversarial network and low-rank linear pooling according to claim 1, characterized in that: Improving the Generator in Generative Adversarial Networks The loss function It is expressed as: （4）； In the formula, It means fighting against loss; Indicates content loss; represents a hyperparameter used to adjust the content loss in The weight in It is expressed as: （5）； In the formula, Indicates expected value; Represents dual-energy X-ray fusion image.

8. The copper ore preselection method based on generative adversarial network and low-rank linear pooling according to claim 7, characterized in that: Improving the discriminator in generative adversarial networks The loss function and the discriminator The loss function It is expressed as: （6）； （7）； In the formula, Represents high energy X-ray transmission image; Represents a low-energy X-ray transmission image.

Images