A mineral image data enhancement method based on a generative adversarial network

Abstract

This invention discloses a mineral image data augmentation method based on generative adversarial networks (GANs). It achieves unique new image data by extracting and representing feature information from individual mineral grain images. Utilizing the SRGAN mineral image data augmentation model, it can quickly and efficiently generate batches of unique mineral grain images. Compared to traditional single-sample image data augmentation methods, this invention effectively expands the feature richness of existing mineral image data. Compared to traditional multi-sample image data augmentation methods, this invention eliminates the need for additional complex preprocessing steps, simplifying the mineral image data augmentation process. By extending the loss function of the traditional GAN ​​model, it improves the generative model's ability to extract features from mineral images, fully extracting important image information such as texture, color, contour, reflection, edges, and gullies.

Description

Technical Field

[0001] This invention relates to a mineral image data enhancement method based on generative adversarial networks, belonging to the field of image data enhancement technology, and particularly to a mineral image data enhancement method based on superpixel generative adversarial networks (SRGAN). Background Technology

[0002] Currently, intelligent mineral sorting technology based on image recognition models has become a new paradigm in the mining industry, enabling pollution-free, high-efficiency, and low-cost mineral processing. The establishment and application of such models require a large amount of mineral image data. However, mineral image recognition differs from general image processing tasks, often facing problems of insufficient data volume and inadequate inter-group data, necessitating an efficient data augmentation method to address these issues.

[0003] Image data augmentation is one of the core issues in the field of data processing, and it is widely used in dataset creation, data expansion, solving data imbalance problems, and training large-scale models. As a branch of data processing technology, image data augmentation has received widespread attention in recent years, and the corresponding research methods can be mainly divided into the following three categories:

[0004] 1. Traditional Single-Sample Image Data Enhancement Methods Based on Pixel-Level Transformation. According to the basic theory of image processing, traditional single-sample image data enhancement methods based on pixel-level transformation emphasize the transformation of overall or local image characteristics. Common methods include image blurring, image equalization, image color space transformation, image noise reduction, and image color interference. Image blurring methods include Gaussian blurring, mean blurring, median blurring, and distortion smoothing; image equalization methods include adaptive histogram equalization and contrast-limited histogram equalization; image color space transformation methods include random channel pixel deactivation, region random deactivation, and color space random swapping; image noise reduction methods include Gaussian noise and salt-and-pepper noise; and image color interference methods include contrast adjustment, saturation adjustment, and brightness adjustment. Since mineral image recognition and sorting require the extraction of apparent features from mineral images, such as color features, contour features, and texture features, pixel-level transformation data enhancement methods such as image blurring, image color space transformation, and image color interference are often not applicable to practical mineral image data enhancement tasks.

[0005] 2. Traditional single-sample image data augmentation methods based on spatial-level transformations. These methods elevate single-sample image data augmentation from the pixel level to the spatial domain level, thereby improving the augmentation capability to some extent. Common traditional single-sample image data augmentation methods based on spatial-level transformations include image cropping, image scaling, image rotation, image flipping, image affine transformation, image elastic transformation, image network distortion, and image optical distortion. Clearly, while the single-sample image after spatial-level transformation differs somewhat from the original image data, it does not significantly enhance the diversity of image features, and therefore cannot be used as an efficient data augmentation method for subsequent downstream tasks.

[0006] 3. Traditional Multi-Sample Image Data Augmentation Methods Based on Multi-Sample Synthesis. With the development of data augmentation technology, some scholars have extended data augmentation methods from single-sample to multi-sample scenarios, proposing traditional multi-sample image data augmentation methods based on multi-sample synthesis. For example, the mixup multi-sample synthesis data augmentation method proposed in the paper "mixup: Beyond Empirical Risk Minimization" first uses single-sample image data augmentation methods, such as rotation, cropping, and equalization, to generate three new sample images. Then, it randomly selects three weights using a Dirichlet distribution and fuses the three new sample images pixel by pixel according to the weight values ​​to generate a new sample image. Generally, this type of method requires a large amount of image preprocessing and has poor augmentation capabilities for mineral image data under complex conditions. Summary of the Invention

[0007] In view of this, the present invention provides a mineral image data augmentation method based on generative adversarial networks, which can quickly generate a large amount of new mineral image data without repetition after model training.

[0008] The technical solution of this invention is:

[0009] A mineral image data augmentation method based on generative adversarial networks includes the following steps:

[0010] Step 1: Acquire multiple raw image data of the mineral to be processed;

[0011] Step 2: Use a threshold segmentation algorithm that includes edge particle processing to segment the multiple original image data obtained in Step 1 to obtain mineral single-grain image data;

[0012] Step 3: Build a data augmentation SRGAN deep learning model for the mineral single-grain image data obtained in Step 2;

[0013] Step 4: Train the data augmentation SRGAN deep learning model built in Step 3 to obtain the loss value and learning rate of the data augmentation SRGAN deep learning model;

[0014] Step 5: Based on the loss value and learning rate obtained in Step 4, terminate the training of the data augmentation SRGAN deep learning model to obtain the trained SRGAN mineral image data augmentation model.

[0015] Step 6: Use the SRGAN mineral image data augmentation model trained in Step 5 to perform data augmentation on the mineral single-grain image data obtained in Step 2, thus completing the mineral single-grain image data augmentation process.

[0016] In step 1, when acquiring multiple raw image data of the mineral to be processed, a mineral image data acquisition platform is set up; the multiple raw image data of the mineral to be processed are taken by an industrial camera directly above the conveyor belt;

[0017] In step 3, the constructed data-augmented SRGAN deep learning model includes a generator G and a discriminator D. The generator uses a convolutional neural network with residual connections, containing 16 residual feature extraction modules and 2 deconvolution modules. This convolutional neural network with residual connections can efficiently extract mineral image feature information. A deconvolution layer is used at the back end of the generator to realize feature remapping and new image generation. The discriminator uses a convolutional neural network, which contains 8 convolutional segments, 2 fully connected layers and 1 sigmoid activation layer. The LeakyReLU activation function is used to compare, identify and judge the generated fake images with real objects, and feed the judgment result back to the generator to achieve overall optimization of the weights of the data-augmented SRGAN deep learning model.

[0018] Step 4, the training process includes:

[0019] The first step is to determine the adversarial loss function for the data-augmented SRGAN deep learning model. for:

[0020]

[0021] In the formula, I LR The low-resolution mineral images are obtained by applying Gaussian filtering to the high-resolution input mineral images of the generator G; n = 1, 2, 3...N, where N is the image data size for each batch;

[0022] The second step is to determine the content loss function of the data-augmented SRGAN deep learning model. for:

[0023]

[0024] In the formula, W, H, and r are the width, height, and downsampling factor of the feature image extracted by the convolutional layer in the generator G, respectively; x and y are the horizontal and vertical coordinates of the feature image extracted by the convolutional layer in the generator G, respectively; and I... LR Provide a low-resolution mineral image as input to generator G; The input is a high-resolution image; x = 1, 2, 3, ..., rW, y = 1, 2, 3, ..., rH;

[0025] The third step is to determine the perceptual loss function L for the data-augmented SRGAN deep learning model. SR for:

[0026]

[0027] In the formula, For content loss function, For adversarial loss function;

[0028] The fourth step is to apply the perceptual loss function l SR Determine the loss value of the data-augmented SRGAN deep learning model so that the learning rate can be obtained during training;

[0029] In step 5, when training is terminated, the termination time is determined based on the threshold of the loss value and the learning rate.

[0030] Step 6, the data augmentation process, includes:

[0031] First, features are extracted from the single mineral grain image to be processed using the convolutional layer of generator G;

[0032] Then, the deconvolution layer of generator G is used to restore the feature vector from the extracted features, generating new mineral single-grain image data, thus completing the mineral single-grain image data enhancement process.

[0033] The present invention has the following beneficial effects:

[0034] (1) The method provided by the present invention utilizes the SRGAN mineral image data augmentation model to quickly and efficiently generate new single-grain images of minerals without repetition in batches. Compared with traditional single-sample image data augmentation methods, the present invention can effectively expand the richness of image features in existing mineral image data.

[0035] (2) Compared with traditional multi-sample image data enhancement methods, the present invention does not require additional complex preprocessing procedures, simplifying the steps of mineral image data enhancement. Therefore, the present invention is more likely to implement mineral image data enhancement tasks.

[0036] (3) The method provided by the present invention extends the loss function of the traditional GAN ​​model, improves the extraction capability of the generation model in mineral image features, and can fully extract important image information such as texture, color, contour, reflection, edge, and gully in mineral images. Therefore, the image generation effect of the present invention is better.

[0037] (4) The method provided by this invention adopts end-to-end deep learning model training technology, so there is no need to interactively change the model object or adjust the model parameters, realizing the integrated training process of generator and discriminator. Attached Figure Description

[0038] Figure 1 A flowchart of the method provided by this invention;

[0039] Figure 2 This is a schematic diagram of the image data acquisition platform of the present invention;

[0040] Figure 3 This is a flowchart of the threshold segmentation algorithm of the present invention;

[0041] Figure 4 This is an example image of a single mineral grain after threshold segmentation according to the present invention;

[0042] Figure 5 Example diagram of generated data provided by the method of the present invention;

[0043] Figure 6 This is an example diagram illustrating the authenticity analysis of the method provided by the present invention. Detailed Implementation

[0044] The present invention will now be described in detail with reference to the accompanying drawings.

[0045] like Figure 1 As shown, this invention provides a mineral image data enhancement method based on generative adversarial networks, specifically including the following steps:

[0046] Step 1: Set up as follows Figure 2 The mineral image data acquisition platform shown comprises five main components: a vibrating screen, a belt conveyor, an industrial camera, a lighting source, and a computer. During the acquisition process, mineral samples meeting the particle size requirements are sieved by the vibrating screen and then evenly distributed onto a conveyor belt. The conveyor belt transports the samples to be photographed to the industrial camera, where they are captured under the illumination of a linear LED strip light source. The images are then stored on the computer. In the image acquisition system, a sensor at the conveyor belt shaft converts the belt rotation speed into a digital signal, which is transmitted to the industrial camera for adaptive adjustment. The industrial camera is a 4K color 3CCD linear scan camera with accompanying image processing software, enabling continuous image capture and storage of samples under constant speed operation. The intensity of the linear LED strip light can be adjusted according to actual needs.

[0047] Step 2: Use as follows Figure 3 The threshold segmentation algorithm shown, which includes edge particle processing, segments the original mineral image to obtain individual mineral particle image data. In the internal individual mineral particle image segmentation stage, the original mineral image undergoes threshold segmentation of the mineral sample region and background region, particle labeling, binary processing, and edge particle removal sequentially. Then, area thresholding and hole filling operations are used to remove noise and fill the internal pores of the particles. For the removed edge particles, the preceding and following frames at the edge particle location are stitched together to form a new binary image, which is then subjected to area thresholding and hole filling operations to remove noise and fill the internal pores of the particles.

[0048] The image processed as described above is subjected to continuous finite erosion operations on the individual mineral sample regions using a 10×10 square structuring element until the area falls below a preset area threshold, and the number of erosion operations is recorded. Based on the number of erosion operations, the individual mineral sample regions are dilated to restore the mineral particle image. The pixels of the individual mineral sample regions are consistent with those before the finite erosion process, effectively solving the problem of adhesion and overlap between mineral particles. Finally, the color image corresponding to each individual mineral sample region in the original image is cropped using the minimum bounding rectangle as the individual mineral image data.

[0049] Step 3: Build an SRGAN mineral image data augmentation model and determine the adversarial loss function (Equation 1), content loss function (Equation 2), and perceptual loss function (Equation 3).

[0050]

[0051]

[0052]

[0053] As described in the SRGAN mineral image data augmentation model design scheme, the network model includes at least a generator G, a discriminator D, convolutional layers, pooling layers, normalization layers, activation layers, residual connection modules, deconvolutional layers, and fully connected layers. The generator comprises 16 residual feature extraction modules and 2 deconvolutional modules. The discriminator contains 8 convolutional segments, 2 fully connected layers, and one sigmoid activation layer, employing the LeakyReLU activation function to compare, identify, and judge the generated forged images with real objects, feeding the judgment result back to the generator segment to achieve overall optimization of the network weights.

[0054] During training, the generator G is responsible for generating new single-grain mineral image data based on the learned single-grain mineral image features. The discriminator is responsible for judging the authenticity of the single-grain mineral images generated by the generator G. When the generated data error is small, the discriminator outputs a value of 1, and when it is large, the output value is 0. The discriminator calculates the loss value according to the loss function and propagates the loss value to the generator G and the discriminator D to adjust the network weights.

[0055] Finally, the training of the model is terminated based on whether the training has reached a preset number of rounds or whether the learning rate meets a preset threshold, and the trained SRGAN mineral image data augmentation model is obtained.

[0056] Step 4: Perform data augmentation on the mineral single-grain image data to be processed using the trained SRGAN mineral image data augmentation model to obtain new non-repeating mineral single-grain image data.

[0057] Step 5: Use the enhanced mineral image data and the original mineral image data as sample data for downstream tasks such as mineral image classification, segmentation, and object detection, and apply them to the specific training tasks of subsequent downstream models.

[0058] Example

[0059] This example uses gas coal from the Shanxi mining area in China as the mineral sample. The sample was manually sieved to select particles of 13-25 mm, and further divided into <1.4 g / cm³ particles based on density distribution. 3 1.4g / cm 3 -1.6g / cm 3 1.6g / cm 3 -1.8g / cm 3 With >1.8g / cm 3 Four groups. The mass percentage and ash content of each density aggregate mineral sample are shown in Table 1, and the results of microscopic component analysis are shown in Table 2.

[0060] Table 1. Mass percentage and ash content of mineral samples of various density clusters (unit: percentage)

[0061]

[0062] Table 2. Microscopic component analysis of mineral samples of different density aggregates (unit: percentage)

[0063]

[0064] A mineral image data acquisition platform as described in step 1 was constructed. In this embodiment, a 4K color 3CCD linear array camera (JAI 3CCD Datasheet_LT-400CL) was selected as the industrial camera, enabling continuous image capture and storage of samples under constant speed operation. A built-in sensor was embedded in the conveyor belt shaft to convert the conveyor belt speed into a digital signal for adaptive adjustment by the industrial camera. The light source selected in this embodiment was a linear LED strip with a length of 500 mm, providing uniform illuminance and a color temperature of 5800-7000 K. The surface brightness of the belt during the imaging process was approximately 250 klux. Since the ambient light brightness was only 130 lx, the influence of ambient light could be completely shielded. In the experiment, the conveyor belt speed was 1 m / s, the acquired coal image size was 4096 × 3000 pixels, and the accuracy was 0.098 mm / pixel.

[0065] The original mineral image is segmented using a threshold segmentation algorithm that includes edge grain processing, as described in step 2. The segmentation result is as follows: Figure 4 As shown.

[0066] An SRGAN mineral image data augmentation model was constructed. The generator comprises one convolutional segment, 16 residual feature extraction modules, two deconvolutional segments, and one additional convolutional layer. The first convolutional segment contains one convolutional layer and one ReLU activation layer. The 16 residual extraction modules have the same structure, containing two convolutional layers, two batch normalization layers, one ReLU activation layer, and one element-wise summing layer; this part is mainly used for single-grain image feature extraction. The two deconvolutional segments have the same structure, each containing one deconvolutional layer and one ReLU activation layer. The additional convolutional layer is used for channel information interaction and final image generation.

[0067] The discriminator consists of 8 convolutional segments, 2 fully connected layers, and 1 sigmoid activation layer: the first convolutional segment contains 1 convolutional layer and 1 LeakyReLU activation layer; the second to seventh convolutional segments have the same structure, each containing 1 convolutional layer, 1 LeakyReLU activation layer, and 1 batch normalization layer; the two fully connected layers contain a LeakyReLU activation layer in between; and the sigmoid activation layer is used to determine the true probability of the generated image.

[0068] The trained SRGAN mineral image data augmentation model is used to perform data augmentation on the single mineral grain images to be processed. The new single mineral grain images generated by SRGAN are as follows: Figure 5 As shown.

[0069] To verify the effectiveness of the proposed SRGAN mineral image data augmentation model, the training dataset of the original mineral images was expanded using the trained SRGAN mineral image data augmentation model. The amount of training data before and after expansion is shown in Table 3.

[0070] Table 3. Mineral Image Dataset from Examples (Unit: Images)

[0071]

[0072] The examples constructed four commonly used mineral image classification models: VGG 19, Inception V3, ResNet 50, and ResNet 101, to test the changes in model accuracy before and after data augmentation. The comparison results are shown in Table 4.

[0073] Table 4. Changes in accuracy of four types of deep learning image classification models before and after training (unit: percentage)

[0074]

[0075] The accuracy improvement demonstrates that the image data generated by the proposed SRGAN mineral image data augmentation model can effectively optimize the detection performance of deep learning models in downstream mineral image classification tasks. Compared to before data augmentation, the VGG 19, Inception V3, ResNet 50, and ResNet 101 models trained on the augmented dataset achieved classification accuracies of 77.34%, 80.91%, 87.17%, and 92.26%, respectively, representing increases of 4.40%, 5.39%, 11.8%, and 20.09%.

[0076] To effectively verify the authenticity of the single-grain mineral images generated by the proposed SRGAN mineral image data augmentation model, the embodiment employs a t-distribution random nearest neighbor embedding dimensionality reduction algorithm to analyze the similarity between the generated and real images at the image feature level. Dimensionality reduction visualization was performed by randomly selecting 100 real samples and 100 generated samples from four-density gas-gathering coal images, and the results are as follows. Figure 6 As shown.

[0077] Visualization results show a high degree of consistency between the generated and real sample images in terms of image feature distribution, indicating that the proposed SRGAN mineral image data augmentation model can effectively preserve the feature information of the ore surface. For >1.8 g / cm³ 3 In the gas coal image data, the red and blue scatter points are very closely distributed, indicating a very high degree of feature similarity between the generated image and the real image; for 1.6-1.8 g / cm³... 3 and 1.4-1.6 g / cm 3 In the gas coal image data, the red and blue scatter dots are relatively closely distributed; compared to the other three categories, for <1.4 g / cm³... 3 The distribution of red and blue scatter points in the gas and coal image data is slightly dispersed, but overall it still maintains a consistent relationship.

[0078] The above analysis results show that the proposed SRGAN mineral image data augmentation model has different data augmentation effects in gas coal image datasets of different densities, but it can still efficiently retain most of the feature information of gas coal in the overall effect. That is, the proposed method can serve as an efficient solution to provide data support for downstream tasks.

[0079] In summary, this invention provides a mineral image data augmentation method based on generative adversarial networks (GANs), applicable to image data augmentation for most minerals. The main process includes building a mineral image data acquisition platform, segmenting mineral single-grain images using a thresholding method, building and training an SRGAN mineral image data augmentation model, and applying the trained SRGAN model. Performance test results for downstream classification tasks and t-SNE visualization analysis show that the proposed method can effectively optimize downstream processing tasks similar to mineral image classification, and the generated new mineral single-grain image data exhibits high feature consistency with real mineral single-grain sample images, demonstrating the effectiveness of the proposed method.

[0080] It should be understood that the above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A mineral image data enhancement method based on generative adversarial networks, characterized in that... Includes the following steps: Step 1: Acquire multiple raw image data of the mineral to be processed; Step 2: Use a threshold segmentation algorithm that includes edge particle processing to segment the multiple original image data obtained in Step 1 to obtain mineral single-grain image data; Step 3: Build a data augmentation SRGAN deep learning model for the mineral single-grain image data obtained in Step 2; Step 4: Train the data augmentation SRGAN deep learning model built in Step 3; Step 5: Terminate the training of the data augmentation SRGAN deep learning model to obtain the trained SRGAN mineral image data augmentation model; Step 6: Use the SRGAN mineral image data augmentation model trained in Step 5 to perform data augmentation on the mineral single-grain image data obtained in Step 2, thus completing the mineral single-grain image data augmentation process. In step 3, the constructed data-augmented SRGAN deep learning model includes a generator G and a discriminator D. The generator uses a convolutional neural network with residual connections, containing 16 residual feature extraction modules and 2 deconvolution modules. This convolutional neural network with residual connections can efficiently extract mineral image feature information. A deconvolution layer is used at the back end of the generator to realize feature remapping and new image generation. The discriminator uses a convolutional neural network, which contains 8 convolutional segments, 2 fully connected layers and 1 sigmoid activation layer. The LeakyReLU activation function is used to compare, identify and judge the generated fake images with real objects, and feed the judgment result back to the generator to achieve overall optimization of the weights of the data-augmented SRGAN deep learning model. Step 4, the training process includes: The first step is to determine the adversarial loss function for the data-augmented SRGAN deep learning model. for: In the formula, The low-resolution mineral image is obtained by Gaussian filtering the high-resolution input mineral image of the generator G; n=1, 2, 3…N, where N is the image data size for each batch. The second step is to determine the content loss function of the data-augmented SRGAN deep learning model. for: In the formula, , , The width, height, and downsampling factor of the feature image extracted by the convolutional layer in the generator G are respectively. , These are the x and y coordinates of the feature image extracted by the convolutional layer in generator G. Provide a low-resolution mineral image as input to generator G; The input is a high-resolution image; x = 1, 2, 3, ... y = 1, 2, 3, ... ; The third step is to determine the perceptual loss function for the data-augmented SRGAN deep learning model. for: In the formula, For content loss function, For adversarial loss function; Based on the perceptual loss function Determine the loss value of the data-augmented SRGAN deep learning model so that the learning rate can be obtained during training; Based on the loss value and learning rate obtained in step 4, the training of the data augmentation SRGAN deep learning model is terminated, and the trained SRGAN mineral image data augmentation model is obtained. Step 6, the data augmentation process, includes: First, features are extracted from the single mineral grain image to be processed using the convolutional layer of generator G; Then, the deconvolution layer of generator G is used to restore the feature vector from the extracted features, generating new mineral single-grain image data, thus completing the mineral single-grain image data enhancement process.

2. The mineral image data enhancement method based on generative adversarial networks according to claim 1, characterized in that: In step 1, when acquiring multiple raw image data of the mineral to be processed, a mineral image data acquisition platform is set up; the multiple raw image data of the mineral to be processed are taken by an industrial camera directly above the conveyor belt.

Images