Flotation foam image classification method integrating denoising coding and multi-scale deep learning

Abstract

The present invention provides a flotation foam image classification method that integrates noise reduction coding and multi-scale deep learning. The method adopts a new model formed by integrating the encoding and decoding layers of different scales in the noise reduction encoder with a multi-scale classification model. The training process is divided into two steps: the first step is noise reduction training, in which random noise is added to the original image before training, and the noisy image is used as the input of the noise reduction training, and the original image is used as the output, so that a better filtering effect is achieved through training. The second step is classification training, in which all parameters are locked, and the layers of different scales in the encoding and decoding stages of the noise reduction model are integrated with the multi-scale classification model, so that the model has a higher recognition rate for flotation conditions through training. The method can better identify flotation foam image conditions with noise points; at the same time, it can shorten the time of model training and reasoning.

Description

Technical Field

[0001] The present invention relates to the technical field of image classification, and in particular to a flotation foam image classification method integrating noise reduction coding and multi-scale deep learning. Background Art

[0002] Froth flotation is a method of separating minerals from impurities or one mineral from another. Unpurified ore powder is stirred with water and reagents. The reagents preferentially adhere to the target ore and change its surface properties. Air is then introduced into the mixture, and the target ore floats to the surface of the mixture with bubbles to form foam. After the foam is removed, the target ore can be obtained.

[0003] Adding too much flotation frother will form larger foam (such as Figure 1 Flotation conditions (1)), which can easily cause gangue minerals to adhere to bubbles, reducing the concentrate grade; if too little flotation frother is added, smaller bubbles will be formed (such as Figure 3 Flotation conditions (3)) The amount of mineral powder carried between bubbles is small, and the flotation efficiency is low; therefore, it is necessary to add a suitable flotation frother to achieve a suitable flotation foam state (such as Figure 2 Flotation conditions (2)).

[0004] Currently, the identification of flotation conditions is done manually, but there is a lot of dust on site, and the dust contains a lot of heavy metals, which can cause a lot of irreparable damage to the human body. Therefore, at this stage, a large number of flotation conditions are identified using machine vision.

[0005] Many scholars use traditional image recognition methods to identify the type of flotation conditions.

[0006] With the development of deep learning, many teams have also applied deep learning to flotation foam image classification in recent years. However, at the flotation site, the lens will often slowly be stained with a lot of mining dust on site. The longer the time, the more noise there will be in the image captured by the lens, resulting in poor image quality and affecting the recognition effect. Summary of the invention

[0007] In view of this, the purpose of the present invention is to provide a flotation foam image classification method that integrates noise reduction coding and multi-scale deep learning, which can better identify the flotation foam image conditions with noise points; at the same time, it can shorten the time of model training and reasoning.

[0008] To achieve the above-mentioned purpose, the present invention adopts the following technical scheme: a flotation foam image classification method integrating noise reduction coding and multi-scale deep learning, including noise reduction training and classification training; the noise reduction training includes a coding stage and a decoding stage; in the coding stage, the image size is gradually reduced by convolution and pooling, and the main outline of the image is refined and the noise is filtered in the process of reducing the image size, and the image size is reduced from 256*256 to 64*64 in the coding stage; in the decoding stage, the image size is restored from 64*64 to 256*256 by deconvolution, and the mean square error between the noise image and the original image is minimized through training; in the multi-scale classification, the convolution layers in the coding layer and the decoding layer are reused, and the flotation foam image is judged by three scales of 256*256, 128*128 and 64*64.

[0009] In a preferred embodiment, the encoding stage has a total of 8 layers, and the texture part of the image is extracted by combining three pooling layers and convolution layers:

[0010] (A1) The first layer is the image input layer, which inputs a floating image with a size of 256*256*3;

[0011] (A2) The image size is converted from 256*256*3 to 256*256*32 through the convolution layer C1 with a convolution kernel of 32;

[0012] (A3) Then pass through the maximum pooling layer P1, let the image lose some information, extract the main part of the image texture, and reduce its size to 128*128*32;

[0013] (A4) The pooled data is then adjusted through a convolution layer C2 with a convolution kernel of 32;

[0014] (A5) Then, through the maximum pooling layer P2, the image loses information again, extracts the texture, and reduces its size to 64*64*32;

[0015] (A6) The pooled data is then adjusted through a convolution layer C3 with a convolution kernel of 32;

[0016] (A7) Next, the maximum pooling layer P3 is used to reduce its size to 32*32*32;

[0017] (A8) Finally, the batch normalization layer B1 is used to prevent the network from experiencing gradient explosion, which makes it impossible to train, and to alleviate the overfitting phenomenon of the network.

[0018] In a preferred embodiment, the decoding stage is divided into 8 layers, and the core information of the image is restored through a combination of three deconvolution layers + convolution layers, by restoring the image from the core to a normal image, and finally batch normalization of the image:

[0019] (B1) First, a deconvolution layer T1 with a stride of 2 is used to restore the image size from 32*32*32 to 64*64*32;

[0020] (B2) A convolutional layer C4 is then used to adjust the deconvolution data, while also increasing the depth of the model;

[0021] (B3) Through the second deconvolution layer T2 with a stride of 2, the image size is restored from 64*64*32 to 128*128*32;

[0022] (B4) Cooperate with the second convolutional layer C5 to adjust the data again;

[0023] (B5) Finally, the image size is restored from 128*128*32 to 256*256*32 through the third deconvolution layer T3 with a stride of 2;

[0024] (B6) The data is then conditioned through the third convolutional layer C6;

[0025] (B7) Then the batch normalization layer B2 is used to prevent the network from experiencing gradient explosion and thus making it impossible to train;

[0026] (B8) Finally, through a convolutional layer C7 with 3 kernels, the image size is restored from 256*256*32 to the normal image size of 256*256*3.

[0027] In a preferred embodiment, the multi-scale classification is specifically as follows:

[0028] (C1) The noisy convolution layer C1 (256*256*32) and the denoised convolution layer C6 (256*256*32) are superimposed to generate a superimposed layer Ca1 (256*256*64);

[0029] (C2) Through a convolutional layer C7 with 32 kernels, the size of the superimposed layer Ca1, i.e. 256*256*64, is reduced to 256*256*32;

[0030] (C3) The noisy convolution layer C2 (128*128*32) and the denoised convolution layer C5 (128*128*32) are superimposed to generate a superimposed layer Ca2 (128*128*64);

[0031] (C4) Through a convolutional layer C8 with 32 kernels, the size of the superimposed layer Ca1, i.e. 128*128*64, is reduced to 128*128*32;

[0032] (C5) The noisy convolution layer C3 (64*64*32) and the denoised convolution layer C4 (64*64*32) are superimposed to generate a superimposed layer Ca3 (64*64*64);

[0033] (C6) Through a convolutional layer C9 with 32 kernels, the size of the superimposed layer Ca1, i.e. 64*64*64, is reduced to 64*64*32;

[0034] (C7) The data is processed by single-layer convolution through three convolution layers C10, C11, and C12 with a convolution kernel of 16 and a kernel size of 1*1. The final output size is C10, i.e. 256*256*16, C11, i.e. 128*128*16, and C12, i.e. 64*64*16;

[0035] (C8) Then, three global average pooling layers GAP1, GAP2, and GAP3 are used to integrate the data of C7 (256*256*16), C8 (128*128*16), and C9 (64*64*16) into one-dimensional data: 16, 16, and 16;

[0036] (C9) superimposing the data of GAP1, namely 16, GAP2, namely 16, and GAP3, namely 16, to form superimposed data GAP4, namely 48;

[0037] (C10) GAP4, i.e., 96 inputs to the fully connected layer, with the input being 48 and the output being 32, to the fully connected layer D1;

[0038] (C11) Dropout layer DP1 is used to randomly change the weights of 50% of the hidden layer nodes to 0;

[0039] (C12) Finally, the classification data of the flotation image is output through the fully connected layer D2 with an input of 32 and an output of 3.

[0040] Compared with the prior art, the present invention has the following beneficial effects: through training, the model has a higher recognition rate for flotation conditions. The method can better identify flotation foam image conditions with noise points; and at the same time, it can shorten the time of model training and reasoning. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 This is the flotation condition (1) in the prior art;

[0042] Figure 2 This is the flotation condition (2) in the prior art;

[0043] Figure 3 This is the flotation condition (3) in the prior art;

[0044] Figure 4It is a schematic diagram of a noise reduction coding fusion multi-scale flotation foam classification model according to a preferred embodiment of the present invention;

[0045] Figure 5 This is a model diagram of a flotation foam image classification method integrating noise reduction coding and multi-scale deep learning in a preferred embodiment of the present invention;

[0046] Figure 6 This is a diagram showing the filtering effect of adding different noises to the test set image in a preferred embodiment of the present invention;

[0047] Figure 7 This is a comparison diagram of the SSIM values ​​of the noise image after filtering and the original image relative to the SSIM values ​​of the noise and the original image in the preferred embodiment of the present invention;

[0048] Figure 8 A comparison diagram of the PSNR values ​​of the noise image after filtering and the original image relative to the PSNR values ​​of the noise and the original image in a preferred embodiment of the present invention;

[0049] Fig. 9 This is a graph showing the accuracy of the training set and the validation set after 100 epochs of training according to a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0050] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0051] It should be noted that the following detailed descriptions are illustrative and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0052] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application; as used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or their combinations.

[0053] Flotation foam image classification method integrating noise reduction coding and multi-scale deep learning, reference Figure 4 and 5, including noise reduction training and classification training; the noise reduction training includes an encoding stage and a decoding stage; in the encoding stage, the image size is gradually reduced by convolution and pooling, the main outline of the image is refined and the noise is filtered in the process of reducing the image size, and the image size is reduced from 256*256 to 64*64 in the encoding stage; in the decoding stage, the image size is restored from 64*64 to 256*256 by deconvolution, and the mean square error between the noise image and the original image is minimized through training; in the multi-scale classification, the convolution layers in the encoding layer and the decoding layer are reused, and the flotation foam image is judged at three scales of 256*256, 128*128 and 64*64.

[0054] The encoding stage has a total of 8 layers, which mainly extract the texture part of the image through the combination of three pooling layers + convolution layers, extract the core (main texture) information of the image, discard the noise, and achieve the encoding effect. Finally, the batch normalization of the image is performed to prevent overfitting and gradient explosion in the network. The specific information of the image is shown in Table 1:

[0055] (A1) The first layer is the image input layer, which inputs a floating image with a size of 256*256*3;

[0056] (A2) The image size is converted from 256*256*3 to 256*256*32 through the convolution layer C1 with a convolution kernel of 32;

[0057] (A3) Then pass through the maximum pooling layer P1 to make the image "lose" some information, extract the main texture of the image, and reduce its size to 128*128*32;

[0058] (A4) The pooled data is then adjusted through a convolution layer C2 with a convolution kernel of 32;

[0059] (A5) Then, through the maximum pooling layer P2, the image is again “lost”, the texture is extracted, and its size is reduced to 64*64*32;

[0060] (A6) The pooled data is then adjusted through a convolution layer C3 with a convolution kernel of 32;

[0061] (A7) Next, the maximum pooling layer P3 is used to reduce its size to 32*32*32;

[0062] (A8) Finally, the batch normalization layer B1 is used to prevent the network from experiencing gradient explosion, which makes it impossible to train, and to alleviate the overfitting phenomenon of the network.

[0063] The decoding stage is also divided into 8 layers. It mainly restores the core information of the image through the combination of three deconvolution layers + convolution layers. The image is restored from the core to the normal image. Since the noise information has been discarded during the encoding process, the restored image will be able to achieve a good noise reduction effect. Finally, the batch normalization of the image is used to prevent overfitting and gradient explosion in the network:

[0064] (B1) First, a deconvolution layer T1 with a stride of 2 is used to restore the image size from 32*32*32 to 64*64*32;

[0065] (B2) A convolutional layer C4 is then used to adjust the deconvolution data, while also increasing the depth of the model;

[0066] (B3) Through the second deconvolution layer T2 with a stride of 2, the image size is restored from 64*64*32 to 128*128*32;

[0067] (B4) Cooperate with the second convolutional layer C5 to adjust the data again;

[0068] (B5) Finally, the image size is restored from 128*128*32 to 256*256*32 through the third deconvolution layer T3 with a stride of 2;

[0069] (B6) After passing through the third convolutional layer C6, the data is conditioned;

[0070] (B7) Here, the batch normalization layer B2 is used to prevent the network from experiencing gradient explosion and thus making it impossible to train;

[0071] (B8) Finally, through a convolutional layer C7 with 3 kernels, the image size is restored from 256*256*32 to the normal image size of 256*256*3.

[0072] Compared with other methods, this method reuses the convolutional layers in the encoding and decoding layers in multi-scale classification, and judges the flotation foam image at three scales (256*256, 128*128, and 64*64).

[0073] (C1) The noisy convolution layer C1 (256*256*32) and the denoised convolution layer C6 (256*256*32) are superimposed to generate the superimposed layer Ca1 (256*256*64).

[0074] (C2) Through a convolutional layer C7 with a kernel number of 32, the size of the superimposed layer Ca1 (256*256*64) is reduced to 256*256*32.

[0075] (C3) The convolution layer C2 (128*128*32) with noise added and the convolution layer C5 (128*128*32) after denoising are superimposed to generate the superimposed layer Ca2 (128*128*64).

[0076] (C4) Through a convolutional layer C8 with 32 kernels, the size of the superimposed layer Ca1 (128*128*64) is reduced to 128*128*32.

[0077] (C5) The convolution layer C3 (64*64*32) with noise added and the convolution layer C4 (64*64*32) after denoising are superimposed to generate a superimposed layer Ca3 (64*64*64).

[0078] (C6) Through a convolutional layer C9 with a kernel number of 32, the size of the superimposed layer Ca1 (64*64*64) is reduced to 64*64*32.

[0079] (C7) Next, the data is processed by single-layer convolution through three convolution layers C10, C11, and C12 with a convolution kernel of 16 and a kernel size of 1*1. The final output size is C10 (256*256*16), C11 (128*128*16), and C12 (64*64*16).

[0080] (C8) Next, the data of C7 (256*256*16), C8 (128*128*16), and C9 (64*64*16) are respectively integrated into one-dimensional data through three global average pooling layers GAP1, GAP2, and GAP3: 16, 16, and 16.

[0081] (C9) Then the data of GAP1 (16), GAP2 (16), and GAP3 (16) are superimposed to form superimposed data GAP4 (48)

[0082] (C10) GAP4(96) is input into the fully connected layer with an input of 48 and an output of 32 in the fully connected layer D1.

[0083] (C11) Through the Dropout layer DP1, the weights of 50% of the hidden layer nodes are randomly changed to 0 to prevent the network from overfitting.

[0084] (C12) Finally, the classification data of the flotation image is output through the fully connected layer D2 with an input of 32 and an output of 3.

[0085] Table 1

[0086]

[0087]

[0088] Experimental verification:

[0089] The dataset consists of three types of flotation foam images taken in the lead ore flotation cell provided by Fujian Jindong Mining Co., Ltd. The image size is 720*480, including 1000 images of flotation condition 1, 1000 images of flotation condition 2, and 1000 images of flotation condition 3. The experimental process is divided into 1800 training data, 600 test data, and 600 verification data according to the ratio of various working conditions (1:1:1).

[0090] Due to the small amount of data, samples were amplified before training and the data was cropped on all sides: the cropping size was 255*255. After data amplification, the corresponding training data, test data, and validation data were 5400, 1800, and 1800, respectively.

[0091] Training process description:

[0092] This method is divided into two steps to train the model:

[0093] The first step is to train the model's denoising encoding performance through 100 batches (epochs) of denoising training;

[0094] In the second step, the parameters of the existing denoising coding part are locked, and then classification training is performed through 100 batches (epochs).

[0095] Ultimately, accurate classification can be achieved even in noisy environments.

[0096] The training is divided into two steps:

[0097] Step 1: Noise reduction training

[0098] Before training, we add random noise with a normal distribution mean of 0.1 and a standard deviation of 0-0.3 to the image as input data, set the original image without noise as the label, and minimize the mean square error between the input image and the output image. After 20 batches (epochs) of training, the filtering effect of adding different noises to the test set images is as follows Figure 6 shown.

[0099] The filtering effect is observed through the two evaluation criteria of SSIM and PSNR.

[0100] SSIM (Structural Similarity) is an indicator to measure the similarity between two images. This indicator was first proposed by the Laboratory for Image and Video Engineering at the University of Texas at Austin. SSIM uses two images, one is an uncompressed, undistorted image, and the other is a distorted image.

[0101] The range of structural similarity is -1 to 1. When two images are exactly the same, the SSIM value is equal to 1, and the closer the two images are, the closer the SSIM value is to 1.

[0102] Depend on Figure 7 It can be seen that the SSIM value of the noise image after filtering and the original image (solid line) is closer to 1 than the SSIM value of the noise and the original image (dashed line), indicating that the SSIM parameter of the filter in this method has a greater improvement on the quality of the image than the SSIM parameter of the noise.

[0103] PSNR is the abbreviation of "Peak Signal to Noise Ratio", which is an objective standard for evaluating images. It has limitations and is generally used for the situation between the maximum signal and the background noise. The unit of PSNR is dB. Therefore, the larger the PSNR value, the less distortion.

[0104] Depend on Figure 8 It can be seen that the PSNR value of the noise image after filtering and the original image (solid line) is larger than the PSNR value of the noise and the original image (dashed line), indicating that the effect after filtering is closer to the real image. The specific values ​​are shown in Table 2.

[0105] Table 2

[0106]

[0107] The second step is classification training

[0108] During the training process, the parameters of the C1, P1, C2, P2, C3, P3, B1, T1, C4, T2, C5, T3, and C6 layers in the encoding and decoding process are frozen. After 100 epochs of training, the accuracy of the training set and the validation set is as follows: Fig. 9 As shown in the figure, it can be seen that the accuracy of the training set is basically stable at 90% after 15 batches (epochs) of training, and the subsequent data gradually increases to 95%. The data of the validation set fluctuates around the training set, and there is no obvious overfitting or underfitting phenomenon. Finally, the test set is used for testing.

[0109] We added random noise with a normal distribution mean of 0.1 and a standard deviation of 0-0.3 as input data to the 1,800 test set images. Among them, 1,694 images were classified correctly and 106 images were misclassified, with an accuracy rate of 94.11%.

Claims

1. A flotation foam image classification method integrating noise reduction coding and multi-scale deep learning, characterized in that The method includes noise reduction training and classification training; the noise reduction training includes an encoding stage and a decoding stage; in the encoding stage, the image size is gradually reduced by convolution and pooling, and the main outline of the image is refined and the noise is filtered in the process of reducing the image size. In the encoding stage, the image size is reduced from 256*256 to 64*64; in the decoding stage, the image size is restored from 64*64 to 256*256 by deconvolution, and the mean square error between the noise image and the original image is minimized by training; in multi-scale classification, the convolution layers in the encoding layer and the decoding layer are reused, and the flotation foam image is judged by three scales of 256*256, 128*128 and 64*64; The multi-scale classification is as follows: (C1) Superimpose the noisy convolutional layer C1 (256*256*32) and the denoised convolutional layer C6 (256*256*32) to generate a superimposed layer Ca1 (256*256*64); (C2) Through a convolutional layer C8 with 32 kernels, the size of the superimposed layer Ca1, i.e. 256*256*64, is reduced to 256*256*32; (C3) The noisy convolution layer C2 (128*128*32) and the denoised convolution layer C5 (128*128*32) are superimposed to generate a superimposed layer Ca2 (128*128*64); (C4) Through a convolutional layer C9 with 32 kernels, the size of the superimposed layer Ca2, i.e. 128*128*64, is reduced to 128*128*32; (C5) The noisy convolution layer C3 (64*64*32) and the denoised convolution layer C4 (64*64*32) are superimposed to generate a superimposed layer Ca3 (64*64*64); (C6) Through a convolutional layer C10 with 32 kernels, the size of the superimposed layer Ca3, i.e. 64*64*64, is reduced to 64*64*32; (C7) The data is processed by single-layer convolution through three convolution layers C11, C12, and C13 with a convolution kernel of 16 and a kernel size of 1*1. The final output size is C11, i.e. 256*256*16, C12, i.e. 128*128*16, and C13, i.e. 64*64*16; (C8) Then, three global average pooling layers GAP1, GAP2, and GAP3 are used to integrate the data of C11 (256*256*16), C12 (128*128*16), and C13 (64*64*16) into one-dimensional data: 16, 16, and 16; (C9) superimposing the data of GAP1, i.e., 16, GAP2, i.e., 16, and GAP3, i.e., 16 to form superimposed data GAP4, i.e., 48; (C10) GAP4, i.e., 48 input, is used as a 48-bit fully connected layer, and the output is a 32-bit fully connected layer D1; (C11) Dropout layer DP1 is used to randomly change the weights of 50% of the hidden layer nodes to 0; (C12) Finally, the classification data of the flotation image is output through the fully connected layer D2 with an input of 32 and an output of 3.

2. The flotation foam image classification method integrating noise reduction coding and multi-scale deep learning according to claim 1 is characterized in that: The encoding stage has a total of 8 layers, and the texture part of the image is extracted through a combination of three pooling layers and convolutional layers: (A1) The first layer is the image input layer, which inputs a flotation image of size 256*256*3; (A2) The image size is converted from 256*256*3 to 256*256*32 through the convolution layer C1 with a convolution kernel of 32; (A3) Then pass through the maximum pooling layer P1 to make the image lose some information, extract the main texture of the image, and reduce its size to 128*128*32; (A4) Then the pooled data is adjusted through the convolution layer C2 with a convolution kernel of 32; (A5) Then pass through the maximum pooling layer P2, let the image lose information again, extract the texture, and reduce its size to 64*64*32; (A6) The pooled data is then adjusted through the convolution layer C3 with a convolution kernel of 32; (A7) Next, the final maximum pooling layer P3 is used to reduce its size to 32*32*32; (A8) Finally, the batch normalization layer B1 is used to prevent the network from experiencing gradient explosion, which can lead to failure in training, and to alleviate the overfitting phenomenon of the network.

3. The flotation foam image classification method integrating noise reduction coding and multi-scale deep learning according to claim 1 is characterized in that: The decoding stage is divided into 8 layers. The core information of the image is restored through a combination of three deconvolution layers + convolution layers, and finally the image is batch normalized: (B1) First, a deconvolution layer T1 with a stride of 2 is used to restore the image size from 32*32*32 to 64*64*32; (B2) A convolutional layer C4 is then used to adjust the deconvolution data, while also increasing the depth of the model; (B3) Through the second deconvolution layer T2 with a stride of 2, the image size is restored from 64*64*32 to 128*128*32; (B4) Cooperate with the second convolutional layer C5 to adjust the data again; (B5) Finally, the image size is restored from 128*128*32 to 256*256*32 through the third deconvolution layer T3 with a step size of 2; (B6) The data is then conditioned through the third convolutional layer C6; (B7) Then the batch normalization layer B2 is used to prevent the network from experiencing gradient explosion and thus making it impossible to train; (B8) Finally, through a convolutional layer C7 with 3 kernels, the image size is restored from 256*256*32 to the normal image size of 256*256*3.

Images