Coal gangue detection method and device based on multi-feature layer fusion

Abstract

The present disclosure proposes a detection method and device for coal gangue based on multi-feature layer fusion, which relates to the field of coal technology, wherein the method comprises: obtaining a first image to be detected, wherein the first image is collected coal image data; inputting the first image into a trained image recognition model to determine the probability value corresponding to each pixel in the first image, wherein the image recognition model comprises multiple depth separable convolutional layers DSC and multiple network layers, and the network layer is used to fuse the features extracted by multiple DSCs; according to the probability value corresponding to each pixel, determining whether the first image contains coal gangue. Thus, after obtaining the coal image to be identified, the trained image recognition model can be used to process it to determine the probability value corresponding to each pixel in the image, and then according to the probability value, it can be determined whether the image contains coal gangue, and the detection of coal gangue can be realized without manual work, thereby improving efficiency and saving time.

Description

Technical Field

[0001] The present disclosure relates to the field of coal technology, and in particular to a detection method and device for coal gangue based on multi-feature layer fusion. Background Art

[0002] In the process of coal mining, the mined coal may be accompanied by coal gangue, which is a solid waste discharge with low carbon content, higher hardness than coal and lower combustion utilization rate. If the coal gangue is separated from the coal, it can not only improve the combustion efficiency of the coal, but also reduce the emission of pollutants during its combustion. Therefore, the rapid identification and detection of coal gangue is of great significance to the separation of coal gangue.

[0003] In the related technology, the detection of coal gangue is usually carried out manually, but the working environment is relatively bad, the labor intensity is high, it takes a lot of time and the efficiency is not high. Therefore, how to improve the efficiency of coal gangue detection is very important. Summary of the invention

[0004] The present disclosure proposes a coal gangue detection method and device based on multi-feature layer fusion, aiming to solve one of the technical problems in the related art at least to a certain extent.

[0005] The first embodiment of the present disclosure proposes a method for detecting coal gangue based on multi-feature layer fusion, including:

[0006] Acquire a first image to be detected, wherein the first image is collected coal image data;

[0007] Inputting the first image into a trained image recognition model to determine a probability value corresponding to each pixel in the first image, wherein the image recognition model comprises a plurality of depth-separable convolutional layers DSC and a plurality of network layers, and the network layers are used to fuse features extracted by the plurality of DSCs;

[0008] Whether the first image contains coal gangue is determined according to the probability value corresponding to each pixel point.

[0009] Optionally, before inputting the first image into the trained image recognition model to determine the probability value corresponding to each pixel in the first image, the method further includes:

[0010] Acquire a training data set, wherein the training data set includes a plurality of second images and a first label corresponding to each second image;

[0011] Inputting each second image into an initial model to determine a prediction probability value corresponding to each second image, wherein the initial model includes an encoder and a decoder;

[0012] Determining a prediction label corresponding to each second image according to the prediction probability value of each second image;

[0013] Determine a loss value corresponding to each second image according to a difference between the predicted label of each second image and the first label;

[0014] Based on each of the loss values, the initial model is modified to generate a trained image recognition model.

[0015] Optionally, inputting each second image into an initial model to determine a prediction probability value corresponding to each second image includes:

[0016] Inputting the second image into an encoder to be processed by a first convolutional layer in the encoder to determine a corresponding first encoding feature, wherein the encoder includes N convolutional layers;

[0017] Inputting the first encoded feature into a second convolutional layer in the encoder to determine a corresponding second encoded feature;

[0018] Input each encoded feature into the next convolutional layer to determine the encoded feature corresponding to the next convolutional layer;

[0019] Until the N-1th encoding feature corresponding to the N-1th convolutional layer is input into the Nth convolutional layer to determine the corresponding Nth encoding feature;

[0020] Inputting the Nth encoding feature and the N-1th encoding feature into a first network layer in a decoder to determine a corresponding first decoding feature;

[0021] Inputting the first decoding feature and the N-2th encoding feature into a second network layer in the decoder to determine a corresponding second decoding feature;

[0022] Inputting the decoding features corresponding to any network layer in the decoder and the encoding features corresponding to the decoding feature specifications into the next network layer in the decoder to determine the decoding features corresponding to the next network layer;

[0023] Until the N-1th decoding feature and the first encoding feature are input into the Nth network layer in the decoder to determine the prediction probability value corresponding to the second image.

[0024] Optionally, the modifying the initial model based on each of the loss values ​​to generate an image recognition model includes:

[0025] Based on each of the loss values, modifying the initial model;

[0026] Based on matrix low-rank decomposition, the modified model is compressed to generate an image recognition model.

[0027] Optionally, determining whether the first image contains gangue according to the probability value of each pixel point includes:

[0028] When the first probability value corresponding to any pixel point is greater than the second probability value, the image information corresponding to the any pixel point is determined to be coal gangue, wherein the first probability value is the probability that the pixel point is coal gangue, and the second probability value is the probability that the pixel point is not coal gangue.

[0029] The second embodiment of the present disclosure proposes a detection device for coal gangue based on multi-feature layer fusion, comprising:

[0030] An acquisition module, used for acquiring a first image to be detected, wherein the first image is collected coal image data;

[0031] A first determination module is used to input the first image into a trained image recognition model to determine a probability value corresponding to each pixel in the first image, wherein the image recognition model includes a plurality of depth-separable convolutional layers DSC and a plurality of network layers, and the network layer is used to fuse features extracted by the plurality of DSCs;

[0032] The second determination module is used to determine whether the first image contains coal gangue according to the probability value corresponding to each pixel point.

[0033] Optionally, the first determining module includes:

[0034] An acquisition unit, configured to acquire a training data set, wherein the training data set includes a plurality of second images and a first label corresponding to each second image;

[0035] A first determining unit, configured to input each of the second images into an initial model to determine a prediction probability value corresponding to each of the second images, wherein the initial model includes an encoder and a decoder;

[0036] A second determining unit, configured to determine a prediction label corresponding to each second image according to a prediction probability value of each second image;

[0037] A third determining unit, configured to determine a loss value corresponding to each second image according to a difference between the predicted label of each second image and the first label;

[0038] A generating unit is used to modify the initial model based on each loss value to generate a trained image recognition model.

[0039] Optionally, the first determining unit is specifically configured to:

[0040] Inputting the second image into an encoder to be processed by a first convolutional layer in the encoder to determine a corresponding first encoding feature, wherein the encoder includes N convolutional layers;

[0041] Inputting the first encoded feature into a second convolutional layer in the encoder to determine a corresponding second encoded feature;

[0042] Input each encoded feature into the next convolutional layer to determine the encoded feature corresponding to the next convolutional layer;

[0043] Until the N-1th encoding feature corresponding to the N-1th convolutional layer is input into the Nth convolutional layer to determine the corresponding Nth encoding feature;

[0044] Inputting the Nth encoding feature and the N-1th encoding feature into a first network layer in a decoder to determine a corresponding first decoding feature;

[0045] Inputting the first decoding feature and the N-2th encoding feature into a second network layer in the decoder to determine a corresponding second decoding feature;

[0046] Inputting the decoding features corresponding to any network layer in the decoder and the encoding features corresponding to the decoding feature specifications into the next network layer in the decoder to determine the decoding features corresponding to the next network layer;

[0047] Until the N-1th decoding feature and the first encoding feature are input into the Nth network layer in the decoder to determine the prediction probability value corresponding to the second image.

[0048] Optionally, the generating unit is specifically used for:

[0049] Based on each of the loss values, modifying the initial model;

[0050] Based on matrix low-rank decomposition, the modified model is compressed to generate an image recognition model.

[0051] Optionally, the second determining module is specifically configured to:

[0052] When the first probability value corresponding to any pixel point is greater than the second probability value, the image information corresponding to the any pixel point is determined to be coal gangue, wherein the first probability value is the probability that the pixel point is coal gangue, and the second probability value is the probability that the pixel point is not coal gangue.

[0053] The detection method and device of coal gangue based on multi-feature layer fusion provided by the present disclosure can first obtain a first image to be detected, wherein the first image is the collected coal image data, and then the first image can be input into a trained image recognition model to determine the probability value corresponding to each pixel in the first image, and then determine whether the first image contains coal gangue according to the probability value corresponding to each pixel. Thus, after obtaining the coal image to be identified, the trained image recognition model can be used to process it to determine the probability value corresponding to each pixel in the image, and then determine whether the image contains coal gangue according to the probability value, and the detection of coal gangue can be realized without manual work, thereby improving efficiency and saving time.

[0054] Additional aspects and advantages of the present disclosure will be given in part in the following description and in part will be obvious from the following description or learned through practice of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The above and / or additional aspects and advantages of the present disclosure will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0056] Figure 1 It is a flow chart of a method for detecting coal gangue based on multi-feature layer fusion provided according to an embodiment of the present disclosure;

[0057] Figure 2 is a flow chart of a method for detecting coal gangue based on multi-feature layer fusion provided according to another embodiment of the present disclosure;

[0058] Figure 2A is a schematic diagram of the structure of an initial model provided according to an embodiment of the present disclosure;

[0059] Figure 3 It is a schematic diagram of a coal gangue detection device based on multi-feature layer fusion provided according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0060] Embodiments of the present disclosure are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present disclosure, and are not to be construed as limitations of the present disclosure. On the contrary, the embodiments of the present disclosure include all changes, modifications, and equivalents that fall within the spirit and connotation of the appended claims.

[0061] It should be noted that the executor of the coal gangue detection method based on multi-feature layer fusion of this embodiment may be a coal gangue detection device based on multi-feature layer fusion, which may be implemented by software and / or hardware, and which may be configured in an electronic device, which may include but is not limited to a terminal, a server, etc.

[0062] Figure 1 is a flow chart of a method for detecting coal gangue based on multi-feature layer fusion according to an embodiment of the present disclosure, such as Figure 1 As shown, the method includes:

[0063] Step 101: Acquire a first image to be detected, wherein the first image is collected coal image data.

[0064] It is understandable that in the process of coal mining, the mined coal may be accompanied by coal gangue, which is a solid emission waste with a low carbon content, higher hardness than coal, and lower combustion utilization rate. If the coal gangue is separated from the coal, it can not only improve the combustion efficiency of the coal, but also reduce the emission of pollutants during its combustion. Therefore, in the embodiment of the present disclosure, the collected coal image data to be identified can be obtained first.

[0065] Among them, the first image can be coal image data collected by a camera, or it can also be coal image data determined by analyzing a video, or it can also be coal image data collected by other means, etc., and the present disclosure does not limit this.

[0066] Step 102, input the first image into the trained image recognition model to determine the probability value corresponding to each pixel in the first image, wherein the image recognition model includes multiple deep separable convolutional layers DSC and multiple network layers, and the network layer is used to fuse the features extracted by multiple DSCs.

[0067] Among them, the image recognition model can be trained in advance, and the first image to be detected is input into the image recognition model. After being processed by the image recognition model, the probability value corresponding to each pixel point in the first image can be determined.

[0068] It is understandable that the probability value may be a probability for characterizing that a pixel point is "coal gangue", or may be a probability value for characterizing that a pixel point is not "coal gangue", etc. Alternatively, the probability value may be multiple, such as a first probability value for the probability that the pixel point is "coal gangue", a second probability value for the probability that the pixel point is "not coal gangue", etc., and the present disclosure does not limit this.

[0069] In addition, the image recognition model can include multiple depthwise separable convolution (DSC) layers and multiple network layers.

[0070] Among them, DSC can be used to extract features from the input first image, and the network layer can be used to fuse the features extracted by multiple DSCs, that is, to fuse multiple feature layers.

[0071] Optionally, the first image can be input into an image recognition model to extract a first feature after being processed by a first DSC, and then the first coded feature can be input into a next DSC to obtain its corresponding feature, that is, the feature depths extracted by different DSCs may also be different. The network layer can fuse the features corresponding to different DSCs to obtain more complete and comprehensive feature information, thereby improving the comprehensiveness of the receptive field, ensuring the effectiveness of feature extraction, etc. The present disclosure does not limit this.

[0072] It can be understood that, in the present disclosure, the depthwise separable convolution technology can make the parameters required for detecting the first image much smaller than the parameters required by the traditional convolution method, thereby achieving the purpose of lightweight model.

[0073] Step 103: Determine whether the first image contains coal gangue according to the probability value corresponding to each pixel point.

[0074] For example, if the probability value corresponding to a pixel point is used to characterize the probability that the pixel point is "coal gangue", then a probability threshold can be set in advance, and the probability value corresponding to each pixel point can be compared with the probability threshold. If the probability value of a certain pixel point is greater than the probability threshold, the pixel point can be considered to be "coal gangue"; or if the probability value of a certain pixel point is less than or equal to the probability threshold, the pixel point can be considered to be "not coal gangue", etc., and the present disclosure does not limit this.

[0075] Optionally, when the first probability value corresponding to any pixel point is greater than the second probability value, it can be determined that the image information corresponding to any pixel point is coal gangue, wherein the first probability value is the probability that the pixel point is coal gangue, and the second probability value is the probability that the pixel point is not coal gangue.

[0076] It can be understood that the first probability value and the second probability value can be compared, and the corresponding image information can be determined according to the larger probability value.

[0077] For example, if the first probability value corresponding to any pixel is 0.95 and the second probability value is 0.05, then it can be determined that the image information corresponding to the pixel is "coal gangue". Or, if the first probability value corresponding to a pixel is 0.3 and the second probability value is 0.7, then it can be determined that the image information corresponding to the pixel is "not coal gangue". And so on, the present disclosure does not limit this.

[0078] Therefore, in the embodiment of the present disclosure, the collected coal images are processed using the trained image recognition model to determine the probability value corresponding to each pixel in the image. Then, based on the probability value, it can be determined whether the image contains gangue, the location of the gangue in the image, etc., without the need for manual identification of gangue, which improves efficiency and saves time.

[0079] In the disclosed embodiment, the first image to be detected can be first obtained, wherein the first image is the collected coal image data, and then the first image can be input into the trained image recognition model to determine the probability value corresponding to each pixel in the first image, and then determine whether the first image contains coal gangue according to the probability value corresponding to each pixel. Thus, after obtaining the coal image to be identified, the trained image recognition model can be used to process it to determine the probability value corresponding to each pixel in the image, and then determine whether the image contains coal gangue according to the probability value, and the detection of coal gangue can be realized without manual work, thereby improving efficiency and saving time.

[0080] Figure 2 is a flow chart of a method for detecting coal gangue based on multi-feature layer fusion according to an embodiment of the present disclosure, such as Figure 2 As shown, the method includes:

[0081] Step 201: Obtain a training data set, wherein the training data set includes a plurality of second images and a first label corresponding to each second image.

[0082] Among them, the second image is a manually annotated coal image; the first label can be a classification label after the second image is manually annotated, such as "coal gangue" or "not coal gangue", etc., which is not limited in the present disclosure.

[0083] Step 202: input each second image into an initial model to determine a prediction probability value corresponding to each second image, wherein the initial model includes an encoder and a decoder.

[0084] The initial model may be any model including an encoder and a decoder, which may be used to process the second image to determine the predicted probability value corresponding to the second image. The encoder may be used to perform feature extraction and other processing on the second image; the decoder may be used to fuse the extracted features to generate the predicted probability value corresponding to the second image, etc., which is not limited in the present disclosure.

[0085] Optionally, the predicted probability value processed by the initial model may also be the predicted probability value corresponding to each pixel point in the second image, etc., which is not limited in the present disclosure.

[0086] Optionally, the second image may be input into the encoder to be processed by the first convolutional layer in the encoder to determine the corresponding first coding feature, wherein the encoder includes N convolutional layers. The first feature may then be input into the second convolutional layer in the encoder to determine the corresponding second coding feature; each coding feature may then be input into the next convolutional layer to determine the coding feature corresponding to the next convolutional layer; until the N-1th coding feature corresponding to the N-1th convolutional layer is input into the Nth convolutional layer to determine the corresponding Nth coding feature.

[0087] The encoder may include multiple convolutional layers, and the number N may be set in advance, such as 5, 6, 7, etc., which is not limited in the present disclosure.

[0088] In addition, the convolution layer may be a depthwise separable convolution layer (DSC), or may be other convolution layers, etc. In addition, multiple convolution layers may be the same, or may be different, etc., and the present disclosure does not limit this.

[0089] For example, when the value of N is 6 and the convolution layer is DSC, the encoder processes the second image as follows: Figure 2A As shown. Figure 2AIt can be seen that the second image A is input into the encoder to generate the corresponding first coding feature F1 after being processed by the first DSC in the encoder; then the first coding feature F1 is input into the second DSC to generate the corresponding second coding feature F2 after being processed by the second DSC; then the second coding feature F2 is input into the third DSC to generate the corresponding third coding feature F3 after being processed by the third DSC; then the third coding feature F3 is input into the fourth DSC to generate the corresponding fourth coding feature F4 after being processed by the fourth DSC; then the fourth coding feature F4 is input into the fifth DSC to generate the corresponding fifth coding feature F5 after being processed by the fifth DSC; then the fifth coding feature F5 is input into the sixth DSC to generate the corresponding sixth coding feature F6 after being processed by the sixth DSC, and the present disclosure does not limit this.

[0090] Optionally, the Nth encoding feature and the N-1th encoding feature can be input into the first network layer in the decoder to determine the corresponding first decoding feature, and then the first decoding feature and the N-2th encoding feature can be input into the second network layer in the decoder to determine the corresponding second decoding feature; then the decoding features corresponding to any network layer in the decoder and the encoding features corresponding to the decoding feature specifications can be input into the next network layer in the decoder to determine the decoding features corresponding to the next network layer; until the N-1th decoding feature and the first encoding feature are input into the Nth network layer in the decoder to determine the predicted probability value corresponding to the second image.

[0091] Optionally, the decoder may fuse the coding features by upsampling to generate corresponding prediction probabilities, etc., which is not limited in the present disclosure.

[0092] In addition, the number of network layers in the decoder may correspond to the number of convolutional layers in the encoder, so that a decoding feature corresponding to any network layer of the decoder has an encoding feature corresponding to a decoding feature specification in the encoder, etc. The specification may be a size, a dimension, etc. This disclosure does not limit this.

[0093] Optionally, the Nth encoding feature can be upsampled first, and then it and the N-1th encoding feature are input into the first network layer in the decoder, so that after processing by the first network layer, such as convolution processing, etc., the corresponding first decoding feature M1, etc. can be determined. The present disclosure does not limit this.

[0094] For example, when the encoder includes 6 DSCs, the process of fusing the encoding features obtained by the encoder can be as follows: Figure 2A As shown. Figure 2AIt can be seen that the sixth encoding feature F6 and the fifth encoding feature F5 obtained by the encoder can be first input into the decoder, so as to be processed by the first network layer in the decoder to determine the first decoding feature M1 corresponding to the first network layer. Among them, the sixth encoding feature F6 can be upsampled first, and then it and the fifth encoding feature F5 are input into the first network layer, so as to be processed by the convolution of the first network to obtain the corresponding first decoding feature M1.

[0095] Afterwards, the first decoding feature M1 can be upsampled first, and then it and the fourth encoding feature F4 can be input into the second network layer in the decoder, so that after the convolution processing of the second network layer, the corresponding second decoding feature M2 can be determined; then the second decoding feature M2 and the third encoding feature F3 can be input into the third network layer in the decoder, so that after the processing of the third network layer, the corresponding third decoding feature M3 can be determined; then the third decoding feature M3 and the second encoding feature F2 can be input into the fourth network layer in the decoder, so that after the processing of the fourth network layer, the corresponding fourth decoding feature M4 can be determined; then the fourth decoding feature M4 and the first encoding feature F1 can be input into the fifth network layer in the decoder, so that after the processing of the fifth network layer, the corresponding fifth decoding feature M5 can be determined, and then the fifth decoding feature M5 can be upsampled to determine the prediction probability value corresponding to B.

[0096] Optionally, when any decoding feature is input into a network layer in a decoder, it can be upsampled first and then input into the corresponding network layer, which is not limited in the present disclosure.

[0097] It should be noted that the above examples are merely illustrative and cannot be used as limitations on the encoder and decoder structures in the embodiments of the present disclosure.

[0098] Therefore, in the disclosed embodiment, the parameters required for processing the second image can be much smaller than those required by the traditional convolution method through the depth-separable convolution technology, thereby achieving the purpose of lightweight model. In addition, by using the DSC in the encoder, coding features of different depths can be generated, and then each network layer in the decoder can be used to fuse the coding features of different depths, so as to achieve a more comprehensive receptive field, thereby improving the effectiveness and reliability of feature extraction.

[0099] Step 203: Determine a prediction label corresponding to each second image according to the prediction probability value of each second image.

[0100] Among them, the predicted probability value can be used to characterize whether the second image contains coal gangue. If the second image contains coal gangue, it can be determined that its corresponding predicted label can be "containing coal gangue"; if the second image does not contain coal gangue, then it can be determined that its corresponding predicted label can be "not coal gangue", etc., and the present disclosure does not limit this.

[0101] Step 204, determining a loss value corresponding to each second image according to the difference between the predicted label of each second image and the first label.

[0102] There are many ways to determine the difference between the predicted label of the second image and the first label. For example, the Manhattan distance formula can be used to determine the difference between the predicted label of the second image and the first label, or the Euclidean distance formula can be used to determine the difference between the predicted label of the second image and the first label. It is understandable that the above method of determining the difference between the predicted label of the second image and the first label is not limited to the Manhattan distance formula, the Euclidean distance formula, etc., and the present disclosure does not limit this.

[0103] Optionally, after determining the difference between the predicted label of the second image and the first label, a loss function may be used to determine a loss value corresponding to each second image.

[0104] The loss function may be of any type, such as a cross entropy loss function, a mean square error loss function, etc. This disclosure does not limit this.

[0105] Step 205, based on each loss value, the initial model is modified to generate a trained image recognition model.

[0106] It is understandable that after the loss value of the second image is determined, the loss value can be used to reversely correct the initial model layer by layer, thereby generating a trained image recognition model. The present disclosure does not limit this.

[0107] It is understandable that in order to further achieve the purpose of model lightweighting, the trained model can also be compressed to facilitate its deployment in edge devices, so that coal gangue can be detected based on the edge, thereby further improving the efficiency of detection.

[0108] Optionally, the initial model may be modified according to each loss value, and then the modified model may be compressed based on matrix low-rank decomposition to generate a trained image recognition model.

[0109] The modified model may also be compressed in any desirable manner to generate an image recognition model, and the present disclosure does not limit this.

[0110] Therefore, in the embodiment of the present disclosure, after generating a trained image recognition model, the image recognition model can be used to process the image to be detected to determine whether the image contains "coal gangue". Due to the encoder and decoder included in the image recognition model, the validity and accuracy of the extracted features are fully considered during the image processing, thereby providing conditions for improving the accuracy and efficiency of coal gangue detection.

[0111] In the disclosed embodiment, a training data set may be first obtained, wherein the training data set includes a plurality of second images and a first label corresponding to each second image, and then each second image may be input into the initial model to determine the predicted probability value corresponding to each second image, wherein the initial model includes an encoder and a decoder, and then the predicted label corresponding to each second image is determined according to the predicted probability value of each second image, and then the loss value corresponding to each second image may be determined according to the difference between the predicted label of each second image and the first label, and based on each loss value, the initial model may be modified to generate an image recognition model. Thus, by processing the second image using the encoder and the decoder, the predicted probability value of the second image may be obtained, and then the predicted label may be obtained, and the initial model may be modified according to the difference between the predicted label and the first label to generate a trained image recognition model, which provides conditions for improving the accuracy and efficiency of coal gangue detection.

[0112] In order to implement the above-mentioned embodiment, the present disclosure also proposes a detection device for coal gangue based on the fusion of multiple feature layers.

[0113] Figure 3 It is a schematic diagram of a coal gangue detection device based on multi-feature layer fusion provided according to an embodiment of the present disclosure.

[0114] like Figure 3 As shown, the detection device 300 for coal gangue based on multi-feature layer fusion includes: an acquisition module 310 , a first determination module 320 and a second determination module 330 .

[0115] The acquisition module 310 is used to acquire a first image to be detected, wherein the first image is collected coal image data.

[0116] The first determination module 320 is used to input the first image into a trained image recognition model to determine the probability value corresponding to each pixel in the first image, wherein the image recognition model includes multiple deep separable convolutional layers DSC and multiple network layers, and the network layer is used to fuse the features extracted by multiple DSCs.

[0117] The second determination module 330 is used to determine whether the first image contains coal gangue according to the probability value corresponding to each pixel point.

[0118] Optionally, the first determining module 320 includes:

[0119] An acquisition unit, configured to acquire a training data set, wherein the training data set includes a plurality of second images and a first label corresponding to each second image;

[0120] A first determining unit, configured to input each of the second images into an initial model to determine a prediction probability value corresponding to each of the second images, wherein the initial model includes an encoder and a decoder;

[0121] A second determining unit, configured to determine a prediction label corresponding to each second image according to a prediction probability value of each second image;

[0122] A third determining unit, configured to determine a loss value corresponding to each second image according to a difference between the predicted label of each second image and the first label;

[0123] A generating unit is used to modify the initial model based on each loss value to generate a trained image recognition model.

[0124] Optionally, the first determining unit is specifically configured to:

[0125] Inputting the second image into an encoder to be processed by a first convolutional layer in the encoder to determine a corresponding first encoding feature, wherein the encoder includes N convolutional layers;

[0126] Inputting the first encoded feature into a second convolutional layer in the encoder to determine a corresponding second encoded feature;

[0127] Input each encoded feature into the next convolutional layer to determine the encoded feature corresponding to the next convolutional layer;

[0128] Until the N-1th encoding feature corresponding to the N-1th convolutional layer is input into the Nth convolutional layer to determine the corresponding Nth encoding feature;

[0129] Inputting the Nth encoding feature and the N-1th encoding feature into a first network layer in a decoder to determine a corresponding first decoding feature;

[0130] Inputting the first decoding feature and the N-2th encoding feature into a second network layer in the decoder to determine a corresponding second decoding feature;

[0131] Inputting the decoding features corresponding to any network layer in the decoder and the encoding features corresponding to the decoding feature specifications into the next network layer in the decoder to determine the decoding features corresponding to the next network layer;

[0132] Until the N-1th decoding feature and the first encoding feature are input into the Nth network layer in the decoder to determine the prediction probability value corresponding to the second image.

[0133] Optionally, the generating unit is specifically used for:

[0134] Based on each of the loss values, modifying the initial model;

[0135] Based on matrix low-rank decomposition, the modified model is compressed to generate an image recognition model.

[0136] Optionally, the second determining module 330 is specifically configured to:

[0137] When the first probability value corresponding to any pixel point is greater than the second probability value, the image information corresponding to the any pixel point is determined to be coal gangue, wherein the first probability value is the probability that the pixel point is coal gangue, and the second probability value is the probability that the pixel point is not coal gangue.

[0138] The functions and specific implementation principles of the above modules in the embodiments of the present disclosure can be referred to the above method embodiments, and will not be repeated here.

[0139] The detection device for coal gangue based on multi-feature layer fusion of the disclosed embodiment can first obtain a first image to be detected, wherein the first image is collected coal image data, and then the first image can be input into a trained image recognition model to determine the probability value corresponding to each pixel in the first image, and then determine whether the first image contains coal gangue based on the probability value corresponding to each pixel. Thus, after obtaining the coal image to be identified, the trained image recognition model can be used to process it to determine the probability value corresponding to each pixel in the image, and then determine whether the image contains coal gangue based on the probability value, and the detection of coal gangue can be achieved without manual work, thereby improving efficiency and saving time.

[0140] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0141] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

[0142] It should be noted that, in the description of the present disclosure, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of the present disclosure, unless otherwise specified, the meaning of "plurality" is two or more.

[0143] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present disclosure includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present disclosure belong.

[0144] It should be understood that the various parts of the present disclosure may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof may be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0145] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0146] In addition, each functional unit in each embodiment of the present disclosure may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0147] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0148] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0149] Although the embodiments of the present disclosure have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present disclosure. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present disclosure.

Claims

1. A detection method for coal gangue based on multi-feature layer fusion, characterized in that: include: Acquire a first image to be detected, wherein the first image is collected coal image data; Inputting the first image into a trained image recognition model to determine a probability value corresponding to each pixel in the first image, wherein the image recognition model comprises a plurality of depth-separable convolutional layers DSC and a plurality of network layers, and the network layers are used to fuse features extracted by the plurality of DSCs; Determining whether the first image contains coal gangue according to the probability value corresponding to each pixel point; Before inputting the first image into the trained image recognition model to determine the probability value corresponding to each pixel in the first image, the method further includes: Acquire a training data set, wherein the training data set includes a plurality of second images and a first label corresponding to each second image; Inputting each second image into an initial model to determine a prediction probability value corresponding to each second image, wherein the initial model includes an encoder and a decoder; Determining a prediction label corresponding to each second image according to the prediction probability value of each second image; Determine a loss value corresponding to each second image according to a difference between the predicted label of each second image and the first label; Based on each of the loss values, the initial model is modified to generate a trained image recognition model; Inputting each of the second images into an initial model to determine a predicted probability value corresponding to each of the second images includes: Inputting the second image into an encoder to be processed by a first convolutional layer in the encoder to determine a corresponding first encoding feature, wherein the encoder includes N convolutional layers; Inputting the first encoded feature into a second convolutional layer in the encoder to determine a corresponding second encoded feature; Input each encoded feature into the next convolutional layer to determine the encoded feature corresponding to the next convolutional layer; Until the N-1th encoding feature corresponding to the N-1th convolutional layer is input into the Nth convolutional layer to determine the corresponding Nth encoding feature; Inputting the Nth encoding feature and the N-1th encoding feature into a first network layer in a decoder to determine a corresponding first decoding feature; Inputting the first decoding feature and the N-2th encoding feature into a second network layer in the decoder to determine a corresponding second decoding feature; Inputting the decoding features corresponding to any network layer in the decoder and the encoding features corresponding to the decoding feature specifications into the next network layer in the decoder to determine the decoding features corresponding to the next network layer; Until the N-1th decoding feature and the first encoding feature are input into the Nth network layer in the decoder to determine the prediction probability value corresponding to the second image.

2. The method according to claim 1, characterized in that The initial model is modified based on each loss value to generate a trained image recognition model, including: Based on each of the loss values, modifying the initial model; Based on matrix low-rank decomposition, the modified model is compressed to generate an image recognition model.

3. The method according to claim 1 or 2, characterized in that The determining, according to the probability value corresponding to each pixel point, whether the first image contains coal gangue includes: When the first probability value corresponding to any pixel point is greater than the second probability value, the image information corresponding to the any pixel point is determined to be coal gangue, wherein the first probability value is the probability that the pixel point is coal gangue, and the second probability value is the probability that the pixel point is not coal gangue.

4. A detection device for coal gangue based on multi-feature layer fusion, characterized in that: include: An acquisition module, used for acquiring a first image to be detected, wherein the first image is collected coal image data; A first determination module is used to input the first image into a trained image recognition model to determine a probability value corresponding to each pixel in the first image, wherein the image recognition model includes a plurality of depth-separable convolutional layers DSC and a plurality of network layers, and the network layer is used to fuse features extracted by the plurality of DSCs; A second determination module, configured to determine whether the first image contains coal gangue according to a probability value corresponding to each pixel point; The first determining module includes: An acquisition unit, configured to acquire a training data set, wherein the training data set includes a plurality of second images and a first label corresponding to each second image; A first determining unit, configured to input each of the second images into an initial model to determine a prediction probability value corresponding to each of the second images, wherein the initial model includes an encoder and a decoder; A second determining unit, configured to determine a prediction label corresponding to each second image according to the prediction probability value of each second image; A third determining unit, configured to determine a loss value corresponding to each second image according to a difference between the predicted label of each second image and the first label; A generating unit, configured to modify the initial model based on each of the loss values ​​to generate a trained image recognition model; The first determining unit is specifically configured to: Inputting the second image into an encoder to be processed by a first convolutional layer in the encoder to determine a corresponding first encoding feature, wherein the encoder includes N convolutional layers; Inputting the first encoded feature into a second convolutional layer in the encoder to determine a corresponding second encoded feature; Input each encoded feature into the next convolutional layer to determine the encoded feature corresponding to the next convolutional layer; Until the N-1th encoding feature corresponding to the N-1th convolutional layer is input into the Nth convolutional layer to determine the corresponding Nth encoding feature; Inputting the Nth encoding feature and the N-1th encoding feature into a first network layer in a decoder to determine a corresponding first decoding feature; Inputting the first decoding feature and the N-2th encoding feature into a second network layer in the decoder to determine a corresponding second decoding feature; Inputting the decoding features corresponding to any network layer in the decoder and the encoding features corresponding to the decoding feature specifications into the next network layer in the decoder to determine the decoding features corresponding to the next network layer; Until the N-1th decoding feature and the first encoding feature are input into the Nth network layer in the decoder to determine the prediction probability value corresponding to the second image.

5. The device according to claim 4, characterized in that The generating unit is specifically used for: Based on each of the loss values, modifying the initial model; Based on matrix low-rank decomposition, the modified model is compressed to generate an image recognition model.

6. The device according to claim 4 or 5, characterized in that The second determining module is specifically used to: When the first probability value corresponding to any pixel point is greater than the second probability value, the image information corresponding to the any pixel point is determined to be coal gangue, wherein the first probability value is the probability that the pixel point is coal gangue, and the second probability value is the probability that the pixel point is not coal gangue.

Images