Method, device and equipment for detecting tailing ash content based on multi-channel neural network and storage medium

Abstract

The invention provides a method, device and equipment for detecting tailing ash content based on a multi-channel neural network and a storage medium, and the method comprises the steps: extracting features of a to-be-detected tailing image through a plurality of channels of a preset multi-channel neural network, and obtaining a plurality of features, the plurality of channels including a color channel, a visual perception channel and a chroma separation channel, the plurality of features comprise a color channel feature, a visual perception channel feature and a chroma separation channel feature; and fusing the plurality of features through a linear transformation layer of the multi-channel neural network to obtain an ash content detection result of the to-be-detected tailing image. Through the method, the effect of accurately detecting the ash content of the tailings can be achieved.

Description

Technical Field

[0001] The present application relates to the field of tailings detection, and in particular, to a method, device, equipment and storage medium for detecting tailings ash based on a multi-channel neural network. Background Art

[0002] Tailings ponds are places used to store tailings or other industrial waste discharged from mines after ore dressing during the mining process. They are essential infrastructure to maintain normal production of mines. Real-time detection of tailings ponds and determination of information such as the number, location, and area of ​​tailings ponds are important means to reduce the occurrence of tailings pond accidents and maintain the normal operation of mines.

[0003] Traditional tailings pond detection is mainly based on manual ground surveys. This method is greatly influenced by human subjectivity, has low detection efficiency and high costs, and cannot meet the needs of tailings pond detection. Existing deep learning algorithms can only detect tailings through simple image recognition when extracting tailings ponds, and there are still problems such as false detection and missed detection.

[0004] Therefore, how to accurately detect the ash content of tailings is a technical problem that needs to be solved. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide a method for detecting tailings ash content based on a multi-channel neural network. The technical solution of the embodiments of the present application can achieve the effect of accurately detecting tailings ash content.

[0006] In a first aspect, an embodiment of the present application provides a method for detecting tailings ash content based on a multi-channel neural network, comprising extracting features of a tailings image to be detected through multiple channels of a preset multi-channel neural network to obtain multiple features, wherein the multiple channels include: color channels, visual perception channels and chromaticity separation channels, and the multiple features include: color channel features, visual perception channel features and chromaticity separation channel features; fusing multiple features through a linear transformation layer of the multi-channel neural network to obtain an ash content detection result of the tailings image to be detected.

[0007] In the above-mentioned embodiments of the present application, feature extraction is performed on multi-source data through each channel of a multi-channel neural network, so that the color features, visual features and brightness features of the tailings image to be detected can be retained, and finally the image features obtained by multiple channels are fused to obtain the final ash detection result, thereby achieving the effect of accurately detecting the ash content of the tailings.

[0008] In some embodiments, before extracting features of the tailings image to be detected through multiple channels of a preset multi-channel neural network to obtain multiple features, it also includes: replacing the deep convolution in the existing ash detection network with a deep separable convolution to obtain a replaced ash detection network, wherein the deep separable convolution includes deep convolution and point-by-point convolution; adding a linear transformation layer to the replaced ash detection network to obtain a multi-channel neural network.

[0009] In the above embodiments of the present application, the depth convolution part can apply a convolution kernel to each input channel respectively without changing the number of channels, and the point-by-point convolution part can achieve the effects of channel fusion and channel number expansion.

[0010] In some embodiments, the deep convolution in the existing ash detection network is replaced with a deep separable convolution to obtain a replaced ash detection network, including: adding a convolution kernel to the input channel of the existing ash detection network through deep convolution to obtain an initial ash detection network; expanding the number of channels of the initial ash detection network through point-by-point convolution to obtain the replaced ash detection network.

[0011] In the above embodiments of the present application, the deep convolution part can apply a convolution kernel to each input channel respectively without changing the number of channels. The point-by-point convolution part can realize channel fusion and channel number expansion, and finally can realize the effect of multi-channel feature fusion when detecting tailings.

[0012] In some embodiments, features of the tailings image to be detected are extracted through multiple channels of a preset multi-channel neural network to obtain multiple features, including: extracting multiple initial features of the tailings image to be detected respectively through multiple channels corresponding to depthwise separable convolution; down-sampling the multiple initial features through the maximum pooling layer of the multi-channel neural network to obtain multiple features.

[0013] In the above-mentioned embodiment of the present application, multiple channels respectively extract different features of the tailings image to be detected, and finally downsample the multiple initial features through the maximum pooling layer, which can reduce the size of the feature map while retaining the most significant features, thereby reducing the amount of calculation and the risk of overfitting of the model.

[0014] In some embodiments, multiple features are fused through the linear transformation layer of a multi-channel neural network to obtain an ash detection result of the tailings image to be detected, including: fusing and linearly processing multiple features through the linear transformation layer of the multi-channel neural network to obtain an ash value; comparing the ash value with a preset ash value to obtain an ash detection result.

[0015] In the above embodiments of the present application, by fusing the features and comparing them with the conventional ash values, the ash content detection results can be accurately obtained.

[0016] In some embodiments, after fusing multiple features through the linear transformation layer of the multi-channel neural network to obtain the ash detection result of the tailings image to be detected, it also includes: adjusting the camera parameters of the camera and adjusting the dosage of the tailings treatment according to the ash detection result.

[0017] In the above embodiment of the present application, the camera parameters can be adjusted according to the ash detection results to find the most suitable monitoring parameters such as exposure, white balance, sensitivity, etc. Adjusting the dosage of tailings treatment can reduce the loss of clean coal and waste of resources.

[0018] In some embodiments, before extracting features of the tailings image to be detected through multiple channels of a preset multi-channel neural network to obtain multiple features, it also includes: collecting visible light tailings images through a high-definition camera; preprocessing the visible light tailings images to obtain processed tailings images; increasing the number of feature extraction channels of deep convolution in the existing ash detection network to obtain an increased ash detection network; training the increased ash detection network by annotating the processed tailings images and corresponding ash values ​​to obtain a multi-channel neural network.

[0019] In the above embodiments of the present application, the gray content detection network can be trained through the acquisition and preprocessing of high-definition images, and the obtained multi-channel neural network can recognize various features of the image, and finally perform feature fusion to achieve accurate gray content detection.

[0020] In a second aspect, an embodiment of the present application provides a device for detecting tailings ash content based on a multi-channel neural network, comprising:

[0021] A feature extraction module is used to extract features of the tailings image to be detected through multiple channels of a preset multi-channel neural network to obtain multiple features, wherein the multiple channels include: a color channel, a visual perception channel and a chromaticity separation channel, and the multiple features include: a color channel feature, a visual perception channel feature and a chromaticity separation channel feature;

[0022] The fusion module is used to fuse multiple features through the linear transformation layer of the multi-channel neural network to obtain the ash detection result of the tailings image to be detected.

[0023] Optionally, the device further comprises:

[0024] A construction module is used for the feature extraction module to replace the deep convolution in the existing ash detection network with a deep separable convolution before extracting the features of the tailings image to be detected through multiple channels of a preset multi-channel neural network to obtain multiple features, so as to obtain the replaced ash detection network, wherein the deep separable convolution includes deep convolution and point-by-point convolution;

[0025] Add a linear transformation layer to the replaced ash detection network to obtain a multi-channel neural network.

[0026] Optionally, the building blocks are specifically used to:

[0027] Add convolution kernels to the input channels of the existing ash detection network through deep convolution to obtain the initial ash detection network;

[0028] The number of channels of the initial ash detection network is expanded by point-by-point convolution to obtain the replaced ash detection network.

[0029] Optionally, the feature extraction module is specifically used for:

[0030] Through multiple channels corresponding to the depthwise separable convolution, multiple initial features of the tailings image to be detected are extracted respectively;

[0031] Through the maximum pooling layer of the multi-channel neural network, multiple initial features are downsampled to obtain multiple features.

[0032] Optionally, the building blocks are specifically used to:

[0033] Through the linear transformation layer of the multi-channel neural network, multiple features are fused and linearly processed to obtain the gray value;

[0034] Compare the ash value with the preset ash value to obtain the ash detection result.

[0035] Optionally, the device further comprises:

[0036] The adjustment module is used to adjust the camera parameters of the camera and the dosage of the tailings treatment according to the ash detection result after the fusion module fuses multiple features through the linear transformation layer of the multi-channel neural network to obtain the ash detection result of the tailings image to be detected.

[0037] Optionally, the device further comprises:

[0038] A training module, used for the feature extraction module to collect visible light tailings images through a high-definition camera before extracting features of the tailings image to be detected through multiple channels of a preset multi-channel neural network to obtain multiple features;

[0039] Preprocessing the visible light tailings image to obtain a processed tailings image;

[0040] Increasing the number of feature extraction channels of deep convolution in the existing ash detection network to obtain an increased ash detection network;

[0041] The ash content detection network after processing tailings images and corresponding ash content value annotation is trained to obtain a multi-channel neural network.

[0042] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor and a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps in the method provided in the first aspect are performed.

[0043] In a fourth aspect, an embodiment of the present application provides a readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps in the method provided in the first aspect are performed.

[0044] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by practicing the embodiments of the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1 A flowchart of a method for detecting tailings ash based on a multi-channel neural network provided in an embodiment of the present application;

[0047] Figure 2 A schematic diagram of the structure of a multi-channel neural network provided in an embodiment of the present application;

[0048] Figure 3 A flowchart of a method for training a multi-channel network model provided in an embodiment of the present application;

[0049] Figure 4 A schematic block diagram of a device for detecting tailings ash content based on a multi-channel neural network provided in an embodiment of the present application;

[0050] Figure 5 A schematic block diagram of the structure of a device for detecting tailings ash content based on a multi-channel neural network provided in an embodiment of the present application. DETAILED DESCRIPTION

[0051] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application usually described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0052] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0053] First, some terms involved in the embodiments of the present application are explained to facilitate understanding by those skilled in the art.

[0054] RGB (color channel) is the most intuitive color space, suitable for capturing the basic color information and brightness information of an image. It directly reflects the combination of the three basic colors of red, green and blue.

[0055] Lab (visual perception channel) is a color space based on human visual perception, which can well separate brightness information (L channel) and chromaticity information (a and b channels). Lab space is robust to lighting changes and is suitable for color correction and enhancement.

[0056] In the YCbCr (chroma separation channel) color space, the Y channel represents luminance, while the Cb and Cr channels represent chrominance. It can more effectively separate luminance and chrominance information, helping to reduce redundant information in the image.

[0057] The present application is applied to the scenario of tailings detection. The specific scenario is to extract features of different aspects of the tailings image to be detected through a multi-channel feature extraction network, and then use linear processing to fuse multiple features to achieve tailings detection.

[0058] Tailings ponds are places used to store tailings or other industrial waste discharged from mines after mineral processing during the mining process. They are necessary infrastructure to maintain normal mine production. Real-time detection of tailings ponds and determination of information such as the number, location, and area of ​​tailings ponds are important means to reduce accidents at tailings ponds and maintain normal mine operations. Traditional tailings pond detection is mainly based on manual ground surveys. This method is greatly affected by human subjectivity, has low detection efficiency and high cost, and cannot meet the needs of tailings pond detection. Existing deep learning algorithms can only detect tailings through simple image recognition when extracting tailings ponds, and there are still problems such as false detection and missed detection.

[0059] To this end, the present application extracts the features of the tailings image to be detected through multiple channels of a preset multi-channel neural network to obtain multiple features, wherein the multiple channels include: color channels, visual perception channels and chromaticity separation channels, and the multiple features include: color channel features, visual perception channel features and chromaticity separation channel features; the multiple features are fused through the linear transformation layer of the multi-channel neural network to obtain the ash content detection result of the tailings image to be detected. By extracting features from multi-source data through each channel of the multi-channel neural network, the color features, visual features and brightness features of the tailings image to be detected can be retained, and finally the image features obtained by the multiple channels are channel-fused to obtain the final ash content detection result, so as to achieve the effect of accurately detecting the ash content of the tailings.

[0060] In the embodiment of the present application, the execution subject can be a multi-channel neural network based tailings ash detection device in a multi-channel neural network based tailings ash detection system. In actual applications, the multi-channel neural network based tailings ash detection device can be an electronic device such as a terminal device and a server, and there is no limitation here.

[0061] Combine the following Figure 1 The method for detecting tailings ash content based on a multi-channel neural network in an embodiment of the present application is described in detail.

[0062] Please see Figure 1 , Figure 1 A flowchart of a method for detecting tailings ash based on a multi-channel neural network provided in an embodiment of the present application is shown in FIG. Figure 1 The method for detecting tailings ash based on a multi-channel neural network includes:

[0063] Step 110: extracting features of the tailings image to be detected through multiple channels of a preset multi-channel neural network to obtain multiple features.

[0064] The multiple channels include: color channel, visual perception channel and chromaticity separation channel, and the multiple features include: color channel feature, visual perception channel feature and chromaticity separation channel feature. The color channel feature represents the combination of three basic colors: red, green and blue, the visual perception channel feature includes visually perceived brightness feature and chromaticity feature, and the chromaticity separation channel feature includes separated brightness and chromaticity features.

[0065] In some embodiments of the present application, before extracting features of the tailings image to be detected through multiple channels of a preset multi-channel neural network to obtain multiple features, Figure 1 The method shown also includes: replacing the deep convolution in the existing ash detection network with a deep separable convolution to obtain a replaced ash detection network, wherein the deep separable convolution includes a deep convolution and a point-by-point convolution; adding a linear transformation layer to the replaced ash detection network to obtain a multi-channel neural network.

[0066] In the above process of the present application, the depth convolution part can apply a convolution kernel to each input channel respectively without changing the number of channels, and the point-by-point convolution part can achieve the effects of channel fusion and channel number expansion.

[0067] Among them, the convolution layer is mainly responsible for extracting local features of the input data, and the convolution kernel performs convolution operations on the input to generate feature maps. Multi-channel neural networks also include activation functions, pooling layers, and output layers. After each convolution layer, a nonlinear activation function, such as ReLU, is usually applied to introduce nonlinear factors so that the network can fit complex models. The pooling layer downsamples the feature map to reduce the size and amount of calculation of the data while retaining important features. Commonly used pooling methods include maximum pooling and average pooling. The output layer can be set appropriately according to task requirements, such as Softmax for multi-classification problems, Sigmod for binary classification, and linear activation for regression tasks. This application can use a linear function as the output layer.

[0068] In some embodiments of the present application, the deep convolution in the existing ash detection network is replaced with a deep separable convolution to obtain a replaced ash detection network, including: adding a convolution kernel to the input channel of the existing ash detection network through deep convolution to obtain an initial ash detection network; expanding the number of channels of the initial ash detection network through point-by-point convolution to obtain a replaced ash detection network.

[0069] In the above process, the deep convolution part of the present application can apply a convolution kernel to each input channel respectively without changing the number of channels. The point-by-point convolution part can realize channel fusion and channel number expansion, and finally can achieve the effect of multi-channel feature fusion when detecting tailings.

[0070] Specifically: the tailings image to be detected can be cropped to H×W×3. The specific cropping size can be set according to demand. Here, H×W×3 is taken as an example. The depth convolution part applies a convolution kernel to each input channel (R, G, B) respectively, without changing the number of channels, and the output feature map size is H×W×3; the point-by-point convolution part uses 16 groups of convolution kernels to fuse the three channels of each pixel, with the aim of expanding the number of channels to 16, and finally outputting H×W×16.

[0071] In some embodiments of the present application, before extracting features of the tailings image to be detected through multiple channels of a preset multi-channel neural network to obtain multiple features, Figure 1 The method shown also includes: collecting visible light tailings images through a high-definition camera; preprocessing the visible light tailings images to obtain processed tailings images; increasing the number of feature extraction channels of deep convolution in the existing ash detection network to obtain an increased ash detection network; training the increased ash detection network through the processed tailings images and corresponding ash value annotations to obtain a multi-channel neural network.

[0072] In the above process, the present application can train the gray content detection network through the collection and preprocessing of high-definition images. The obtained multi-channel neural network can recognize various features of the image and finally perform feature fusion to achieve accurate gray content detection.

[0073] Among them, preprocessing includes image size, image space transformation, image cropping, rotation, flipping, etc. Gray value annotation can be done by relevant personnel.

[0074] Optionally, the improved multi-channel neural network structure provided by the present application can be Figure 2 The multi-channel neural network structure schematic diagram shown is described in detail.

[0075] Please see Figure 2 , Figure 2 A schematic diagram of the structure of a multi-channel neural network provided for this application includes:

[0076] The color channel (RGB), visual perception channel (Lab) and chromaticity separation channel (YCbCr) are input with multi-source tailings image data, and pass through a 3×3 16-channel convolution layer (3×3DSConv16), a ReLU function, a 2×2 16-channel maximum pooling layer (2×2MaxPooling, 16), and a 3×3 32-channel convolution layer (3×3DSConv32), a ReLU function and a 2×2 32-channel maximum pooling layer (2×2MaxPooling, 32), and finally pass through the first linear transformation layer (Linear, 64) to output 64 eigenvalues ​​and the second linear transformation layer (Linear, 1) to output a gray value.

[0077] also, Figure 2 The structure and use of the multi-channel neural network shown can be Figure 1 The method shown will not be described in detail here.

[0078] Optionally, the multi-channel neural network training process can be found in Figure 3 The process shown.

[0079] Please see Figure 3 , Figure 3 A flowchart of a training method for a multi-channel network model provided in this application, such as Figure 3 The training method of the multi-channel network model shown includes:

[0080] Step 310: Data preparation.

[0081] Specifically: a visible light tailings image is collected by a high-definition camera; the visible light tailings image is preprocessed to obtain a processed tailings image.

[0082] Step 320: Model architecture design.

[0083] Specifically: the number of feature extraction channels of deep convolution in the existing ash detection network is increased to obtain an increased ash detection network.

[0084] Step 330: Training process.

[0085] Specifically: the ash detection network after processing the tailings image and the corresponding ash value annotation is trained to obtain a multi-channel neural network.

[0086] Step 340: Model evaluation and validation.

[0087] Specifically: Define the optimization algorithm through the loss function to select model training and data enhancement method for verification.

[0088] Among them, data enhancement uses center cropping and random image flipping; the learning rate can be set to 0.001, and an adaptive learning rate method is used to gradually reduce the learning rate during training to improve the stability and convergence speed of training; the optimization algorithm can use the adaptive moment estimate Adam; the number of training times can be set to 200; the loss function uses the mean square error MSE. By evaluating and verifying the multi-channel network model with these data and set values, the accuracy of feature extraction by the model can be improved.

[0089] Step 350: Reasoning process.

[0090] Specifically: Input the tailings image to be tested for model prediction and output the ash detection result.

[0091] also, Figure 3The detailed introduction of the multi-channel network model shown can be found in Figure 1 The use and training of the multi-channel neural network shown will not be described in detail here.

[0092] Optionally, capture visible tailings images with a high-definition camera A 2-megapixel high-definition camera can be used to capture visible RGB images. Each time a test is started, the system captures an image every 6 seconds over a two-minute period, for a total of 20 images for video analysis. This acquisition method ensures image clarity and timeliness, and helps to accurately analyze the ash content of coal mines.

[0093] In some embodiments of the present application, features of the tailings image to be detected are extracted through multiple channels of a preset multi-channel neural network to obtain multiple features, including: extracting multiple initial features of the tailings image to be detected through multiple channels corresponding to depthwise separable convolution; down-sampling the multiple initial features through the maximum pooling layer of the multi-channel neural network to obtain multiple features.

[0094] In the above process, the present application uses multiple channels to extract different features of the tailings image to be detected, and finally downsamples multiple initial features through the maximum pooling layer, which can reduce the size of the feature map while retaining the most significant features, thereby reducing the amount of calculation and the risk of overfitting of the model.

[0095] The initial features include initial brightness features, color features, chromaticity features, etc., and may also include image size and shape, etc.

[0096] Specifically, a 2×2 maximum pooling layer can be used. Convolution and pooling layers at different levels can perform channel fusion, channel expansion, and feature downsampling. The feature map is then flattened into a one-dimensional tensor through an image processing tool (flattening). After a linear transformation layer, for example, a 1×1 convolution can be used to output 64 values. At this time, the role of the 1×1 convolution here is to mix the information between feature channels and further increase the complexity of feature representation. Finally, 64 values ​​can be fused through features to obtain a gray value.

[0097] Step 120: Multiple features are fused through the linear transformation layer of the multi-channel neural network to obtain the ash detection result of the tailings image to be detected.

[0098] Among them, the ash test results include relevant data such as the tailings ash value and the clean coal content in the tailings.

[0099] In some embodiments of the present application, multiple features are fused through the linear transformation layer of a multi-channel neural network to obtain an ash detection result of the tailings image to be detected, including: fusing and linearly processing multiple features through the linear transformation layer of the multi-channel neural network to obtain an ash value; comparing the ash value with a preset ash value to obtain an ash detection result.

[0100] In the above process, the present application can accurately obtain the ash content detection result by fusing the features and comparing them with the conventional ash content value.

[0101] In some embodiments of the present application, after fusing multiple features through the linear transformation layer of the multi-channel neural network to obtain the ash detection result of the tailings image to be detected, Figure 1 The method shown also includes: adjusting the camera parameters of the camera and adjusting the dosage of the tailings treatment according to the ash detection result.

[0102] In the above process, the present application can adjust the camera parameters according to the ash detection results to find the most suitable monitoring parameters such as exposure, white balance, sensitivity, etc. Adjusting the dosage of tailings treatment can reduce the loss of clean coal and waste of resources.

[0103] Among them, the dosage is adjusted by the ash value. A higher tailings ash value may mean that more useful minerals have entered the tailings. It is necessary to reduce the medium density to increase the separation effect of coal and gangue and reduce the loss of clean coal.

[0104] In the above Figure 1 In the process shown, the present application extracts the features of the tailings image to be detected through multiple channels of a preset multi-channel neural network to obtain multiple features, wherein the multiple channels include: color channel, visual perception channel and chromaticity separation channel, and the multiple features include: color channel features, visual perception channel features and chromaticity separation channel features; multiple features are fused through the linear transformation layer of the multi-channel neural network to obtain the ash content detection result of the tailings image to be detected. By extracting features from multi-source data through each channel of the multi-channel neural network, the color features, visual features and brightness features of the tailings image to be detected can be retained, and finally the image features obtained by the multiple channels are channel-fused to obtain the final ash content detection result, thereby achieving the effect of accurately detecting the ash content of the tailings.

[0105] Optionally, the present application provides an image recognition system, including a camera and an industrial computer. The camera is used to collect visible light tailings images, and the industrial computer can be used as the control center of the entire tailings ash recognition system to execute Figure 1The method shown coordinates the work between the video analysis task and other automation equipment, and transmits the collected images to the industrial computer through the camera. The industrial computer performs video analysis, including: image preprocessing, feature extraction and model reasoning, and finally obtains the ash value. The factors affecting the coal preparation quality such as light intensity, camera parameters and added medium are adjusted based on the analysis results.

[0106] Previous article Figure 1 The method of detecting tailings ash content based on multi-channel neural network is described below. Figure 4-Figure 5 A device for detecting tailings ash content based on a multi-channel neural network is described.

[0107] Please refer to Figure 4 , is a schematic block diagram of a device 400 for detecting tailings ash based on a multi-channel neural network provided in an embodiment of the present application. The device 400 may be a module, program segment or code on an electronic device. The device 400 is similar to the above Figure 1 The method embodiment corresponds to and can be executed Figure 1 The various steps involved in the method embodiment and the specific functions of the device 400 can be found in the description below. To avoid repetition, the detailed description is appropriately omitted here.

[0108] Optionally, the device 400 includes:

[0109] The feature extraction module 410 is used to extract features of the tailings image to be detected through multiple channels of a preset multi-channel neural network to obtain multiple features, wherein the multiple channels include: a color channel, a visual perception channel and a chromaticity separation channel, and the multiple features include: a color channel feature, a visual perception channel feature and a chromaticity separation channel feature;

[0110] The fusion module 420 is used to fuse multiple features through the linear transformation layer of the multi-channel neural network to obtain the ash detection result of the tailings image to be detected.

[0111] Optionally, the device further comprises:

[0112] A construction module is used for the feature extraction module to extract the features of the tailings image to be detected through multiple channels of a preset multi-channel neural network to obtain multiple features, replace the deep convolution in the existing ash detection network with a deep separable convolution to obtain a replaced ash detection network, wherein the deep separable convolution includes deep convolution and point-by-point convolution; and add a linear transformation layer to the replaced ash detection network to obtain a multi-channel neural network.

[0113] Optionally, the building blocks are specifically used to:

[0114] By adding convolution kernels to the input channels of the existing ash detection network through deep convolution, an initial ash detection network is obtained; by expanding the number of channels of the initial ash detection network through point-by-point convolution, a replaced ash detection network is obtained.

[0115] Optionally, the feature extraction module is specifically used for:

[0116] Through multiple channels corresponding to the depthwise separable convolution, multiple initial features of the tailings image to be detected are extracted respectively; through the maximum pooling layer of the multi-channel neural network, the multiple initial features are downsampled to obtain multiple features.

[0117] Optionally, the building blocks are specifically used to:

[0118] The multiple features are fused and linearly processed through the linear transformation layer of the multi-channel neural network to obtain the ash value; the ash value is compared with the preset ash value to obtain the ash detection result.

[0119] Optionally, the device further comprises:

[0120] The adjustment module is used to adjust the camera parameters of the camera and the dosage of the tailings treatment according to the ash detection result after the fusion module fuses multiple features through the linear transformation layer of the multi-channel neural network to obtain the ash detection result of the tailings image to be detected.

[0121] Optionally, the device further comprises:

[0122] A training module, used for the feature extraction module to collect visible light tailings images through a high-definition camera before extracting features of the tailings image to be detected through multiple channels of a preset multi-channel neural network to obtain multiple features;

[0123] Preprocessing the visible light tailings image to obtain a processed tailings image;

[0124] The number of feature extraction channels of deep convolution in the existing ash detection network is increased to obtain an increased ash detection network; the increased ash detection network is trained by annotating the processed tailings images and corresponding ash values ​​to obtain a multi-channel neural network.

[0125] Please refer to Figure 5 The structure schematic block diagram of a device for detecting tailings ash content based on a multi-channel neural network provided in an embodiment of the present application, the device may include a memory 510 and a processor 520. Optionally, the device may also include: a communication interface 530 and a communication bus 540. The device and the above Figure 1 The method embodiment corresponds to and can be executed Figure 1 The various steps involved in the method embodiment and the specific functions of the device can be found in the description below.

[0126] Specifically, the memory 510 is used to store computer-readable instructions.

[0127] Processor 520, configured to process readable instructions stored in the memory, capable of executing Figure 1 The steps in the method.

[0128] The communication interface 530 is used for signaling or data communication with other node devices, for example, for communication with a server or a terminal, or for communication with other device nodes, but the embodiments of the present application are not limited thereto.

[0129] The communication bus 540 is used to realize direct connection and communication among the above components.

[0130] The device for detecting tailings ash content by multi-channel neural network can also include: a camera, two light sources, a black box environment and a light intensity sensor. Among them, the camera is connected to the industrial computer, the automatic white balance of the camera is turned off, the saturation is adjusted to restore the default value, the aperture, sensitivity, and shutter are recorded in real time and transmitted to the industrial computer; two light sources are placed opposite to each other, and two light sources with controllable light intensity are used to provide a stable lighting environment in the black box; the black box provides a dark environment to avoid the influence of external light pollution on image acquisition, so that the shooting light comes entirely from the two light sources, and the influence of the reflected light from the inner wall of the device on the tailings shooting is eliminated; the light intensity sensor is connected to the industrial computer, and the industrial computer controls the light intensity.

[0131] The communication interface 530 of the device in the embodiment of the present application is used to communicate signals or data with other node devices. The memory 510 can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. The memory 510 can also be at least one storage device located away from the aforementioned processor. The memory 510 stores computer-readable instructions. When the computer-readable instructions are executed by the processor 520, the electronic device executes the aforementioned Figure 1 The method process shown. The processor 520 can be used on the device 400 and used to perform the functions in the present application. Exemplarily, the above-mentioned processor 520 can be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, and the embodiments of the present application are not limited thereto.

[0132] The embodiment of the present application also provides a readable storage medium, when the computer program is executed by a processor, Figure 1 The method process in the method embodiment shown is executed by the electronic device.

[0133] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the device described above can refer to the corresponding process in the aforementioned method, and will not be described in detail here.

[0134] In summary, the embodiments of the present application provide a method, device, equipment and storage medium for detecting tailings ash content based on a multi-channel neural network. The method includes extracting features of a tailings image to be detected through multiple channels of a preset multi-channel neural network to obtain multiple features, wherein the multiple channels include: color channels, visual perception channels and chromaticity separation channels, and the multiple features include: color channel features, visual perception channel features and chromaticity separation channel features; and fusing multiple features through a linear transformation layer of a multi-channel neural network to obtain an ash content detection result of the tailings image to be detected. This method can achieve the effect of accurately detecting the ash content of tailings.

[0135] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0136] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0137] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0138] The above description is only an embodiment of the present application and is not intended to limit the scope of protection of the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0139] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0140] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

Claims

1. A method for detecting tailings ash content based on a multi-channel neural network, characterized in that: include: Extracting features of the tailings image to be detected through multiple channels of a preset multi-channel neural network to obtain multiple features, wherein the multiple channels include: color channels, visual perception channels and chromaticity separation channels, and the multiple features include: color channel features, visual perception channel features and chromaticity separation channel features; The plurality of features are fused through the linear transformation layer of the multi-channel neural network to obtain an ash detection result of the tailings image to be detected.

2. The method according to claim 1, characterized in that Before extracting the features of the tailings image to be detected through multiple channels of a preset multi-channel neural network to obtain multiple features, the method further includes: Replacing the depth convolution in the existing ash detection network with a depth-wise separable convolution to obtain a replaced ash detection network, wherein the depth-wise separable convolution includes a depth-wise convolution and a point-wise convolution; The replaced ash detection network linear transformation layer is added to obtain the multi-channel neural network.

3. The method according to claim 2, characterized in that The method replaces the deep convolution in the existing ash content detection network with a deep separable convolution to obtain the replaced ash content detection network, including: Adding a convolution kernel to the input channel of the existing ash detection network through the deep convolution to obtain an initial ash detection network; The number of channels of the initial ash detection network is expanded through the point-by-point convolution to obtain the replaced ash detection network.

4. The method according to claim 3, characterized in that The features of the tailings image to be detected are extracted through multiple channels of a preset multi-channel neural network to obtain multiple features, including: Extracting a plurality of initial features of the tailings image to be detected respectively through the plurality of channels corresponding to the depthwise separable convolution; The multiple initial features are downsampled through the maximum pooling layer of the multi-channel neural network to obtain the multiple features.

5. The method according to any one of claims 1 to 3, characterized in that: The step of fusing the multiple features through the linear transformation layer of the multi-channel neural network to obtain the ash content detection result of the tailings image to be detected includes: Performing fusion processing and linear processing on the multiple features through the linear transformation layer of the multi-channel neural network to obtain a gray value; The ash content value is compared with a preset ash content value to obtain the ash content detection result.

6. The method according to any one of claims 1 to 3, characterized in that: After fusing the multiple features through the linear transformation layer of the multi-channel neural network to obtain the ash detection result of the tailings image to be detected, the method further includes: According to the ash detection result, the camera parameters of the camera and the dosage of the tailings treatment are adjusted.

7. The method according to any one of claims 1 to 3, characterized in that: Before extracting the features of the tailings image to be detected through multiple channels of a preset multi-channel neural network to obtain multiple features, the method further includes: Visible light tailings images are collected through high-definition cameras; Preprocessing the visible light tailings image to obtain a processed tailings image; Increasing the number of feature extraction channels of deep convolution in the existing ash detection network to obtain an increased ash detection network; The multi-channel neural network is obtained by training the increased ash detection network through the processed tailings images and corresponding ash value annotations.

8. A device for detecting tailings ash content based on a multi-channel neural network, characterized in that: include: A feature extraction module is used to extract features of the tailings image to be detected through multiple channels of a preset multi-channel neural network to obtain multiple features, wherein the multiple channels include: color channels, visual perception channels and chromaticity separation channels, and the multiple features include: color channel features, visual perception channel features and chromaticity separation channel features; A fusion module is used to fuse the multiple features through the linear transformation layer of the multi-channel neural network to obtain the ash detection result of the tailings image to be detected.

9. An electronic device, characterized in that: include: A memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps in the method according to any one of claims 1 to 7 are executed.

10. A computer-readable storage medium, characterized in that: include: A computer program, when the computer program is run on a computer, causes the computer to execute the method according to any one of claims 1 to 7.

Images