Fine coal ash content detection method and system based on image recognition

Abstract

The invention relates to the technical field of coal engineering, in particular to a pulverized coal ash content detection method and system based on image recognition, and the method comprises the following steps: collecting a coal sample and preprocessing the coal sample; coal sample target images corresponding to different modes are obtained for the preprocessed coal sample; a trained detection model is obtained, the detection model is used for outputting a gray content predicted value and a gray content category based on an input image, the detection model comprises a feature adjustment module, a feature extraction module, a feature fusion module and an analysis output module which are connected in sequence, and the feature extraction module comprises a plurality of convolutional layer modules and a Transform self-attention layer module; each module in the feature extraction module is connected with the feature fusion module, and the analysis output module is used for outputting an ash content predicted value and an ash content category; and inputting the coal sample target images corresponding to different modes into the trained detection model to obtain ash content predicted values and ash content categories of the collected coal samples. By utilizing the method disclosed by the invention, the ash content of the slack coal product can be detected more quickly and accurately.

Description

Technical Field

[0001] The present invention relates to the field of coal engineering technology, and in particular to a method and system for detecting fine coal ash based on image recognition. Background Art

[0002] In the process of coal production and processing, the ash content of coal samples is an important indicator for evaluating coal quality. Traditional ash determination methods usually rely on laboratory chemical analysis, which is time-consuming, cumbersome, and requires manual intervention. This traditional ash detection method is not only inefficient, but also easily affected by human operation, resulting in unstable measurement results and unable to meet the growing demand of the modern coal industry for real-time, online detection. Existing automated detection equipment for fine coal ash is mostly based on X-ray detection, which has potential radioactive hazards and is gradually not recognized by the market.

[0003] In recent years, intelligent detection methods based on image processing and deep learning have been gradually applied in industrial production. However, most existing research focuses on the processing and detection of foam images in the mineral flotation process, while there is still a research gap in processing fine coal images to achieve ash content prediction of fine coal products. Summary of the invention

[0004] The present invention aims to solve one of the technical problems in the related art at least to a certain extent.

[0005] To this end, the first purpose of the present invention is to propose a method for detecting the ash content of fine coal based on image recognition, so as to solve the problem that the ash content detection of fine coal products is currently time-consuming and labor-intensive.

[0006] The second object of the present invention is to provide a fine coal ash detection system based on image recognition.

[0007] To achieve the above-mentioned purpose, the first aspect of the present invention proposes a method for detecting the ash content of fine coal based on image recognition, comprising:

[0008] Collecting coal samples and pre-treating the coal samples;

[0009] Perform multi-modal image acquisition on the pre-processed coal sample to obtain target images of the coal sample corresponding to different modalities;

[0010] Acquire a trained detection model, the detection model is used to output a gray prediction value and a gray category based on an input image, the detection model includes a feature adjustment module, a feature extraction module, a feature fusion module and an analysis output module connected in sequence, the feature extraction module includes a plurality of convolutional layer modules and a Transformer self-attention layer module, each module in the feature extraction module is connected to the feature fusion module, the analysis output module includes a first output submodule and a second output submodule, the first output submodule is used to output a gray prediction value, and the second output submodule is used to output a gray category;

[0011] The coal sample target images corresponding to the different modes are input into the trained detection model to obtain the ash content prediction value and ash content category of the collected coal sample.

[0012] In the method of the first aspect of the present invention, the pretreatment includes crushing, shrinking and shaping.

[0013] In the method of the first aspect of the present invention, the coal sample target images corresponding to the different modalities include visible light coal sample target images, infrared coal sample target images and thermal imaging coal sample target images.

[0014] In the method of the first aspect of the present invention, the feature adjustment module includes a grouped convolution layer, a normalization layer, a GELU nonlinear function layer and a spatial pyramid pooling layer.

[0015] In the method of the first aspect of the present invention, the convolution layer module in the feature extraction module adopts a deformable convolution module.

[0016] In the method of the first aspect of the present invention, the feature fusion module includes a feature selection submodule and a feature splicing submodule, and each module in the feature extraction module is connected to the feature selection submodule.

[0017] In the method of the first aspect of the present invention, the feature selection submodule adopts a global maximum or average pooling layer, and the feature splicing submodule adopts an expansion layer.

[0018] In the method of the first aspect of the present invention, both the first output submodule and the second output submodule adopt a fully connected layer.

[0019] In the method of the first aspect of the present invention, the coal sampling comprises: real-time monitoring of the amount of coal on the conveyor belt, and when the amount of coal reaches a set amount of coal, sampling the coal on the conveyor belt.

[0020] To achieve the above-mentioned purpose, the second aspect of the present invention proposes a fine coal ash detection system based on image recognition, comprising:

[0021] A sampling device, used for collecting coal samples and pre-processing the coal samples;

[0022] An image acquisition device, used for performing multi-modal image acquisition on the pre-processed coal sample to obtain target images of the coal sample corresponding to different modalities;

[0023] A modeling device, used to obtain a trained detection model, wherein the detection model is used to output a gray content prediction value and a gray content category based on an input image, wherein the detection model includes a feature adjustment module, a feature extraction module, a feature fusion module, and an analysis output module connected in sequence, wherein the feature extraction module includes a plurality of convolutional layer modules and a Transformer self-attention layer module, wherein each module in the feature extraction module is connected to the feature fusion module, and wherein the analysis output module includes a first output submodule and a second output submodule, wherein the first output submodule is used to output a gray content prediction value, and the second output submodule is used to output a gray content category;

[0024] The detection device is used to input the coal sample target images corresponding to the different modes into the trained detection model to obtain the ash content prediction value and ash content category of the collected coal sample.

[0025] The present invention provides a method and system for detecting ash content in final coal based on image recognition, which collects coal samples and preprocesses the coal samples; collects multimodal images of the preprocessed coal samples to obtain target images of coal samples corresponding to different modes; obtains a trained detection model, which is used to output ash prediction values ​​and ash categories based on the input image, and the detection model includes a feature adjustment module, a feature extraction module, a feature fusion module and an analysis output module connected in sequence, the feature extraction module includes a plurality of convolutional layer modules and a Transformer self-attention layer module, each module in the feature extraction module is connected to the feature fusion module, and the analysis output module includes a first output submodule and a second output submodule, the first output submodule is used to output the ash prediction value, and the second output submodule is used to output the ash category; the target images of coal samples corresponding to different modes are input into the trained detection model to obtain the ash prediction value and ash category of the collected coal samples. In this case, the target images of coal samples corresponding to different modes are obtained for the preprocessed coal samples, and then sent to the trained detection model for ash prediction to obtain the ash prediction value and ash category of the coal samples. Among them, the detection model includes a feature adjustment module, a feature extraction module, a feature fusion module and an analysis output module connected in sequence, the feature extraction module includes multiple convolutional layer modules and a Transformer self-attention layer module, each module in the feature extraction module is connected to the feature fusion module, and the analysis output module includes a first output submodule and a second output submodule. The detection model can fully mine the features in the target image of the coal sample corresponding to each modality, improve the accuracy of ash content prediction, and the entire ash content prediction does not require human participation, which improves the efficiency of ash content prediction. Therefore, the ash content of the final coal product can be detected faster and more accurately, solving the problem of time-consuming and labor-intensive ash content detection of the final coal product.

[0026] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0028] Figure 1 A schematic flow chart of a method for detecting ash content in fine coal based on image recognition provided by an embodiment of the present invention;

[0029] Figure 2 A schematic diagram of the structure of a detection model provided by an embodiment of the present invention;

[0030] Figure 3 A schematic diagram of a specific process of a method for detecting ash content in fine coal based on image recognition provided by an embodiment of the present invention;

[0031] Figure 4 A block diagram of a coal ash detection system based on image recognition provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0032] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0033] The following describes a method and system for detecting fine coal ash based on image recognition according to an embodiment of the present invention with reference to the accompanying drawings.

[0034] The embodiment of the present invention provides a method for detecting the ash content of fine coal based on image recognition to solve the problem that the current detection of the ash content of fine coal products is time-consuming and labor-intensive.

[0035] Figure 1 A schematic flow chart of a method for detecting ash content in fine coal based on image recognition provided in an embodiment of the present invention. Figure 2 A schematic diagram of the structure of a detection model provided in an embodiment of the present invention.

[0036] like Figure 1 As shown, the method for detecting the ash content of fine coal based on image recognition comprises the following steps:

[0037] Step S101, collecting coal samples and preprocessing the coal samples.

[0038] In step S101, coal samples are collected, including: real-time monitoring of the amount of coal on the conveyor belt, and when the amount of coal reaches a set amount of coal, the coal samples on the conveyor belt are collected. In other embodiments, coal samples may also be collected at set time intervals.

[0039] In step S101, the preprocessing includes fragmentation, reduction and shaping.

[0040] Specifically, the steps of step S101 include:

[0041] Sampling process: monitor the amount of coal on the conveyor belt in real time, and use a high-precision weight sensor to keenly capture subtle changes in the amount of coal to trigger the sampling action. In some embodiments, sampling can be performed at set time intervals, or sampling can be triggered when the real-time coal amount reaches the set amount of coal. This step is implemented by the sampling unit in the sampling device. The sampling unit is installed above the coal flow conveyor belt and is responsible for automatically collecting coal samples. The sampling unit integrates a high-precision weight sensor and an extraction tube assembly. According to the set time interval or when the amount of coal monitored by the weight sensor reaches the preset amount of coal, the sampling action is triggered and the coal sample is sucked out of the extraction tube. As a result, the sampling unit can be dynamically adjusted according to the fluctuations of the coal flow to ensure the timeliness and representativeness of the sampling.

[0042] Crushing: The collected coal sample is crushed to meet the set particle size standard. This step is achieved by the crushing unit in the sampling device. The crushing unit crushes the sampled coal sample to meet the set particle size standard. The crushing unit crushes the coal sample using a jaw crusher or a double-roll crusher. Therefore, the crushing unit can dynamically adjust according to the particle size requirements of the coal sample so that the particle size of the sample meets the subsequent processing requirements.

[0043] Reduction processing: The crushed coal sample is reduced into four parts, and the reduction processing is optimized according to the properties of the sample. This step is achieved by the reduction unit in the sampling device. The reduction unit is responsible for reducing the crushed coal sample. The reduction unit adopts the reduction method of rotation, extraction or free fall, and adjusts according to the volume and properties of the sample to ensure the uniformity of the sample after reduction.

[0044] Shaping: Collect the shrunken coal samples into the collection tank container, stir and mix the coal samples thoroughly, and flatten the coal sample surface with high precision so that the samples can be evenly distributed. This step is achieved by the shaping unit in the sampling device. The shaping unit includes stirring and flattening equipment, which is used to evenly mix and flatten the shrunken coal samples. The shaping unit can ensure that the coal sample has a uniform surface when collecting images to improve detection accuracy.

[0045] Step S102, performing multi-modal image acquisition on the pre-processed coal sample to obtain target images of the coal sample corresponding to different modalities.

[0046] In step S102, the coal sample target images corresponding to different modalities include visible light coal sample target images, infrared coal sample target images and thermal imaging coal sample target images.

[0047] The steps of step S102 specifically include: using visible light, infrared band, thermal imaging multimodal information fusion method to collect coal sample images: transferring the container with the shaped coal sample to the bottom of the image acquisition device to collect coal sample images. This step is implemented by the image acquisition device. The image acquisition device includes a high-resolution, multi-spectral image acquisition device, which can perform multi-spectral image acquisition in visible light, infrared band and newly added near-infrared band, and use a thermal imager to obtain the temperature information of the coal sample, and capture the multi-dimensional image information of the coal sample in all directions, providing comprehensive data support for ash content prediction. Thus, the coal sample is subjected to multi-spectral image acquisition in visible light and infrared bands and thermal imager image acquisition by the image acquisition device to obtain a multi-modal coal sample target image of the coal sample. The multi-modal coal sample target image includes a visible light coal sample target image, an infrared coal sample target image and a thermal imaging coal sample target image.

[0048] Step S103, obtaining a trained detection model.

[0049] In step S103, the detection model is used to output the ash content prediction value and ash content category based on the input image. The input image of the detection model is the target image of the coal sample corresponding to all the modes collected in step S102.

[0050] In step S103, the detection model is a multi-scale multi-task neural network model. The detection model includes a feature adjustment module, a feature extraction module, a feature fusion module and an analysis output module which are connected in sequence.

[0051] In step S103, the feature adjustment module is used to process the input image to adjust the coal sample target images corresponding to multiple modes with different resolutions acquired by the image acquisition device into feature images of uniform size and format to meet the requirements of the subsequent feature extraction layer for input size. Figure 2 As shown, the input of the feature adjustment module (also called the feature adjustment layer) is a multimodal input, and the input images are the infrared image of the coal sample after crushing and shaping, the visible light image of the coal sample after crushing and shaping, and the thermal imaging image of the coal sample after crushing and shaping.

[0052] In step S103, the feature adjustment module includes a group convolution layer, a normalization layer, a GELU (Gaussian Error Linear Unit) nonlinear function layer and a spatial pyramid pooling layer.

[0053] Specifically, the feature adjustment module includes two grouped convolutional layers, a normalization layer, a GELU nonlinear function, and a spatial pyramid pooling layer connected in sequence. The grouped convolutional layer performs grouped convolution operations on the input multimodal image data (i.e., coal sample target images corresponding to multiple modalities), retains the independent features of different modal data, and reduces information interference between modalities. The normalization layer uses layer normalization to keep the data in a similar distribution to avoid gradient explosion or vanishing problems caused by excessive differences in data distribution. The GELU nonlinear function activates the features. The spatial pyramid pooling layer uses pooling operations of different scales to capture the multi-scale feature information of the image, and the pooled feature maps are uniformly output as a fixed size (e.g., 224×224).

[0054] In step S103, the feature extraction module is used to receive the output of the feature adjustment module as input, perform feature extraction and output a feature map tensor. The feature extraction module includes multiple convolutional layer modules and a Transformer self-attention layer module, and each module in the feature extraction module is connected to the feature fusion module.

[0055] The convolutional layer module in the feature extraction module can adopt a deformable convolution (DCN) module. In this case, the feature extraction module is also called a DCTANet module (see Figure 2 ).

[0056] The DCTANet module combines the advantages of the deformable convolutional neural network and the Transformer self-attention mechanism. Figure 2 As shown in the figure, the DCTANet module first extracts local features through three deformable convolution modules, and then captures global features through a Transformer self-attention layer module. The DCTANet module obtains feature maps of different scales by extracting features layer by layer. At the output of the Transformer self-attention layer module, the feature fusion module is connected, and the three deformable convolution modules are connected to the feature fusion module respectively. The feature map tensors extracted from each layer are globally max-pooled or average-pooled, expanded, and spliced ​​through the feature fusion module to achieve multi-scale feature fusion.

[0057] Among them, the three deformable convolution modules are formed by stacking N1, N2 and N3 DCNBlocks (DCN blocks) respectively. Each DCN Block consists of a 3×3 DeformConv2d convolution layer with a stride of 1 and the number of groups equal to the number of input channels, a layer normalization layer, two 1×1 point-by-point convolution layers with a stride of 1, and a GELU activation function. The Transformer self-attention layer module consists of stacking M Transformer encoder modules. The Transformer encoder module includes a multi-head self-attention layer, a jump connection and layer normalization layer, a feedforward neural network layer, and a second jump connection and layer normalization. Among them, the calculation method of the multi-head self-attention layer is:

[0058]

[0059] MultiHead(Q,K,V)=Concat(head1,head2,…,head h )W O

[0060] Where Q, K, and V are the query, key, and value matrices generated after the input is linearly transformed; K T is the transpose of K; d k is the dimension of the query vector and key vector; W O is a learnable parameter matrix; Concat means concatenating the attention mechanisms of each head into a matrix. head1, head2, ..., head h The attention mechanism of each head. h is the number of heads.

[0061] In step S103, the feature fusion module is used to receive the output of the feature extraction module as input, perform multi-scale feature fusion and output a multi-scale fusion feature map. The feature fusion module includes a feature selection submodule and a feature splicing submodule, and each module in the feature extraction module is connected to the feature selection submodule. Figure 2 As shown, the feature selection submodule uses the global maximum or average pooling layer to perform global maximum or average pooling, and the feature splicing submodule uses the expansion layer (i.e., Flatten layer) to expand the features and splice the features along the channel.

[0062] In step S103, the analysis output module is used to receive the output of the feature fusion module as input, and perform multi-task prediction to obtain the ash prediction value and ash category. The analysis output module includes a first output submodule and a second output submodule, the first output submodule is used to output the ash prediction value, and the second output submodule is used to output the ash category. Both the first output submodule and the second output submodule use a fully connected layer.

[0063] Specifically, Figure 2 As shown, the first branch of the fully connected layer is the first output submodule, which receives and processes the multi-scale fusion feature map and outputs the ash content prediction value (for example, ash content regression prediction) corresponding to the target image of the coal sample. The second branch of the fully connected layer is the second output submodule, which receives and processes the multi-scale fusion feature map and outputs the ash content category corresponding to the target image of the coal sample. For example, there are 200 categories of ash content, and the second output submodule outputs the specific category of the 200 categories to which the ash content value belongs, so as to realize ash content classification prediction.

[0064] In step S103, after the detection model is acquired, the detection model needs to be trained using the training set to obtain a trained detection model.

[0065] The training set consists of labeled visible light images of coal samples, labeled infrared images of coal samples, and labeled thermal imaging images of coal samples. The visible light images of coal samples, labeled infrared images of coal samples, and labeled thermal imaging images of coal samples are collected from coal samples that have undergone the same preprocessing as step S101. Labels include ash value labels and ash category labels. The ash value in the ash value label can be obtained by preparing the sample and then manually measuring it by burning the ash in a muffle furnace. The ash category label is manually classified and labeled according to the ash value range. The ash value label corresponds to the first output submodule, and the ash category label corresponds to the second output submodule.

[0066] The training set is used to supervise the detection model and obtain a robust model. Finally, the trained detection model can be applied to the actual ash content detection.

[0067] Step S104, inputting the coal sample target images corresponding to different modalities into the trained detection model to obtain the ash content prediction value and ash content category of the collected coal sample.

[0068] In step S104, the visible light coal sample target image, infrared coal sample target image and thermal imaging coal sample target image are input into the trained detection model to analyze the multi-modal and multi-scale features of the image using the model, and the ash content is detected in the form of multi-task prediction, thereby obtaining the ash content prediction value and ash content category of the collected coal sample. As a result, it is no longer necessary to manually prepare samples and burn ash in a time-consuming manner. The ash content of the current sample can be directly predicted through the model, thereby reducing work pressure and being able to display the ash content information of the product on the belt in a more real-time manner.

[0069] In some embodiments, the system can also output the prediction results (i.e., ash prediction value and ash category) in a standardized and normalized format, and store the ash data in a storage device with high security and high reliability for further in-depth analysis and refined management. The storage device can be a hard disk, cloud storage, etc., for storing prediction results and historical data.

[0070] Figure 3 A schematic diagram of a specific process of a method for detecting ash content in fine coal based on image recognition provided in an embodiment of the present invention.

[0071] Combine the following Figure 3 The method for detecting the fine coal ash content based on image recognition of the present invention is described.

[0072] 1) Build a coal ash detection system:

[0073] 11) Installation and configuration of sampling unit. Install the sampling unit at a suitable position above the coal conveyor belt and ensure that it can fully cover the coal flow range. The sampling unit integrates a high-precision weight sensor and an extraction tube assembly. The sampling threshold of the weight sensor is set to trigger the sampling action when the coal flow reaches 500 kilograms per cubic meter. At the same time, set a timed sampling every 10 minutes as a supplementary sampling method. Debug the sampling device so that it can be dynamically adjusted according to the fluctuation of the coal flow.

[0074] 12) Selection and setting of crushing unit. A jaw crusher is selected as the main equipment of the crushing unit and installed at a suitable position downstream of the sampling point to facilitate timely receipt of the collected coal samples for crushing. The particle size standard of the crushed coal sample is set to be less than 6 mm. The crushing unit is dynamically adjusted according to the particle size requirements of the coal sample. When the input coal sample particle size is large, the crushing force is automatically increased and the crushing time is appropriately extended to ensure that the particle size of the sample meets the subsequent processing requirements.

[0075] 13) Setting and adjustment of the reduction unit. Install the reduction unit after the crushing unit, and use a combination of rotation, extraction and free fall. Perform the initial setting according to the volume and properties of the coal sample. First, use the rotation reduction method to divide the coal sample into two roughly equal parts, then use the extraction reduction method to extract a part from each of the two parts to form two new parts, and finally use the free fall reduction method to drop the two new parts of coal sample into different collection containers, so as to achieve reduction into four parts.

[0076] 14) Installation and debugging of the shaping unit. Install the shaping unit downstream of the reduction unit, which includes a stirring and flattening device. Collect the reduced coal sample into the collection tank container of the shaping unit. Debug the parameters of the stirring and flattening device according to the actual volume and density of the coal sample. Set the stirring speed to 60 revolutions per minute and the flattening force to a moderate intensity so that the sample can be evenly distributed and ensure that the coal sample has a uniform surface during image acquisition to improve detection accuracy.

[0077] 15) Image acquisition device configuration. Install a high-resolution, multi-spectral image acquisition system as an image acquisition device, place it behind the shaping unit, and collect images of the shaped coal sample. Configure the image acquisition device to enable it to perform multi-spectral image acquisition in visible light, infrared bands, and the newly added near-infrared band. At the same time, use a thermal imager to obtain the temperature information of the coal sample and capture the multi-dimensional image information of the coal sample in all directions. Collect coal sample images every 5 seconds to provide comprehensive data support for ash content prediction.

[0078] 16) Modeling device preparation. The modeling device is equipped with a detection model, which is deployed on a computing device connected to the image acquisition device to promptly receive and process the collected multimodal image data. The multi-scale multi-task neural network deep learning model is trained and optimized, using a large amount of coal sample image data and corresponding ash content data accumulated in the past as a training set. Through multiple iterative training, the model can accurately analyze the multimodal and multi-scale features of the image, and detect the ash content in the form of multi-task prediction.

[0079] 17) Preparation of the detection device: The modeling device outputs the trained detection model to the detection device for actual ash content detection.

[0080] 18) Storage device settings. Choose a combination of hard disk storage and cloud storage as the storage device, configure a large-capacity hard disk on the local server to store recent prediction results and historical data, and back up important data to a reliable cloud storage platform. Set the format and specifications for data storage, and classify and store the ash data of each test according to date, coal sample batch, test time and other information to facilitate subsequent query, analysis and management.

[0081] 2) If Figure 3 As shown, the final coal ash content is fully automatically detected:

[0082] 21) Coal products are sampled by weight and time. The sampling unit monitors the coal flow on the conveyor belt in real time, and the high-precision weight sensor keenly captures the slight changes in the coal flow. When the coal flow reaches the set 500 kg per cubic meter or every 10 minutes (whichever comes first), the sampling action is triggered and the coal sample is sucked out of the extraction tube. When the coal flow speed increases and the coal flow increases rapidly, the sampling device responds immediately, increases the extraction frequency, and successfully collects a coal sample that can represent the coal quality of that period.

[0083] 22) The coal samples are crushed by jaw and roller crushing. The collected coal samples are quickly transferred to the jaw crusher of the crushing unit for crushing. The crusher automatically adjusts the crushing force and time according to the initial particle size of the coal sample. If the coal sample particle size is large, such as the diameter of some coal blocks reaching 50 mm, the crusher will increase the crushing force. After a period of crushing, the coal sample reaches the set particle size standard of less than 6 mm, meeting the subsequent processing requirements.

[0084] 23) The coal sample is reduced by rotation, extraction, and free fall. The crushed coal sample enters the reduction unit for reduction. According to the pre-set reduction method, the coal sample is first divided into two roughly equal parts by rotation reduction, and then part of it is extracted to form two new parts. Finally, the two new parts of coal sample are dropped into different collection containers by free fall reduction, and successfully reduced into four parts. In the reduction process, according to the actual properties of the coal sample, if the particle size distribution of the coal sample is found to be uneven, the proportion and order of the reduction method are adjusted in time to ensure the uniformity of the sample after reduction.

[0085] 24) The coal sample is shaped by stirring and flattening. The shrunken coal sample is collected in the collection tank container of the shaping unit. The stirring and flattening device stirs and mixes the coal sample comprehensively and flattens the surface of the coal sample with high precision. In view of the large volume and high density of the coal sample, the stirring device stirs at a speed of 60 revolutions per minute, and the flattening device flattens at a moderate intensity so that the sample can be evenly distributed and ready for image acquisition.

[0086] 25) The coal sample image is collected by fusion of visible light, infrared band and thermal imaging multi-modal information. The shaped coal sample container is transferred to the bottom of the image acquisition device. The image acquisition device collects multi-spectral images and thermal imager images of the coal sample in visible light, infrared band and near-infrared band according to the set parameters of 1280×720 pixels and acquisition every 5 seconds, so as to capture the multi-dimensional image information of the coal sample in all directions and obtain the multi-modal information data of the coal sample.

[0087] 26) Predict ash content based on image intelligence. The collected multimodal image data is transmitted to the detection device in real time. The detection model of the detection device is developed by the modeling device based on the multi-scale multi-task neural network deep learning model. The detection model uses its internal feature adjustment layer to pre-process the input multimodal data, and adjusts the image inputs of different resolutions collected by different image acquisition devices to a unified size and format (such as 224×224 size) to meet the input size requirements of the subsequent feature extraction layer. The image data processed by the feature adjustment layer enters the DCTANet module. Each module combines deformable convolution with the Transformer self-attention mechanism. It first extracts local features through deformable convolution, and then captures global features through the Transformer self-attention mechanism. Feature maps of different scales are obtained through layer-by-layer feature extraction. The feature fusion module performs global maximum or average pooling, expansion and splicing on the feature map tensors extracted from each layer to achieve multi-scale feature fusion. The fused multi-scale feature map is transmitted to two branch fully connected layers, one of which is used to output the ash value corresponding to the coal sample image, and the other branch is used to output the ash category of the coal sample image, thereby completing the ash detection.

[0088] 27) The prediction results are output and stored in storage devices such as hard disks and cloud platforms. The system outputs the ash prediction results output by the detection device in a standardized and normalized format, and outputs the ash content in percentage form with an accuracy of two decimal places. At the same time, the ash data is stored in the hard disk and cloud storage platform of the local server according to the set classification storage method for further in-depth analysis and refined management. For example, in the subsequent production process, the changing trend of coal quality can be analyzed based on the ash data of different batches of coal samples, so as to adjust the production process in time.

[0089] In order to verify the effect of the method of the present invention, verification was performed.

[0090] After a period of actual operation, the effect of the fully automatic ash content intelligent detection method and system of fine coal adopted in this embodiment was verified:

[0091] The fully automatic sampling, sample preparation and testing process has greatly reduced the amount of manual intervention. In the past, manual ash testing took several hours to complete the testing of a batch of coal samples, but now the entire testing process can be completed within half an hour, greatly improving the testing efficiency.

[0092] Compared with the test results of traditional chemical analysis methods and some automated detection equipment in the past, the combination of multimodal images and deep learning technology adopted in this embodiment significantly improves the detection accuracy. After multiple sampling comparisons and verifications, the error between the ash content detected by the new method and the actual ash content is controlled within ±1%, which can more accurately reflect the ash content of the coal sample.

[0093] In the actual production process, facing different types of coal and different working conditions, the system can dynamically adjust the parameters and operation methods of each link according to the specific properties of the coal sample. For example, when dealing with a new type of coal, the sampling unit adjusts the sampling frequency and method according to its coal flow characteristics, the crushing unit adjusts the crushing parameters according to the particle size and hardness of the coal sample, the reduction unit adjusts the reduction method according to the nature of the coal sample, the shaping unit adjusts the stirring and flattening parameters according to the volume and density of the coal sample, the image acquisition device adjusts the image acquisition parameters according to the characteristics of the coal sample, the modeling device fine-tunes the model according to the new coal sample data, and the storage module stores data according to the new classification method, thus adapting to the application needs of various industrial environments and having wide applicability and good compatibility.

[0094] In order to implement the above-mentioned embodiment, the present invention also proposes a fine coal ash detection system based on image recognition.

[0095] Figure 4 A block diagram of a coal ash detection system based on image recognition provided by an embodiment of the present invention.

[0096] like Figure 4 As shown, the coal ash detection system based on image recognition includes a sampling device 11, an image acquisition device 12, a modeling device 13 and a detection device 14, wherein:

[0097] The sampling device 11 is used to collect coal samples and pre-treat the coal samples;

[0098] An image acquisition device 12 is used to perform multi-modal image acquisition on the pre-processed coal sample to obtain target images of the coal sample corresponding to different modalities;

[0099] A modeling device 13 is used to obtain a trained detection model, the detection model is used to output a gray prediction value and a gray category based on an input image, the detection model includes a feature adjustment module, a feature extraction module, a feature fusion module and an analysis output module connected in sequence, the feature extraction module includes a plurality of convolutional layer modules and a Transformer self-attention layer module, each module in the feature extraction module is connected to the feature fusion module, the analysis output module includes a first output submodule and a second output submodule, the first output submodule is used to output a gray prediction value, and the second output submodule is used to output a gray category;

[0100] The detection device 14 is used to input the coal sample target images corresponding to different modes into the trained detection model to obtain the ash content prediction value and ash content category of the collected coal sample.

[0101] Furthermore, in a possible implementation of the embodiment of the present invention, in the sampling device 11, collecting coal samples includes: real-time monitoring of the amount of coal on the conveyor belt, and when the amount of coal reaches a set amount of coal, collecting the coal sample on the conveyor belt.

[0102] Furthermore, in a possible implementation manner of the embodiment of the present invention, in the sampling device 11, the preprocessing includes a crushing process, a shrinking process and a shaping process.

[0103] Furthermore, in a possible implementation of an embodiment of the present invention, in the image acquisition device 12, the coal sample target images corresponding to different modalities include visible light coal sample target images, infrared coal sample target images and thermal imaging coal sample target images.

[0104] Furthermore, in a possible implementation of the embodiment of the present invention, in the modeling device 13, the feature adjustment module includes a grouped convolution layer, a normalization layer, a GELU nonlinear function layer and a spatial pyramid pooling layer.

[0105] Furthermore, in a possible implementation of the embodiment of the present invention, in the modeling device 13, the convolution layer module in the feature extraction module adopts a deformable convolution module.

[0106] Furthermore, in a possible implementation of the embodiment of the present invention, in the modeling device 13, the feature fusion module includes a feature selection submodule and a feature splicing submodule, and each module in the feature extraction module is connected to the feature selection submodule.

[0107] Furthermore, in a possible implementation of the embodiment of the present invention, in the modeling device 13, the feature selection submodule adopts a global maximum or average pooling layer, and the feature splicing submodule adopts an expansion layer.

[0108] Furthermore, in a possible implementation of the embodiment of the present invention, in the modeling device 13, the first output submodule and the second output submodule both use a fully connected layer.

[0109] It should be noted that the aforementioned explanation of the embodiment of the fine coal ash detection method based on image recognition is also applicable to the fine coal ash detection system based on image recognition in this embodiment, and will not be repeated here.

[0110] In the embodiment of the present invention, coal samples are collected and preprocessed; multimodal image collection is performed on the preprocessed coal samples to obtain coal sample target images corresponding to different modalities; a trained detection model is obtained, and the detection model is used to output ash content prediction values ​​and ash content categories based on the input image. The detection model includes a feature adjustment module, a feature extraction module, a feature fusion module, and an analysis output module connected in sequence. The feature extraction module includes multiple convolutional layer modules and a Transformer self-attention layer module. Each module in the feature extraction module is connected to the feature fusion module. The analysis output module includes a first output submodule and a second output submodule. The first output submodule is used to output the ash content prediction value, and the second output submodule is used to output the ash content category; the coal sample target images corresponding to different modalities are input into the trained detection model to obtain the ash content prediction value and ash content category of the collected coal sample. In this case, the coal sample target images corresponding to different modalities are obtained for the preprocessed coal sample, and then sent to the trained detection model for ash content prediction to obtain the ash content prediction value and ash content category of the coal sample. Among them, the detection model includes a feature adjustment module, a feature extraction module, a feature fusion module and an analysis output module connected in sequence, the feature extraction module includes multiple convolutional layer modules and a Transformer self-attention layer module, each module in the feature extraction module is connected to the feature fusion module, and the analysis output module includes a first output submodule and a second output submodule. The detection model can fully mine the features in the target image of the coal sample corresponding to each modality, improve the accuracy of ash content prediction, and the entire ash content prediction does not require human participation, which improves the efficiency of ash content prediction. Therefore, the ash content of the final coal product can be detected faster and more accurately, solving the problem of time-consuming and labor-intensive ash content detection of the final coal product.

[0111] The method and system of the present invention belong to the field of coal engineering, and are specifically a method and system for fully automatic intelligent detection of ash content in fine coal.

[0112] Through automated coal product sampling, sample preparation, and multimodal image acquisition and processing, a method and system for fully automatic intelligent ash content detection of fine coal is realized, which is suitable for rapid prediction of the ash content of coal sample products during coal production, processing, and quality inspection. The problem of time-consuming and labor-intensive ash content detection of fine coal products is solved, and it can ensure that ash content detection can be completed efficiently, accurately, safely, and automatically. Therefore, the present invention not only helps to improve detection efficiency and meet the high requirements for quality control in the coal production process, but also has the practical value of reducing the manual burden and improving the level of production automation.

[0113] The method and system of the present invention have the following advantages:

[0114] 1) Most existing technologies are based on visible light images, while the present invention adds infrared and thermal imaging images and adopts multimodal input. This multimodal input method can capture the characteristic information of coal samples more comprehensively and provide richer data dimensions, thereby improving the model's adaptability to complex scenarios and prediction accuracy.

[0115] 2) The prediction model uses a multi-scale feature extraction method, which enables it to express both details and global semantic information at different scales. At the same time, combined with a multi-task learning strategy, the model can simultaneously complete ash regression and classification tasks, improving the collaborative learning effect of the tasks and further improving the accuracy and robustness of the prediction. Real-time and automated ash detection is achieved, which not only greatly improves the detection efficiency, but also reaches a higher level of detection accuracy, and can more accurately reflect the ash content of coal samples.

[0116] 3) We designed a new feature extraction backbone network DCTANet, which combines the local modeling capability of deformable convolution and the global feature capture capability of Transformer self-attention, and provides powerful expression capabilities for multimodal input through a more efficient feature extraction method.

[0117] 4) No radioactive hazards: Abandoning the traditional gamma-ray detection method, the advanced image processing technology is adopted to completely eliminate potential radioactive hazards and ensure the health of operators and environmental safety.

[0118] 5) High degree of automation: The fully automatic sampling, sample preparation and testing process greatly reduces human intervention, improves the level of production automation, reduces labor costs, and also reduces errors caused by human factors.

[0119] 6) Strong adaptability: The system can dynamically adjust the parameters and operation methods of each link according to different coal types and different working conditions, adapt to the application needs of various industrial environments, and has wide applicability and good compatibility.

[0120] In order to implement the above embodiments, the present invention also proposes an electronic device, comprising: a processor, and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the method provided by the above embodiments.

[0121] In order to implement the above embodiments, the present invention further proposes a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the methods provided by the above embodiments.

[0122] In order to implement the above embodiments, the present invention further provides a computer program product, including a computer program, which implements the methods provided by the above embodiments when executed by a processor.

[0123] In the description of the aforementioned embodiments, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0124] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0125] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present invention belong.

[0126] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute the instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0127] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0128] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0129] In addition, each functional unit in each embodiment of the present invention may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0130] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present invention. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A method for detecting ash content in fine coal based on image recognition, characterized in that: include: Collecting coal samples and pre-treating the coal samples; Perform multi-modal image acquisition on the pre-processed coal sample to obtain target images of the coal sample corresponding to different modalities; Acquire a trained detection model, the detection model is used to output a gray prediction value and a gray category based on an input image, the detection model includes a feature adjustment module, a feature extraction module, a feature fusion module and an analysis output module connected in sequence, the feature extraction module includes a plurality of convolutional layer modules and a Transformer self-attention layer module, each module in the feature extraction module is connected to the feature fusion module, the analysis output module includes a first output submodule and a second output submodule, the first output submodule is used to output a gray prediction value, and the second output submodule is used to output a gray category; The coal sample target images corresponding to the different modes are input into the trained detection model to obtain the ash content prediction value and ash content category of the collected coal sample.

2. The method for detecting ash content in fine coal based on image recognition according to claim 1, characterized in that: The pretreatment includes crushing, shrinking and shaping.

3. The method for detecting ash content in fine coal based on image recognition according to claim 1, characterized in that: The coal sample target images corresponding to the different modes include visible light coal sample target images, infrared coal sample target images and thermal imaging coal sample target images.

4. The method for detecting the ash content of fine coal based on image recognition according to claim 1, characterized in that: The feature adjustment module includes a grouped convolution layer, a normalization layer, a GELU nonlinear function layer and a spatial pyramid pooling layer.

5. The method for detecting the ash content of fine coal based on image recognition according to claim 1, characterized in that: The convolution layer module in the feature extraction module adopts a deformable convolution module.

6. The method for detecting the ash content of fine coal based on image recognition according to claim 1, characterized in that: The feature fusion module includes a feature selection submodule and a feature splicing submodule, and each module in the feature extraction module is connected to the feature selection submodule.

7. The method for detecting the ash content of fine coal based on image recognition according to claim 6 is characterized in that: The feature selection submodule adopts a global maximum or average pooling layer, and the feature splicing submodule adopts an expansion layer.

8. The method for detecting ash content in fine coal based on image recognition according to claim 1, characterized in that: The first output submodule and the second output submodule both use fully connected layers.

9. The method for detecting ash content in fine coal based on image recognition according to claim 1, characterized in that: The coal sampling comprises: The amount of coal on the conveyor belt is monitored in real time, and when the amount of coal reaches the set amount of coal, the coal sample on the conveyor belt is collected.

10. A coal ash detection system based on image recognition, characterized in that: include: A sampling device, used for collecting coal samples and pre-processing the coal samples; An image acquisition device, used for performing multi-modal image acquisition on the pre-processed coal sample to obtain target images of the coal sample corresponding to different modalities; A modeling device, used to obtain a trained detection model, wherein the detection model is used to output a gray content prediction value and a gray content category based on an input image, wherein the detection model includes a feature adjustment module, a feature extraction module, a feature fusion module, and an analysis output module connected in sequence, wherein the feature extraction module includes a plurality of convolutional layer modules and a Transformer self-attention layer module, wherein each module in the feature extraction module is connected to the feature fusion module, and wherein the analysis output module includes a first output submodule and a second output submodule, wherein the first output submodule is used to output a gray content prediction value, and the second output submodule is used to output a gray content category; The detection device is used to input the coal sample target images corresponding to the different modes into the trained detection model to obtain the ash content prediction value and ash content category of the collected coal sample.

Images