Dual-modal coal and rock microscopic component recognition method based on dual decoder fusion UNetFormer architecture

Abstract

The present invention discloses a dual-modal coal-rock microscopic component identification method based on a dual-decoder fusion UNetFormer architecture. The method aims to accurately identify the microscopic components in coal and rock to guide their efficient utilization in industry. The present invention combines fluorescence and reflected light coal and rock microscopic image information, and proposes an end-to-end dual-modal semantic segmentation method based on a dual-decoder UNetFormer, including: using ResNet18 and Transformer structures as encoders and decoders of the basic network respectively; in order to improve the segmentation results of the edges of the microscopic components, the present invention additionally introduces an edge decoder into the basic network, and designs an auxiliary loss function to optimize network training. Experimental results show that the method proposed in the present invention has achieved excellent results in the semantic segmentation of microscopic components, with an average intersection-over-union ratio of 75.07%, which exceeds any other identification method in the field of coal and rock microscopic component identification. The dual-modal microscopic component identification method provides valuable insights for coal and rock quality prediction and efficient utilization.

Description

Technical Field

[0001] The present invention relates to the fields of coal mining industry and computer vision, and in particular to a method for identifying coal rock microscopic components, in particular to a dual-modal coal rock microscopic component identification method based on a dual-decoder fusion UNetFormer architecture. Background Art

[0002] Coal is an important source of global electricity supply, especially in some developing countries, where it provides a vital source of electricity when other forms of energy are difficult to obtain. However, coal utilization has always been a focus of attention due to the pollution problems caused by coal combustion. In order to use coal efficiently while meeting strict emission standards, accurate assessment of coal properties has become crucial. The microscopic components of coal are closely related to its chemical composition and greatly affect the properties of coal. However, traditional microscopic component analysis is usually performed manually by highly skilled coal petrologists or geologists, which is time-consuming, labor-intensive and highly subjective.

[0003] In recent years, with the development of machine learning and computer vision, computer-assisted coal petrological evaluation has shown great potential. The widespread application of deep learning algorithms has brought revolutionary changes to the field of image analysis. The algorithm has the ability to automatically learn features, thereby improving the accuracy and efficiency of analysis. However, there are still some challenges when the existing deep learning methods are applied to the identification of coal and rock microscopic components.

[0004] The Chinese patent application, publication number CN111968136A, describes a coal rock microscopic image analysis method and analysis system. The patent preprocesses the coal rock microscopic image to be processed, extracts the effective area in the preprocessed image; divides the effective area into multiple image areas of preset size, and performs feature detection of the target object on the multiple image areas respectively through the trained first neural network to obtain the candidate image area; classifies the target image area corresponding to the candidate image area in the coal rock microscopic image to be processed through the second neural network to obtain the confidence that the target image area belongs to the preset category; screens the target image area, retains the target image area with a confidence greater than the first threshold, and measures the characteristic index of the retained target image area. The invention completes the feature extraction of coal rock microscopic images through a deep neural network, realizes automatic semantic segmentation, identifies the different coal rock components and their proportions in the image, and realizes the accurate detection and recognition of coal rock microscopic components. This method relies on multiple neural networks to extract candidate areas and classify and screen the candidate areas, which can preliminarily realize the semantic segmentation of microscopic images, but because it does not consider the characteristics of microscopic images for special network improvement, this method has the disadvantages of not being able to achieve semantic segmentation end-to-end and poor recognition results.

[0005] The Chinese patent application, publication number CN111723738A, records a method and system for classifying microscopic images of coal rock shell group based on transfer learning, including the following steps: S1: collect and expand the number of samples; S2: obtain a pre-trained model; S3: construct a coal rock shell group microscopic component identification model; S4: perform component identification. In the step S1, the data enhancement process is as follows: S11, divide the acquired samples into a training set and a test set; S12, randomly scale, randomly horizontally translate, and vertically translate the images in the training set to achieve data expansion. The invention completes the training of the target data set based on the transfer learning method by sharing the parameters of the convolution layer and the pooling layer in the pre-trained network, so that a model with good generalization ability can be trained even when the number of shell group samples is limited, and the effective classification of coal rock shell images is achieved. The method trains a model that can classify the shell group based on transfer learning, but can only classify the entire image and the classification category is limited to the shell group, and cannot distinguish images containing multiple microscopic components. Summary of the invention

[0006] 1. Technical problems to be solved

[0007] In view of the problem that the prior art cannot accurately, quickly and end-to-end identify a variety of coal and rock microscopic components, the present invention proposes a dual-modal coal and rock microscopic component identification method based on a dual decoder fusion UNetFormer architecture for coal and rock microscopic component identification. The present invention can achieve the pixel-level classification (semantic segmentation) task of microscopic components end-to-end, that is, only one neural network can be used to classify different pixel points of the microscopic image into four categories: background resin, vitrinite, exinite and inertinite; at the same time, after fusing the fluorescence and reflected light dual-modal microscopic image information and adding an edge decoder to optimize the edge recognition results of the microscopic components, the present invention can complete the pixel-level classification task of the four microscopic components more finely.

[0008] 2. Technical solution

[0009] The purpose of the present invention is achieved through the following technical solutions.

[0010] One aspect of the embodiments of this specification provides a dual-modal coal and rock microscopic component identification method based on a dual decoder fusion UNetFormer architecture, and the specific steps are as follows:

[0011] Step 1: Obtain dual-modal microscopic images of coal and rock samples;

[0012] Step 2: Annotate the dual-modal microscopic image at the pixel level to obtain the coal and rock component category labels of the corresponding pixels;

[0013] Step 3: Input the dual-modal microscopic image into the improved UNetFormer network;

[0014] Step 4: Use the cross entropy loss function and Dice loss function as the main loss, combine the auxiliary losses generated by the two decoder outputs, construct a comprehensive loss function, and perform network training;

[0015] Step 5: Input the coal-rock dual-modal microscopic image into the trained network, identify different microscopic components, and output the recognition results.

[0016] Preferably, the dual-modal microscopic image of the coal rock sample obtained in step 1 is a microscopic image obtained by fusing the microscopic images under fluorescence and reflected light in the channel dimension, so as to better identify the microscopic components.

[0017] Preferably, the component type in step 2 is one of background resin, exinite, vitrinite or inertinite.

[0018] Preferably, the improved UNetFormer network described in step 3 has a specific structure as follows:

[0019] A lightweight CNN-based encoder based on ResNet18 is used, which contains multiple Resblocks. Each Resblock performs 2x downsampling on the feature map and is connected to two decoders through skip connections.

[0020] A Transformer-based decoder is used, which includes three Global-Local Transformer Blocks (GLTB) and a Feature Refinement Head (FRH) to capture global and local contextual information at multiple scales.

[0021] An additional edge decoder is introduced to extract edge features of microscopic components, which is connected to the ResNet18 encoder through a skip connection;

[0022] The Spatial and Channel Attention Block is used to fuse the outputs of the Transformer-based decoder and the edge decoder to obtain the final coal and rock microscopic component recognition results.

[0023] Preferably, the encoder of the improved UNetFormer network in step 3 obtains the features of the coal and rock samples through Resblocks, and is connected to the Transformer-based decoder and the edge decoder through jump connections. When the edge decoder obtains the edge features of the microscopic components, it extracts semantic and spatial features of different scales through the E2B edge extraction module and the Sobel module. The output of the Transformer decoder is fused with the output of the edge decoder through the spatial channel attention module (Spatialand Channel Attention Block) to obtain accurate microscopic component recognition results.

[0024] Preferably, the comprehensive loss function constructed by introducing the auxiliary loss function in step 4 is calculated as follows:

[0025]

[0026] in, is a constant, is the comprehensive loss function. represents the auxiliary loss at the edge decoder output, represents the auxiliary loss based on the Transformer decoder output, Represents the auxiliary loss of the auxiliary output head of the Transformer decoder. All three auxiliary losses are cross entropy losses. . The main loss function used to deal with class imbalance problems is Loss by Dice and cross entropy loss The loss calculation formulas are as follows:

[0027]

[0028] in, and is a constant, represents the main loss function, represents the cross entropy loss, represents the Dice loss, represents the total number of pixels in each batch of images, represents the number of categories, Represents pixels Belongs to category The predicted probability of Represents pixels Belongs to category The actual label.

[0029] Preferably, the training of the network in step 4 adopts a five-fold cross-validation strategy to ensure the independence of the data set and provide a more comprehensive and objective network performance evaluation. During the training process, the AdamW optimizer was used with an initial learning rate of 2×10 -4 , achieving efficient feature learning and recognition.

[0030] Preferably, when performing network training in step 4, performance evaluation indicators including pixel accuracy (PA), mean pixel accuracy (MPA), and mean intersection over union (MIOU) are used to evaluate the performance of the network in the task of identifying microscopic components of coal and rock. The formulas of each evaluation indicator are as follows:

[0031]

[0032] In the above formula, we assume that there is t Class, in the present invention there are 4 classes. P mm Indicates that it belongs to the class m and is predicted to be class m The number of pixels, P mn Indicates that it belongs to the class m But it is predicted to be n The number of pixels.

[0033] Preferably, the classification categories of the microscopic components implemented in step 5 include four categories: background resin, exinite, vitrinite and inertinite.

[0034] 3. Beneficial effects

[0035] Compared with the prior art, the advantages of the present invention are:

[0036] (1) Use of dual-modal image data. The present invention makes full use of the dual-modal information of fluorescence and reflected light microscopic images to achieve more accurate and comprehensive identification of microscopic components. The comprehensive application of fluorescence and reflected light microscopic images provides more dimensions and depth for the analysis of coal and rock microscopic components.

[0037] (2) Improved deep learning network. The present invention provides a bimodal semantic segmentation method based on an improved UnetFormer architecture, which can accurately identify microscopic components, especially microscopic components with similar grayscale in a single modality.

[0038] (3) Edge feature extraction. Our invention introduces an advanced edge feature extractor that can accurately extract the edge features of microscopic components. This innovation enables our system to clearly and accurately identify the edges of microscopic components, providing a reliable guarantee for the analysis of coal and rock microscopic components.

[0039] (4) Auxiliary loss function. Our invention introduces an auxiliary loss function, which significantly improves the network training effect by guiding the network to learn intermediate representations. This innovative training strategy not only makes our system more stable and reliable, but also enhances the robustness of the system in various complex scenarios, ensuring the accuracy and credibility of the recognition results. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Attached Figure 1 The overall structure of the method, wherein the improved UNetFormer network includes a ResNet18 encoder, a Transformer-based decoder, an edge decoder, and a spatial and channel attention block;

[0041] Figure 2 The structure of GLTB in the Transformer-based decoder;

[0042] Figure 3 The structure of FRH in Transformer-based decoder;

[0043] Figure 4 is the detailed structure of the edge decoder;

[0044] Figure 5 The specific structure of the Spatial and Channel Attention Block;

[0045] Figure 6 The following is a comparison of the recognition results of the method of the present invention with other popular methods in recent years. DETAILED DESCRIPTION

[0046] Vitrinite, inertinite and exinite are the three main categories of organic matter in coal and petroleum. They play an important role in the study of petrology and geology. Through their analysis, geologists can have a more comprehensive understanding of the origin, maturity, sedimentary environment and other aspects of rocks.

[0047] Vitrinite: Vitrinite mainly consists of organic matter with a glassy luster, usually the cell wall residues of plants. These residues gradually transform into coal under high temperature and high pressure conditions. The study of vitrinite is very important for understanding the formation of coal and the maturity of coal.

[0048] Inertinite: Inertinite includes relatively stable organic matter, usually formed at high temperatures, and has strong resistance to decay. This organic matter usually comes from hard plant tissues, such as wood. The presence of inertinite can provide information about the environment when the rock was formed, such as fire and oxygen supply.

[0049] Exinite: Exinite mainly includes some organic substances with shell-like structures, usually from algae, phytoplankton, etc. These organic substances may retain their original shapes during the rock formation process, providing important clues for inferring ancient water environment and vegetation types.

[0050] The analysis of these coal components can be carried out by microscopic observation, spectroscopy, chemical analysis and other methods. The composition of coal has an important influence on its combustion performance, chemical properties and utilization methods, so a deep understanding of coal components is of great significance for the development and utilization of coal. Traditional microscopic component analysis is usually performed manually by highly skilled coal petrologists or geologists, which is time-consuming, labor-intensive and highly subjective. The present invention provides a dual-modal coal rock microscopic component identification method based on a dual decoder fusion UNetFormer architecture for coal rock microscopic component identification, which can achieve pixel-level classification (semantic segmentation) of microscopic components end-to-end, that is, only one neural network can be used to classify different pixel points of the microscopic image into four categories: background resin, vitrinite, exinite and inertinite; at the same time, after fusing fluorescence and reflected light dual-modal microscopic image information and adding an edge decoder to optimize the edge recognition results of the microscopic components, the present invention can complete the pixel-level classification task of the four microscopic components more finely.

[0051] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0052] Example 1

[0053] The technical solution of the present invention is described in detail below in conjunction with the accompanying drawings, but the content of the present invention is not limited thereto.

[0054] The present invention provides a dual-modal coal and rock microscopic component identification method based on a dual decoder fusion UNetFormer architecture, which can quickly identify and classify coal and rock microscopic components. The specific steps are as follows:

[0055] Step 1: Obtain dual-modal microscopic images of coal and rock samples;

[0056] Step 2: Annotate the dual-modal microscopic image at the pixel level to obtain the coal and rock component category labels of the corresponding pixels;

[0057] Step 3: Input the dual-modal microscopic image into the improved UNetFormer network;

[0058] Step 4: Use the cross entropy loss function and Dice loss function as the main loss, combine the auxiliary losses generated by the two decoder outputs, construct a comprehensive loss function, and perform network training;

[0059] Step 5: Input the coal-rock dual-modal microscopic image into the trained network, identify different microscopic components, and output the recognition results.

[0060] Preferably, the coal rock microscopic image obtained in step 1 is a dual-modal microscopic image, that is, the present invention uses microscopic images under fluorescence and reflected light to perform information fusion in the channel dimension to better identify microscopic components.

[0061] Preferably, the component categories in step 2 are background resin, exinite, vitrinite and inertinite.

[0062] Preferably, the improved UNetFormer network described in step 3 is as follows Figure 1 As shown, its specific structure is:

[0063] (1) A lightweight CNN-based encoder based on ResNet18 is used. ResNet enables the creation of deeper networks, making them easier to train and performing better on challenging tasks. ResNet18 is a lightweight version of ResNet that is particularly suitable for training on small datasets, such as the micrographs used in this study. It consists of four stages of Resblocks, each of which downsamples the feature map by a factor of 2. The feature maps generated in each stage are connected to the two decoders through skip connections.

[0064] (2) A Transformer-based decoder is used, which combines three global-local transformer blocks (GLTB) and a feature refinement head (FRH) to build a four-stage lightweight decoder. The overall structure of the decoder is Figure 1The design allows the decoder to capture global and local context at multiple scales while maintaining computational efficiency. The feature maps of Resblocks and GLTB are fused through a learnable weighted sum operation, and the weights are dynamically adjusted according to their contribution to segmentation accuracy. GLTB adopts efficient global-local attention, including a global branch for global context and a local branch for local context, to effectively capture inter-window relationships. UNetFormer further narrows the gap between the spatial information of shallow convolutional features and the semantic information of deep GLTB features through FRH. The specific structures of GLTB and FRH are respectively described in. Figure 2 and Figure 3 Displayed in.

[0065] (3) To improve the accuracy of coal-rock component segmentation, we not only use UNetFormer to extract global-local information, but also introduce an additional edge decoder on this basis to improve edge recognition. It has the following two main functions: first, by introducing an auxiliary loss function specific to the edge decoder, it helps the encoder optimize parameters; second, it predicts the edge position of the microscopic component and further optimizes the segmentation image of the Transformer-based decoder through the attention mechanism based on this.

[0066] As mentioned above, the encoder ResNet18 has four stages, which output feature maps of different scales. Shallow features mainly contain fine-grained spatial information, while deep features contain rich semantic information. In the coal and rock micrograph segmentation task, in order to accurately identify edges, it is necessary to fuse semantic and spatial features of different scales. We adopt a shallow-to-deep edge decoder structure instead of a common decoder structure, such as Figure 1 The edge decoder part and Figure 4 As shown. The feature maps of the four stages of Resblock are respectively input into the corresponding edge extraction block (E2B), and then accumulated layer by layer from shallow to deep layers. In order to prevent the influence of accumulation, they will enter E2B again after accumulation. This edge decoder can simultaneously obtain accurate edge information from the low stage and semantic information from the high stage, and eliminate some original edges that lack semantic information. In this way, high-level semantic information can refine detailed edge information in each stage. At the same time, in order to further enhance the extraction of edge information, we introduce Sobel blocks for the features of each stage. The detailed structure of E2B and Sobel blocks is shown in Figure 4 As shown in Figure 2. During the training process, the edge GroundTruth can be obtained from the semantic segmentation GroundTruth through traditional image processing methods.

[0067] (4) Use the spatial and channel attention block to fuse the outputs of the two decoders to obtain the final coal rock microscopic component recognition results. We use the spatial and channel attention block for feature fusion, such as Figure 5 Specifically, by adding the output features of the two decoders and applying 1×1 convolution and adaptive average pooling, the corresponding weights in the spatial and channel dimensions are generated. These weights are then used to weight the segmentation results, effectively combining the output information of the two decoders to obtain the final segmentation result.

[0068] Preferably, the encoder of the improved UNetFormer network in step 3 obtains the features of the coal and rock samples through Resblocks and is connected to the Transformer-based decoder through a jump connection. When the edge decoder obtains the edge features of the microscopic components, it extracts semantic and spatial features of different scales through E2B and Sobel modules. The output of the Transformer-based decoder and the output of the edge decoder are weighted fused through the Spatial and ChannelAttention Block to obtain accurate microscopic component recognition results.

[0069] Preferably, the comprehensive loss function constructed by introducing the auxiliary loss function in step 4 is calculated as follows:

[0070]

[0071] in, is a constant, is the comprehensive loss function. represents the auxiliary loss at the edge decoder output, represents the auxiliary loss based on the Transformer decoder output, Represents the auxiliary loss of the auxiliary output head of the Transformer decoder. All three auxiliary losses are cross entropy losses. . The main loss function used to deal with class imbalance problems is Loss by Dice and cross entropy loss The loss calculation formulas are as follows:

[0072]

[0073] in, and is a constant, represents the main loss function, represents the cross entropy loss, represents the Dice loss, represents the total number of pixels in each batch of images, represents the number of categories, Represents pixels Belongs to category The predicted probability of Represents pixels Belongs to category The actual label.

[0074] Preferably, the training of the network in step 4 adopts a five-fold cross-validation strategy to ensure the independence of the data set and provide a more comprehensive and objective network performance evaluation. During the training process, the AdamW optimizer was used with an initial learning rate of 2×10 -4 ,The network model is trained on a computer, achieving efficient feature learning and recognition.

[0075] Preferably, when performing network training in step 4, performance evaluation indicators including pixel accuracy (PA), mean pixel accuracy (MPA), and mean intersection over union (MIOU) are used to evaluate the performance of the network in the task of identifying microscopic components of coal and rock. The formulas of each evaluation indicator are as follows:

[0076]

[0077] In the above formula, we assume that there is t Class, in the present invention there are 4 classes. P mm Indicates that it belongs to the class m and is predicted to be class m The number of pixels, P mn Indicates that it belongs to the class m But it is predicted to be n The number of pixels.

[0078] Preferably, the classification categories of the microscopic components implemented in step 5 include four categories: background resin, exinite, vitrinite and inertinite.

[0079] We conducted extensive experimental evaluations on the proposed method to verify its performance in identifying components in coal and rock microscopic images. The experimental data includes coal and rock microscopic images under two lighting conditions: reflected light and fluorescent light. We compared the model with a variety of advanced microscopic component identification methods, including Res-Unet, DeepLabV3+, and Res-Unet-EAGS. In the following experimental results, we will focus on the performance of our model under two lighting conditions and the comparison results with other methods.

[0080] In Table 1, we compare the proposed model with three other recently published coal and rock microscopic image component recognition methods on single modal data, all of which use advanced deep learning techniques and convert the component recognition problem into a pixel-level segmentation task. These methods achieved significant segmentation performance under reflected light conditions, but the proposed method surpassed these methods in terms of accuracy, average precision, and mean intersection over union (PA, mPA, and mIoU), demonstrating the excellent performance of our model in feature extraction and representation.

[0081] In addition, in Table 2, we show the performance comparison of our model on unimodal and bimodal data. On the bimodal dataset, our model achieved 95.34%, 81.10%, and 75.07% accuracy on PA, mPA, and mIoU, respectively, which is 7.74%, 14.98%, and 18.06% higher than that on the unimodal dataset, respectively. This demonstrates the importance of bimodal data in improving model performance.

[0082] We compare our model with other commonly used semantic segmentation methods in Table 3. Under bimodal data, our model achieves 95.34%, 81.10%, and 75.07% accuracy on PA, mPA, and mIoU, respectively, outperforming other methods. Figure 6 The comparison of the recognition results of the method of the present invention and other methods is shown. The first two rows and the last two rows in the figure belong to two different examples. Sub-images (a) and (b) are micrographs under reflected light and fluorescence, respectively; sub-image (c) is the corresponding label; sub-images (d), (e) and (f) respectively represent the microscopic component recognition results of DeepLabV3+, Res-Unet-EAGS and the method proposed in the present invention. The rectangular boxes in sub-images (d), (e) and (f) indicate that the recognition results of the areas within the rectangular boxes are significantly different from the actual situation. These examples further demonstrate the advantages of our model in processing complex coal and rock microscopic images.

[0083] Table 1: Performance comparison with state-of-the-art microscopic component identification methods (single modality)

[0084]

[0085] Table 2: Performance comparison of the proposed method in unimodal and bimodal data

[0086]

[0087] Table 3: Performance comparison of different segmentation networks in a bimodal setting

[0088]

[0089] The above schematically describes the invention and its implementation methods, which is not restrictive. Without departing from the spirit or basic features of the invention, the invention can be implemented in other specific forms. What is shown in the accompanying drawings is only one of the implementation methods of the invention. The actual structure is not limited to this, and any figure mark in the claims should not limit the claims involved. Therefore, if a person of ordinary skill in the art is inspired by it, without departing from the purpose of the invention, a structural method and an embodiment similar to the technical solution are designed without creativity, which should all fall within the scope of protection of this patent. In addition, the word "including" does not exclude other elements or steps, and the word "one" before the element does not exclude the inclusion of "multiple" elements. The multiple elements stated in the product claim can also be implemented by one element through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any specific order.

Claims

1. A dual-modal coal and rock microscopic component identification method based on a dual decoder fusion UNetFormer architecture, comprising the following steps: Step 1: Obtain a dual-modal microscopic image of a coal rock sample; the dual-modal microscopic image of the coal rock sample obtained is a dual-modal microscopic image obtained by fusing information of microscopic images under fluorescence and reflected light in the channel dimension; Step 2: Annotate the dual-modal microscopic image at the pixel level to obtain the coal and rock component category labels of the corresponding pixels; Step 3: Input the bimodal microscopic image into the improved UNetFormer network; the improved UNetFormer network adopts a lightweight CNN-based encoder based on ResNet18, which contains multiple Resblocks, each Resblock performs 2x downsampling on the feature map and is connected to two decoders through jump connections; a Transformer-based decoder is adopted, including three GLTBs and one FRH, which is used to capture global and local context information at multiple scales; an edge decoder is also included, which is used to extract edge features of microscopic components, which is connected to the encoder of UNetFormer through jump connections; the outputs of the Transformer-based decoder and the edge decoder are fused using a spatial channel attention module to obtain the final recognition result of coal and rock microscopic components; Step 4: Use the cross entropy loss function and Dice loss function as the main loss, combine the auxiliary loss generated by the dual decoder output, construct a comprehensive loss function, and perform network training; Step 5: Input the coal-rock dual-modal microscopic image into the trained network, identify different microscopic components, and output the recognition results.

2. According to claim 1, a dual-modal coal and rock microscopic component identification method based on dual decoder fusion UNetFormer architecture is characterized in that: The component type in step 2 is one of background resin, exinite, vitrinite or inertinite.

3. According to claim 2, a dual-mode coal and rock microscopic component identification method based on dual decoder fusion UNetFormer architecture is characterized in that: The comprehensive loss function constructed in step 4 is calculated as follows: Among them, γ is a constant, is the comprehensive loss function, represents the auxiliary loss at the edge decoder output, represents the auxiliary loss based on the Transformer decoder output, Represents the auxiliary loss of the auxiliary output head of the Transformer decoder. All three auxiliary losses are cross entropy losses. , and the main loss function used to deal with class imbalance problems Loss by Dice and cross entropy loss The loss calculation formula is as follows: Among them, α and β are constants, represents the main loss function, represents the cross entropy loss, represents the Dice loss, N represents the total number of pixels in each batch of images, and K represents the number of categories. represents the predicted probability that pixel n belongs to category k, represents the actual label that pixel n belongs to category k.

4. A dual-modal coal and rock microscopic component identification method based on a dual decoder fusion UNetFormer architecture according to any one of claims 1 to 3, characterized in that: The network was trained using a five-fold cross-validation strategy.

5. According to claim 4, a dual-modal coal and rock microscopic component identification method based on dual decoder fusion UNetFormer architecture is characterized in that: During the training of the network, the AdamW optimizer was used with an initial learning rate of 2×10 -4 .

Images