Coke microscopic optical tissue extraction method based on multi-attention neural network

Abstract

The invention discloses a coke microscopic optical tissue extraction method based on a multi-attention neural network, and the method comprises the following steps: obtaining and cutting a coke microscopic image, and constructing a data set containing a manually labeled optical tissue label and a background label; dividing the data set into a training set, a verification set and a test set, and performing data enhancement on the training set; constructing a multi-attention U-shaped neural network model, wherein the multi-attention U-shaped neural network model comprises an encoder, a bridging module, a bottleneck module and a decoder which are connected in sequence; a training set is adopted to train the multi-attention U-type neural network model, multi-scale prediction masks are generated at all stages of a decoder through a deep supervision mechanism, and the multi-scale prediction masks and final output jointly participate in loss function calculation; and inputting a to-be-processed coke microscopic image into the trained multi-attention U-shaped neural network model, and outputting an optical tissue extraction result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of coke microscopic image processing, in particular to a coke microscopic optical structure extraction method based on a multi-attention neural network. BACKGROUND

[0002] Metallurgical coke is a key raw material in the blast furnace ironmaking process, and its quality directly affects the ironmaking efficiency and energy consumption. Accurate analysis of the coke microscopic optical structure is an important basis for evaluating its quality. The traditional method mainly relies on the manual identification and statistics under the microscope according to the "Determination method of coke optical structure" (YB / T 077-2017), which is not only time-consuming and laborious, but also the analysis results are greatly influenced by the subjectivity of the operator. With the development of computer vision technology, automated analysis has become possible, but the particularity of the coke microscopic image brings many challenges to automated processing.

[0003] In the coke microscopic image, the contrast between the optical structure and the background is usually low, and due to the impurities such as polishing agents mixed in the preparation process and the exposure phenomenon during shooting, there are a large number of noise interferences in the image. These noises are very similar to the real optical structure in shape and brightness characteristics, making it difficult for traditional segmentation methods such as K-means clustering and watershed algorithm to accurately distinguish them. Although deep learning methods have made some progress in this field in recent years, there are still obvious deficiencies in the existing technology. For example, the feature extraction capability of shallow convolutional neural networks is limited, making it difficult to identify the details of complex structures; although the multi-scale U-shaped neural network can suppress small impurity interference, the segmentation effect of low-contrast regions and impurity reflections is still not ideal; the method based on improved DeepLabv3+ performs well in areas with unclear boundaries, but has poor adaptability to background noise.

[0004] These technical defects mainly manifest in the following aspects: first, in the case of unclear features and close gray scale distribution, the model's perception ability of the foreground region is insufficient; second, when facing interference caused by impurity reflections and exposure phenomenon, it is difficult to retain reliable structural information; finally, when the target region and the interference region are highly similar in color, the existing methods have weak discrimination ability for fine-grained features. These problems seriously restrict the accuracy and efficiency of coke microscopic optical structure analysis.

[0005] In view of the above problems, the existing technology needs to be improved. SUMMARY

[0006] The purpose of the present application is to provide a coke microscopic optical structure extraction method based on a multi-attention neural network to solve the problems existing in the prior art.

[0007] The above technical purpose of the present application is achieved by the following technical scheme:

[0008] A method for extracting microscopic optical tissue from coke based on a multi-attention neural network includes the following steps:

[0009] S1. Acquire and crop coke microscopic images to construct a dataset containing manually labeled optical tissue labels and background labels;

[0010] S2. Divide the dataset into training, validation and test sets, and perform data augmentation on the training set;

[0011] S3. Construct a multi-attention U-shaped neural network model, wherein the multi-attention U-shaped neural network model includes an encoder, a bridging module, a bottleneck module, and a decoder connected in sequence;

[0012] S4. The multi-attention U-shaped neural network model is trained using a training set. Multi-scale prediction masks are generated at each stage of the decoder through a deep supervision mechanism. The multi-scale prediction masks and the final output are used together to calculate the loss function.

[0013] S5. Input the microscopic image of the coke to be processed into the trained multi-attention U-shaped neural network model, and output the optical tissue extraction results.

[0014] Furthermore, the multi-attention U-shaped neural network model includes:

[0015] The encoder contains six feature extraction layers, including:

[0016] The first to third layers consist of standard convolutional layers and max pooling layers;

[0017] The fourth and fifth layers consist of a cross-group connection attention module and a max pooling layer;

[0018] The sixth layer consists of cross-group connection attention modules;

[0019] Bridge module: contains six layers of adaptive hierarchical fusion attention modules, which connect to adjacent layers of the encoder respectively;

[0020] Bottleneck module: Composed of bottleneck attention modules;

[0021] Decoder: Contains a six-layer feature reconstruction module, where:

[0022] The first to third layers consist of standard convolutional layers and bilinear interpolation upsampling layers;

[0023] The fourth and fifth layers consist of a cross-group connection attention module and a bilinear interpolation upsampling layer;

[0024] The sixth layer consists of cross-group connection attention modules.

[0025] Furthermore, the cross-group connection attention module includes:

[0026] Learnable shared space parameter layer;

[0027] A multiplier that divides the input feature map into multiple feature groups and multiplies them element-wise with features in the shared space;

[0028] Depth-separable convolutional layers for local context extraction of grouped features;

[0029] A connection layer that enables cross-group feature interaction;

[0030] A stitching layer that performs channel stitching on the interactive features;

[0031] Features are integrated through 1×1 convolutions, and the output is activated by a non-linear layer.

[0032] Furthermore, the adaptive hierarchical fusion attention module includes:

[0033] An adaptive spatial attention mask generator for generating spatial attention masks;

[0034] A fusion layer that groups and splices together high-order and low-order features;

[0035] A feature selector that suppresses noise using spatial attention masking;

[0036] A feature extraction layer that extracts contextual information through multi-scale grouped convolution;

[0037] An integration layer that outputs a unified feature map through 1×1 convolution.

[0038] Furthermore, the bottleneck attention module includes:

[0039] Channel attention branch: includes a global average pooling layer and a fully connected layer, which generate channel weights and weighted features;

[0040] Spatial attention branch: Contains dimensionality reduction convolutional layers and dilated convolutional layers to generate a spatial attention map;

[0041] Fusioner: Adds the outputs of the two branches to form a fused attention map, and multiplies it element-wise with the input features;

[0042] Residual connection structures that retain supplementary features.

[0043] Furthermore, the data augmentation operation in step S2 includes:

[0044] A combination of horizontal flip, vertical flip, and rotation operations is randomly performed with probability p = 0.5.

[0045] Furthermore, the loss function in step S4 is a hybrid weighted loss function, which is composed of the Dice loss function and the binary cross-entropy loss function, and the multi-stage prediction results are weighted and calculated.

[0046] Furthermore, step S5 specifically includes:

[0047] S51. The encoder extracts low-order features through standard convolution and max pooling, and extracts high-order features through a cross-group connection attention module;

[0048] S52. Higher-order features are jointly modeled in the channel-space manner using the bottleneck attention module;

[0049] S53. Features from adjacent encoder levels are fused via skip connections using an adaptive hierarchical fusion attention module;

[0050] S54. The decoder restores the resolution through bilinear interpolation upsampling and cross-group connection attention modules, and outputs the segmentation result after fusing skip connection features.

[0051] In summary, the present invention has the following beneficial effects:

[0052] By constructing a multi-attention U-shaped neural network model, and combining cross-group connection attention modules, adaptive hierarchical fusion attention modules, and deep supervision mechanisms, the traditional methods effectively solve the problems of insufficient perception of low-contrast regions and poor noise interference suppression. It has the advantages of improving the accuracy of optical tissue recognition, effectively suppressing noise interference, and enhancing the ability to extract complex features. Attached Figure Description

[0053] Figure 1 This is a schematic diagram of a coke micrograph used in this invention.

[0054] Figure 2 This is a structural diagram of the multi-attention U-shaped neural network of the present invention.

[0055] Figure 3 This is a structural diagram of the cross-group connection attention module used in this invention.

[0056] Figure 4 This is a structural diagram of the adaptive hierarchical fusion attention module used in this invention.

[0057] Figure 5 This is a structural diagram of the bottleneck attention module used in this invention.

[0058] Figure 6 This is a comparison diagram of the extraction results of coke microscopic optical structures using the network model described in this invention and the U-Net network.

[0059] Figure 7This is a flowchart of the coke microscopic optical tissue extraction method based on a multi-attention neural network as described in this invention. Detailed Implementation

[0060] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below with reference to the figures and specific embodiments.

[0061] In existing technologies, coke microscopic image analysis relies on manual microscopic identification and statistics, which suffers from high subjectivity and low efficiency. Traditional segmentation methods such as K-means clustering and watershed algorithms suffer from severe missegmentation under noise interference. Existing deep learning methods such as shallow convolutional neural networks are insufficient for recognizing complex tissue details, and multi-scale U-shaped networks have limited segmentation accuracy in low-contrast regions. Improvements are needed.

[0062] The DeepLabv3+ model exhibits poor adaptability to background noise interference. When there is polishing agent residue, noise interference caused by reflective exposure in the coke micrograph, or when the grayscale distribution of the target area is similar to that of the background, existing methods struggle to accurately distinguish between optical structures and interfering factors.

[0063] To address the aforementioned issues, we first observed that the core challenges in coke microscopic image segmentation lie in feature similarity interference and the weakening of effective features. Traditional U-shaped networks do not consider noise suppression during cross-level feature fusion, resulting in the reconstruction of features containing interfering information during the decoding stage. Based on this, we attempted to introduce a grouped feature interaction mechanism in the encoding stage to enhance the expression of effective features through cross-group connections. We further discovered that direct concatenation of high- and low-order features during skip connections introduces noise, necessitating the design of an adaptive fusion strategy. To address the gradient decay problem in deep networks, we proposed a multi-scale supervision mechanism to promote feature learning. Ultimately, we arrived at a complete solution incorporating the synergistic effect of multiple attention modules and deep supervision.

[0064] Referring to the figure, this application proposes a method for extracting optical microstructures from coke based on a multi-attention neural network, including the following steps:

[0065] S1. Acquire and crop coke microscopic images to construct a dataset containing manually labeled optical tissue labels and background labels;

[0066] S2. Divide the dataset into training, validation and test sets, and perform data augmentation on the training set;

[0067] S3. Construct a multi-attention U-shaped neural network model, wherein the multi-attention U-shaped neural network model includes an encoder, a bridging module, a bottleneck module, and a decoder connected in sequence;

[0068] S4. The multi-attention U-shaped neural network model is trained using a training set. Multi-scale prediction masks are generated at each stage of the decoder through a deep supervision mechanism. The multi-scale prediction masks and the final output are used together to calculate the loss function.

[0069] S5. Input the microscopic image of the coke to be processed into the trained multi-attention U-shaped neural network model, and output the optical tissue extraction results.

[0070] Data augmentation refers to increasing sample diversity through random geometric transformations, specifically through a combination of horizontal flipping, vertical flipping, and rotation operations. The augmentation strength is controlled probabilistically to balance generalization ability and overfitting risk. The multi-attention U-shaped neural network model refers to an encoder-decoder structure with multi-level attention mechanisms. Specifically, it achieves feature interaction through cross-group connection attention modules, noise suppression through adaptive hierarchical fusion attention modules, and channel-space joint modeling through a bottleneck attention module. Deep supervision mechanisms generate prediction masks at each stage of the decoder, specifically by backpropagating gradient information through multi-scale supervision signals to promote the optimization of shallow network parameters.

[0071] Specifically, the encoder extracts image features layer by layer through standard convolution and pooling operations. In layers four through six, a cross-group connection attention module is introduced to divide the input features into multiple subgroups for interactive learning, enhancing the ability to perceive features in low-contrast regions. The bridging module uses an adaptive hierarchical fusion attention mechanism to weightedly fuse features from adjacent encoder layers, suppressing noise propagation during skip connections. The bottleneck module employs joint channel and spatial attention modeling to strengthen the representation of key features. The decoder restores feature resolution through upsampling operations and reconstructs the segmentation result by combining the spatial information provided by skip connections. During training, the multi-scale predicted masks output by the decoder at each stage participate in loss calculation along with the final output, forming a feature optimization path from coarse to fine.

[0072] Compared to existing technologies, traditional shallow convolutional networks struggle to capture global contextual information due to their limited receptive field. This solution achieves multi-scale feature interaction by connecting attention modules across groups. Conventional U-shaped networks directly concatenate high- and low-order features, leading to noise amplification. This solution employs an adaptive hierarchical fusion strategy for feature optimization. Existing single-stage supervision methods are prone to gradient vanishing in deep networks; this solution achieves gradient equalization propagation through a multi-scale supervision mechanism.

[0073] Through the above technical solutions, this application can effectively distinguish optical tissues from noise interference with similar morphology in coke microscopic images, achieve accurate boundary division in low-contrast areas, suppress artifact interference caused by reflective exposure, and improve the stability and accuracy of microscopic tissue extraction under complex imaging conditions.

[0074] This application further proposes a multi-attention U-shaped neural network model, which includes an encoder, a bridging module, a bottleneck module, and a decoder. The encoder comprises six feature extraction layers: layers one through three consist of standard convolutional layers and max-pooling layers; layers four through five consist of cross-group connection attention modules and max-pooling layers; and layer six consists of a cross-group connection attention module. The bridging module comprises six adaptive hierarchical fusion attention modules, connecting adjacent layers of the encoder. The bottleneck module consists of a bottleneck attention module. The decoder comprises six feature reconstruction layers: layers one through three consist of standard convolutional layers and bilinear interpolation upsampling layers; layers four through five consist of cross-group connection attention modules and bilinear interpolation upsampling layers; and layer six consists of a cross-group connection attention module.

[0075] The cross-group connection attention module divides the input feature map into multiple feature groups, achieves feature interaction through a shared spatial parameter layer, and extracts local context using depthwise separable convolution. Specifically, it can use a learnable shared spatial parameter layer to multiply the grouped features element-wise, and then use depthwise separable convolution and cross-group connections to enhance features, thereby enhancing the response of effective features in low-contrast regions. The adaptive hierarchical fusion attention module dynamically suppresses noise interference and fuses high- and low-order features by generating a spatial attention mask. Specifically, it can use multi-scale grouped convolution to extract contextual information and select effective features through the spatial attention mask, thereby improving the reliability of feature fusion during skip connections. The bottleneck attention module jointly models channel attention and spatial attention. Specifically, it can use global average pooling to generate channel weights and combine them with dilated convolution to generate a spatial attention map, thereby enhancing the ability to distinguish texture differences in color-similar regions.

[0076] Specifically, the encoder's first three layers extract basic features through standard convolution and pooling operations. The fourth and fifth layers introduce a cross-group connection attention module, enhancing the perception of low-contrast regions through feature grouping and interaction. The sixth layer cancels pooling to preserve complete spatial information. The adaptive hierarchical fusion attention module in the bridging module suppresses noise interference such as impurities and reflections through spatial attention masks during skip connections, while simultaneously employing multi-scale grouped convolution to fuse contextual features from different levels. The decoder's first three layers restore resolution through standard convolution and upsampling, while the last three layers combine the cross-group connection attention module to reconstruct detailed features, avoiding information loss during upsampling. The bottleneck module enhances the differentiation of regions with similar colors but different textures through joint modeling of channel attention and spatial attention. Through a collaborative mechanism of deep attention in the encoder enhancing feature discrimination, dynamic interference suppression in the bridging module, and attention-assisted detail restoration in the decoder, effective modeling of complex microscopic image features is achieved.

[0077] Compared to existing technologies, traditional U-shaped neural networks directly concatenate high- and low-order features during skip connections, neglecting the impact of noise interference on feature fusion and lacking targeted processing for low-contrast regions. This solution introduces a layered cross-group connection attention module to enhance feature interaction capabilities at a deeper level of the encoder; it employs an adaptive layered fusion attention module to dynamically suppress noise interference in skip connections; and it combines channel and spatial attention for joint modeling at the bottleneck layer to improve the ability to capture subtle texture differences. Compared to a single attention mechanism or shallow network structure, this solution optimizes feature extraction and fusion under complex interference conditions through multi-module collaboration.

[0078] Through the above technical solutions, this application can effectively improve the feature response intensity of low-contrast regions in coke micrographs, suppress the interference of impurity reflection and exposure noise on feature fusion, and enhance the ability to identify texture differences in areas with similar colors. The encoder's deep attention module can capture richer contextual information, the bridging module's adaptive fusion mechanism reduces invalid feature transmission, and the decoder's attention assists in recovering detailed features, thereby achieving high-precision segmentation of optical structures under complex interference conditions.

[0079] This application further proposes a cross-group connection attention module, which includes a learnable shared space parameter layer, a multiplier that divides the input feature map into multiple feature groups and multiplies them element-wise with the shared space features, a deep separable convolutional layer that extracts local context for grouped features, a connection layer that enables cross-group feature interaction, a concatenation layer that performs channel concatenation on the interacted features, and a nonlinear layer that integrates features through 1×1 convolution and activates the output.

[0080] The learnable shared spatial parameter layer refers to a learnable parameter layer that dynamically adjusts spatial weights. This can be achieved by dynamically generating convolutional kernel parameters, and is used to establish global feature relationships. The multiplier that divides the input feature map into multiple feature groups and multiplies them element-wise with shared spatial features refers to grouping the feature map by channel dimension and then performing element-wise multiplication with the shared features. This can be implemented using a grouped matrix multiplier, and is used to enhance local features guided by global features. The depthwise separable convolutional layer refers to a lightweight structure that decomposes standard convolution into depthwise convolution and pointwise convolution. This can be implemented by concatenating 3×3 convolutional kernels and 1×1 convolutions, and is used to extract local contextual information. The cross-group feature interaction connection layer establishes information transmission channels between different feature groups. This can be implemented using a cross-group skip connection structure, and is used to promote the fusion of complementary features between groups. The channel concatenation layer re-integrates multiple groups of features along the channel dimension. This can be implemented using tensor concatenation operations, and is used to form composite semantic features. A 1×1 convolutional integration layer refers to adjusting the channel dimension of the concatenated features. Specifically, it can be implemented using a single-layer convolutional network structure to generate discriminative attention weights.

[0081] Specifically, in low-contrast scenes of coke micrographs, the shared spatial parameter layer establishes relationships between different regions by dynamically learning global spatial weights. After the input feature map is divided into multiple feature groups, each group feature is multiplied element-wise with the shared spatial feature, allowing global contextual information to guide local feature enhancement, thereby amplifying subtle differences between the foreground and background. Depthwise separable convolutional layers process each group feature independently, extracting local contextual information while maintaining parameter efficiency, enhancing the ability to capture blurred boundaries. Cross-group feature interaction connection layers establish inter-group information transmission channels, enabling different feature groups to share key information; for example, edge features detected by one group can be passed to adjacent groups for shape completion. After interaction, multiple groups of features are concatenated through channels to form composite features, which are then integrated and non-linearly activated by a 1×1 convolutional layer, ultimately generating an attention weight map that focuses on key regions.

[0082] Compared to existing technologies, traditional grouped convolution methods typically process each feature group independently, leading to isolated information between groups. Our proposed solution establishes global feature relationships through a shared spatial parameter layer and combines this with cross-group interactive connection layers to achieve complementary information between groups. This allows the model to comprehensively utilize both global context and local detail features even under low-contrast conditions. Existing methods that simply use standard convolution or ordinary attention mechanisms struggle to achieve fine-grained feature enhancement while maintaining computational efficiency. Our solution, however, combines depthwise separable convolution with grouped processing, reducing computational cost while improving the ability to perceive subtle features.

[0083] Through the above technical solutions, this application effectively enhances the model's feature perception capability in low-contrast regions, accurately distinguishes the grayscale distribution differences between the foreground and background, and improves the positioning accuracy of blurred boundaries. In cases where exposure interference or impurity reflection exists in coke micrographs, this module significantly reduces the missegmentation rate and achieves accurate extraction of optical tissue regions through cross-group feature interaction and global-local feature fusion mechanisms.

[0084] This application further proposes an adaptive hierarchical fusion attention module, which includes an adaptive spatial attention mask generator that generates spatial attention masks, a fusion layer that groups and concatenates high-order and low-order features, a feature selector that uses spatial attention masks to suppress noise, a feature extraction layer that extracts contextual information through multi-scale grouped convolutions, and an integration layer that outputs a unified feature map through 1×1 convolutions.

[0085] The adaptive spatial attention mask generator dynamically generates spatial attention weights based on the contextual information of the input feature map. This can be implemented using a combination of global average pooling and fully connected layers to locate noisy regions and generate corresponding suppression weights. The fusion layer groups and concatenates feature maps from different stages of the encoder, typically using channel-dimensional grouping and concatenation to integrate semantic information at different levels. The feature selector uses a spatial attention mask to weight and filter the fused features, typically through channel-wise multiplication, to suppress the response intensity in noisy regions. The multi-scale grouped convolutional layer uses depthwise separable convolutions with different kernel sizes to process features in parallel, typically using 3×3 or 5×5 convolutional kernels in parallel, to capture contextual information at different scales. The integration layer compresses multi-scale features using channels, typically with 1×1 convolutional kernels, to generate a unified-dimensional output feature map.

[0086] Specifically, the adaptive spatial attention mask generator first analyzes the global information of the input feature map to generate a spatial attention mask reflecting the noise distribution. The fusion layer concatenates high-order semantic features and low-order detail features of the encoder by channel grouping, forming a fused feature containing multi-level information. The feature selector applies the spatial attention mask to the fused feature, reducing activation values ​​in noisy regions while preserving response strength in effective regions. Multi-scale grouped convolutional layers extract local details and large-scale contextual features separately through parallel execution of depthwise separable convolutions of different sizes, enhancing the model's ability to distinguish complex textures. The integration layer compresses and fuses the multi-scale features by channel compression, generating an output feature map with unified representation capabilities. This process effectively solves the feature confusion problem caused by impurity reflections by dynamically suppressing noise interference and fusing multi-scale features.

[0087] Compared to existing technologies, traditional methods typically employ fixed thresholds or single-scale convolutions to handle noise interference, making it difficult to adapt to the dynamic changes in noise morphology in coke micrographs. Existing skip connection structures directly add or concatenate features without considering the differences in noise distribution across different feature levels. This approach dynamically adjusts feature weights through an adaptive spatial attention mask and enhances contextual modeling capabilities by combining multi-scale grouped convolutions, thus suppressing noise while preserving key structural details.

[0088] Through the above technical solution, this application can effectively identify bright spot noise in coke micrographs caused by polishing agent residue or exposure, and actively suppress the response intensity of interference regions during the feature fusion stage. The introduction of multi-scale grouped convolution enhances the ability to capture edge features of low-contrast regions, avoiding the loss of details caused by traditional single-scale convolution. This module achieves high-precision differentiation between noise interference regions and real optical structures under complex lighting conditions, significantly improving the reliability of the segmentation results.

[0089] This application further proposes a bottleneck attention module comprising a channel attention branch, a spatial attention branch, a fusion unit, and a residual connection structure. The channel attention branch includes a global average pooling layer and a fully connected layer to generate channel weights and weighted features; the spatial attention branch includes a dimensionality-reducing convolutional layer and a dilated convolutional layer to generate a spatial attention map; the fusion unit adds the outputs of the two branches to form a fused attention map and multiplies it element-wise with the input features; the residual connection structure preserves supplementary features.

[0090] The channel attention branch involves compressing the channel dimension of the input feature map using a global average pooling layer, followed by generating channel weight coefficients through a fully connected layer. This can be achieved by adding a non-linear activation function between two fully connected layers, enhancing the ability to represent key channel features. The spatial attention branch reduces the number of channels using dimensionality-reducing convolutions, then expands the receptive field using dilated convolutions. This can be achieved using 1×1 convolutions for dimensionality reduction and 3×3 convolutions with a dilation rate of 2 for spatial feature extraction, capturing a wider range of contextual spatial information. The fusion layer adds the channel attention weights and spatial attention map element-wise to form a joint attention map, achieved using tensor addition. This structure enables synergistic feature enhancement of both channel and spatial dimensions. The residual connection structure superimposes the original features before attention is applied with the attention-weighted features, typically achieved using skip connections. This structure preserves detailed features not captured by the attention mechanism.

[0091] Specifically, the channel attention branch acquires global statistical features of each channel through global average pooling. The fully connected layer learns the dependencies between channels to generate weight coefficients, and reweights the input features according to the channel dimension, strengthening the feature representation of important channels. The spatial attention branch reduces computational complexity through dimensionality reduction convolution and expands the receptive field using dilated convolution to extract a spatial attention map containing multi-scale contextual information. The fusion unit superimposes the channel weights with the spatial attention map to form a joint attention mechanism with both channel sensitivity and spatial localization capabilities, achieving feature recalibration by multiplying with the input features. The residual connection adds the original features to the attention-weighted features, preserving details not focused on by the attention mechanism and avoiding the loss of effective features due to excessive suppression by the attention mechanism. Through the synergistic effect of the dual-branch attention and residual structure, this module can effectively distinguish subtle texture differences in highly similar color regions, improving the ability to distinguish regions with weak detail interference.

[0092] Compared to existing technologies, traditional methods typically employ only a single attention mechanism when processing color-similar regions, such as channel attention or spatial attention alone, making it difficult to simultaneously consider global channel features and local spatial context information. While some improved schemes attempt to combine two attention mechanisms, the cascaded structure increases computational complexity, and the lack of residual protection mechanisms easily leads to the loss of detailed features. This application achieves simultaneous computation of channel and spatial attention through a parallel dual-branch structure, uses additive fusion to reduce computational load, and combines residual connections to preserve original features, thereby improving the fine-grained recognition capability of color-similar regions while ensuring computational efficiency.

[0093] Through the above technical solution, this application can effectively distinguish between optical structures and interference areas in coke micrographs that are similar in color but differ in texture details, reducing missegmentation caused by color similarity. When the target area has weak detail information, the application enhances the representation of key features through joint channels and spatial attention mechanisms, while utilizing residual structures to supplement potentially lost detail information, significantly improving the segmentation accuracy for weak detail interference areas.

[0094] This application further proposes that the data augmentation operation in step S2 includes a combination of horizontal flipping, vertical flipping and rotation operations performed randomly with probability p = 0.5.

[0095] Horizontal flipping refers to mirroring the image along the vertical axis, which expands the model's ability to recognize left-right symmetry features in coke microstructures. Vertical flipping refers to mirroring the image along the horizontal axis, which enhances the model's ability to perceive vertically symmetrical structures. Rotation refers to rotating the image at random angles, which can be implemented using an affine transformation matrix. This operation simulates the directional randomness that may exist in coke microstructures during sample preparation. A probability p = 0.5 means that there is a 50% chance that each data augmentation operation will be performed, which can be implemented using a random number generator. This parameter setting can balance the relationship between data augmentation strength and feature fidelity.

[0096] Specifically, in the preprocessing stage of coke micrographs, diverse training samples are generated by randomly combining three geometric transformation operations. The combination of horizontal and vertical flipping simulates different orientations of the sample on the microscope stage, while rotation covers any angular distribution of coke particles that may occur during sample preparation. Probability parameters control the execution probability of each enhancement operation, ensuring that different enhancement combinations are obtained in each training cycle while avoiding excessive distortion of the original image features. This enhancement strategy, by introducing controllable geometric deformation, exposes the model to more diverse data distributions during training, thereby improving its generalization ability to noise patterns and low-contrast region features in coke micrographs caused by polishing agent residue, reflective interference, etc.

[0097] Compared with existing technologies, this solution uses a combination of geometric transformations and probabilistic control mechanisms to more accurately simulate the spatial distribution characteristics of actual samples, maximizing the diversity of training samples while ensuring data authenticity.

[0098] Through the above technical solution, this application effectively solves the problem of limited model generalization ability caused by insufficient training data diversity, enabling the neural network to learn the invariant characteristics of coke microstructure under different spatial orientations, significantly improving the model's robustness to image noise interference, enhancing the segmentation accuracy of low-contrast regions, and providing a reliable data foundation for subsequent optical tissue statistics.

[0099] This application further proposes that the loss function in the step is a hybrid weighted loss function, which is composed of the Dice loss function and the binary cross-entropy loss function, and the multi-stage prediction results are weighted and calculated.

[0100] The Dice loss function optimizes the model output by calculating the overlap between the predicted segmented region and the ground truth labeled region. Specifically, it can be implemented mathematically by calculating the ratio of the intersection area to the union area of ​​the predicted mask and the ground truth mask. This function effectively mitigates model training bias caused by the uneven distribution of optical tissue areas in coke microscopic images. The binary cross-entropy loss function calculates the binary classification error for each pixel. Specifically, it can be implemented by calculating the logarithmic loss between the predicted probability and the ground truth label pixel-by-pixel. This function improves the model's ability to distinguish subtle boundaries between coke microscopic tissue and the background. The weighted calculation of multi-stage prediction results assigns different weight coefficients to the multi-scale prediction masks output at each level of the decoder. Specifically, it can be implemented using a linear weighting method, where the prediction results from each level and the final output participate in the backpropagation process. This mechanism enhances the network's ability to capture optical tissue features of different sizes in coke microscopic images through deep supervision.

[0101] Specifically, during model training, the Dice loss function maximizes the intersection-union ratio (IU) between the predicted region and the ground truth annotation, prompting the network to focus on the overall morphological integrity of the coke microstructure and avoiding fragmented segmentation regions due to local noise interference. The binary cross-entropy loss function enhances the model's classification accuracy for edge pixels in low-contrast regions through pixel-by-pixel error backpropagation. The linear combination coefficients of the two loss functions can be set to empirical values, such as a Dice loss weight of 0.6 and a cross-entropy loss weight of 0.4, to balance the dual requirements of region consistency optimization and pixel-level classification accuracy. For the multi-scale prediction mask output by the decoder, a layer-by-layer decreasing weight allocation strategy is adopted, for example, setting the prediction weight of deep features to 0.5, the middle layer to 0.3, and the shallow layer to 0.2, guiding the network to learn discriminative representations at different feature levels through gradient backpropagation.

[0102] This scheme enhances the ability to model regional continuity while maintaining pixel-level classification accuracy through loss function combination and multi-stage supervision mechanism. By weighted fusion of multi-scale prediction results, the model can simultaneously adapt to optical tissue features of different sizes from tens to hundreds of micrometers in coke microscopic images.

[0103] Through the above technical solutions, this application effectively balances the contradictory relationship between pixel-level classification accuracy and region consistency in coke microstructure segmentation tasks, and solves the problem of unstable segmentation results under complex noise interference by traditional single loss functions. The multi-stage prediction weighting mechanism enhances the model's ability to capture multi-scale features in microscopic images, and significantly improves the generalization performance of the segmentation results in scenarios where the optical structure of coke is diverse and impurity interference coexists.

[0104] This application further proposes specific implementation steps for the coke microscopic optical tissue extraction method, including: the encoder extracts low-order features through standard convolution and max pooling, then extracts high-order features through a cross-group connection attention module; the high-order features are jointly modeled by channel and space through a bottleneck attention module; adjacent-level features of the encoder are fused by skip connections through an adaptive hierarchical fusion attention module; and the decoder restores the resolution and fuses the skip connection features through bilinear interpolation upsampling and a cross-group connection attention module to output the segmentation result.

[0105] The cross-group connection attention module is a computational unit that divides the input feature map into multiple feature groups and interacts with shared spatial parameters. Specifically, it can be implemented using a combination of grouped convolution and spatial attention mechanisms, enhancing the feature representation ability of low-contrast regions through local context extraction and cross-group feature interaction. The bottleneck attention module is an attention computation unit that operates simultaneously on both the channel and spatial dimensions. Specifically, it can be implemented using a dual-branch structure that generates channel weights through global average pooling and generates spatial attention maps through dimensionality reduction convolution, addressing feature confusion through joint modeling. The adaptive hierarchical fusion attention module is a feature fusion unit with spatial attention mask generation capabilities. Specifically, it can be implemented using a combination of multi-scale grouped convolution and feature concatenation, preserving effective structural information by suppressing noise interference. Bilinear interpolation upsampling is a computational method that restores resolution based on weighted calculations of adjacent pixels. Specifically, it can use linear interpolation kernel functions to spatially expand low-resolution feature maps, reducing detail loss by maintaining feature continuity.

[0106] Specifically, the encoder extracts low-order features such as edge textures by performing 3×3 convolution operations using standard convolutional kernels. After compressing the spatial dimension through a max pooling layer, the cross-group attention module divides the features into four groups for interaction, enhancing the response intensity of the target region by sharing spatial parameters. Higher-order features are processed in the bottleneck module through global average pooling in the channel branch and 3×1 dilated convolution in the spatial branch, generating channel weight coefficients which are then superimposed and fused with the spatial attention map to highlight effective features and suppress background noise. The features from the third and fourth layers of the encoder are concatenated by an adaptive hierarchical fusion module, and then multi-scale contextual information is extracted using 5×5 group convolutions. Spatial attention masks are used to filter interference signals from polishing reflective areas. During bilinear interpolation upsampling, the decoder gradually restores resolution using a scaling factor of 0.5. The cross-group attention module compensates for the loss of detail caused by pooling operations during the feature reconstruction stage through cross-group interaction, and finally, the multi-scale features provided by skip connections are fused to output a segmentation mask.

[0107] Compared with existing technologies, this scheme employs a bottleneck attention module with channel-space joint modeling, which enhances the feature discrimination capability in low-contrast regions through a dual-branch structure. Conventional decoders using transposed convolution for upsampling are prone to chessboard effects; this scheme uses bilinear interpolation combined with a cross-group connected attention module to maintain feature continuity while restoring resolution.

[0108] Through the above technical solutions, this application solves the problems of indistinct features due to low contrast, missegmentation caused by noise interference, and loss of details caused by pooling operation in coke microscopic images. It achieves segmentation results with clear optical tissue boundaries and complete structure of coke under complex imaging conditions, and significantly improves the accuracy and anti-interference ability of microscopic tissue extraction.

[0109] Example

[0110] A method for extracting microscopic optical tissue from coke based on a multi-attention U-shaped neural network includes the following steps:

[0111] S1. Use a polarizing microscope to photograph the cross-section of the coke sample to obtain a microscopic image of the coke. The microscopic image of the coke is shown below. Figure 1 As shown.

[0112] S2. Each microscopic image is cropped into nine smaller images with a resolution of 917*736 to form a coke microscopic image dataset.

[0113] S3. Perform pixel-level annotation on the image data, where the background of the coke microscopic image is annotated as 0, and the optical tissue of the coke microscopic image is annotated as 1.

[0114] S4. Divide all labeled datasets into training, validation, and test sets in a 6:2:2 ratio.

[0115] S5. Constructing a coke segmentation model: Constructing a network segmentation model based on a multi-attention U-shaped neural network to extract the microscopic optical structure of coke.

[0116] Based on the multi-attention U-shaped neural network model structure, as follows: Figure 2 As shown, the model comprises six feature extraction layers, five bridging layers, and one bottleneck layer. In the encoder stage, the first three layers consist of standard convolutional layers and max-pooling layers, respectively; the fourth and fifth layers consist of cross-group connection attention modules and max-pooling layers; and the last layer consists only of cross-group connection attention modules. In the decoder stage, the first three layers consist of standard convolutional layers and linear interpolation upsampling layers, respectively; the fourth and fifth layers consist of cross-group connection attention modules and linear interpolation upsampling layers; and the last layer consists only of cross-group connection attention modules. All bridging modules employ adaptive hierarchical fusion attention modules, and the bottleneck layer uses a bottleneck attention module.

[0117] Cross-group connection attention module structure as follows Figure 3 As shown, the cross-group connection attention module mainly includes learnable shared spatial parameters, cross-group convolutions, depthwise separable convolutions, and normalization layers. The input feature map is divided into four groups and multiplied element-wise with the corresponding shared spatial feature map; each group extracts local contextual features through depthwise separable convolutions, and then the features interact through cross-group connections; finally, the features are concatenated along the channel dimension, integrated through 1×1 convolutions, and output through a non-linear activation function.

[0118] Adaptive hierarchical fusion attention module structure as follows Figure 4 As shown, this module mainly includes an adaptive spatial attention mask generator, multi-scale grouped convolutions, depthwise separable convolutions, and normalization layers. This module divides high-order and low-order features into four groups and concatenates them along the channel dimension. The spatial attention mask generator then generates an attention mask to suppress irrelevant noise and highlight salient regions. The fused features extract multi-scale contextual information through multiple groups of dilated convolutions, and finally, the concatenated features are integrated through a 1×1 convolution to output a unified feature map.

[0119] Bottleneck attention module structure as follows Figure 5 As shown, the bottleneck attention module consists of a channel attention branch and a spatial attention branch. The channel attention branch generates weights and weights each channel feature through global average pooling and a fully connected network; the spatial attention branch generates a spatial attention map through dimensionality reduction convolution and dilated convolution. The results of the two branches are added to form a fused attention map, which is multiplied element-wise with the input features; residual connections are also introduced to add the original input with the weighted features, highlighting key information while retaining supplementary features.

[0120] S6. Augment the training set and input the training and validation sets into the constructed network for training.

[0121] S61. Data augmentation methods include random horizontal image flipping, random vertical image flipping, and image rotation. These three image transformation methods are combined together with a probability of p = 0.5 as a data augmentation method. That is, in a data augmentation, random horizontal image flipping, random vertical image flipping, and image rotation each have a 50% probability of being randomly combined to form this data augmentation method.

[0122] S62. The training cycle of the multi-attention U-shaped neural network model is set to 200. During training, the AdamW optimizer is used to optimize the learning rate. The initial learning rate of the AdamW optimizer is set to 0.001, and the weight decay is set to 0.01 to prevent overfitting during model training.

[0123] S63. During training, the multi-attention U-shaped neural network model, after performing feature extraction and feature fusion operations using the pre-built network structure, utilizes a deep supervision mechanism to generate prediction masks of different scales. These masks, along with the final prediction results, participate in the calculation of the loss function, thereby enhancing the model's training effect and prediction accuracy. Specifically, the multi-attention U-shaped neural network employs a hybrid weighted loss function: combining the Dice loss function (LDice) with the binary cross-entropy loss function (LBCE), and introducing a multi-stage prediction deep supervision mechanism. The loss is calculated by weighting the prediction results at different stages, i.e.:

[0124] L BceDice =ω b ·L BCE +ω d ·L Dice

[0125]

[0126] Where, ω b ,ω d ω represents the weights of the binary cross-entropy loss function and the Dice loss function. During training, ω b ω d Set to 1. gt_pre i It is an auxiliary prediction system with five different stages, and a weight α. i Growth with each stage, specifically:

[0127] α5=0.1, α4=0.2, α3=0.3, α2=0.4, α1=0.5.

[0128] This design guides the model through multi-stage weighted supervision, enabling the prediction results at different levels to better fit the true labels, thereby improving the overall accuracy and stability of segmentation.

[0129] S7: The model with the smallest loss value obtained from training is used as the final inference model. The prepared test set is input into the inference model in sequence for inference, and the model output results are spliced ​​together to obtain the final coke microscopic optical tissue extraction results.

[0130] In this document, the terms "upper," "lower," "front," "back," "left," "right," "top," "bottom," "inner," "outer," "vertical," and "horizontal," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only used for the clarity of expressing the technical solution and for the convenience of description, and therefore should not be construed as limiting the present invention.

[0131] In this document, the terms “comprising,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, which includes not only the elements listed but also other elements not expressly listed.

[0132] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A method for extracting microscopic optical tissue from coke based on a multi-attention neural network, characterized in that, Includes the following steps: S1. Acquire and crop coke microscopic images to construct a dataset containing manually labeled optical tissue labels and background labels; S2. Divide the dataset into training, validation, and test sets, and perform data augmentation on the training set; S3. Construct a multi-attention U-shaped neural network model, wherein the multi-attention U-shaped neural network model includes an encoder, a bridging module, a bottleneck module, and a decoder connected in sequence; S4. The multi-attention U-shaped neural network model is trained using a training set. Multi-scale prediction masks are generated at each stage of the decoder through a deep supervision mechanism. The multi-scale prediction masks and the final output are used together to calculate the loss function. S5. Input the microscopic image of the coke to be processed into the trained multi-attention U-shaped neural network model, and output the optical tissue extraction results.

2. The method for extracting coke microscopic optical tissue based on a multi-attention neural network according to claim 1, characterized in that, The multi-attention U-shaped neural network model includes: The encoder contains six feature extraction layers, including: The first to third layers consist of standard convolutional layers and max pooling layers; The fourth and fifth layers consist of a cross-group connection attention module and a max pooling layer; The sixth layer consists of cross-group connection attention modules; Bridge module: contains six layers of adaptive hierarchical fusion attention modules, which connect to adjacent layers of the encoder respectively; Bottleneck module: Composed of bottleneck attention modules; Decoder: Contains a six-layer feature reconstruction module, where: The first to third layers consist of standard convolutional layers and bilinear interpolation upsampling layers; The fourth and fifth layers consist of a cross-group connection attention module and a bilinear interpolation upsampling layer; The sixth layer consists of cross-group connection attention modules.

3. The method for extracting coke microscopic optical tissue based on a multi-attention neural network according to claim 2, characterized in that, The cross-group connection attention module includes: Learnable shared space parameter layer; A multiplier that divides the input feature map into multiple feature groups and multiplies them element-wise with features in the shared space; Depth-separable convolutional layers for local context extraction of grouped features; A connection layer that enables cross-group feature interaction; A stitching layer that performs channel stitching on the interactive features; Features are integrated through 1×1 convolutions, and the output is activated by a non-linear layer.

4. The method for extracting coke microscopic optical tissue based on a multi-attention neural network according to claim 2, characterized in that, The adaptive hierarchical fusion attention module includes: An adaptive spatial attention mask generator for generating spatial attention masks; A fusion layer that groups and splices together high-order and low-order features; A feature selector that suppresses noise using spatial attention masking; A feature extraction layer that extracts contextual information through multi-scale grouped convolution; An integration layer that outputs a unified feature map through 1×1 convolution.

5. The method for extracting coke microscopic optical tissue based on a multi-attention neural network according to claim 1, characterized in that, The bottleneck attention module includes: Channel attention branch: includes a global average pooling layer and a fully connected layer, which generate channel weights and weighted features; Spatial attention branch: Contains dimensionality reduction convolutional layers and dilated convolutional layers to generate a spatial attention map; Fusioner: Adds the outputs of the two branches to form a fused attention map, and multiplies it element-wise with the input features; Residual connection structures that retain supplementary features.

6. The method for extracting coke microscopic optical tissue based on a multi-attention neural network according to claim 1, characterized in that, The data augmentation operation in step S2 includes: A combination of horizontal flip, vertical flip, and rotation operations is randomly performed with probability p = 0.

5.

7. The method for extracting coke microscopic optical tissue based on a multi-attention neural network according to claim 1, characterized in that, The loss function in step S4 is a hybrid weighted loss function, which is composed of the Dice loss function and the binary cross-entropy loss function, and the multi-stage prediction results are weighted and calculated.

8. The method for extracting coke microscopic optical tissue based on a multi-attention neural network according to claim 1, characterized in that, Step S5 specifically includes: S51. The encoder extracts low-order features through standard convolution and max pooling, and extracts high-order features through a cross-group connection attention module; S52. Higher-order features are jointly modeled in the channel-space manner using the bottleneck attention module; S53. Features from adjacent encoder levels are fused via skip connections using an adaptive hierarchical fusion attention module; S54. The decoder restores the resolution through bilinear interpolation upsampling and cross-group connection attention modules, and outputs the segmentation result after fusing skip connection features.

Images