Semantic segmentation method and system for medium manganese steel microstructure

Abstract

The invention discloses a semantic segmentation method and system for a medium manganese steel microstructure, relates to the technical field of crossing of material science and computer science, and aims at solving the problem that an existing semantic segmentation method for medium manganese steel is low in accuracy. The method comprises the steps of obtaining to-be-segmented data of medium manganese steel; inputting the to-be-segmented data into the target semantic segmentation model to obtain a segmentation result of the to-be-segmented data; the target semantic segmentation model at least comprises an encoder and decoder structure, a multi-scale feature fusion structure and a shallow network processing module; the multi-scale feature fusion structure is used for fusing the output of the encoder and the output of the decoder; and the shallow network processing module is used for carrying out key feature extraction on the output of the multi-scale feature fusion structure through a parallel channel attention mechanism and a space attention mechanism. The semantic segmentation method for the medium manganese steel microstructure is used for improving the semantic segmentation accuracy of austenite and ferrite of medium manganese steel.

Description

Technical Field

[0001] The present invention relates to the interdisciplinary field of materials science and computer science, and in particular to a semantic segmentation method and system for the microstructure of medium manganese steel. Background Art

[0002] Medium manganese steel, a candidate for third-generation advanced high-strength steel, exhibits multiphase, multiscale, and metastable properties, low manganese content, and excellent strength and toughness. By employing a specific rolling process with a customized combination of type, temperature, and reduction rate, as well as intercritical annealing at a temperature that allows for austenite reduction transformation and a partitioning process that promotes manganese-carbon partitioning, a more stable microstructure composed of ferrite and retained austenite can be achieved at the microscale. The high strength and high plasticity are attributed to the continuous phase transformation-induced plasticity of retained austenite. Through process adjustments, the target material can achieve a tensile strength of 800-1600 MPa and an elongation of 10-70%. To maximize the tensile strength of medium manganese steel, semantic segmentation of the steel is required to determine the proportion and morphology of retained austenite in the steel microstructure.

[0003] Existing semantic segmentation models usually use fully convolutional neural networks with data augmentation technology to distinguish complex phases. However, medium manganese steel has a complex microstructure, and its grayscale depth changes, similar grayscale values ​​of different phases, low contrast, and unclear boundaries between phases all increase the possibility of segmentation errors. Summary of the Invention

[0004] The purpose of the present invention is to provide a semantic segmentation method and system for the microstructure of medium manganese steel, which is used to solve the problem of low accuracy of existing semantic segmentation methods for medium manganese steel.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] The first invention provides a semantic segmentation method for the microstructure of medium manganese steel, comprising:

[0007] Obtain the data to be segmented of medium manganese steel;

[0008] The data to be segmented is input into a target semantic segmentation model to obtain segmentation results of ferrite and austenite in the data to be segmented; the target semantic segmentation model includes at least an encoder and a decoder structure, a multi-scale feature fusion structure and a shallow network processing module; the multi-scale feature fusion structure is used to fuse the output of the encoder and the output of the decoder; the shallow network processing module is used to extract key features of the output of the multi-scale feature fusion structure through a parallel channel attention mechanism and a spatial attention mechanism and output the segmentation result.

[0009] Optionally, the step of inputting the image to be segmented into a target semantic segmentation model to obtain segmentation results of ferrite and austenite in the image to be segmented further includes:

[0010] Obtaining sample data of medium manganese steel; the sample data includes a scanning electron microscope image and a label image; the label image is a microstructural label marked with the distribution of ferrite and austenite of the medium manganese steel;

[0011] Constructing an initial semantic segmentation model, and inputting the scanning electron microscope image of the training set in the sample data into the initial semantic segmentation model, wherein the encoder of the initial semantic segmentation model performs feature extraction on the scanning electron microscope image to obtain a multi-layer shallow feature map; the initial semantic segmentation model is an MFF-UMamba neural network model;

[0012] Based on the multiple layers of shallow feature maps, a target fusion feature map and a multiple layers of deep feature map are obtained by feature fusion;

[0013] Extracting key features of the target fusion feature map based on a parallel channel attention mechanism and a spatial attention mechanism to obtain a prediction result of the label image;

[0014] Aggregating and enhancing the depth feature map of the preset layer to obtain a depth supervision output;

[0015] Optimizing the initial semantic segmentation model based on the prediction result, the deep supervision output, and the label image to obtain a trained semantic segmentation model;

[0016] Testing the trained semantic segmentation model based on the test set of the sample data to obtain a test result;

[0017] Performing index evaluation on the test results to obtain an accuracy index;

[0018] If the accuracy index meets the preset conditions, the trained semantic segmentation model is determined as the target semantic segmentation model.

[0019] Optionally, the initial semantic segmentation model is an MFF-UMamba neural network model, the multi-layer deep feature map includes a first deep feature map and a second deep feature map; and the target fused feature map and the multi-layer deep feature map are obtained by feature fusion based on the multi-layer shallow feature map, including:

[0020] Decode the last shallow feature map to obtain the first deep feature map;

[0021] Fusing any layer of deep feature map with any layer of shallow feature map except the last shallow feature map to obtain a multi-layer fused feature map;

[0022] Decode the fused feature maps of each layer to obtain a second depth feature map;

[0023] The third deep feature map and the first shallow feature map are fused to obtain the target fused feature.

[0024] Optionally, fusing the third deep feature map and the first shallow feature map to obtain a target fused feature includes:

[0025] Using the formula:

[0026] XL1=ReLU(BatchNorm(Cov(α×XS1+β×XD1)))

[0027] Perform weighted fusion processing to obtain the target fusion feature map;

[0028] Among them, XL1 is the target fusion feature map, XS1 is the result of convolution on the first shallow feature map, XD1 is the third deep feature map, and α and β are weight coefficients.

[0029] Optionally, extracting key features of the target fusion feature map based on the parallel channel attention mechanism and the spatial attention mechanism to obtain a prediction result of the label image includes:

[0030] Performing convolution processing on the target fusion feature map to obtain a first convolution result;

[0031] Processing the first convolution result using a channel attention mechanism to obtain a first attention processing result;

[0032] Using a spatial attention mechanism to process the first convolution result to obtain a second attention processing result;

[0033] Adding the first attention processing result and the second attention processing result to obtain a first summation result;

[0034] Processing the first summation result using a depthwise convolution layer to obtain a second convolution result;

[0035] Performing batch normalization and convolution on the second convolution result to obtain a third convolution result;

[0036] Summing the third convolution result and the target fusion feature to obtain a second summation result;

[0037] The second summation result is subjected to convolution processing and upsampling processing to obtain a prediction result of the label image.

[0038] Optionally, the optimizing the initial semantic segmentation model based on the prediction result, the deep supervision output, and the label image to obtain a trained semantic segmentation model includes:

[0039] Using the formula:

[0040]

[0041] Determine a first loss function;

[0042] Among them, L p is the first loss function, N is the number of pixels of the label image, y i is the i-th pixel value of the label image, is the i-th pixel value of the prediction result, N>0;

[0043] Using the formula:

[0044]

[0045] Determine a second loss function;

[0046] Among them, L aux is the second loss function, is the i-th pixel value output by deep supervision;

[0047] Using the formula:

[0048] L=L p +δ×L aux

[0049] Determine the target loss function;

[0050] Among them, L is the target loss function, δ is the loss coefficient;

[0051] Based on the target loss function, the AdamW optimizer is used to update the parameters of the initial semantic segmentation model to obtain a trained semantic segmentation model.

[0052] Optionally, the multiple layers of shallow feature maps include a first shallow feature map and a second shallow feature map; the decoder of the initial semantic segmentation model performs feature extraction on the scanning electron microscope image to obtain the multiple layers of shallow feature maps including:

[0053] Processing the scanning electron microscope image using a convolutional layer to obtain a first shallow feature map;

[0054] Performing block embedding processing on the first shallow feature map to obtain an embedding result;

[0055] A visual state space module is used to perform feature extraction on the embedding result layer by layer to obtain a second shallow feature map.

[0056] Optionally, the using a visual state space module to perform feature extraction on the embedding result layer by layer to obtain a second shallow feature map includes:

[0057] For any visual state space module, perform layer normalization on the input data to obtain the normalized result;

[0058] Performing convolution processing on the normalized result, and applying a SiLU activation function to the convolution processing result to obtain an activation feature map;

[0059] Flattening the activation feature map into a one-dimensional vector along four different directions to obtain multiple long sequence features;

[0060] Using a selective state space model algorithm to process the long sequence features to obtain multiple processing results;

[0061] Determining an extraction feature corresponding to the visual state space module based on the processing result;

[0062] The extracted features corresponding to each visual state space module are merged to obtain the second shallow feature map.

[0063] Optionally, performing index evaluation on the test results to obtain an accuracy index includes:

[0064] Using the formula:

[0065]

[0066] Determining an accuracy index of the test result;

[0067] Among them, mIoU is the accuracy index, p (a，b) Indicates the number of pixels on the label whose category a is incorrectly predicted to be category b, p (a，a) Indicates the number of pixels on the label whose category a is correctly predicted as category a, p (b，a) Indicates the number of pixels on the label whose category b is incorrectly predicted as category a, and k represents the total number of categories;

[0068] Compared with the prior art, the present invention provides a semantic segmentation method for the microstructure of medium manganese steel, comprising: obtaining data to be segmented of medium manganese steel; inputting the data to be segmented into a target semantic segmentation model to obtain segmentation results of ferrite and austenite in the data to be segmented; the semantic segmentation model at least includes an encoder and a decoder structure, a multi-scale feature fusion structure, and a shallow network processing module; the multi-scale feature fusion structure is used to fuse the output of the encoder and the output of the decoder; the shallow network processing module is used to extract key features of the output of the multi-scale feature fusion structure through a parallel channel attention mechanism and a spatial attention mechanism and output the segmentation result. The present application can fuse the feature map information of different scales of medium manganese steel through the multi-scale feature fusion structure, and then further extract key information from the fused features through the shallow network processing module, so that the semantic segmentation result is more accurate.

[0069] In a second aspect, the present invention further provides a semantic segmentation system for the microstructure of medium manganese steel, comprising:

[0070] A module for acquiring images of the scanning electron microscope to be segmented is used to acquire the data of the medium manganese steel to be segmented;

[0071] A semantic segmentation module is used to input the data to be segmented into a target semantic segmentation model to obtain segmentation results of ferrite and austenite in the data to be segmented; the target semantic segmentation model includes at least an encoder and a decoder structure, a multi-scale feature fusion structure and a shallow network processing module; the multi-scale feature fusion structure is used to fuse the output of the encoder and the output of the decoder; the shallow network processing module is used to extract key features of the output of the multi-scale feature fusion structure through a parallel channel attention mechanism and a spatial attention mechanism and output the segmentation result.

[0072] Compared with the prior art, the beneficial effects of the semantic segmentation system for the microstructure of medium manganese steel provided by the present invention are the same as the beneficial effects of the semantic segmentation method for the microstructure of medium manganese steel described in the above technical solution, which will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0074] Figure 1 A flowchart of a semantic segmentation method for the microstructure of medium manganese steel provided by the present invention;

[0075] Figure 2 A schematic diagram of the structure of the MFF-UMamba neural network model provided by the present invention;

[0076] Figure 3 A schematic structural diagram of the VSS module provided by the present invention;

[0077] Figure 4 A schematic structural diagram of the W-PSB module provided by the present invention;

[0078] Figure 5 A schematic structural diagram of the SFRB module provided by the present invention;

[0079] Figure 6 A schematic diagram of the structure of the DFRB module provided by the present invention;

[0080] Figure 7 Schematic diagram of the phase content calculated by EBSD, U-Net and MFF-UMamba for the sample annealed for 60 minutes provided by the present invention;

[0081] Figure 8 Schematic diagram of the phase content calculated by EBSD, U-Net and MFF-UMamba for the sample annealed for 720 min provided by the present invention;

[0082] Figure 9 This is a structural schematic diagram of a semantic segmentation system for the microstructure of medium manganese steel provided by the present invention. DETAILED DESCRIPTION

[0083] To facilitate a clear description of the technical solutions of the embodiments of the present invention, the words "first" and "second" are used in the embodiments of the present invention to distinguish between identical or similar items with substantially the same functions and effects. For example, the first threshold and the second threshold are merely used to distinguish between different thresholds and do not limit their order. Those skilled in the art will understand that the words "first" and "second" do not limit the quantity or execution order, and the words "first" and "second" do not necessarily mean different.

[0084] It should be noted that, in the present invention, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present invention should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0085] In the present invention, "at least one" refers to one or more, and "more" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can represent: a, b, c, the combination of a and b, the combination of a and c, the combination of b and c, or the combination of a, b and c, where a, b, c can be single or multiple.

[0086] Before introducing the embodiments of the present invention, the following definitions are given for the relevant terms involved in the embodiments of the present invention:

[0087] Electron Backscattered Diffraction, Electron Backscattered Diffraction, abbreviated as EBSD.

[0088] Scanning Electron Microscope, Scanning Electron Microscope, referred to as SEM.

[0089] The Visual State Space Model (VSS) is an innovative architecture that combines the state space model (SSM) with the characteristics of visual tasks. It aims to efficiently model global dependencies in images while maintaining linear computational complexity.

[0090] The SS2D algorithm, or two-dimensional selective scanning algorithm, is an algorithm used in the visual state space module (VSSM). It traverses the image through four scanning paths to obtain contextual information from different directions.

[0091] The S6 algorithm, the selective state space model algorithm, is a deep learning algorithm based on the state space model SSM, which aims to improve the efficiency and flexibility of sequence modeling.

[0092] State space model (Visual State Space Model, VSS), the S6 algorithm uses the framework of the state space model to predict the future state of the system through the relationship between the input signal and the intermediate state. Its core is to model the relationship between input, state and output through learnable matrices A, B, C, and D.

[0093] The selective scanning algorithm is an algorithm that allows the model to dynamically filter information based on the relevance of the input and is widely used in deep learning and state-space models.

[0094] Hardware-aware algorithms consider the characteristics and performance of hardware when designing and implementing algorithms to optimize execution efficiency and resource utilization. Hardware-aware algorithms typically involve techniques such as parallelization and hardware acceleration to fully utilize the hardware's computing power and memory bandwidth.

[0095] The classification and quantification of microstructures bridges the gap between chemical composition, processing conditions, and mechanical properties. Therefore, microstructural characterization and analysis are crucial tasks in metallic materials research. Microstructural classification and quantification, in essence, involve segmentation of the material's microstructure image. Microstructural classification and quantification first require obtaining a microstructural image of the material. Subsequently, the pretreated material surface is observed and analyzed using microscopic equipment and various physical and chemical testing methods. Common microscopy instruments include optical microscopes and scanning electron microscopes. Scanning electron microscopes can record the material's microstructure at high resolution, but they only provide surface topography or contrast information for microstructural classification and lack the ability to distinguish microstructural entities. While electron backscatter diffraction (EBSD) can accurately segment two-phase features, the experimental scanning cost is high, the sample preparation success rate is low, and the process is time-consuming. With the advancement of modern computer technology and semantic segmentation models, fully convolutional neural networks with data augmentation have been introduced to distinguish complex phases. However, the multiphase microstructure of medium-manganese steel presents unique challenges for segmentation. Characteristics such as varying grayscale depth, similar grayscale values ​​across different phases, low contrast, and unclear phase boundaries increase the likelihood of segmentation errors. The microstructure of medium manganese steel materials has rich contextual information. How to fuse the feature map information of different scales and how to accurately extract the two-phase characteristics such as austenite and ferrite from this information are the main challenges faced by researchers.

[0096] To address these issues, the present invention provides a semantic segmentation method and system for medium-manganese steel microstructure. This method extracts features using a state-space model and fuses deep features using a multi-scale feature fusion method to achieve more accurate two-phase feature segmentation and two-phase ratios. This is described below with reference to the accompanying figures.

[0097] See also Figure 1 The present invention provides a semantic segmentation method for the microstructure of medium manganese steel, comprising the following steps:

[0098] Step 101: Obtaining data to be segmented of medium manganese steel;

[0099] The data to be segmented is a scanning electron microscope image to be semantically segmented;

[0100] Step 102: inputting the data to be segmented into a target semantic segmentation model to obtain segmentation results of ferrite and austenite in the data to be segmented;

[0101] The target semantic segmentation model includes at least an encoder and decoder structure, a multi-scale feature fusion structure and a shallow network processing module; the multi-scale feature fusion structure is used to fuse the output of the encoder and the output of the decoder; the shallow network processing module is used to extract key features of the output of the multi-scale feature fusion structure through parallel channel attention mechanism and spatial attention mechanism and output segmentation results.

[0102] Specifically, the encoder includes a convolutional layer and a multi-layer VSS layer, each VSS layer is composed of a VSS module, the decoder is composed of a multi-layer VSS layer, the encoder and the decoder have the same number of VSS layers and the same number of VSS modules, the multi-scale feature fusion structure includes a multi-layer W-PSB module, the W-PSB module uses a convolutional layer and a weighted fusion algorithm to fuse the output of the encoder and the output of the decoder, the segmentation steps are: first, the encoder is used to extract features layer by layer on the data to be segmented, and then the last layer of the decoder processes the output of the last layer of the encoder, the multi-scale feature fusion structure fuses the output of the encoder and the output of the decoder, the fusion feature map output by each layer except the fusion feature map finally output by the multi-scale feature fusion module is input into the decoder for decoding, the decoded features are input into the W-PSB module of the next layer, the fusion feature map finally output by the multi-scale feature fusion structure is input into the shallow network processing module SFRB, the shallow network processing module uses a parallel channel attention mechanism and a spatial attention mechanism to process the received fusion feature map to obtain the segmentation result.

[0103] Figure 1 The above method can fuse the feature map information of different scales of medium manganese steel through the multi-scale feature fusion structure, and then further extract key information from the fused features through the shallow network processing module, making the semantic segmentation results more accurate.

[0104] As an optional method, before inputting the scanning electron microscope image to be segmented into the target semantic segmentation model to obtain the segmentation results of ferrite and austenite in the scanning electron microscope image to be segmented, it is necessary to construct the target semantic segmentation model, including the following steps:

[0105] Step 110: Obtain a data set of medium manganese steel and process the data set to obtain sample data.

[0106] The sample data includes scanning electron microscope images and labeled images. The sample data can be divided into training sets, test sets, and validation sets. The labeled images are electron backscatter diffraction images, or EBSD images, which are labeled with microstructural labels indicating the distribution of ferrite and austenite in medium manganese steel.

[0107] The dataset was generated through sample annealing, SEM, and EBSD experiments. To extend the model's application to the entire dual-phase region of the target medium-manganese steel, samples were annealed at the same temperature with varying holding times. This resulted in variations in the dual-phase ratio of the medium-manganese steel, yielding samples with varying micromorphologies. For example, three identical medium-manganese steel samples were heated at 650°C for 1, 5, and 12 hours, labeled 1, 2, and 3, respectively. The three samples were air-cooled to room temperature to produce samples with varying austenite contents. The samples were then ground and polished using low forces to avoid mechanically induced martensitic transformation of the austenite. After grinding and polishing using 800-1200 grit sandpaper, electropolishing was performed using 700 ml of glacial acetic acid and 100 ml of perchloric acid at 25°C, 24 V, and 20 s. Finally, scanning electron microscope images (SEM) and labeled EBSD images of the three samples were obtained, containing the desired ratios and shapes of the two phases. The experiments were performed using a Zeiss Sigma 300 system and a C-Nano EBSD system at an accelerating voltage of 20 kV. Scanning electron microscopy (SEM) experiments used a scanning step size of 0.033 μm and secondary electron imaging to obtain microscopic images at a magnification of 4000x (2048 × 1536 pixels). The dataset includes SEM images and labeled images of the three samples.

[0108] Exemplarily, the steps for processing the dataset are as follows: uniformly scale and crop the scanning electron microscope images and label images in the dataset to 1536×1024 pixels, then crop all images in the dataset into 960 128×128 pixel scanning electron microscope image sub-images and 960 images from the label images. 576 sub-images consisting of three images of sample No. 1 and three images of sample No. 3 are selected to form the training set; 192 sub-images consisting of one image of sample No. 1 and one image of sample No. 3 are selected to form the validation set; and 192 sub-images consisting of two images of sample No. 2 are selected to form the test set.

[0109] Step 120: constructing an initial semantic segmentation model, and inputting the scanning electron microscope image of the training set in the sample data into the initial semantic segmentation model, wherein the encoder of the initial semantic segmentation model performs feature extraction on the scanning electron microscope image to obtain a multi-layer shallow feature map;

[0110] like Figure 2As shown, the initial semantic segmentation model is the MFF-UMamba neural network model, which includes an encoder and decoder structure, a multi-scale fusion structure, and a deep supervision structure. The encoder includes convolutional layers and multiple VSS layers. A block merging layer is provided between two adjacent VSS layers. The block merging layer is used to merge the output data received from each VSS module. A block embedding layer is provided between the convolutional layer and the first VSS layer. The decoder includes the same number of VSS layers as the encoder, and the VSS layers are composed of multiple VSS modules. For example, the number of VSS modules in the encoder VSS layer is 2, 2, 9, and 2, respectively, from top to bottom. The number of VSS modules in the decoder VSS layer is 2, 2, 9, and 2, respectively, from top to bottom. The multi-scale fusion structure consists of multiple W-PSB modules. The W-PSB module uses convolutional layers and a weighted fusion algorithm to combine the shallow feature maps output by the encoder with the deep feature maps output by the decoder. The deep supervision structure includes a shallow network processing module SFRB and a deep data fusion module DFAB. The output of each layer of the encoder is a shallow feature map, the output of the convolutional layer is the first shallow feature map, and the output of each VSS layer is the second shallow feature map.

[0111] As an optional method, step 120 includes the following steps:

[0112] Step 1201: Processing the scanning electron microscope image using a convolutional layer to obtain a first shallow feature map;

[0113] Step 1202: The block embedding layer performs block embedding processing on the first shallow feature map to obtain an embedding result;

[0114] Step 1203: Each VSS layer uses a visual state space module to extract features from the embedding result layer by layer to obtain a second shallow feature map;

[0115] Specifically, such as Figure 3As shown, for any visual state space module VSS module: perform layer normalization on the input data to obtain the first normalization result; use a linear layer to process the first normalization result, perform convolution on the processed normalized result to obtain a convolution result and apply the SiLU activation function to the convolution result to obtain an activation feature map; use a two-dimensional selective scanning algorithm to process the activation feature map, specifically, flatten the activation feature map into a one-dimensional vector along four different directions, the flattened one-dimensional vector is a long sequence feature, and obtain multiple long sequence features; use the S6 algorithm to process the long sequence feature to obtain multiple processing results: y1, y2, y3, y 4, etc.; determining the extraction features corresponding to the visual state space module based on the processing results; specifically, performing summation and transposition processing on the processing results to obtain a summation and transposition result, and performing layer normalization processing on the summation and transposition result to obtain a second normalization result, performing linear processing on the first normalization result, and applying the SiLU activation function to the linear processing result, multiplying the obtained result with the second layer normalization processing result, and performing linear processing again, adding the processed result to the input data to obtain the extraction features of the VSS module, wherein the input data of the VSS layer adjacent to the convolution layer in the encoder is the embedding result, and the input data of the remaining VSS layers are the block-merged data.

[0116] Finally, the extracted features corresponding to each visual state space module in each VSS layer are merged to obtain the second shallow feature map corresponding to each VSS layer.

[0117] The S6 algorithm is a selective state space model algorithm based on the SSM algorithm. The SSM principle expression is shown in formula (1):

[0118] h′(t)=Ah(t)+Bx(t)

[0119] y(t)=Ch(t)+Dx(t) (1)

[0120] in, x(t) is the input, h(t) is the hidden state, and y(t) is the output; A is the state matrix, that is, the coupling relationship matrix between state variables, which is used to describe the internal dynamic evolution of the system state composed of the sequence transformed by the feature graph; B is the input matrix, that is, the sequence transformed by the feature graph of the external input. The input matrix is ​​a specific description matrix that affects the system state; C is the output matrix, which maps the internal state of the system to the observable output; D is the through matrix, which describes the dynamic impact of the input signal on the output by directly bypassing the internal state of the system.

[0121] The VSS module utilizes a linear state space layer. Its core concept is to discretize the continuous-time SSM using the zero-order hold technique, resulting in two discrete representations: cyclic and convolutional. The VSS module utilizes a one-dimensional convolution kernel to aggregate features, enabling parallel training. The VSS module utilizes the basic methodology of recurrent neural networks, calculating the predicted output at each time step based on the impact of the current input on the previous state. The VSS module utilizes high-order polynomial projection operators to more accurately reconstruct new signals. The VSS module also utilizes a selective scanning algorithm and a hardware-aware algorithm, enabling excellent performance in processing long sequences.

[0122] In practical applications, data needs to be downsampled before being input into the convolutional layer, block embedding layer, each VSS layer, and block merging layer.

[0123] Step 130: Based on the multiple layers of shallow feature maps, obtain a target fused feature map and a multiple layer deep feature map by feature fusion;

[0124] like Figure 2 As shown in the figure, each layer of the decoder outputs a depth feature map. The VSS layer that performs the first decoding outputs the first depth feature map, and the other VSS layers output the second depth feature map. The shallow feature maps output by each VSS layer of the encoder are skip-connected and input into the corresponding W-PSB module. The deep features output by each VSS layer of the decoder are upsampled and input into the corresponding W-PSB module. The W-PSB module fuses the received shallow feature maps and deep feature maps, outputs a fused feature map, and upsamples the fused feature map and inputs it into the decoder. The final fused feature map is input into the SFRB module, and the SFRB module outputs the segmentation result.

[0125] Specifically, step 130 can be implemented based on the following steps:

[0126] Step 1301: Decode the last shallow feature map to obtain a first deep feature map;

[0127] The decoder decodes the input first shallow feature map and fusion feature map through the VSS layer. The principle of the VSS module used is the same as that of the encoder's VSS module, which will not be repeated here.

[0128] Step 1302: Fusing any layer of deep feature map with any layer of shallow feature map except the last shallow feature map to obtain a multi-layer fused feature map;

[0129] like Figure 4As shown in the figure, the input shallow feature map is convolved, and the weighted sum of the shallow feature map after convolution and the input deep feature map is calculated. Then, the calculation result is convolved, batch normalized and ReLU activation function is applied to obtain the final fusion feature map.

[0130] Specifically, use formula (2):

[0131] XL j =ReLU(BatchNorm(Conv(α×XS j +β×XD j ))) (2)

[0132] Perform weighted fusion processing to obtain a fusion feature map;

[0133] Among them, XL j is the fusion feature map output by the j-th layer W-PSB, XS j is the convolution result of the j-th shallow feature map, XD j is the j-th layer depth feature map, α and β are weight coefficients.

[0134] It should be noted that the encoder, multi-scale feature fusion structure and each layer of the decoder are numbered in a gradually increasing manner from top to bottom; the encoder is the first layer, the second layer, the third layer, the fourth layer and the fifth layer from top to bottom, the decoder is the first layer, the second layer, the third layer and the fourth layer from top to bottom, the multi-scale feature fusion structure is the first layer, the second layer, the third layer and the fourth layer from top to bottom, the shallow feature map of the first layer of the encoder and the deep feature map of the first layer of the decoder are input into the first layer of the multi-scale feature fusion structure, and so on. In addition, the decoder performs the first decoding from the last layer.

[0135] Step 1303: Decode the fused feature maps of each layer to obtain a second depth feature map;

[0136] The second depth feature map is a depth feature map among all depth feature maps except the first depth feature map.

[0137] Step 1304: Fuse the third deep feature map and the first shallow feature map to obtain target fused features.

[0138] The third depth feature map is the depth feature map finally output by the decoder, that is, the depth feature map output by the first layer of the decoder.

[0139] The third depth feature map is the depth feature map finally output by the decoder in the second depth feature map, that is, Figure 2 The deep feature map of the first layer of the decoder is shown.

[0140] Specifically, use formula (3):

[0141] XL1=ReLU(BatchNorm(Conv(α×XS1+β×XD1))) (3)

[0142] Perform weighted fusion processing to obtain the target fusion feature map;

[0143] Among them, XL1 is the target fusion feature, that is, Figure 2 The fusion features of the first layer output of the multi-scale feature fusion structure, XS1 is the result after convolution of the first shallow feature map, that is, Figure 2 The shallow feature map of the first layer output of the encoder, XD1 is the third deep feature map, that is Figure 2 The depth feature map output by the first layer of the decoder, α and β are weight coefficients.

[0144] Step 140: The shallow network processing module SFRB module extracts key features of the target fusion feature map based on the parallel channel attention mechanism and the spatial attention mechanism to obtain a prediction result of the label image;

[0145] like Figure 5 As shown, step 140 can be implemented based on the following steps:

[0146] The target fusion feature map is convolved to obtain a first convolution result; the convolution kernel is 3×3.

[0147] The first convolution result is processed using a channel attention mechanism to obtain a first attention processing result. Specifically, the first convolution result is sequentially subjected to average pooling processing, convolution processing, application of ReLU activation function, convolution processing, and Sigmoid function processing, and the convolution kernels used in the two convolution processing are both 1×1.

[0148] The first convolution result is processed using a spatial attention mechanism to obtain a second attention processing result. Specifically, the first convolution result is sequentially subjected to convolution processing and Sigmoid function processing, and the convolution kernel used in the convolution processing is 3×3.

[0149] Adding the first attention processing result and the second attention processing result to obtain a first summation result;

[0150] Using a deep convolution layer DWConv to process the first summation result to obtain a second convolution result;

[0151] Performing batch normalization and convolution on the second convolution result to obtain a third convolution result;

[0152] Summing the third convolution result and the target fusion feature to obtain a second summation result;

[0153] The second summation result is subjected to convolution and upsampling to obtain a prediction result of the label image, and the convolution result is subjected to a ReLU activation function.

[0154] The parallel channel attention mechanism and the spatial attention mechanism can more accurately extract key information from the target fusion feature map.

[0155] Step 150: Aggregate and enhance the depth feature map of the preset layer to obtain a depth supervision output;

[0156] Specifically, the depth feature map of the preset layer can be the depth feature map of the second layer, the third layer and the fourth layer, and the depth feature map of the second layer, the third layer and the fourth layer is summed up, and then the summation result is processed by the DFAB module to obtain the depth supervision output. Figure 6 As shown in the figure, the DFAB module processes the summation result through a convolution layer with a convolution kernel size of 3×3, a batch normalization layer, a ReLU activation function, and a convolution layer with a convolution kernel of 1×1, and then upsamples the result to an image of size (128, 128).

[0157] Step 160: Optimizing the initial semantic segmentation model based on the prediction result, the deep supervision output, and the label image to obtain a trained semantic segmentation model;

[0158] To achieve fast convergence, the Adam optimizer is used to train the initial semantic segmentation model. Based on a multi-head design, the training of the entire network is guided by a combination of a main loss function and an auxiliary loss function. L1Loss is used as the loss function for shallow supervision, while DiceBCELoss is used for deep supervision.

[0159] Specifically, use formula (4):

[0160]

[0161] Determine a first loss function;

[0162] Among them, L p is the first loss function, N is the number of pixels of the label image, y i is the i-th pixel value of the label image, is the i-th pixel value of the prediction result, N>0;

[0163] Using formula (5):

[0164]

[0165] Determine a second loss function;

[0166] Among them, L aux is the second loss function, is the i-th pixel value output by deep supervision;

[0167] Using formula (6):

[0168] L=L p +δ×L aux (6)

[0169] Determine the target loss function;

[0170] Among them, L is the target loss function, δ is the loss coefficient;

[0171] Based on the target loss function, the AdamW optimizer is used to update the parameters of the initial semantic segmentation model to obtain a trained semantic segmentation model.

[0172] Specifically, the operational logic of the AdamW optimizer formula, that is, the optimization process, is:

[0173] Step 1: Calculate the gradient without weight decay, as shown in formula (7):

[0174]

[0175] Where g t Indicates that the objective function f has parameters θ t-1 The gradient at θ, that is, the original gradient, does not contain the L2 regularization term, t-1 is the value of the model parameter at time step t-1.

[0176] Step 2: Update the first-order moment and second-order moment. The first-order moment, that is, the momentum update is shown in formula (8):

[0177] m t =β1·m t-1 +(1-β1)·g t (8)

[0178] Where m t is the first-order moment estimate, i.e., the momentum term, which is used to smooth the historical gradient direction and alleviate gradient noise. β1 is the exponential decay rate of the first-order moment, which is 0.9.

[0179] The second-order moment, that is, the adaptive learning rate update is shown in formula (9):

[0180]

[0181] Where, v tIt is the second-order moment estimation, that is, the adaptive learning rate, the moving average of the squared gradient, which is used to adjust the learning rate of the parameters. β2 represents the exponential decay rate of the first-order moment, which is 0.999.

[0182] Step 3: Bias correction. The first-order moment correction is shown in formula (10), and the second-order moment correction is shown in formula (11):

[0183]

[0184] Where, is the first-order estimate after bias correction, is the bias-corrected second moment estimate. and It decays exponentially with time step t, and the correction makes the initial estimate closer to the true statistic.

[0185] Step 4: Parameter update. The update formula is shown in formula (12):

[0186]

[0187] Where η is the learning rate, which is 6e-4, λ is the weight decay coefficient, and ε is a numerical stability constant to prevent division by zero.

[0188] Step 170: Testing the trained semantic segmentation model based on the test set of the sample data to obtain a test result;

[0189] Apply the training weight file stored in the training to input the scanning electron microscope images of the test set into the trained semantic segmentation model to predict the label images and obtain the test results.

[0190] Step 180: performing an index evaluation on the test results to obtain an accuracy index;

[0191] As an optional method, the mIoU indicator is used for evaluation, specifically using formula (13):

[0192]

[0193] Determining an accuracy index of the test result;

[0194] Among them, mIoU is the accuracy index, p (a，b) Indicates the number of pixels on the label where category a is incorrectly predicted to be category b, that is, the number of false negatives, p (a，a) Indicates the number of pixels on the label that are correctly predicted to be category a, that is, the number of true positives, p (b，a) It represents the number of pixels on the label whose category b is incorrectly predicted as category a, that is, the number of false positives, and k represents the total number of categories.

[0195] As another optional method, the PA index is used for evaluation. Specifically, formula (14) is used:

[0196]

[0197] Determining an accuracy index of the test result;

[0198] Among them, p (a，b) Indicates the number of pixels on the label where category a is incorrectly predicted to be category b, that is, the number of false negatives, p (a，a) It represents the number of pixels on the label that are correctly predicted as category a, that is, the number of true positives, and k represents the number of categories.

[0199] Step 190: If the accuracy index meets a preset condition, the trained semantic segmentation model is determined as the target semantic segmentation model. The preset condition may be that the accuracy index meets a preset threshold, which is set as needed. If the accuracy index does not meet the preset condition, more sample data is collected to train the model until the accuracy index meets the preset condition.

[0200] Step 210: Use the validation set to validate the trained semantic segmentation model.

[0201] The predicted results using the MFF-UMamba neural network model of this application are basically consistent with the EBSD label image, while the results predicted by models such as U-Net, SwinUNet, TransUNet, and UNetFrmer all have large errors compared with the EBSD label.

[0202] like Figure 7 and Figure 8 As shown in the figure, the ratio of austenite and ferrite predicted by the MFF-UMamba neural network model is very different from the ratio of austenite and ferrite in the EBSD label image, which is only 1%. This shows that the MFF-UMamba neural network model still has higher accuracy for data under different annealing times.

[0203] The encoder and decoder of the semantic segmentation model utilize the VSS module, enabling pixel-by-pixel classification of SEM images and precise semantic segmentation of the rich contextual information inherent in the microstructure of medium-manganese steel materials. This allows for pixel-level understanding of micromorphology, thereby improving the accuracy of extracted features. The multi-scale feature fusion structure deeply fuses the shallow feature maps output by the encoder with the deep feature maps output by the decoder, and inputs the fused features into the decoder for decoding. This allows for the fusion of information from feature maps at different scales, and enhances local features with contextual information and fuses them with global features, thereby improving the accuracy of complex texture and boundary segmentation. Finally, extraction through the shallow network processing module SFRB further enhances the accuracy of the segmentation results. The MFF-UMamba neural network model uses EBSD images as label images during training, significantly improving model accuracy. Furthermore, training based on the outputs of the shallow network processing module SFRB and the deep data fusion module DFAB accelerates convergence, improves training efficiency, and enhances the model's feature discrimination. The target semantic segmentation model of the present application performs better in models with the same or similar volume, and performs better in models with smaller volume. With the same number of layers or similar volume, the target semantic segmentation model of the present application has higher accuracy.

[0204] The embodiments of the present invention can be divided into functional modules according to the above-mentioned method examples. For example, each functional module can be divided according to each function, or two or more functions can be integrated into a single processing module. The above-mentioned integrated modules can be implemented in the form of hardware or software functional modules. It should be noted that the division of modules in the embodiments of the present invention is illustrative and is only a logical functional division. In actual implementation, other division methods may be used.

[0205] In the case of dividing each functional module into corresponding functional modules, Figure 9 FIG. 1 shows a schematic diagram of the structure of a semantic segmentation system for the microstructure of medium manganese steel provided by the present invention. Figure 9 As shown, the system includes:

[0206] The data to be segmented acquisition module 910 is used to acquire the data to be segmented of the medium manganese steel;

[0207] The semantic segmentation module 920 is used to input the data to be segmented into the target semantic segmentation model to obtain the segmentation results of ferrite and austenite in the data to be segmented; the target semantic segmentation model includes at least an encoder and a decoder structure, a multi-scale feature fusion structure and a shallow network processing module; the multi-scale feature fusion structure is used to fuse the output of the encoder and the output of the decoder; the shallow network processing module is used to extract key features of the output of the multi-scale feature fusion structure through a parallel channel attention mechanism and a spatial attention mechanism and output the segmentation result.

[0208] Optionally, the system further includes a target semantic segmentation model determination module, wherein the target semantic segmentation model determination module includes:

[0209] A sample data acquisition unit is used to acquire sample data of medium manganese steel; the sample data includes a scanning electron microscope image and a label image; the label image is a microstructural label marked with the distribution of ferrite and austenite of the medium manganese steel;

[0210] An initial semantic segmentation model construction unit is used to construct an initial semantic segmentation model, and input the scanning electron microscope image of the training set in the sample data into the initial semantic segmentation model, wherein the encoder of the initial semantic segmentation model performs feature extraction on the scanning electron microscope image to obtain a multi-layer shallow feature map; the initial semantic segmentation model is an MFF-UMamba neural network model;

[0211] A feature fusion unit, configured to obtain a target fused feature map and a multi-layer deep feature map by feature fusion based on the multi-layer shallow feature map;

[0212] A parallel mechanism feature extraction unit, configured to extract key features of the target fusion feature map based on a parallel channel attention mechanism and a spatial attention mechanism to obtain a prediction result of the label image;

[0213] A depth supervision unit, configured to aggregate and enhance the depth feature map of a preset layer to obtain a depth supervision output;

[0214] a parameter optimization unit, configured to optimize the initial semantic segmentation model based on the prediction result, the deep supervision output, and the label image to obtain a trained semantic segmentation model;

[0215] A testing unit, configured to test the trained semantic segmentation model based on a test set of the sample data to obtain a test result;

[0216] An index evaluation unit is used to perform index evaluation on the test results to obtain an accuracy index; if the accuracy index meets a preset condition, the trained semantic segmentation model is determined as the target semantic segmentation model.

[0217] Optionally, the initial semantic segmentation model is an MFF-UMamba neural network model, the multi-layer depth feature map includes a first depth feature map and a second depth feature map; the feature fusion unit may include:

[0218] A first depth feature map determining subunit is used to decode the last shallow feature map to obtain a first depth feature map;

[0219] A fusion subunit is used to fuse any layer of deep feature map with any layer of shallow feature map except the last layer of shallow feature map to obtain a multi-layer fused feature map;

[0220] A second depth feature map determining subunit is used to decode the fused feature maps of each layer to obtain a second depth feature map;

[0221] The target fusion feature determination subunit is used to fuse the third deep feature map and the first shallow feature map to obtain the target fusion feature.

[0222] Optionally, the target fusion feature determination subunit may be specifically used to:

[0223] Using the formula:

[0224] Using the formula:

[0225] XL1=ReLU(BatchNorm(Conv(α×XS1+β×XD1)))

[0226] Perform weighted fusion processing to obtain the target fusion feature map;

[0227] Among them, XL1 is the target fusion feature map, XS1 is the result of convolution on the first shallow feature map, XD1 is the third deep feature map, and α and β are weight coefficients.

[0228] Optionally, the parallel mechanism feature extraction unit may be specifically used to:

[0229] Performing convolution processing on the target fusion feature map to obtain a first convolution result;

[0230] Processing the first convolution result using a channel attention mechanism to obtain a first attention processing result;

[0231] Using a spatial attention mechanism to process the first convolution result to obtain a second attention processing result;

[0232] Adding the first attention processing result and the second attention processing result to obtain a first summation result;

[0233] Processing the first summation result using a depthwise convolution layer to obtain a second convolution result;

[0234] Performing batch normalization and convolution on the second convolution result to obtain a third convolution result;

[0235] Summing the third convolution result and the target fusion feature to obtain a second summation result;

[0236] The second summation result is subjected to convolution processing and upsampling processing to obtain a prediction result of the label image.

[0237] Optionally, the parameter optimization unit may be specifically used to:

[0238] Using the formula:

[0239]

[0240] Determine a first loss function;

[0241] Among them, L p is the first loss function, N is the number of pixels of the label image, y i is the i-th pixel value of the label image, is the i-th pixel value of the prediction result, N>0;

[0242] Using the formula:

[0243]

[0244] Determine a second loss function;

[0245] Among them, L aux is the second loss function, is the i-th pixel value output by deep supervision;

[0246] Using the formula:

[0247] L=L p +δ×L aux

[0248] Determine the target loss function;

[0249] Among them, L is the target loss function, δ is the loss coefficient;

[0250] Based on the target loss function, the AdamW optimizer is used to update the parameters of the initial semantic segmentation model to obtain a trained semantic segmentation model.

[0251] Optionally, the multiple layers of shallow feature maps include a first shallow feature map and a second shallow feature map; and the initial semantic segmentation model construction unit may include:

[0252] a first shallow feature map determining subunit, configured to process the scanning electron microscope image using a convolutional layer to obtain a first shallow feature map;

[0253] an embedding subunit, configured to perform block embedding processing on the first shallow feature map to obtain an embedding result;

[0254] The second shallow feature map determining subunit is used to extract features from the embedding result layer by layer using a visual state space module to obtain a second shallow feature map.

[0255] Optionally, the second shallow feature map determination subunit may be specifically used to:

[0256] For any visual state space module, perform layer normalization on the input data to obtain the normalized result;

[0257] Performing convolution processing on the normalized result, and applying a SiLU activation function to the convolution processing result to obtain an activation feature map;

[0258] Flattening the activation feature map into a one-dimensional vector along four different directions to obtain multiple long sequence features;

[0259] Using a selective state space model algorithm to process the long sequence features to obtain multiple processing results;

[0260] Determining an extraction feature corresponding to the visual state space module based on the processing result;

[0261] The extracted features corresponding to each visual state space module are merged to obtain the second shallow feature map.

[0262] Optionally, the indicator evaluation unit may be specifically used to:

[0263] Using the formula:

[0264]

[0265] Determining an accuracy index of the test result;

[0266] Among them, mIoU is the accuracy index, p (a，b) Indicates the number of pixels on the label whose category a is incorrectly predicted to be category b, p (a,a) Indicates the number of pixels on the label whose category a is correctly predicted as category a, p (b，a)Indicates the number of pixels on the label whose category b is incorrectly predicted as category a, and k represents the total number of categories;

[0267] The trained semantic segmentation model whose accuracy index meets the preset conditions is determined as the target semantic segmentation model.

[0268] The above mainly introduces the solution provided by the embodiment of the present invention from the perspective of the interaction between the various modules. It can be understood that in order to realize the above functions, it includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present invention can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0269] In the above embodiments, all or part of the embodiments can be implemented using software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer programs or instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present invention are performed in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network model, a terminal, a user device, or other programmable device. The computer program or instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another. For example, the computer program or instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, such as a floppy disk, hard disk, or magnetic tape; an optical medium, such as a digital video disc (DVD); or a semiconductor medium, such as a solid-state drive (SSD).

[0270] Although the present invention is described herein in conjunction with various embodiments, in the process of implementing the claimed invention, those skilled in the art can understand and implement other variations of the disclosed embodiments by reviewing the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple situations. A single processor or other unit can implement several functions listed in the claims. Certain measures are recorded in different dependent claims, but this does not mean that these measures cannot be combined to produce good results.

[0271] Although the present invention has been described with reference to specific features and embodiments thereof, it will be apparent that various modifications and combinations may be made thereto without departing from the spirit and scope of the invention. Accordingly, this specification and drawings are merely illustrative of the invention as defined by the appended claims and are deemed to cover any and all modifications, variations, combinations or equivalents within the scope of the invention. It will be apparent that various modifications and variations may be made to the present invention by those skilled in the art without departing from the spirit and scope of the invention. Thus, the present invention is intended to include such modifications and variations as fall within the scope of the claims of the present invention and their equivalents.

Claims

1. A semantic segmentation method for medium manganese steel microstructure, characterized in that: include: Obtain the data to be segmented of medium manganese steel; Inputting the data to be segmented into a target semantic segmentation model to obtain segmentation results of ferrite and austenite in the data to be segmented; the target semantic segmentation model includes at least an encoder and a decoder structure, a multi-scale feature fusion structure, and a shallow network processing module; the multi-scale feature fusion structure is used to fuse the output of the encoder and the output of the decoder; The shallow network processing module is used to extract key features of the output of the multi-scale feature fusion structure through a parallel channel attention mechanism and a spatial attention mechanism and output the segmentation result.

2. The semantic segmentation method of the medium manganese steel microstructure according to claim 1, characterized in that: The step of inputting the image to be segmented into a target semantic segmentation model to obtain segmentation results of ferrite and austenite in the image to be segmented further includes: Obtaining sample data of medium manganese steel; the sample data includes a scanning electron microscope image and a label image; the label image is a microstructural label marked with the distribution of ferrite and austenite of the medium manganese steel; Constructing an initial semantic segmentation model, and inputting the scanning electron microscope image of the training set in the sample data into the initial semantic segmentation model, wherein the encoder of the initial semantic segmentation model performs feature extraction on the scanning electron microscope image to obtain a multi-layer shallow feature map; the initial semantic segmentation model is an MFF-UMamba neural network model; Based on the multiple layers of shallow feature maps, a target fusion feature map and a multiple layers of deep feature map are obtained by feature fusion; Extracting key features of the target fusion feature map based on a parallel channel attention mechanism and a spatial attention mechanism to obtain a prediction result of the label image; Aggregating and enhancing the depth feature map of the preset layer to obtain a depth supervision output; Optimizing the initial semantic segmentation model based on the prediction result, the deep supervision output, and the label image to obtain a trained semantic segmentation model; Testing the trained semantic segmentation model based on the test set of the sample data to obtain a test result; Performing index evaluation on the test results to obtain an accuracy index; If the accuracy index meets the preset conditions, the trained semantic segmentation model is determined as the target semantic segmentation model.

3. The semantic segmentation method of the medium manganese steel microstructure according to claim 2, characterized in that: The multi-layer depth feature map includes a first depth feature map and a second depth feature map; the target fusion feature map and the multi-layer depth feature map are obtained by feature fusion based on the multi-layer shallow feature map, including: Decode the last shallow feature map to obtain the first deep feature map; Fusing any layer of deep feature map with any layer of shallow feature map except the last shallow feature map to obtain a multi-layer fused feature map; Decode the fused feature maps of each layer to obtain a second depth feature map; The third deep feature map and the first shallow feature map are fused to obtain the target fused feature.

4. The semantic segmentation method of the medium manganese steel microstructure according to claim 3, characterized in that: The fusing process of the third deep feature map and the first shallow feature map to obtain a target fused feature includes: Using the formula: XL1=ReLU(BatchNorm(Conv(α×XS1+β×XD1))) Perform weighted fusion processing to obtain the target fusion feature map; Among them, XL1 is the target fusion feature map, XS1 is the result of convolution on the first shallow feature map, XD1 is the third deep feature map, and α and β are weight coefficients.

5. The semantic segmentation method of the medium manganese steel microstructure according to claim 2, characterized in that: The key features of the target fusion feature map are extracted based on the parallel channel attention mechanism and the spatial attention mechanism to obtain the prediction result of the label image, including: Performing convolution processing on the target fusion feature map to obtain a first convolution result; Processing the first convolution result using a channel attention mechanism to obtain a first attention processing result; Using a spatial attention mechanism to process the first convolution result to obtain a second attention processing result; Adding the first attention processing result and the second attention processing result to obtain a first summation result; Processing the first summation result using a depthwise convolution layer to obtain a second convolution result; Performing batch normalization and convolution on the second convolution result to obtain a third convolution result; Summing the third convolution result and the target fusion feature to obtain a second summation result; The second summation result is subjected to convolution processing and upsampling processing to obtain a prediction result of the label image.

6. The semantic segmentation method of the medium manganese steel microstructure according to claim 2, characterized in that: The optimizing the initial semantic segmentation model based on the prediction result, the deep supervision output, and the label image to obtain a trained semantic segmentation model includes: Using the formula: Determine a first loss function; Among them, L p is the first loss function, N is the number of pixels of the label image, y i is the i-th pixel value of the label image, is the i-th pixel value of the prediction result, N>0; Using the formula: Determine a second loss function; Among them, L aux is the second loss function, is the i-th pixel value output by deep supervision; Using the formula: L=L p +δ×L aux Determine the target loss function; Among them, L is the target loss function, δ is the loss coefficient; Based on the target loss function, the AdamW optimizer is used to update the parameters of the initial semantic segmentation model to obtain a trained semantic segmentation model.

7. The semantic segmentation method of the medium manganese steel microstructure according to claim 2, characterized in that: The multiple layers of shallow feature maps include a first shallow feature map and a second shallow feature map; the decoder of the initial semantic segmentation model extracts features from the scanning electron microscope image to obtain multiple layers of shallow feature maps including: Processing the scanning electron microscope image using a convolutional layer to obtain a first shallow feature map; Performing block embedding processing on the first shallow feature map to obtain an embedding result; A visual state space module is used to perform feature extraction on the embedding result layer by layer to obtain a second shallow feature map.

8. The semantic segmentation method of the medium manganese steel microstructure according to claim 7, characterized in that: The visual state space module is used to extract features from the embedding result layer by layer to obtain a second shallow feature map, which includes: For any visual state space module, perform layer normalization on the input data to obtain the normalized result; Performing convolution processing on the normalized result, and applying a SiLU activation function to the convolution processing result to obtain an activation feature map; Flattening the activation feature map into a one-dimensional vector along four different directions to obtain multiple long sequence features; Using a selective state space model algorithm to process the long sequence features to obtain multiple processing results; Determining an extraction feature corresponding to the visual state space module based on the processing result; The extracted features corresponding to each visual state space module are merged to obtain the second shallow feature map.

9. The semantic segmentation method of medium manganese steel microstructure according to claim 2, characterized in that: The test results are evaluated to obtain accuracy indicators including: Using the formula: Determining an accuracy index of the test result; Among them, mIoU is the accuracy index, p (a，a) Indicates the number of pixels on the label whose category a is incorrectly predicted to be category b, p (a，a) Indicates the number of pixels on the label whose category a is correctly predicted as category a, p (b，a) Indicates the number of pixels on the label whose category b is incorrectly predicted as category a, and k represents the total number of categories; The trained semantic segmentation model whose accuracy index meets the preset conditions is determined as the target semantic segmentation model.

10. A semantic segmentation system for medium manganese steel microstructure, characterized in that: include: A module for acquiring images of the scanning electron microscope to be segmented is used to acquire the data of the medium manganese steel to be segmented; A semantic segmentation module is configured to input the data to be segmented into a target semantic segmentation model to obtain segmentation results of ferrite and austenite in the data to be segmented; the target semantic segmentation model comprises at least an encoder and a decoder structure, a multi-scale feature fusion structure, and a shallow network processing module; the multi-scale feature fusion structure is configured to fuse the output of the encoder and the output of the decoder; The shallow network processing module is used to extract key features of the output of the multi-scale feature fusion structure through a parallel channel attention mechanism and a spatial attention mechanism and output the segmentation result.

Images