A semantic segmentation method for low-contrast images based on global semantic feature fusion

Abstract

The present invention relates to a low-contrast image semantic segmentation method based on global semantic feature fusion, comprising the following steps: obtaining a data set containing multiple sample images, annotating the sample images in the data set to assign the category to which each pixel in the image belongs; preprocessing the annotated sample images to obtain a preprocessed data set; inputting the preprocessed data set into a constructed image semantic segmentation model to perform model training to obtain a trained image semantic segmentation model; performing standardization processing on the low-contrast image to be segmented; inputting the standardized low-contrast image to be segmented into the trained image semantic segmentation model to output the corresponding segmentation result. Compared with the prior art, the present invention effectively solves the problem of low-contrast target segmentation in image semantic segmentation, and can accurately perform semantic segmentation on low-contrast images.

Description

Technical Field

[0001] The present invention relates to the technical field of image semantic segmentation, and in particular to a low-contrast image semantic segmentation method based on global semantic feature fusion. Background Art

[0002] With the large-scale application of deep learning in the field of artificial intelligence in recent years, the semantic segmentation technology in deep learning has been widely used in various image processing tasks, such as unmanned driving, aerial image analysis, and medical image analysis. However, the existing image semantic segmentation technology cannot solve the image segmentation problem of low-contrast segmentation targets. Since the color and texture of the segmented target are similar to the background and the contour is fuzzy, it is difficult for the model to segment the target. Traditional image semantic segmentation algorithms generally use threshold-based, region-based, or clustering-based algorithm models, but these algorithm models have low robustness. In real-world scene applications, due to the large changes in the image acquisition environment, such as different shooting angles, different lighting conditions, and different forms of the same target, traditional image semantic segmentation models are difficult to apply in low-contrast target segmentation tasks.

[0003] The difficulty of the current deep learning-based image semantic segmentation technology in solving the problem of low-contrast target segmentation is that the model is affected by the inherent properties of the convolution operator in the convolutional network. The model can only extract local features based on the convolution operator to classify the pixels in the image, but the features extracted by the model lack global information, making the performance of the low-contrast target segmentation task more susceptible to noise. In addition, in order to reduce the impact of the detailed features lost during the downsampling of the image, the existing model uses skip connections to splice the features output by the encoder with the features output by the decoder. However, skip connections ignore the large amount of redundant information in the low-level semantic features, which will reduce the accuracy of the model output results. Summary of the invention

[0004] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide a low-contrast image semantic segmentation method based on global semantic feature fusion, so as to accurately perform semantic segmentation on low-contrast images.

[0005] The object of the present invention can be achieved by the following technical scheme: A low-contrast image semantic segmentation method based on global semantic feature fusion comprises the following steps:

[0006] S1. Obtain a data set containing multiple sample images, and annotate the sample images in the data set to assign a category to each pixel in the image;

[0007] S2, preprocessing the labeled sample images to obtain a preprocessed data set;

[0008] S3, inputting the preprocessed data set into the constructed image semantic segmentation model to perform model training, thereby obtaining a trained image semantic segmentation model;

[0009] S4, standardizing the low-contrast image to be segmented;

[0010] S5. Input the standardized low-contrast image to be segmented into the trained image semantic segmentation model, and output the corresponding segmentation result.

[0011] Furthermore, the step S2 specifically includes the following steps:

[0012] S21, performing data value standardization processing on the labeled sample image;

[0013] S22, rotating, flipping, scaling, and randomly covering the sample image after the standardization processing according to the set probability value;

[0014] S23. Obtain the preprocessed data set.

[0015] Furthermore, the image semantic segmentation model includes an encoder layer, a global information extraction module, a high-level semantic guidance module, a decoder layer and a prediction layer, wherein the encoder in the encoder layer is used to obtain image features at different semantic levels of the image;

[0016] The global information extraction module is used to obtain the global information of the encoder and decoder output features, and to mine the potential relationship between the sample images of the data set;

[0017] The high-level semantic guidance module is used to obtain a spatial attention matrix from the decoder output and the encoder output features;

[0018] The decoder in the decoder layer is used to decode the input features of the previous layer;

[0019] The prediction layer is used to perform segmentation prediction on the decoder layer output features.

[0020] Furthermore, the number of encoders in the encoder layer is consistent with the number of decoders in the decoder layer.

[0021] Furthermore, the encoder and the decoder are respectively connected to a global information extraction module, and the global information extraction module is connected to a high-level semantic guidance module.

[0022] Furthermore, the output of the encoder is multiplied by the output of the high-level semantic guidance module, and then summed with the output of the decoder before inputting into the next layer encoder.

[0023] Furthermore, the step S3 specifically includes the following steps:

[0024] S31, inputting the sample image in the preprocessed data set into the encoder layer, and the encoder obtains image features of different semantic levels of the image;

[0025] S32, obtaining global information about the features output by the encoder through a global information extraction module, and mining potential relationships between sample images in the data set using the memory capability of the global information extraction module;

[0026] S35, decoding the input features of the previous layer through the decoder;

[0027] S36, input the features output by the decoder into the global information extraction module, and concatenate the output features with the global semantic features of the encoder, and input the concatenated features into the next layer of decoder to obtain the output result;

[0028] S37, inputting the output features of the decoder and the encoder containing global information into the high-level semantic guidance module to obtain a spatial attention matrix, calculating the Hadamard product of the spatial attention matrix and the low-level semantic features output by the encoder, then concatenating the calculated Hadamard product with the high-level semantic features output by the decoder, and then inputting the next layer of decoder to reduce the influence of redundant information in the low-level semantic features;

[0029] S38, predicting the decoder output features through the prediction layer, and outputting the segmentation prediction result;

[0030] S39, until each sample image in the data set completes the process of steps S31 to S38, one training is completed, and then iterative training is performed until the model converges to obtain a trained image semantic segmentation model.

[0031] Furthermore, the specific working process of the global information extraction module is as follows:

[0032] Input feature X;

[0033] Flatten the input X according to the channel dimension and input it into the multi-layer perceptron;

[0034] The feature X obtained in the previous step is split in the channel dimension, and the split features are input into the multi-layer perceptron;

[0035] Input the feature X obtained in the previous step into the numerical normalization layer;

[0036] Input the feature X obtained in the previous step into the multi-layer perceptron and merge the segmented features;

[0037] Input the feature X obtained in the previous step into the multi-layer perceptron.

[0038] Furthermore, the specific working process of the high-level semantic guidance module is as follows:

[0039] Input high-level semantic features X h and low-level semantic features X l ;

[0040] Concatenate high-level semantic features X h and low-level semantic features X l ;

[0041] Input the feature X obtained from the upper layer into the attention layer;

[0042] The attention weights α, 1-α obtained from the upper level are combined with the high-level semantic features X h and low-level semantic features X l Multiply and sum them separately;

[0043] The feature X obtained in the previous layer is convolved and normalized.

[0044] Furthermore, the condition for the model convergence in step S39 is specifically that the loss function reaches convergence.

[0045] Compared with the prior art, the present invention sets a global information extraction module in the image semantic segmentation model to obtain global semantic information of features at different semantic levels and to mine the potential relationship between different samples in the data set; sets a high-level semantic guidance module in the image semantic segmentation model to fuse high-level global semantic features with low-level global semantic features, thereby obtaining a spatial attention matrix, guiding the jump connection of low-level semantic feature graphs, and effectively reducing the influence of redundant information in features. Thus, the present invention solves the problem of low-contrast target segmentation in image semantic segmentation, and can accurately segment low-contrast images. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a schematic diagram of the method flow of the present invention;

[0047] Figure 2 Schematic diagram of the working process of the image semantic segmentation model in the present invention;

[0048] Figure 3 Schematic diagram of the architecture of the image semantic segmentation model in the present invention;

[0049] Figure 4 It is a schematic diagram of the structure of the global information extraction module in the image semantic segmentation model;

[0050] Figure 5 Schematic diagram of the structure of the high-level semantic guidance module in the image semantic segmentation model. DETAILED DESCRIPTION

[0051] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0052] Example

[0053] A low-contrast image semantic segmentation method based on global semantic feature fusion includes the following steps:

[0054] S1. Obtain a data set containing multiple sample images, and annotate the sample images in the data set to assign a category to each pixel in the image;

[0055] S2, preprocessing the labeled sample images to obtain a preprocessed data set. Specifically, firstly, the labeled sample images are subjected to data value normalization processing; then, the standardized sample images are rotated, flipped, scaled, and randomly masked with a set probability value;

[0056] S3. Input the preprocessed data set into the constructed image semantic segmentation model to perform model training to obtain a trained image semantic segmentation model. In this technical solution, the image semantic segmentation model includes an encoder layer, a global information extraction module, a high-level semantic guidance module, a decoder layer and a prediction layer, wherein the number of encoders in the encoder is consistent with the number of decoders in the decoder layer, the encoder and the decoder are respectively connected to the global information extraction module, the global information extraction module is connected to the high-level semantic guidance module, the output of the encoder is multiplied by the output of the high-level semantic guidance module, and then summed with the output of the decoder before inputting into the next layer of encoder;

[0057] Specifically, the encoder in the encoder layer is used to obtain image features at different semantic levels of the image;

[0058] The global information extraction module is used to obtain the global information of the encoder output features and to mine the potential relationship between the sample images of the dataset;

[0059] The high-level semantic guidance module is used to obtain the spatial attention matrix from the decoder output and encoder output features;

[0060] The decoder in the decoder layer is used to decode the input features of the previous layer;

[0061] The prediction layer is used to perform segmentation prediction on the output features of the decoder layer;

[0062] When training the model, the following steps are mainly included:

[0063] S31, inputting the sample image in the preprocessed data set into the encoder layer, and the encoder obtains image features of different semantic levels of the image;

[0064] S32, obtaining global information about the features output by the encoder through a global information extraction module, and mining potential relationships between sample images in the data set using the memory capability of the global information extraction module;

[0065] S35, decoding the input features of the previous layer through the decoder;

[0066] S36, input the features output by the decoder into the global information extraction module, and concatenate the output features with the global semantic features of the encoder, and input the concatenated features into the next layer of decoder to obtain the output result;

[0067] S37, inputting the output features of the decoder and the encoder containing global information into the high-level semantic guidance module to obtain a spatial attention matrix, calculating the Hadamard product of the spatial attention matrix and the low-level semantic features output by the encoder, then concatenating the calculated Hadamard product with the high-level semantic features output by the decoder, and then inputting the next layer of decoder to reduce the influence of redundant information in the low-level semantic features;

[0068] S38, predicting the decoder output features through the prediction layer, and outputting the segmentation prediction result;

[0069] S39, until each sample image in the data set completes the process of steps S31 to S38, then one training is completed, and then iterative training is performed until the model converges to obtain a trained image semantic segmentation model;

[0070] S4, standardizing the low-contrast image to be segmented;

[0071] S5. Input the standardized low-contrast image to be segmented into the trained image semantic segmentation model, and output the corresponding segmentation result.

[0072] This embodiment applies the above technical solution. Figure 2 The following is the workflow of the image semantic segmentation module. Let X be the input image. First, the data set required for training is annotated, and each pixel in the sample image in the data set is assigned a category to which the pixel belongs.

[0073] Then the input model data is preprocessed: the data values ​​are normalized; the input image is rotated, flipped, scaled, and randomly masked with a certain probability;

[0074] Then, the encoder is used to obtain image features at different semantic levels of the image, and the global semantic information acquisition module is used to obtain global information about the features output by the encoder. The memory capacity of the global semantic information acquisition module is used to mine the potential relationship between samples in the data set.

[0075] The decoder decodes the input features of the previous layer, inputs the output features of the decoder and encoder containing global information into the high-level semantic guidance module to obtain the spatial attention matrix, calculates the Hadamard product of the spatial attention matrix and the low-level semantic features output by the encoder, and then concatenates the calculated Hadamard product with the high-level semantic features output by the decoder, and then inputs it into the decoder of the next layer to reduce the influence of redundant information in the low-level semantic features;

[0076] Use the model prediction layer to predict the decoder output features and output the prediction results. Iterate all the training processes of the model until the model converges and obtains the image semantic segmentation model.

[0077] like Figure 3 As shown in Figure 1, the image semantic segmentation model architecture of this technical solution is divided into five parts: image encoding layer, image decoding layer, global information feature extraction layer, semantic guidance module, and prediction layer. During the training process, the global information extraction module works as follows: Figure 4 As shown:

[0078] 1: Input feature X. => 1,X.

[0079] 2: Flatten the input X according to the channel dimension and input it into the multi-layer perceptron. 1,=>2,MLP(flaten(X)).

[0080] 3: Split the feature X obtained in the previous step in the channel dimension and input the split features into the multi-layer perceptron. 2=>3, MLP(split(X)).

[0081] 4: Input the feature X obtained in the previous step into the numerical normalization layer. 3,=>4,Norm(X).

[0082] 5: Input the feature X obtained in the previous step into the multi-layer perceptron and merge the split features. 4,=>5,Concat(MLP(X)).

[0083] 6: Input the feature X obtained in the previous step into the multi-layer perceptron, 5,=>6,MLP(X).

[0084] For the high-level semantic guidance module, its working process is as follows: Figure 5 As shown:

[0085] 1: Input high-level semantic features X h and low-level semantic features X l . =>1,X l ,X h .

[0086] 2: Concatenate high-level semantic features X h and low-level semantic features Xl . 1,=>2,Concat(X l ,X h ).

[0087] 3: Input the feature X obtained from the previous layer into the attention layer. 2,=>3,Sigmoid(Conv2d(X)).

[0088] 4: Combine the attention weights α, 1-α obtained from the upper level with the high-level semantic features X h and low-level semantic features X l Multiply them and sum them. 3,=>4,α×X l +(1-α)×X h .

[0089] 5: Convolve and normalize the feature X obtained in the previous layer. 4,=>5,Norm(Conv2d(X)).

[0090] Among them, the symbol => indicates that the output of the layer represented by the Arabic numeral before the symbol is input into the layer represented by the Arabic numeral after the symbol. MLP stands for multi-layer perceptron network; flatten represents feature flattening operation, for example, flattening 4×7×7 features into 4×49; Split represents feature segmentation, for example, segmenting 4×49 features into 4×1×49 features; Norm represents normalization, which is implemented using the SoftMax function in this embodiment; Concat represents concatenation, for example, concatenating two 1×1×49 dimensional features into 2×1×49 features; Conv2d represents a convolutional layer with a step size of 3×3 and a step size of 1 to fill 1 pixel, and a Relu function is used for activation after the convolutional layer.

Claims

1. A low-contrast image semantic segmentation method based on global semantic feature fusion, characterized in that: The following steps are involved: S1. Obtain a data set containing multiple sample images, and annotate the sample images in the data set to assign a category to each pixel in the image; S2, preprocessing the labeled sample images to obtain a preprocessed data set; S3, inputting the preprocessed data set into the constructed image semantic segmentation model to perform model training, thereby obtaining a trained image semantic segmentation model; S4, standardizing the low-contrast image to be segmented; S5, inputting the standardized low-contrast image to be segmented into the trained image semantic segmentation model, and outputting the corresponding segmentation result; The image semantic segmentation model includes an encoder layer, a global information extraction module, a high-level semantic guidance module, a decoder layer and a prediction layer, wherein the encoder in the encoder layer is used to obtain image features at different semantic levels of the image; The global information extraction module is used to obtain the global information of the encoder and decoder output features, and to mine the potential relationship between the sample images of the data set; The high-level semantic guidance module is used to obtain a spatial attention matrix from the decoder output and the encoder output features; The decoder in the decoder layer is used to decode the input features of the previous layer; The prediction layer is used to perform segmentation prediction on the output features of the decoder layer; The encoder and the decoder are respectively connected to a global information extraction module, and the global information extraction module is connected to a high-level semantic guidance module; The output of the encoder is multiplied by the output of the high-level semantic guidance module, and then summed with the output of the decoder before being input into the next layer encoder.

2. The low-contrast image semantic segmentation method based on global semantic feature fusion according to claim 1, characterized in that: The step S2 specifically includes the following steps: S21, performing data value standardization processing on the labeled sample image; S22, rotating, flipping, scaling, and randomly masking the sample image after the standardization processing according to the set probability value; S23. Obtain the preprocessed data set.

3. The low-contrast image semantic segmentation method based on global semantic feature fusion according to claim 1, characterized in that: The number of encoders in the encoder layer is consistent with the number of decoders in the decoder layer.

4. The low-contrast image semantic segmentation method based on global semantic feature fusion according to claim 1, characterized in that: The step S3 specifically comprises the following steps: S31, inputting the sample image in the preprocessed data set into the encoder layer, and the encoder obtains image features of different semantic levels of the image; S32, obtaining global information about the features output by the encoder through a global information extraction module, and mining potential relationships between sample images in the data set using the memory capability of the global information extraction module; S35, decoding the input features of the previous layer through the decoder; S36, input the features output by the decoder into the global information extraction module, and concatenate the output features with the global semantic features of the encoder, and input the concatenated features into the next layer of decoder to obtain the output result; S37, inputting the output features of the decoder and the encoder containing global information into the high-level semantic guidance module to obtain a spatial attention matrix, calculating the Hadamard product of the spatial attention matrix and the low-level semantic features output by the encoder, then concatenating the calculated Hadamard product with the high-level semantic features output by the decoder, and then inputting the next layer of decoder to reduce the influence of redundant information in the low-level semantic features; S38, predicting the decoder output features through the prediction layer, and outputting the segmentation prediction result; S39, until each sample image in the data set completes the process of steps S31 to S38, then one training is completed, and then iterative training is performed until the model converges to obtain a trained image semantic segmentation model.

5. The low-contrast image semantic segmentation method based on global semantic feature fusion according to claim 4, characterized in that: The specific working process of the global information extraction module is as follows: Input feature X; Flatten the input X according to the channel dimension and input it into the multi-layer perceptron; The feature X obtained in the previous step is split in the channel dimension, and the split features are input into the multi-layer perceptron; Input the feature X obtained in the previous step into the numerical normalization layer; Input the feature X obtained in the previous step into the multi-layer perceptron and merge the segmented features; Input the feature X obtained in the previous step into the multi-layer perceptron.

6. The low-contrast image semantic segmentation method based on global semantic feature fusion according to claim 4, characterized in that: The specific working process of the high-level semantic guidance module is as follows: Input high-level semantic features X h and low-level semantic features X l ; Concatenate high-level semantic features X h and low-level semantic features X l ; Input the feature X obtained from the upper layer into the attention layer; The attention weights α, 1-α obtained from the upper level are combined with the high-level semantic features X h and low-level semantic features X l Multiply and sum them separately; The feature X obtained in the previous layer is convolved and normalized.

7. The low-contrast image semantic segmentation method based on global semantic feature fusion according to claim 4, characterized in that: The condition for model convergence in step S39 is specifically that the loss function reaches convergence.

Images