A method for extracting Antarctic ice cracks based on semi-supervised learning

Abstract

The invention relates to an Antarctic ice crack extraction method based on semi-supervised learning, comprising: labeling part of original Antarctic ice crack images to obtain binary image labels; pre-training an Antarctic ice crack extraction model using the labeled original Antarctic ice crack images; the Antarctic ice crack extraction model comprises: a PVT encoder, a spatial pyramid pooling layer, an attention mechanism module, a 1*1 convolution module, and a 3*3 convolution module; cutting two image blocks with overlapping areas from unlabeled original Antarctic ice crack images, respectively inputting the two image blocks into the pre-trained Antarctic ice crack extraction model to extract original comprehensive enhancement features and crack segmentation maps of the image blocks; constructing a directional contrast loss function according to the original comprehensive enhancement features and crack segmentation maps corresponding to the two image blocks, and re-training the Antarctic ice crack extraction model; and using the trained Antarctic ice crack extraction model to perform crack extraction on the Antarctic ice crack image to be extracted.

Description

Technical Field

[0001] The invention belongs to the technical field of image segmentation, and in particular relates to an Antarctic ice crack extraction method based on semi-supervised learning. Background Art

[0002] Antarctic ice shelves play an important role in the mass balance of the ice sheet and the associated global sea level changes. The disintegration and collapse of these ice shelves will accelerate Antarctic ice flow and mass loss, which will accelerate the rate of sea level rise. Ice cracks are widely present in Antarctica and are a direct result of glacier movement. They are affected by and affect the stress and surface mass balance of glaciers. Glacier collapse is caused by the expansion of cracks under pressure. Therefore, the detection of ice cracks is a key step in analyzing the characteristics of crack propagation, understanding the glacier collapse process, and predicting glacier collapse.

[0003] In previous studies, optical images, radar data, and altimetry data have been widely used to detect and extract surface cracks. However, manual feature extraction is time-consuming and laborious, and it is difficult to quickly map cracks across the entire continent. Therefore, with the rise of machine learning, machine learning-based methods have been widely used in cryosphere research.

[0004] In machine learning, sufficient labeled training data is necessary for the model to achieve satisfactory performance, which is particularly important for deep learning-based methods. However, the cracks on the surface of the Antarctic ice sheet vary in scale and distribution type, and each pixel in the satellite image can represent a large area on the ground, so domain knowledge and experience are required to annotate pixel-level labels, which makes accurate pixel-level annotation of all cracks in the study area time-consuming and labor-intensive. In order to alleviate this high demand for labeled training data, many semi-supervised learning methods have been proposed to make full use of a large number of unlabeled images. These semi-supervised methods extract information from a small amount of labeled data in a fully supervised manner, and extract information from a large amount of unlabeled data in an unsupervised manner, thereby reducing the heavy labeling work in the fully supervised scenario and achieving experimental results comparable to or even exceeding those in the fully supervised scenario.

[0005] The segmentation network cannot predict the label of each pixel based only on the RGB value of the image. The contextual information of the image is crucial for semantic segmentation. Recently, contrastive learning methods have successfully promoted the development of image segmentation in unsupervised and semi-supervised learning, which focuses on high-level features to distinguish different classes in the absence of true values. However, in a semi-supervised environment, the model is prone to overfitting the limited training data, resulting in poor generalization performance of the model. In this case, the model is prone to over-relying on contextual information for prediction, resulting in low accuracy of crack extraction. Summary of the invention

[0006] In order to solve the problems existing in the background technology, the present invention provides an Antarctic ice crack extraction method based on semi-supervised learning, comprising:

[0007] S1: Obtain the original Antarctic ice crack image set, use Labelme to annotate some of the original Antarctic ice crack images, and obtain the corresponding binary image labels;

[0008] S2: Pre-training the Antarctic ice crack extraction model using the annotated original Antarctic ice crack images, wherein the Antarctic ice crack extraction model includes: a PVT encoder, a spatial pyramid pooling layer, an attention mechanism module, a 1*1 convolution module, and a 3*3 convolution module;

[0009] The pre-training of the Antarctic ice crack extraction model includes:

[0010] S21: Input the annotated original Antarctic ice crack image into the PVT encoder to obtain original features at multiple scales;

[0011] S22: Input the original features with the smallest scale into the spatial pyramid pooling layer for semantic enhancement processing to obtain the original semantic enhancement features; input the original semantic enhancement features into the attention mechanism module for attention enhancement to obtain the original comprehensive enhancement features;

[0012] S23: Perform a 1*1 convolution operation on the original feature with the largest scale to obtain a first feature map; upsample the original comprehensive enhanced feature to the same size as the first feature map to obtain a second feature map; concatenate the first feature map and the second feature map using the ConCat function to obtain a third feature map;

[0013] S24: performing a 3*3 convolution operation on the third feature map to obtain a fourth feature map; upsampling the fourth feature map to the same size as the binary label of the original Antarctic ice crack image to obtain a crack segmentation map of the original Antarctic ice crack image;

[0014] S25: construct a cross entropy loss function based on the crack segmentation map of the original Antarctic ice crack image and the binary label of the original Antarctic ice crack image, use the adam optimizer to optimize the parameters of the Antarctic ice crack extraction model, and complete the pre-training of the Antarctic ice crack extraction model;

[0015] S3: Randomly crop two overlapping image patches x from the unlabeled original Antarctic ice crack image u1 and x u2 ;

[0016] S4: Two image blocks with overlapping areas x u1 and x u2 Input the pre-trained Antarctic ice crack extraction model to extract image blocks x u1 and x u2The corresponding original comprehensive enhanced feature f u1 and f u2 , and the image patch x u1 and x u2 The corresponding crack segmentation map y u1 and u2 ;

[0017] S6: Transform the image block x u1 and x u2 The corresponding original comprehensive enhanced feature f u1 and f u2 Perform nonlinear mapping to obtain mapping feature β u1 =β(f u1 ) and β u2 =β(f u2 ), β represents the nonlinear mapping function;

[0018] S7: According to the image block x u1 and x u2 The overlapping area of ​​the mapping feature β u1 and β u2 The feature representation in, and the image block x u1 and x u2 The overlapping area of ​​the crack segmentation map y u1 and u2 The feature representation in constructs the directional contrast loss function;

[0019] S8: According to the directional contrast loss function, the parameters of the Antarctic ice crack extraction model are optimized using the adam optimizer to complete the training of the Antarctic ice crack extraction model, and the trained Antarctic ice crack extraction model is used to extract cracks from the Antarctic ice crack images to be extracted.

[0020] Preferably, the use of Labelme to label part of the Antarctic ice crack images includes: setting the pixel value of the crack area to 1, and setting the pixel value of the non-crack area to 0.

[0021] Preferably, the step of enhancing the original features comprises:

[0022] CHannelattention=X*Sigmoid(MLP(Avgpool(X)+Maxpool(X)))

[0023] Among them, CHannelattention represents the original comprehensive enhanced features, X represents the original features with the smallest scale, Sigmoid represents the activation function, MLP represents the multi-layer perceptron, Avgpool represents the global average pooling, and Maxpool represents the global maximum pooling.

[0024] Preferably, the cross entropy loss function includes:

[0025]

[0026] in, represents the cross entropy loss function, y i Represents the binary label of the original Antarctic ice crack image, P(x i ) represents the crack segmentation map of the original Antarctic ice crack image, x i represents the original Antarctic ice crack image, and N represents the number of sample points.

[0027] Preferably, the directional contrast loss function includes:

[0028]

[0029] in, Represents a binary mask indicating two pixel features and β n The corresponding pixel point in the crack segmentation map y u1 and u2 Features in and n Is it different? Used to indicate two pixel features and β n The corresponding pixel point in the crack segmentation map y u1 and u2 Features in and n Is it different? Indicates the direction of loss, and Represents image block x u1 and x u2 The overlapping area of ​​the mapping feature β u1 and β u2 The feature representation in ; γ represents the threshold for filtering low confidence positive samples; Fu represents the pixel feature set of non-overlapping areas, and B represents the batch size of training.

[0030] The present invention has at least the following beneficial effects

[0031] The present invention uses a hybrid encoder composed of a multi-scale visual transformer and a spatial pyramid pooling layer structure based on channel attention as the basic encoder for Antarctic ice crack extraction, and uses a method for maintaining context-aware consistency between pixels in different environments to make the framework more robust. Specifically, a multi-scale visual transformer is used as the basic unit of the encoder. Unlike the standard visual transformer, the introduced multi-scale transformer realizes progressive contraction through a pyramid structure, so that the network can flexibly learn multi-scale and high-resolution characteristics, and the spatial convolution pooling pyramid structure based on channel attention further improves the ability of the network to learn multi-scale and high-resolution characteristics. At the same time, two patches are randomly cropped from an unlabeled image and confined to an overlapping area, so that the overlapping area is placed in two different environments. In this case, even if the true value is unknown, because there is a one-to-one correspondence at the pixel level between the overlapping area of ​​the two patches, the consistency of high-level features in different environments can be maintained. At the same time, in order to achieve consistency, the word uses a directional contrast loss to align low-quality features with high-quality features. The present invention can effectively learn from unlabeled images and alleviate the workload of labeling data. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a schematic diagram of the method flow of the present invention;

[0033] Figure 2 It is a schematic diagram of the model structure of the present invention;

[0034] Figure 3 Schematic diagram of the structure of the multi-scale visual transformer of the present invention. DETAILED DESCRIPTION

[0035] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0036] Among them, the drawings are only used for illustrative explanations, and they only represent schematic diagrams rather than actual pictures, and should not be understood as limitations on the present invention. In order to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0037] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if the terms "upper", "lower", "left", "right", "front", "rear", etc. indicate the orientation or position relationship, they are based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation. Therefore, the terms describing the position relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0038] See also Figure 1 The present invention provides a method for extracting Antarctic ice cracks based on semi-supervised learning, comprising:

[0039] S1: Obtain the original Antarctic ice crack image set, use Labelme to annotate some of the original Antarctic ice crack images, and obtain the corresponding binary image labels;

[0040] See also Figure 2 , S2: pre-training the Antarctic ice crack extraction model using the annotated original Antarctic ice crack image, the Antarctic ice crack extraction model includes: a PVT encoder, a spatial pyramid pooling layer, an attention mechanism module, a 1*1 convolution module, and a 3*3 convolution module;

[0041] The pre-training of the Antarctic ice crack extraction model includes:

[0042] S21: Input the annotated original Antarctic ice crack image into the PVT encoder to obtain original features at multiple scales;

[0043] S22: Input the original features with the smallest scale into the spatial pyramid pooling layer for semantic enhancement processing to obtain the original semantic enhancement features; input the original semantic enhancement features into the attention mechanism module for attention enhancement to obtain the original comprehensive enhancement features;

[0044] The attention mechanism module is integrated with the spatial pyramid pooling layer module, and different weights are assigned to different feature maps to enhance the channel features of the input features extracted from different scales by the spatial convolution pooling pyramid, and then the final feature extraction result is obtained, so as to more effectively extract the features of image segmentation.

[0045] S23: Perform a 1*1 convolution operation on the original feature with the largest scale to obtain a first feature map; upsample the original comprehensive enhanced feature to the same size as the first feature map to obtain a second feature map; concatenate the first feature map and the second feature map using the ConCat function to obtain a third feature map;

[0046] S24: performing a 3*3 convolution operation on the third feature map to obtain a fourth feature map; upsampling the fourth feature map to the same size as the binary label of the original Antarctic ice crack image to obtain a crack segmentation map of the original Antarctic ice crack image;

[0047] S25: construct a cross entropy loss function based on the crack segmentation map of the original Antarctic ice crack image and the binary label of the original Antarctic ice crack image, use the adam optimizer to optimize the parameters of the Antarctic ice crack extraction model, and complete the pre-training of the Antarctic ice crack extraction model;

[0048] S3: Randomly crop two overlapping image patches x from the unlabeled original Antarctic ice crack image u1 and x u2 ;

[0049] S4: Two image blocks with overlapping areas x u1 and x u2 Input the pre-trained Antarctic ice crack extraction model to extract image blocks x u1 and x u2 The corresponding original comprehensive enhanced feature f u1 and f u2 , and the image patch x u1 and x u2 The corresponding crack segmentation map y u1 and u2 ;

[0050] In this case, even if the true value is unknown, the consistency of high-level features in different environments can be maintained because there is a one-to-one correspondence at the pixel level in the overlapping area between the two patches.

[0051] S6: Transform the image block x u1 and x u2 The corresponding original comprehensive enhanced feature f u1 and f u2 Perform nonlinear mapping to obtain mapping feature β u1 =β(f u1 ) and β u2 =β(f u2 ), β represents the nonlinear mapping function;

[0052] S7: According to the image block x u1 and xu2 The overlapping area of ​​the mapping feature β u1 and β u2 The feature representation in, and the image block x u1 and x u2 The overlapping area of ​​the crack segmentation map y u1 and u2 The feature representation in constructs the directional contrast loss function;

[0053] S8: According to the directional contrast loss function, the parameters of the Antarctic ice crack extraction model are optimized using the adam optimizer to complete the training of the Antarctic ice crack extraction model, and the trained Antarctic ice crack extraction model is used to extract cracks from the Antarctic ice crack images to be extracted.

[0054] Preferably, the PVT feature encoder is used to extract the global semantic features of the image. After an image is input into the PVT feature encoder, multiple original features of different scales can be obtained. In this embodiment, the PVT feature encoder includes 4 stages, each stage corresponds to a stage, and the original Antarctic ice crack image is input into the PVT feature encoder. An original feature is output at each stage. The scale of the original feature output from each stage is different. The scales of the original features output from stage1 to stage4 are from large to small. Each stage is regarded as a stage, and P is used to represent the original features of the image. j Represents the patch size of stage j. At the beginning of stage j, the original features of the input are first Divide evenly into Each patch is then flattened and mapped to C j In the latitude and longitude embedding, after linear mapping, the shape of the embedding patch can be regarded as Among them C j Indicates the number of channels at stage j.

[0055] See also Figure 3 ,Preferably, because it is based on a pyramid structure for processing dense prediction, its computational complexity is relatively large, so SRA needs to be used in each stage to reduce the size of the feature map to achieve the purpose of reducing the amount of calculation. SRA accepts query, key and value as input. The difference is that the SRA layer will reduce the spatial scale of key and value before the attention operation, thereby reducing the computational overhead.

[0056] The details of SRA in stage j are as follows:

[0057]

[0058] Among them, N j represents the number of heads in the attention layer of the jth stage, is the linear projection parameter, and the SR operation is as follows:

[0059] SR(x)=Norm(Reshape(x,R i )W S )

[0060] Among them, R j Reshape(x,R j ) is the input image Resize to operation.

[0061] Preferably, the use of Labelme to label part of the Antarctic ice crack images includes: setting the pixel value of the crack area to 1, and setting the pixel value of the non-crack area to 0.

[0062] Preferably, the step of enhancing the original features comprises:

[0063] CHannelattention=X*Sigmoid(MLP(Avgpool(X)+Maxpool(X)))

[0064] Among them, CHannelattention represents the original comprehensive enhanced features, X represents the original features with the smallest scale, Sigmoid represents the activation function, MLP represents the multi-layer perceptron, Avgpool represents the global average pooling, and Maxpool represents the global maximum pooling.

[0065] Preferably, the cross entropy loss function includes:

[0066]

[0067] in, represents the cross entropy loss function, y i Represents the binary label of the original Antarctic ice crack image, P(x i ) represents the crack segmentation map of the original Antarctic ice crack image, x i represents the original Antarctic ice crack image, and N represents the number of sample points.

[0068] Preferably, the directional contrast loss function includes:

[0069]

[0070] in, Represents a binary mask indicating two pixel features and β n The corresponding pixel point in the crack segmentation map y u1 and u2 Features in and n Is it different? Used to indicate two pixel features and β n The corresponding pixel point in the crack segmentation map y u1 and u2 Features in and n Is it different? Indicates the direction of loss, and Represents image block x u1 and x u2 The overlapping area of ​​the mapping feature β u1 and β u2 The feature representation in ; γ represents the threshold for filtering low confidence positive samples; Fu represents the pixel feature set of non-overlapping areas, and B represents the batch size of training.

[0071] The present invention uses a hybrid encoder composed of a multi-scale visual transformer and a spatial pyramid pooling layer structure based on channel attention as the basic encoder for Antarctic ice crack extraction, and uses a method for maintaining context-aware consistency between pixels in different environments to make the framework more robust. Specifically, a multi-scale visual transformer is used as the basic unit of the encoder. Unlike the standard visual transformer, the introduced multi-scale transformer realizes progressive contraction through a pyramid structure, so that the network can flexibly learn multi-scale and high-resolution characteristics, and the spatial convolution pooling pyramid structure based on channel attention further improves the ability of the network to learn multi-scale and high-resolution characteristics. At the same time, two patches are randomly cropped from an unlabeled image and confined to an overlapping area, so that the overlapping area is placed in two different environments. In this case, even if the true value is unknown, because there is a one-to-one correspondence at the pixel level between the overlapping area of ​​the two patches, the consistency of high-level features in different environments can be maintained. At the same time, in order to achieve consistency, the word uses a directional contrast loss to align low-quality features with high-quality features. The present invention can effectively learn from unlabeled images and alleviate the workload of labeling data.

[0072] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A method for extracting Antarctic ice cracks based on semi-supervised learning, characterized in that: include: S1: Obtain the original Antarctic ice crack image set, use Labelme to annotate some of the original Antarctic ice crack images, and obtain the corresponding binary image labels; S2: Pre-training the Antarctic ice crack extraction model using the annotated original Antarctic ice crack images, wherein the Antarctic ice crack extraction model includes: a PVT encoder, a spatial pyramid pooling layer, an attention mechanism module, a 1*1 convolution module, and a 3*3 convolution module; The pre-training of the Antarctic ice crack extraction model includes: S21: Input the annotated original Antarctic ice crack image into the PVT encoder to obtain original features at multiple scales; S22: Input the original features with the smallest scale into the spatial pyramid pooling layer for semantic enhancement processing to obtain the original semantic enhancement features; input the original semantic enhancement features into the attention mechanism module for attention enhancement to obtain the original comprehensive enhancement features; S23: Perform a 1*1 convolution operation on the original feature with the largest scale to obtain a first feature map; upsample the original comprehensive enhanced feature to the same size as the first feature map to obtain a second feature map; concatenate the first feature map and the second feature map using the ConCat function to obtain a third feature map; S24: performing a 3*3 convolution operation on the third feature map to obtain a fourth feature map; upsampling the fourth feature map to the same size as the binary label of the original Antarctic ice crack image to obtain a crack segmentation map of the original Antarctic ice crack image; S25: construct a cross entropy loss function based on the crack segmentation map of the original Antarctic ice crack image and the binary label of the original Antarctic ice crack image, use the adam optimizer to optimize the parameters of the Antarctic ice crack extraction model, and complete the pre-training of the Antarctic ice crack extraction model; S3: Randomly crop two overlapping image patches x from the unlabeled original Antarctic ice crack image u1 and x u2 ; S4: Two image blocks with overlapping areas x u1 and x u2 Input the pre-trained Antarctic ice crack extraction model to extract image blocks x u1 and x u2 The corresponding original comprehensive enhanced feature f u1 and f u2 , and the image patch x u1 and x u2 The corresponding crack segmentation map y u1 and u2 ; S6: Transform the image block x u1 and x u2 The corresponding original comprehensive enhanced feature f u1 and f u2 Perform nonlinear mapping to obtain mapping feature β u1 =β(f u1 ) and β u2 =β(f u2 ), β represents the nonlinear mapping function; S7: According to the image block x u1 and x u2 The overlapping area of ​​the mapping feature β u1 and β u2 The feature representation in, and the image block x u1 and x u2 The overlapping area of ​​the crack segmentation map y u1 and u2 The feature representation in constructs the directional contrast loss function; S8: According to the directional contrast loss function, the parameters of the Antarctic ice crack extraction model are optimized using the adam optimizer to complete the training of the Antarctic ice crack extraction model, and the trained Antarctic ice crack extraction model is used to extract cracks from the Antarctic ice crack images to be extracted.

2. The method for extracting Antarctic ice cracks based on semi-supervised learning according to claim 1, characterized in that: The labeling of some Antarctic ice crack images using Labelme includes: setting the pixel value of the crack area to 1, and setting the pixel value of the non-crack area to 0.

3. The method for extracting Antarctic ice cracks based on semi-supervised learning according to claim 2, characterized in that: The feature enhancement of the original feature comprises: CHannelattention=X*Sigmoid(MLP(Avgpool(X)+Maxpool(X))) Among them, CHannelattention represents the original comprehensive enhanced features, X represents the original features with the smallest scale, Sigmoid represents the activation function, MLP represents the multi-layer perceptron, Avgpool represents the global average pooling, and Maxpool represents the global maximum pooling.

4. The method for extracting Antarctic ice cracks based on semi-supervised learning according to claim 2, characterized in that: The cross entropy loss function includes: in, represents the cross entropy loss function, y i Represents the binary label of the original Antarctic ice crack image, P(x i ) represents the crack segmentation map of the original Antarctic ice crack image, x i represents the original Antarctic ice crack image, and N represents the number of sample points.

5. The method for extracting Antarctic ice cracks based on semi-supervised learning according to claim 2, characterized in that: The directional contrast loss function includes: in, Represents a binary mask indicating two pixel features and β n The corresponding pixel point in the crack segmentation map y u1 and u2 Features in and n Is it different? Used to indicate two pixel features and β n The corresponding pixel point in the crack segmentation map y u1 and u2 Features in and n Is it different? Indicates the direction of loss, and Represents image block x u1 and x u2 The overlapping area of ​​the mapping feature β u1 and β u2 The feature representation in ; γ represents the threshold for filtering low confidence positive samples; Fu represents the pixel feature set of non-overlapping areas, and B represents the batch size of training.

Images