A semi-supervised remote sensing image semantic segmentation model training method and device

Abstract

The application discloses a semi-supervised remote sensing image semantic segmentation model training method and device, and belongs to the technical field of computer vision. The method comprises the following steps: based on the labeled remote sensing image data, the consistency score of the unlabeled remote sensing image data is calculated, and the unlabeled remote sensing image data is screened; the screened unlabeled remote sensing image data is input into a teacher module to obtain a teacher prediction result; the labeled remote sensing image data and the screened unlabeled remote sensing image data are input into a student module, and the student module is supervised through the teacher prediction result to obtain a student prediction result; and based on the teacher prediction result and the student prediction result, a loss function of the semantic segmentation model is constructed to update the parameters of the semantic segmentation model. The application improves the training effect of the semantic segmentation model from three aspects of unlabeled training data screening, model design and training strategy, and can effectively alleviate the problem that the performance of the semantic segmentation model decreases with the increase of the unlabeled data.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision and relates to a semi-supervised remote sensing image semantic segmentation model training method and device. Background Art

[0002] Semantic segmentation of remote sensing images has important applications in tasks such as land cover classification, but the labeling process is expensive and time-consuming, resulting in a scarcity of labeled data. Semi-supervised semantic segmentation offers a promising solution to this problem. Due to limited data resources, current research dynamically adjusts the ratio of labeled to unlabeled data, simulating the optimal allocation under fixed resources to achieve optimal performance. However, this approach limits the exploration of the model's performance potential in scenarios with large-scale unlabeled data. For example, as the amount of unlabeled data increases, the model often cannot effectively process the increased amount of unlabeled information.

[0003] Research has shown that the main reasons for performance degradation are the uneven quality of unlabeled data, the model's susceptibility to interference from unknown categories, and improper training strategies leading to semantic drift. First, the semi-supervised semantic segmentation task assumes that labeled and unlabeled data share the same distribution. However, in practical applications, this assumption is often difficult to hold. Unlabeled data may contain samples with a distribution that differs significantly from the training data, which interferes with the model's pseudo-label generation and feature learning. Second, existing semi-supervised semantic segmentation models generate labels for unlabeled data through a pseudo-labeling mechanism and utilize these labels during training. However, because unlabeled data may contain unknown categories, the model easily confuses these unknown categories with known categories, affecting classification and segmentation accuracy. Finally, current training methods can lead to the accumulation of erroneous pseudo-labels, causing the model's decision boundary to gradually deviate from the actual data distribution, exacerbating the problem of semantic drift. These three issues do not exist in isolation throughout the training process, but rather influence each other. As the model is iteratively trained, these problems gradually accumulate at different stages, ultimately affecting the overall performance of the model. Summary of the Invention

[0004] The purpose of the present invention is to overcome the deficiencies in the prior art and provide a semi-supervised remote sensing image semantic segmentation model training method and device, which can effectively alleviate the problem of semantic segmentation model performance degradation caused by the increase of unlabeled data.

[0005] To achieve the above object, the present invention is implemented by adopting the following technical solutions:

[0006] In a first aspect, the present invention provides a semi-supervised remote sensing image semantic segmentation model training method, comprising:

[0007] Based on labeled remote sensing image data, calculate the consistency score of unlabeled remote sensing image data;

[0008] The unlabeled remote sensing image data are screened according to their consistency scores;

[0009] Input the filtered unlabeled remote sensing image data into the teacher module of the semantic segmentation model to obtain the teacher's prediction results;

[0010] Input the labeled remote sensing image data and the filtered unlabeled remote sensing image data into the student module of the semantic segmentation model, and supervise the student module through the teacher prediction result to obtain the student prediction result;

[0011] Constructing a loss function of a semantic segmentation model based on the teacher prediction results and the student prediction results;

[0012] The parameters of the semantic segmentation model are updated through the loss function of the semantic segmentation model.

[0013] Furthermore, based on the labeled remote sensing image data, the consistency score of the unlabeled remote sensing image data is calculated, including:

[0014] Training a fully supervised semantic segmentation network model using the labeled remote sensing image data;

[0015] Freezing decoder parameters of the fully supervised semantic segmentation network model;

[0016] Inputting the labeled remote sensing image data into the encoder of the fully supervised semantic segmentation network model to obtain features of the labeled remote sensing image data;

[0017] Inputting the unlabeled remote sensing image data into the encoder of the fully supervised semantic segmentation network model to obtain features of the unlabeled remote sensing image data;

[0018] By performing consistency measurement on the features of the labeled remote sensing image data and the features of the unlabeled remote sensing image data, a consistency score of the unlabeled remote sensing image data is obtained.

[0019] Furthermore, by performing consistency measurement on the features of the labeled remote sensing image data and the features of the unlabeled remote sensing image data, a consistency score of the unlabeled remote sensing image data is obtained, including:

[0020] ,

[0021] in, score [ i ] Indicates the The consistency score of unlabeled remote sensing image data; Represents mapping the eigenvector to the reproducing kernel Hilbert space ; representing features of the labeled remote sensing image data, representing features of the first unlabeled remote sensing image data; representing the number of unlabeled remote sensing image data.

[0022] Further, according to the consistency score of the unlabeled remote sensing image data, the unlabeled remote sensing image data is filtered, comprising:

[0023] The set of unlabeled remote sensing image data selected by the teacher module and the student module in the current training round is:

[0024] ,

[0025] ,

[0026] wherein, the unlabeled remote sensing image data arranged in order from high to low according to the consistency score, i ∈ [ m , m + increment ] ; representing the set of unlabeled remote sensing image data; representing the initial data amount, representing the new data amount of each training round, representing the current training round.

[0027] Further, the teacher module and the student module each comprise an encoder and a decoder;

[0028] The decoder of the student module comprises a main decoder and an auxiliary decoder;

[0029] The prediction result of the main decoder is supervised by the pseudo label generated by the teacher module;

[0030] The main decoder divides the pixel points of the unlabeled remote sensing image data input into the encoder of the student module into known categories and unknown categories;

[0031] The auxiliary decoder is used to determine the position of the unknown categories.

[0032] Further, it also comprises: constraining the features of the unlabeled remote sensing image data output by the encoder of the student module, comprising:

[0033] If the features of the labeled remote sensing image data are input into the auxiliary decoder, the supervision signal of the auxiliary decoder ;

[0034] If the features of the unlabeled remote sensing image data are input into the auxiliary decoder, the supervision signal of the auxiliary decoder ,

[0035] in, Indicates confidence, represents the confidence threshold;

[0036] Based on the location of unknown categories determined by the auxiliary decoder, the features of unlabeled remote sensing image data are divided into known category features and unknown category features ;

[0037] Use K-means clustering algorithm to calculate the regional center of each known category feature and regional centers of unknown class characteristics ;

[0038] Will The clustering result of is regarded as an unknown category and regarded as a positive sample. The clustering results of are regarded as negative samples, then the feature loss function between known categories and unknown categories is for:

[0039] ,

[0040] ,

[0041] ,

[0042] in, represents the feature similarity loss between positive samples, Represents the feature similarity loss between negative samples; express and The cosine similarity of represents a hyperparameter.

[0043] Furthermore, the loss function of the semantic segmentation model is:

[0044] ,

[0045] in, represents the supervised loss function; represents the loss function of the auxiliary decoder; represents the consistency loss function; and Represents the weight of the corresponding loss;

[0046] The auxiliary decoder is supervised by both labeled remote sensing image data and unlabeled remote sensing image data, and its loss function The calculation formula is:

[0047] ,

[0048] ,

[0049] L u =− 1 N ∑ i = 1 N [ y i log( p i ) + ( 1 − y i )log( 1 − p i )] ,

[0050] in, Represents the loss function of labeled remote sensing image data in the auxiliary decoder, represents the loss function for unlabeled remote sensing image data in the auxiliary decoder; represents the predicted value, represents the true value;

[0051] Consistency loss function The calculation formula is:

[0052] ,

[0053] in, Represents the unlabeled remote sensing image data set participating in the training in the current training round, represents the parameters of the student model, Represents the segmentation map output grid The pixel address, Indicates The segmentation predictions from the teacher module, p θ s ( x )( ω ) ∈ [ 0 , 1 ] Y Indicates The segmentation predictions from the student model, Indicates The confidence of the segmentation prediction from the teacher module is given at .

[0054] In a second aspect, the present invention further provides a training device for a semi-supervised remote sensing image semantic segmentation model, the device comprising:

[0055] A consistency score calculation module is used to calculate the consistency score of unlabeled remote sensing image data based on labeled remote sensing image data;

[0056] The unlabeled data screening module is used to screen the unlabeled remote sensing image data according to their consistency scores;

[0057] The teacher prediction result acquisition module is used to input the filtered unlabeled remote sensing image data into the teacher module of the semantic segmentation model to obtain the teacher prediction results;

[0058] A student prediction result acquisition module is used to input the labeled remote sensing image data and the filtered unlabeled remote sensing image data into the student module of the semantic segmentation model, and supervise the student module through the teacher's prediction results to obtain the student prediction results;

[0059] A loss function construction module, configured to construct a loss function of a semantic segmentation model based on the teacher prediction results and the student prediction results;

[0060] The model parameter update module is used to update the parameters of the semantic segmentation model through the loss function of the semantic segmentation model.

[0061] In a third aspect, the present invention further provides a computer device, comprising:

[0062] Memory for storing computer programs;

[0063] A processor is used to execute the computer program to implement the steps of the above-mentioned semi-supervised remote sensing image semantic segmentation model training method.

[0064] In a fourth aspect, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement the steps of the above-mentioned semi-supervised remote sensing image semantic segmentation model training method.

[0065] Compared with the prior art, the present invention has the following beneficial effects:

[0066] The semi-supervised remote sensing image semantic segmentation model training method provided by the present invention calculates the consistency degree of labeled remote sensing image data and unlabeled remote sensing image data to evaluate the distribution similarity between labeled data and unlabeled data, thereby screening and sorting unlabeled remote sensing image data. The present invention proposes a training strategy for the problem of semantic drift. In the early stage of training, priority is given to unlabeled remote sensing image data that is closely related to the distribution of labeled remote sensing image data, that is, unlabeled remote sensing image data with a high consistency score, ensuring that the semantic segmentation model learns in a clear and unambiguous semantic context, which helps to establish an accurate decision boundary. As the training progresses, more complex unlabeled remote sensing image data, that is, unlabeled remote sensing image data with a low consistency score, is gradually introduced. These data are relatively inconsistent with the distribution of labeled remote sensing image data, which can enhance the robustness of the model and effectively reduce the risk of semantic drift when the semantic segmentation model encounters complex unlabeled data. In addition, the present invention develops a consistency-based semi-supervised semantic segmentation network architecture, and the decoder can identify unknown category areas and distinguish their features from known category features, thereby reducing the interference of unknown categories. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 A flowchart of a semi-supervised remote sensing image semantic segmentation model training method provided by an embodiment of the present invention;

[0068] Figure 2 A schematic diagram of a structure for obtaining consistency scores for unlabeled remote sensing image data according to an embodiment of the present invention;

[0069] Figure 3 Schematic diagram of a framework for training a semantic segmentation model in an embodiment of the present invention;

[0070] Figure 4 A schematic diagram of the structure of a semi-supervised remote sensing image semantic segmentation model training device provided by an embodiment of the present invention;

[0071] Figure 5 This is a diagram of the internal structure of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0072] The technical solution of the present invention is described in detail below through the accompanying drawings and specific embodiments. The same reference numerals in the drawings indicate the same or similar components or parts. It should be understood by those skilled in the art that these drawings are not necessarily drawn to scale. The embodiments of the present application and the specific features in the embodiments are detailed descriptions of the technical solution of the present application, rather than limitations of the technical solution of the present application. In the absence of conflict, the embodiments of the present application and the technical features in the embodiments can be combined with each other.

[0073] The term "and / or" in this document simply describes a relationship between related objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, or B exists alone. Additionally, the character " / " in this document generally indicates that the related objects are in an "or" relationship.

[0074] Example 1:

[0075] like Figures 1 to 3 As shown, an embodiment of the present invention provides a semi-supervised remote sensing image semantic segmentation model training method. Figure 1 This is a flow chart of the semi-supervised remote sensing image semantic segmentation model training method. This flow chart only shows the logical sequence of the method described in this embodiment. Under the premise of no conflict, in other possible embodiments of the present invention, different methods can be used. Figure 1 The steps shown or described are accomplished in the order shown.

[0076] The semi-supervised remote sensing image semantic segmentation model training method provided in this embodiment can be applied to a terminal and can be executed by a semi-supervised remote sensing image semantic segmentation model training device, which can be implemented by software and / or hardware and can be integrated into a terminal.

[0077] The embodiment of the present invention is implemented using the Pytorch deep learning framework under the Windows operating system. The specific experimental environment configuration is as follows:

[0078]

[0079] See also Figure 1 The method of the embodiment of the present invention specifically includes the following steps:

[0080] Step 1: Based on the labeled remote sensing image data, calculate the consistency score of the unlabeled remote sensing image data.

[0081] Labeled remote sensing image data set of the present invention , unlabeled remote sensing image data set , where M represents the number of labeled remote sensing image data, N represents the number of unlabeled remote sensing image data, M< <N。

[0082] When it comes to semantic segmentation of remote sensing images in large-scale and complex scenes, the quality of unlabeled data varies, and not all unlabeled data need to be involved in training. Therefore, before model training, all unlabeled remote sensing image data are first evaluated for consistency.

[0083] like Figure 2 As shown, step 1 specifically includes the following steps:

[0084] Step 11: training a fully supervised semantic segmentation network model using the labeled remote sensing image data;

[0085] Step 12: Freeze the decoder parameters of the fully supervised semantic segmentation network model;

[0086] Step 13: Input the labeled remote sensing image data into the encoder of the fully supervised semantic segmentation network model to obtain the features of the labeled remote sensing image data;

[0087] Step 14: Input the unlabeled remote sensing image data into the encoder of the fully supervised semantic segmentation network model to obtain the features of the unlabeled remote sensing image data;

[0088] Step 15: Obtaining a consistency score of the unlabeled remote sensing image data by performing consistency measurement on the features of the labeled remote sensing image data and the features of the unlabeled remote sensing image data, including:

[0089] ,

[0090] in, score [ i ] Indicates the The consistency score of unlabeled remote sensing image data; Represents mapping the eigenvector to the reproducing kernel Hilbert space ; Represents the characteristics of labeled remote sensing image data, Indicates the Features of unlabeled remote sensing image data; Represents the number of unlabeled remote sensing image data.

[0091] Step 2: Filter the unlabeled remote sensing image data according to their consistency scores.

[0092] According to the consistency scores of the unlabeled remote sensing image data obtained in step 1, all unlabeled remote sensing image data are sorted from high to low according to the consistency scores, and the unlabeled remote sensing image data are input into the semantic segmentation model for training in this order.

[0093] The set of unlabeled remote sensing image data selected by the semantic segmentation model in the current training round for:

[0094] ,

[0095] ,

[0096] in, Represents unlabeled remote sensing image data arranged in descending order according to consistency scores. i ∈ [ m , m + increment ] ; Represents a collection of unlabeled remote sensing image data; Indicates the initial data volume, represents the amount of new data in each training round, Indicates the current training round.

[0097] The present invention proposes a training strategy for the problem of semantic drift. In the early stage of semantic segmentation model training, priority is given to unlabeled remote sensing image data that is closely related to the distribution of labeled remote sensing image data, that is, unlabeled remote sensing image data with a high consistency score, to ensure that the semantic segmentation model learns in a clear and unambiguous semantic context at this time, which helps to establish an accurate decision boundary. As the training progresses, more complex unlabeled remote sensing image data, that is, unlabeled remote sensing image data with a low consistency score, are gradually introduced. These data are relatively inconsistent with the distribution of labeled remote sensing image data, which can enhance the robustness of the semantic segmentation model. Through this progressive training strategy, starting from simple training data and gradually transitioning to more challenging training data, the risk of semantic drift occurring when the model encounters complex unlabeled data is effectively reduced. It can improve the adaptability to potential data changes while maintaining the stability of the semantic segmentation model, and ultimately improve the generalization performance of the semantic segmentation model.

[0098] Step 3: Input the filtered unlabeled remote sensing image data into the teacher module of the semantic segmentation model to obtain the teacher prediction results.

[0099] Step 4: Input the labeled remote sensing image data and the filtered unlabeled remote sensing image data into the student module of the semantic segmentation model, and supervise the student module through the teacher's prediction results to obtain the student prediction results.

[0100] like Figure 3 As shown, the semantic segmentation model of the present invention includes a teacher module and a student module. The student module provides parameter updates to the teacher module, and the teacher module provides pseudo labels to the student module. The two interact with each other to achieve better training purposes.

[0101] Both the teacher module and the student module include an encoder and a decoder.

[0102] In an embodiment of the present invention, the labeled remote sensing image data and the filtered unlabeled remote sensing image data need to be preprocessed before being input into the model, including: cropping the labeled remote sensing image data and the filtered unlabeled remote sensing image data according to a size of 512×512; performing data enhancement on the cropped unlabeled remote sensing image data, specifically including weak enhancement (rotation, translation, etc.) and strong enhancement (color transformation, etc.), and inputting the weakly enhanced data into the encoder of the student module, and inputting the strongly enhanced data into the encoder of the teacher module.

[0103] In addition, before inputting the training data, the encoder of the teacher module and the encoder of the student module need to be randomly initialized to ensure that they are different.

[0104] The decoder of the student module is divided into a main decoder and an auxiliary decoder. The prediction results of the main decoder are supervised by the pseudo labels generated by the teacher module, and the output of the main decoder is , where B represents the batch size, K represents the number of known categories plus 1, H represents the feature height, and W represents the feature width. The main decoder of the present invention divides the pixels of the unlabeled remote sensing image data input to the encoder of the student module into known categories and unknown categories, and designs a channel to store the unknown categories. To alleviate the interference of unknown categories, an auxiliary decoder is also designed to determine the location of the unknown categories.

[0105] Specifically, the filtered unlabeled remote sensing image data is input into the teacher module of the semantic segmentation model to obtain the teacher's prediction results. :

[0106] ,

[0107] in, represents the encoder of the teacher module, Represents the decoder of the teacher module.

[0108] For the student module, it is trained by the labeled remote sensing image data and the screened unlabeled remote sensing image data. First, the unknown category region needs to be distinguished:

[0109] ,

[0110] Among them, represents the mask of the unknown category, represents the encoder of the student module, represents the auxiliary decoder of the student module.

[0111] The position of the unknown category is obtained by using , and the features of the unlabeled remote sensing image data are divided into known category features and unknown category features based on the position:

[0112] [ F 1 , F 2 ,..., F k ] = F S u ⋅ mask ,

[0113] ,

[0114] Among them, represents the feature of the unlabeled remote sensing image data output by the encoder of the student module, which is input into the main encoder and combined with to calculate the final prediction :

[0115] .

[0116] The application also constrains the feature of the unlabeled remote sensing image data output by the encoder of the student module, including:

[0117] If the feature of the labeled remote sensing image data is input into the auxiliary decoder, the supervision signal of the auxiliary decoder is ;

[0118] If the feature of the unlabeled remote sensing image data is input into the auxiliary decoder, the supervision signal of the auxiliary decoder is ,

[0119] Among them, represents the confidence, represents the confidence threshold;

[0120] Based on the position of the unknown category determined by the above auxiliary decoder, the features of the unlabeled remote sensing image data are divided into known category features and unknown category features ;

[0121] The region center of each known category feature is calculated by using the K-means clustering algorithm and regional centers of unknown class characteristics ;

[0122] Will The clustering result of is regarded as an unknown category and regarded as a positive sample. The clustering results of are regarded as negative samples, then the feature loss function between known categories and unknown categories is for:

[0123] ,

[0124] ,

[0125] ,

[0126] in, represents the feature similarity loss between positive samples, Represents the feature similarity loss between negative samples; express and The cosine similarity of represents a hyperparameter.

[0127] Step 5: Based on the teacher prediction results and the student prediction results, construct a loss function for the semantic segmentation model.

[0128] The loss function of the semantic segmentation model of the present invention is:

[0129] ,

[0130] in, represents the supervised loss function; represents the loss function of the auxiliary decoder; represents the consistency loss function; and Represents the weight of the corresponding loss.

[0131] Supervised loss function The student module is mainly supervised by labeled data, which is calculated by multi-class cross entropy loss, which is defined as follows:

[0132] ,

[0133] Where M represents the number of samples, Indicates the number of categories; samples If the true category is equal to ,but Take 1, otherwise take 0; Represents an observation sample Belong to category The loss guides the feature extraction of the student model by comparing the difference between the predicted probability distribution of the network and the true label distribution.

[0134] The auxiliary decoder is supervised by both labeled remote sensing image data and unlabeled remote sensing image data, and its loss function The calculation formula is:

[0135] ,

[0136] ,

[0137] L u =− 1 N ∑ i = 1 N [ y i log( p i ) + ( 1 − y i )log( 1 − p i )] ,

[0138] in, Represents the loss function of labeled remote sensing image data in the auxiliary decoder, represents the loss function for unlabeled remote sensing image data in the auxiliary decoder; represents the predicted value, represents the true value;

[0139] The teacher's prediction results supervise the student module and are reflected in the consistency loss function , and its calculation formula is:

[0140] ,

[0141] in, Represents the unlabeled remote sensing image data set participating in the training in the current training round, represents the parameters of the student model, Represents the segmentation map output grid The pixel address, Indicates The segmentation predictions from the teacher module, p θ s ( x )( ω ) ∈ [ 0 , 1 ] Y Indicates The segmentation predictions from the student model, Indicates The confidence of the segmentation prediction from the teacher module is given at .

[0142] Step 6: Update the parameters of the semantic segmentation model through the loss function of the semantic segmentation model.

[0143] In this embodiment of the present invention, the batch size is set to 16, consisting of 8 labeled remote sensing image data and 8 unlabeled remote sensing image data. The teacher and student modules are pre-trained on the labeled data for 5 cycles before training begins. The cluster learning method uses a stochastic gradient descent optimizer with the following settings: a base learning rate of 0.01, a momentum of 0.9, and a weight decay of 0.0001. The training process lasts for 40 cycles, with an initial learning rate of 0.01. After each iteration, the learning rate is updated using the following formula: ; The momentum coefficient is set to 0.9 and the weight decay coefficient is set to 0.0001.

[0144] Example 2:

[0145] Based on the same inventive concept as Example 1, this embodiment of the present invention also provides a semi-supervised remote sensing image semantic segmentation model training device for implementing the above-mentioned semi-supervised remote sensing image semantic segmentation model training method. The implementation solution provided by this device is similar to the implementation solution described in the above-mentioned method. Therefore, the specific limitations in the embodiment of the semi-supervised remote sensing image semantic segmentation model training device provided below can be found in the above-mentioned limitations on the method for enhancing ionospheric data over the ocean, and will not be repeated here.

[0146] like Figure 4 As shown, an embodiment of the present invention provides a semi-supervised remote sensing image semantic segmentation model training device, comprising:

[0147] A consistency score calculation module is used to calculate the consistency score of unlabeled remote sensing image data based on labeled remote sensing image data;

[0148] The unlabeled data screening module is used to screen the unlabeled remote sensing image data according to their consistency scores;

[0149] The teacher prediction result acquisition module is used to input the filtered unlabeled remote sensing image data into the teacher module of the semantic segmentation model to obtain the teacher prediction results;

[0150] A student prediction result acquisition module is used to input the labeled remote sensing image data and the filtered unlabeled remote sensing image data into the student module of the semantic segmentation model, and supervise the student module through the teacher's prediction results to obtain the student prediction results;

[0151] A loss function construction module, configured to construct a loss function of a semantic segmentation model based on the teacher prediction results and the student prediction results;

[0152] The model parameter update module is used to update the parameters of the semantic segmentation model through the loss function of the semantic segmentation model.

[0153] Example 3:

[0154] The embodiment of the present invention further provides a computer device, which may be a server, and its internal structure diagram may be as shown in FIG. Figure 5 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the steps of the semi-supervised remote sensing image semantic segmentation model training method in the aforementioned embodiment are implemented.

[0155] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0156] Example 4:

[0157] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the following method:

[0158] Based on labeled remote sensing image data, calculate the consistency score of unlabeled remote sensing image data;

[0159] The unlabeled remote sensing image data are screened according to their consistency scores;

[0160] Input the filtered unlabeled remote sensing image data into the teacher module of the semantic segmentation model to obtain the teacher's prediction results;

[0161] Input the labeled remote sensing image data and the filtered unlabeled remote sensing image data into the student module of the semantic segmentation model, and supervise the student module through the teacher prediction result to obtain the student prediction result;

[0162] Constructing a loss function of a semantic segmentation model based on the teacher prediction results and the student prediction results;

[0163] The parameters of the semantic segmentation model are updated through the loss function of the semantic segmentation model.

[0164] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0165] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0166] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0167] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0168] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the purpose of the present invention and the claims, which are all protected by the present invention.

Claims

1. A semi-supervised remote sensing image semantic segmentation model training method, characterized in that: include: Based on labeled remote sensing image data, the consistency score of unlabeled remote sensing image data is calculated, including: Training a fully supervised semantic segmentation network model using the labeled remote sensing image data; Freezing decoder parameters of the fully supervised semantic segmentation network model; Inputting the labeled remote sensing image data into the encoder of the fully supervised semantic segmentation network model to obtain features of the labeled remote sensing image data; Inputting the unlabeled remote sensing image data into the encoder of the fully supervised semantic segmentation network model to obtain features of the unlabeled remote sensing image data; Obtaining a consistency score of the unlabeled remote sensing image data by performing consistency measurement on the features of the labeled remote sensing image data and the features of the unlabeled remote sensing image data includes: ， in, Indicates the The consistency score of unlabeled remote sensing image data; Represents mapping the eigenvector to the reproducing kernel Hilbert space ; Represents the characteristics of labeled remote sensing image data, Indicates the Features of unlabeled remote sensing image data; Represents the number of unlabeled remote sensing image data; According to the consistency score of the unlabeled remote sensing image data, the unlabeled remote sensing image data is screened, including: The unlabeled remote sensing image data set selected by the teacher module and the student module in the current training round for: ， ， in, Represents unlabeled remote sensing image data arranged in descending order according to consistency scores. ; Represents a collection of unlabeled remote sensing image data; Indicates the initial data volume, represents the amount of new data in each training round, Indicates the current training round; Input the filtered unlabeled remote sensing image data into the teacher module of the semantic segmentation model to obtain the teacher's prediction results; Input the labeled remote sensing image data and the filtered unlabeled remote sensing image data into the student module of the semantic segmentation model, and supervise the student module through the teacher prediction result to obtain the student prediction result; Constructing a loss function of a semantic segmentation model based on the teacher prediction results and the student prediction results; The parameters of the semantic segmentation model are updated through the loss function of the semantic segmentation model.

2. The semi-supervised remote sensing image semantic segmentation model training method according to claim 1, characterized in that: The teacher module and the student module both include an encoder and a decoder; The decoder of the student module includes a main decoder and an auxiliary decoder; The prediction results of the main decoder are supervised by the pseudo labels generated by the teacher module; The main decoder divides the pixels of the unlabeled remote sensing image data input to the encoder of the student module into known categories and unknown categories; The auxiliary decoder is used to determine the location of the unknown class.

3. The semi-supervised remote sensing image semantic segmentation model training method according to claim 2, characterized in that: Also includes: Constrain the features of the unlabeled remote sensing image data output by the encoder of the student module, including: If the input to the auxiliary decoder is the features of labeled remote sensing image data, the supervision signal of the auxiliary decoder ; If the input to the auxiliary decoder is the features of unlabeled remote sensing image data, the supervision signal of the auxiliary decoder , in, Indicates confidence, represents the confidence threshold; Based on the location of unknown categories determined by the auxiliary decoder, the features of unlabeled remote sensing image data are divided into known category features and unknown category features ; Use K-means clustering algorithm to calculate the regional center of each known category feature and regional centers of unknown class characteristics ; Will The clustering result of is regarded as an unknown category and regarded as a positive sample. The clustering results of are regarded as negative samples, then the feature loss function between known categories and unknown categories is for: ， ， ， in, represents the feature similarity loss between positive samples, Represents the feature similarity loss between negative samples; express and The cosine similarity of represents a hyperparameter.

4. The semi-supervised remote sensing image semantic segmentation model training method according to claim 3, characterized in that: The loss function of the semantic segmentation model is: ， in, represents the supervised loss function; represents the loss function of the auxiliary decoder; represents the consistency loss function; and Represents the weight of the corresponding loss; The auxiliary decoder is supervised by both labeled remote sensing image data and unlabeled remote sensing image data, and its loss function The calculation formula is: ， ， ， in, Represents the loss function of labeled remote sensing image data in the auxiliary decoder, represents the loss function for unlabeled remote sensing image data in the auxiliary decoder; represents the predicted value, represents the true value; Consistency loss function The calculation formula is: ， in, Represents the unlabeled remote sensing image data set participating in the training in the current training round, represents the parameters of the student model, Represents the segmentation map output grid The pixel address, Indicates The segmentation predictions from the teacher module, Indicates The segmentation predictions from the student model, Indicates The confidence of the segmentation prediction from the teacher module is given at .

5. A semi-supervised remote sensing image semantic segmentation model training device, characterized in that: include: The consistency score calculation module is used to calculate the consistency score of unlabeled remote sensing image data based on labeled remote sensing image data, including: Training a fully supervised semantic segmentation network model using the labeled remote sensing image data; Freezing decoder parameters of the fully supervised semantic segmentation network model; Inputting the labeled remote sensing image data into the encoder of the fully supervised semantic segmentation network model to obtain features of the labeled remote sensing image data; Inputting the unlabeled remote sensing image data into the encoder of the fully supervised semantic segmentation network model to obtain features of the unlabeled remote sensing image data; Obtaining a consistency score of the unlabeled remote sensing image data by performing consistency measurement on the features of the labeled remote sensing image data and the features of the unlabeled remote sensing image data includes: ， in, Indicates the The consistency score of unlabeled remote sensing image data; Represents mapping the eigenvector to the reproducing kernel Hilbert space ; Represents the characteristics of labeled remote sensing image data, Indicates the Features of unlabeled remote sensing image data; Represents the number of unlabeled remote sensing image data; The unlabeled data screening module is used to screen unlabeled remote sensing image data based on their consistency scores, including: The unlabeled remote sensing image data set selected by the teacher module and the student module in the current training round for: ， ， in, Represents unlabeled remote sensing image data arranged in descending order according to consistency scores. ; Represents a collection of unlabeled remote sensing image data; Indicates the initial data volume, represents the amount of new data in each training round, Indicates the current training round; The teacher prediction result acquisition module is used to input the filtered unlabeled remote sensing image data into the teacher module of the semantic segmentation model to obtain the teacher prediction results; A student prediction result acquisition module is used to input the labeled remote sensing image data and the filtered unlabeled remote sensing image data into the student module of the semantic segmentation model, and supervise the student module through the teacher's prediction results to obtain the student prediction results; A loss function construction module, configured to construct a loss function of a semantic segmentation model based on the teacher prediction results and the student prediction results; The model parameter update module is used to update the parameters of the semantic segmentation model through the loss function of the semantic segmentation model.

6. A computer device, characterized in that: include: Memory for storing computer programs; A processor, configured to execute the computer program to implement the steps of the semi-supervised remote sensing image semantic segmentation model training method according to any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the semi-supervised remote sensing image semantic segmentation model training method described in any one of claims 1 to 4 are implemented.

Images