Unsupervised semantic segmentation method in open scene and storage medium

Abstract

The invention discloses an unsupervised semantic segmentation method in an open scene and a storage medium. The method comprises the following steps: acquiring a first image data set only containing a known pixel category and a second image data set containing an unknown pixel category; and constructing a first network for feature learning and reconstruction and a second network for classification learning. In the training stage, the first image data set is input into the first network for training, reconstruction loss is determined, prototype representation of each pixel category is learned, and a correction coefficient is determined based on the relation between pixel features and the prototype representation so as to correct the reconstruction loss. And a test stage: inputting the second image data set into the trained first network, determining and correcting reconstruction loss, representing classification by using a prototype, selecting a negative sample from pixels with differences in classification results, and selecting a positive sample to carry out comparative learning so as to train a second network. And finally, carrying out pixel classification on the to-be-segmented image by utilizing the trained second network. According to the method, the semantic segmentation task in the open scene can be effectively processed.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image processing, and relates to an unsupervised semantic segmentation method in an open scene and a storage medium. BACKGROUND

[0002] Reasonable planning of land resource utilization is the key to realizing global sustainable development, which requires timely and accurate land cover classification and monitoring. However, existing methods are mostly based on static closed set assumption for classification, which is difficult to meet in practical applications, because it is both time-consuming and laborious to make the training set cover all possible scene categories. Promoting the development of land cover technology in an open scene can not only improve classification accuracy, but also enhance the adaptability to unknown or emerging land cover types, which is crucial for coping with challenges such as rapid urbanization, agricultural expansion and natural disasters.

[0003] Current researches mostly focus on image-level classification problems in an open scene, while relatively few researches focus on pixel-level classification problems. However, land cover classification usually needs to achieve pixel-level accuracy. Due to its dense labeling characteristics, the pixel-level classification task in an open scene is more difficult than the image-level classification. For example, the input data dimension handled by pixel-level recognition is usually very high, because each pixel is regarded as an independent sample and contains multiple features, which leads to the need for more computing resources and storage space in the training and inference process of the model. In addition, in pixel-level classification, the number of pixels of different categories may differ significantly, leading the model to focus on the categories with more number, which may cause the model performance to decrease in training. Therefore, in this context, developing a pixel-level classification method suitable for an open scene is a difficult problem to be solved in the current field of remote sensing image processing. SUMMARY

[0004] The technical purpose of the present application is to provide an unsupervised semantic segmentation method in an open scene and a storage medium to solve the technical problem of the lack of pixel-level classification methods for an open scene.

[0005] To achieve the above technical purpose, the present application adopts the following technical solutions.

[0006] In a first aspect, the embodiments of the present application provide an unsupervised semantic segmentation method in an open scene, comprising:

[0007] obtaining a first image data set containing only known pixel categories and a second image data set containing unknown pixel categories;

[0008] constructing a first network for feature learning and reconstruction and a second network for classification learning;

[0009] In a training phase, the first image dataset is input into a first network for training, a reconstruction loss corresponding to the first image dataset is determined, and a prototype representation of each pixel category in the first image dataset is learned; for each pixel in the first image dataset, a correction coefficient of the pixel is determined based on a relationship between pixel features and the prototype representation, and the reconstruction loss of the pixel is corrected using the correction coefficient;

[0010] During a testing phase, the second image dataset is input into the trained first network, a reconstruction loss corresponding to the second image dataset is determined, a correction coefficient for each pixel in the second image dataset is determined based on a relationship between pixel features and the prototype representation, and the reconstruction loss of the pixel is corrected using the correction coefficient; pixels in the second image dataset and its reconstructed image are classified using the prototype representation, negative samples are selected from pixels with different classification results, and positive samples are selected from the first image dataset and / or the second image dataset; and the positive and negative samples are used for comparative learning to train the second network;

[0011] An image to be segmented is obtained, and pixel classification of the image to be segmented is performed using the trained second network.

[0012] Furthermore, the method includes: using L2-norm as reconstruction loss, expressed as:

[0013]

[0014] Among them, L recon is the reconstruction error, N is the number of samples in a batch, X n,i and are the input and reconstructed value of the i-th element of the n-th sample, respectively.

[0015] Furthermore, determining a correction coefficient of the pixel based on the relationship between the pixel feature and the prototype representation, and correcting the reconstruction loss of the pixel using the correction coefficient, includes:

[0016] Determine the cosine similarity between the pixel and the prototype representation of each category, and determine the prototype representation with the highest similarity to the pixel The cosine similarity between Subtract the cosine similarity from 1 The difference is used as the correction coefficient to correct the reconstruction loss. The expression is as follows:

[0017]

[0018] in is the reconstruction error after correction, L recon is the reconstruction error to be corrected.

[0019] Further, the method further comprises: in the training stage, fitting the tail part of the reconstruction loss of each pixel in the first image data set after correction by using the POT method in the extreme value theory, and determining the generalized Pareto distribution parameters by maximum likelihood estimation;

[0020] In the testing stage, for the tail part of the reconstruction loss of each pixel in the second image data set after correction, the tail part probability of the reconstruction loss after correction is calculated by using the generalized Pareto distribution parameters obtained in the training stage.

[0021] If the tail part probability exceeds a preset threshold, it is determined that the pixel belongs to an unknown pixel class; if the tail part probability does not exceed the threshold, it is determined that the pixel belongs to a known pixel class.

[0022] The pixels in the second image data set that are determined to be unknown pixel classes are excluded, and the pixels with different classification results are taken as negative samples from the remaining pixels that are determined to be known pixel classes.

[0023] Further, the positive samples are selected from the first image data set and / or the second image data set, comprising:

[0024] The prototype representation of each pixel class in the first image data set learned by the first network is taken as a positive sample.

[0025] Further, when the positive samples are selected from the second image data set, it is required that the cosine similarity between the pixel features of the positive samples and the prototype representation of the corresponding class is greater than 0.8.

[0026] Further, InfoNCE loss function is used for contrastive learning to pull the positive samples closer and push the negative samples farther.

[0027] Further, the first image data set is derived from a public land cover image database, and the second image data set is derived from actually collected remote sensing images.

[0028] Further, the network structures of the first network and the second network are the same, and both are encoder-decoder structures, and the first network and the second network are both image reconstruction networks constructed based on ResNet101 and DeepLabv3+.

[0029] In a second aspect, a non-transitory computer readable storage medium has a computer program stored thereon, wherein the computer program, when executed by a processor, implements the unsupervised semantic segmentation method in an open scene provided in any possible implementation manner of the first aspect.

[0030] Compared with the prior art, the beneficial technical effects obtained by the present application are: through the image reconstruction network, the reconstruction error after the pixels containing unknown classes and only containing known classes are reconstructed is used for distinguishing. The reconstruction error is corrected by the known class prototype, so that the reconstruction error can be more accurately used as the basis for distinguishing between known classes and unknown classes. The inaccurate pixels are identified based on the classification results of the class prototype before and after reconstruction, and these pixels are effectively used to improve the reconstruction accuracy of the model for known classes through contrast learning. The method provided by the present application can process remote sensing images containing unknown classes without a large amount of labeled data, reduces the data labeling cost, enhances the adaptability of the model to emerging or unknown land cover types, and is suitable for dynamic actual scenes. DETAILED DESCRIPTION

[0031] In order for those skilled in the art to better understand the technical solutions in the present application, the technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should be within the scope of protection of the present application.

[0032] In the description of the present application, the terms "first", "second" are only used for description purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more features.

[0033] The existing remote sensing image land cover classification method can only classify images containing only known classes, and when unknown classes appear in the image, the classification accuracy of known classes decreases. In addition, in pixel-level classification, the number of pixels of different classes may differ significantly, and the performance of the existing method in pixel-level classification in an open scene needs to be improved.

[0034] The embodiments of the present application provide an unsupervised semantic segmentation method in an open scene, including the following steps. It should be noted that the serial numbers of the steps in the present embodiment do not represent the limitation of the execution order of the steps.

[0035] Step S1: obtaining a first image data set P containing only known pixel classes l and a second image data set P containing unknown pixel classes u .

[0036] In some embodiments, the first image data set is derived from a public land cover image database, and the second image data set is derived from actually collected remote sensing images.

[0037] In some other embodiments, assuming that the dataset has C classes in total, u classes in the dataset are selected as unknown classes, and the pixel points corresponding to these classes are designated as "ignored" and not included in the loss function during training of the first network (which can be a teacher network), simulating an open set scenario.

[0038] The present application can obtain a first image dataset containing only known pixel classes and a second image dataset containing unknown pixel classes, which adapts to the diversity characteristics of data in an open scene. By respectively processing the two datasets, a foundation is laid for subsequent segmentation and classification tasks, and the applicability of the method in complex actual scenarios is improved.

[0039] Step S2: constructing a first network for feature learning and reconstruction and a second network for classification learning.

[0040] The second network can be regarded as a student network. In some embodiments, the teacher network and the student network have the same network structure, both of which are encoder-decoder structures. The network is an image reconstruction network based on ResNet101 and DeepLabv3+, using ResNet101 as the backbone and DeepLabv3+ as the decoder. Some embodiments are based on ResNet101 and DeepLabv3+, but the land cover classification scheme designed by the present application is not limited to these two networks, and other networks can also be used in specific implementation.

[0041] The present application constructs a first network for feature learning and reconstruction and a second network for classification learning. The first network focuses on feature extraction and image reconstruction and can learn the prototype representation of known pixel classes. The second network is responsible for classification learning, and the two networks work together to improve the overall performance and accuracy of semantic segmentation.

[0042] Step S3: in the training phase, input the first image data (containing only known pixel classes) set to the first network for training, determine the reconstruction loss corresponding to the first image data set, learn the prototype representation of each pixel class in the first image data set, and determine the correction coefficient of each pixel in the first image data set based on the relationship between the pixel feature and the prototype representation, and correct the reconstruction loss of the pixel using the correction coefficient.

[0043] In the embodiments, the entire training process adopts an unsupervised learning paradigm, and the reconstruction loss is calculated by comparing the differences between the original image and the reconstructed image. Subsequently, gradient backpropagation is performed according to the reconstruction loss value, and the network weight parameters are updated. This process is iterated until the model converges, and the trained model is finally obtained.

[0044] In an embodiment, the L2-norm may be used as the reconstruction loss:

[0045]

[0046] Among them, L recon is the reconstruction error, N is the number of samples in a batch, X n,i and are the input and reconstructed value of the i-th element of the n-th sample, respectively.

[0047] In remote sensing image classification, the prototype or prototype representation refers to the representation space P of category c. c The representation space Pc contains the feature representations of all labeled pixels belonging to class c. The class prototype is obtained through an encoder. Specifically, the encoder maps the features of the input pixels into a low-dimensional feature space, and the class prototype is the mean or center position of the features belonging to the same class in this space. This representation effectively captures the essential characteristics of the class and provides a compact and representative feature description for subsequent classification tasks.

[0048] The class prototype of a known class is expressed by the following formula:

[0049]

[0050] Among them, |P c |It is P c The number of elements in z c is the feature representation of pixels belonging to category c.

[0051] For each pixel in the first image data set, a correction coefficient of the pixel is determined based on the relationship between the pixel feature and the prototype representation, and the reconstruction loss of the pixel is corrected using the correction coefficient, which can be expressed as:

[0052] L recon * =L recon *K;

[0053] in is the reconstruction error after correction, L recon is the reconstruction error to be corrected, and K is the correction coefficient.

[0054] In this process, the method for correcting using prototype representation in a specific embodiment can use a variety of different similarity measurement methods, and the specific selection depends on the characteristics of the data and the requirements of the classification task.

[0055] As an example, the following similarity measures can be used:

[0056] Example 1. Based on cosine similarity

[0057] determine the cosine similarity between the pixel and the prototype representation of each class determine the cosine similarity between the pixel and the prototype representation of each class subtract the cosine similarity from 1 as a correction coefficient, correct the reconstruction loss, the expression is as follows:

[0058]

[0059] wherein is the corrected reconstruction error, L recon is the reconstruction error to be corrected.

[0060] Example 2. Based on the Euclidean distance

[0061] The Euclidean distance measures the straight-line distance between two vectors in the feature space.

[0062]

[0063] wherein x is the feature vector, x i is the i-th element feature, c □ is the prototype representation. The Euclidean distance can be converted into a similarity measure.

[0064] Example 3. Manhattan distance

[0065] The Manhattan distance measures the sum of the absolute differences between two vectors in the feature space.

[0066]

[0067] The Manhattan distance can also be converted into a similarity measure. In addition, there are similarity measure methods such as Mahalanobis distance.

[0068] After determining the similarity measure value in the above examples 2, 3, etc., the following formula can be used to determine the correction coefficient, for example:

[0069] In addition to determining the correction coefficient based on the similarity between the pixel features and the prototype representation, some can also consider the following relationships to determine the correction coefficient:

[0070] 4. Feature difference

[0071] First, calculate the difference vector between the pixel feature φ(x) and the class prototype μ c

[0072] Δ(x,c)=φ(x)-μ c (7)

[0073] ​Next, the norm (e.g., Euclidean distance) of the feature difference is calculated.

[0074]

[0075] The norm of the feature difference is used as a correction factor to adjust the reconstruction error.

[0076]

[0077] where γ is an adjustment parameter that controls the influence of the feature difference on the reconstruction error.

[0078] Step S4: In the test phase, the second image dataset is input into the trained first network to determine the reconstruction loss corresponding to the second image dataset. For each pixel in the second image dataset, a correction coefficient of the pixel is determined based on the relationship between the pixel feature and the prototype representation. The reconstruction loss of the pixel is corrected using the correction coefficient, and whether the pixel is an unknown class pixel is determined according to the corrected reconstruction loss and a set threshold. The pixels in the second image dataset and its reconstructed image are classified using the prototype representation, and negative samples are selected from the pixels with different classification results. Positive samples are selected from the first image dataset and / or the second image dataset. The positive samples and the negative samples are used for contrastive learning to train the second network.

[0079] In the test phase, the teacher network is used to reconstruct the unlabeled image in step S4 of the embodiment, the reconstruction loss of each pixel is calculated, and the correction coefficient is determined according to the relationship between the pixel feature and the class prototype learned in the training phase. The reconstruction loss is corrected using the correction coefficient, and this process is synchronized with step S3.

[0080] Since part of the known class pixels have a large reconstruction loss after reconstruction, and part of the unknown class pixels have a small reconstruction loss after reconstruction, the correction based on the class prototype can make the reconstruction loss better reflect the matching degree of the pixel and the known class, thereby more accurately distinguishing the unknown class pixels and the known class pixels.

[0081] In the training and test phases, the application determines a correction coefficient for each pixel based on the relationship between its feature and the prototype representation, and corrects the reconstruction loss of the pixel using the coefficient. This way can more accurately measure the reconstruction error of each pixel, making the model handle different pixels more precisely, thereby improving the quality of image reconstruction and feature learning, and further improving the accuracy of semantic segmentation.

[0082] This step uses the class prototype to classify the images in the second image dataset Pu containing unknown classes and their reconstructed images, and compares the classification results to identify pixels with inaccurate reconstruction, which are used as negative samples of low probability classes for contrastive learning to train the student model and further improve the reconstruction accuracy of the student model for known classes.

[0083] In the implementation of the embodiment, when processing the image containing unknown class information, the feature representation of each pixel in the image is compared with the prototype feature of all known classes (class prototype) one by one, and then the class to which the pixel is most likely to belong is determined. As an example, this process can be realized by calculating the cosine similarity between the pixel feature vector and the class prototype vector; the cosine similarity can effectively measure the closeness of two vectors in direction, thereby providing a quantitative basis for the class attribution of the pixel. That is, for each pixel p, the cosine similarity between its feature representation z p and all class prototypes z c can be calculated, and the class c * with the highest similarity is selected as the classification result of the pixel. Finally, the classification result Y of the image is obtained, where each element y ij represents the class label of the pixel at position (i, j).

[0084] The reconstructed image of the unlabeled image is processed in the same way to obtain another classification result Y Since the teacher model is trained only with known class data, and the images in the second image data set P u contain unknown classes, which will affect the reconstruction of known class pixels, therefore, by comparing the two classification results Y, the possible unknown class pixels are excluded, and the pixels with inconsistent results among the remaining pixels are screened out as the pixels with poor reconstruction effect.

[0085] After identifying the inaccurate reconstructed pixels, they are used as negative samples of low probability classes for contrast learning, effectively utilizing the inaccurate reconstructed pixels to train the student model. For each class, anchor pixels and corresponding positive samples (prototypes of the class) can be selected, and then negative samples are sampled from the memory bank. The positive samples and negative samples are used for contrast learning to train the second network.

[0086] In some embodiments, when selecting positive samples from the first image data set or the second image data set, the cosine similarity between the pixel feature of the positive sample and the prototype representation of the corresponding class is required to be greater than 0.8.

[0087] In some embodiments, in the training stage of step S3, the POT method in the Extreme Value Theory (EVT) is introduced to fit the tail data of the corrected reconstruction loss of each pixel in the first image data set, and the generalized Pareto distribution parameters are determined by maximum likelihood estimation.

[0088] The extreme value theory is a statistical method specially used for analyzing the distribution of extreme events, which can effectively capture the abnormal characteristics of the tail of the data. Through this theory, the tail of the reconstruction error can be modeled. Specifically, the Peaks-Over-Threshold (POT) method is used to model the tail of the reconstruction error. The POT method assumes that the data in the tail follows a Generalized Pareto Distribution (GPD), and the parameters of the GPD are determined by maximum likelihood estimation. In this way, the distribution characteristics of the tail of the reconstruction error can be effectively estimated, which prepares for the subsequent unknown class pixel identification.

[0089] When the maximum likelihood estimation exceeds a certain threshold ω that is large enough, the cumulative distribution function can be approximated by a Generalized Pareto Distribution (GPD):

[0090]

[0091] wherein,

[0092]

[0093] wherein, ν is the reconstruction error, F ν (ω+ν) and F V (ν) are the cumulative distribution function values of the distribution V at ω+ν and ν, respectively; ξ is the shape parameter, and μ is the scale parameter.

[0094] The embodiment introduces the extreme value theory (POT) to fit the tail data of the reconstruction loss, and accurately distinguishes the unknown class.

[0095] In the test stage of step S4, for the corrected reconstruction loss of each pixel in the second image data set, the tail probability of the corrected reconstruction loss is calculated using the Generalized Pareto Distribution parameters obtained in the training stage; if the tail probability exceeds a preset threshold, it is determined that the pixel belongs to the unknown pixel class; if the tail probability does not exceed the threshold, it is determined that the pixel belongs to the known pixel class; the pixels in the second image data set that are determined to belong to the unknown pixel class are excluded, and among the remaining pixels that are determined to belong to the known pixel class, the pixels with different classification results are taken as negative samples of their low probability classes. In this way, pixels that may belong to the unknown class can be avoided as negative samples, so that the negative samples more purely reflect the situation that the classification of the known pixel classes is different, thereby improving the quality of the negative samples, helping the second network to more accurately learn the classification boundary and feature difference of the known pixel classes, improving the effect of contrast learning, and thereby improving the performance and accuracy of the entire semantic segmentation model.

[0096] In an embodiment, whether a pixel belongs to an unknown pixel class is determined based on the comparison between the tail probability and a preset threshold. This method can utilize the tail features of the data to more accurately identify unknown pixel classes that have large differences from the features of known pixel classes, thereby improving the detection capability of unknown pixels.

[0097] In some embodiments, the InfoNCE loss function is introduced in contrastive learning to encourage the model to bring positive samples closer while pushing negative samples further away, thereby improving the model's generalization ability. The InfoNCE loss function is as follows:

[0098]

[0099] Where C is the total number of categories in the dataset, u is the number of unknown categories, and M is the total number of anchor pixels. ci is the feature representation of the i-th anchor pixel of category c. +ci is a positive sample of category c, z -cij is the jth negative sample of category c. 〈·,·〉 represents the cosine similarity between the two vectors. T is the temperature parameter used to adjust the scaling of the similarity. N is the number of negative samples for each anchor point.

[0100] The weights of the teacher network are adjusted via momentum updates, i.e. the weights of the teacher network are the exponential moving average (EMA) of the weights of the student network.

[0101] Step S5: Obtain the image to be segmented, and use the trained second network to perform pixel classification on the image to be segmented.

[0102] The unsupervised semantic segmentation method in an open scene provided by the embodiment of the present application is an unsupervised semantic segmentation method that does not require a large amount of labeled sample data. In practical applications, labeling data often requires a lot of manpower, material resources and time, while this unsupervised method can achieve semantic segmentation with less manual intervention, reducing the cost of data labeling, and can also handle new and unknown pixel categories that continue to appear in open scenes. The trained second network performs pixel classification on the image to be segmented. Since the second network has undergone a series of effective training processes, including contrast learning, etc., it can accurately classify pixels, thereby achieving more accurate semantic segmentation and meeting the needs of image semantic understanding in practical applications.

[0103] Embodiments of the present invention also provide a non-transitory computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the unsupervised semantic segmentation method described above for the open scenario. The non-transitory computer-readable storage medium may include ROM, RAM, a magnetic disk, or an optical disk. The steps of the wireless roaming method are described above and are not repeated here.

[0104] The unsupervised semantic segmentation method in an open scene and the storage medium provided by the present application are described in detail above, and the principles and implementation manners of the present application are described by using specific examples in this paper. The above description of the embodiments is only used to help understand the concept of the present application, and should not be understood as a limitation on the protection scope of the present application.

Claims

1. An unsupervised semantic segmentation method in an open scenario, characterized by: include: Obtaining a first image dataset containing only known pixel categories and a second image dataset containing unknown pixel categories; Constructing a first network for feature learning and reconstruction and a second network for classification learning; In a training phase, the first image dataset is input into a first network for training, a reconstruction loss corresponding to the first image dataset is determined, and a prototype representation of each pixel category in the first image dataset is learned; For each pixel in the first image data set, determining a correction coefficient of the pixel based on a relationship between a pixel feature and the prototype representation, and correcting a reconstruction loss of the pixel using the correction coefficient; During a testing phase, the second image dataset is input into the trained first network, a reconstruction loss corresponding to the second image dataset is determined, a correction coefficient for each pixel in the second image dataset is determined based on a relationship between pixel features and the prototype representation, and the reconstruction loss of the pixel is corrected using the correction coefficient; pixels in the second image dataset and its reconstructed image are classified using the prototype representation, negative samples are selected from pixels with different classification results, and positive samples are selected from the first image dataset and / or the second image dataset; and the positive and negative samples are used for comparative learning to train the second network; An image to be segmented is obtained, and pixel classification of the image to be segmented is performed using the trained second network.

2. The unsupervised semantic segmentation method in an open scenario according to claim 1, characterized in that The method includes: using L2-norm as reconstruction loss, which is expressed as: Among them, L recon is the reconstruction error, N is the number of samples in a batch, X n,i and are the input and reconstructed value of the i-th element of the n-th sample, respectively.

3. The unsupervised semantic segmentation method in an open scenario according to claim 1, characterized in that Determining a correction coefficient of a pixel based on a relationship between a pixel feature and the prototype representation, and correcting a reconstruction loss of the pixel using the correction coefficient, comprising: Determine the cosine similarity between the pixel and the prototype representation of each category, and determine the prototype representation with the highest similarity to the pixel The cosine similarity between Subtract the cosine similarity from 1 The difference is used as the correction coefficient to correct the reconstruction loss. The expression is as follows: in is the reconstruction error after correction, L recon is the reconstruction error to be corrected.

4. The unsupervised semantic segmentation method in an open scenario according to claim 1, characterized in that The method further comprises: During the training phase, the POT method in extreme value theory is introduced to fit the tail data of the reconstruction loss after correction of each pixel in the first image dataset, and the generalized Pareto distribution parameters are determined by maximum likelihood estimation; In the testing phase, for the corrected reconstruction loss of each pixel in the second image data set, the tail probability of the corrected reconstruction loss is calculated using the generalized Pareto distribution parameters obtained in the training phase; If the tail probability exceeds a preset threshold, the pixel is determined to belong to the unknown pixel class; if the tail probability does not exceed the threshold, the pixel is determined to belong to the known pixel class; Pixels in the second image data set that are determined to be of unknown pixel class are excluded, and from the remaining pixels determined to be of known class, pixels with different classification results are used as negative samples.

5. The unsupervised semantic segmentation method in an open scenario according to claim 1, characterized in that Selecting positive samples from the first image dataset and / or the second image dataset includes: The prototype representation of each pixel category in the first image dataset learned by the first network is used as a positive sample.

6. The unsupervised semantic segmentation method in an open scenario according to claim 1, characterized in that When selecting a positive sample from the first image dataset / or the second image dataset, it is required that the cosine similarity between the pixel features of the positive sample and the prototype representation of the corresponding category is greater than 0.

8.

7. The unsupervised semantic segmentation method in an open scenario according to claim 1, characterized in that The InfoNCE loss function is used for contrastive learning to bring positive samples closer while pushing negative samples further away.

8. The unsupervised semantic segmentation method in an open scenario according to claim 1, characterized in that The first image dataset is derived from a public land cover image database, and the second image dataset is derived from actually collected remote sensing images.

9. The unsupervised semantic segmentation method in an open scenario according to claim 1, characterized in that The first network and the second network have the same network structure, both of which are encoder-decoder structures. The first network and the second network are both image reconstruction networks built based on ResNet101 and DeepLabv3+.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the unsupervised semantic segmentation method in an open scenario as described in any one of claims 1 to 8 is implemented.

Images