A method, system, medium and device for constructing remote sensing sample data set

Abstract

The present invention belongs to the field of remote sensing data, and in particular, relates to a method, system, medium and device for constructing a remote sensing sample data set. The method constructs pseudo labels through unsupervised semantic segmentation technology, optimizes sample labels based on a knowledge-guided intelligent interactive interpretation method, constructs a deep convolutional neural network that can be used for sample self-production, uses sample intelligent augmentation technology to improve sample diversity, and uses a cyclic iterative training method to complete the construction of a sample data set. The present invention fully reflects the advantages of remote sensing big data and expert knowledge, and greatly improves the efficiency of remote sensing sample set production under the joint drive of data and knowledge, and can provide a technical basis for large-area remote sensing object classification applications based on deep learning technology.

Description

Technical Field

[0001] The present invention belongs to the field of remote sensing data, and in particular relates to a method, system, medium and equipment for constructing a remote sensing sample data set. Background Art

[0002] In recent years, the rapid development of deep learning technology in the field of remote sensing has promoted the application of convolutional neural networks in land object classification. The gradual accumulation of cloud computing resources and remote sensing big data has provided a solid foundation for the development of large-area remote sensing applications. From a methodological point of view, land object classification is still dominated by supervised classification, and the model and network structure are becoming more and more regionally complex, and the demand for samples is becoming more and more dependent. From the perspective of application scale, it has gradually expanded from regional demonstration to provincial, national and even global scales. The rise of remote sensing cloud computing platforms such as Google Earth Engine has further reduced the cost of acquiring massive data. However, the main factor restricting the development of regional-scale remote sensing applications is the construction of massive remote sensing sample data sets, which further limits the expansion of artificial intelligence and remote sensing technology in the fields of remote sensing monitoring of wide-area natural resource elements and comprehensive remote sensing identification of geological disaster hazards.

[0003] Remote sensing data not only has the concept of spatial resolution, but also has multi-spectral and multi-temporal characteristics. Remote sensing data obtained by different payload platforms also have different physical meanings. These characteristics determine the difference between remote sensing data and ordinary natural images. The field of remote sensing is different from the field of computer vision. Due to the huge differences in remote sensing data, it is difficult to build a unified data set to meet the application requirements of various remote sensing scenarios. The construction of traditional remote sensing data sets is mainly based on manual interpretation, which is in great conflict with the need to carry out large-area remote sensing applications. Therefore, the main challenge currently faced is how to develop a fast remote sensing data set construction method to meet the growing application needs.

[0004] With the development of deep learning technology, the emergence of a series of unsupervised semantic segmentation algorithms and intelligent interactive interpretation methods has provided new solutions for the production of remote sensing sample data sets. The present invention proposes a method for quickly constructing remote sensing sample data sets driven by data-knowledge. Pseudo-labels are constructed through unsupervised semantic segmentation technology, sample labels are optimized based on knowledge-guided intelligent interactive interpretation methods, deep convolutional neural networks that can be used for sample self-production are constructed according to the characteristics of application scenarios and targets, sample intelligent augmentation technology is used to improve sample diversity, and a cyclic iterative training method is used to complete the construction of sample data sets. The method proposed in the present invention fully reflects the advantages of remote sensing big data and expert knowledge, and greatly improves the efficiency of remote sensing sample set production driven by data-knowledge, which can provide a technical basis for large-area remote sensing object classification applications based on deep learning technology. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a method, system, medium and equipment for constructing a remote sensing sample data set.

[0006] The technical solution of the present invention to solve the above technical problem is as follows: A method for constructing a remote sensing sample data set, comprising:

[0007] S1, processing each slice data in a slice data set corresponding to any set of remote sensing data through an unsupervised semantic segmentation network to obtain first pseudo label data corresponding to each slice data;

[0008] S2, processing the revised slice data through the unsupervised semantic segmentation network to obtain third pseudo label data, combining the third pseudo label data with the second pseudo label data to obtain preliminary label data, wherein the revised slice data is obtained by correcting the slice data corresponding to each pseudo label data having errors, and the second pseudo label data is pseudo label data without errors in the first pseudo label data;

[0009] S3, training a deep convolutional neural network by using the sliced ​​data set and the preliminary labeled data set consisting of the preliminary labeled data to obtain a first deep convolutional neural network;

[0010] S4, performing prediction processing on the remote sensing data of the sample data set to be constructed based on the first deep convolutional neural network to obtain primary label data, and selecting slice data with a confidence level exceeding a threshold from all primary label data for augmentation processing;

[0011] S5, using the slice data after the augmentation processing and the primary label data corresponding to the slice data as a new training set, training the deep convolutional neural network to obtain a second deep convolutional neural network, using the second deep convolutional neural network as the first deep convolutional neural network to perform prediction processing on the remote sensing data of the sample data set to be constructed, and repeatedly executing S4 and S5 to update the remote sensing sample data set composed of the primary label data until the requirements of the remote sensing data of the sample data set to be constructed are met, and the construction of the sample data set is completed.

[0012] The beneficial effects of the present invention are: constructing pseudo labels through unsupervised semantic segmentation technology, optimizing sample labels based on knowledge-guided intelligent interactive interpretation methods, constructing a deep convolutional neural network that can be used for sample self-production, improving sample diversity using sample intelligent augmentation technology, and constructing a complete sample data set using a cyclic iterative training method. The present invention fully reflects the advantages of remote sensing big data and expert knowledge, and greatly improves the efficiency of remote sensing sample set production under the joint drive of data and knowledge, and can provide a technical basis for large-area remote sensing object classification applications based on deep learning technology.

[0013] Based on the above technical solution, the present invention can also be improved as follows.

[0014] Further, the slice data set includes:

[0015] There are m slice data, and each slice data is obtained by randomly trimming any set of remote sensing data and then normalizing it.

[0016] Further, the correction process includes:

[0017] A number of pixels are selected from the foreground and background of the slice data corresponding to the erroneous pseudo-label data, and prompt information is added to each pixel, wherein the prompt information includes the image coordinates corresponding to the pixel and the type of the pixel.

[0018] Furthermore, S4 also includes:

[0019] It is determined whether there is an error in the primary label data whose confidence level does not exceed a threshold, and the primary label data without an error is used as the second pseudo label data to execute S2.

[0020] Another technical solution of the present invention to solve the above technical problem is as follows: A system for constructing a remote sensing sample data set, comprising:

[0021] The first prediction module is used to: process each slice data in the slice data set corresponding to any set of remote sensing data through an unsupervised semantic segmentation network to obtain first pseudo label data corresponding to each slice data;

[0022] The second prediction module is used to: process the revised slice data through the unsupervised semantic segmentation network to obtain third pseudo label data, and combine the third pseudo label data with the second pseudo label data to obtain preliminary label data, wherein the revised slice data is obtained by correcting the slice data corresponding to each pseudo label data with errors, and the second pseudo label data is pseudo label data without errors in the first pseudo label data;

[0023] The training module is used to: train the deep convolutional neural network by using the sliced ​​data set and the preliminary labeled data set composed of preliminary labeled data to obtain a first deep convolutional neural network;

[0024] The third prediction module is used to: perform prediction processing on the remote sensing data of the sample data set to be constructed based on the first deep convolutional neural network to obtain primary label data, and select slice data with a confidence level exceeding a threshold from all primary label data for augmentation processing;

[0025] The construction module is used to: use the slice data after augmentation processing and the primary label data corresponding to the slice data as a new training set, train the deep convolutional neural network, obtain a second deep convolutional neural network, use the second deep convolutional neural network as the first deep convolutional neural network to perform prediction processing on the remote sensing data of the sample data set to be constructed, and repeatedly execute the third prediction module and the construction module to update the remote sensing sample data set composed of the primary label data until the requirements of the remote sensing data of the sample data set to be constructed are met, thereby completing the construction of the sample data set.

[0026] The beneficial effects of the present invention are: constructing pseudo labels through unsupervised semantic segmentation technology, optimizing sample labels based on knowledge-guided intelligent interactive interpretation methods, constructing a deep convolutional neural network that can be used for sample self-production, improving sample diversity using sample intelligent augmentation technology, and constructing a complete sample data set using a cyclic iterative training method. The present invention fully reflects the advantages of remote sensing big data and expert knowledge, and greatly improves the efficiency of remote sensing sample set production under the joint drive of data and knowledge, and can provide a technical basis for large-area remote sensing object classification applications based on deep learning technology.

[0027] Further, the slice data set includes:

[0028] There are m slice data, and each slice data is obtained by randomly trimming any set of remote sensing data and then normalizing it.

[0029] Further, the correction process includes:

[0030] A number of pixels are selected from the foreground and background of the slice data corresponding to the erroneous pseudo-label data, and prompt information is added to each pixel, wherein the prompt information includes the image coordinates corresponding to the pixel and the type of the pixel.

[0031] Furthermore, the third prediction module is also used for:

[0032] It is determined whether there is an error in the primary label data whose confidence does not exceed a threshold, and the primary label data without errors is used as the second pseudo label data to execute the second prediction module.

[0033] Another technical solution of the present invention to solve the above technical problem is as follows: a storage medium, in which instructions are stored, and when a computer reads the instructions, the computer executes any of the methods described above.

[0034] The beneficial effects of the present invention are: constructing pseudo labels through unsupervised semantic segmentation technology, optimizing sample labels based on knowledge-guided intelligent interactive interpretation methods, constructing a deep convolutional neural network that can be used for sample self-production, improving sample diversity using sample intelligent augmentation technology, and constructing a complete sample data set using a cyclic iterative training method. The present invention fully reflects the advantages of remote sensing big data and expert knowledge, and greatly improves the efficiency of remote sensing sample set production under the joint drive of data and knowledge, and can provide a technical basis for large-area remote sensing object classification applications based on deep learning technology.

[0035] Another technical solution of the present invention to solve the above technical problem is as follows: an electronic device comprises the above storage medium and a processor for executing instructions in the above storage medium.

[0036] The beneficial effects of the present invention are: constructing pseudo labels through unsupervised semantic segmentation technology, optimizing sample labels based on knowledge-guided intelligent interactive interpretation methods, constructing a deep convolutional neural network that can be used for sample self-production, improving sample diversity using sample intelligent augmentation technology, and constructing a complete sample data set using a cyclic iterative training method. The present invention fully reflects the advantages of remote sensing big data and expert knowledge, and greatly improves the efficiency of remote sensing sample set production under the joint drive of data and knowledge, and can provide a technical basis for large-area remote sensing object classification applications based on deep learning technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 A schematic diagram of a process flow provided for a method for constructing a remote sensing sample data set according to an embodiment of the present invention;

[0038] Figure 2 A schematic diagram of a remote sensing sample data set construction system embodiment provided by the present invention;

[0039] Figure 3 An unsupervised semantic segmentation network structure diagram provided for an embodiment of a method for constructing a remote sensing sample data set of the present invention;

[0040] Figure 4 An example diagram of a deep convolutional neural network adapted to a business scenario provided in an embodiment of a method for constructing a remote sensing sample data set of the present invention;

[0041] Figure 5 A basic flow chart of constructing a remote sensing sample data set driven by data-knowledge combination is provided for a method for constructing a remote sensing sample data set according to an embodiment of the present invention. DETAILED DESCRIPTION

[0042] The principles and features of the present invention are described below. The examples given are only used to explain the present invention and are not used to limit the scope of the present invention.

[0043] like Figure 1 As shown, a method for constructing a remote sensing sample data set includes:

[0044] S1, processing each slice data in a slice data set corresponding to any set of remote sensing data through an unsupervised semantic segmentation network to obtain first pseudo label data corresponding to each slice data;

[0045] S2, processing the revised slice data through the unsupervised semantic segmentation network to obtain third pseudo label data, combining the third pseudo label data with the second pseudo label data to obtain preliminary label data, wherein the revised slice data is obtained by correcting the slice data corresponding to each pseudo label data having errors, and the second pseudo label data is pseudo label data without errors in the first pseudo label data;

[0046] S3, training a deep convolutional neural network by using the sliced ​​data set and the preliminary labeled data set consisting of the preliminary labeled data to obtain a first deep convolutional neural network;

[0047] S4, performing prediction processing on the remote sensing data of the sample data set to be constructed based on the first deep convolutional neural network to obtain primary label data, and selecting slice data with a confidence level exceeding a threshold from all primary label data for augmentation processing;

[0048] S5, using the slice data after the augmentation processing and the primary label data corresponding to the slice data as a new training set, training the deep convolutional neural network to obtain a second deep convolutional neural network, using the second deep convolutional neural network as the first deep convolutional neural network to perform prediction processing on the remote sensing data of the sample data set to be constructed, and repeatedly executing S4 and S5 to update the remote sensing sample data set composed of the primary label data until the requirements of the remote sensing data of the sample data set to be constructed are met, and the construction of the sample data set is completed.

[0049] In some possible implementations, pseudo labels are constructed through unsupervised semantic segmentation technology, sample labels are optimized based on knowledge-guided intelligent interactive interpretation methods, deep convolutional neural networks that can be used for sample self-production are constructed, sample intelligent augmentation technology is used to improve sample diversity, and a complete sample data set is constructed using a cyclic iterative training method. The present invention fully reflects the advantages of remote sensing big data and expert knowledge, and the efficiency of remote sensing sample set production is greatly improved under the joint drive of data and knowledge, which can provide a technical basis for large-area remote sensing object classification applications based on deep learning technology.

[0050] S1, such as Figure 5As shown, the specific process of processing each slice data in the slice data set corresponding to any set of remote sensing data through the unsupervised semantic segmentation network to obtain the first pseudo-label data corresponding to each slice data is as follows:

[0051] The unsupervised semantic segmentation network is: a general unsupervised semantic segmentation network is built in the Pytorch environment. The unsupervised semantic segmentation network is mainly composed of two parts: the encoder and the decoder. The decoder is an image decoder, which mainly includes an upsampling layer, a patch feature embedding layer, four isometric global attention modules, and a dynamic prompt layer. Figure 3 The Transformer module in is the isometric global attention module.

[0052] Remote sensing data include but are not limited to: optical remote sensing image data, radar remote sensing image data and terrain data acquired through satellite or aviation payloads;

[0053] Pseudo-labeled data refers to labeled data generated by the model without manual labeling.

[0054] S11, using the MAE model and SAM to initialize the parameters of the unsupervised semantic segmentation network;

[0055] Among them, the MAE model stands for Masked Autoencoders, which is an autoencoding network model proposed by He Kaiming.

[0056] SAM stands for Segment Anything Model, which is a large semantic segmentation visual model launched by Meta.

[0057] MAE and SAM both prefer the ViT-H / 16 version of ViT, namely the ViT module, whose parameters are initialized from the trained MAE model. The dynamic prompt layer of the unsupervised semantic segmentation network consists of three groups of convolutional layers Conv, layer normalization LN and GELU activation layers.

[0058] Among the three groups of convolutional layers, except for the middle group of convolutional layers which uses 3×3 convolution, the other two groups of convolutional layers use 1×1 convolution;

[0059] The dynamic prompt layer allows the unsupervised semantic segmentation network to fully utilize the extraction capabilities of the pre-trained ViT large model while iteratively finding suitable prompt word information for specific tasks, thereby enabling the model to achieve transfer learning at a lower computational cost. The pre-trained ViT large model is the MAE model and the SAM model.

[0060] The image decoder includes a ViT module and two layers of MLP, which are used to restore the image feature embedding information into label data of the same size as the input image. In addition to receiving the image embedding features output by the image encoder, the image decoder will also receive prompt information consisting of coordinate points, coordinate boxes or text information. The prompt information will be input into the keyword prompter and mapped into a feature vector and then passed into the image decoder to optimize the extraction results.

[0061] The parameters in parameter initialization generally refer to: In machine learning and neural networks, parameter initialization refers to the process of assigning initial values ​​to the parameters in the model before training the model. Parameters are learnable variables in the model, such as the weights and biases of a neural network.

[0062] MLP refers to: Multi-Layer Perceptron, a forward artificial neural network model, which consists of several neural network layers (including input layer, hidden layer and output layer). Each neural network layer consists of multiple neurons, which perform weighted summation and nonlinear transformation on the input signal through weights and activation functions, thereby realizing complex nonlinear mapping.

[0063] S12, slicing the remote sensing data to obtain a slice data set, and normalizing each slice data in the data set;

[0064] The slicing process is as follows:

[0065] Randomly crop the remote sensing data to form m slices of size h×w×3, where h and w are close to 1024. All slices will be upsampled or padded to 1024×1024 after entering the unsupervised semantic segmentation network. The m slices are obtained as the slice dataset D1;

[0066] The normalization process is:

[0067] Each slice data in D1 is processed by the following formula to obtain data set D2;

[0068] X'=(x-xmin) / (xmax-xmin)

[0069] Among them, X' represents the slice data after normalization, and xmin and xmax represent the minimum and maximum values ​​of the data respectively.

[0070] S13, processing the normalized slice data set through an unsupervised semantic segmentation network to obtain pseudo label data (i.e., first pseudo label data) corresponding to each slice data, thereby forming a pseudo label data set P1;

[0071] Among them, there are m pseudo-label data in total, and the size is 3×1024×1024;

[0072] The processing process of the unsupervised semantic segmentation network is:

[0073] pred ini =Model gen (X),

[0074] Among them, pred ini represents pseudo-labeled data;

[0075] Model gen (X) is the unsupervised semantic segmentation network.

[0076] S2, the revised slice data is processed by the unsupervised semantic segmentation network to obtain third pseudo label data, and the third pseudo label data is combined with the second pseudo label data to obtain preliminary label data, wherein the revised slice data is obtained by correcting the slice data corresponding to each pseudo label data having errors, and the second pseudo label data is the pseudo label data without errors in the first pseudo label data.

[0077] S21, selecting the pseudo-label data with errors through a preset scheme to obtain the pseudo-label data with errors;

[0078] Errors refer to situations where the pseudo-label results obtained are inconsistent with the judgment of expert knowledge.

[0079] Pre-set solutions include but are not limited to:

[0080] 1) Screening through expert knowledge;

[0081] Expert knowledge refers to:

[0082] 2) All pseudo-label data are filtered through a preset filtering model.

[0083] S22, determining the remote sensing data corresponding to all the false label data with errors, that is, the corresponding slice data, processing the slice data corresponding to the false label data with errors by adding foreground and background prompt words to obtain revised slice data, and forming a revised slice data set D3 through all the revised slice data;

[0084] The process of adding foreground and background prompt words is as follows:

[0085] Through visual interpretation, several points are selected in the foreground and background of the image, respectively. pos and Point nagTo indicate that each point includes three pieces of information (x, y, t), where x and y represent the surface coordinates of the point, and t represents the type of point (foreground coordinate point t is 0, background coordinate point t is 1), and these pieces of information are used as prompt information.

[0086] Visual interpretation refers to the process of interpreting data by directly observing features and patterns in the data.

[0087] S23, using an unsupervised semantic segmentation network to process the revised slice data set to obtain third pseudo-label data, i.e., the processing result, and the third pseudo-label data and the pseudo-label data without obvious errors in the first pseudo-label data, i.e., the second pseudo-label data, are combined into preliminary label data P2.

[0088] pred opt '=Model gen {X”,(Point pos ,Point nag )},

[0089]

[0090] X" indicates the revised slice data, pred opt ' represents the pseudo label data corresponding to "X", that is, the third pseudo label data.

[0091] S3, training the deep convolutional neural network by the sliced ​​data set and the preliminary labeled data set composed of the preliminary labeled data, and obtaining the process of the first deep convolutional neural network is:

[0092] A deep convolutional neural network does not refer to a specific network, but an end-to-end semantic segmentation network based on business needs.

[0093] Taking building information extraction as an example, the structure of the classic semantic segmentation model Unet is used as a reference for optimization design (such as Figure 4 As shown in the figure, all 3×3 ordinary convolutions are replaced with depth-separable convolutions. In the decoder part of the model, the features of each level are merged, and the channel attention mechanism is added to perform weighted processing of effective features to complete the business network model. proThe construction of the channel attention module. In the structure of the channel attention module, the feature maps obtained at different scales are first merged through the feature superposition operation, and then reduced in dimension through the global pooling layer. After two consecutive 1×1 convolutions and Sigmoid activation functions, the feature gain coefficient is obtained. This coefficient is multiplied by the merged features input by the channel attention module to complete the re-weighting of the specific channel features, thereby achieving the optimization of features. The image first enters the encoder of the network for scale compression, and then passes through the decoder for scale restoration. In the process, the features of different scales from the encoder will be merged with the decoder's scale features. The merged features of each scale in the decoder are then processed by two layers of deep separable convolutional machine groups (composed of two groups of "deep separable convolutional layer DW-Conv+ batch normalization layer BN+ReLU activation layer"), and the obtained features are added to the channel attention machine module for weighted processing of effective features, and finally the mask result with the same size as the input original image is output.

[0094] S4, based on the first deep convolutional neural network, the remote sensing data of the sample data set to be constructed is predicted and processed to obtain the first-level label data P3, and the process of selecting the slice data with a confidence level exceeding the threshold value from all the first-level label data for augmentation processing is as follows:

[0095] The threshold can be set manually according to actual needs. In this solution, when the probability value in the primary label data is within the range of 0.7-1, it is considered to be high confidence, and within the range of 0.1-0.4, it is considered to be the second confidence. Within the range of 0.4-0.7, it is considered as: no judgment is made.

[0096] When the confidence level is lower than the threshold, the slice data with the confidence level lower than the threshold is reprocessed to add foreground and background prompt words, and the subsequent operations are repeated.

[0097] The augmentation process is:

[0098] The augmentation processing in this scheme includes geometric augmentation and color augmentation, both of which are common operations in deep learning applications.

[0099] The geometric augmentation operation requires synchronous processing of the remote sensing data corresponding to the erroneous pseudo-label data and its corresponding first-level label data P3. Through scaling, ninety-degree rotation and flipping operations, after geometric augmentation, the data volume of the remote sensing data corresponding to the erroneous pseudo-label data and its corresponding first-level label data P3 is expanded to 8 times the original.

[0100] The color augmentation operation only processes the remote sensing data corresponding to the erroneous pseudo-label data. Specifically, it randomly changes the brightness and contrast by 0-10%. After color augmentation, the amount of remote sensing data corresponding to the erroneous pseudo-label data is expanded to 4 times the original amount.

[0101] The remote sensing data corresponding to the erroneous pseudo-label data and its corresponding primary label data P3 are band-merged to form a new data set DP3, and a scene change operation is performed on DP3.

[0102] The scene change operation is: X m1 , X m2 are any two slice data in DP3, that is First, each deviation data is cut into 4 mini slices in both horizontal and vertical directions, and the size of each mini slice is one quarter of the original slice data. The specific formula is as follows:

[0103]

[0104] Among them, f clip () indicates that horizontal and vertical cropping are performed simultaneously. mn1 and X mn2 Two sets of mini slice data are randomly mosaicked and combined. During the mosaicking process, the mini slice data can be randomly rotated or flipped (multiples of 90°) to form two new sets of slice data. The specific formula is as follows:

[0105] X' m1 , X' m2 =f stack ([f fl90 (X mn1 ),f fl90 (X mn2 )])

[0106] where f stack represents random mosaic processing, f fl90 represents a random rotation or flip operation, X' m1 and X' m2 The data is randomly mosaicked. All slices in the dataset DP3 are traversed to perform the above operations, the image and label parts in DP3 are separated again, and the dataset is expanded to twice the original size through scene transformation.

[0107] S5, using the slice data after the augmentation processing and the primary label data corresponding to the slice data as a new training set, training the deep convolutional neural network to obtain a second deep convolutional neural network, using the second deep convolutional neural network as the first deep convolutional neural network to perform prediction processing on the remote sensing data of the sample data set to be constructed, and repeatedly executing S4 and S5 to update the remote sensing sample data set composed of the primary label data until the requirements of the remote sensing data of the sample data set to be constructed are met, and the construction of the sample data set is completed.

[0108] Through the geometric augmentation, color augmentation and scene transformation in the above steps, the volume of the data set is expanded to 14 times of the original data set. According to the new slice data set (the slice data set composed of the slice data after augmentation processing) and the new pseudo-label data set (the first-level label data set corresponding to the above slice data), the deep convolutional neural network adapted to the business scenario is repeatedly trained to obtain the second deep convolutional neural network.

[0109] The remote sensing data for constructing the sample data set are processed according to the second deep convolutional neural network to obtain new processing results, and the confidence judgment and augmentation processing are repeated, thereby realizing the cyclic iterative generation of sample data.

[0110] The termination time of the loop iteration process is determined according to the requirements or demands of the remote sensing data for constructing the sample data set, for example, the requirement is to iterate four times, or the accuracy reaches a threshold value, etc.

[0111] This solution can be used to construct a sample data set of terrain data, for example. The sample data set can be used for further analysis of terrain remote sensing data to obtain landform features, etc. The sample data set can also be used to identify landform images.

[0112] Preferably, in any of the above embodiments, the slice data set includes:

[0113] There are m slice data, and each slice data is obtained by randomly trimming any set of remote sensing data and then normalizing it.

[0114] Preferably, in any of the above embodiments, the correction process includes:

[0115] A number of pixels are selected from the foreground and background of the slice data corresponding to the erroneous pseudo-label data, and prompt information is added to each pixel, wherein the prompt information includes the image coordinates corresponding to the pixel and the type of the pixel.

[0116] Preferably, in any of the above embodiments, S4 further includes:

[0117] It is determined whether there is an error in the primary label data whose confidence level does not exceed a threshold, and the primary label data without an error is used as the second pseudo label data to execute S2.

[0118] like Figure 2 As shown, a system for constructing a remote sensing sample data set includes:

[0119] The first prediction module 100 is used to: process each slice data in the slice data set corresponding to any set of remote sensing data through an unsupervised semantic segmentation network to obtain first pseudo label data corresponding to each slice data;

[0120] The second prediction module 200 is used to: process the revised slice data through the unsupervised semantic segmentation network to obtain third pseudo label data, and combine the third pseudo label data with the second pseudo label data to obtain preliminary label data, wherein the revised slice data is obtained by correcting the slice data corresponding to each pseudo label data with errors, and the second pseudo label data is pseudo label data without errors in the first pseudo label data;

[0121] The training module 300 is used to: train the deep convolutional neural network by using the slice data set and the preliminary label data set composed of preliminary label data to obtain a first deep convolutional neural network;

[0122] The third prediction module 400 is used to: perform prediction processing on the remote sensing data for constructing the sample data set based on the first deep convolutional neural network to obtain primary label data, and select slice data with a confidence level exceeding a threshold from all primary label data for augmentation processing;

[0123] The construction module 400 is used to: use the slice data after augmentation processing and the primary label data corresponding to the slice data as a new training set, train the deep convolutional neural network, obtain a second deep convolutional neural network, use the second deep convolutional neural network as the first deep convolutional neural network to perform prediction processing on the remote sensing data of the sample data set to be constructed, and repeatedly execute the third prediction module 400 and the construction module 400 to update the remote sensing sample data set composed of the primary label data until the requirements of the remote sensing data of the sample data set to be constructed are met, thereby completing the construction of the sample data set.

[0124] In some possible implementations, pseudo labels are constructed through unsupervised semantic segmentation technology, sample labels are optimized based on knowledge-guided intelligent interactive interpretation methods, deep convolutional neural networks that can be used for sample self-production are constructed, sample intelligent augmentation technology is used to improve sample diversity, and a complete sample data set is constructed using a cyclic iterative training method. The present invention fully reflects the advantages of remote sensing big data and expert knowledge, and the efficiency of remote sensing sample set production is greatly improved under the joint drive of data and knowledge, which can provide a technical basis for large-area remote sensing object classification applications based on deep learning technology.

[0125] Preferably, in any of the above embodiments, the slice data set includes:

[0126] There are m slice data, and each slice data is obtained by randomly trimming any set of remote sensing data and then normalizing it.

[0127] Preferably, in any of the above embodiments, the correction process includes:

[0128] A number of pixels are selected from the foreground and background of the slice data corresponding to the erroneous pseudo-label data, and prompt information is added to each pixel, wherein the prompt information includes the image coordinates corresponding to the pixel and the type of the pixel.

[0129] Preferably, in any of the above embodiments, the third prediction module 400 is further used for:

[0130] It is determined whether there is an error in the primary label data whose confidence does not exceed a threshold, and the primary label data without errors is used as the second pseudo label data to execute the second prediction module 200.

[0131] Another technical solution of the present invention to solve the above technical problem is as follows: a storage medium, in which instructions are stored, and when a computer reads the instructions, the computer executes any of the methods described above.

[0132] In some possible implementations, pseudo labels are constructed through unsupervised semantic segmentation technology, sample labels are optimized based on knowledge-guided intelligent interactive interpretation methods, deep convolutional neural networks that can be used for sample self-production are constructed, sample intelligent augmentation technology is used to improve sample diversity, and a complete sample data set is constructed using a cyclic iterative training method. The present invention fully reflects the advantages of remote sensing big data and expert knowledge, and the efficiency of remote sensing sample set production is greatly improved under the joint drive of data and knowledge, which can provide a technical basis for large-area remote sensing object classification applications based on deep learning technology.

[0133] Another technical solution of the present invention to solve the above technical problem is as follows: an electronic device comprises the above storage medium and a processor for executing instructions in the above storage medium.

[0134] In some possible implementations, pseudo labels are constructed through unsupervised semantic segmentation technology, sample labels are optimized based on knowledge-guided intelligent interactive interpretation methods, deep convolutional neural networks that can be used for sample self-production are constructed, sample intelligent augmentation technology is used to improve sample diversity, and a complete sample data set is constructed using a cyclic iterative training method. The present invention fully reflects the advantages of remote sensing big data and expert knowledge, and the efficiency of remote sensing sample set production is greatly improved under the joint drive of data and knowledge, which can provide a technical basis for large-area remote sensing object classification applications based on deep learning technology.

[0135] The reader should understand that in the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples without contradiction.

[0136] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the method embodiments described above are only illustrative, for example, the division of steps is only a logical function division, and there may be other division methods in actual implementation, such as multiple steps can be combined or integrated into another step, or some features can be ignored or not executed.

[0137] If the above method is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0138] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for constructing a remote sensing sample data set, characterized in that: include: S1, processing each slice data in a slice data set corresponding to any set of remote sensing data through an unsupervised semantic segmentation network to obtain first pseudo label data corresponding to each slice data; S2, processing the revised slice data through the unsupervised semantic segmentation network to obtain third pseudo label data, combining the third pseudo label data with the second pseudo label data to obtain preliminary label data, wherein the revised slice data is obtained by correcting the slice data corresponding to each pseudo label data having errors, and the second pseudo label data is pseudo label data without errors in the first pseudo label data; S3, training a deep convolutional neural network by using the sliced ​​data set and the preliminary labeled data set consisting of the preliminary labeled data to obtain a first deep convolutional neural network; S4, performing prediction processing on the remote sensing data of the sample data set to be constructed based on the first deep convolutional neural network to obtain primary label data, and selecting slice data with a confidence level exceeding a threshold from all primary label data for augmentation processing; S5, using the slice data after the augmentation processing and the primary label data corresponding to the slice data as a new training set, training the deep convolutional neural network to obtain a second deep convolutional neural network, using the second deep convolutional neural network as the first deep convolutional neural network to perform prediction processing on the remote sensing data of the sample data set to be constructed, and repeatedly executing S4 and S5 to update the remote sensing sample data set composed of the primary label data until the requirements of the remote sensing data of the sample data set to be constructed are met, and the construction of the sample data set is completed.

2. The method for constructing a remote sensing sample data set according to claim 1, characterized in that: The slice data set includes: There are m slice data, and each slice data is obtained by randomly trimming any set of remote sensing data and then normalizing it.

3. The method for constructing a remote sensing sample data set according to claim 1, characterized in that: The correction process includes: A number of pixels are selected from the foreground and background of the slice data corresponding to the erroneous pseudo-label data, and prompt information is added to each pixel, wherein the prompt information includes the image coordinates corresponding to the pixel and the type of the pixel.

4. The method for constructing a remote sensing sample data set according to claim 1, characterized in that: S4 also includes: It is determined whether there is an error in the primary label data whose confidence level does not exceed a threshold, and the primary label data without an error is used as the second pseudo label data to execute S2.

5. A system for constructing a remote sensing sample data set, characterized in that: include: The first prediction module is used to: process each slice data in the slice data set corresponding to any set of remote sensing data through an unsupervised semantic segmentation network to obtain first pseudo label data corresponding to each slice data; The second prediction module is used to: process the revised slice data through the unsupervised semantic segmentation network to obtain third pseudo label data, and combine the third pseudo label data with the second pseudo label data to obtain preliminary label data, wherein the revised slice data is obtained by correcting the slice data corresponding to each pseudo label data with errors, and the second pseudo label data is pseudo label data without errors in the first pseudo label data; The training module is used to: train the deep convolutional neural network by using the sliced ​​data set and the preliminary labeled data set composed of preliminary labeled data to obtain a first deep convolutional neural network; The third prediction module is used to: perform prediction processing on the remote sensing data of the sample data set to be constructed based on the first deep convolutional neural network to obtain primary label data, and select slice data with a confidence level exceeding a threshold from all primary label data for augmentation processing; The construction module is used to: use the slice data after augmentation processing and the primary label data corresponding to the slice data as a new training set, train the deep convolutional neural network, obtain a second deep convolutional neural network, use the second deep convolutional neural network as the first deep convolutional neural network to perform prediction processing on the remote sensing data of the sample data set to be constructed, and repeatedly execute the third prediction module and the construction module to update the remote sensing sample data set composed of the primary label data until the requirements of the remote sensing data of the sample data set to be constructed are met, thereby completing the construction of the sample data set.

6. A remote sensing sample data set construction system according to claim 5, characterized in that: The slice data set includes: There are m slice data, and each slice data is obtained by randomly trimming any set of remote sensing data and then normalizing it.

7. A remote sensing sample data set construction system according to claim 5, characterized in that: The correction process includes: A number of pixels are selected from the foreground and background of the slice data corresponding to the erroneous pseudo-label data, and prompt information is added to each pixel, wherein the prompt information includes the image coordinates corresponding to the pixel and the type of the pixel.

8. The system for constructing a remote sensing sample data set according to claim 5, characterized in that: The third prediction module is also used for: It is determined whether there is an error in the primary label data whose confidence does not exceed a threshold, and the primary label data without errors is used as the second pseudo label data to execute the second prediction module.

9. A storage medium, characterized in that: The medium stores instructions, and when a computer reads the instructions, the computer executes the method according to any one of claims 1 to 4.

10. An electronic device, characterized in that: The invention comprises the storage medium as claimed in claim 9 and a processor for executing instructions in the storage medium.

Images