A remote sensing image self-supervised training method, device and electronic equipment

Abstract

The application provides a remote sensing image self-supervised training method and device and electronic equipment. The method comprises the following steps: dividing a remote sensing image into multiple patches based on target achievement data, each patch corresponding to a category attribute; performing block processing on the remote sensing image after the division and screening the blocks based on the attributes of the patches in the blocks to obtain a remote sensing sub-image set; obtaining positive samples and negative samples of each remote sensing sub-image based on the similarity between each remote sensing sub-image and the remaining remote sensing sub-images in the remote sensing sub-image set; and training an initial remote sensing image feature extraction model using each remote sensing sub-image and the corresponding positive samples and negative samples to obtain a remote sensing image feature extraction model. The method can quickly and effectively construct a positive and negative sample pair set, and can also quickly train a converged remote sensing image feature extraction model through the constructed positive and negative sample pair set.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of remote sensing image processing, and in particular to a remote sensing image self-supervised training method and device and electronic equipment. BACKGROUND

[0002] Remote sensing images have been widely used in the field of earth observation, such as land cover classification, forest change monitoring, etc., among which, remote sensing image classification is an important technical means for converting remote sensing data into remote sensing information, and a remote sensing image classification model often needs a large number of labeled samples to train and optimize the parameters of the remote sensing image classification model.

[0003] In 2012, Krizhevsky A. published AlexNet work in the top conference of machine learning field 2012 Conference on Neural Information Processing Systems (NIPS), which made a major breakthrough on the classic image recognition task, and increased the recognition rate by nearly 10% compared with the traditional support vector machine-based recognition, which aroused great interest in the academic and industrial circles for the relatively classic neural network. In the academic field, neural networks were a mainstream machine learning school in the 1980s, but due to the lack of sufficient training data and relatively weak computing power at that time, the training of multi-layer neural networks was difficult and time-consuming, and often produced over-fitting classification networks, so the recognition effect in practical application was not good. In recent years, the emergence of large data sets with sufficient data and class labels (such as ImageNet, etc.), and the enhancement of GPU computing power of graphics cards have made it possible to train good network model parameters for multi-layer neural networks in an acceptable time. The most basic structural unit of convolutional neural network includes convolution layer, pooling layer and full connection layer. The convolution layer performs convolution operation on the image in the window to extract various types of features; the pooling layer typically includes averaging and maximizing, which is a further abstraction of the extracted features; the full connection layer is to stack the outputs of the convolution layer and the pooling layer using a vector to form one or more full connection layers to realize high-order derivation capability. After the great success of AlexNet, subsequent researchers have extensively studied the increase of network layers, network optimization methods, etc., and have appeared GoogleNet, VGGNet, ResNet, etc. which are influential and typical convolutional neural networks.

[0004] An important prerequisite for the success of deep learning models is the reliance on a large number of labeled samples. Inspired by the limitations of relying on a large number of labeled samples, in recent years, self-supervised learning has received widespread attention, resulting in some influential algorithm models such as MoCo (reference paper: Momentum Contrast for Unsupervised Visual Representation Learning, CVPR2020), SimCLR (reference paper: A Simple Framework for Contrastive Learning of Visual Representations, 2020). Self-supervised learning mainly uses auxiliary tasks (pretext) to mine supervision information from large-scale unsupervised data, and trains the network through the supervision information constructed in this way, so as to learn representations valuable for downstream tasks. That is, the supervision information of self-supervised learning is not manually labeled, but is automatically constructed by the algorithm in large-scale unsupervised data to perform supervised learning or training. Self-supervised learning can be regarded as an "ideal state" of machine learning, where the model learns directly from unlabeled data without the need for labeled data. The core of self-supervised learning lies in how to automatically generate labels for data, for example, inputting a picture, randomly rotating the picture by an angle, and then taking the rotated picture as input and the rotation angle as label. For another example, divide the input picture into a 3*3 grid, and take the content in each grid as a patch. Randomly shuffle the order of the patches, then use the shuffled patches as input and the correct order as the class label. Similar to this automatically generated annotation, no human intervention is required.

[0005] Another type of self-supervised learning method is based on contrast constraints, which constructs representations by learning to encode the similarity or dissimilarity of two things. The performance of this method is very strong at present, that is, by constructing positive samples (positive) and negative samples (negative), and then measuring the distance between positive and negative samples to achieve self-supervised learning. However, based on remote sensing images, the construction of positive and negative samples by manual annotation method is extremely low in efficiency, and there is a lack of fast and effective method for constructing positive and negative samples. SUMMARY

[0006] The present application provides a remote sensing image self-supervised training method and device and electronic equipment to solve the defect of low construction efficiency of positive and negative samples constructed by manual annotation method in the prior art model self-supervised training, and to realize a self-supervised training method of a remote sensing image feature extraction model capable of quickly and effectively constructing positive and negative samples.

[0007] The application provides a remote sensing image self-supervised training method, comprising:

[0008] The remote sensing image is divided into multiple image patches based on target result data, and each image patch corresponds to a category attribute;

[0009] The remote sensing image after the region division is subjected to block processing, and the blocks are screened based on the attributes of the image patches in the blocks to obtain a remote sensing sub-image set;

[0010] Based on the similarity between each remote sensing sub-image in the remote sensing sub-image set and the remaining remote sensing sub-images, the positive samples and negative samples of each remote sensing sub-image are obtained;

[0011] Each remote sensing sub-image and the corresponding positive samples and negative samples are used to perform self-supervised training on an initial remote sensing image feature extraction model to obtain a remote sensing image feature extraction model.

[0012] According to the remote sensing image self-supervised training method provided by the application, the remote sensing image is divided into multiple image patches based on target result data, and each image patch corresponds to a category attribute, comprising:

[0013] Target result data is obtained based on the remote sensing image;

[0014] The target result data and the remote sensing image are subjected to spatial overlap registration, so that the remote sensing image is divided into multiple image patches according to the target result data;

[0015] The category attribute of each image patch is defined according to the target result data.

[0016] According to the remote sensing image self-supervised training method provided by the application, the remote sensing image after the region division is subjected to block processing, comprising:

[0017] A sliding window and a sliding step are set, and the size of the sliding step is less than or equal to the width of the sliding window in the sliding direction;

[0018] The remote sensing image after the region division is subjected to block processing by moving the sliding window in the sliding step from left to right and from top to bottom.

[0019] According to the remote sensing image self-supervised training method provided by the application, the remote sensing sub-image set is obtained by screening the blocks based on the attributes of the image patches in the blocks, comprising:

[0020] The area proportion of the image patches of each category attribute in each block is calculated;

[0021] when the area proportion of the graph patch of each category attribute in the sub-block is lower than a preset threshold, the sub-block is excluded;

[0022] when the area proportion of the graph patch of one category attribute in the sub-block is higher than the preset threshold, the sub-block is retained to constitute a remote sensing sub-image set.

[0023] According to the remote sensing image self-supervision training method provided by the application, the similarity between each remote sensing sub-image in the remote sensing sub-image set and the remaining remote sensing sub-images is obtained, and the positive samples and negative samples of each remote sensing sub-image are obtained, including:

[0024] The similarity between each remote sensing sub-image in the remote sensing sub-image set and the remaining remote sensing sub-images is calculated, the remote sensing sub-image with similarity is defined as a positive sample, and the remote sensing sub-image without similarity is defined as a negative sample.

[0025] Among each remote sensing sub-image, the main category attributes are the same, and the main category attributes are different.

[0026] According to the remote sensing image self-supervision training method provided by the application, the initial remote sensing image feature extraction model is self-supervised trained by using each remote sensing sub-image and the corresponding positive sample and negative sample, and a remote sensing image feature extraction model is obtained, including:

[0027] The initial remote sensing image feature extraction model is established.

[0028] The initial remote sensing image feature extraction model is self-supervised trained by using each remote sensing sub-image and the corresponding positive sample and negative sample.

[0029] According to the positive sample and the negative sample, a loss function is established, the loss function is optimized, the parameters of the initial remote sensing image feature extraction model are updated until convergence, and a remote sensing image feature extraction model is obtained.

[0030] The application also provides a remote sensing image self-supervision training device, comprising:

[0031] The region division module is used for dividing the remote sensing image into a plurality of graph patches based on the target result data, and each graph patch corresponds to a category attribute.

[0032] The screening module is used for performing sub-block processing on the remote sensing image after region division and screening the sub-block based on the attribute of the graph patch in the sub-block to obtain a remote sensing sub-image set.

[0033] a sample obtaining module configured to obtain positive samples and negative samples of each of the remote sensing sub-images based on similarities between each of the remote sensing sub-images and the rest of the remote sensing sub-images in the set of remote sensing sub-images;

[0034] a training module configured to perform self-supervised training on an initial remote sensing image feature extraction model using each of the remote sensing sub-images and the corresponding positive samples and negative samples to obtain a remote sensing image feature extraction model.

[0035] The present application also provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the remote sensing image self-supervised training method according to any one of the preceding embodiments when executing the program.

[0036] The present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the remote sensing image self-supervised training method according to any one of the preceding embodiments.

[0037] The present application also provides a computer program product comprising a computer program, wherein the computer program, when executed by a processor, implements the remote sensing image self-supervised training method according to any one of the preceding embodiments.

[0038] The remote sensing image self-supervised training method, device, and electronic device provided by the present application can quickly and effectively construct a set of positive and negative sample pairs by using target achievement data such as Open Street Map (open source map data) or historical achievement data accumulated by various industries, and can quickly train a converged remote sensing image feature extraction model by using the constructed set of positive and negative sample pairs, thereby providing support for subsequent remote sensing image classification. BRIEF DESCRIPTION OF DRAWINGS

[0039] In order to more clearly illustrate the technical solutions of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings described below are some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without any creative effort.

[0040] Figure 1 is a flowchart of the remote sensing image self-supervised training method provided by the present application;

[0041] Figure 2 is a schematic diagram of the remote sensing image after being divided into regions according to the present application;

[0042] Figure 3 is a flowchart of the method for dividing the remote sensing image into a plurality of image patches according to the present application.

[0043] Figure 4 This is a flowchart illustrating the method for segmenting remote sensing images after region division, as provided by the present invention.

[0044] Figure 5 This is a schematic diagram of the completed segmentation of a remote sensing image provided by the present invention;

[0045] Figure 6 This is a flowchart illustrating the method for filtering blocks provided by the present invention;

[0046] Figure 7 This is a flowchart illustrating the method for training an initial remote sensing image feature extraction model provided by the present invention.

[0047] Figure 8 This is a schematic diagram of the structure of the remote sensing image self-supervised training device provided by the present invention;

[0048] Figure 9 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0050] The following is combined Figures 1 to 7 The present invention describes a self-supervised training method for remote sensing images, comprising:

[0051] S101: Based on the target result data, the remote sensing image is divided into regions, and the remote sensing image is divided into multiple patches, each patch corresponding to a category attribute.

[0052] Specifically, the target outcome data can be open-source vector data such as Open Street Map, or historical outcome data completed in previous periods by different industries or departments. For example, the Third National Land Survey (referred to as the Third Survey) led by the China Land Surveying and Planning Institute under the Ministry of Natural Resources of my country has formed a basic base map information on the current land use status in my country.

[0053] For remote sensing images, refer to Figure 2 As shown, by dividing the remote sensing image into regions based on the target result data corresponding to the entire area of ​​the remote sensing image, the remote sensing image can be divided into multiple patches, such as... Figure 2The plurality of region blocks have white borders. After the polygons are divided, the category attributes corresponding to each polygon can be obtained according to the target result data. The category attribute can be a classification of each polygon based on the use or other attributes, for example, Figure 2 The category attributes of ① and ⑤ in the above are residential land polygons, ② is a traffic land polygon, ③ is a commercial land polygon, and ④ is a park green land polygon, but the present application is not limited thereto.

[0054] S102: The divided remote sensing image is subjected to block processing, and the blocks are screened based on the attributes of the polygons in the blocks to obtain a remote sensing sub-image set.

[0055] Specifically, the divided remote sensing image is subjected to block processing, that is, the remote sensing image is divided into a plurality of blocks. The blocks are screened based on the attributes of the polygons in the blocks, and the purpose is to screen out blocks with a large area of a certain category attribute, that is, blocks with a main category attribute. The blocks form a remote sensing sub-image set.

[0056] S103: Based on the similarity between each remote sensing sub-image in the remote sensing sub-image set and the other remote sensing sub-images, the positive samples and the negative samples of each remote sensing sub-image are obtained.

[0057] Specifically, in the remote sensing sub-image set, the similarity between each remote sensing sub-image and the other remote sensing sub-images is calculated, the remote sensing sub-images with similarity are classified as positive samples, and the remote sensing sub-images without similarity are classified as negative samples. The positive samples and the negative samples of a plurality of remote sensing sub-images form a positive and negative sample pair set of the remote sensing image.

[0058] S104: The initial remote sensing image feature extraction model is subjected to self-supervised training by using each remote sensing sub-image and the corresponding positive sample and negative sample, and a remote sensing image feature extraction model is obtained.

[0059] Specifically, the initial remote sensing image feature extraction model is trained by using each remote sensing sub-image and the corresponding positive sample and negative sample obtained by the above steps, and a remote sensing image feature extraction model is obtained after convergence. The remote sensing image feature extraction model can be used for feature extraction of the remote sensing image, and the regions in the remote sensing image can be classified based on the extracted features.

[0060] The remote sensing image self-supervised training method provided by the present application can quickly and effectively construct a positive and negative sample pair set by using target result data such as open source vector data of Open Street Map or historical result data accumulated by various industries. The positive and negative sample pair set can be quickly trained to obtain a converged remote sensing image feature extraction model, which provides support for subsequent remote sensing image classification.

[0061] In one embodiment, such as Figure 3 As shown, the remote sensing image is divided into regions based on the target result data, and the remote sensing image is divided into multiple patches, each patch corresponding to a category attribute, including:

[0062] S301: Obtain target result data based on the remote sensing image.

[0063] Specifically, based on the location of the region included in the remote sensing image, the target result data corresponding to that region is obtained. For example, if the remote sensing image contains region X of city A, then the result data of region X of city A is obtained as the target result data.

[0064] S302: Spatial overlap registration is performed between the target result data and the remote sensing image, thereby dividing the remote sensing image into multiple patches based on the target result data.

[0065] Specifically, since the target data and the remote sensing image correspond to the same area, spatially overlaying and registering the target data and the remote sensing image allows us to determine which block in the target data corresponds to each area in the remote sensing image. This enables the segmentation of the remote sensing image. For example, if the target data contains a block M, indicating that block M is school land and specifying its size and shape, mapping this block data to the remote sensing image creates a patch of the same size as block M, with the attribute of school land.

[0066] S303: Define the category attributes of each of the map patches based on the target result data.

[0067] Specifically, the target result data includes the type, size, and attributes of each block. After registration, the attributes of the blocks can be determined based on the target result data while dividing the blocks.

[0068] In one embodiment, such as Figure 4 As shown, the block-based processing of the remote sensing image after region division includes:

[0069] S401: Set the sliding window and the sliding step size, wherein the size of the sliding step size is less than or equal to the width of the sliding window in the sliding direction.

[0070] Specifically, since remote sensing images can be segmented by sliding a window with a set step size, the set step size must be less than or equal to the width of the sliding window in the sliding direction to ensure continuous segmentation. That is, the sliding window slides either with partial overlap or without overlap, depending on the step size. Figure 5 As shown,Figure 5 The middle section uses a sliding block method that involves sliding without overlapping or adjacency.

[0071] When designing the sliding step size and sliding window, they can be set according to the size of the remote sensing image so that the entire remote sensing image can be divided into blocks. Alternatively, as shown below... Figure 5 As shown, after the remote sensing image is divided into blocks with a specific sliding step size and sliding window, areas at the edge of the remote sensing image that are smaller than a window size are discarded.

[0072] S402: The remote sensing image after region division is processed by moving the sliding window from left to right and from top to bottom with the sliding step size.

[0073] Specifically, starting from the first pixel in the upper left corner of the remote sensing image, the sliding window is moved from left to right and from top to bottom to divide the remote sensing image into blocks using a set sliding step size. The length of the sliding step size can be an integer number of pixels.

[0074] In one embodiment, such as Figure 6 As shown, the step of filtering the blocks based on the attributes of the patches in the blocks to obtain a remote sensing sub-image set includes:

[0075] S601: Calculate the area ratio of each category attribute in each of the aforementioned blocks.

[0076] Specifically, a block may include multiple category attribute patch regions, such as Figure 5 The blocks to the left of block 'a' contain patches with more than one category attribute. Calculate the area ratio of patches of each category attribute in each block, that is, the proportion of the area of ​​each category attribute patch to the area of ​​the block.

[0077] S602: When the area ratio of each category attribute of the patch in the block is lower than a preset threshold, the block is filtered out.

[0078] Specifically, when the area ratio of each type of patch region in a block is lower than a preset threshold, the block is filtered out. The preset threshold can be set to 80%, 85%, 87%, 93%, etc., and the threshold should be at least no less than 70%.

[0079] S603: When the area ratio of a patch of a certain category attribute in the block is higher than the preset threshold, the block is retained to form a remote sensing sub-image set.

[0080] Specifically, if the area proportion of a certain type of patch region in a block exceeds a preset threshold, that block is retained. Through this filtering process, all the retained blocks form the remote sensing sub-image set N. i(i = 1, 2, 3,... n), wherein n is the number of reserved blocks.

[0081] In one embodiment, the positive samples and the negative samples of each remote sensing sub-image are obtained based on the similarity between each remote sensing sub-image and the remaining remote sensing sub-images in the remote sensing sub-image set, including: calculating the similarity between each remote sensing sub-image and the remaining remote sensing sub-images in the remote sensing image set, defining the remote sensing sub-images with similarity as positive samples, and defining the remote sensing sub-images without similarity as negative samples; wherein in each remote sensing sub-image, the same main category attribute is with similarity, and the different main category attribute is without similarity.

[0082] Specifically, the similarity between each remote sensing sub-image and the remaining remote sensing sub-images in the remote sensing sub-image set N i (i = 1, 2, 3,... n) is calculated. For example, for remote sensing sub-image N1, the remaining remote sensing sub-images N2~N n The main category attribute is the category attribute of the graphon with an area ratio greater than a preset threshold, and the category attribute of the graphon with an area ratio greater than a preset threshold in N1 is the same as that of N1. The category attribute of the graphon with an area ratio greater than a preset threshold is different from that of N1. As shown in FIG. 1B, for remote sensing sub-image a, remote sensing sub-images b, c, d, aa, bb, cc, and dd are positive samples of a, and remote sensing sub-images A, B, C, and D are negative samples of a. Figure 5

[0083] In one embodiment, as shown in FIG. 1C, the initial remote sensing image feature extraction model is trained using each remote sensing sub-image and the corresponding positive sample and negative sample to obtain a remote sensing image feature extraction model, including: Figure 7

[0084] S701: Establishing the initial remote sensing image feature extraction model.

[0085] S702: Training the initial remote sensing image feature extraction model using each remote sensing sub-image and the corresponding positive sample and negative sample.

[0086] S703: Establishing a loss function according to the positive sample and the negative sample, optimizing the loss function, updating the parameters of the initial remote sensing image feature extraction model until convergence, and obtaining a remote sensing image feature extraction model.

[0087] ​​Specifically, after obtaining the positive samples and the negative samples, the remote sensing sub-images and the corresponding positive samples and negative samples are used to train an initial remote sensing image feature extraction model, the parameters of the initial remote sensing image feature extraction model are updated until convergence, and a remote sensing image feature extraction model can be obtained. The remote sensing image feature extraction model obtained above can be directly used for feature extraction of a remote sensing image. For example, the remote sensing image feature extraction model can be a MoCo network model, a MoCo v2 network model or a SimCLR network model.

[0088] Reference will be made to the following Figure 8 The remote sensing image self-supervised training device provided by the present application is described below, and the remote sensing image self-supervised training device described below can be correspondingly referred to the remote sensing image self-supervised training method described above.

[0089] The remote sensing image self-supervised training device comprises a region division module 801, a screening module 802, a sample obtaining module 803 and a training module 804; the region division module 801 is used for dividing a remote sensing image into multiple patches based on target result data, and each patch corresponds to a category attribute; the screening module 802 is used for performing block processing on the remote sensing image after region division and screening the blocks based on the attributes of the patches in the blocks to obtain a remote sensing sub-image set; the sample obtaining module 803 is used for obtaining positive samples and negative samples of each remote sensing sub-image based on the similarity between each remote sensing sub-image in the remote sensing sub-image set and the remaining remote sensing sub-images; and the training module 804 is used for performing self-supervised training on an initial remote sensing image feature extraction model using each remote sensing sub-image and the corresponding positive samples and negative samples to obtain a remote sensing image feature extraction model.

[0090] The remote sensing image self-supervised training device provided by the present application can quickly and effectively construct a positive and negative sample pair set by using target result data such as open source vector data of Open Street Map or historical result data accumulated by various industries, and can quickly train a converged remote sensing image feature extraction model through the constructed positive and negative sample pair set, thereby providing support for subsequent remote sensing image classification.

[0091] In one embodiment, the region division module 801 comprises an acquisition submodule, a division submodule and a definition submodule; the acquisition submodule is used for obtaining target result data based on the remote sensing image; the division submodule is used for spatially overlapping and registering the target result data and the remote sensing image, so as to divide the remote sensing image into multiple patches according to the target result data; and the definition submodule is used for defining the category attributes of each patch according to the target result data.

[0092] In an embodiment, the screening module 802 comprises a setting sub-module and a blocking sub-module; the setting sub-module is configured to set a sliding window and a sliding step, the size of the sliding step being less than or equal to the width of the sliding window in the sliding direction; the blocking sub-module is configured to block process the region-divided remote sensing image by moving the sliding window in the sliding step from left to right and from top to bottom.

[0093] In an embodiment, the screening module 802 further comprises a calculating sub-module, a screening-out sub-module and a screening sub-module; the calculating sub-module is configured to calculate the area proportion of the map patch of each category attribute in each block; the screening-out sub-module is configured to screen out the block when the area proportion of the map patch of each category attribute in the block is lower than a preset threshold; and the screening sub-module is configured to retain the block to form a remote sensing sub-image set when the area proportion of the map patch of one category attribute in the block is higher than the preset threshold.

[0094] In an embodiment, the sample obtaining module 803 is specifically configured to: calculate the similarity between each remote sensing sub-image and the remaining remote sensing sub-images in the remote sensing image set, define the remote sensing sub-image with similarity as a positive sample, and define the remote sensing sub-image without similarity as a negative sample; wherein, in each remote sensing sub-image, the main category attributes being the same means having similarity, and the main category attributes being different means not having similarity.

[0095] In an embodiment, the training module 804 comprises an establishing sub-module, a training sub-module and an optimization updating sub-module. The establishing sub-module is configured to establish the initial remote sensing image feature extraction model. The training sub-module is configured to train the initial remote sensing image feature extraction model by using each remote sensing sub-image and the corresponding positive sample and negative sample. The optimization updating sub-module is configured to establish a loss function according to the positive sample and the negative sample, optimize the loss function, update the parameters of the initial remote sensing image feature extraction model until convergence, and obtain a remote sensing image feature extraction model.

[0096] Figure 9 An example of an entity structure diagram of an electronic device is shown in FIG. 1. Figure 9As shown, the electronic device can include a processor 910, a communications interface 920, a memory 930, and a communications bus 940, wherein the processor 910, the communications interface 920, and the memory 930 complete mutual communication through the communications bus 940. The processor 910 can invoke a logical instruction in the memory 930 to execute a remote sensing image self-supervised training method, which includes: dividing a remote sensing image into multiple patches based on target achievement data, each patch corresponding to a category attribute; performing block processing on the remote sensing image after the region division and screening the blocks based on the attributes of the patches in the blocks to obtain a remote sensing sub-image set; obtaining positive samples and negative samples of each remote sensing sub-image based on the similarity between each remote sensing sub-image in the remote sensing sub-image set and the remaining remote sensing sub-images; and training an initial remote sensing image feature extraction model using each remote sensing sub-image and the corresponding positive samples and negative samples to obtain a remote sensing image feature extraction model.

[0097] In addition, the logical instructions in the memory 930 described above can be implemented in the form of a software functional unit and sold or used as an independent product, which can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium, includes several instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various program code storage media.

[0098] In another aspect, the present application also provides a computer program product comprising a computer program, which can be stored on a non-transitory computer readable storage medium, and the computer program can be executed by a processor to enable a computer to perform the remote sensing image self-supervised training method provided by the above-mentioned methods, which comprises: dividing a remote sensing image into regions based on target result data, dividing the remote sensing image into a plurality of map patches, each of which corresponds to a category attribute; performing block processing on the remote sensing image after the region division and screening the blocks based on the attributes of the map patches in the blocks to obtain a set of remote sensing sub-images; obtaining positive samples and negative samples of each remote sensing sub-image in the set of remote sensing sub-images based on the similarity between each remote sensing sub-image and the remaining remote sensing sub-images; and training an initial remote sensing image feature extraction model using each remote sensing sub-image and the corresponding positive samples and negative samples to obtain a remote sensing image feature extraction model.

[0099] In yet another aspect, the present application also provides a non-transitory computer readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the remote sensing image self-supervised training method provided by the above-mentioned methods, which comprises: dividing a remote sensing image into regions based on target result data, dividing the remote sensing image into a plurality of map patches, each of which corresponds to a category attribute; performing block processing on the remote sensing image after the region division and screening the blocks based on the attributes of the map patches in the blocks to obtain a set of remote sensing sub-images; obtaining positive samples and negative samples of each remote sensing sub-image in the set of remote sensing sub-images based on the similarity between each remote sensing sub-image and the remaining remote sensing sub-images; and training an initial remote sensing image feature extraction model using each remote sensing sub-image and the corresponding positive samples and negative samples to obtain a remote sensing image feature extraction model.

[0100] The device embodiments described above are only schematic, wherein the units shown as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment scheme. Those skilled in the art can understand and implement it without creative labor.

[0101] Those skilled in the art can clearly understand the technical solutions of the various embodiments from the above description of the embodiments, and the various embodiments can be implemented by means of software with the necessary general hardware platforms, and of course, can also be implemented by hardware. Based on such understanding, the above technical solutions, essentially or in other words, the part of the prior art that makes a contribution, can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, and the like, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0102] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for some technical features therein; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for self-supervised training of remote sensing images, characterized in that, The method comprises the following steps: obtaining target achievement data based on a remote sensing image; spatially overlapping and registering the target achievement data and the remote sensing image, thereby dividing the remote sensing image into multiple image patches according to the target achievement data; defining the category attributes of each image patch according to the target achievement data; performing block processing on the remote sensing image after region division and screening the blocks based on the attributes of the image patches in the blocks to obtain a set of remote sensing sub-images; calculating the similarity between each remote sensing sub-image and the remaining remote sensing sub-images in the set of remote sensing sub-images, defining the remote sensing sub-images with similarity as positive samples, and defining the remote sensing sub-images without similarity as negative samples; wherein, in each remote sensing sub-image, the remote sensing sub-images with the same main category attribute have similarity, and the remote sensing sub-images with different main category attributes do not have similarity; performing self-supervised training on an initial remote sensing image feature extraction model using each remote sensing sub-image and the corresponding positive samples and negative samples to obtain a remote sensing image feature extraction model.

2. The remote sensing image self-supervised training method of claim 1, wherein, The block processing on the remote sensing image after region division comprises the following steps: setting a sliding window and a sliding step, wherein the size of the sliding step is less than or equal to the width of the sliding window in the sliding direction; performing block processing on the remote sensing image after region division by moving the sliding window in the sliding step from left to right and from top to bottom.

3. The remote sensing image self-supervised training method of claim 2, wherein, The screening of the blocks based on the attributes of the image patches in the blocks to obtain a set of remote sensing sub-images comprises the following steps: calculating the area proportion of the image patches of each category attribute in each block; when the area proportion of the image patches of each category attribute in the block is lower than a preset threshold, the block is excluded; when the area proportion of the image patches of one category attribute in the block is higher than the preset threshold, the block is retained to form a set of remote sensing sub-images.

4. The remote sensing image self-supervised training method of claim 1, wherein, The self-supervised training on an initial remote sensing image feature extraction model using each remote sensing sub-image and the corresponding positive samples and negative samples to obtain a remote sensing image feature extraction model comprises the following steps: establishing the initial remote sensing image feature extraction model; performing self-supervised training on the initial remote sensing image feature extraction model using each remote sensing sub-image and the corresponding positive samples and negative samples; establishing a loss function according to the positive samples and the negative samples, optimizing the loss function, updating the parameters of the initial remote sensing image feature extraction model until convergence, and obtaining a remote sensing image feature extraction model.

5. A remote sensing image self-supervised training apparatus, characterized in that, The method comprises the following steps: a region division module for obtaining target achievement data based on a remote sensing image; spatially overlapping and registering the target achievement data and the remote sensing image, thereby dividing the remote sensing image into multiple image patches according to the target achievement data; defining the category attributes of each image patch according to the target achievement data; a screening module for performing block processing on the remote sensing image after region division and screening the blocks based on the attributes of the image patches in the blocks to obtain a set of remote sensing sub-images; The sample obtaining module is configured to calculate the similarity between each remote sensing sub-image in the remote sensing image set and the rest remote sensing sub-images, define the remote sensing sub-image with the similarity as a positive sample, and define the remote sensing sub-image without the similarity as a negative sample; wherein, in each remote sensing sub-image, the remote sensing sub-images with the same main category attribute have the similarity, and the remote sensing sub-images with different main category attributes do not have the similarity; The training module is configured to perform self-supervised training on an initial remote sensing image feature extraction model by using each remote sensing sub-image and the corresponding positive sample and negative sample, to obtain a remote sensing image feature extraction model.

6. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the remote sensing image self-supervised training method according to any one of claims 1 to 4 when executing the program.

7. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the remote sensing image self-supervised training method according to any one of claims 1 to 4.

8. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the remote sensing image self-supervised training method according to any one of claims 1 to 4.

Images