A self-supervised learning network training method for remote sensing images and related devices

Abstract

The application discloses a self-supervised learning network training method for remote sensing images and related devices. Without labeled information, the self-supervised training improves the representation ability of a basic model for real texture and structure information of ground objects, can realize accurate representation of remote sensing images, effectively reduces the probability of scale mismatch, texture statistical difference and geometric topological structure fracture during transfer training compared with related technologies, has more robust feature learning ability for remote sensing images of cross-resolution, cross-region and cross-imaging conditions, and effectively improves the generalization ability of the basic model.

Description

Technical Field

[0001] This application relates to the field of remote sensing technology, specifically to a self-supervised learning network training method and related apparatus for remote sensing images. Background Technology

[0002] Remote sensing images refer to digital images or film photographs formed by receiving electromagnetic wave information reflected or radiated by ground objects (such as visible light, infrared light, radar, etc.). They have a wide coverage, complex sources, high cost of manual annotation, and significant differences in imaging conditions in different regions or at different times, making it difficult for traditional small-scale annotated training to achieve good generalization.

[0003] In related technologies, a basic image processing model is trained based on natural images. Then, the pre-trained basic model can be transferred to the processing of remote sensing images, thereby enabling the processing of remote sensing images without the need for the label information of the remote sensing images.

[0004] However, the aforementioned basic models are insufficient in representing remote sensing images. Summary of the Invention

[0005] In view of this, this application provides a self-supervised learning network training method and related apparatus for remote sensing images. Without labeled information, self-supervised training improves the ability of the base model to represent the real texture and structural information of ground objects, enabling accurate representation of remote sensing images. Compared with related technologies, it effectively reduces the probability of scale mismatch, texture statistical differences and geometric topological structure breaks during transfer training. It has a more robust feature learning ability for remote sensing images across resolutions, regions and imaging conditions, and effectively improves the generalization ability of the base model.

[0006] To solve the above problems, the technical solution provided in this application is as follows:

[0007] On the one hand, embodiments of this application provide a method for training a self-supervised learning network for remote sensing images, the method comprising:

[0008] Acquire multiple remote sensing image samples;

[0009] Image processing is performed on the plurality of remote sensing image samples. For the first remote sensing image sample among the plurality of remote sensing image samples, a first part of the region within the first remote sensing image sample is masked. The masked region within the first remote sensing image sample is replaced by a second part of the region within the second remote sensing image sample among the plurality of remote sensing image samples to obtain a target remote sensing image sample. The resolution of the first remote sensing image sample is different from the resolution of the second remote sensing image sample.

[0010] The target remote sensing image sample is used to extract features through an initial basic model to obtain a one-dimensional feature sequence.

[0011] The features in the one-dimensional feature sequence are organized according to their relative position information to obtain multiple local features;

[0012] Image reconstruction is performed based on the multiple local features to obtain a reconstructed image for the first remote sensing image sample;

[0013] Based on the error between the reconstructed image of the multiple remote sensing image samples and the corresponding remote sensing image samples, the parameters of the initial basic model are adjusted to obtain the basic model.

[0014] As one possible implementation, the method further includes:

[0015] Add location information to each of the plurality of local features, the location information being used to identify the spatial location of each local feature in the corresponding remote sensing image sample;

[0016] The process of image reconstruction based on the multiple local features to obtain a reconstructed image for the first remote sensing image sample includes:

[0017] Based on the multiple local features and the corresponding location information, image reconstruction is performed to obtain a reconstructed image for the first remote sensing image sample.

[0018] As one possible implementation, the adjustment of the parameters of the initial basic model based on the error between the reconstructed images corresponding to the plurality of remote sensing images and the corresponding remote sensing image samples to obtain the basic model includes:

[0019] The mean squared error loss is used to measure the error between the reconstructed image corresponding to each of the multiple remote sensing images and the same pixel position corresponding to the remote sensing image sample.

[0020] Based on the error, the parameters of the initial basic model are iteratively adjusted to obtain the basic model.

[0021] As one possible implementation, the iterative adjustment of the parameters of the initial basic model based on the error to obtain the basic model includes:

[0022] Based on the error, the parameters of the initial base model are iteratively adjusted using a learning rate strategy of linear preheating and cosine annealing to obtain the base model.

[0023] As one possible implementation, the method further includes:

[0024] The initial base model is constructed based on convolutional layers and Transformer layers.

[0025] In another aspect, this application provides a training device for a self-supervised learning network based on remote sensing imagery, the device comprising:

[0026] The acquisition unit is used to acquire multiple remote sensing image samples;

[0027] An image processing unit is used to perform image processing on the plurality of remote sensing image samples. For a first remote sensing image sample among the plurality of remote sensing image samples, a first part of the region within the first remote sensing image sample is masked, and the masked region within the first remote sensing image sample is replaced by a second part of the region within a second remote sensing image sample among the plurality of remote sensing image samples to obtain a target remote sensing image sample. The resolution of the first remote sensing image sample is different from the resolution of the second remote sensing image sample.

[0028] The feature extraction unit is used to extract features from the target remote sensing image sample through the initial basic model to obtain a one-dimensional feature sequence;

[0029] The feature recombination unit is used to organize the features in the one-dimensional feature sequence according to their relative position information to obtain multiple local features;

[0030] An image reconstruction unit is used to perform image reconstruction based on the multiple local features to obtain a reconstructed image for the first remote sensing image sample.

[0031] The training unit is used to adjust the parameters of the initial base model based on the error between the reconstructed image of the plurality of remote sensing image samples and the corresponding remote sensing image sample, so as to obtain the base model.

[0032] As one possible implementation, the device further includes an adding unit for:

[0033] Add location information to each of the plurality of local features, the location information being used to identify the spatial location of each local feature in the corresponding remote sensing image sample;

[0034] The image reconstruction unit is used to perform image reconstruction based on the multiple local features and the corresponding location information to obtain a reconstructed image for the first remote sensing image sample.

[0035] As one possible implementation, the training unit is used for:

[0036] The mean squared error loss is used to measure the error between the reconstructed image corresponding to each of the multiple remote sensing images and the same pixel position corresponding to the remote sensing image sample.

[0037] Based on the error, the parameters of the initial basic model are iteratively adjusted to obtain the basic model.

[0038] As one possible implementation, the training unit is used for:

[0039] Based on the error, the parameters of the initial base model are iteratively adjusted using a learning rate strategy of linear preheating and cosine annealing to obtain the base model.

[0040] As one possible implementation, the device further includes a construction unit for:

[0041] The initial base model is constructed based on convolutional layers and Transformer layers.

[0042] In another aspect, this application provides a computer device, which includes a processor and a memory:

[0043] The memory is used to store computer programs;

[0044] The processor is configured to execute the method described in any of the above-described embodiments according to the computer program.

[0045] In another aspect, this application provides a computer-readable storage medium for storing a computer program that, when executed by a computer device, implements the method described in any of the above-mentioned embodiments.

[0046] In another aspect, this application provides a computer program product including a computer program, which, when run on a computer device, causes the computer device to perform any of the methods described above.

[0047] As can be seen from the above technical solution, this solution first acquires multiple remote sensing image samples and then processes them. Specifically, a first region within a first remote sensing image sample is masked, and the masked region within the first remote sensing image sample is replaced by a second region within a second remote sensing image sample with a different resolution. This allows image information with different resolutions and ground feature characteristics to be embedded within the same target remote sensing image sample through image processing. Then, using an initial base model, features are extracted from multiple target remote sensing image samples with mixed resolutions, making the initial base model focus more on the texture and boundary information of the image, thus obtaining a one-dimensional feature sequence to represent the first remote sensing image sample. The features in the one-dimensional feature sequence are then organized according to their relative position information to obtain multiple local features, and their spatial relationships are represented using window-level features. Image reconstruction is then performed based on these multiple local features to obtain a reconstructed image of the first remote sensing image sample. Finally, based on the error between the reconstructed image of the multiple remote sensing image samples and the corresponding remote sensing image samples, the parameters of the initial base model are adjusted to improve the initial base model's ability to model the structural relationships of remote sensing images. Therefore, this application improves the ability of the base model to represent the real texture and structural information of ground objects through self-supervised training without annotation information, and can achieve accurate representation of remote sensing images. Compared with related technologies, it effectively reduces the probability of scale mismatch, texture statistical differences and geometric topological structure breaks during transfer training. It has a more robust feature learning ability for remote sensing images across resolutions, regions and imaging conditions, and effectively improves the generalization ability of the base model. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 A schematic diagram illustrating a self-supervised learning network training method for remote sensing images provided in this application embodiment;

[0050] Figure 2 A schematic diagram of a model structure for self-supervised training provided in an embodiment of this application;

[0051] Figure 3 This is a schematic diagram of a self-supervised learning network training device for remote sensing images, provided as an embodiment of this application. Detailed Implementation

[0052] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0053] As described in the background section, in related technologies, a basic model for image processing is trained based on natural images, and then the pre-trained basic model can be transferred to the processing of remote sensing images, thereby enabling the processing of remote sensing images without the need for the label information of the remote sensing images.

[0054] However, the basic model obtained by the above training method is unable to learn features from the real texture and structural information of remote sensing images. When directly transferred to remote sensing scenes, it is prone to problems such as scale mismatch, texture statistical differences and geometric topological structure breaks. As a result, the basic model has poor representation ability for remote sensing images across resolutions, regions and imaging conditions, resulting in insufficient robustness and generalizability of the basic model in complex real scenes.

[0055] To address the aforementioned issues, this application provides a self-supervised learning network training method for remote sensing images. In the absence of labeled information, self-supervised training improves the base model's ability to represent the true texture and structural information of ground features, enabling accurate representation of remote sensing images. Compared to related technologies, this method effectively reduces the probability of scale mismatch, texture statistical differences, and geometric topological structure breaks during transfer training. It also exhibits more robust feature learning capabilities for remote sensing images across resolutions, regions, and imaging conditions, effectively improving the generalization ability of the base model.

[0056] The solutions provided in this application relate to the field of remote sensing technology, and are specifically illustrated through the following embodiments.

[0057] See Figure 1 The diagram shown is a flowchart of a self-supervised learning network training method for remote sensing images provided in an embodiment of this application. In this embodiment, it can be executed by a computer device.

[0058] S101: Acquire multiple remote sensing image samples.

[0059] The remote sensing image samples in this application include remote sensing images of various resolutions acquired in different geographical regions. Remote sensing images from different geographical regions possess different ground feature characteristics, containing different textures and boundary information.

[0060] For example, multiple remote sensing image samples with different imaging conditions, different geographical regions, and different resolutions can be obtained from open source websites.

[0061] S102: Perform image processing on multiple remote sensing image samples. For the first remote sensing image sample among the multiple remote sensing image samples, perform masking processing on the first part of the region within the first remote sensing image sample, and replace the masked region within the first remote sensing image sample with the second part of the region within the second remote sensing image sample among the multiple remote sensing image samples to obtain the target remote sensing image sample.

[0062] The first part of the region refers to a portion of the first remote sensing image sample, such as 20%, 30%, or 50% of the total region of the first remote sensing image sample. This application does not impose specific limitations on this.

[0063] The second part of the region refers to a portion of the second remote sensing image sample, such as 20%, 30%, or 50% of the total region of the second remote sensing image sample. This application does not impose specific limitations on this.

[0064] The resolution of the first remote sensing image sample differs from that of the second remote sensing image sample. In other words, the target remote sensing image obtained after the replacement contains image information of different scales and different ground features.

[0065] One possible approach is to randomly flip multiple remote sensing image samples, then pair two remote sensing image samples of different resolutions. Randomly occlude the first and second remote sensing image samples in each pair, for example, by dividing both samples into four equal parts. The upper right and lower left regions of the first remote sensing image sample are used as the first part for masking, and the upper left and lower right regions of the second remote sensing image sample are used as the second part for masking. Then, the unoccluded regions in the second remote sensing image sample replace the unoccluded regions in the first remote sensing image sample to obtain the target remote sensing image sample. This process is repeated to obtain N / 2 target remote sensing image samples from N remote sensing image samples.

[0066] This allows the target remote sensing image samples to include mixed-resolution image information, enabling the model to simultaneously access information at different scales during training, thereby learning a consistent representation across resolutions.

[0067] S103: Extract features from multiple target remote sensing image samples using the initial basic model to obtain a one-dimensional feature sequence.

[0068] The initial base model refers to a general model for feature extraction from remote sensing images, which is part of a self-supervised learning network. For example, a general visual model with hundreds of millions of parameters can be built as the backbone network of the initial base model. Furthermore, the initial base model can be a model based on convolutional neural networks, a model based on Transformers, or a model based on a hybrid architecture, etc. This application does not impose specific limitations on this.

[0069] As one possible implementation, an initial base model is built based on convolutional layers and Transformer layers.

[0070] Convolutional layers can capture local dependency information of remote sensing images, while Transformers can capture long-range dependency information, thus helping to improve the initial base model's ability to capture local textures and model global dependencies of remote sensing images.

[0071] For example, multiple target remote sensing image samples are input into an initial base model to perform image block segmentation, then feature extraction is performed on each image block, and then the features of multiple image blocks are mapped into an image token sequence to obtain a one-dimensional feature sequence, which loses spatial information.

[0072] S104: Organize the features in the one-dimensional feature sequence according to their relative position information to obtain multiple local features.

[0073] The relative position information refers to the position of each pixel represented by each sequence in the one-dimensional feature sequence within the image. Local features are used to identify local information in remote sensing image samples and are represented in a window-level feature form, such as in a 2D or 3D multi-dimensional form. This application does not impose specific limitations on this.

[0074] To force the initial base model to focus on the true texture and structure of ground features rather than a single imaging condition, for each feature in the one-dimensional feature sequence, corresponding relative position information is assigned according to the pixel position. This information can be calculated based on positional encoding methods. This enables the initial base model to identify the spatial adjacency relationships of these features when processing local regions, which is beneficial for mining the structural continuity between adjacent pixels or features. Based on this spatial position information, a consistent spatial position prior can be formed, and the features can be rearranged according to local windows of a fixed size, so that the original one-dimensional feature representation is reorganized into spatially continuous window blocks.

[0075] Furthermore, each local feature includes multiple spatially adjacent features, and the window-level feature representation contains the location information of each feature. This enables the initial basic model to recognize remote sensing images across resolutions and ground features, as well as to model the local structure of remote sensing images, further enhancing its ability to model the continuity of linear structures such as roads and water systems. It also allows subsequent image reconstruction to continue processing under a consistent data format, ensuring consistency in data dimensionality and sequence between steps and achieving smooth workflow transitions.

[0076] S105: Reconstruct the image based on multiple local features to obtain a reconstructed image for the first remote sensing image sample.

[0077] One possible implementation is to obtain the features of the masked region in the first remote sensing image sample, then reorganize the local features belonging to the first remote sensing image according to the spatial location information corresponding to each feature, and at the same time organize the features of the masked region according to the spatial location information, and output the reconstructed image corresponding to the first remote sensing image sample through the decoder.

[0078] As another possible implementation, the masked region of the first remote sensing image is inferred based on the local features belonging to the first remote sensing image. Then, the spatial location information of each local feature and the spatial location information of the masked region are reorganized to obtain the reconstructed image corresponding to the first remote sensing image.

[0079] Therefore, by treating multiple remote sensing image samples as the first remote sensing image sample and performing image processing, corresponding reconstructed images can be obtained respectively.

[0080] S106: Based on the error between the reconstructed image and the corresponding remote sensing image sample from multiple remote sensing image samples, the parameters of the initial basic model are adjusted to obtain the basic model.

[0081] For example, after image processing of multiple remote sensing images, corresponding reconstructed images are obtained. Then, the error between each reconstructed image and its corresponding remote sensing image sample is measured. For example, the error is calculated by determining the loss function by comparing the error of pixels or the similarity of structures. Then, with the goal of reducing the error, the parameters of the initial network model are iteratively adjusted until the preset number of iterations is reached or the error is reduced to the preset difference condition. Then, the iteration is stopped, and the trained basic model is obtained.

[0082] Therefore, this technical solution first acquires multiple remote sensing image samples and then processes them. Specifically, a first region within a first remote sensing image sample is masked, and the masked region within the first remote sensing image sample is replaced by a second region within a second remote sensing image sample with a different resolution. This allows image information with different resolutions and ground feature characteristics to be embedded within the same target remote sensing image sample through image processing. Then, using an initial base model, features are extracted from multiple target remote sensing image samples with mixed resolutions, making the initial base model focus more on the texture and boundary information of the images, thus obtaining a one-dimensional feature sequence to represent the first remote sensing image sample. The features in the one-dimensional feature sequence are then organized according to their relative position information to obtain multiple local features, and their relative positional relationships are represented by window-level features. Image reconstruction is then performed based on these multiple local features to obtain a reconstructed image of the first remote sensing image sample. Finally, based on the error between the reconstructed image of the multiple remote sensing image samples and the corresponding remote sensing image samples, the parameters of the initial base model are adjusted to improve the initial base model's ability to model the structural relationships of remote sensing images. Therefore, this application improves the ability of the base model to represent the real texture and structural information of ground objects through self-supervised training without annotation information, and can achieve accurate representation of remote sensing images. Compared with related technologies, it effectively reduces the probability of scale mismatch, texture statistical differences and geometric topological structure breaks during transfer training. It has a more robust feature learning ability for remote sensing images across resolutions, regions and imaging conditions, and effectively improves the generalization ability of the base model.

[0083] As one possible implementation, this application embodiment also provides a method for reconstructing an image, specifically in the instants A1-A2. A2 is a specific implementation of S105, namely, a specific implementation of image reconstruction based on multiple local features to obtain a reconstructed image for the first remote sensing image sample.

[0084] A1: Add location information to each of the multiple local features.

[0085] A2: Based on multiple local features and corresponding location information, image reconstruction is performed to obtain a reconstructed image for the first remote sensing image sample.

[0086] Location information is used to identify the spatial location of each local feature in the corresponding remote sensing image sample.

[0087] For example, after reorganizing the original one-dimensional feature sequence into spatially continuous window blocks, i.e. local features, for each local feature, corresponding spatial location information is assigned, and this spatial location information is uniformly added to all features within that local feature. That is, the spatial location information at the feature granularity (one feature in the local feature corresponds to one spatial location information) is converted into spatial location information at the window granularity (all features in the local feature correspond to one spatial location information). Then, image reconstruction is performed using the absolute location information corresponding to the local features. That is, for the multiple local features corresponding to the first remote sensing image, which are organized in a windowed manner, the overall feature sequence is restored according to their spatial order in the original remote sensing image, and the reconstructed image of the first remote sensing image is obtained.

[0088] Therefore, using the local features of windowed organization as the basic unit and reconstructing the image according to the corresponding spatial location information can effectively accelerate the efficiency of image reconstruction.

[0089] This application provides a method for measuring the error between a reconstructed image and a corresponding remote sensing image sample. See B1-B2 for details.

[0090] B1: The mean squared error loss measures the error between the reconstructed image and the corresponding remote sensing image sample at the same pixel position, corresponding to multiple reconstructed images.

[0091] B2: Based on the error, the parameters of the initial basic model are iteratively adjusted to obtain the basic model.

[0092] The mean squared error loss refers to a loss function that measures the mean of the squared differences between the predicted or estimated value and the true value. In this embodiment, it refers to a loss function that measures the mean of the squared differences between the same pixel positions of multiple reconstructed images and corresponding remote sensing image samples. It can penalize both brightness deviation and structure deviation. This embodiment does not impose specific limitations on the implementation of the mean squared error loss. A smaller mean squared error indicates that the reconstructed image is more similar to the original remote sensing image, and also represents that the initial base network has a stronger ability to represent the features of the remote sensing image.

[0093] Gradient information can be generated through mean squared error loss, which can then be used to iteratively adjust the parameters of the initial base model.

[0094] For example, for each reconstructed image and its corresponding remote sensing image sample, the error between the same pixel position is measured by mean squared error loss. The errors obtained from multiple reconstructed images and multiple remote sensing image samples are summed and averaged to obtain the error between the same pixel position of multiple reconstructed images and multiple remote sensing image samples. Then, with the goal of reducing this error, the parameters of the initial base model are continuously adjusted during the iteration process. When the initial base model is inaccurate in some local areas, the error in that area will be amplified by square, thereby generating a larger gradient in backpropagation. This drives the network to focus on improving these difficult-to-reconstruct detailed areas, thus obtaining a well-trained base model.

[0095] Therefore, the mean squared error loss is not only used to measure the current reconstruction performance of the initial base model, but also guides the initial base model to be optimized step by step through the gradient feedback mechanism, so that it can learn image texture, structure and spatial patterns more accurately in subsequent iterations, and achieve more stable and effective self-supervised training.

[0096] This application provides a method for iteratively adjusting the parameters of an initial base model. Specifically, based on the error, the parameters of the initial base model are iteratively adjusted using a learning rate strategy of linear preheating and cosine annealing to obtain the base model.

[0097] The calculated error is used to generate gradients during backpropagation of the network, indicating how the parameters should be updated to further reduce the error in the next forward propagation. For example, the parameters of the initial base model are stabilized and updated through adaptive moment estimation and weight decay optimizer. Regular weight decay is applied to the image reconstruction module (to suppress overfitting and parameter inflation), while no weight decay is applied to the normalization and bias terms to maintain numerical stability. At the same time, linear warm-up and cosine annealing learning rate strategies are combined to make the initial update smooth and the later convergence refined in the low loss region. If necessary, gradient clipping is used to limit gradient spikes caused by abnormal batches, avoiding gradient oscillations and parameter drift common in unlabeled training, making the overall training process more stable and controllable in convergence.

[0098] Therefore, by quantifying the error through mean squared error loss and providing the gradient direction, the adaptive moment estimation and weight decay optimizer transform the gradient into a stable and regularized parameter update. Combined with linear warm-up and cosine annealing learning rate strategies, the training rhythm is controlled, so that the mean squared error loss value during the training process decreases continuously and controllably.

[0099] To more clearly describe this self-supervised learning network training method for remote sensing imagery, the overall network structure for implementing this method is illustrated below. See [link to documentation]. Figure 2The diagram shown is a schematic of a model structure for self-supervised training provided in an embodiment of this application, including an unlabeled influence masking mixing module, a base model, and an image reconstruction module.

[0100] The unlabeled image masking and mixing module pairs multiple remote sensing image samples and processes them to obtain multiple target remote sensing image samples with mixed resolution image information. These samples serve as input for self-supervised learning, enabling the self-supervised learning network to simultaneously analyze the texture and structural information of different images during reconstruction. This increases the difficulty of self-supervised training and enhances the robustness of the feature representation of the base model. Taking one sample pair as an example, the unlabeled image masking and mixing module randomly masks approximately 50% of the region of each remote sensing image in the sample pair and replaces the masked region with the visible region of another remote sensing image, resulting in a target remote sensing image sample with mixed resolution information. The base model extracts features from multiple target remote sensing image samples through the image encoder to obtain multiple one-dimensional feature sequences. The image reconstruction module merges the features (represented in the form of image tokens) based on the spatial location information of each feature in each one-dimensional feature sequence. Features belonging to the same remote sensing image sample are merged into the corresponding local features according to their spatial location information. Then, the image decoder reassembles the local features belonging to the same remote sensing image sample and the corresponding masked region, outputting the corresponding reconstructed image. Then, the error between the reconstructed image and the corresponding remote sensing image sample is measured by mean squared error loss. Based on this error, the parameters of the initial base model are iteratively trained using an adaptive moment estimation and weight decay optimizer, as well as a learning rate strategy of linear preheating and cosine annealing, to obtain the trained base model.

[0101] Therefore, the technical solution of this application has the following advantages:

[0102] (1) Combining hybrid occlusion and dual reconstruction for high efficiency and robustness: The hybrid strategy of "replacing occlusion blocks with visible blocks" avoids the inconsistency between training and inference caused by the use of placeholders; dual reconstruction learns the details and structure of two original images at the same time, balancing efficiency and accuracy (corresponding to the core advantages of "image modeling with hybrid occlusion").

[0103] (2) Cross-resolution consistent learning to alleviate scale mismatch: Multi-resolution data training enables the network to learn the same type of land cover representation at different scales, improving the generalization robustness across regions, weather conditions and different ground resolutions.

[0104] (3) Structure enhancement without manual annotation: pixel reconstruction and structure reconstruction are carried out in parallel, so that the model can take into account both fine-grained texture and global structure.

[0105] (4) Balancing parameters and training stability: Adaptive moment estimation and weight decay optimizer are used in conjunction with warm-up and annealing, which makes it easier to converge in large-scale training without labels, and the backbone can be basically frozen, requiring only a small number of adaptation modules to be trained. The parameters are economical and engineering-friendly.

[0106] Based on the above embodiments, this application provides a self-supervised learning network training device for remote sensing images, with reference to... Figure 3 The diagram shown is a schematic of a self-supervised learning network training device for remote sensing images provided in an embodiment of this application. The device 300 includes:

[0107] Acquisition unit 301 is used to acquire multiple remote sensing image samples;

[0108] Image processing unit 302 is used to perform image processing on the plurality of remote sensing image samples. For a first remote sensing image sample among the plurality of remote sensing image samples, a first part of the region within the first remote sensing image sample is masked, and the masked region within the first remote sensing image sample is replaced by a second part of the region within a second remote sensing image sample among the plurality of remote sensing image samples to obtain a target remote sensing image sample. The resolution of the first remote sensing image sample is different from the resolution of the second remote sensing image sample.

[0109] The feature extraction unit 303 is used to extract features from the target remote sensing image sample through an initial basic model to obtain a one-dimensional feature sequence;

[0110] The feature recombination unit 304 is used to organize the features in the one-dimensional feature sequence according to their relative position information to obtain multiple local features;

[0111] Image reconstruction unit 305 is used to perform image reconstruction based on the multiple local features to obtain a reconstructed image for the first remote sensing image sample.

[0112] Training unit 306 is used to adjust the parameters of the initial base model based on the error between the reconstructed image of the plurality of remote sensing image samples and the corresponding remote sensing image sample, so as to obtain the base model.

[0113] As one possible implementation, the device further includes an adding unit for:

[0114] Add location information to each of the plurality of local features, the location information being used to identify the spatial location of each local feature in the corresponding remote sensing image sample;

[0115] The image reconstruction unit is used to perform image reconstruction based on the multiple local features and the corresponding location information to obtain a reconstructed image for the first remote sensing image sample.

[0116] As one possible implementation, the training unit is used for:

[0117] The mean squared error loss is used to measure the error between the reconstructed image corresponding to each of the multiple remote sensing images and the same pixel position corresponding to the remote sensing image sample.

[0118] Based on the error, the parameters of the initial basic model are iteratively adjusted to obtain the basic model.

[0119] As one possible implementation, the training unit is used for:

[0120] Based on the error, the parameters of the initial base model are iteratively adjusted using a learning rate strategy of linear preheating and cosine annealing to obtain the base model.

[0121] As one possible implementation, the device further includes a construction unit for:

[0122] The initial base model is constructed based on convolutional layers and Transformer layers.

[0123] Based on the above embodiments, this application provides a computer device, which includes a processor and a memory:

[0124] The memory is used to store computer programs;

[0125] The processor is used to execute the above-described self-supervised learning network training method for remote sensing images according to the computer program.

[0126] Based on the above embodiments, this application provides a computer-readable storage medium for storing a computer program that, when executed by a computer device, implements the above-described self-supervised learning network training method for remote sensing images.

[0127] Based on the above embodiments, this application provides a computer program product including a computer program, which, when run on a computer device, causes the computer device to execute the above-described self-supervised learning network training method for remote sensing images.

[0128] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems or apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and relevant parts can be referred to the method section.

[0129] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for training self-supervised learning networks for remote sensing images, characterized in that, The method includes: Acquire multiple remote sensing image samples; Image processing is performed on the plurality of remote sensing image samples. For the first remote sensing image sample among the plurality of remote sensing image samples, a first part of the region within the first remote sensing image sample is masked. The masked region within the first remote sensing image sample is replaced by a second part of the region within the second remote sensing image sample among the plurality of remote sensing image samples to obtain a target remote sensing image sample. The resolution of the first remote sensing image sample is different from the resolution of the second remote sensing image sample. The target remote sensing image sample is used to extract features through an initial basic model to obtain a one-dimensional feature sequence. The features in the one-dimensional feature sequence are organized according to their relative position information to obtain multiple local features; Image reconstruction is performed based on the multiple local features to obtain a reconstructed image for the first remote sensing image sample; Based on the error between the reconstructed image of the multiple remote sensing image samples and the corresponding remote sensing image samples, the parameters of the initial basic model are adjusted to obtain the basic model.

2. The method according to claim 1, characterized in that, The method further includes: Add location information to each of the plurality of local features, the location information being used to identify the spatial location of each local feature in the corresponding remote sensing image sample; The process of image reconstruction based on the multiple local features to obtain a reconstructed image for the first remote sensing image sample includes: Based on the multiple local features and the corresponding location information, image reconstruction is performed to obtain a reconstructed image for the first remote sensing image sample.

3. The method according to claim 1, characterized in that, The parameters of the initial basic model are adjusted based on the error between the reconstructed images corresponding to the multiple remote sensing images and the corresponding remote sensing image samples, resulting in a basic model including: The mean squared error loss is used to measure the error between the reconstructed image corresponding to each of the multiple remote sensing images and the same pixel position corresponding to the remote sensing image sample. Based on the error, the parameters of the initial basic model are iteratively adjusted to obtain the basic model.

4. The method according to claim 1, characterized in that, The step of iteratively adjusting the parameters of the initial basic model based on the error to obtain the basic model includes: Based on the error, the parameters of the initial base model are iteratively adjusted using a learning rate strategy of linear preheating and cosine annealing to obtain the base model.

5. The method according to claim 1, characterized in that, The method further includes: The initial base model is constructed based on convolutional layers and Transformer layers.

6. A self-supervised learning network training device for remote sensing images, characterized in that, The device includes: The acquisition unit is used to acquire multiple remote sensing image samples; An image processing unit is used to perform image processing on the plurality of remote sensing image samples. For a first remote sensing image sample among the plurality of remote sensing image samples, a first part of the region within the first remote sensing image sample is masked, and the masked region within the first remote sensing image sample is replaced by a second part of the region within a second remote sensing image sample among the plurality of remote sensing image samples to obtain a target remote sensing image sample. The resolution of the first remote sensing image sample is different from the resolution of the second remote sensing image sample. The feature extraction unit is used to extract features from the target remote sensing image sample through the initial basic model to obtain a one-dimensional feature sequence; The feature recombination unit is used to organize the features in the one-dimensional feature sequence according to their relative position information to obtain multiple local features; An image reconstruction unit is used to perform image reconstruction based on the multiple local features to obtain a reconstructed image for the first remote sensing image sample. The training unit is used to adjust the parameters of the initial base model based on the error between the reconstructed image of the plurality of remote sensing image samples and the corresponding remote sensing image sample, so as to obtain the base model.

7. The apparatus according to claim 6, characterized in that, The device further includes an adding unit for: Add location information to each of the plurality of local features, the location information being used to identify the spatial location of each local feature in the corresponding remote sensing image sample; The image reconstruction unit is used to perform image reconstruction based on the multiple local features and the corresponding location information to obtain a reconstructed image for the first remote sensing image sample.

8. A computer device, characterized in that, The computer device includes a processor and memory: The memory is used to store computer programs; The processor is configured to perform the method according to any one of claims 1-5 according to the computer program.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program that, when executed by a computer device, performs the method described in any one of claims 1-5.

10. A computer program product comprising a computer program, characterized in that, When it is run on a computer device, it causes the computer device to perform the method according to any one of claims 1-5.

Images