Remote sensing image pixel level interpretation method and device based on attention mechanism

Abstract

The invention provides a remote sensing image pixel level interpretation method and device based on an attention mechanism, and the method comprises the steps: obtaining a remote sensing image; the remote sensing image is input into a pre-trained remote sensing image interpretation model, a classification result image is obtained, and the pre-trained remote sensing image interpretation model interprets the remote sensing image through the following steps that the remote sensing image is cut into a plurality of cut images with the same size, embedding processing is carried out, and a classification result is obtained; obtaining an input sequence; the embedded input sequence is input into a transformer encoder based on an attention mechanism, and encoding is carried out; and adjusting a result output by the transformer encoder to a dimension conforming to the output, and performing up-sampling to obtain a classification result image. The ability of the model to capture global dependence is improved, an encoder based on an attention mechanism is applied, and the interpretation precision is further improved on the basis of the convolutional neural network.

Description

Technical Field

[0001] The present application relates to the technical field of remote sensing image recognition, and more specifically, to a remote sensing image pixel-level interpretation method and device based on an attention mechanism. Background Art

[0002] In the past decade, my country has made breakthrough progress in the construction of earth observation infrastructure, launching a series of remote sensing satellites such as Resources and Gaofen, which have greatly improved the ability to investigate and monitor all elements, full coverage, and all-weather natural resources such as mountains, water, forests, fields, lakes, and grasses. Remote sensing data plays an increasingly important role due to its significant advantages such as objectivity, accuracy, comprehensiveness, and timeliness, helping government management departments to understand the changes in the quantity, quality, and ecology of natural resources in a timely and accurate manner, and providing reliable data support for natural resource monitoring, land space planning, and ecological and environmental protection.

[0003] In the field of computer vision, convolutional neural networks have achieved great success. Since convolutional neural networks have a weight sharing mechanism, the features extracted by the convolution layer are translation invariant. Therefore, the feature extraction method based on convolutional neural networks can be widely used in image classification, target detection and semantic segmentation. In the field of natural resources, remote sensing images can be interpreted for surface coverage elements and identify ground objects. However, due to the nature of the convolution operator, the image only focuses on the features within the "receptive field". Therefore, the convolution feature map has local sensitivity, that is, each convolution operation will only consider a small part of the local information of the original data. Summary of the invention

[0004] In view of this, the purpose of this application is to provide a remote sensing image pixel-level interpretation method and device based on an attention mechanism, which can interpret remote sensing images through a trained remote sensing image interpretation model, solve the problem that convolution calculations in the prior art only consider a small part of the local information of the original data, improve the model's ability to capture global dependencies, and apply an encoder based on the attention mechanism to further improve the interpretation accuracy based on the convolutional neural network.

[0005] In a first aspect, an embodiment of the present application provides a remote sensing image pixel-level interpretation method based on an attention mechanism, the method comprising: acquiring a remote sensing image; inputting the remote sensing image into a pre-trained remote sensing image interpretation model to obtain a classification result image, wherein the pre-trained remote sensing image interpretation model interprets the remote sensing image through the following steps: cropping the remote sensing image into multiple cropped images of the same size, and embedding them to obtain an input sequence; inputting the embedded input sequence into a transformer encoder based on an attention mechanism for encoding; adjusting the result output by the transformer encoder to match the output dimension, and upsampling to obtain a classification result image.

[0006] Optionally, the remote sensing image interpretation model is trained by the following steps: constructing a remote sensing image interpretation sample library, the remote sensing image interpretation sample library comprising multiple remote sensing images and mask label images corresponding to multiple target remote sensing images; inputting the multiple remote sensing images into a transformer encoder based on an attention mechanism for encoding; for each remote sensing image, adjusting the result output by the transformer encoder to a dimension that conforms to the output, and performing upsampling to obtain a training image; for each training image, comparing the training image with the mask label image corresponding to the training image, and calculating the error value based on a cross entropy loss algorithm;

[0007] Based on the stochastic gradient descent optimization algorithm, the remote sensing image interpretation model is trained according to the error value, and when the error value reaches a preset error range or the number of training steps reaches a target number of steps, a trained remote sensing image interpretation model is obtained.

[0008] Optionally, a remote sensing image interpretation sample library is constructed by the following steps: acquiring multiple remote sensing images; performing mask annotation on the multiple remote sensing images to obtain mask label images corresponding to multiple target remote sensing images; performing normalization calculation on the multiple remote sensing images, and performing image processing on the multiple mask label images to obtain cropped mask label images corresponding to the multiple remote sensing images; and constructing a remote sensing image interpretation sample library based on the multiple remote sensing images and the multiple cropped mask label images.

[0009] Optionally, embedding processing is performed on multiple sub-remote sensing images corresponding to each remote sensing image through the following steps: determining sequence data based on the multiple sub-remote sensing images, wherein the multiple sub-remote sensing images are obtained by cropping the remote sensing image; performing linear transformation on the sequence data to obtain first target data, wherein the first target data includes sub-first target data corresponding to each sub-remote sensing image; splicing a randomly generated category label on each sub-first target data to obtain multiple embedded data; determining a position code for each embedded data based on the number of the multiple sub-remote sensing images and a preset dimension, wherein the position code is used to characterize the input order of the multiple sub-remote sensing images; determining the input sequence of each remote sensing image based on the multiple embedded data and the position code of each embedded data.

[0010] Optionally, the embedded input sequence is input into a transformer encoder based on an attention mechanism, and the encoding step includes: inputting the input sequence of the remote sensing image into a transformer encoder based on an attention mechanism, performing a preset number of encoding calculations based on the attention mechanism, and obtaining the encoding result of the remote sensing image, wherein an encoding calculation is performed based on the attention mechanism through the following steps: normalizing the input sequence of the remote sensing image, and based on the multi-head attention mechanism, concatenating the result of the normalization calculation and the input sequence of the remote sensing image to obtain a second intermediate sequence; performing normalization calculation and fully connected layer calculation on the first intermediate sequence to obtain a first intermediate sequence.

[0011] Optionally, the step of adjusting the result output by the transformer encoder to match the output dimension and upsampling to obtain a classification result image includes: performing a reshape calculation on the encoding result of the remote sensing image to obtain an encoding result of a target dimension; performing two convolution calculations on the encoding result of the target dimension to obtain a first intermediate code, wherein the number of output channels of the first intermediate code is the same as the number of categories of pixel classification; and using a bilinear interpolation algorithm to upsample the image to obtain a classification result image of the same size as the remote sensing image.

[0012] In a second aspect, the embodiment of the present application further provides a remote sensing image pixel-level interpretation device based on an attention mechanism, the device comprising:

[0013] Remote sensing image acquisition module, used to acquire remote sensing images;

[0014] The remote sensing image interpretation module is used to input the remote sensing image into a pre-trained remote sensing image interpretation model to obtain a classification result image, wherein the remote sensing image interpretation module is specifically used to crop the remote sensing image into multiple cropped images of the same size, and perform embedding processing to obtain an input sequence; input the embedded input sequence into a transformer encoder based on an attention mechanism for encoding; adjust the result output by the transformer encoder to match the output dimension, and perform upsampling to obtain a classification result image.

[0015] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps of the remote sensing image pixel-level interpretation method based on the attention mechanism are performed as described above.

[0016] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above-mentioned remote sensing image pixel-level interpretation method based on the attention mechanism are executed.

[0017] The attention-based remote sensing image pixel-level interpretation method and device provided in the embodiments of the present application can interpret remote sensing images through a trained remote sensing image interpretation model, thereby solving the problem in the prior art that convolution calculations only consider a small part of the local information of the original data, improving the model's ability to capture global dependencies, and applying an encoder based on the attention mechanism to further improve the interpretation accuracy based on the convolutional neural network.

[0018] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 A flowchart of a remote sensing image pixel-level interpretation method based on an attention mechanism provided in an embodiment of the present application;

[0021] Figure 2A schematic diagram of the network architecture of the remote sensing image interpretation model provided in the embodiment of the present application;

[0022] Figure 3 A visualization of the image provided in the embodiments of the present application and the pixel-level interpretation results of the remote sensing image based on the attention mechanism.

[0023] Figure 4 A schematic diagram of the structure of a remote sensing image pixel-level interpretation device based on an attention mechanism provided in an embodiment of the present application;

[0024] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application usually described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, each other embodiment obtained by those skilled in the art without making creative work belongs to the scope of protection of the present application.

[0026] First, the application scenarios to which this application is applicable are introduced. This application can be applied to the field of remote sensing image recognition technology.

[0027] Research has found that in the past decade, my country has made breakthrough progress in the construction of earth observation infrastructure, launched a series of remote sensing satellites such as Resources and High Resolution, and greatly improved the full-element, full-coverage, all-weather survey and monitoring capabilities of natural resources such as mountains, water, forests, fields, lakes, and grass. Remote sensing data plays an increasingly important role due to its significant advantages such as objectivity, accuracy, comprehensiveness, and timeliness, helping government management departments to timely and accurately understand the changes in the quantity, quality, and ecology of natural resources, and providing reliable data support for natural resource monitoring, land space planning, and ecological and environmental protection.

[0028] In the field of computer vision, convolutional neural networks have achieved great success. Since convolutional neural networks have a weight sharing mechanism, the features extracted by the convolution layer are translation invariant. Therefore, the feature extraction method based on convolutional neural networks can be widely used in image classification, target detection and semantic segmentation. In the field of natural resources, remote sensing images can be interpreted for surface coverage elements and identify ground objects. However, due to the nature of the convolution operator, the image only focuses on the features within the "receptive field". Therefore, the convolution feature map has local sensitivity, that is, each convolution operation will only consider a small part of the local information of the original data.

[0029] Based on this, the embodiments of the present application provide a remote sensing image pixel-level interpretation method and device based on an attention mechanism, which can interpret remote sensing images through a trained remote sensing image interpretation model, solve the problem that the convolution calculation in the prior art only considers a small part of the local information of the original data, improves the model's ability to capture global dependencies, and applies an encoder based on the attention mechanism to further improve the interpretation accuracy based on the convolutional neural network.

[0030] See also Figure 1 , Figure 1 Flow chart of a remote sensing image pixel-level interpretation method based on attention mechanism provided in an embodiment of the present application. Figure 1 As shown in , the remote sensing image pixel-level interpretation method based on the attention mechanism provided in the embodiment of the present application includes:

[0031] S101. Acquire remote sensing images.

[0032] For example, see Figure 4 , Figure 4 This is a visualization of the image provided in the embodiment of the present application and the pixel-level interpretation result of the remote sensing image based on the attention mechanism. Figure 4 As shown in Figure 4 Figure a in the figure is the remote sensing image obtained in this application.

[0033] S102: input the remote sensing image into a pre-trained remote sensing image interpretation model to obtain a classification result image.

[0034] Specifically, the pre-trained remote sensing image interpretation model interprets the remote sensing image through the following steps: cropping the remote sensing image into multiple cropped images of the same size, and embedding them to obtain an input sequence; inputting the embedded input sequence into a transformer encoder based on an attention mechanism for encoding; adjusting the result output by the transformer encoder to a dimension that conforms to the output, and upsampling it to obtain a classified result image.

[0035] Among them, the remote sensing image interpretation model can be trained through the following steps: constructing a remote sensing image interpretation sample library, the remote sensing image interpretation sample library includes multiple remote sensing images and mask label images corresponding to multiple target remote sensing images; inputting the multiple remote sensing images into a transformer encoder based on an attention mechanism for encoding; for each remote sensing image, adjusting the result output by the transformer encoder to a dimension that conforms to the output, and upsampling to obtain a training image; for each training image, comparing the training image with the mask label image corresponding to the training image, and calculating the error value based on the cross entropy loss algorithm; based on the stochastic gradient descent optimization algorithm, training the remote sensing image interpretation model according to the error value, and obtaining the trained remote sensing image interpretation model when the error value reaches a preset error range or the number of training steps reaches the target number of steps.

[0036] For example, the constructed model can be used for training, the output data of the model can be compared with the mask label data, and the difference between the two can be evaluated by cross entropy loss. The stochastic gradient descent algorithm (SGD) can be used to minimize the loss. When the training reaches a preset number of times or the loss value is reduced to a specified threshold, the training is stopped, and the obtained model is the trained remote sensing image interpretation model.

[0037] Among them, a remote sensing image interpretation sample library can be constructed through the following steps: obtaining multiple remote sensing images; performing mask annotation on the multiple remote sensing images to obtain mask label images corresponding to the multiple target remote sensing images; performing normalization calculation on the multiple remote sensing images, and performing image processing on the multiple mask label images to obtain cropped mask label images corresponding to the multiple remote sensing images; based on the multiple remote sensing images and the multiple cropped mask label images, a remote sensing image interpretation sample library is constructed.

[0038] Specifically, the multiple sub-remote sensing images corresponding to each remote sensing image can be embedded through the following steps: determine sequence data based on the multiple sub-remote sensing images, and the multiple sub-remote sensing images are obtained by cropping the remote sensing image; perform linear transformation on the sequence data to obtain first target data, and the first target data includes sub-first target data corresponding to each sub-remote sensing image; splice a randomly generated category label on each sub-first target data to obtain multiple embedded data; determine the position code of each embedded data based on the number of multiple sub-remote sensing images and a preset dimension, and the position code is used to characterize the input order of multiple sub-remote sensing images; determine the input sequence of each remote sensing image based on the multiple embedded data and the position code of each embedded data.

[0039] Among them, the embedded input sequence is input into the transformer encoder based on the attention mechanism, and the encoding step includes: inputting the input sequence of the remote sensing image into the transformer encoder based on the attention mechanism, performing a preset number of encoding calculations based on the attention mechanism, and obtaining the encoding result of the remote sensing image.

[0040] Among them, a coding calculation can be performed based on the attention mechanism through the following steps: normalizing the input sequence of the remote sensing image, and based on the multi-head attention mechanism, concatenating the result of the normalization calculation and the input sequence of the remote sensing image to obtain a second intermediate sequence; normalizing and fully connected layer calculating the first intermediate sequence to obtain a first intermediate sequence.

[0041] Specifically, the steps of adjusting the result output by the transformer encoder to conform to the output dimension and performing upsampling to obtain a classification result image include: performing a reshape calculation on the encoding result of the remote sensing image to obtain an encoding result of a target dimension; performing two convolution calculations on the encoding result of the target dimension to obtain a first intermediate code, wherein the number of output channels of the first intermediate code is the same as the number of categories of pixel classification; and performing an upsampling calculation on the image using a bilinear interpolation algorithm to obtain a classification result image of the same size as the remote sensing image.

[0042] For example, see Figure 3 , Figure 3 This is a visualization of the image provided in the embodiment of the present application and the pixel-level interpretation result of the remote sensing image based on the attention mechanism. Figure 3 As shown in Figure 3 Figure a in the figure is a remote sensing image. Figure 3 Figure b in the figure is the target remote sensing image with mask labels.

[0043] For example, taking the southern Jiangsu region as an example, please refer to Figure 3 , Figure 3 Figure a in the figure is the southern Jiangsu region, which is a satellite remote sensing image with sufficient representativeness and diversity.

[0044] like Figure 3 As shown in Figure a, the southern Jiangsu region contains various surface conditions such as buildings, roads, cultivated land, woodland, gardens, grasslands, and water systems. The image data has a resolution of 0.8 meters and clear imaging. It was taken between 2020 and 2021. Mask annotation: 0-background, 1-cultivated land, 2-orchard, 3-woodland, 4-turf, 5-water system, 6-road, 7-building.

[0045] In this way, the satellite remote sensing images and mask data of the southern Jiangsu region can be used to build an image interpretation sample library for training.

[0046] For example, see Figure 2 , Figure 2 This is a schematic diagram of the network architecture of the remote sensing image interpretation model provided in the embodiment of the present application. Figure 2 As shown in , first, for the 512×512 input image, it is randomly cropped to 256×256 size, and the batch size is 8 during training.

[0047] Secondly, Python can be used to write data enhancement algorithms to implement operations such as horizontal flipping, upside down flipping, and rotation with a probability of 0.5, and complete the normalization of the input image, thereby forming a remote sensing image interpretation sample library.

[0048] Next, the input remote sensing image needs to be divided into blocks based on the remote sensing image interpretation model, and these small blocks are merged together for linear transformation. The result is then added with a randomly generated class token on dimension 1 to obtain embedding data. Then, according to the number of blocks of the image and the dimension dim preset by the model, a random tensor with a dimension of (1, number of blocks + 1, dim) is generated, called position embedding, which is used to represent the order information of the input data. Then the embedding and position embedding data are added together to obtain the final input sequence.

[0049] Thirdly, it is necessary to encode the image based on the remote sensing image interpretation model. The core of encoding lies in the multi-head attention mechanism. The role of the multi-head attention mechanism is that when given the same set of queries, keys and values, the multi-head attention mechanism can learn different behaviors based on the same attention mechanism, and then combine the different behaviors as knowledge to capture various ranges of dependencies in the sequence, similar to learning different features. The result of embedding+positionembedding is used as the final input data and input into the standard transformer encoding module (i.e., target encoder). In the transformer encoding module, the input sequence is first layer normalized. According to the preset number of heads H, the input is first linearly transformed independently to obtain H groups of outputs (including queries, keys and values), and then the self-attention is calculated separately. The H groups of self-attention outputs are spliced ​​together, and then a linear transformation is performed to obtain the output of the multi-head self-attention. The output of the multi-head attention mechanism module is layer normalized and fully connected again to complete an encoding operation. Such an encoding operation can be repeated L times according to the preset parameter L to obtain the encoding result.

[0050] Thirdly, the encoded result needs to be decoded based on the remote sensing image interpretation model, and the encoded result needs to be reshaped to be adjusted into a tensor in the format of [batch size, number of channels, image width, image height], and the image needs to be convolved twice in succession so that the number of output channels is equal to the number of pixel classification categories. Then the output data needs to be upsampled and enlarged to the size of the original image to obtain the pixel classification result (i.e. the target remote sensing image with mask labels).

[0051] The remote sensing image pixel-level interpretation method based on the attention mechanism provided in the embodiment of the present application can interpret the remote sensing image through a trained remote sensing image interpretation model, solve the problem that the convolution calculation in the prior art only considers a small part of the local information of the original data, improves the model's ability to capture global dependencies, and applies an encoder based on the attention mechanism to further improve the interpretation accuracy based on the convolutional neural network.

[0052] Based on the same inventive concept, the embodiments of the present application also provide a remote sensing image pixel-level interpretation device based on an attention mechanism corresponding to the remote sensing image pixel-level interpretation method based on an attention mechanism. Since the principle of solving the problem by the device in the embodiments of the present application is similar to the above-mentioned remote sensing image pixel-level interpretation method based on the attention mechanism in the embodiments of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0053] See also Figure 4 , Figure 4 The schematic diagram of the structure of a remote sensing image pixel-level interpretation device based on the attention mechanism provided in the embodiment of the present application is shown in FIG. Figure 4 As shown in , the remote sensing image pixel-level interpretation device 400 based on the attention mechanism includes:

[0054] A remote sensing image acquisition module 401 is used to acquire remote sensing images;

[0055] The remote sensing image interpretation module 402 is used to input the remote sensing image into a pre-trained remote sensing image interpretation model to obtain a classification result image.

[0056] Among them, the remote sensing image interpretation module 402 is specifically used to crop the remote sensing image into multiple cropped images of the same size, and embed them to obtain an input sequence; input the embedded input sequence into a transformer encoder based on an attention mechanism for encoding; adjust the result output by the transformer encoder to match the output dimension, and upsample it to obtain a classified result image.

[0057] The remote sensing image pixel-level interpretation device based on the attention mechanism provided in the embodiment of the present application can interpret remote sensing images through a trained remote sensing image interpretation model, solve the problem that convolution calculations in the prior art only consider a small part of the local information of the original data, improve the model's ability to capture global dependencies, and apply an encoder based on the attention mechanism to further improve the interpretation accuracy based on the convolutional neural network.

[0058] See also Figure 5 , Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 5 As shown in , the electronic device 500 includes a processor 510 , a memory 520 and a bus 530 .

[0059] The memory 520 stores machine-readable instructions executable by the processor 510. When the electronic device 500 is running, the processor 510 communicates with the memory 520 via the bus 530. When the machine-readable instructions are executed by the processor 510, the above-mentioned Figure 1 The steps of the remote sensing image pixel-level interpretation method based on the attention mechanism in the method embodiment shown, the specific implementation method can be found in the method embodiment, and will not be repeated here.

[0060] The present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the computer program can execute the above-mentioned Figure 1 The steps of the remote sensing image pixel-level interpretation method based on the attention mechanism in the method embodiment shown, the specific implementation method can be found in the method embodiment, and will not be repeated here.

[0061] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0062] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0063] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0064] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0065] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application can essentially be embodied in the form of a software product, or in other words, the part that contributes to the prior art or the part of the technical solution. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0066] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A remote sensing image pixel-level interpretation method based on attention mechanism, characterized in that: The method comprises: Acquisition of remote sensing images; The remote sensing image is input into a pre-trained remote sensing image interpretation model to obtain a classification result image. The pre-trained remote sensing image interpretation model interprets the remote sensing image through the following steps: The remote sensing image is cropped into a plurality of cropped images of the same size, and embedded into the cropped images to obtain an input sequence; The embedded input sequence is input into the attention-based transformer encoder for encoding; The output of the transformer encoder is adjusted to match the output dimension and upsampled to obtain the classification result image.

2. The method according to claim 1, characterized in that: The remote sensing image interpretation model is trained by the following steps: Constructing a remote sensing image interpretation sample library, wherein the remote sensing image interpretation sample library includes a plurality of remote sensing images and mask label images corresponding to a plurality of target remote sensing images; Inputting the multiple remote sensing images into a transformer encoder based on an attention mechanism for encoding; For each remote sensing image, the output of the transformer encoder is adjusted to match the output dimension, and up-sampled to obtain a training image; For each training image, compare the training image with the mask label image corresponding to the training image, and calculate the error value based on the cross entropy loss algorithm; Based on the stochastic gradient descent optimization algorithm, the remote sensing image interpretation model is trained according to the error value, and when the error value reaches a preset error range or the number of training steps reaches a target number of steps, a trained remote sensing image interpretation model is obtained.

3. The method according to claim 2, characterized in that The remote sensing image interpretation sample library is constructed through the following steps: Acquire multiple remote sensing images; Perform mask annotation on multiple remote sensing images to obtain mask label images corresponding to multiple target remote sensing images; Performing normalization calculation on the multiple remote sensing images and performing image processing on the multiple mask label images to obtain cropped mask label images corresponding to the multiple remote sensing images; Based on multiple remote sensing images and multiple cropped mask label images, a remote sensing image interpretation sample library is constructed.

4. The method according to claim 1, characterized in that Embed multiple sub-remote sensing images corresponding to each remote sensing image through the following steps: Determining sequence data according to the plurality of remote sensing sub-images, wherein the plurality of remote sensing sub-images are obtained by cropping the remote sensing image; Performing a linear transformation on the sequence data to obtain first target data, wherein the first target data includes sub-first target data corresponding to each sub-remote sensing image; Splice a randomly generated category label on each sub-first target data to obtain multiple embedded data Determining, according to the number of the plurality of sub-remote sensing images and a preset dimension, a position code for each embedded data, wherein the position code is used to characterize an input order of the plurality of sub-remote sensing images; An input sequence of each remote sensing image is determined according to the plurality of embedded data and the position code of each embedded data.

5. The method according to claim 4, characterized in that The embedded input sequence is input into the attention-based transformer encoder. The encoding steps include: The input sequence of the remote sensing image is input into the transformer encoder based on the attention mechanism. Based on the attention mechanism, a preset number of encoding calculations are performed to obtain the encoding result of the remote sensing image. Among them, an encoding calculation is performed based on the attention mechanism through the following steps: Performing normalization calculation on the input sequence of the remote sensing image, and based on the multi-head attention mechanism, concatenating the result obtained by the normalization calculation with the input sequence of the remote sensing image to obtain a second intermediate sequence; Normalization calculation and full connection layer calculation are performed on the first intermediate sequence to obtain a first intermediate sequence.

6. The method according to claim 5, characterized in that The steps of adjusting the output of the transformer encoder to conform to the output dimension and up-sampling to obtain the classification result image include: Perform reshape calculation on the encoding result of the remote sensing image to obtain the encoding result of the target dimension; Performing two convolution calculations on the encoding result of the target dimension to obtain a first intermediate code, wherein the number of output channels of the first intermediate code is the same as the number of categories of pixel classification; The bilinear interpolation algorithm is used to upsample the image to obtain a classification result image with the same size as the remote sensing image.

7. A remote sensing image pixel-level interpretation device based on attention mechanism, characterized in that: The device comprises: Remote sensing image acquisition module, used to acquire remote sensing images; The remote sensing image interpretation module is used to input the remote sensing image into a pre-trained remote sensing image interpretation model to obtain a classification result image. Among them, the remote sensing image interpretation module is specifically used to crop the remote sensing image into multiple cropped images of the same size, and embed them to obtain an input sequence; input the embedded input sequence into a transformer encoder based on an attention mechanism for encoding; adjust the result output by the transformer encoder to match the output dimension, and upsample it to obtain a classified result image.

8. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the memory via the bus, and the processor executes the machine-readable instructions to perform the steps of any method as claimed in claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are executed.

Images