Remote sensing basic model network rapid fine-tuning method and device based on parameter freezing

Abstract

The present invention provides a feature extraction network training method, device, equipment and storage medium for remote sensing images, which can be applied to the field of remote sensing model training technology. The method includes: using convolutional network blocks and backbone network blocks to construct a feature extraction network, the feature extraction network is used to extract the task features of the training data, and the training data is a remote sensing image; the task features of the training data are input into a task prediction module to obtain the predicted value of the task; according to the predicted value of the task and the actual value of the task, the loss value of the task is calculated; according to the loss value of the task, the parameters of the feature extraction network are iteratively updated to obtain a trained feature extraction network. The training time and computing resource requirements are greatly reduced.

Description

Technical Field

[0001] The present invention relates to the field of remote sensing model training, and in particular to a remote sensing image-oriented feature extraction network training method, device, equipment and storage medium. Background Art

[0002] With the development of deep learning and large-scale pre-trained models, they have shown strong capabilities in solving downstream tasks. However, comprehensive fine-tuning of large-scale pre-trained models faces the challenge of consuming a lot of time and computing resources. To solve this problem, a series of parameter efficient fine-tuning (PEFT) technologies have been widely used in the fields of language models and visual models. In the field of visual models, commonly used PEFT technologies such as AdaptFormer, VPT, and NOAH are mainly suitable for models with medium-sized parameters (tens of millions), but not for larger-scale pre-trained models. Summary of the invention

[0003] In view of the above problems, the present invention provides a method, device, equipment and storage medium for feature extraction network training for remote sensing images.

[0004] According to a first aspect of the present invention, a feature extraction network training method for remote sensing images is provided, comprising:

[0005] Using the convolutional network block and the backbone network block, a feature extraction network is constructed, wherein the feature extraction network is used to extract task features of the training data, wherein the training data is a remote sensing image;

[0006] Inputting the task features of the training data into a task prediction module to obtain a predicted value of the task;

[0007] Calculating the loss value of the task according to the predicted value of the task and the actual value of the task;

[0008] According to the loss value of the task, the parameters of the feature extraction network are iteratively updated to obtain a trained feature extraction network.

[0009] Optionally, the task includes at least one of image segmentation, object detection and scene classification.

[0010] Optionally, the feature extraction network includes at least one convolutional network block connected in sequence and at least one backbone network block connected in sequence, and the number of the convolutional network blocks and the number of the backbone network blocks are the same;

[0011] Each of the convolutional network blocks is connected to a backbone network block via two two-dimensional convolutional modules.

[0012] Optionally, the convolutional network block includes two two-dimensional convolution modules;

[0013] The backbone network block includes a multi-head self-attention module, a feedforward network module and two layer normalization modules.

[0014] Optionally, iteratively updating the parameters of the feature extraction network according to the loss value of the task includes:

[0015] Maintaining parameters of the at least one backbone network block unchanged;

[0016] According to the loss value of the task, parameters of the at least one convolutional network block and the two two-dimensional convolutional modules connecting the convolutional network blocks are updated.

[0017] Optionally, the task features of extracting the training data include:

[0018] Obtain N stage blocks included in the feature extraction network, each of the stage blocks includes at least one convolutional network block and at least one backbone network block, each stage block is connected through two downsampling modules, and the two downsampling modules are respectively connected to the last convolutional network block and the last backbone network block of the stage block, and N is a positive integer;

[0019] Calculate the output of each stage block, where the output of each stage block is the sum of the output of the last convolutional network block in the stage block and the output of the last backbone network block in the stage block;

[0020] The outputs of the N stage blocks are determined to be task features of the training data.

[0021] Optionally, the convolution mode of the convolutional network block is group convolution.

[0022] A second aspect of the present invention provides a remote sensing image feature extraction network training device, comprising:

[0023] A network construction module, used to construct a feature extraction network using a convolutional network block and a backbone network block, wherein the feature extraction network is used to extract task features of the training data, wherein the training data is a remote sensing image;

[0024] An input module, used to input the task features of the training data into a task prediction module to obtain a predicted value of the task;

[0025] A calculation module, used for calculating the loss value of the task according to the predicted value of the task and the actual value of the task;

[0026] The updating module is used to iteratively update the parameters of the feature extraction network according to the loss value of the task to obtain a trained feature extraction network.

[0027] A third aspect of the present invention provides an electronic device, comprising: one or more processors; a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the above method.

[0028] The fourth aspect of the present invention also provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the above method.

[0029] According to the feature extraction network training method, device, equipment and storage medium for remote sensing images provided by the present invention, compared with the traditional full fine-tuning training method, the present invention focuses on the convolutional network blocks and connection parts when fine-tuning parameters, rather than fine-tuning the entire network. Such a fast fine-tuning training method significantly reduces the number of parameters that need to be adjusted (approximately 9% of the total fine-tuning parameter amount), thereby greatly reducing the training time and computing resource requirements. Compared with full fine-tuning, the present invention also performs well in performance and does not cause the model's performance to drop significantly in downstream tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The above contents and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0031] Figure 1 A flowchart of a feature extraction network training method for remote sensing images according to an embodiment of the present invention is schematically shown;

[0032] Figure 2 A schematic diagram of a convolutional network block and a backbone network block according to an embodiment of the present invention is schematically shown;

[0033] Figure 3 A schematic diagram of a feature extraction network and a task prediction module according to an embodiment of the present invention is schematically shown;

[0034] Figure 4 A schematic diagram of an architecture of task prediction according to an embodiment of the present invention is shown;

[0035] Figure 5 A schematic diagram of a structure of a network training device for extracting features from remote sensing images according to an embodiment of the present invention is shown;

[0036] Figure 6 A block diagram of an electronic device suitable for implementing a feature extraction network training method for remote sensing images according to an embodiment of the present invention is schematically shown. DETAILED DESCRIPTION

[0037] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.

[0038] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0039] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0040] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0041] In the technical solution of the present invention, the collection, storage, use, processing, transmission, provision, invention and application of user personal information involved are in compliance with the provisions of relevant laws and regulations, necessary confidentiality measures are taken, and do not violate public order and good morals.

[0042] In the technical solution of the present invention, the acquisition, collection, storage, use, processing, transmission, provision, invention and application of data all comply with the provisions of relevant laws and regulations, take necessary confidentiality measures, and do not violate public order and good morals.

[0043] Figure 1 The flowchart of the feature extraction network training method for remote sensing images according to an embodiment of the present invention is schematically shown.

[0044] like Figure 1 As shown, the feature extraction network training method for remote sensing images in this embodiment includes operations S110 to S160.

[0045] In operation S110, a feature extraction network is constructed using a convolutional network block and a backbone network block. The feature extraction network is used to extract task features of the training data, where the training data is a remote sensing image.

[0046] In operation S120, the task features of the training data are input into a task prediction module to obtain a predicted value of the task.

[0047] In operation S130 , a loss value of the task is calculated according to the predicted value of the task and the actual value of the task.

[0048] In operation S140, the parameters of the feature extraction network are iteratively updated according to the loss value of the task to obtain a trained feature extraction network.

[0049] In one embodiment of the present invention, the feature extraction network includes at least one convolutional network block connected in sequence and at least one backbone network block connected in sequence, the number of the convolutional network blocks and the backbone network blocks are the same, and each of the convolutional network blocks is connected to the backbone network block via two two-dimensional convolution modules.

[0050] Figure 2 Schematically shows a schematic diagram of a convolutional network block and a backbone network block according to an embodiment of the present invention. Figure 2 , where the backbone network block includes a multi-head self-attention module (Attention), a feedforward network module (MLP) and two layer normalization modules (Layer Norm), the convolutional network block includes two two-dimensional convolutional modules (Conv2d), and a convolutional network block is connected to a backbone network block through two two-dimensional convolutional modules.

[0051] Figure 3 Schematically shows a schematic diagram of a feature extraction network and a task prediction module according to an embodiment of the present invention. Figure 3, the feature extraction network includes several convolutional network blocks and backbone network blocks. In the entire feature extraction network, there are N stage blocks Stage, where N is a positive integer. Several convolutional network blocks and backbone network blocks are connected front and back to form a Stage. The output result of the Stage is the sum of the output result of the last convolutional network block in the Stage and the output result of the last backbone network block in the Stage. The output result of the Stage will be input to the downstream task prediction module. In addition, each Stage is connected through two downsampling modules, which are respectively connected to the last convolutional network block and the last backbone network block of the Stage. The output result of the last convolutional network block of the Stage and the output result of the last backbone network block will be downsampled at the same time and input to the next Stage, that is, the input of the first convolutional network block in the non-first stage is the result of downsampling the output of the last convolutional network block in the previous stage, and the input of the first backbone network block in the non-first stage is the result of downsampling the output of the last backbone network block in the previous stage.

[0052] The specific value of N is related to the specific type of the backbone network block, and the present invention does not impose any specific limitation thereto.

[0053] In one embodiment of the present invention, using a feature extraction network to extract the task features of the training data includes: obtaining N stage blocks included in the feature extraction network; calculating the output of each stage block, the output of each stage block is the sum of the output of the last convolutional network block in the stage block and the output of the last backbone network block in the stage block; determining the output of the N stage blocks as the task features of the training data.

[0054] In one embodiment of the present invention, the convolution method of the convolutional network block is group convolution, which can reduce the amount of fine-tuning parameters, improve migration efficiency and ensure migration effect.

[0055] In one embodiment of the present invention, the iterative updating of the parameters of the feature extraction network according to the loss value of the task includes: keeping the parameters of the at least one backbone network block unchanged; updating the parameters of the at least one convolutional network block and the two two-dimensional convolutional modules connecting the convolutional network blocks according to the loss value of the task.

[0056] First, the training data is imported and task features are extracted through the feature extraction network. Next, according to different downstream tasks, the extracted task features are input into the corresponding task prediction module to obtain the predicted value of the task. Subsequently, the loss value of the predicted value of the task is calculated with the actual value of the task, and the feature extraction network is reversely iterated and updated according to the loss value of the task. Different from the traditional training method, the present invention adopts the method of freezing the parameters of the backbone network block, and only iteratively updates the parameters of the convolutional network block and the two-dimensional convolutional module connected in the middle. Compared with the traditional training method, the parameter update is more finely controlled, thereby improving the training efficiency of the network.

[0057] In one embodiment of the present invention, the task includes at least one of image segmentation, object detection and scene classification. Figure 4 As shown, the feature extraction network trained by the present invention can be used to train image segmentation tasks, target detection tasks, and scene classification tasks, thereby constructing image segmentation models, target detection models, scene classification models, etc. based on the trained feature extraction network.

[0058] Based on the above-mentioned feature extraction network training method for remote sensing images, the present invention also provides a feature extraction network training device for remote sensing images. Figure 5 The device is described in detail.

[0059] Figure 5 The structural block diagram of the feature extraction network training device for remote sensing images according to an embodiment of the present invention is schematically shown.

[0060] like Figure 5 As shown, the remote sensing image feature extraction network training device 500 of this embodiment includes a network construction module 510, an input module 520, a calculation module 530 and an update module 540.

[0061] A network construction module 510 is used to construct a feature extraction network using a convolutional network block and a backbone network block, wherein the feature extraction network is used to extract task features of the training data, wherein the training data is a remote sensing image;

[0062] An input module 520, used to input the task features of the training data into a task prediction module to obtain a predicted value of the task;

[0063] A calculation module 530, used to calculate the loss value of the task according to the predicted value of the task and the actual value of the task;

[0064] Update module 540 is used to iteratively update the parameters of the feature extraction network according to the loss value of the task to obtain a trained feature extraction network.

[0065] According to an embodiment of the present invention, the task includes at least one of image segmentation, object detection and scene classification.

[0066] According to an embodiment of the present invention, the feature extraction network includes at least one convolutional network block connected in sequence and at least one backbone network block connected in sequence, and the number of the convolutional network blocks and the backbone network blocks is the same;

[0067] Each of the convolutional network blocks is connected to a backbone network block via two two-dimensional convolutional modules.

[0068] According to an embodiment of the present invention, the convolutional network block includes two two-dimensional convolution modules;

[0069] The backbone network block includes a multi-head self-attention module, a feed-forward network module and two layer normalization modules.

[0070] According to an embodiment of the present invention, iteratively updating the parameters of the feature extraction network according to the loss value of the task includes:

[0071] maintaining parameters of the at least one backbone network block unchanged;

[0072] According to the loss value of the task, parameters of the at least one convolutional network block and two two-dimensional convolutional modules connecting the convolutional network blocks are updated.

[0073] According to an embodiment of the present invention, the task feature of extracting the training data includes:

[0074] Obtain N stage blocks included in the feature extraction network, each of which includes at least one convolutional network block and at least one backbone network block, each of which is connected through two downsampling modules, and the two downsampling modules are respectively connected to the last convolutional network block and the last backbone network block of the stage block, and N is a positive integer;

[0075] Calculate the output of each block at this stage. The output of each block at this stage is the sum of the output of the last convolutional network block in the block at this stage and the output of the last backbone network block in the block at this stage.

[0076] The outputs of the N stage blocks are determined to be the task features of the training data.

[0077] According to an embodiment of the present invention, the convolution method of the convolutional network block is grouped convolution.

[0078] According to an embodiment of the present invention, any multiple modules among the network construction module 510, the input module 520, the calculation module 530 and the update module 540 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present invention, at least one of the network construction module 510, the input module 520, the calculation module 530 and the update module 540 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in any appropriate combination of any of them. Alternatively, at least one of the network construction module 510, the input module 520, the calculation module 530 and the update module 540 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be executed.

[0079] Figure 6 A block diagram of an electronic device suitable for implementing a feature extraction network training method for remote sensing images according to an embodiment of the present invention is schematically shown.

[0080] like Figure 6 As shown, the electronic device 600 according to an embodiment of the present invention includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage part 608 to a random access memory (RAM) 603. The processor 601 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 601 may also include an onboard memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0081] In RAM 603, various programs and data required for the operation of electronic device 600 are stored. Processor 601, ROM 602 and RAM 603 are connected to each other through bus 604. Processor 601 performs various operations of the method flow according to the embodiment of the present invention by executing the program in ROM 602 and / or RAM 603. It should be noted that the program can also be stored in one or more memories other than ROM 602 and RAM 603. Processor 601 can also perform various operations of the method flow according to the embodiment of the present invention by executing the program stored in the one or more memories.

[0082] According to an embodiment of the present invention, the electronic device 600 may further include an input / output (I / O) interface 605, which is also connected to the bus 604. The electronic device 600 may further include one or more of the following components connected to the I / O interface 605: an input portion 606 including a keyboard, a mouse, etc.; an output portion 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 608 including a hard disk, etc.; and a communication portion 609 including a network interface card such as a LAN card, a modem, etc. The communication portion 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage portion 608 as needed.

[0083] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0084] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 602 and / or RAM 603 described above and / or one or more memories other than ROM 602 and RAM 603.

[0085] The embodiment of the present invention also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the method provided by the embodiment of the present invention.

[0086] The computer program executes the above functions defined in the system / device of the embodiment of the present invention when it is executed by the processor 601. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0087] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 609, and / or installed from a removable medium 611. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0088] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the processor 601, the above functions defined in the system of the embodiment of the present invention are performed. According to the embodiment of the present invention, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0089] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0090] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0091] It will be appreciated by those skilled in the art that the features described in the various embodiments of the present invention may be combined and / or combined in various ways, even if such combinations and / or combinations are not explicitly described in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features described in the various embodiments of the present invention may be combined and / or combined in various ways. All of these combinations and / or combinations fall within the scope of the present invention.

[0092] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment is described above, it does not mean that the measures in each embodiment cannot be used in combination advantageously. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A feature extraction network training method for remote sensing images, characterized in that: include: A feature extraction network is constructed by using a convolutional network block and a backbone network block, wherein the feature extraction network is used to extract task features of training data, and the training data is a remote sensing image, wherein the feature extraction network includes at least one convolutional network block and at least one backbone network block connected in sequence, and the number of the convolutional network blocks and the backbone network blocks is the same; each of the convolutional network blocks is connected to a backbone network block through two two-dimensional convolutional modules, and the convolutional network block includes two two-dimensional convolutional modules; the backbone network block includes a multi-head self-attention module, a feedforward network module and two layer normalization modules; The remote sensing image is input from the backbone network block, processed by the multi-head self-attention module, and output to the first layer normalization module for processing. After the first layer normalization module is processed, it is output to the feedforward network module for processing. After the feedforward network module is processed, it is output to the second layer normalization module for processing; Wherein, after being processed by the normalization module of each layer in the backbone network block, the output of each layer normalization module is mapped to the convolutional network block by the two correspondingly connected two-dimensional convolutional modules for processing, and the two two-dimensional convolutional modules in the convolutional network block respectively process the data mapped by the two connected two-dimensional convolutional modules; Superimposing the output results of the backbone network block and the convolutional network block to obtain the task features of the training data; Inputting the task features of the training data into a task prediction module to obtain a predicted value of the task; Calculating the loss value of the task according to the predicted value of the task and the actual value of the task; According to the loss value of the task, iteratively update the parameters of the feature extraction network to obtain a trained feature extraction network, including: keeping the parameters of the at least one backbone network block unchanged; according to the loss value of the task, updating the parameters of the at least one convolutional network block and the two two-dimensional convolutional modules connecting the convolutional network blocks.

2. The feature extraction network training method for remote sensing images according to claim 1, characterized in that: The tasks include at least one of image segmentation, object detection and scene classification.

3. The feature extraction network training method for remote sensing images according to claim 1, characterized in that: The task features of extracting the training data include: Obtain N stage blocks included in the feature extraction network, each of the stage blocks includes at least one convolutional network block and at least one backbone network block, each stage block is connected through two downsampling modules, and the two downsampling modules are respectively connected to the last convolutional network block and the last backbone network block of the stage block, and N is a positive integer; Calculate the output of each stage block, where the output of each stage block is the sum of the output of the last convolutional network block in the stage block and the output of the last backbone network block in the stage block; Determine the outputs of the N stage blocks as task features of the training data.

4. The feature extraction network training method for remote sensing images according to claim 1, characterized in that: The convolution mode of the convolutional network block is grouped convolution.

5. A feature extraction network training device for remote sensing images, characterized in that: include: A network construction module, used to construct a feature extraction network using a convolutional network block and a backbone network block, wherein the feature extraction network is used to extract task features of training data, and the training data is a remote sensing image, wherein the feature extraction network includes at least one convolutional network block connected in sequence and at least one backbone network block connected in sequence, and the number of the convolutional network blocks and the backbone network blocks is the same; each of the convolutional network blocks is connected to a backbone network block through two two-dimensional convolutional modules, and the convolutional network block includes two two-dimensional convolutional modules; the backbone network block includes a multi-head self-attention module, a feedforward network module and two layer normalization modules; The remote sensing image is input from the backbone network block, processed by the multi-head self-attention module, and output to the first layer normalization module for processing. After the first layer normalization module is processed, it is output to the feedforward network module for processing. After the feedforward network module is processed, it is output to the second layer normalization module for processing; Wherein, after being processed by the normalization module of each layer in the backbone network block, the output of each layer normalization module is mapped to the convolutional network block by the two correspondingly connected two-dimensional convolutional modules for processing, and the two two-dimensional convolutional modules in the convolutional network block respectively process the data mapped by the two connected two-dimensional convolutional modules; Superimposing the output results of the backbone network block and the convolutional network block to obtain the task features of the training data; An input module, used to input the task features of the training data into a task prediction module to obtain a predicted value of the task; A calculation module, used for calculating the loss value of the task according to the predicted value of the task and the actual value of the task; An updating module is used to iteratively update the parameters of the feature extraction network according to the loss value of the task to obtain a trained feature extraction network, including: keeping the parameters of the at least one backbone network block unchanged; updating the parameters of the at least one convolutional network block and the two two-dimensional convolutional modules connecting the convolutional network blocks according to the loss value of the task.

6. An electronic device comprising: one or more processors; as well as a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors execute the method according to any one of claims 1 to 4.

7. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the method according to any one of claims 1 to 4.

Images