Lightweight target detection method, device, equipment and medium for remote sensing images

Abstract

The present invention relates to a lightweight target detection method, device, equipment and medium for remote sensing images. The method comprises: constructing a backbone network of a lightweight target detection model including an initialization module and two or more feature extraction layers connected in sequence; wherein, the initialization module includes two or more frequency-sensitive convolution layers arranged in parallel, and the feature extraction layer includes two or more cascaded frequency-attention convolution layers; inputting the acquired remote sensing image into the initialization module, performing feature extraction through the frequency-sensitive convolution layers arranged in parallel, and then inputting the acquired remote sensing image into the feature extraction layers connected in sequence, and performing feature extraction in each feature extraction layer through two or more cascaded frequency-attention convolution layers, and finally obtaining a second output feature; processing the second output feature through other modules of the lightweight target detection model, and outputting a target detection image. The present invention realizes a lightweight design of the model structure while ensuring detection accuracy.

Description

Technical Field

[0001] The present invention relates to the field of target detection technology, and in particular to a lightweight target detection method, device, equipment and medium for remote sensing images. Background Art

[0002] With the rapid development of neural network and deep learning technologies, many applications have penetrated into all aspects of industry and life, such as spectral data applications, millimeter wave imaging, medical diagnosis, traffic rescue, and more. Remote sensing technology, in particular, is increasingly reliant on deep learning models, especially with the widespread use of drones and satellites, which has made it relatively easy to acquire large quantities of high-resolution remote sensing images.

[0003] Currently, many researchers are working to improve the accuracy and efficiency of object detection in remote sensing images. Methods such as super-resolution, feature fusion, data augmentation, and semi-supervised learning can significantly improve detection accuracy. However, multi-scale feature fusion structures can increase model parameters, while data augmentation and semi-supervised methods can increase data preprocessing and training time. Convolution is one of the most basic operations used for feature extraction in neural networks. However, standard convolution operations require significant computational overhead, and the number of parameters increases exponentially with the size of the convolution kernel.

[0004] However, for remote sensing image target detection applications, in many cases, they are run on edge devices with limited computing resources. Therefore, model lightweighting is one of the most important research directions in remote sensing image target detection tasks. Summary of the Invention

[0005] Based on this, it is necessary to provide a lightweight target detection method, device, equipment and medium for remote sensing images that is more suitable for lightweight remote sensing image target detection tasks in response to the above technical problems.

[0006] A lightweight target detection method for remote sensing images, the method comprising:

[0007] Constructing a lightweight object detection model, wherein the backbone network of the lightweight object detection model includes an initialization module and two or more sequentially connected feature extraction layers; wherein the initialization module includes two or more parallel frequency-sensitive convolutional layers, and the feature extraction layer includes two or more cascaded frequency-attention convolutional layers;

[0008] Inputting the acquired remote sensing image into the initialization module, performing feature extraction on the remote sensing image through the parallel frequency-sensitive convolutional layers to obtain a first output feature;

[0009] Input the first output feature into the feature extraction layers connected in sequence, and in each feature extraction layer, extract features through two or more cascaded frequency-directed attention convolutional layers, and use the output features of the previous feature extraction layer as the input features of the next feature extraction layer, to finally obtain the second output feature;

[0010] The second output feature is processed by other modules of the lightweight target detection model to output a target detection image.

[0011] A lightweight target detection device for remote sensing images, comprising:

[0012] A model construction module for constructing a lightweight object detection model, wherein the backbone network of the lightweight object detection model includes an initialization module and two or more sequentially connected feature extraction layers; wherein the initialization module includes two or more parallel frequency-sensitive convolutional layers, and the feature extraction layer includes two or more cascaded frequency-attention convolutional layers;

[0013] A first output feature extraction module is used to input the acquired remote sensing image into the initialization module, perform feature extraction on the remote sensing image through a frequency-sensitive convolution layer set in parallel, and obtain a first output feature;

[0014] A second output feature extraction module is configured to input the first output feature into sequentially connected feature extraction layers, perform feature extraction at each feature extraction layer through two or more cascaded frequency-directed attention convolutional layers, and use the output feature of the previous feature extraction layer as the input feature of the next feature extraction layer, ultimately obtaining a second output feature;

[0015] The target detection image output module is used to process the second output feature through other modules of the lightweight target detection model and output a target detection image.

[0016] A computer device includes a memory and a processor, wherein the memory stores a computer program and the processor implements the steps of the lightweight target detection method for remote sensing images when executing the computer program.

[0017] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the lightweight target detection method for remote sensing images.

[0018] The above-mentioned lightweight target detection method, device, equipment and medium for remote sensing images construct a lightweight target detection model, and the backbone network of the lightweight target detection model includes an initialization module and two or more feature extraction layers connected in sequence; wherein, the initialization module includes two or more frequency-sensitive convolution layers arranged in parallel, and the feature extraction layer includes two or more cascaded frequency-sensitive attention convolution layers; the acquired remote sensing image is input into the initialization module, and the remote sensing image is feature extracted through the parallel frequency-sensitive convolution layers to obtain a first output feature; the first output feature is input into the feature extraction layers connected in sequence, and in each feature extraction layer, feature extraction is performed through two or more cascaded frequency-sensitive attention convolution layers, and the output feature of the previous feature extraction layer is used as the input feature of the next feature extraction layer, and finally the second output feature is obtained; the second output feature is processed by other modules of the lightweight target detection model to output a target detection image.

[0019] The beneficial effects of the present invention are as follows: by reconstructing the backbone network and setting two or more parallel frequency-sensitive convolution layers in the initialization module, with the help of wavelet transform, remote sensing image features can be captured from different frequency domain angles, thereby improving the frequency domain feature capture capability; two or more feature extraction layers are connected in sequence, and the frequency-sensitive convolution layers in each layer are cascaded, which can refine and enhance the features layer by layer, making the final output features more discriminative.

[0020] The frequency-sensitive convolution layer is combined with the frequency-attention convolution layer. The former obtains rich frequency domain features, while the latter focuses on key features through the attention mechanism. The two work together to improve the detection accuracy of targets in remote sensing images, reduce missed detections and false detections, and thus capture target features more accurately.

[0021] The backbone network is constructed using frequency-sensitive and frequency-attention convolutional layers. Compared to traditional complex convolutional structures, this reduces the number of model parameters and computational complexity, thus achieving a lightweight design. While maintaining detection accuracy, this approach reduces the demand for computing resources and can be deployed on resource-constrained edge devices, expanding its application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

[0023] Figure 1 Schematic diagram of the process of the lightweight target detection method for remote sensing images provided in Example 1;

[0024] Figure 2 This is a schematic diagram of the backbone network structure framework provided in Example 1;

[0025] Figure 3 This is a schematic diagram of the initialization module structure framework provided in Example 1;

[0026] Figure 4 Schematic diagram of the feature extraction layer structure framework provided in Example 1;

[0027] Figure 5 This is a schematic diagram of the frequency-sensitive convolutional layer structure framework provided in Example 1;

[0028] Figure 6 Schematic diagram of the frequency-directed attention convolutional layer structure framework provided in Example 1;

[0029] Figure 7 This is a schematic diagram comparing the first set of detection effects provided in Example 1. Figure 7 (a) is a schematic diagram of the detection results obtained using the default ResNet-50. Figure 7 (b) is a schematic diagram of the detection results obtained using the method proposed by the present invention;

[0030] Figure 8 This is a schematic diagram comparing the second set of detection results provided in Example 1. Figure 8 (a) is a schematic diagram of the detection results obtained using the default ResNet-50. Figure 8 (b) is a schematic diagram of the detection results obtained using the method proposed by the present invention;

[0031] Figure 9 This is a structural block diagram of the lightweight target detection device for remote sensing images provided in Example 2;

[0032] Figure 10 This is a diagram of the internal structure of the computer device provided in Example 3.

[0033] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0034] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0035] It should be understood that the terms "first," "second," and so on, used in this disclosure are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referenced. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of this disclosure, "plurality" means at least two, such as two or three, unless otherwise specifically defined.

[0036] In addition, the technical solutions between the various embodiments of the present invention can be combined with each other, but it must be based on the fact that ordinary technicians in this field can implement it. When the combination of technical solutions is mutually contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0037] The following describes the implementation of the present invention in detail with reference to the accompanying drawings in the embodiments of the present invention.

[0038] Example 1

[0039] This embodiment discloses a lightweight target detection method for remote sensing images. By reconstructing the backbone network and setting two or more parallel frequency-sensitive convolution layers in the initialization module, with the help of wavelet transform, remote sensing image features can be captured from different frequency domain angles, thereby improving the frequency domain feature capture capability; two or more feature extraction layers are connected in sequence, and the frequency-attention convolution layers in each layer are cascaded, which can refine and enhance the features layer by layer, making the final output features more discriminative.

[0040] The frequency-sensitive convolution layer is combined with the frequency-attention convolution layer. The former obtains rich frequency domain features, while the latter focuses on key features through the attention mechanism. The two work together to improve the detection accuracy of targets in remote sensing images, reduce missed detections and false detections, and thus capture target features more accurately.

[0041] The backbone network is constructed using frequency-sensitive convolution layers and frequency-attention convolution layers. Compared with traditional complex convolution structures, this can reduce the number of model parameters and computational complexity, lower the model complexity, and achieve a lightweight design of the model structure. While ensuring detection accuracy, it reduces the demand for computing resources and can be deployed on resource-constrained edge devices to expand application scenarios.

[0042] like Figure 1 As shown, the lightweight target detection method for remote sensing images provided in this embodiment includes the following steps:

[0043] Step 201: construct a lightweight target detection model. The backbone network of the lightweight target detection model includes an initialization module and two or more feature extraction layers connected in sequence; wherein, the initialization module includes two or more frequency-sensitive convolutional layers arranged in parallel, and the feature extraction layer includes two or more cascaded frequency-attention convolutional layers.

[0044] In step 202 , the acquired remote sensing image is input into an initialization module, and features of the remote sensing image are extracted through parallel frequency-sensitive convolutional layers to obtain first output features.

[0045] Step 203: Input the first output feature into the feature extraction layers connected in sequence. In each feature extraction layer, feature extraction is performed through two or more cascaded frequency-wise attention convolutional layers, and the output feature of the previous feature extraction layer is used as the input feature of the next feature extraction layer, and finally the second output feature is obtained.

[0046] Step 204: Process the second output feature through other modules of the lightweight target detection model to output a target detection image.

[0047] It can be understood that the lightweight target detection model structure constructed in this embodiment is mainly a lightweight design of the backbone network structure in the model, and the other modules and loss functions are reused from the existing basic detection model. By seamlessly replacing the backbone network in the existing basic detector with the backbone network constructed in this embodiment, and then performing fine-tuning training, the lightweight target detection model constructed in this embodiment is obtained. Therefore, this embodiment mainly describes the backbone network, and other modules and loss functions are not described in detail.

[0048] like Figure 2 The figure shows the backbone network (FOSNet) provided in this embodiment, which includes an initialization module (MSLayer) and two or more feature extraction layers. The initialization module is located in the first layer of the backbone network, followed by the feature extraction layers. The feature extraction layers have the same structure and are arranged in order from top to bottom.

[0049] like Figure 3 As shown, the initialization module provided in this embodiment includes a downsampling layer, two or more parallel frequency-sensitive convolution layers (FOSConv), and an average pooling layer. The two or more parallel frequency-sensitive convolution layers are located between the downsampling layer and the average pooling layer. The frequency-sensitive convolution layers have the same structure, differing in the scale of the convolution kernel.

[0050] like Figure 4 As shown, the feature extraction layer provided in this embodiment includes a downsampling layer and two or more cascaded frequency-direction attention convolution layers (abbreviated as RABlock). The frequency-direction attention convolution layers have the same structure and are connected behind the downsampling module.

[0051] It is worth noting that the number of feature extraction layers, frequency-sensitive convolution layers, and frequency-attention convolution layers is set according to the needs. For the convenience of subsequent explanation, in this implementation, the number of feature extraction layers is set to 3; the number of frequency-sensitive convolution layers is set to 5, and the convolution kernel scales are respectively and ; In the three feature extraction layers, the number of frequency-directed attention convolutional layer cascades set The number of channels of the output vector at each stage is 3, 4, and 6 respectively, and the number of channels of the output vector at each stage is 128, 256, and 512 respectively, and the output resolution is one-quarter, one-eighth, and one-sixteenth of the output image respectively. It is worth noting that the above specific quantity settings are only one of the examples given to facilitate understanding of the scheme and are not intended to be a specific limitation of the present invention. The specific quantity settings mentioned below have the same meaning and will not be repeated here.

[0052] In one embodiment, Figure 5 As shown, this is a schematic diagram of the frequency-sensitive convolution layer structure framework provided in this embodiment. The frequency-sensitive convolution layer includes a wavelet transform layer and an inverse wavelet transform layer. Two or more parallel first convolution layers are arranged between the wavelet transform layer and the inverse wavelet transform layer; a second convolution layer is also arranged after the inverse wavelet transform layer.

[0053] Specifically, the wavelet transform layer is mainly used to perform wavelet transform on the input feature map, decompose it into different frequency bands, and obtain the frequency sub-maps The number is set according to the requirements. In this embodiment, 4 wavelet transform coefficients are set in the wavelet transform layer, namely and , output 4 frequency subgraphs The first convolutional layer is mainly used to frequency-convolution molecule graph Perform feature extraction, the number of which is based on the frequency subgraph Therefore, the first convolution layer is also set to 4, and the convolution kernel size is set according to the characteristics of the frequency subgraph. In this embodiment, the convolution kernel size of the first convolution layer is divided into 、 、 and ; The inverse wavelet transform layer is mainly used to reconstruct features, which is the inverse process of wavelet transform; the second convolution layer is set to 1, which is mainly used to restore the number of channels, and the convolution kernel size is ,in, The convolution kernel scale set for the frequency-sensitive convolution layer. In this embodiment, the wavelet transform layer can be constructed using wavelets such as db1, db2, db3, and db4, with db1 being preferred. The frequency-sensitive convolution layer decomposes input features into different frequencies and directions, and uses convolution of a specific shape to extract features, achieving better feature extraction capabilities while reducing parameter and time overhead.

[0054] It is worth noting that, because the frequency-sensitive convolution layer is used in the initialization module and the frequency-sensitive attention convolution layer of the feature extraction layer, the feature maps input to the frequency-sensitive convolution layer are different at different stages. Figure 5 The figure shows the input of the intermediate frequency sensitive convolution layer of the initialization module. and output In the frequency-sensitive convolution layer, the input of the frequency-sensitive convolution layer is the second sub-feature map , the output is the fifth feature map .

[0055] In one embodiment, the remote sensing image is input into the initialization module, and before the remote sensing image is subjected to feature extraction through the parallel frequency-sensitive convolution layer, the remote sensing image is further processed. Downsample to get the first feature map .

[0056] In one embodiment, the acquired remote sensing image is input into the initialization module, and feature extraction is performed on the remote sensing image through parallel frequency-sensitive convolution layers to obtain a first output feature, including:

[0057] The first feature map The parallel frequency-sensitive convolution layers are input separately for feature extraction.

[0058] In each frequency-sensitive convolution layer, the first feature map is first transformed through the wavelet transform layer. Perform wavelet transform to obtain two or more frequency subgraphs .

[0059] Frequency numerator Input the corresponding first convolution layer for feature extraction to obtain two or more second feature maps .

[0060] Two or more second feature maps After the inverse wavelet transform is performed on the input layer, it is input into the second convolutional layer for processing to obtain the third feature map. .

[0061] Each third feature map After adding, average pooling is performed to obtain the first output feature .

[0062] Specifically, remote sensing images Input initialization module, first through a downsampling layer to obtain the first feature map , then the first feature map Input the five parallel frequency-sensitive convolution layers for feature extraction to obtain the third feature map , the third feature maps output by the five frequency-sensitive convolutional layers are expressed as 、 、 、 、 . The feature map 、 、 、 、 After adding, The average pooling layer outputs the first output feature ,Right now:

[0063] .

[0064] In each frequency-sensitive convolution layer, the first feature map of the input is first transformed through the wavelet transform layer. Perform wavelet transform and obtain the low-frequency molecular graphs , horizontal high-frequency molecular map , vertical high-frequency molecular map and diagonal high-frequency molecular graphs These frequency subgraphs are input into the corresponding first convolutional layer for feature extraction to obtain the second feature graph 、 、 and Through the operation of the first convolutional layer, the frequency subgraph The number of channels is reduced to a quarter of the original, thus reducing the amount of network parameters, and then performing inverse wavelet transform through the inverse wavelet transform layer, and using The second convolutional layer restores it to the original number of channels and outputs the third feature map .

[0065] In one embodiment, Figure 6 As shown, this is a schematic diagram of the structural framework of the frequency-direction attention convolution layer provided in this embodiment. The frequency-direction attention convolution layer includes an attention layer and a frequency-direction sensitive convolution layer arranged in parallel. A third convolution layer is connected to the output ends of the attention layer and the frequency-direction sensitive convolution layer; and a residual connection is made between the input end and the output end of the frequency-direction attention convolution layer.

[0066] Specifically, the attention layer includes channel attention and spatial attention, which are mainly used to improve the ability to focus on key information. The frequency-sensitive convolution layer is set in parallel with the attention layer, and the convolution kernel scale is The third convolution layer is set to 1, which is mainly used to restore the number of channels, and the convolution kernel size is .

[0067] In one embodiment, before extracting features through two or more cascaded frequency-direction attention convolution layers at each feature extraction layer, the method further includes: Downsample to get the fourth feature map .

[0068] In one embodiment, at each feature extraction layer, feature extraction is performed through two or more cascaded frequency-wise attention convolutional layers, including:

[0069] In the first frequency-wise attention convolution layer, the fourth feature map Divide into the first sub-feature map and the second sub-feature map .

[0070] The first sub-feature map Input the attention layer for feature extraction to obtain the attention vector ; The second sub-feature map Input frequency-sensitive convolution layer for feature extraction to obtain the fifth feature map .

[0071] The attention vector With the fifth characteristic diagram After multiplication, input the third convolution layer and output the sixth feature map .

[0072] For the fourth characteristic map With the sixth characteristic diagram Perform residual operation to obtain the seventh feature map output by each frequency-direction attention convolution layer .

[0073] The seventh feature map Input the next frequency-directed attention convolution layer for processing to obtain the output features of the feature extraction layer;

[0074] The output features of the previous feature extraction layer are used as the input features of the next feature extraction layer, and finally the second output features are obtained. .

[0075] It can be understood that for the feature extraction layers connected in sequence, the input features of each feature extraction layer are the features output by the previous feature extraction layer. .feature Enter feature extraction layer, first through a The downsampling layer obtains the fourth feature map , and then the fourth feature map Input two or more cascaded frequency-directed attention convolution layers for feature extraction and output the second output feature Among them, in In the feature extraction layer, the number of cascades of frequency-directed attention convolution layers is .

[0076] The two or more cascaded frequency-direction attention convolution layers are regarded as a group of frequency-direction attention convolution modules. In a group of frequency-direction attention convolution modules, the fourth feature map Input the first frequency-wise attention convolution layer and first divide it into the first sub-feature map along the channel dimension. and the second sub-feature map Among them, the first sub-feature map Input the attention layer for feature extraction to obtain the attention vector The attention layer includes channel attention and spatial attention, so the attention vectors are recorded as and ; The second sub-feature map Enter a The frequency-sensitive convolution layer is used to extract features and obtain the fifth feature map .

[0077] The attention vector 、 With the fifth characteristic diagram After multiplication, pass a The third convolutional layer restores its channel number and outputs the sixth feature map .

[0078] For the fourth characteristic map With the sixth characteristic diagram Perform residual operation to obtain the seventh feature map output by each frequency-direction attention convolution layer , the expression is:

[0079] .

[0080] The present invention can be easily applied to existing basic detectors by reconstructing the backbone network. Among them, the frequency-sensitive convolution layer is the core component, which takes into account the characteristics of frequency and direction at the same time, decomposes the input features into different frequencies and directions, and uses a convolution layer of a specific shape to extract features, which can not only achieve better performance, but also significantly reduce parameters and time overhead. Based on this construction, the backbone network including the initialization module and the feature extraction layer can be used to replace the backbone network in the traditional model, so as to better cope with the large-scale scale changes of the target objects in the remote sensing images, further reduce the network parameters, not only ensure the detection accuracy, but also achieve a lightweight design.

[0081] In one embodiment, in order to intuitively compare the improvement in target detection accuracy brought about by the application of the method proposed in the present invention, the classic two-stage general target detector Faster R-CNN is used as a benchmark, and training and testing are performed on the remote sensing image target detection dataset AI-TOD. The detection effects of using the default ResNet-50 and the FOSNet proposed in the present invention as the backbone network are compared. Except for the backbone network, the other settings of the model are the same. The specific detection effect is as follows Figure 7 and Figure 8 As shown, Figure 7 (a) with Figure 8 (a) is the detection effect using the default ResNet-50. Figure 7 (b) with Figure 8 (b) shows the detection results using the proposed FOSNet as the backbone network. The green, blue, and red rectangles represent correctly detected, falsely detected, and missed targets, respectively. By comparing the results, we can intuitively see that the proposed method significantly improves detection accuracy.

[0082] In addition to the visual comparison, detailed experimental verification was conducted on the AI-TOD dataset. The specific results are shown in Table 1. Rows 1 to 3 show the detection results on a commonly used one-stage target detector, rows 4 to 7 show the detection results on a commonly used two-stage target detector, rows 8 to 12 show the detection results of some recently proposed backbone networks for remote sensing image target detection and conventional image target detection tasks, and rows 13 and 14 show the detection results based on the method proposed in this paper. For each row of results, in addition to comparing the detection accuracy, we also compare the performance of each detector and backbone network in terms of parameter quantity, which is an important indicator of the computational efficiency of the detector and backbone network.

[0083] The FOSNet proposed in this invention can easily replace the backbone network in commonly used basic detectors. As can be seen from the results in Table 1, compared with the basic detectors Faster R-CNN and Cascade R-CNN, the FOSNet proposed in this invention reduces the number of parameters by 20M, while also bringing an improvement in accuracy by 5.5 and 4.6 percentage points, respectively. In addition, the parameters and accuracy are compared with the latest backbone networks ARC-R50, LSKNet-S, and PKINet-S for remote sensing image target detection, as well as the lightweight networks FastViT-T12 and RepViT-M1.1 for conventional target detection. The experimental results show that the FOSNet proposed in this invention has achieved satisfactory results in both the number of parameters and the accuracy, which fully demonstrates the efficiency and effectiveness of the FOSNet proposed in this invention.

[0084] Table 1 Comparison of test results accuracy on the AI-TOD dataset

[0085]

[0086] In summary, the method proposed in this paper can well handle the task of remote sensing image target detection, which can not only achieve accurate detection results but also significantly reduce the number of model parameters.

[0087] Although this embodiment Figure 1 The steps in the diagram are shown in the order indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0088] Example 2

[0089] Based on the lightweight target detection method for remote sensing images in Example 1, this embodiment discloses a lightweight target detection device for remote sensing images, such as Figure 9 As shown, the lightweight target detection device for remote sensing images includes: a model construction module 401, a first output feature extraction module 402, a second output feature extraction module 403 and a target detection image output module 404, wherein:

[0090] The model construction module 401 is used to construct a lightweight target detection model. The backbone network of the lightweight target detection model includes an initialization module and two or more feature extraction layers connected in sequence; wherein, the initialization module includes two or more frequency-sensitive convolution layers arranged in parallel, and the feature extraction layer includes two or more cascaded frequency-attention convolution layers.

[0091] The first output feature extraction module 402 is used to input the acquired remote sensing image into the initialization module, extract features from the remote sensing image through the parallel frequency-sensitive convolution layers, and obtain the first output feature;

[0092] The second output feature extraction module 403 is used to input the first output feature into the feature extraction layers connected in sequence. At each feature extraction layer, feature extraction is performed through two or more cascaded frequency-directed attention convolutional layers, and the output feature of the previous feature extraction layer is used as the input feature of the next feature extraction layer to finally obtain the second output feature.

[0093] The target detection image output module 404 is configured to process the second output feature through other modules of the lightweight target detection model and output a target detection image.

[0094] In this embodiment, the specific working process and working principle of the model construction module 401, the first output feature extraction module 402, the second output feature extraction module 403 and the target detection image output module 404 are the same as those in the method of Example 1, and therefore will not be described in detail in this embodiment. Each unit module can be implemented in whole or in part by software, hardware or a combination thereof. Each unit module can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above unit modules.

[0095] Example 3

[0096] like Figure 10 The terminal device disclosed in this embodiment includes a transmitter, a receiver, a memory, and a processor. The transmitter is used to send instructions and data, the receiver is used to receive instructions and data, the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions stored in the memory to implement the method in the above-mentioned embodiment 1.

[0097] It should be noted that the above memory can be independent or integrated with the processor. When the memory is independently provided, the terminal device further includes a bus for connecting the memory and the processor.

[0098] Example 4

[0099] This embodiment discloses a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the method in the above-mentioned embodiment 1 is implemented.

[0100] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0101] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0102] The above-described embodiments merely illustrate several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that a person skilled in the art would be able to make numerous modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. A lightweight target detection method for remote sensing images, characterized in that: The method comprises: Constructing a lightweight object detection model, wherein the backbone network of the lightweight object detection model includes an initialization module and two or more sequentially connected feature extraction layers; wherein the initialization module includes two or more frequency-sensitive convolutional layers arranged in parallel, and the feature extraction layer includes two or more cascaded frequency-attention convolutional layers; wherein the frequency-sensitive convolutional layers have the same structure except for different convolution kernel scales; Inputting the acquired remote sensing image into the initialization module, performing feature extraction on the remote sensing image through the parallel frequency-sensitive convolutional layers to obtain a first output feature; Input the first output feature into the feature extraction layers connected in sequence, and in each feature extraction layer, extract features through two or more cascaded frequency-directed attention convolutional layers, and use the output features of the previous feature extraction layer as the input features of the next feature extraction layer, to finally obtain the second output feature; Processing the second output feature through other modules of the lightweight object detection model to output an object detection image; The frequency-sensitive convolution layer includes a wavelet transform layer and an inverse wavelet transform layer, wherein two or more parallel first convolution layers are arranged between the wavelet transform layer and the inverse wavelet transform layer; and a second convolution layer is further arranged after the inverse wavelet transform layer. The frequency-direction attention convolution layer includes an attention layer and a frequency-direction sensitive convolution layer arranged in parallel. A third convolution layer is connected to the output ends of the attention layer and the frequency-direction sensitive convolution layer; and a residual connection is made between the input end and the output end of the frequency-direction attention convolution layer.

2. The lightweight target detection method for remote sensing images according to claim 1, characterized in that: The acquired remote sensing image is input into the initialization module, and before the remote sensing image is subjected to feature extraction through the parallel frequency-sensitive convolutional layers, the method further includes: The remote sensing image Downsample to get the first feature map .

3. The lightweight target detection method for remote sensing images according to claim 2, characterized in that: Inputting the acquired remote sensing image into the initialization module, performing feature extraction on the remote sensing image through the frequency-sensitive convolution layers arranged in parallel to obtain a first output feature, including: The first feature map The parallel frequency-sensitive convolution layers are inputted respectively for feature extraction; In each frequency-sensitive convolution layer, the first feature map is first transformed by the wavelet transform layer. Perform wavelet transform to obtain two or more frequency subgraphs ; The frequency subgraph Input the corresponding first convolution layer for feature extraction to obtain two or more second feature maps ; Two or more second feature maps After the inverse wavelet transform is performed on the input layer, it is input into the second convolutional layer for processing to obtain the third feature map. ; Each third feature map After adding, average pooling is performed to obtain the first output feature .

4. The lightweight target detection method for remote sensing images according to claim 1, characterized in that: At each feature extraction layer, before performing feature extraction through two or more cascaded frequency-direction attention convolution layers, the method further includes: Downsample to get the fourth feature map .

5. The lightweight target detection method for remote sensing images according to claim 4, characterized in that: At each feature extraction layer, feature extraction is performed through two or more cascaded frequency-wise attention convolutional layers, including: In the first frequency-wise attention convolution layer, the fourth feature map Divide into the first sub-feature map and the second sub-feature map ; The first sub-feature map Input the attention layer for feature extraction to obtain the attention vector ; The second sub-feature map Input frequency-sensitive convolution layer for feature extraction to obtain the fifth feature map ; The attention vector With the fifth characteristic diagram After multiplication, the third convolution layer is input and the sixth feature map is output. ; For the fourth characteristic map With the sixth characteristic diagram Perform residual operation to obtain the seventh feature map output by each frequency-direction attention convolution layer ; The seventh feature map Input the next frequency-directed attention convolution layer for processing to obtain the output features of the feature extraction layer; The output features of the previous feature extraction layer are used as the input features of the next feature extraction layer, and finally the second output features are obtained. .

6. A lightweight target detection device for remote sensing images, characterized in that: The lightweight target detection method for remote sensing images according to any one of claims 1 to 5 is adopted, wherein the device comprises: A model construction module for constructing a lightweight object detection model, wherein the backbone network of the lightweight object detection model includes an initialization module and two or more sequentially connected feature extraction layers; wherein the initialization module includes two or more parallel frequency-sensitive convolutional layers, and the feature extraction layer includes two or more cascaded frequency-attention convolutional layers; A first output feature extraction module is used to input the acquired remote sensing image into the initialization module, perform feature extraction on the remote sensing image through a frequency-sensitive convolution layer set in parallel, and obtain a first output feature; A second output feature extraction module is configured to input the first output feature into sequentially connected feature extraction layers, perform feature extraction at each feature extraction layer through two or more cascaded frequency-directed attention convolutional layers, and use the output feature of the previous feature extraction layer as the input feature of the next feature extraction layer, ultimately obtaining a second output feature; The target detection image output module is used to process the second output feature through other modules of the lightweight target detection model and output a target detection image.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the lightweight target detection method for remote sensing images according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the lightweight target detection method for remote sensing images according to any one of claims 1 to 5 are implemented.

Images