A deep learning-based remote sensing image small target detection and recognition system and method

Abstract

The application discloses a kind of remote sensing image small target detection identification system and method based on deep learning, solve the problem that current remote sensing image target detection method is difficult to detect small target under complex background, bring difficulty to detection task.The remote sensing image small target detection identification system and method provided by the application include three innovative light-weight modules of spatial local information enhancement module (SPDI module), hierarchical feature self-adaptive weighted fusion module (CBCC module) and global large field of view perception module (MAGI), which effectively improve the detection performance from three aspects of spatial local information utilization, feature self-adaptive fusion and global large field of view feature perception, while maintaining light weight, the ability to detect small targets in complex remote sensing environment has been significantly improved, while taking into account resource consumption, showing advanced performance for small targets under complex remote sensing background.

Description

Technical Field

[0001] This invention relates to a remote sensing image target detection system and method, specifically to a remote sensing image small target detection and recognition system and method based on deep learning. Background Technology

[0002] With the rapid development of remote sensing technology, the application of optical remote sensing imagery in emergency rescue, traffic monitoring, and building inspection has increased significantly. Target detection tasks based on optical remote sensing imagery are also booming, including vehicle inspection and aircraft inspection. Optical remote sensing imagery typically has a large field of view and can acquire a large amount of information; however, the objects of interest are usually small targets (approximately 12.8 pixels in size) submerged in complex backgrounds. These small targets are usually characterized by low resolution, poor quality, and complex backgrounds.

[0003] Existing technologies typically employ the YOLOv5 detection system to perform target detection and recognition while maintaining detection speed. The overall framework of YOLOv5 mainly consists of three parts: the backbone network, the neck network, and the detection head.

[0004] The backbone network comprises the CBS module, CSP module, and SPPF module. The CBS module is a multi-layered module containing convolutional layers, batch normalization layers, and activation function layers, capable of feature extraction and non-linear transformation of the input feature map. The CSP module is a nested module combining the CBS module and convolutional layers through residual structures; introducing the CSP module improves network efficiency while extracting deep features. The SPPF module is an improved spatial pyramid pooling module, consisting of one convolutional layer and three 5x5 max pooling layers, capable of aggregating multi-scale contextual information without significantly increasing computational cost.

[0005] The neck network consists of a CBS module, a CSP module, and a Concat module. The Concat module concatenates feature maps along the channel dimension, achieving feature fusion.

[0006] The detection heads include Detect1, Detect2 and Detect3, which are used to collect feature maps of different scales for target detection, thereby improving the multi-scale performance of the model in detecting targets of different sizes.

[0007] The YOLOv5 detection system described above typically has the following shortcomings:

[0008] (1) In the process of feature extraction through the hierarchical structure of downsampling and upsampling, the downsampling operation will lose some spatial local information, and the preservation and utilization of spatial local information is not sufficient.

[0009] (2) In the process of feature fusion of different feature maps, the Concat module has the disadvantage of poor flexibility and inability to adjust the emphasis on different receptive field feature maps.

[0010] (3) The detection system has insufficient ability to perceive global long-distance information.

[0011] Therefore, existing remote sensing target detection methods have difficulty accurately detecting and identifying these small targets. At the same time, the diversity of their scale, orientation, and background also brings difficulties to the task of detecting small targets in remote sensing images. Summary of the Invention

[0012] To address the technical problem that existing remote sensing image target detection methods struggle to detect small targets against complex backgrounds, posing a significant challenge to detection tasks, this invention provides a deep learning-based remote sensing image small target detection and recognition system and method.

[0013] To achieve the above objectives, the present invention adopts the following technical solution:

[0014] A deep learning-based remote sensing image small target detection and recognition system includes a backbone network, a neck network, and a detection head connected in sequence. The backbone network includes a first CBS module, a first CSP module, a second CBS module, a second CSP module, and an SPPF module. The neck network includes a third CBS module, a fourth CBS module, a fifth CBS module, a sixth CBS module, a third CSP module, a fourth CSP module, a fifth CSP module, and a sixth CSP module. The detection head includes a first Detect module, a second Detect module, and a third Detect module. Its unique feature is that:

[0015] The backbone network also includes a first SPID module and a second SPID module.

[0016] The first CBS module, the first SPID module, the first CSP module, the second CBS module, the second CSP module, the second SPID module, and the SPPF module are connected in sequence;

[0017] The neck network also includes a first CBCC module, a second CBCC module, a third CBCC module, a fourth CBCC module, a first MAGI module, a second MAGI module, and a third MAGI module;

[0018] The first input terminal of the first CBCC module is connected to the output terminal of the second CBS module and the first output terminal of the fourth CBS module, respectively. Its second input terminal is connected to the output terminal of the third CBS module, and its output terminal is connected to the second input terminal of the fourth CBCC module via the third CSP module and the sixth CBS module in sequence.

[0019] The first input terminal of the second CBCC module is connected to the output terminal of the first CBS module, its second input terminal is connected to the second output terminal of the fourth CBS module, and its output terminal is connected to the input terminal of the third CBS module via the fourth CSP module.

[0020] The first input terminal of the third CBCC module is connected to the output terminal of the second CBS module, its second input terminal is connected to the second output terminal of the fifth CBS module, and its output terminal is connected to the input terminal of the fourth CBS module via the fifth CSP module; the input terminal of the fifth CBS module is connected to the output terminal of the SPPF module.

[0021] The first input terminal of the fourth CBCC module is connected to the first output terminal of the fifth CBS module;

[0022] The input of the first MAGI module is connected to the output of the third CSP module, and its output is connected to the first Detect module.

[0023] The input of the second MAGI module is connected to the output of the fourth CSP module, and its output is connected to the second Detect module.

[0024] The input of the third MAGI module is connected to the output of the fourth CBCC module through the sixth CSP module, and its output is connected to the third Detect module.

[0025] Furthermore, the first SPID module and the second SPID module have the same structure;

[0026] The first SPID module includes a spatial dimension equal-step extraction unit, a channel dimension staggered splicing unit, and a fully convolutional activation unit connected in sequence;

[0027] The spatial dimension equal step size extraction unit is connected to the output end of the first CBS module;

[0028] The fully convolutional activation unit is connected to the input of the first CSP module.

[0029] Furthermore, the first CBCC module, the second CBCC module, the third CBCC module, and the fourth CBCC module have the same structure;

[0030] The first CBCC module includes interconnected block-level adaptive weighting units and channel-level adaptive weighting units;

[0031] The input terminals of the block-level adaptive weighting unit are respectively connected to the second CBS module, the fourth CBS module, and the third CBS module;

[0032] The output of the channel-level adaptive weighting unit is connected to the third CSP module.

[0033] Furthermore, the first MAGI module, the second MAGI module, and the third MAGI module have the same structure;

[0034] The first MAGI module includes a global perception unit, a receptive field expansion unit, and a convolution weighting unit connected in sequence;

[0035] The input terminal of the global sensing unit is connected to the third CSP module, and its output terminal is connected to the first input terminal of the receptive field expansion unit.

[0036] The second input terminal of the receptive field expansion unit is connected to the output terminal of the third CSP module, and its output terminal is connected to the first input terminal of the convolution weighting unit.

[0037] The second input of the convolution weighting unit is connected to the output of the third CSP module, and its output is connected to the first Detect module.

[0038] A method for detecting and recognizing small targets in remote sensing images based on deep learning, employing the aforementioned deep learning-based remote sensing image small target detection and recognition system, is characterized by including the following steps:

[0039] Step 1: The first CBS module receives the initial remote sensing image and performs shallow feature extraction on it to obtain the first feature map, which is then sent to the first SPID module and the second CBCC module respectively.

[0040] Step 2: The first SPID module performs staggered downsampling processing on the first feature map from spatial dimension to channel depth to obtain the second feature map, which is then sent to the first CSP module.

[0041] Step 3: The first CSP module and the second CBS module sequentially perform deep feature extraction on the second feature map to obtain the sixth feature map. The second CBS module sends the sixth feature map to the second CSP module, the first CBCC module, and the third CBCC module respectively. The second CSP module performs deep feature extraction on the sixth feature map to obtain the third feature map.

[0042] Step 4: The second SPID module performs staggered downsampling processing on the third feature map from spatial dimension to channel depth to obtain the fourth feature map, which is then sent to the SPPF module.

[0043] Step 5: The SPPF module aggregates context information on the fourth feature map, and then the fifth feature map is obtained after processing by the fifth CBS module. It is then sent to the fourth CBCC module, and the fifth feature map is upsampled and sent to the third CBCC module.

[0044] Step 6: The third CBCC module performs two-layer adaptive weighting of the fifth and sixth feature maps at the block and channel levels, and then processes them sequentially through the fifth CSP module and the fourth CBS module to obtain the seventh feature map, which is then sent to the first CBCC module. At the same time, the seventh feature map is upsampled and sent to the second CBCC module.

[0045] Step 7: The second CBCC module performs two-layer adaptive weighting of the first feature map and the seventh feature map at the block level and the channel level, and then processes the twelfth feature map by the fourth CSP module and sends it to the third CBS module and the second MAGI module respectively. The third CBS module processes the twelfth feature map to obtain the eighth feature map, upsamples it and sends it to the first CBCC module.

[0046] Step 8: The first CBCC module performs two-layer adaptive weighting of the sixth, seventh, and eighth feature maps at the block and channel levels, and then processes them through the third CSP module to obtain the tenth feature map, which is sent to the sixth CBS module and the first MAGI module respectively. The sixth CBS module processes the tenth feature map to obtain the ninth feature map, which is then sent to the fourth CBCC module.

[0047] Step 9: The fourth CBCC module performs two-level adaptive weighting on the fifth and ninth feature maps at the block and channel levels, and then processes them through the sixth CSP module to obtain the eleventh feature map, which is then sent to the third MAGI module.

[0048] Step 10: The first MAGI module, the second MAGI module, and the third MAGI module perform global large-view feature perception on the tenth feature map, the twelfth feature map, and the eleventh feature map, respectively, to obtain the thirteenth feature map, the fourteenth feature map, and the fifteenth feature map, and send them to the first Detect module, the second Detect module, and the third Detect module, respectively.

[0049] Step 11: The first Detect module, the second Detect module, and the third Detect module detect and identify the thirteenth feature map, the fourteenth feature map, and the fifteenth feature map, respectively, and then perform a comprehensive calculation on the detection and identification results to obtain the detection matrix;

[0050] Step 12: Combine the detection matrix with the initial remote sensing image to draw the remote sensing image detection map, thus completing the detection and recognition of small targets in remote sensing images based on deep learning.

[0051] Furthermore, step 2 specifically includes:

[0052] 2.1 The spatial dimension equal step size extraction unit extracts pixels from the first feature map with a preset step size and reassembles them to obtain feature sub-maps. The number of feature sub-maps is the square of the step size.

[0053] 2.2 The channel-dimensional staggered stitching unit performs staggered stitching of all feature sub-images from spatial dimension to channel dimension to form a stitched feature map, wherein the number of channels in the stitched feature map is the square of the preset step size;

[0054] 2.3 The fully convolutional activation unit performs channel compression convolution on all the spliced ​​feature maps to obtain a second feature map with the same number of channels as the first feature map, and sends it to the first CSP module.

[0055] Furthermore, step 6 specifically includes:

[0056] 6.1 The block-level adaptive weighting unit uses an iterable block-weighting tensor to perform the first-layer block-level adaptive weighting on the fifth and sixth feature maps to obtain the block-level weighted feature maps;

[0057] 6.2 The channel-level adaptive weighting unit uses an iterable channel weighting tensor to perform a second-layer channel-level adaptive weighting on the block-level weighted feature map. After processing by the fifth CSP module and the fourth CBS module, the seventh feature map is sent to the first CBCC module and then upsampled to the second CBCC module.

[0058] Furthermore, in step 10, the step of the first MAGI module performing global large-view feature perception on the tenth feature map specifically includes:

[0059] The global perception unit extracts global long-range information from the tenth feature map through global attention and activation function operations;

[0060] The receptive field expansion unit obtains local large field information of the tenth feature map through dilated convolution, and then fuses the local large field information and global long distance information through matrix alignment multiplication. After fusion, the receptive field expansion feature map is obtained by 1×1 convolution.

[0061] After performing a 1×1 convolution on the tenth feature map, the weighted convolution unit fuses it with the receptive field expansion feature map through matrix positional multiplication. After fusion, the global large field-of-view perceptual feature map is obtained after activation function transformation. The global large field-of-view perceptual feature map and the tenth feature map are then processed with residuals to obtain the thirteenth feature map.

[0062] Furthermore, in step 6.1:

[0063] The range of the iterable block-weighted tensor is 0 to 1.

[0064] Furthermore, in step 6.2:

[0065] The range of the iterable channel-weighted tensor is 0 to 1.

[0066] The beneficial effects of this invention are:

[0067] 1. The remote sensing image small target detection and recognition system and method provided by the present invention includes three innovative lightweight modules: a spatial local information enhancement module (SPDI module), a hierarchical feature adaptive weighted fusion module (CBCC module), and a global large field of view perception module (MAGI). These three modules effectively improve detection performance from three aspects: utilization of spatial local information, adaptive feature fusion, and global large field of view feature perception. While maintaining a lightweight design, the ability to detect small targets in complex remote sensing environments is significantly improved. While taking into account resource consumption, it shows advanced performance for small targets in complex remote sensing backgrounds.

[0068] 2. In the remote sensing image small target detection and recognition system and method provided by the present invention, the SPID module retains more spatial local information of the input feature map during downsampling by interleaving and recombining spatial information in the channel dimension; the CBCC module further enhances the flexibility and efficiency of multi-scale feature fusion through a two-layer adaptive weighting structure at the block level and channel level; the MAGI module fuses information from the global large field of view, the long-distance global information, the local large field of view information and the input information feature map through a three-branch approach, which strengthens the ability to distinguish between background and target and further improves the accuracy of detection and recognition.

[0069] 3. This invention verifies the effectiveness and feasibility of a deep learning-based remote sensing image small target detection and recognition system and method on the AI-TOD public high-difficulty dataset for small remote sensing targets. Experimental results show that the detection and recognition system and method achieve leading performance under the same parameter scale. Attached Figure Description

[0070] Figure 1 This is a schematic diagram of the structure of an embodiment of a remote sensing image small target detection and recognition system based on deep learning according to the present invention;

[0071] Figure 2 This is a schematic diagram of the structure of the first SPID module or the second SPID module in an embodiment of the present invention;

[0072] Figure 3 This is a schematic diagram of the structure of the first CBCC module, the second CBCC module, the third CBCC module, or the fourth CBCC module in an embodiment of the present invention;

[0073] Figure 4 This is a schematic diagram of the structure of the first MAGI module, the second MAGI module, or the third MAGI module according to an embodiment of the present invention;

[0074] Figure 5 This is a detection and recognition effect diagram of small target detection in remote sensing images on the AI-TOD dataset in an embodiment of the present invention. Detailed Implementation

[0075] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings and embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0076] This invention provides a deep learning-based system for detecting and recognizing small targets in remote sensing images, such as... Figure 1 As shown, it includes a backbone network, a neck network, and a detection head connected in sequence.

[0077] The backbone network is primarily structured around CBS and CSP modules, and incorporates a Spatial Local Information Enhancement (SPDI) module to fully preserve spatial local information while performing downsampling. Specifically, the backbone network includes a first CBS module, a first CSP module, a second CBS module, a second CSP module, and an SPPF module, as well as a first SPID module and a second SPID module. The first CBS module is used for the initial input remote sensing imagery.

[0078] The input of the first SPID module is connected to the output of the first CBS module, and its output is connected to the input of the first CSP module; the output of the first CSP module is connected to the input of the second CSP module via the second CBS module; the input of the second SPID module is connected to the output of the second CSP module, and its output is connected to the input of the SPPF module.

[0079] The first SPID module and the second SPID module have the same structure. This embodiment will use the first SPID module as an example for introduction:

[0080] like Figure 2 As shown, the first SPID module includes a spatial dimension equal-stride extraction unit, a channel dimension interleaved splicing unit, and a fully convolutional activation unit connected in sequence; the spatial dimension equal-stride extraction unit is connected to the output of the first CBS module accordingly; and the fully convolutional activation unit is connected to the input of the first CSP module accordingly.

[0081] In other embodiments, the input terminal of the first CBS module is also connected to the seventh CSP module and the seventh CBS module in sequence; then the seventh CBS module is used to input the initial remote sensing image.

[0082] After the feature maps enter the neck network, they undergo upsampling and feature fusion is performed through the Feature Adaptive Weighted Fusion (CBCC) module. Specifically, the neck network includes the third CBS module, the fourth CBS module, the fifth CBS module, the sixth CBS module, the third CSP module, the fourth CSP module, the fifth CSP module, the sixth CSP module, the first CBCC module, the second CBCC module, the third CBCC module, the fourth CBCC module, the first MAGI module, the second MAGI module, and the third MAGI module.

[0083] The first input terminal of the first CBCC module is connected to the output terminal of the second CBS module and the first output terminal of the fourth CBS module, respectively. Its second input terminal is connected to the output terminal of the third CBS module, and its output terminal is connected to the input terminal of the third CSP module. The first input terminal of the second CBCC module is connected to the output terminal of the first CBS module, its second input terminal is connected to the second output terminal of the fourth CBS module, and its output terminal is connected to the input terminal of the fourth CSP module. The first input terminal of the third CBCC module is connected to the output terminal of the second CBS module, its second input terminal is connected to the second output terminal of the fifth CBS module, its output terminal is connected to the input terminal of the fifth CSP module, and the output terminal of the fifth CSP module is connected to the input terminal of the fourth CBS module. The first input terminal of the fourth CBCC module is connected to the first output terminal of the fifth CBS module, its second input terminal is connected to the output terminal of the sixth CBS module, its output terminal is connected to the input terminal of the sixth CSP module, and the input terminal of the sixth CBS module is connected to the output terminal of the third CSP module.

[0084] The first CBCC module, the second CBCC module, the third CBCC module, and the fourth CBCC module have the same structure; this embodiment uses the first CBCC module as an example for description:

[0085] like Figure 3 As shown, the first CBCC module includes interconnected block-level adaptive weighting units and channel-level adaptive weighting units; the block-level adaptive weighting units are connected to the second CBS module, the fourth CBS module, and the third CBS module, respectively; and the channel-level adaptive weighting units are connected to the third CSP module.

[0086] The input of the first MAGI module is connected to the output of the third CSP module, and its output is connected to the first Detect module; the input of the second MAGI module is connected to the output of the fourth CSP module, and its output is connected to the second Detect module; the output of the fourth CSP module is also connected to the input of the third CBS module; the input of the third MAGI module is connected to the output of the sixth CSP module, and its output is connected to the third Detect module.

[0087] The first MAGI module, the second MAGI module, and the third MAGI module have the same structure. This embodiment will use the first MAGI module as an example for introduction:

[0088] like Figure 4 As shown, the first MAGI module includes a global perception unit, a receptive field expansion unit, and a convolutional weighting unit connected in sequence; the input of the global perception unit is connected to the third CSP module, and its output is connected to the first input of the receptive field expansion unit; the second input of the receptive field expansion unit is connected to the output of the third CSP module, and its output is connected to the first input of the convolutional weighting unit; the second input of the convolutional weighting unit is connected to the output of the third CSP module, and its output is connected to the first Detect module.

[0089] The detection head inputs the fusion results from the neck network through a three-layer CSP module into a global large-field perception module (MAGI module) to introduce context awareness. Finally, the output feature map of the MAGI module is input into the detection head for detection output. Specifically, the detection head includes a first detection module, a second detection module, and a third detection module.

[0090] This remote sensing image small target detection and recognition system includes three innovative lightweight modules: Spatial Local Information Enhancement (SPDI) module, Hierarchical Feature Adaptive Weighted Fusion (CBCC) module, and Global Large Field of View (MAGI) module. These three modules effectively improve the performance of the detector in terms of feature extraction, feature fusion, and feature perception. While maintaining a lightweight design, the ability to detect small targets in complex remote sensing environments has been significantly enhanced.

[0091] The above-mentioned deep learning-based remote sensing image small target detection and recognition system is used to detect and recognize small targets in remote sensing images, including the following steps:

[0092] Step 1: The first CBS module receives the initial remote sensing image and performs shallow feature extraction on it to obtain the first feature map, which is then sent to the first SPID module and the second CBCC module. The size of the first feature map is (C, W, H), where C represents the number of channels in the input feature map, W represents the pixel width of the input feature map, and H represents the pixel height of the input feature map.

[0093] Step 2: The first SPID module performs interleaved downsampling processing on the first feature map from spatial dimension to channel depth to obtain the second feature map, which is then sent to the first CSP module; specifically including:

[0094] 2.1 Spatial Dimension Equal Step Size Extraction Unit: Based on a preset step size scale, pixels are extracted from the same starting point and recombined into a new feature sub-map X.

[0095] X 0,0 =X[C,0::scale,0::scale]

[0096] X scale-1,0 =X[C,1::scale,0::scale]

[0097] X 0,scale-1 =X[C,0::scale,1::scale]

[0098] X scale-1,scale-1 =X[C,1::scale,1::scale]

[0099] Here, ::scale represents the information extraction of the input feature map in the spatial dimension.

[0100] The width and height of feature sub-map X are half that of the first feature map, and the number of channels is the same as that of the first feature map. Figure 1 The number of feature sub-maps obtained by recombination is scale. 2 .

[0101] 2.2 The channel-dimensional staggered stitching unit performs staggered stitching of all feature sub-maps from the spatial dimension to the channel dimension in the channel dimension to form a stitched feature map F. Output Send to the first CSP module;

[0102]

[0103] splicing feature map F Output The number of channels is scale 2 The above operations ensure that adjacent pixels in the spatial dimension of the input feature map remain adjacent in the channel dimension of the output feature map, thus preserving more spatial local information.

[0104] 2.3 The fully convolutional activation unit performs channel compression convolution on all the spliced ​​feature maps to obtain a second feature map with the same number of channels as the first feature map.

[0105] Step 3: The first CSP module and the second CBS module sequentially perform deep feature extraction on the second feature map to obtain the sixth feature map. The second CBS module sends the sixth feature map to the second CSP module, the first CBCC module, and the third CBCC module respectively. The second CSP module performs deep feature extraction on the sixth feature map to obtain the third feature map.

[0106] Step 4: The second SPID module performs interleaved downsampling processing on the third feature map from spatial dimension to channel depth to obtain the fourth feature map, which is then sent to the SPPF module. The specific steps of the second SPID module performing interleaved downsampling processing on the third feature map from spatial dimension to channel depth are the same as those of the first SPID module performing interleaved downsampling processing on the first feature map from spatial dimension to channel depth.

[0107] Step 5: The SPPF module aggregates context information for the fourth feature map, and then processes it through the fifth CBS module to obtain the fifth feature map, which is then sent to the fourth CBCC module and simultaneously upsampled and sent to the third CBCC module.

[0108] Step 6: The third CBCC module performs two-layer adaptive weighting (block-level and channel-level) on the fifth and sixth feature maps, then processes them sequentially through the fifth CSP module and the fourth CBS module to obtain the seventh feature map, which is then sent to the first CBCC module. Simultaneously, it is upsampled and sent to the second CBCC module. Specifically, this includes:

[0109] 6.1 Block-level adaptive weighted units use iterable block-weighted tensors W ranging from 0 to 1. block A first-level block-level adaptive weighting is performed on N fifth and sixth feature maps of size (C, W, H) to achieve preliminary adaptive focusing on key information blocks, resulting in a block-level weighted feature map F of dimension (N×C, W, H). block ;

[0110] 6.2 The channel-level adaptive weighting unit uses an iterable channel-weighted tensor W with a range of 0 to 1. channel Block-level weighted feature map F block After performing channel-level adaptive weighting in the second layer, a feature map F of size (N×C, W, H) is obtained. output Feature map F output The seventh feature map is obtained by sequentially processing the fifth CSP module and the fourth CBS module and then sent to the first CBCC module. Simultaneously, it is upsampled and sent to the second CBCC module.

[0111] The semantic information represented by feature maps at different levels varies. By fusing multi-level feature maps, the representation of features related to small objects can be effectively enhanced. The CBCC module adaptively allocates weights by block and channel through a two-level weighted structure, thereby achieving more flexible and efficient feature fusion.

[0112] Step 7: After the second CBCC module performs two-layer adaptive weighting at the block level and channel level on the first and seventh feature maps, the fourth CSP module processes the resulting twelfth feature map, which is then sent to the third CBS module and the second MAGI module. The third CBS module processes the twelfth feature map to obtain the eighth feature map, which is then upsampled and sent to the first CBCC module. The specific steps of the second CBCC module performing two-layer adaptive weighting at the block level and channel level on the first and seventh feature maps are the same as those of the third CBCC module performing two-layer adaptive weighting at the block level and channel level on the fifth and sixth feature maps.

[0113] Step 8: After the first CBCC module performs two-level adaptive weighting (block-level and channel-level) on the sixth, seventh, and eighth feature maps, the tenth feature map is obtained by the third CSP module and sent to the sixth CBS module and the first MAGI module respectively. The sixth CBS module processes the tenth feature map to obtain the ninth feature map and sends it to the fourth CBCC module. The specific steps of the first CBCC module performing two-level adaptive weighting (block-level and channel-level) on the sixth, seventh, and eighth feature maps are the same as the specific steps of the third CBCC module performing two-level adaptive weighting (block-level and channel-level) on the fifth and sixth feature maps.

[0114] Step 9: The fourth CBCC module performs two-level adaptive weighting (block-level and channel-level) on the fifth and ninth feature maps, and then processes them through the sixth CSP module to obtain the eleventh feature map, which is then sent to the third MAGI module. The specific steps of the fourth CBCC module performing two-level adaptive weighting (block-level and channel-level) on the fifth and ninth feature maps are the same as those of the third CBCC module performing two-level adaptive weighting (block-level and channel-level) on the fifth and sixth feature maps.

[0115] Step 10: The first MAGI module, the second MAGI module, and the third MAGI module perform global wide-view feature perception on the tenth, twelfth, and eleventh feature maps, respectively, to obtain the thirteenth, fourteenth, and fifteenth feature maps, which are then sent to the first Detect module, the second Detect module, and the third Detect module, respectively. Specifically, the first MAGI module performs global wide-view feature perception on the tenth feature map, including the following steps:

[0116] 10.11. The global perception unit obtains the global perception feature map q branch through global attention operation, enabling the model to perceive global context information and better understand the multiple semantic relationships in the image. Then, it extracts the global long-distance information of the tenth feature map through activation function operation.

[0117] 10.12 The receptive field expansion unit expands the receptive field through dilated convolution to obtain the large-field receptive feature map k branch, which obtains the local large-field information of the tenth feature map. Then, the q branch and the k branch are subjected to matrix alignment multiplication to obtain the qk branch, which is used to perform attention operation on the large-field receptive feature map from the global field of view, thereby fusing the local large-field information and the global long-distance information. After fusion, the receptive field expansion feature map is obtained after 1×1 convolution.

[0118] 10.13. The convolutional weighted unit performs a 1×1 convolution on the tenth feature map to obtain the input information feature map v branch. The qk branch and v branch are fused by positional multiplication to further enhance the features and suppress the background. After fusion, the global large-view perception feature map is obtained after activation function transformation. The global large-view perception feature map and the tenth feature map are processed by residual processing to obtain the thirteenth feature map, which enhances the robustness of the model. The size of the tenth feature map is (N, C, W, H).

[0119] The specific steps of the second MAGI module for global large-view feature perception of the twelfth feature map and the specific steps of the third MAGI module for global large-view feature perception of the eleventh feature map are the same as the specific steps of the first MAGI module for global large-view feature perception of the tenth feature map.

[0120] Step 11: The first Detect module, the second Detect module, and the third Detect module detect and identify the thirteenth feature map, the fourteenth feature map, and the fifteenth feature map, respectively, and then perform a comprehensive calculation on the detection and identification results to obtain the detection matrix;

[0121] Step 12: Combine the detection matrix with the initial remote sensing image to draw the remote sensing image detection map, thus completing the detection and recognition of small targets in remote sensing images based on deep learning.

[0122] Experiments were conducted on the AI-TOD dataset using the above system and methods:

[0123] like Figure 5 As shown, the effectiveness of the proposed detection method is verified by performing detection on the AI-TOD remote sensing small target dataset. The average target size in the AI-TOD dataset is approximately 12.8 pixels, which is much smaller than in other datasets. The test results on the AI-TOD dataset are as follows:

[0124] Table 1. Test Results of the AI-TOD Dataset

[0125]

[0126]

[0127] mAP50 and mAP50-95 are comprehensive performance metrics that consider both detection precision and recall. Experimental results show that the method proposed in this invention has better detection performance than mainstream lightweight target detection models with approximately the same number of parameters. Compared with YOLOv5s, the proposed method improves the mAP50 metric by 5.493%, the mAP50-95 metric by 3.931%, and reduces the number of parameters by approximately 24.5%. Compared with YOLOv8s, the mAP50 metric improves by 5.293%, the recall metric improves by 4.037%, and the precision metric improves by 6.267%.

[0128] Comparative experiments on the AI-TOD public remote sensing small target detection dataset verified the effectiveness and lightweight nature of the monitoring and identification method provided in this invention. Experiments on the AI-TOD dataset show that the mAP50 and mAP50-95 indices reached 64.053% and 27.283%, respectively, significantly outperforming other detection models with the same parameter range.

[0129] In summary, the monitoring and identification system and method provided by this invention have the best performance in small target detection under remote sensing background. Higher performance can be achieved by adding only a few parameters, while maintaining the characteristics of lightweight design, and can be adapted to deployment in more resource-constrained scenarios.

[0130] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present invention should be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A deep learning-based remote sensing image small target detection and recognition system, comprising a backbone network, a neck network, and a detection head connected in sequence; the backbone network includes a first CBS module, a first CSP module, a second CBS module, a second CSP module, and an SPPF module; the neck network includes a third CBS module, a fourth CBS module, a fifth CBS module, a sixth CBS module, a third CSP module, a fourth CSP module, a fifth CSP module, and a sixth CSP module; the detection head includes a first Detect module, a second Detect module, and a third Detect module; characterized in that: The backbone network also includes a first SPID module and a second SPID module. The first CBS module, the first SPID module, the first CSP module, the second CBS module, the second CSP module, the second SPID module, and the SPPF module are connected in sequence; The first SPID module and the second SPID module have the same structure; the first SPID module includes a spatial dimension equal-step extraction unit, a channel dimension staggered splicing unit and a fully convolutional activation unit connected in sequence. The neck network also includes a first CBCC module, a second CBCC module, a third CBCC module, a fourth CBCC module, a first MAGI module, a second MAGI module, and a third MAGI module; The first input terminal of the first CBCC module is connected to the output terminal of the second CBS module and the first output terminal of the fourth CBS module, respectively. Its second input terminal is connected to the output terminal of the third CBS module, and its output terminal is connected to the second input terminal of the fourth CBCC module via the third CSP module and the sixth CBS module in sequence. The first input terminal of the second CBCC module is connected to the output terminal of the first CBS module, its second input terminal is connected to the second output terminal of the fourth CBS module, and its output terminal is connected to the input terminal of the third CBS module via the fourth CSP module. The first input terminal of the third CBCC module is connected to the output terminal of the second CBS module, its second input terminal is connected to the second output terminal of the fifth CBS module, and its output terminal is connected to the input terminal of the fourth CBS module via the fifth CSP module; the input terminal of the fifth CBS module is connected to the output terminal of the SPPF module. The first input terminal of the fourth CBCC module is connected to the first output terminal of the fifth CBS module; The first CBCC module, the second CBCC module, the third CBCC module, and the fourth CBCC module have the same structure; the first CBCC module includes interconnected block-level adaptive weighting units and channel-level adaptive weighting units; The input of the first MAGI module is connected to the output of the third CSP module, and its output is connected to the first Detect module. The input of the second MAGI module is connected to the output of the fourth CSP module, and its output is connected to the second Detect module. The input of the third MAGI module is connected to the output of the fourth CBCC module through the sixth CSP module, and its output is connected to the third Detect module. The first MAGI module, the second MAGI module, and the third MAGI module have the same structure; the first MAGI module includes a global sensing unit, a receptive field expansion unit, and a convolution weighting unit connected in sequence.

2. The remote sensing image small target detection and recognition system based on deep learning according to claim 1, characterized in that: The spatial dimension equal step size extraction unit is connected to the output end of the first CBS module; The fully convolutional activation unit is connected to the input of the first CSP module.

3. The remote sensing image small target detection and recognition system based on deep learning according to claim 1 or 2, characterized in that: The input terminals of the block-level adaptive weighting unit are respectively connected to the second CBS module, the fourth CBS module, and the third CBS module; The output of the channel-level adaptive weighting unit is connected to the third CSP module.

4. The remote sensing image small target detection and recognition system based on deep learning according to claim 3, characterized in that: The input terminal of the global sensing unit is connected to the third CSP module, and its output terminal is connected to the first input terminal of the receptive field expansion unit. The second input terminal of the receptive field expansion unit is connected to the output terminal of the third CSP module, and its output terminal is connected to the first input terminal of the convolution weighting unit. The second input of the convolution weighting unit is connected to the output of the third CSP module, and its output is connected to the first Detect module.

5. A method for detecting and recognizing small targets in remote sensing images based on deep learning, employing the remote sensing image small target detection and recognition system based on deep learning as described in any one of claims 1-4, characterized in that, Includes the following steps: Step 1: The first CBS module receives the initial remote sensing image and performs shallow feature extraction on it to obtain the first feature map, which is then sent to the first SPID module and the second CBCC module respectively. Step 2: The first SPID module performs staggered downsampling processing on the first feature map from spatial dimension to channel depth to obtain the second feature map, which is then sent to the first CSP module. Step 3: The first CSP module and the second CBS module sequentially perform deep feature extraction on the second feature map to obtain the sixth feature map. The second CBS module sends the sixth feature map to the second CSP module, the first CBCC module, and the third CBCC module respectively. The second CSP module performs deep feature extraction on the sixth feature map to obtain the third feature map. Step 4: The second SPID module performs staggered downsampling processing on the third feature map from spatial dimension to channel depth to obtain the fourth feature map, which is then sent to the SPPF module. Step 5: The SPPF module aggregates context information on the fourth feature map, and then the fifth feature map is obtained after processing by the fifth CBS module. It is then sent to the fourth CBCC module, and the fifth feature map is upsampled and sent to the third CBCC module. Step 6: The third CBCC module performs two-layer adaptive weighting of the fifth and sixth feature maps at the block and channel levels, and then processes them sequentially through the fifth CSP module and the fourth CBS module to obtain the seventh feature map, which is then sent to the first CBCC module. At the same time, the seventh feature map is upsampled and sent to the second CBCC module. Step 7: The second CBCC module performs two-layer adaptive weighting of the first feature map and the seventh feature map at the block level and the channel level, and then processes the twelfth feature map by the fourth CSP module and sends it to the third CBS module and the second MAGI module respectively. The third CBS module processes the twelfth feature map to obtain the eighth feature map, upsamples it and sends it to the first CBCC module. Step 8: The first CBCC module performs two-layer adaptive weighting of the sixth, seventh, and eighth feature maps at the block and channel levels, and then processes them through the third CSP module to obtain the tenth feature map, which is sent to the sixth CBS module and the first MAGI module respectively. The sixth CBS module processes the tenth feature map to obtain the ninth feature map, which is then sent to the fourth CBCC module. Step 9: The fourth CBCC module performs two-level adaptive weighting on the fifth and ninth feature maps at the block and channel levels, and then processes them through the sixth CSP module to obtain the eleventh feature map, which is then sent to the third MAGI module. Step 10: The first MAGI module, the second MAGI module, and the third MAGI module perform global large-view feature perception on the tenth feature map, the twelfth feature map, and the eleventh feature map, respectively, to obtain the thirteenth feature map, the fourteenth feature map, and the fifteenth feature map, and send them to the first Detect module, the second Detect module, and the third Detect module, respectively. Step 11: The first Detect module, the second Detect module, and the third Detect module detect and identify the thirteenth feature map, the fourteenth feature map, and the fifteenth feature map, respectively, and then perform a comprehensive calculation on the detection and identification results to obtain the detection matrix; Step 12: Combine the detection matrix with the initial remote sensing image to draw the remote sensing image detection map, thus completing the detection and recognition of small targets in remote sensing images based on deep learning.

6. The method for detecting and recognizing small targets in remote sensing images based on deep learning according to claim 5, characterized in that, Step 2 specifically includes: 2.1 The spatial dimension equal step size extraction unit extracts pixels from the first feature map with a preset step size and reassembles them to obtain a feature sub-map; 2.2 The channel-dimensional staggered stitching unit performs staggered stitching of all feature sub-maps from spatial dimension to channel dimension to form a stitched feature map; 2.3 The fully convolutional activation unit performs channel compression convolution on all the spliced ​​feature maps to obtain a second feature map with the same number of channels as the first feature map, and sends it to the first CSP module.

7. The method for detecting and recognizing small targets in remote sensing images based on deep learning according to claim 6, characterized in that, Step 6 specifically includes: 6.1 The block-level adaptive weighting unit uses an iterable block-weighting tensor to perform the first-layer block-level adaptive weighting on the fifth and sixth feature maps to obtain the block-level weighted feature maps; 6.2 The channel-level adaptive weighting unit uses an iterable channel weighting tensor to perform a second-layer channel-level adaptive weighting on the block-level weighted feature map. After processing by the fifth CSP module and the fourth CBS module, the seventh feature map is sent to the first CBCC module and then upsampled to the second CBCC module.

8. The method for detecting and recognizing small targets in remote sensing images based on deep learning according to claim 7, characterized in that, In step 10, the first MAGI module performs global large-view feature perception on the tenth feature map to obtain the thirteenth feature map, specifically including: The global perception unit extracts global long-range information from the tenth feature map through global attention and activation function operations; The receptive field expansion unit obtains local large field information of the tenth feature map through dilated convolution, and then fuses the local large field information and global long distance information through matrix alignment multiplication. After fusion, the receptive field expansion feature map is obtained by 1×1 convolution. After performing a 1×1 convolution on the tenth feature map, the weighted convolution unit fuses it with the receptive field expansion feature map through matrix positional multiplication. After fusion, the global large field-of-view perceptual feature map is obtained after activation function transformation. The global large field-of-view perceptual feature map and the tenth feature map are then processed with residuals to obtain the thirteenth feature map.

9. The method for detecting and recognizing small targets in remote sensing images based on deep learning according to claim 8, characterized in that, In step 6.1: The range of the iterable block-weighted tensor is 0 to 1.

10. The method for detecting and recognizing small targets in remote sensing images based on deep learning according to claim 9, characterized in that, In step 6.2: The range of the iterable channel-weighted tensor is 0 to 1.

Images