Aerial image camouflage target detection method and device based on multi-scale attention and boundary sensing network, equipment and medium

Abstract

The invention discloses an aerial image camouflage target detection method and device based on a multi-scale attention and boundary sensing network, equipment and a medium. A pyramid vision converter (PVT) is adopted as a trunk network, multi-scale features are extracted as attention-based feature extraction, and a high-resolution feature map is generated; a boundary sensing module (BAM) is developed, and the module aggregates local boundary information and global position information so as to fully mine boundary semantic information of the camouflage target; designing a boundary guidance module (BGM) which integrates the boundary features of the boundary sensing module with the multi-scale features of the backbone network to guide feature learning; a context aggregation module (CAM) is constructed, the module aggregates multi-layer fusion features, context semantic information of a camouflage target is fully mined by adopting extended convolution, and feature representation is further enhanced. According to the method, the problem of detecting the aerial image camouflage target is solved.

Description

Technical Field

[0001] The present invention relates to the field of camouflaged target detection in unmanned aerial vehicle (UAV) aerial images, and in particular to a method, device, equipment and medium for camouflaged target detection in aerial images based on a multi-scale attention and boundary perception network. Background Art

[0002] In recent years, drones have played an important role in target detection tasks due to their special imaging perspective. When drones conduct reconnaissance on targets, the targets usually camouflage themselves with the help of various complex backgrounds to avoid reconnaissance strikes. There is a high degree of texture similarity between the target and the complex background, making the target in the image highly camouflaged. Camouflaged target detection in drone aerial images is a valuable but challenging task that can provide technical support for accurate and preemptive strikes on real battlefields. However, camouflage technology has developed rapidly and has been widely used for key targets, making it difficult to effectively distinguish key targets that are extremely similar to the background.

[0003] Many deep learning-based methods have been used to detect camouflaged targets: some methods design advanced network structures to explore the discriminative features of camouflaged targets; some methods combine auxiliary tasks with target detection tasks to improve the performance of camouflaged target detection; some methods adopt bionic mechanisms inspired by the behavior of predators in nature to perform camouflaged target detection; some methods propose infrared and visible light image fusion algorithms to assist in camouflaged target detection.

[0004] Although the above methods have made significant progress, less attention has been paid to camouflaged target detection in UAV aerial imagery, and the following problems still exist: (1) The above methods mainly focus on the semantic segmentation of plants and animals rather than camouflaged target detection; (2) The above methods are all based on camouflaged target detection in natural images and do not involve UAV aerial scenes. Currently, there are almost no datasets designed specifically for camouflaged target detection in UAV aerial imagery; (3) When the camouflaged target does not generate heat to avoid reconnaissance, the infrared and visible light image fusion algorithm is ineffective; (4) The scale of the camouflaged target is variable, and they are usually located in complex backgrounds such as battlefields, jungles, and towns, and have a high texture similarity with the complex background. Summary of the invention

[0005] The purpose of the present invention is to provide a method, device, equipment and medium for detecting camouflaged targets in aerial images of unmanned aerial vehicles based on a multi-scale attention and boundary perception network, so as to realize the detection of camouflaged targets in aerial images of unmanned aerial vehicles.

[0006] The technical solution to achieve the purpose of the present invention is: a method for detecting camouflaged targets in aerial images based on a multi-scale attention and boundary perception network, comprising:

[0007] Input the aerial image, use the pyramid visual transformer PVT as the backbone network, extract features through PVT, and obtain the multi-scale feature map f of the backbone network i , i = {1, 2, 3, 4};

[0008] The boundary perception module BAM is used to fully mine the boundary semantic information of the camouflaged target from the bottom feature map f1 with local boundary information and the top feature map f4 with global position information to generate the boundary feature map f b ;

[0009] The boundary feature map f of the boundary perception module BAM is transformed into b Multi-scale feature map f with backbone network i Integration, generate fusion feature map f i a , to guide feature learning;

[0010] Construct a context aggregation module CAM to gradually aggregate multiple layers of fusion feature maps f in a top-down manner i a , i = {1, 2, 3, 4}, and obtain the output feature map f of the context aggregation module CAM i c , i = {1,2,3}, using 1×1 convolution to change f1 in the output feature map c The number of channels will be changed, and finally the output feature map f1 with the changed channel number will be c Input into the detection head to detect camouflaged targets in drone aerial images.

[0011] Furthermore, the method of using the boundary perception module BAM to fully mine the boundary semantic information of the camouflaged target from the bottom feature map f1 with local boundary information and the top feature map f4 with global position information is as follows:

[0012] First, two 1×1 convolutions are used to change the channels of f1 and f4 respectively to obtain f1' and f4'; then, the connection operation is used to aggregate f1' and f4'; finally, two 3×3 convolutions, one 1×1 convolution and Sigmoid function are used to aggregate the aggregated features of f1' and f4' to obtain the boundary feature map f b , which contains rich boundary semantic information of the camouflaged target.

[0013] Furthermore, the boundary feature map f of the boundary perception module BAM is transformed into b Multi-scale feature map f with backbone network i Integration, generate fusion feature map f i a The method is:

[0014] The boundary guidance module BGM is a multi-scale feature map f of the backbone network. i and the boundary feature map f b , and then a skip connection and a 3×3 convolution are used to obtain the fused feature map f. i b , expressed as:

[0015]

[0016] Where Conv3 represents 3×3 convolution, D represents downsampling, Represents element-wise multiplication; represents element-wise addition;

[0017] For the fusion feature map f i b The global average pooling GAP, one-dimensional convolution and Sigmoid function are used to obtain the fused feature map f i b The channel attention weight of

[0018] The channel attention weight is combined with f i b Multiply, use a 1×1 convolution to reduce the number of channels, and get the final output feature map f i a

[0019]

[0020] Among them, Conv1 represents 1×1 convolution, Conv1D k represents a one-dimensional convolution with kernel k, σ represents the Sigmoid function; the kernel size k is set to k = |1+log2(C)) / 2|, |·| represents an odd number operation, and C represents f i b The number of channels.

[0021] Furthermore, the context aggregation module CAM is constructed to gradually aggregate multiple layers of fusion feature maps f in a top-down manner. i a Get the output feature map f of the context aggregation module CAM i c The method is:

[0022] First of all, Upsample and Connect and obtain the aggregated feature map f through 1×1 convolution m3 , and then transform f in the channel dimension m3 Evenly divided into four feature maps The feature maps of adjacent branches are fused using dilated convolutions with dilation rates of {1, 2, 3, 4} respectively. Generate multi-scale context feature map Finally, a 1×1 convolution is performed on the multi-scale context feature map Connect them together, and the connected context feature map f m3 'After residual connection and a 3×3 convolution, the output feature map of the third context aggregation module is obtained right Upsample and Connect and obtain the aggregated feature map f through 1×1 convolution m2 , and then transform f in the channel dimension m2 Evenly divided into four feature maps The feature maps of adjacent branches are fused using dilated convolutions with dilation rates of {1, 2, 3, 4} respectively. Generate multi-scale context feature maps Finally, a 1×1 convolution is performed on the multi-scale context feature map Connect them together, and the connected context feature map f m2 'After residual connection and a 3×3 convolution, the output feature map of the second context aggregation module is obtained right Upsample and combine with f1 a Connect and obtain the aggregated feature map f through 1×1 convolution m1 , and then transform f in the channel dimension m1 Evenly divided into four feature maps The feature maps of adjacent branches are fused using dilated convolutions with dilation rates of {1, 2, 3, 4} respectively. Generate multi-scale context feature maps Finally, a 1×1 convolution is performed on the multi-scale context feature map Connect them together, and the connected context feature map f m1 'After residual connection and a 3×3 convolution, the output feature map f1 of the first context aggregation module is obtained c ; Finally, f1 c The output is sent to the detection head consisting of the classification branch and the regression branch to obtain the final disguised target detection result.

[0023] Furthermore, the detection head includes a classification branch and a regression branch, and the classification branch uses a binary cross entropy loss function to calculate the target classification loss L cls , the CIoU function is used in the regression branch to calculate the target bounding box loss L bbox .

[0024] A device for detecting camouflaged targets in aerial images based on a multi-scale attention and boundary perception network, the device comprising:

[0025] The feature extraction module inputs the aerial image and uses the pyramid visual transformer PVT as the backbone network. It extracts features through PVT and obtains the multi-scale feature map f of the backbone network. i , i = {1, 2, 3, 4};

[0026] The boundary perception module BAM fully mines the boundary semantic information of the camouflaged target from the bottom feature map f1 with local boundary information and the top feature map f4 with global position information to generate a boundary feature map f b ;

[0027] The boundary guidance module BGM uses the boundary guidance module BGM to transform the boundary feature map f of the boundary perception module BAM b Multi-scale feature map f with backbone network i i={1,2,3,4} integration, generate fusion feature map f i a , to guide feature learning;

[0028] The context aggregation module CAM gradually aggregates multiple layers of fusion feature maps f in a top-down manner i a , i = {1, 2, 3, 4}, and obtain the output feature map f of the context aggregation module CAM i c , i = {1,2,3}, using 1×1 convolution to change f1 in the output feature map c The number of channels will be changed, and finally the output feature map f1 with the changed channel number will be c Input into the detection head to detect camouflaged targets in drone aerial images.

[0029] Furthermore, the detection head includes a classification branch and a regression branch, and the classification branch uses a binary cross entropy loss function to calculate the target classification loss L cls , the CIoU function is used in the regression branch to calculate the target bounding box loss L bbox .

[0030] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the following steps are implemented:

[0031] Input the aerial image, use the pyramid visual transformer PVT as the backbone network, extract features through PVT, and obtain the multi-scale feature map f of the backbone network i , i = {1, 2, 3, 4};

[0032] The boundary perception module BAM is used to fully mine the boundary semantic information of the camouflaged target from the bottom feature map f1 with local boundary information and the top feature map f4 with global position information to generate the boundary feature map f b ;

[0033] The boundary feature map f of the boundary perception module BAM is transformed into b Multi-scale feature map f with backbone network i Integration, generate fusion feature map f i a , to guide feature learning;

[0034] Construct a context aggregation module CAM to gradually aggregate multiple layers of fusion feature maps f in a top-down manner i a , i = {1, 2, 3, 4}, and obtain the output feature map f of the context aggregation module CAM i c , i = {1,2,3}, using 1×1 convolution to change f1 in the output feature map c The number of channels will be changed, and finally the output feature map f1 with the changed channel number will be c Input into the detection head to detect camouflaged targets in drone aerial images.

[0035] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0036] Input the aerial image, use the pyramid visual transformer PVT as the backbone network, extract features through PVT, and obtain the multi-scale feature map f of the backbone network i , i = {1, 2, 3, 4};

[0037] The boundary perception module BAM is used to fully mine the boundary semantic information of the camouflaged target from the bottom feature map f1 with local boundary information and the top feature map f4 with global position information to generate the boundary feature map f b ;

[0038] The boundary feature map f of the boundary perception module BAM is transformed into b Multi-scale feature map f with backbone network i Integration, generate fusion feature map f i a , to guide feature learning;

[0039] Construct a context aggregation module CAM to gradually aggregate multiple layers of fusion feature maps f in a top-down manner i a, i = {1, 2, 3, 4}, and obtain the output feature map f of the context aggregation module CAM i c , i = {1,2,3}, using 1×1 convolution to change f1 in the output feature map c The number of channels will be changed, and finally the output feature map f1 with the changed channel number will be c Input into the detection head to detect camouflaged targets in drone aerial images.

[0040] Compared with the prior art, the present invention has the following significant advantages: (1) The method of the present invention is mainly based on aerial images generated by unmanned aerial vehicles, which solves the problem of detecting camouflaged targets. (2) The present invention adopts a pyramid visual transformer as the backbone network, extracts multi-scale features of the target, and generates a high-resolution feature map. (3) The present invention adopts a boundary extraction module to fully mine the boundary semantic information of the camouflaged target. (4) The present invention adopts a boundary guidance module to integrate the boundary features of the boundary extraction module with the multi-scale features of the backbone network to guide feature learning. (5) The present invention adopts a context fusion module to fully mine the context semantic information of the camouflaged target and further enhance the feature representation. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 This is the overall framework diagram of the multi-scale attention and boundary perception network of the present invention.

[0042] Figure 2 Schematic diagram of the boundary perception module of the present invention.

[0043] Figure 3 Schematic diagram of the boundary guidance module of the present invention.

[0044] Figure 4 It is the context aggregation module of the present invention.

[0045] Figure 5 Some example diagrams of the MCOD-UAV dataset in one embodiment of the present invention.

[0046] Figure 6 This is a qualitative comparison diagram on the MCOD-UAV dataset in one embodiment of the present invention, where (a) is C2FNet, (b) is SINet, (c) is TCPM, (d) is CPDN, ​​and (e) is MHNet.

[0047] Figure 7 The figure is a flow chart of a method for detecting camouflaged targets in aerial images based on a multi-scale attention and boundary perception network in one embodiment of the present invention. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0049] An embodiment of the method for detecting camouflaged targets in aerial images based on a multi-scale attention and boundary perception network is provided in the present application. The framework is as follows: Figure 1 As shown, the method flow is as follows Figure 7 As shown, including:

[0050] (1) The input image is subjected to feature extraction through the pyramid visual transformer (PVT) to obtain the multi-scale feature map f of the backbone network i (i={1,2,3,4}).

[0051] (2) The boundary awareness module (BAM) is used to fully mine the boundary semantic information of the camouflaged target from the bottom-level feature map f1 with local boundary information and the top-level feature map f4 with global position information to generate a boundary feature map f b .

[0052] (3) Use the boundary guidance module (BGM) to transform the boundary feature map f of the boundary perception module (BAM) b Multi-scale feature map f with backbone network i (i={1,2,3,4}) integration to generate fusion feature map f i a , to guide feature learning.

[0053] (4) Construct a context aggregation module CAM to gradually aggregate multiple layers of fusion feature maps f in a top-down manner i a , i = {1, 2, 3, 4}, and obtain the output feature map f of the context aggregation module CAM i c , i = {1,2,3}, using 1×1 convolution to change f1 in the output feature map c The number of channels will be changed, and finally the output feature map f1 with the changed channel number will be c Input into the detection head to detect camouflaged targets in drone aerial images.

[0054] The detection head consists of a classification branch and a regression branch. The classification branch uses a binary cross entropy loss function to calculate the target classification loss L cls , the CIoU function is used in the regression branch to calculate the target bounding box loss L bbox .

[0055] In one embodiment, in the method for detecting camouflaged targets in aerial images based on multi-scale attention and boundary perception network, in the boundary perception module, the bottom feature map f1 and the top feature map f4 from the backbone network are combined to fully mine the boundary semantic information of the camouflaged target, such as Figure 2 As shown. First, two 1×1 convolutions are used to change the channels of f1 and f4 respectively to obtain f1' and f4'. Then, the connection operation is used to aggregate f1' and f4'. Finally, two 3×3 convolutions, one 1×1 convolution and Sigmoid function are used to obtain the boundary feature map f after the aggregation of f1' and f4'. b , which contains rich boundary semantic information of the camouflaged target.

[0056] In one embodiment, a method for detecting camouflaged objects in aerial images based on a multi-scale attention and boundary-aware network is described. Figure 3 As shown in the figure, the boundary guidance module first builds on the multi-scale feature map f of the backbone network. i (i={1,2,3,4}) and boundary feature map f b , and then a skip connection and a 3×3 convolution are used to obtain the fused feature map f. i b , which can be expressed as:

[0057]

[0058] Where Conv3 represents 3×3 convolution, D represents downsampling, represents element-wise multiplication, Represents element-wise addition.

[0059] First, global average pooling (GAP) is used to aggregate the fusion feature map f i b Then, one-dimensional convolution and Sigmoid function are used to obtain the channel attention weight. Finally, the channel attention weight is combined with f i b Multiply, use a 1×1 convolution to reduce the number of channels, and get the final output feature map f i a .

[0060]

[0061] Among them, Conv1 represents 1×1 convolution, Conv1D k represents a one-dimensional convolution with kernel k, σ represents the Sigmoid function. The kernel size k is set to k = |(1+log2(C)) / 2|, |·| represents an odd number operation, and C represents f i bIn summary, the boundary guidance module captures key feature channels and suppresses redundant feature channels by introducing local channel attention, thereby enhancing the feature representation of the disguised target.

[0062] In one embodiment, in order to aggregate multi-level fusion features, a context aggregation module is designed to fully mine the contextual semantic information of the camouflaged target in an aerial image based on a multi-scale attention and boundary-aware network. Figure 4 As shown. and For example, first, Upsample and connect through 1×1 convolution to obtain the aggregated feature map f m Next, we transform f along the channel dimension. m Evenly divided into four feature maps Then, the extended convolution is used to aggregate the features of adjacent branches and extract the multi-scale context features of the camouflaged target. It can be expressed as:

[0063]

[0064] in Indicates the expansion rate is n j 3×3 dilated convolution. In the experiment, n is set j ={1,2,3,4}. Finally, the four multi-scale feature maps are convolved through a 1×1 convolution. Connect them together, and then use residual connection and a 3×3 convolution to obtain the final output feature map f of the context aggregation module. i c , which can be expressed as:

[0065]

[0066] where [·] is the residual connection operation, f i c is the output of the context aggregation module. It is worth noting that for i = {1, 2}, the previous context aggregation module The output will be the same as f i a Together as the input of the next context aggregation module to obtain f i c . Use 1×1 convolution to change f1 c The number of channels will be changed, and finally the output feature map f1 with the changed channel number will be c Input into the detection head to detect camouflaged targets in drone aerial images.

[0067] In one embodiment, a method for detecting camouflaged targets in aerial images based on a multi-scale attention and boundary-aware network is used to design a joint loss function for the output detection results, and a binary cross entropy loss function is used to obtain the target classification loss L. cls , use the CIoU function to get the target bounding box loss L bbox Total loss L all for:

[0068] L all =L cls +L bbox (5)

[0069] The target classification loss L is calculated using the binary cross entropy loss function cls , which forces the model to converge to the ground truth image at the pixel level.

[0070]

[0071]

[0072] Among them A 1 Determine whether there is an object in the bounding box (if so, set its value to 1, otherwise it will be set to 0), A 2 Determine if there is no object in the bounding box (if not, its value is set to 1, otherwise it is set to 0). α1 and α2 represent L cls The weight coefficient is . S×S represents the number of grids, N represents the number of bounding boxes generated for each grid, represents the ground truth value, Represents the predicted value.

[0073] Use CIoU function to calculate the target bounding box loss L bbox The CIoU function includes the overlapping area, aspect ratio and center point distance of the ground truth target bounding box and the predicted target bounding box, and is defined as:

[0074]

[0075] Among them, bb and bb r are the predicted target bounding box and the ground truth target bounding box, ww, hh, ww r and hh r are the width and height of the predicted target bounding box and the ground truth target bounding box, respectively. d represents the center point distance between the predicted target bounding box and the ground truth target bounding box, dd represents the length of the farthest diagonal line connecting the predicted target bounding box and the ground truth target bounding box, and α represents the weight coefficient.

[0076] In one embodiment, a device for detecting camouflaged targets in aerial images based on a multi-scale attention and boundary perception network is provided, comprising:

[0077] The feature extraction module inputs the aerial image and uses the pyramid visual transformer PVT as the backbone network to extract features through PVT to obtain the multi-scale backbone feature f i , i = {1, 2, 3, 4};

[0078] The boundary perception module BAM fully mines the boundary semantic information of the camouflaged target from the bottom feature map f1 with local boundary information and the top feature map f4 with global position information to generate the boundary feature f b ;

[0079] The boundary guidance module BGM uses the boundary guidance module BGM to transform the boundary feature f of the boundary perception module BAM b Multi-scale features f of the backbone network i i={1,2,3,4} integration, generate fusion feature f i a , to guide feature learning;

[0080] The context aggregation module CAM gradually aggregates multiple layers of fusion features f in a top-down manner. i a , i = {1, 2, 3, 4}, and obtain the output feature map f of the context aggregation module CAM i c , i = {1,2,3}, using 1×1 convolution to change f1 in the output feature map c The number of channels will be changed, and finally the output feature map f1 with the changed channel number will be c Input into the detection head to detect camouflaged targets in drone aerial images.

[0081] In one embodiment, the boundary perception module BAM first uses two 1×1 convolutions to change the channels of f1 and f4 respectively; then, a connection operation is applied to aggregate f1' and f4'; finally, two 3×3 convolutions, one 1×1 convolution and a Sigmoid function are used to obtain the boundary feature f b , which contains rich boundary semantic information of the camouflaged target.

[0082] In one embodiment, the context aggregation module CAM Upsample and connect through 1×1 convolution to obtain the aggregated feature f m ;

[0083] Along the channel dimension, f m Evenly divided into four feature maps

[0084] The extended convolution is used to aggregate the features of adjacent branches and extract the multi-scale context features of the camouflaged target, which can be expressed as:

[0085]

[0086] in Indicates the expansion rate is n j 3×3 dilated convolution; in the experiment, n is set j ={1,2,3,4}; four multi-scale features are transformed into Connect them together, and then go through a residual connection and a 3×3 convolution to obtain the final output f of the context aggregation module. i c , expressed as:

[0087]

[0088] where [·] is the residual connection operation, f i c is the output of the context aggregation module; for i = {1, 2}, the previous context aggregation module The output will be the same as f i a Together as the input of the next context aggregation module to obtain f i c ; Use 1×1 convolution to change f1 c The number of channels is calculated, and finally the detection result of the camouflaged target is obtained.

[0089] In one embodiment, a computer device is provided, which may be a server. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data required for the detection of camouflaged targets in aerial images based on a multi-scale attention and boundary perception network. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a method for detecting camouflaged targets in aerial images based on a multi-scale attention and boundary perception network is implemented.

[0090] Those skilled in the art will appreciate that the above structure does not constitute a limitation on the computer device to which the present application is applied, and a specific computer device may include more or fewer components than those described above, or combine certain components, or have a different arrangement of components.

[0091] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0092] Input the aerial image, use the pyramid visual transformer PVT as the backbone network, extract features through PVT, and obtain the multi-scale feature map f of the backbone network i , i = {1, 2, 3, 4};

[0093] The boundary perception module BAM is used to fully mine the boundary semantic information of the camouflaged target from the bottom feature map f1 with local boundary information and the top feature map f4 with global position information to generate the boundary feature f b ;

[0094] The boundary feature f of the boundary perception module BAM is transformed into b Multi-scale features f of the backbone network i i={1,2,3,4} integration, generate fusion feature f i a , to guide feature learning;

[0095] Construct a context aggregation module CAM to gradually aggregate multiple layers of fusion features f in a top-down manner i a , i = {1,2,3,4}, and finally detect the camouflaged targets in the UAV aerial images.

[0096] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0097] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0098] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

[0099] The effect of the aerial image camouflage target detection method based on multi-scale attention and boundary perception network of the present invention is verified by experiments below.

[0100] Datasets play an important role in camouflaged object detection based on deep learning. However, currently popular camouflaged object detection datasets mainly focus on the semantic segmentation of plants and animals, and there are almost no datasets designed specifically for camouflaged object detection in drone aerial images.

[0101] In order to create a dataset for camouflaged target detection in UAV aerial images, a large number of camouflaged images captured by UAVs were collected by setting multiple keywords on the Internet. In addition, 1,000 tank targets hidden in complex environments were collected for camouflaged target detection in UAV aerial images. In summary, the MCOD-UAV dataset includes four categories: soldiers, aircraft, military vehicles, and tanks, and its camouflage involves various real-life scenes such as battlefields, jungles, and towns.

[0102] After the image acquisition is completed, image processing and manual filtering are used to eliminate low-quality images. The collected images are filtered by taking into account multiple scenes, directions and angles, as well as target size, clarity, occlusion degree and other factors. Images with serious visual problems due to motion blur, low resolution, no camouflaged targets and high repetition are deleted before annotation. Finally, after rigorous screening, 3,000 camouflaged images were retained to ensure the quality of the MCOD-UAV dataset, and each image contained 1-5 camouflaged targets. Subsequently, each image was carefully annotated with the target category and a tightly wrapped rectangular box. Obviously, the MCOD-UAV dataset is suitable for camouflaged target detection in UAV aerial images, and can provide data support for camouflaged target detection in UAV aerial images. Some examples of the MCOD-UAV dataset are as follows. Figure 5 shown.

[0103] In order to demonstrate the camouflaged target detection effect of the scheme of the present invention, extensive comparative experiments were conducted on the MCOD-UAV dataset. In order to quantitatively estimate the camouflaged target detection effect of different network models, the mean average precision (mAP) was used to evaluate the target detection results.

[0104] Dataset: The camouflaged target detection effect of multi-scale attention and boundary-aware network is evaluated on the MCOD-UAV dataset. The MCOD-UAV dataset is divided into training set and test set in a ratio of 8:2. The images in the training set are expanded by 5 times through classic data enhancement strategies such as random flipping and rotation. Therefore, there are 12,000 images for training and the remaining 600 images for testing.

[0105] Implementation details: The PyTorch framework is used to implement the multi-scale attention and boundary-aware networks. All experiments are trained and tested on images of a single scale on an NVIDIA RTX 3090Ti GPU with 24GB of memory. During the training phase, all input images are resized to 1000×600 pixels to balance efficiency and performance. Stochastic gradient descent (SGD) with momentum and weight decay initialized to 0.9 and 0.0001 is used as the optimizer, and the learning rate is initialized to 0.0025 and follows a linear decay strategy. The maximum training epoch is set to 12 and the batch size is set to 4.

[0106] Comparison with state-of-the-art image deblurring methods

[0107] The multi-scale attention and boundary-aware networks are quantitatively and qualitatively studied with existing camouflaged object detection methods on the MCOD-UAV dataset.

[0108] (1) Quantitative Analysis: We quantitatively compare the MSCAB with existing camouflaged object detection methods. As shown in Table 1, the MSCAB achieves superior performance in terms of mAP and significantly outperforms other methods. Specifically, the MSCAB increases the mAP by 2.27% over the second-best method on the MCOD-UAV dataset and has the highest mAP for all four object categories.

[0109] Table 1. Quantitative comparison on the MCOD-UAV dataset

[0110]

[0111] (2) Qualitative analysis: The multi-scale attention and boundary-aware network is qualitatively compared with existing camouflaged object detection methods. Figure 6 As shown in Figure 2, the multi-scale attention and boundary-aware network can accurately detect targets in various camouflage environments, such as indistinguishable backgrounds, similar colors, and dense occlusions. Figure 6 In the first column, for a camouflaged target that is difficult to distinguish from the surrounding background, the multi-scale attention and boundary-aware network can accurately determine the outline boundary of the sniper and then detect the sniper. Figure 6 In the second column, for camouflaged targets with similar colors to the surroundings, the multi-scale attention and boundary-aware network can accurately match the shape of the aircraft and then detect the aircraft with similar colors to the surroundings. Figure 6 In the third column, for densely occluded camouflaged targets, the multi-scale attention and boundary-aware network can accurately detect the tank densely occluded by trees.

Claims

1. A method for detecting camouflaged targets in aerial images based on multi-scale attention and boundary-aware networks, characterized in that: include: Input the aerial image, use the pyramid visual transformer PVT as the backbone network, extract features through PVT, and obtain the multi-scale feature map f of the backbone network i , i = {1, 2, 3, 4}; The boundary perception module BAM is used to fully mine the boundary semantic information of the camouflaged target from the bottom feature map f1 with local boundary information and the top feature map f4 with global position information to generate the boundary feature map f b ; The boundary feature map f of the boundary perception module BAM is transformed into b Multi-scale feature map f with backbone network i Integration, generating fusion feature map To guide feature learning; Construct a context aggregation module CAM to gradually aggregate multiple layers of fusion feature maps in a top-down manner i={1,2,3,4}, and get the output feature map of the context aggregation module CAM i={1,2,3}, using 1×1 convolution to change the output feature map The number of channels will be changed, and finally the output feature map of the number of channels will be changed Input into the detection head to detect camouflaged targets in drone aerial images.

2. The method for detecting camouflaged targets in aerial images based on multi-scale attention and boundary-aware networks according to claim 1 is characterized in that: The method of using the boundary perception module BAM to fully mine the boundary semantic information of the camouflaged target from the bottom feature map f1 with local boundary information and the top feature map f4 with global position information is as follows: First, two 1×1 convolutions are used to change the channels of f1 and f4 respectively to obtain f1' and f4'; then, the connection operation is used to aggregate f1' and f4'; finally, two 3×3 convolutions, one 1×1 convolution and Sigmoid function are used to aggregate the aggregated features of f1' and f4' to obtain the boundary feature map f b , which contains rich boundary semantic information of the camouflaged target.

3. The method for detecting camouflaged targets in aerial images based on multi-scale attention and boundary-aware networks according to claim 1 is characterized in that: The boundary feature map f of the boundary perception module BAM is transformed by using the boundary guidance module BGM. b Multi-scale feature map f with backbone network i Integration, generating fusion feature map The method is: The boundary guidance module BGM is in the multi-scale feature map f of the backbone network. i and the boundary feature map f b Element-wise multiplication is performed between them, and then a skip connection and a 3×3 convolution are used to obtain the fused feature map. It is expressed as: Where Conv3 represents 3×3 convolution, D represents downsampling, Represents element-wise multiplication; represents element-wise addition; Fusion feature map The fused feature map is obtained by using global average pooling GAP, one-dimensional convolution and Sigmoid function. The channel attention weight of The channel attention weights are Multiply and use a 1×1 convolution to reduce the number of channels to get the final output feature map Among them, Conv1 represents 1×1 convolution, Conv1D k represents a one-dimensional convolution with kernel k, σ represents the Sigmoid function; the kernel size k is set to k = |(1+log2(C)) / 2|, |·| represents an odd number operation, and C represents The number of channels.

4. The method for detecting camouflaged targets in aerial images based on multi-scale attention and boundary-aware networks according to claim 1, characterized in that: The context aggregation module CAM is constructed to gradually aggregate multiple layers of fusion feature maps in a top-down manner. Get the output feature map of the context aggregation module CAM The method is: First of all, Upsample and Connect and obtain the aggregated feature map f through 1×1 convolution m3 , and then transform f in the channel dimension m3 Evenly divided into four feature maps The feature maps of adjacent branches are fused using dilated convolutions with dilation rates of {1, 2, 3, 4} respectively. Generate multi-scale context feature maps Finally, a 1×1 convolution is performed on the multi-scale context feature map Connect them together, and the connected context feature map f m3 'After residual connection and a 3×3 convolution, the output feature map of the third context aggregation module is obtained right Upsample and combine with f2 a Connect and obtain the aggregated feature map f through 1×1 convolution m2 , and then transform f in the channel dimension m2 Evenly divided into four feature maps The feature maps of adjacent branches are fused using dilated convolutions with dilation rates of {1, 2, 3, 4} respectively. Generate multi-scale context feature maps Finally, a 1×1 convolution is performed on the multi-scale context feature map Connect them together, and the connected context feature map f m2 'After residual connection and a 3×3 convolution, the output feature map of the second context aggregation module is obtained right Upsample and combine with f1 a Connect and obtain the aggregated feature map f through 1×1 convolution m1 , and then transform f in the channel dimension m1 Evenly divided into four feature maps The feature maps of adjacent branches are fused using dilated convolutions with dilation rates of {1, 2, 3, 4} respectively. Generate multi-scale context feature maps Finally, a 1×1 convolution is performed on the multi-scale context feature map Connect them together, and the connected context feature map f m1 'After residual connection and a 3×3 convolution, the output feature map of the first context aggregation module is obtained at last, The output is sent to the detection head consisting of the classification branch and the regression branch to obtain the final disguised target detection result.

5. The method for detecting camouflaged targets in aerial images based on multi-scale attention and boundary-aware networks according to claim 1, characterized in that: The detection head includes a classification branch and a regression branch. The classification branch uses a binary cross entropy loss function to calculate the target classification loss L cls , the CIoU function is used in the regression branch to calculate the target bounding box loss L bbox .

6. A device for detecting camouflaged targets in aerial images based on a multi-scale attention and boundary-aware network, characterized in that: The device comprises: The feature extraction module inputs the aerial image and uses the pyramid visual transformer PVT as the backbone network. It extracts features through PVT and obtains the multi-scale feature map f of the backbone network. i , i = {1, 2, 3, 4}; The boundary perception module BAM fully mines the boundary semantic information of the camouflaged target from the bottom feature map f1 with local boundary information and the top feature map f4 with global position information to generate a boundary feature map f b ; The boundary guidance module BGM uses the boundary guidance module BGM to transform the boundary feature map f of the boundary perception module BAM b Multi-scale feature map f with backbone network i i={1,2,3,4} integration, generate fusion feature map To guide feature learning; Context aggregation module CAM, which gradually aggregates multiple layers of fusion feature maps in a top-down manner i={1,2,3,4}, and get the output feature map of the context aggregation module CAM i={1,2,3}, using 1×1 convolution to change the output feature map The number of channels will be changed, and finally the output feature map f1 with the changed channel number will be c Input into the detection head to detect camouflaged targets in drone aerial images.

7. The aerial image camouflage target detection device based on multi-scale attention and boundary perception network according to claim 6 is characterized by: The detection head includes a classification branch and a regression branch. The classification branch uses a binary cross entropy loss function to calculate the target classification loss L cls , the CIoU function is used in the regression branch to calculate the target bounding box loss L bbox .

8. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images