A multispectral unmanned aerial vehicle aerial target detection model optimization method and device

Abstract

This invention discloses a method and apparatus for optimizing a multispectral UAV aerial target detection model, belonging to the field of target detection technology. The method includes: acquiring a multispectral image dataset and preprocessing the multispectral images in the dataset to reduce the dimensionality of the multispectral images and convert the annotations of the multispectral images into a preset form; constructing a target detection model; wherein the target detection model includes a multi-scale contextual feature enhancement module and a convolutional block attention module connected in series; and training the target detection model using the preprocessed multispectral image dataset. This invention is applicable to the task of detecting small targets in multispectral UAV aerial photography scenarios and can effectively cope with complex challenges such as low light, low visibility, cluttered backgrounds, and occlusion.

Description

Technical Field

[0001] This invention relates to the field of target detection technology, and in particular to a method and apparatus for optimizing a multispectral unmanned aerial vehicle (UAV) target detection model. Background Technology

[0002] In the field of drone aerial photography and remote sensing monitoring, the detection of weak targets based on multispectral images is of great significance and is widely used in scenarios such as agricultural monitoring, disaster assessment, and military reconnaissance.

[0003] However, traditional object detection models still face significant challenges in this task, especially in detecting small targets under multispectral, complex background, and low signal-to-noise ratio conditions, exhibiting three main limitations. First, traditional models have significant shortcomings in the feature extraction stage. These models typically rely on a single scale or a limited receptive field for feature extraction, making it difficult to adapt to the large scale variations of targets in UAV aerial images. For small targets, their low pixel proportion and weak visual saliency make it difficult to simultaneously and effectively preserve semantic information and spatial details, leading to the gradual loss of key features in deep networks and affecting detection reliability. Second, small targets inherently possess characteristics such as low contrast, blurred edges, and varied shapes. In complex natural backgrounds or high-texture areas, target signals are easily submerged by background noise. Existing models often assign the same weight to all feature channels or spatial locations, lacking enhancement mechanisms for key features and failing to effectively suppress redundant background information, resulting in insufficient target feature response and prominent false negatives and false negatives. Furthermore, multispectral data contains rich band information, exhibiting complementarity under different lighting, occlusion, and weather conditions. However, most traditional detection models are designed for RGB images and do not fully consider the cross-band correlation and differential representation capabilities of multispectral data, resulting in poor robustness of the models in complex scenes such as low light and shadow occlusion, and significant fluctuations in detection performance.

[0004] Specifically, the main shortcomings of existing technologies can be summarized into the following three points: 1. Limitations of feature extraction mechanism: Features extracted by a single or limited receptive field cannot fully adapt to the characteristics of multi-scale, especially very small targets, and it is difficult to capture semantic information and spatial details at the same time, resulting in insufficient effective feature representation of weak targets.

[0005] 2. Imbalance in feature fusion and weight allocation: The lack of targeted feature selection and enhancement mechanisms in the network limits its ability to distinguish key target features from background information, causing target features to be diluted during the fusion process and reducing detection sensitivity.

[0006] 3. Insufficient utilization of multispectral information: The complementary and differentiated features between multispectral bands are not effectively explored, and there is a lack of cross-band feature fusion and adaptation mechanisms, resulting in insufficient generalization ability and robustness of the model under complex imaging conditions. Summary of the Invention

[0007] This invention provides a method and apparatus for optimizing a multispectral UAV aerial target detection model, in order to solve the technical problems of insufficient effective feature representation of small targets, low detection sensitivity, and insufficient generalization ability and robustness of the model under complex imaging conditions in the existing technology.

[0008] To solve the above-mentioned technical problems, the present invention provides the following technical solution: On one hand, this invention provides a method for optimizing a multispectral UAV aerial target detection model, including: A multispectral image dataset is acquired, and the multispectral images in the dataset are preprocessed to reduce the dimensionality of the multispectral images and convert the annotations of the multispectral images into a preset form. Construct an object detection model; wherein the object detection model includes a multi-scale context feature enhancement module and a convolutional block attention module connected in series; The target detection model is trained using the preprocessed multispectral image dataset.

[0009] Furthermore, the multispectral images in the multispectral image dataset are preprocessed to reduce the dimensionality of the multispectral images and convert the annotations of the multispectral images into a preset form, including: Principal component analysis is used to reduce the dimensionality of multispectral images in a multispectral image dataset, mapping the original multiband data to a three-dimensional feature space, and selecting the first three principal components to construct pseudo-RGB images of the corresponding multispectral images; wherein, the multispectral image dataset is the MODA dataset; For the rotated bounding box annotations in the MODA dataset, we convert them into COCO format bounding box annotations using coordinate transformation and the minimum bounding horizontal rectangle method. The preprocessed multispectral image dataset consists of pseudo-RGB images with horizontal bounding boxes in COCO format.

[0010] Furthermore, the target detection model includes a cascaded backbone network, a feature pyramid network, a multi-scale contextual feature enhancement module, and a convolutional block attention module; wherein, after the input image is processed sequentially through the backbone network, the feature pyramid network, the multi-scale contextual feature enhancement module, and the convolutional block attention module, the resulting feature map is processed by a region proposal network and region of interest pooling, and then classified and bounding box regressed by the detection head.

[0011] Furthermore, the backbone network is a ResNet50 network.

[0012] Furthermore, the data processing procedure of the multi-scale contextual feature enhancement module is as follows: For the input feature map, three 3×3 convolution kernels with different dilation rates are used for convolution operations to obtain feature information of local small targets, medium-sized targets and large targets respectively, and output the first feature map F1, the second feature map F2 and the third feature map F3 accordingly; F1, F2 and F3 are concatenated in the channel dimension, and the enhanced feature map is obtained by dimensionality reduction through 1×1 convolution, and then residual connection is performed with the input feature map to avoid information loss and promote network training, and finally the enhanced feature map is output.

[0013] Furthermore, the data processing procedure of the convolutional block attention module is as follows: Global average pooling and global max pooling are performed on the input feature map in the spatial dimension to obtain two types of channel description vectors. After nonlinear mapping by a multilayer perceptron with shared weights, channel attention weights are generated by the Sigmoid function, and the feature map is weighted channel by channel. The channel-weighted feature maps are subjected to average pooling and max pooling in the channel dimension, respectively. After concatenation, spatial attention weights are generated through a 7×7 convolutional layer to spatially weight the feature maps and highlight the features of the target region.

[0014] Furthermore, the multilayer perceptron includes two fully connected layers with ReLU activation function.

[0015] Furthermore, training the target detection model using the preprocessed multispectral image dataset includes: The preprocessed multispectral image dataset is input into the target detection model, and the model is trained using a stochastic gradient descent optimizer. The model parameters are optimized through multiple iterations.

[0016] Furthermore, when training the model, the initial learning rate was set to 0.0025, momentum to 0.9, training epochs to 24, and batch size to 4; a step decay strategy was adopted, with the learning rate decayed by 0.1 at the 16th and 22nd epochs.

[0017] On the other hand, the present invention also provides a multispectral UAV aerial target detection model optimization device, the multispectral UAV aerial target detection model optimization device comprising: The dataset preprocessing module is used to acquire a multispectral image dataset and preprocess the multispectral images in the multispectral image dataset to reduce the dimensionality of the multispectral images and convert the annotations of the multispectral images into a preset form. The model improvement module is used to construct an object detection model; wherein, the object detection model includes a multi-scale context feature enhancement module and a convolutional block attention module connected in series. The training module is used to train the object detection model using the preprocessed multispectral image dataset.

[0018] In another aspect, the present invention also provides an electronic device comprising a processor and a memory; wherein the memory stores at least one instruction, which is loaded and executed by the processor to implement the above-described method.

[0019] In another aspect, the present invention also provides a computer-readable storage medium storing at least one instruction, which is loaded and executed by a processor to implement the above-described method.

[0020] The beneficial effects of the technical solution provided by this invention include at least the following: This invention enhances the model's ability to represent features of multi-scale targets and its ability to focus on salient features of weak targets by designing a cascaded structure of a multi-scale contextual feature enhancement module (MCEM) and a convolutional block attention module (CBAM). Furthermore, by preprocessing multispectral data using principal component analysis (PCA) and combining this with the improved model architecture, it achieves efficient and accurate detection of weak targets in multispectral UAV aerial images, significantly improving detection accuracy and reducing false detection rate, making it suitable for multispectral target detection tasks in complex scenarios. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of the execution flow of the multispectral UAV aerial target detection model optimization method provided in an embodiment of the present invention; Figure 2 This is the overall architecture diagram of the model provided in the embodiment of the present invention (feature pyramid network, MCEM module, CBAM module in series structure); Figure 3 This is a schematic diagram of the MCEM module structure (multi-hole rate convolution, feature splicing, residual connection) provided in an embodiment of the present invention. Figure 4 This is a schematic diagram of the CBAM module structure (channel attention module and spatial attention module connected in series) provided in an embodiment of the present invention. Figure 5 This is a flowchart of the PCA preprocessing of multispectral data provided in an embodiment of the present invention; Figure 6This is a pseudo-RGB image after PCA processing provided in this embodiment of the invention; wherein, (a) is the explained variance of PC1 (primary luminance / information): 0.757; (b) is the explained variance of PC2 (secondary contrast): 0.207; (c) is the explained variance of PC3 (detail): 0.022; and (d) is PCA pseudo-color synthesis (R:G:B corresponds to PC1:P2:PC3). Figure 7 This is a comparison image of the MODA dataset rotated bounding box annotations and horizontal bounding box annotations provided in this embodiment of the invention; wherein, (a) is the rotated bounding box annotation for a car; (b) is the rotated bounding box annotation for a truck; (c) is the rotated bounding box annotation for a pedestrian; (d) is the horizontal bounding box annotation for a car; (e) is the horizontal bounding box annotation for a truck; and (f) is the horizontal bounding box annotation for a pedestrian. Figure 8 This is a comparison chart of the PR curves of Faster R-CNN, Deformable-DETR, RetinaNet, YOLOv8 and the model of this invention provided in the embodiments of this invention; Figure 9 This is a system block diagram of the electronic device provided in the embodiments of the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0024] First, it should be noted that in the embodiments of the present invention, the words "exemplarily," "for example," etc., are used to indicate that they are examples, illustrations, or descriptions. Any embodiment or design scheme described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the term "exemplarily" is intended to present the concept in a specific manner. Furthermore, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either one or the other.

[0025] First Embodiment This embodiment provides an optimization method for a multispectral UAV aerial target detection model. Through three core stages—data preprocessing, feature enhancement and attention weighting, and model training and inference—it achieves high-quality detection of small targets in complex scenes. It is applicable to small target detection tasks in multispectral UAV aerial photography scenarios and can effectively address complex challenges such as low light, low visibility, cluttered backgrounds, and occlusion.

[0026] This method can be implemented by an electronic device, which can be a terminal or a server. The execution flow of this method is as follows: Figure 1 As shown, it includes the following steps:

[0027] S1, acquire a multispectral image dataset, and preprocess the multispectral images in the multispectral image dataset to reduce the dimensionality of the multispectral images and convert the annotations of the multispectral images into a preset form; Specifically, in this embodiment, such as Figure 5 As shown, the implementation process of S1 above is as follows: S11, Multispectral Image Dimensionality Reduction, as detailed below: Principal Component Analysis (PCA) is used to reduce the dimensionality of multispectral images in the multispectral image dataset. An orthogonal transformation is performed on the original band data of the multispectral images to generate a set of linearly uncorrelated principal components. The original multispectral data is mapped to a three-dimensional feature space, the covariance matrix of the original data is calculated, and the eigenvalues ​​and eigenvectors are solved. The principal components are arranged in descending order of eigenvalues, and the three principal components with the largest variances are selected to construct pseudo-RGB images of the corresponding multispectral images. Figure 6 As shown, redundant information is removed while retaining over 95% of the spectral information. Specifically, for the MODA dataset: the original spectral range is 395-950nm, containing 8 bands, and after dimensionality reduction, a pseudo-RGB image of 1280×960 pixels is generated; for the HOD3K dataset: the original spectral range is 470-620nm, containing 16 bands, and after dimensionality reduction, a pseudo-RGB image of 512×256 pixels is generated. The spectral information retention rates for the MODA dataset training and test sets exceed 98.6% and 98.8%, respectively.

[0028] S12, annotation format conversion, details are as follows: For the MODA dataset, the original annotations are rotated bounding boxes (four-point coordinates, arranged counterclockwise), with the origin of the coordinate system located at the top left corner of the image, the x-axis corresponding to the image height H, and the y-axis corresponding to the image width W. During transformation, coordinate transformation and the minimum bounding rectangle method are used to convert them into COCO format horizontal bounding box annotations, ensuring compatibility with traditional object detection networks, such as... Figure 7 As shown.

[0029] S13, Data partitioning, as detailed below: The preprocessed dataset consists of pseudo-RGB images with horizontal bounding boxes in COCO format. For the MODA dataset, the training set contains 9156 images and the test set contains 4885 images; for the HOD3K dataset, which contains 3242 images, they are divided into training and test sets proportionally.

[0030] S2, Construct an object detection model; wherein, the object detection model includes a multi-scale contextual feature enhancement module and a convolutional block attention module connected in series; The object detection model in this embodiment can be obtained by improving upon the existing Faster RCNN model.

[0031] Specifically, in this embodiment, the target detection model is constructed as follows: A multi-scale contextual feature enhancement module (MCEM) and a convolutional block attention module (CBAM) are connected in series on the feature pyramid network; that is, the MCEM module and the CBAM module are connected in series between the feature extraction network and the detection head. The MCEM module obtains multi-scale receptive fields through dilated convolutions with different dilation rates, and enhances feature representation by combining feature concatenation and residual connections. The CBAM module adaptively weights the feature map through channel attention and spatial attention mechanisms, highlighting key target features and suppressing background interference. Finally, the target detection model constructed in this embodiment is as follows: Figure 2 As shown, the system includes a cascaded backbone network, a feature pyramid network, a multi-scale contextual feature enhancement module, and a convolutional block attention module. The input image is processed sequentially through the backbone network, feature pyramid network, multi-scale contextual feature enhancement module, and convolutional block attention module. The resulting feature map is then processed by a region proposal network and region of interest pooling, and finally classified and bounding box regressed by a detection head. The backbone network is a ResNet50 network.

[0032] like Figure 3 As shown, the innovative design of the Multi-Scale Contextual Feature Enhancement Module (MCEM) is as follows: (1) Construction of multi-scale receptive fields: For the input feature map F, convolution operation is performed using 3×3 convolution kernels with three different dilation rates (1, 2, 3) to obtain feature information of local small targets, medium-sized targets and large targets respectively, and output feature maps F1, F2 and F3 respectively.

[0033] (2) Feature fusion and residual connection: F1, F2, and F3 are concatenated along the channel dimension, and then dimensionality-reduced using a 1×1 convolution to obtain an enhanced feature map. This enhanced feature map is then residually connected to the input feature map to avoid information loss and promote network training, ultimately outputting the enhanced feature map F'. The specific implementation steps for feature map F can be expressed by the following formula:

[0034] Where F1, F2, and F3 represent the convolution outputs under different dilation rates.

[0035] in, Represents the characteristics after splicing. This represents the features after dimensionality reduction.

[0036] Where F is the input feature map, It is the enhanced output feature map.

[0037] (3) Lightweight design: By using dilated convolutions, the receptive field can be expanded without increasing the number of parameters and computational cost, ensuring lightweight modules and not significantly improving model inference time.

[0038] like Figure 4 As shown, the core design of the Convolutional Block Attention Module (CBAM) is as follows: (1) Channel attention mechanism: The input feature map is subjected to global average pooling and global max pooling in the spatial dimension to obtain two types of channel description vectors. After nonlinear mapping by a multilayer perceptron (MLP) with shared weights, channel attention weights are generated by the sigmoid function and applied to the feature map channel by channel. The channel attention MLP contains two fully connected layers (the number of channels is reduced in dimensionality according to the reduction ratio and then increased in dimensionality to the original size), and the activation function is ReLU.

[0039] (2) Spatial attention mechanism: The channel-weighted feature maps are subjected to average pooling and max pooling in the channel dimension, respectively. After concatenation, spatial attention weights are generated through a 7×7 convolutional layer to spatially weight the feature maps and highlight the features of the target region.

[0040] (3) Dual-weighted synergy: The channel attention and spatial attention modules are executed in series to sequentially distinguish the importance of feature channels and model the saliency of spatial location, thereby improving feature discrimination ability while maintaining low computational complexity.

[0041] For the input feature map F, the specific implementation steps of the channel attention part are as follows: in, The overall statistical characteristics of the representative feature map. The most salient feature of the feature map.

[0042] in, and These represent the features after MLP mapping. This represents the weights of the first fully connected layer. This indicates the weights of the second fully connected layer. This represents the ReLU activation function.

[0043] in, Represents channel attention weight. Represents the Sigmoid function. This represents the output characteristics of the channel attention module. This indicates a channel-by-channel multiplication operation.

[0044] Following the channel attention module, a spatial attention mechanism is introduced to model the spatial saliency of the feature map. The specific steps are as follows:

[0045] in, To reflect the characteristics of average activation level, This represents a characteristic that reflects the maximum degree of activation.

[0046] in, Indicates the features after splicing. Represents spatial attention weights. This represents the Sigmoid function.

[0047] in, This indicates the output characteristics of the CBAM module. This indicates a channel-by-channel multiplication operation.

[0048] S3, The target detection model is trained using the preprocessed multispectral image dataset; Specifically, in this embodiment, the implementation process of S3 is as follows: The preprocessed dataset is input into the improved model, which is trained using a stochastic gradient descent (SGD) optimizer. The model parameters are then optimized through multiple iterations. The trained model can then be used to achieve multispectral UAV aerial target detection. Specifically, during the inference phase, a pseudo-RGB image of the multispectral UAV aerial image to be detected is first acquired. This pseudo-RGB image is then input into the trained model. The model performs feature extraction, attention weighting, and target detection on the input pseudo-RGB image, outputting detection results such as target category and bounding box coordinates, achieving accurate identification of multi-scale, small targets.

[0049] Specifically, the loss function is a weighted sum of classification loss (cross-entropy loss) and regression loss (smoothing L1 loss), which are commonly used in object detection tasks.

[0050] The model training parameters were as follows: SGD optimizer was used, with an initial learning rate of 0.0025, momentum of 0.9, and weight decay coefficients adjusted experimentally; 24 training epochs were used, with a batch size of 4; a step decay strategy was adopted, decreasing the learning rate by 0.1 at epochs 16 and 22; the training hardware environment consisted of an Intel Core i7-10700K processor and an NVIDIA RTX 3090 graphics card, and the software environment included Python 3.9.25, PyTorch 2.0.1, and CUDA 11.8. Performance was evaluated on the validation set after each training epoch, and the model weights with the lowest validation loss were saved as the optimal model. After training, the mAP performance of the optimal model on the validation set for each class is shown in Table 1 (using the MODA dataset as an example):

[0051] Table 1. mAP performance of the optimal model on the validation set for each target class. The model inference process is as follows: (1) Input: Preprocessed multispectral pseudo-RGB image.

[0052] (2) Feature extraction: Initial features are extracted through the basic feature network, and multi-scale feature maps are generated through the feature pyramid network.

[0053] (3) Feature enhancement and weighting: The MCEM module enhances and fuses multi-scale feature maps, while the CBAM module performs channel and spatial attention weighting.

[0054] (4) Target detection: The detection head classifies the enhanced feature map and performs bounding box regression, outputting the target category, confidence score, bounding box coordinates and other results.

[0055] Furthermore, this embodiment employs multi-dimensional evaluation metrics to comprehensively measure model performance, including: (1) Detection performance: average accuracy (mAP@[0.5:0.95]), mAP50, mAP75.

[0056] (2) Multispectral characteristics: average recall (AR).

[0057] (3) Robustness: F1 parameter.

[0058] The model performance evaluation results are shown in Table 2: Table 2 Model performance evaluation results Furthermore, this embodiment uses comparative experiments to verify the superiority of the model of the present invention.

[0059] On the MODA dataset, comparative experiments were conducted with four mainstream models: Faster R-CNN, Deformable-DETR, RetinaNet, and YOLOv8. Figure 8 As shown, the model of this invention outperforms key metrics such as mAP, AP_Small, and FAR, demonstrating the effectiveness of multi-scale feature enhancement and attention-weighted design.

[0060] The optimized model maintains low computational complexity while ensuring detection accuracy. The average inference time per image includes the entire process of preprocessing, forward propagation, and postprocessing, meeting the real-time requirements of practical applications.

[0061] Specifically, the optimized model has the following advantages compared to the existing model: 1. Significant improvement in multi-scale target detection capabilities Traditional models are difficult to adapt to the needs of multi-scale target detection. This invention achieves full coverage of the receptive field at multiple scales through the multi-hole ratio design of the MCEM module, effectively solving the problem of imbalance in feature extraction between small and large targets, and significantly improving the detection performance of multi-scale targets.

[0062] 2. Precise enhancement of the expression of weak target characteristics By employing the dual attention mechanism of the CBAM module, the importance of feature channels and spatial location is adaptively distinguished, enhancing the salient feature response of weak targets, suppressing interference from redundant information in complex backgrounds, and significantly reducing false detection and false negative rates.

[0063] 3. Efficient utilization of multispectral data The PCA preprocessing method is adopted to maximize the preservation of multispectral information while removing data redundancy, so that the model can make full use of the advantages of multispectral data and improve its robustness in complex scenarios such as low light and low visibility.

[0064] 4. Good engineering practicality and compatibility The improved model is based on the traditional object detection framework and is optimized. It has a clear structure, is easy to implement, and is highly compatible with existing datasets and detection platforms. It adopts a lightweight design and can run efficiently in ordinary hardware environments, making it easy to deploy and promote in practical engineering.

[0065] In summary, this embodiment provides an optimization method for a multispectral UAV aerial target detection model. By designing a cascaded structure of a multi-scale contextual feature enhancement module (MCEM) and a convolutional block attention module (CBAM), the model's ability to represent features of multi-scale targets and its ability to focus on the salient features of weak targets are enhanced. Furthermore, by preprocessing the multispectral data through principal component analysis (PCA) and combining it with the improved model architecture, efficient and accurate detection of weak targets in multispectral UAV aerial images can be achieved, significantly improving detection accuracy and reducing false detection rate. This method is suitable for multispectral target detection tasks in complex scenarios.

[0066] Second Embodiment This embodiment provides a multispectral UAV aerial target detection model optimization device, including: The dataset preprocessing module is used to acquire a multispectral image dataset and preprocess the multispectral images in the multispectral image dataset to reduce the dimensionality of the multispectral images and convert the annotations of the multispectral images into a preset form. The model improvement module is used to construct an object detection model; wherein, the object detection model includes a multi-scale context feature enhancement module and a convolutional block attention module connected in series. The training module is used to train the object detection model using the preprocessed multispectral image dataset.

[0067] It should be noted that the multispectral UAV aerial target detection model optimization device of this embodiment corresponds to the multispectral UAV aerial target detection model optimization method of the first embodiment described above; the functions implemented by each functional module in the multispectral UAV aerial target detection model optimization device of this embodiment correspond one-to-one with the process steps in the multispectral UAV aerial target detection model optimization method of the first embodiment described above; therefore, they will not be described again here.

[0068] Third Embodiment This embodiment provides an electronic device, such as... Figure 9As shown, the electronic device includes a processor and a memory; wherein the processor and the memory can be connected via a communication bus; the memory stores at least one instruction, which is loaded and executed by the processor to implement the method of the first embodiment described above. Furthermore, the electronic device may also include a transceiver, the processor and the transceiver can be connected via a communication bus, and the transceiver is used to communicate with other devices.

[0069] Below, in conjunction with Figure 9 A detailed introduction to each component of this electronic device is provided below: The processor is the control center of the electronic device. The electronic device may include multiple processors, each of which can be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The term "processor" can refer to a single processor or a collective term for multiple processing elements. For example, a processor can be one or more central processing units (CPUs), other general-purpose processors, application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs), one or more field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor can perform various functions of the electronic device by running or executing software programs stored in memory and by calling data stored in memory.

[0070] In a specific implementation, as one example, the processor may include one or more CPUs, for example... Figure 9 CPU0 and CPU1 shown are, of course, merely illustrative examples.

[0071] The memory is used to store the software program that executes the solution of the present invention, and the processor controls its execution. For specific implementation methods, please refer to the above method embodiments, which will not be repeated here.

[0072] Optionally, the memory may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory may be integrated with the processor or exist independently, and may be accessed through the interface circuit of the electronic device ( Figure 9 (Not shown in the image) is coupled to the processor; however, this embodiment of the invention does not impose specific limitations on this.

[0073] The transceiver may include a receiver and a transmitter. Figure 9 (Not shown separately). The receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function. The transceiver can be integrated with the processor or exist independently, and can be connected through the interface circuit of the electronic device (…). Figure 9 (Not shown in the image) is coupled to the processor, and this embodiment of the invention does not specifically limit this.

[0074] In addition, it should be noted that, Figure 9 The structure of the electronic device shown is not intended to limit the device. Actual devices may include more or fewer components than shown, or combine certain components, or have different component arrangements. Furthermore, the technical effects achieved by this electronic device when performing the method of the first embodiment described above can be referenced to the technical effects described in the first embodiment; therefore, they will not be repeated here.

[0075] Fourth embodiment This embodiment provides a computer-readable storage medium storing at least one instruction, which is loaded and executed by a processor to implement the method of the first embodiment described above. The computer-readable storage medium may be a ROM, random access memory, CD-ROM, magnetic tape, floppy disk, or optical data storage device, etc. The instruction stored therein can be loaded and executed by a processor in a terminal.

[0076] Furthermore, it should be noted that the present invention can be provided as a method, apparatus, or computer program product. Therefore, embodiments of the present invention can take the form of a completely or partially hardware embodiment, a completely or partially software embodiment, or an embodiment combining software and hardware aspects. Moreover, when implemented in software, embodiments of the present invention can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer program are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any usable medium accessible to a computer or a data storage device such as a server or data center containing one or more sets of usable media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive (SSD).

[0077] Embodiments of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0078] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing terminal equipment to cause a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0079] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element. Furthermore, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone, where A and B can be singular or plural. Additionally, the character " / " in this text generally indicates an "or" relationship between the preceding and following objects, but it can also indicate an "AND / OR" relationship. Please refer to the context for specific interpretations. "At least one" refers to one or more items, while "more than" refers to two or more items. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, or c can be represented as: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0080] Furthermore, it is understood that in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0081] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0082] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of functional modules / units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the shown or discussed mutual couplings or direct couplings or communication connections may be through some interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms. Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, i.e., they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs. Additionally, the functional units in the various embodiments of this invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0083] If the method is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0084] Finally, it should be noted that the above description is merely a preferred embodiment of the present invention. It should be pointed out that although preferred embodiments of the present invention have been described, those skilled in the art, once they understand the basic inventive concept of the present invention, can make several improvements and modifications without departing from the principles described herein. These improvements and modifications should also be considered within the scope of protection of the present invention. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present invention.

Claims

1. A method for optimizing a multispectral UAV aerial target detection model, characterized in that, include: A multispectral image dataset is acquired, and the multispectral images in the dataset are preprocessed to reduce the dimensionality of the multispectral images and convert the annotations of the multispectral images into a preset form. Construct an object detection model; wherein the object detection model includes a multi-scale context feature enhancement module and a convolutional block attention module connected in series; The target detection model is trained using the preprocessed multispectral image dataset.

2. The multispectral UAV aerial target detection model optimization method as described in claim 1, characterized in that, Preprocessing the multispectral images in the multispectral image dataset to reduce the dimensionality of the multispectral images and convert the annotations of the multispectral images into a preset form includes: Principal component analysis is used to reduce the dimensionality of multispectral images in a multispectral image dataset, mapping the original multiband data to a three-dimensional feature space, and selecting the first three principal components to construct pseudo-RGB images of the corresponding multispectral images; wherein, the multispectral image dataset is the MODA dataset; For the rotated bounding box annotations in the MODA dataset, we convert them into COCO format bounding box annotations using coordinate transformation and the minimum bounding horizontal rectangle method. The preprocessed multispectral image dataset consists of pseudo-RGB images with horizontal bounding boxes in COCO format.

3. The multispectral UAV aerial target detection model optimization method as described in claim 1, characterized in that, The target detection model includes a cascaded backbone network, a feature pyramid network, a multi-scale contextual feature enhancement module, and a convolutional block attention module. The input image is processed sequentially through the backbone network, the feature pyramid network, the multi-scale contextual feature enhancement module, and the convolutional block attention module. The resulting feature map is then processed by a region proposal network and a region of interest pooling, and finally classified and bounding box regressed by the detection head.

4. The multispectral UAV aerial target detection model optimization method as described in claim 3, characterized in that, The backbone network is a ResNet50 network.

5. The multispectral UAV aerial target detection model optimization method as described in claim 3, characterized in that, The data processing procedure of the multi-scale contextual feature enhancement module is as follows: For the input feature map, three 3×3 convolution kernels with different dilation rates are used for convolution operations to obtain the feature information of local small targets, medium-sized targets and large targets respectively, and output the first feature map F1, the second feature map F2 and the third feature map F3 respectively; F1, F2 and F3 are concatenated in the channel dimension, and the enhanced feature map is obtained by dimensionality reduction through 1×1 convolution, and then residual connection is performed with the input feature map to avoid information loss and promote network training, and finally the enhanced feature map is output.

6. The multispectral UAV aerial target detection model optimization method as described in claim 3, characterized in that, The data processing procedure for the convolutional block attention module is as follows: Global average pooling and global max pooling are performed on the input feature map in the spatial dimension to obtain two types of channel description vectors. After nonlinear mapping by a multilayer perceptron with shared weights, channel attention weights are generated by the Sigmoid function, and the feature map is weighted channel by channel. The channel-weighted feature maps are subjected to average pooling and max pooling in the channel dimension, respectively. After concatenation, spatial attention weights are generated through a 7×7 convolutional layer to spatially weight the feature maps and highlight the features of the target region.

7. The multispectral UAV aerial target detection model optimization method as described in claim 6, characterized in that, The multilayer perceptron includes two fully connected layers with ReLU activation function.

8. The method for optimizing the multispectral UAV aerial target detection model as described in claim 1, characterized in that, The step of training the target detection model using the preprocessed multispectral image dataset includes: The preprocessed multispectral image dataset is input into the target detection model, and the model is trained using a stochastic gradient descent optimizer. The model parameters are optimized through multiple iterations.

9. The multispectral UAV aerial target detection model optimization method as described in claim 8, characterized in that, When training the model, the initial learning rate was set to 0.0025, momentum to 0.9, training epochs to 24, and batch size to 4. A step decay strategy was adopted, with the learning rate decayed by 0.1 at the 16th and 22nd epochs.

10. A multispectral unmanned aerial vehicle (UAV) target detection model optimization device, characterized in that, include: The dataset preprocessing module is used to acquire a multispectral image dataset and preprocess the multispectral images in the multispectral image dataset to reduce the dimensionality of the multispectral images and convert the annotations of the multispectral images into a preset form. The model improvement module is used to construct an object detection model; wherein, the object detection model includes a multi-scale context feature enhancement module and a convolutional block attention module connected in series. The training module is used to train the object detection model using the preprocessed multispectral image dataset.

Images