A Visual Inspection Method for Unmanned Aerial Vehicles Based on Improved YOLOv5s

Abstract

This invention relates to the field of computer vision technology and discloses a UAV visual detection method based on an improved YOLOv5s. It addresses the problems of existing detection methods failing to meet the lightweight deployment requirements of embedded UAV platforms and the difficulty in fully capturing contextual information during lightweight deployment, thus affecting detection performance. The Stem layer of this invention enhances initial feature extraction capabilities while maintaining high resolution through a multi-branch parallel structure; the Shuffle module achieves efficient feature transformation through channel segmentation and channel shuffling; the Shuffle downsampling module implements weighted feature fusion during downsampling through an adaptive gating mechanism; the C3Ghost module utilizes Ghost convolutions to first perform a small number of standard convolutions to generate intrinsic feature maps, and then generates more feature maps through low-cost linear transformation; the NextConv module optimizes the Conv structure in PANet, effectively expanding the receptive field of the feature maps without significantly increasing computational cost.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a UAV visual inspection method based on an improved YOLOv5s. Background Technology

[0002] In recent years, with the rapid development of drone technology, its applications in logistics, aerial commuting, urban planning, and disaster relief have become increasingly widespread. To ensure the legal and compliant use of drones and prevent potential security threats, accurate identification, location, and tracking of drones has become a key technical challenge. Currently, mainstream detection methods include radar detection, photoelectric detection, visual inspection, audio analysis, and radio frequency signal analysis. Among these, visual inspection, with its advantages of being non-contact, providing rich information, high scalability, high real-time performance, and low cost, has been widely used in low-altitude target detection tasks.

[0003] Deep learning-based visual detection algorithms have become a hot research topic, mainly divided into two-stage algorithms (such as R-CNN and its derivatives) and single-stage algorithms (such as SSD and the YOLO series). Among them, the YOLO series of algorithms balances detection speed and accuracy, and performs particularly well in real-time detection tasks. Researchers have carried out a lot of improvement work for real-time detection of small targets by low-altitude UAVs.

[0004] However, existing UAV visual detection still has the following shortcomings: Although the YOLOv5s model improves detection accuracy, its high number of parameters and computational complexity make it difficult to meet the lightweight deployment requirements of UAV embedded platforms; lightweight improvements to the YOLOv5s model are mostly focused on optimizing the backbone network or a single module, with insufficient attention paid to improving efficiency in the feature fusion stage, and the traditional C3 module still relies on standard convolution stacking, resulting in high computational overhead; in response to the problem of limited receptive field of targets in complex low-altitude UAV scenarios, existing UAV visual detection lacks an effective large-kernel convolution design, making it difficult to fully capture contextual information and affecting detection performance. Summary of the Invention

[0005] The purpose of this invention is to improve the existing detection methods, which are difficult to meet the lightweight deployment requirements of UAV embedded platforms and are difficult to fully capture contextual information when lightweighting, thus affecting detection performance. This invention provides a UAV visual detection method based on an improved YOLOv5s.

[0006] To achieve the above-mentioned objectives, the embodiments of the present invention provide the following technical solutions:

[0007] A UAV visual inspection method based on improved YOLOv5s includes the following steps:

[0008] Collect drone flight datasets, preprocess the drone flight datasets, and obtain drone detection images;

[0009] A visual inspection model for UAVs is constructed based on a lightweight backbone network, a lightweight neck network, and an inspection head.

[0010] The drone detection image is input into the drone vision detection model, and the multi-scale feature map is output through the lightweight backbone network and sent to the lightweight neck network. The multi-scale feature map is then output through the lightweight neck network and sent to the detection head to obtain the total loss function.

[0011] The UAV visual detection model is trained iteratively using the total loss function. The trained UAV visual detection model is then used to identify UAVs in real-time acquired images, and the final UAV identification result is output.

[0012] While existing YOLOv5s models offer high detection accuracy, their large number of parameters and computational complexity make them unsuitable for the stringent lightweight deployment requirements of UAV embedded platforms. Furthermore, the limited receptive field during lightweighting hinders the capture of contextual information from UAV targets in complex low-altitude scenarios, thus impacting detection performance. This application addresses these issues by constructing a lightweight backbone network and a lightweight neck network: The lightweight backbone network replaces the original YOLOv5s model backbone structure, significantly reducing the number of parameters and computational complexity while maintaining feature extraction capabilities, thus resolving the difficulty of deploying existing models on resource-constrained UAV embedded platforms. The lightweight neck network replaces the original YOLOv5s model neck network, further reducing computational overhead in the feature fusion stage while effectively expanding the receptive field of the feature map. This allows the model to fully capture contextual information from UAV targets in complex low-altitude scenarios while maintaining a lightweight design, enhancing feature discrimination capabilities and resolving the performance degradation caused by the limited receptive field during lightweighting. This application, through the collaborative design of a lightweight backbone network and a lightweight neck network, effectively maintains the detection accuracy of UAV targets while achieving model lightweighting, providing a technical solution for UAV detection in low-altitude complex scenarios on resource-constrained platforms.

[0013] Compared with existing technologies, the beneficial effects of this invention are as follows: The lightweight backbone network adopts a collaborative design of Stem layer, Shuffle module, and Shuffle downsampling module: The Stem layer enhances the initial feature extraction capability while maintaining high resolution through a multi-branch parallel structure; the Shuffle module achieves efficient feature transformation through channel segmentation and channel shuffling; the Shuffle downsampling module achieves feature weighted fusion during the downsampling process through an adaptive gating mechanism; the lightweight neck network introduces the C3Ghost module to replace the traditional C3 module, using Ghost convolution to first perform a small number of standard convolutions to generate inherent feature maps, and then generating more feature maps through low-cost linear transformation, significantly reducing the number of parameters and FLOPs in the feature fusion stage while keeping the number of output feature maps unchanged; at the same time, the NextConv module is used to optimize the Conv structure in PANet, effectively expanding the receptive field of the feature maps without significantly increasing the computational load.

[0014] Furthermore, a UAV visual detection method based on improved YOLOv5s includes the following sub-steps for preprocessing the UAV flight dataset:

[0015] Perform data cleaning on the UAV flight dataset and standardize the annotation format of the cleaned UAV flight dataset;

[0016] Image enhancement is performed on the labeled UAV flight dataset, including pixel value normalization and size unification, to obtain UAV detection images.

[0017] Furthermore, a UAV visual inspection method based on improved YOLOv5s is provided, wherein the lightweight backbone network includes a Stem layer, a Shuffle module, and a Shuffle downsampling module; the Shuffle module includes a Shuffle-1 module, a Shuffle-2 module, and a Shuffle-3 module; the Shuffle downsampling module includes a Shuffle-1 downsampling module, a Shuffle-2 downsampling module, and a Shuffle-3 downsampling module; the Stem layer, the Shuffle-1 downsampling module, the Shuffle-1 module, the Shuffle-2 downsampling module, the Shuffle-2 module, the Shuffle-3 downsampling module, and the Shuffle-3 module are connected sequentially.

[0018] Furthermore, a UAV visual detection method based on improved YOLOv5s is provided, wherein the Stem layer includes a convolutional layer s1, a convolutional layer s2, a convolutional layer s3, a convolutional layer s4, and a max pooling layer. The UAV detection image is input to the convolutional layer s1. The output feature map of the convolutional layer s1 is divided into two branches. The first branch enters the convolutional layer s2. The convolutional layer s2 processes the feature map through a convolutional kernel and outputs the feature map to the convolutional layer s3. The convolutional layer s3 processes the feature map through a convolutional kernel. The second branch enters the max pooling layer. The max pooling layer processes the feature map through a pooling kernel. The output feature map of the convolutional layer s3 and the output feature map of the max pooling layer are concatenated and input to the convolutional layer s4. The convolutional layer s4 processes the feature map through a convolutional kernel and outputs the feature map to the Shuffle-1 downsampling module.

[0019] Furthermore, a UAV visual inspection method based on improved YOLOv5s is provided, wherein the lightweight neck network includes a CBL layer, an upsample module, a C3ghost module, and a NextConv module; the CBL layer includes a CBL1 layer and a CBL2 layer; the upsample module includes an upsample1 module and an upsample2 module; the C3ghost module includes a C3ghost1 module, a C3ghost2 module, a C3ghost3 module, and a C3ghost4 module; and the NextConv module includes a NextConv1 module and a NextConv2 module.

[0020] The output of the Shuffle-3 module is connected to the CBL1 layer. The first output of the CBL1 layer is connected to the upsample1 module. The output of the upsample1 module and the second output of the Shuffle-2 module are concatenated and then connected to the C3ghost1 module. The output of the C3ghost1 module is connected to the CBL2 layer. The first output of the CBL2 layer is connected to the upsample2 module. The output of the upsample2 module and the second output of the Shuffle-1 module are concatenated and then connected to the C3ghost2 module. The first output of the C3ghost2 module is connected to the NextConv1 module. The output of the NextConv1 module and the second output of the CBL2 layer are concatenated and then connected to the C3ghost3 module. The first output of the C3ghost3 module is connected to the NextConv2 module. The output of the NextConv2 module and the second output of the CBL1 layer are concatenated and then connected to the C3ghost4 module.

[0021] The detection head includes a detection head 1 module, a detection head 2 module, and a detection head 3 module.

[0022] Furthermore, in a UAV visual inspection method based on improved YOLOv5s, the processing procedure of the C3ghost1 module is as follows:

[0023] The input feature map is divided into two branches. One branch is input to the CBL3 layer. The CBL3 layer outputs a feature map to the ghost1 module. After batch normalization and ReLU activation, the feature map output by the ghost1 module is input to the ghost2 module. The feature map output by the ghost2 module is batch normalized and then added to the feature map output by the CBL3 layer before being input to the convolutional s2 layer. The convolutional s2 layer outputs a feature map. The other branch is input to the convolutional s1 layer. The feature map output by the convolutional s1 layer is concatenated with the feature map output by the convolutional s2 layer. The concatenated feature map is input to the CBL4 layer, and the CBL4 layer outputs a feature map.

[0024] Furthermore, a UAV visual detection method based on improved YOLOv5s, wherein obtaining the total loss function includes the following sub-steps:

[0025] The drone detection images are processed by extracting preliminary features from the Stem layer of the lightweight backbone network, expanding the channel dimension, and then undergoing in-depth processing through the Shuffle module. The Shuffle downsampling module is used for spatial downsampling and channel expansion, and multi-scale feature maps C2, C3, and C4 are output to the lightweight neck network.

[0026] Multi-scale feature maps C2, C3, and C4 undergo channel compression through the CBL layer of the lightweight neck network, and the spatial resolution is restored by the upsample module. They are then concatenated and fused with the feature maps output by the corresponding scale Shuffle module. Lightweight feature processing is performed through the C3ghost module, and downsampling and channel halving are achieved using the NextConv module. After multi-level feature fusion and enhancement, multi-scale feature maps P2, P3, and P4 are output to the detection head.

[0027] Multi-scale feature maps P2, P3, and P4 are output as prediction tensors by the two-dimensional convolutional layers of each detection head module. The prediction tensors are divided into classification prediction and bounding box regression prediction in the channel dimension, and are output to the classification loss module and the bounding box regression loss module respectively to calculate the loss. The total loss function is obtained by weighting and summing according to the preset weights.

[0028] In the above scheme, this invention effectively solves the technical problems of low feature utilization efficiency and insufficient multi-scale adaptability in traditional target detection models by constructing a differentiated multi-scale feature extraction, fusion, and prediction process. This invention forms a complete feature processing flow from image input to loss calculation by constructing a backbone network for step-by-step extraction, a neck network for multi-level fusion, and a detection head for parallel prediction. Specifically, the backbone network, through the synergistic effect of the Stem layer, the Shuffle module, and the Shuffle downsampling module, achieves step-by-step extraction and channel expansion from high-resolution input to multi-scale feature maps; the neck network, through multi-level operations such as CBL layer channel compression, upsampling to restore resolution, cross-scale feature splicing and fusion, lightweight processing of the C3ghost module, and receptive field enhancement of the NextConv module, achieves deep fusion and optimization of multi-scale feature maps; the detection head performs parallel prediction of feature maps at each scale through two-dimensional convolutional layers and decouples the prediction tensor into classification and regression tasks to calculate losses separately. This invention achieves a complete feature representation of UAV targets from local details to global semantics, providing a reliable guarantee for high-precision training and efficient convergence of the model.

[0029] Furthermore, a UAV visual inspection method based on improved YOLOv5s, wherein the spatial downsampling and channel expansion using the Shuffle downsampling module includes the following steps:

[0030] Input feature map X to the channel segmentation module. The channel segmentation module segments the input feature map into feature map X1 and feature map X2. Output feature map X1 to branch one and output feature map X2 to branch two.

[0031] Branch A performs a 2×3 depthwise convolution operation on the input feature map. The features after the depthwise convolution operation are batch normalized. The features after the batch normalization operation are then performed a 1×1 convolution operation. The features after the convolution operation are then batch normalized. The batch normalized features are processed by the ReLU activation function to obtain the feature map F1, which is output to the preprocessing module and the feature fusion module.

[0032] Input feature map X2 to branch two. Branch two performs a 1×1 convolution operation on the input feature map. The features after the convolution operation are batch normalized. The batch normalized features are processed by the ReLU activation function. The processed features are then subjected to a 3×3 depthwise convolution operation with a stride of 2. The features after the depthwise convolution operation are batch normalized. The batch normalized features are then subjected to a 1×1 convolution operation. The features after the convolution operation are batch normalized. The batch normalized features are then processed by the ReLU activation function to obtain feature map F2, which is output to the preprocessing module 2 and the feature fusion module.

[0033] The preprocessing module performs a 1×1 convolution operation on the feature map F1. The convolutional features are then batch normalized, and the batch normalized features are processed by the LeakyReLU activation function to obtain the preprocessed features. Output to adaptive gating;

[0034] The second preprocessing module performs a 1×1 convolution operation on the feature map F2. The convolutional features are then batch normalized, and the batch normalized features are processed by the LeakyReLU activation function to obtain the preprocessed features. Output to adaptive gating;

[0035] Adaptive gated splicing preprocessed feature one and preprocessed feature two are combined. After performing a 1×1 convolution operation on the spliced ​​features, the convolutional features are processed by the LeakyReLU activation function to obtain gate weight one, which is then output to the feature fusion module.

[0036] The feature fusion module multiplies the first gate weight and the feature map F1 element by element, multiplies the second gate weight and the feature map F2 element by element, and concatenates the multiplied features to obtain the fused feature F3, which is then output to the channel shuffling module.

[0037] The channel shuffling module performs channel shuffling by fusing features, promoting cross-channel information exchange, and outputting the features. .

[0038] Furthermore, a UAV visual inspection method based on improved YOLOv5s, wherein the formula for spatial downsampling and channel expansion using the Shuffle downsampling module is as follows:

[0039] ;

[0040] Where F1 represents the output features of branch one, Conv is the convolution operation, DWConv is the depthwise convolution operation, ReLU is the ReLU activation function, BN is batch normalization, X1 and X2 are the feature maps output by the channel segmentation module, and F2 is the output features of branch two. For preprocessing feature one, LeakyReLU is the LeakyReLU activation function. For preprocessing feature 2, Concat is used for concatenation, W1 is the first gate weight, W2 is the second gate weight, and F3 is the fused feature. For element-wise multiplication, For channel mixed washing operation, For output features.

[0041] In the above scheme, this invention effectively solves the technical problems of feature information loss and insufficient response of key features in the traditional downsampling process by constructing an adaptive gating and dual-branch collaborative processing downsampling mechanism. This invention achieves efficient feature selection and fusion during downsampling by constructing a dual-branch parallel processing and adaptive gating weighting mechanism. Specifically, branch one adopts a structure of pre-deep convolution and post-1×1 convolution, focusing on rapid compression of spatial information and channel expansion; branch two adopts a structure of pre-1×1 convolution, mid-deep convolution, and post-1×1 convolution, focusing on spatial downsampling and feature reconstruction after channel transformation. The two branches process the input features from different angles, forming complementary feature representations. Based on this, this invention innovatively designs an adaptive gating mechanism based on bidirectional splicing. A preprocessing module extracts compact representations of branch features, then generates gating weights matching the size of the original feature map, achieving dynamic weighted selection of branch features. Finally, the weighted features are spliced ​​and fused with channel mixing, which not only expands the number of channels but also promotes full interaction of cross-channel information. This design enables the model to adaptively enhance key feature responses and suppress redundant information during downsampling, significantly improving the discriminativeness and richness of feature representation while expanding the receptive field and increasing the number of channels.

[0042] Furthermore, a UAV visual detection method based on improved YOLOv5s, wherein the iterative training of the UAV visual detection model through the total loss function includes the following sub-steps:

[0043] The training dataset is input into the UAV visual detection model to perform multi-level feature extraction and complex feature reasoning to obtain recognition results.

[0044] By comparing the detection results with the real label data using the total loss function, the gradient descent algorithm is used for backpropagation to dynamically adjust the weight parameters of each layer of the UAV visual detection model.

[0045] Repeated training continuously optimizes the parameters of the lightweight UAV visual detection model until the training loss converges to a stable state or reaches the preset maximum number of iterations.

[0046] In the above scheme, this invention achieves efficient optimization of a lightweight model in UAV detection tasks by constructing an end-to-end iterative training mechanism. Specifically, this training process compares the recognition results obtained from forward propagation with the real labels, uses the total loss function to comprehensively evaluate the errors of classification and regression tasks, and adjusts the network weights layer by layer through gradient backpropagation, enabling the model to gradually learn more discriminative feature representations. The iterative training process ensures that the model parameters converge to the optimal solution, and, combined with preset convergence conditions or maximum number of iterations, effectively avoids overfitting and underfitting problems. This training method is fully compatible with the lightweight backbone network, neck network, and detection head structure constructed in this invention, enabling the model to fully mine the feature information in the training data while maintaining low parameter and computational costs, ultimately obtaining a UAV visual detection model with high detection accuracy and strong generalization ability. Attached Figure Description

[0047] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 A flowchart for improving the UAV visual inspection method of YOLOv5s.

[0049] Figure 2 This is a schematic diagram of the structure of a UAV visual inspection model.

[0050] Figure 3 This is a schematic diagram of the Stem layer structure.

[0051] Figure 4 This is a structural diagram of the Shuffle basic module.

[0052] Figure 5 This is a schematic diagram of the Shuffle1 downsampling module.

[0053] Figure 6 This is a schematic diagram of the structure of C3ghost1.

[0054] Figure 7 This is a structural diagram of the NextConv module.

[0055] Figure 8 A comparison chart showing the effects achieved by the model. Detailed Implementation

[0056] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0057] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this invention, the terms "first," "second," etc., are used only for distinguishing descriptions and should not be construed as indicating or implying relative importance, or suggesting any such actual relationship or order between these entities or operations. Additionally, the terms "connected," "linked," etc., can refer to a direct connection between elements or an indirect connection via other elements.

[0058] It is important to note that modules prefixed with "Shuffle" all have the same structure as Shuffle modules; modules containing "Shuffle-downsampling" modules all have the same structure as Shuffle-downsampling modules; modules prefixed with "CBL" all have the same structure as CBL layers; modules prefixed with "upsample" all have the same structure as upsample modules; modules prefixed with "C3ghost" all have the same structure as C3ghost modules; modules prefixed with "NextConv" all have the same structure as NextConv modules; modules prefixed with "detector head" all have the same structure as detector head modules; modules prefixed with "convolution" all have the same structure as convolutional layers; modules prefixed with "depthmography" all have the same structure as depthmography convolutional layers; and modules prefixed with "ghost" all have the same structure as ghost modules. The numbers "-1, -2, 1, 2, s1, s2, d1, d2, N1, N2" are added after the connection to distinguish the connection relationships.

[0059] Example 1: A visual inspection method for unmanned aerial vehicles based on improved YOLOv5s.

[0060] This invention is achieved through the following technical solutions, such as... Figure 1 As shown, a UAV visual inspection method based on an improved YOLOv5s includes the following steps:

[0061] like Figure 1 As shown, S1: Collect UAV flight dataset, preprocess the UAV flight dataset, and obtain UAV detection images.

[0062] The drone flight dataset includes the Det-Fly dataset, publicly available online datasets, and self-built datasets.

[0063] The Det-Fly dataset includes front view, top view, and bottom view. Figure 3 One perspective;

[0064] In this example, the Det-Fly dataset consists of images of air-to-air flying target drones, with over 13,000 images. The proportions of images in the front view, top view, and bottom view are 36.4%, 32.5%, and 31.1%, respectively.

[0065] The publicly available dataset contains images of drones of various sizes and shapes;

[0066] The self-built dataset consists of drone images collected through on-site photography, covering drone samples under different lighting conditions, background environments, and flight attitudes;

[0067] It is important to note that publicly available online datasets are used to improve the ability to identify various types of drones, while self-built datasets are used to supplement the insufficient scene coverage of existing datasets.

[0068] S11: Clean the UAV flight dataset and standardize the annotation format of the cleaned UAV flight dataset;

[0069] S12: Perform image enhancement on the labeled UAV flight dataset, normalize pixel values ​​and unify dimensions to obtain UAV detection images.

[0070] In the embodiments, the dimensions are uniformly 640×640.

[0071] In existing technologies, the traditional YOLOv5s model is relatively suitable for drone recognition in low-altitude drone detection. Although there are gaps in model accuracy and algorithm, low-altitude drone detection mainly involves mounting the detection equipment on the drone. Therefore, the selected network model should be more lightweight and minimize the impact of computation and memory usage. The traditional YOLOv5s model has significant advantages in terms of parameter count and speed, making it more suitable for mobile devices. Therefore, this paper uses the traditional YOLOv5s model as the base network model and further improves its backbone and neck network to construct a drone visual detection model, creating a lighter model that is more suitable for drone recognition.

[0072] S2: Construct a UAV visual inspection model based on a lightweight backbone network, a lightweight neck network, and an inspection head.

[0073] Specifically, such as Figure 2As shown, the UAV visual inspection model includes a lightweight backbone network, a lightweight neck network, and an inspection head.

[0074] The lightweight backbone network includes a Stem layer, a Shuffle module, and a Shuffle downsampling module; the Shuffle module includes a Shuffle-1 module, a Shuffle-2 module, and a Shuffle-3 module; the Shuffle downsampling module includes a Shuffle-1 downsampling module, a Shuffle-2 downsampling module, and a Shuffle-3 downsampling module.

[0075] The Stem layer, Shuffle-1 downsampling module, Shuffle-1 module, Shuffle-2 downsampling module, Shuffle-2 module, Shuffle-3 downsampling module, and Shuffle-3 module are connected in sequence.

[0076] The Shuffle-1 module outputs a multi-scale feature map C2, the Shuffle-2 module outputs a multi-scale feature map C3, and the Shuffle-3 module outputs a multi-scale feature map C4.

[0077] In existing technologies, the backbone network of traditional YOLOv5s models is mainly composed of Focus layers, Conv layers, Bottleneck CSP / C3 layers, and SPP modules stacked together. The problems are: a large number of parameters, high computational complexity, and insufficient feature extraction capability under lightweight requirements. Specifically, its Focus layer structure is simple, performing fast downsampling and channel transformation through a single path. While this reduces subsequent computation, it struggles to fully extract and retain key details such as edges and textures in shallow image layers, especially in scenarios with dense small targets, such as low-altitude drone detection, where feature loss is significant. Furthermore, while the Bottleneck CSP / C3 modules in the middle layers possess strong feature processing capabilities, the repeated stacking of standard convolutions leads to high computational overhead, hindering real-time inference on resource-constrained mobile platforms. Simultaneously, its downsampling process lacks a dynamic filtering mechanism for feature importance; while expanding the receptive field and increasing the number of channels, it easily introduces redundant information, limiting the model's feature representation capability in complex environments.

[0078] Therefore, the lightweight backbone network of this application includes a Stem layer, a Shuffle module, and a Shuffle downsampling module. The Stem layer differs from the single-path design of the traditional YOLOv5s model entry point. The Stem layer of this application adopts a multi-branch parallel structure, which splices and fuses the preliminary features extracted from different receptive fields, and then performs channel compression and information integration through 1×1 convolution. While maintaining high spatial resolution, it effectively enhances the ability to extract low-level features such as image edges and textures in the initial stage. Under the premise of basically no increase in computation, it makes up for the inherent loss of the lightweight network in shallow feature representation.

[0079] The Shuffle module applies the existing Shuffle module to the traditional YOLOv5s model. It divides the input feature map into two branches through channel segmentation. One branch performs identity mapping, and the other branch performs feature transformation through convolution. Finally, it achieves cross-channel information fusion by concatenation and channel shuffling. While keeping the feature map size and number of channels unchanged, it significantly reduces the number of parameters and computational complexity within the module, and achieves efficient deep feature processing.

[0080] The Shuffle downsampling module is a self-created core module in this application, designed to solve the feature weighting and filtering problem in the lightweight downsampling process. First, it adopts the channel segmentation strategy to divide the input into two branches. In both branches, depthwise convolution is introduced to achieve halving of spatial size and channel expansion. Crucially, the module integrates an adaptive gating mechanism. Compact features are generated through an independent preprocessing module, and then gating weights matching the size of the original feature map are generated by bidirectional concatenation and convolution. These weights are used to perform element-wise weighted filtering of the original features, and the weighted features are concatenated and fused, and output after channel shuffling. This allows the Shuffle downsampling module to dynamically enhance the response of key features during the lightweight downsampling process, while effectively suppressing redundant information. Thus, while expanding the receptive field and increasing the number of channels, it significantly improves the expressive power and discriminative power of features, and the overall parameter and computational costs remain at a low level.

[0081] Specifically, the internal connections of the lightweight backbone network are as follows: the output of the Stem layer is connected to the input of the Shuffle-1 downsampling module; the output of the Shuffle-1 downsampling module is connected to the input of the Shuffle-1 module; the first output of the Shuffle-1 module is connected to the input of the Shuffle-2 downsampling module; the output of the Shuffle-2 downsampling module is connected to the input of the Shuffle-2 module; the first output of the Shuffle-2 module is connected to the input of the Shuffle-3 downsampling module; and the output of the Shuffle-3 downsampling module is connected to the Shuffle-3 module.

[0082] like Figure 3 As shown, the Stem layer includes a convolutional layer s1, a convolutional layer s2, a convolutional layer s3, a convolutional layer s4, and a max pooling layer.

[0083] The Shuffle-1 module includes 3 basic Shuffle modules, the Shuffle-2 module includes 7 basic Shuffle modules, and the Shuffle-3 module includes 3 basic Shuffle modules.

[0084] Specifically, such as Figure 4 As shown, the Shuffle basic module includes a channel segmentation module, a convolutional d1 layer, a depthwise convolutional d1 layer, a convolutional d2 layer, and a channel shuffling module;

[0085] Specifically, such as Figure 5 As shown, the Shuffle-1 downsampling module includes a channel segmentation module, branch one, branch two, preprocessing one module, preprocessing two module, adaptive gating, feature fusion module, and channel shuffling module;

[0086] The lightweight neck network includes a CBL layer, an upsample module, a C3ghost module, and a NextConv module; the CBL layer includes a CBL1 layer and a CBL2 layer; the upsample module includes an upsample1 module and an upsample2 module; the C3ghost module includes a C3ghost1 module, a C3ghost2 module, a C3ghost3 module, and a C3ghost4 module; and the NextConv module includes a NextConv1 module and a NextConv2 module.

[0087] Specifically, the internal connectivity of the lightweight neck network is as follows:

[0088] The output of the Shuffle-3 module is connected to the CBL1 layer. The first output of the CBL1 layer is connected to the upsample1 module. The output of the upsample1 module and the second output of the Shuffle-2 module are concatenated and then connected to the C3ghost1 module. The output of the C3ghost1 module is connected to the CBL2 layer. The first output of the CBL2 layer is connected to the upsample2 module. The output of the upsample2 module and the second output of the Shuffle-1 module are concatenated and then connected to the C3ghost2 module.

[0089] The first output of the C3ghost2 module is connected to the NextConv1 module. The output of the NextConv1 module and the second output of the CBL2 layer are spliced ​​together and then connected to the C3ghost3 module. The first output of the C3ghost3 module is connected to the NextConv2 module. The output of the NextConv2 module and the second output of the CBL1 layer are spliced ​​together and then connected to the C3ghost4 module.

[0090] More specifically, the CBL1 layer includes a convolutional L layer, batch normalization, and the Leakyrelu activation function.

[0091] More specifically, such as Figure 6 As shown, the C3ghost1 module includes a CBL3 layer, a ghost1 module, a ghost2 module, a convolutional s1 layer, a convolutional s2 layer, and a CBL4 layer.

[0092] The ghost1 module is actually an existing Ghost module that can generate the same number of feature maps as convolutional layers, while reducing the number of parameters and computational cost.

[0093] More specifically, such as Figure 7 As shown, the NextConv1 module includes a depthwise convolutional layer N1, a convolutional layer N2, a max pooling layer, a convolutional layer N3, and a convolutional layer N4.

[0094] The connection structure of the NextConv1 module is as follows: the first input of the NextConv1 module is connected to the input of the depthwise convolutional layer N1, the output of the depthwise convolutional layer N1 is connected to the input of the convolutional layer N1, the second input of the NextConv1 module is connected to the input of the max pooling layer, the output of the max pooling layer is connected to the input of the convolutional layer N2, and the outputs of the convolutional layers N1 and N2 are concatenated and then connected to the input of the convolutional layer N3.

[0095] In existing technologies, the traditional YOLOv5s model's neck network adopts a bidirectional fusion structure of feature pyramid network + path aggregation network. The core feature processing module is the C3 module, which is composed of multiple stacked Bottleneck structures. Each Bottleneck uses standard convolution for feature transformation. This has the following problems: The stacking of standard convolutions within the C3 module results in a high overall parameter count in the neck network, leading to high computational costs, which is not conducive to deployment on resource-constrained UAV embedded platforms. The receptive field of the standard convolutions used is limited, making it difficult to fully capture the contextual information of large-scale UAV targets. In complex low-altitude scenarios, UAV targets are often affected by background interference, and the limited receptive field leads to insufficient perception of the surrounding environment information of the target. In low-altitude UAV detection tasks, the target scale varies, the background is complex, and the lighting conditions change drastically. Although the neck network can perform multi-scale fusion, its ability to retain small target features is insufficient, and shallow detail information is easily lost during multiple feature transfers. At the same time, it lacks an adaptive filtering mechanism for key features, which limits the model's ability to discriminate UAVs in complex backgrounds.

[0096] This application introduces the C3Ghost and NextConv modules into a lightweight neck network. The C3Ghost module replaces the standard convolutional layers within the Bottleneck of the C3 module with Ghost convolutional layers. The Ghost convolution generates inherent feature maps by first performing a small number of standard convolutions, and then generating more feature maps through a low-cost linear transformation. This significantly reduces the number of parameters within the module while maintaining the same number of output feature maps. Drawing inspiration from the design of the ConvNeXt network, the NextConv module is constructed to optimize the standard convolutional layers in the path aggregation network. The NextConv module effectively expands the receptive field of the feature maps without significantly increasing computation, enabling the model to aggregate a wider range of contextual information and enhancing the feature discrimination capability for UAV targets. This solves the problem of insufficient feature extraction caused by the limited receptive field in lightweight networks.

[0097] The detection head includes a detection head 1 module, a detection head 2 module, and a detection head 3 module.

[0098] Specifically, the connection between the lightweight neck network and the detection head is as follows:

[0099] Module 1 of the detection head is connected to the second output terminal of module 2 of C3ghost2, module 2 of the detection head is connected to the second output terminal of module 3ghost3, and module 3 of the detection head is connected to the second output terminal of module 4 of C3ghost4.

[0100] The detection head module 1 includes a two-dimensional convolutional layer, which is responsible for simultaneously outputting classification and regression prediction results.

[0101] S3: Input the UAV detection image into the UAV visual detection model, process it through the lightweight backbone network and output multi-scale feature maps C2, C3, and C4 to the lightweight neck network, output multi-scale feature maps P2, P3, and P4 through the lightweight neck network to the detection head, and obtain the total loss function.

[0102] S31: The drone detection image extracts preliminary features through the Stem layer of the lightweight backbone network, expands the channel dimension, performs in-depth processing through the Shuffle module, and uses the Shuffle downsampling module to perform spatial downsampling and channel expansion, outputting multi-scale feature maps C2, C3, and C4 to the lightweight neck network.

[0103] Specifically, the lightweight backbone network's processing procedure is as follows: A 640×640×3 UAV detection image is input to the Stem layer of the lightweight backbone network. The Stem layer outputs a 160×160×64 feature map to the Shuffle-1 downsampling module. The Shuffle-1 downsampling module outputs an 80×80×128 feature map to the Shuffle-1 module. The first output of the Shuffle-1 module outputs a multi-scale feature map C2 (80×80×128) to the Shuffle-2 downsampling module. The Shuffle-2 downsampling module outputs a 40×40×256 feature map to the Shuffle-2 module. The first output of the Shuffle-2 module outputs a multi-scale feature map C3 (40×40×256) to the Shuffle-3 downsampling module. The Shuffle-3 downsampling module outputs a 20×20×512 feature map to the Shuffle-3 module. The Shuffle-3 module outputs a multi-scale feature map C4 (20×20×512) to the lightweight neck network.

[0104] More specifically, the processing of the STEM layer is as follows: The 640×640×3 drone detection image is input to the convolutional layer s1. The convolutional layer s1 processes the drone detection image using 32 convolutional kernels with a stride of 2 and padding of 1, and a size of 3x3. The output feature map is 320×320×32, which is divided into two branches. The first branch enters the convolutional layer s2. The convolutional layer s2 processes the feature map using 32 convolutional kernels with a stride of 1 and a size of 3x3, and outputs the feature map 320×320×32 to the convolutional layer s3. The convolutional layer s3 uses 32 convolutional kernels with a stride of 2 and padding of 1, and a size of 3x3. The feature map is processed, and a 160×160×32 feature map is output. The second branch enters the max pooling layer. The max pooling layer processes the feature map with a 3x3 size pooling kernel with a stride of 2 and padding of 1, and outputs a 160×160×32 feature map. The output feature map of the convolutional s3 layer and the output feature map of the max pooling layer are concatenated and input into the convolutional s4 layer, which processes the feature map with 64 convolutional kernels with a stride of 1 and a size of 1x1, and outputs a 160×160×64 feature map to the Shuffle-1 downsampling module.

[0105] More specifically, the processing procedure of the Shuffle basic module is as follows:

[0106] The input feature map H×W×C is fed into the channel segmentation module. The channel segmentation module divides the input feature map H×W×C into feature maps H×W×C1 and H×W×C2 (C1=C2=C / 2). Feature map H×W×C1 is fed into convolutional layer d1. Convolutional layer d1 performs a 1×1 convolution operation on the feature map. The output feature map H×W×C1' is processed by batch normalization and ReLU activation function and then fed into depthwise convolutional layer d1. Depthwise convolutional layer d1 performs a 3×3 depthwise convolution operation on the feature map and then performs batch normalization before outputting feature map H×W×C1' to convolutional layer d2. Convolutional layer d2 performs a 1×1 convolution operation on the feature map. The output feature map H×W×C1 is processed by batch normalization and ReLU activation function and then concatenated with feature map H×W×C2. The output feature map H×W×C is fed into the channel shuffling module. The channel shuffling module performs channel rearrangement operation on the feature map and outputs feature map H×W×C.

[0107] Where H is the height of the feature map, W is the width of the feature map, and C is the number of channels in the feature map.

[0108] It should be noted that the channel shuffling module is used to facilitate cross-group information exchange and enhance feature representation capabilities.

[0109] More specifically, the processing procedure of the Shuffle-1 downsampling module is as follows:

[0110] S311: Input feature map X (H×W×C) to the channel segmentation module. The channel segmentation module divides the input feature map H×W×C into feature map X1 (H×W×C1) and feature map X2 (H×W×C2) (C1=C2=C / 2). It outputs feature map X1 to branch one and feature map X2 to branch two.

[0111] S312: Branch A pair of input feature maps is subjected to a 2×3 depthwise convolution operation. The features after the depthwise convolution operation are batch normalized. The features after the batch normalization operation are then subjected to a 1×1 convolution operation. The features after the convolution operation are then batch normalized. The batch normalized features are processed by the ReLU activation function to obtain feature map F1, which is output to the preprocessing module and the feature fusion module.

[0112] The formula for processing branch one is:

[0113] F1=ReLU(BN(Conv(BN(DWConv(X1)))));

[0114] Where F1 is the output feature of branch one, Conv is the convolution operation, DWConv is the depthwise convolution operation, ReLU is the ReLU activation function, BN is batch normalization, and X1 is the feature map output by the channel segmentation module.

[0115] It should be noted that the number of convolution kernels in a 1×1 convolution operation is C, which is the output feature map F1 (H / 2×W / 2×C).

[0116] S312: Input feature map X2 (H×W×C2) to branch two. Branch two performs a 1×1 convolution operation on the input feature map. The features after the convolution operation are batch normalized. The batch normalized features are processed by the ReLU activation function. The processed features are then subjected to a 3×3 depthwise convolution operation with a stride of 2. The features after the depthwise convolution operation are batch normalized. The batch normalized features are then subjected to a 1×1 convolution operation. The features after the convolution operation are batch normalized. The batch normalized features are then processed by the ReLU activation function to obtain feature map F2, which is output to the preprocessing module 2 and the feature fusion module.

[0117] The formula for processing branch two is:

[0118] F2=ReLU(BN(Conv(BN(DWConv(ReLU(BN(Conv(X2))))))));

[0119] Where F2 is the output feature of branch two, and X2 is the feature map output by the channel segmentation module.

[0120] It should be noted that the number of kernels in the first 1×1 convolution operation in branch 2 is C / 2, and the number of kernels in the second 1×1 convolution operation is C, which is the output feature map F2 (H / 2×W / 2×C).

[0121] S313: The preprocessing module performs a 1×1 convolution operation on the feature map F1. The features after the convolution operation are batch normalized, and the batch normalized features are processed by the LeakyReLU activation function to obtain the preprocessed features. Output to adaptive gating;

[0122] The processing formula for the preprocessing module is as follows:

[0123] ;

[0124] in, For preprocessing feature one, LeakyReLU is the LeakyReLU activation function.

[0125] S314: The preprocessing module performs a 1×1 convolution operation on the feature map F2. The features after the convolution operation are then batch normalized, and the batch normalized features are processed by the LeakyReLU activation function to obtain the preprocessed features. Output to adaptive gating;

[0126] The processing formula for the second preprocessing module is:

[0127] ;

[0128] in, This is the second preprocessing feature.

[0129] S315: Adaptive gated concatenation of preprocessed features 1 and 2. After performing a 1×1 convolution operation on the concatenated features, the convolutional features are processed by the LeakyReLU activation function to obtain gate weight 1, which is then output to the feature fusion module.

[0130] The processing formula for adaptive gating is:

[0131] ;

[0132] Wherein, Concat is for concatenation, W1 is the first gating weight, and W2 is the second gating weight.

[0133] It should be noted that the number of convolution kernels in adaptive gating convolution operations is C.

[0134] S316: The feature fusion module multiplies the first gate weight and the feature map F1 element by element, and multiplies the second gate weight and the feature map F2 element by element. The multiplied features are then concatenated to obtain the fused features, which are then output to the channel shuffling module.

[0135] The processing formula for the feature fusion module is:

[0136] ;

[0137] in, As a feature of fusion, This is an element-wise product.

[0138] S317: The channel shuffling module performs channel shuffling of fused features to promote cross-channel information exchange and output features.

[0139] The processing formula for the channel shuffling module is as follows:

[0140] ;

[0141] in, For channel mixed washing operation, For output features.

[0142] It is important to note that the output features processed by the Shuffle-1 downsampling module... (H / 2×W / 2×2C) achieves a halved size and an increased number of channels.

[0143] S32: Multi-scale feature maps C2, C3, and C4 undergo channel compression through the CBL layer of the lightweight neck network, and the spatial resolution is restored by the upsample module. They are then concatenated and fused with the feature maps output by the corresponding scale Shuffle module. Lightweight feature processing is performed through the C3ghost module, and downsampling and channel halving are performed using the NextConv module. After multi-level feature fusion and enhancement, multi-scale feature maps P2, P3, and P4 are output to the detection head.

[0144] More specifically, the lightweight neck network's processing procedure is as follows: The multi-scale feature map C4 (20×20×512) is input to the CBL1 layer of the lightweight neck network. The CBL1 layer outputs a 20×20×128 feature map, which is then fed to the upsample1 module. The upsample1 module outputs a 40×40×128 feature map, which, along with the Shuffle-1 module's output multi-scale feature map C3 (40×40×256), are concatenated to obtain a concatenated feature map of 40×40×384, which is then input to the C3ghost1 module. The C3ghost1 module outputs a 40×40×384 feature map, which is fed to the CBL2 layer. The CBL2 layer outputs a 40×40×128 feature map, which is fed to the upsample2 module. The upsample2 module outputs an 80×80×128 feature map, which, along with the Shuffle-1 module's output multi-scale feature map C2 (80×80×128), are then fed to the upsample2 module. The features are spliced ​​together to obtain a spliced ​​feature map of 80×80×256, which is then input to the C3ghost2 module. The C3ghost2 module outputs feature map P2 (80×80×256) to the NextConv1 module. The NextConv1 module outputs feature map 40×40×128, which is spliced ​​together with the feature map output from the CBL2 layer. The spliced ​​feature map 40×40×256 is then input to the C3ghost3 module. The C3ghost3 module outputs feature map P3 (40×40×256) to the NextConv2 module. The NextConv2 module outputs feature map 20×20×128, which is spliced ​​together with the feature map output from the CBL1 layer. The spliced ​​feature map 20×20×256 is then input to the C3ghost4 module. The C3ghost4 module outputs feature map P4 (20×20×256) to the detection head.

[0145] Specifically, the processing procedure of the C3ghost1 module is as follows:

[0146] The input feature map (H×W×C) is divided into two branches. One branch is input to the CBL3 layer. The CBL3 layer outputs a feature map (H×W×C / 2) to the ghost1 module. The ghost1 module outputs a feature map (H×W×C / 2), which is then processed by batch normalization and ReLU activation function before being input to the ghost2 module. The ghost2 module outputs a feature map (H×W×C / 2), which is batch normalized and then added to the CBL3 layer output feature map (H×W×C / 2) before being input to the convolutional s2 layer. The convolutional s2 layer outputs a feature map (H×W×C / 4). The other branch is input to the convolutional s1 layer. The convolutional s1 layer outputs a feature map (H×W×C / 4) and the convolutional s2 layer outputs a feature map (H×W×C / 2). The concatenated feature map is input to the CBL4 layer, which outputs a feature map (H×W×C / 2).

[0147] The C3ghost1, C3ghost2, C3ghost3, and C3ghost4 modules have the same structure and are only used to distinguish connection relationships.

[0148] Specifically, the processing procedure of the NextConv module is as follows:

[0149] The input feature map (H×W×C) is divided into two branches. One branch is input to the depthwise convolutional layer N1, and the output feature map (H / 2×W / 2×C) of the depthwise convolutional layer N1 is input to the convolutional layer N1. The second branch is input to the max pooling layer, and the output feature map (H / 2×W / 2×C) of the max pooling layer is input to the convolutional layer N2. The output feature map (H / 2×W / 2×C / 2) of the convolutional layer N1 and the output feature map (H / 2×W / 2×C / 2) of the convolutional layer N2 are concatenated to obtain the concatenated feature (H / 2×W / 2×C), which is input to the convolutional layer N3, and the output feature map (H / 2×W / 2×C / 2) of the convolutional layer N3 is input.

[0150] S33: Multi-scale feature maps P2, P3, and P4 output prediction tensors through the two-dimensional convolutional layers of each detection head module. The prediction tensors are divided into classification prediction and bounding box regression prediction in the channel dimension, and are output to the classification loss module and the bounding box regression loss module respectively to calculate the loss. The total loss function is obtained by weighted summation according to the preset weights.

[0151] Specifically, the processing procedure of the detection head 1 module is as follows:

[0152] The multi-scale feature map P2 is input to the two-dimensional convolutional layer, and the two-dimensional convolutional layer outputs a prediction tensor. The prediction tensor is divided into classification prediction and bounding box regression prediction in the channel dimension.

[0153] The classification prediction part includes the target category probability and objectness, and the number of channels is equal to the number of anchor boxes multiplied by (number of categories + 1).

[0154] The bounding box regression prediction includes the offset of the bounding box, and the number of channels is equal to the number of anchor boxes multiplied by 4. These are output to the classification loss module and the bounding box regression loss module, respectively.

[0155] The classification loss module and the bounding box regression loss module calculate the classification loss function and the bounding box regression loss function, respectively. The two modules then perform a weighted summation based on a preset weight coefficient (e.g., 0.05) to obtain the total loss function.

[0156] It should be noted that the detection head has the same structural hierarchy as the traditional YOLOv5s model in existing technologies, and has not been improved.

[0157] S4: Iteratively train the UAV visual detection model through the total loss function, use the trained UAV visual detection model to identify UAVs in real-time acquired images, and finally output the UAV identification results.

[0158] Specifically, S4 includes the following sub-steps:

[0159] S41: Input the training dataset into the UAV visual detection model to perform multi-level feature extraction and complex feature reasoning to obtain the recognition results;

[0160] S42: By comparing the detection results with the real label data through the total loss function, the gradient descent algorithm is used for backpropagation to dynamically adjust the weight parameters of each layer of the UAV visual detection model.

[0161] S43: Repeatedly train to continuously optimize the parameters of the lightweight UAV visual detection model until the training loss converges to a stable state or the preset maximum number of iterations is reached.

[0162] In this embodiment, the training dataset is trained 100 times, the batch size is 16, the optimizer is stochastic gradient descent, the initial learning rate is set to 0.01, and a cosine annealing strategy is used for dynamic adjustment.

[0163] Example 2: Comparison of Model Target Detection Performance for UAVs

[0164] The YOLOv5 model, YOLOv4 model, YOLOv3 model, Faster RCNN model, and UAV visual detection model were compared and implemented, and the final results are shown in Table 1.

[0165] Table 1: Comparison of model performance in detecting UAV targets:

[0166] ;

[0167] Where Para represents the number of parameters, which characterizes the total scale of the model's learnable parameters and is related to memory usage and initialization efficiency; FLOPs represents the number of floating-point operations, which measures the number of multiply-accumulate operations required for forward inference of the model and directly reflects its demand for hardware computing resources; mAP0.5 represents the average detection accuracy of all categories when the loss function threshold is set to 0.5; M represents millions; and G represents billions.

[0168] The UAV visual detection model achieved an mAP of 0.934, which is roughly on par with the highest-performing YOLOv5 model and significantly higher than YOLOv4, YOLOv3, and Faster R-CNN models. In terms of parameter count, the UAV visual detection model requires only 2.60M parameters, far fewer than other comparative models. This extremely low parameter count means a smaller model file, effectively reducing storage requirements on UAV embedded platforms and facilitating model deployment and updates. Regarding computational complexity, the UAV visual detection model requires only 9.5 GFLOPs, far lower than other models. This low computational load means fewer operations are needed during forward inference, significantly accelerating inference speed and meeting the low-latency requirements of real-time UAV detection tasks.

[0169] like Figure 8 As shown, the anchor boxes of the YOLOv5 model and the UAV visual detection model are highly consistent with the target contour, and the model detection results have high confidence and high positioning accuracy, enabling accurate identification of UAV targets. The UAV visual detection model is more lightweight than the YOLOv5 model and is more suitable for UAV target detection.

[0170] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A UAV visual inspection method based on an improved YOLOv5s, characterized in that, Includes the following steps: Collect drone flight datasets, preprocess the drone flight datasets, and obtain drone detection images; A visual inspection model for UAVs is constructed based on a lightweight backbone network, a lightweight neck network, and an inspection head. The drone detection image is input into the drone vision detection model, and the multi-scale feature map is output through the lightweight backbone network and sent to the lightweight neck network. The multi-scale feature map is then output through the lightweight neck network and sent to the detection head to obtain the total loss function. The UAV visual detection model is trained iteratively using the total loss function. The trained UAV visual detection model is then used to identify UAVs in real-time acquired images, and the final UAV identification result is output.

2. The UAV visual inspection method based on improved YOLOv5s according to claim 1, characterized in that, The preprocessing of the UAV flight dataset includes the following sub-steps: Perform data cleaning on the UAV flight dataset and standardize the annotation format of the cleaned UAV flight dataset; Image enhancement is performed on the labeled UAV flight dataset, including pixel value normalization and size unification, to obtain UAV detection images.

3. The UAV visual inspection method based on improved YOLOv5s according to claim 1, characterized in that, The lightweight backbone network includes a Stem layer, a Shuffle module, and a Shuffle downsampling module; the Shuffle module includes a Shuffle-1 module, a Shuffle-2 module, and a Shuffle-3 module; the Shuffle downsampling module includes a Shuffle-1 downsampling module, a Shuffle-2 downsampling module, and a Shuffle-3 downsampling module; the Stem layer, the Shuffle-1 downsampling module, the Shuffle-1 module, the Shuffle-2 downsampling module, the Shuffle-2 module, the Shuffle-3 downsampling module, and the Shuffle-3 module are connected sequentially.

4. The UAV visual inspection method based on improved YOLOv5s according to claim 3, characterized in that, The Stem layer includes a convolutional layer s1, a convolutional layer s2, a convolutional layer s3, a convolutional layer s4, and a max pooling layer. The UAV detection image is input to the convolutional layer s1. The output feature map of the convolutional layer s1 is divided into two branches. The first branch enters the convolutional layer s2. The convolutional layer s2 processes the feature map through convolutional kernels and outputs the feature map to the convolutional layer s3. The convolutional layer s3 processes the feature map through convolutional kernels. The second branch enters the max pooling layer. The max pooling layer processes the feature map through pooling kernels. The output feature map of the convolutional layer s3 and the output feature map of the max pooling layer are concatenated and input to the convolutional layer s4. The convolutional layer s4 processes the feature map through convolutional kernels and outputs the feature map to the Shuffle-1 downsampling module.

5. The UAV visual inspection method based on improved YOLOv5s according to claim 1, characterized in that, The lightweight neck network includes a CBL layer, an upsample module, a C3ghost module, and a NextConv module; the CBL layer includes a CBL1 layer and a CBL2 layer; the upsample module includes an upsample1 module and an upsample2 module; the C3ghost module includes a C3ghost1 module, a C3ghost2 module, a C3ghost3 module, and a C3ghost4 module; and the NextConv module includes a NextConv1 module and a NextConv2 module. The output of the Shuffle-3 module is connected to the CBL1 layer. The first output of the CBL1 layer is connected to the upsample1 module. The output of the upsample1 module and the second output of the Shuffle-2 module are concatenated and then connected to the C3ghost1 module. The output of the C3ghost1 module is connected to the CBL2 layer. The first output of the CBL2 layer is connected to the upsample2 module. The output of the upsample2 module and the second output of the Shuffle-1 module are concatenated and then connected to the C3ghost2 module. The first output of the C3ghost2 module is connected to the NextConv1 module. The output of the NextConv1 module and the second output of the CBL2 layer are concatenated and then connected to the C3ghost3 module. The first output of the C3ghost3 module is connected to the NextConv2 module. The output of the NextConv2 module and the second output of the CBL1 layer are concatenated and then connected to the C3ghost4 module. The detection head includes a detection head 1 module, a detection head 2 module, and a detection head 3 module.

6. The UAV visual inspection method based on improved YOLOv5s according to claim 5, characterized in that, The processing procedure of the C3ghost1 module is as follows: The input feature map is divided into two branches. One branch is input to the CBL3 layer. The CBL3 layer outputs a feature map to the ghost1 module. After batch normalization and ReLU activation, the feature map output by the ghost1 module is input to the ghost2 module. The feature map output by the ghost2 module is batch normalized and then added to the feature map output by the CBL3 layer before being input to the convolutional s2 layer. The convolutional s2 layer outputs a feature map. The other branch is input to the convolutional s1 layer. The feature map output by the convolutional s1 layer is concatenated with the feature map output by the convolutional s2 layer. The concatenated feature map is input to the CBL4 layer, and the CBL4 layer outputs a feature map.

7. The UAV visual inspection method based on improved YOLOv5s according to claim 1, characterized in that, The process of obtaining the total loss function includes the following sub-steps: The drone detection images are processed by extracting preliminary features from the Stem layer of the lightweight backbone network, expanding the channel dimension, and then undergoing in-depth processing through the Shuffle module. The Shuffle downsampling module is used for spatial downsampling and channel expansion, and multi-scale feature maps C2, C3, and C4 are output to the lightweight neck network. Multi-scale feature maps C2, C3, and C4 undergo channel compression through the CBL layer of the lightweight neck network, and the spatial resolution is restored by the upsample module. They are then concatenated and fused with the feature maps output by the corresponding scale Shuffle module. Lightweight feature processing is performed through the C3ghost module, and downsampling and channel halving are performed using the NextConv module. After multi-level feature fusion and enhancement, multi-scale feature maps P2, P3, and P4 are output to the detection head. Multi-scale feature maps P2, P3, and P4 are output as prediction tensors by the two-dimensional convolutional layers of each detection head module. The prediction tensors are divided into classification prediction and bounding box regression prediction in the channel dimension, and are output to the classification loss module and the bounding box regression loss module respectively to calculate the loss. The total loss function is obtained by weighting and summing according to the preset weights.

8. The UAV visual inspection method based on improved YOLOv5s according to claim 7, characterized in that, The spatial downsampling and channel expansion using the Shuffle downsampling module includes the following steps: Input feature map X to the channel segmentation module. The channel segmentation module segments the input feature map into feature map X1 and feature map X2. Output feature map X1 to branch one and output feature map X2 to branch two. Branch A performs a 2×3 depthwise convolution operation on the input feature map. The features after the depthwise convolution operation are batch normalized. The features after the batch normalization operation are then performed a 1×1 convolution operation. The features after the convolution operation are then batch normalized. The batch normalized features are processed by the ReLU activation function to obtain the feature map F1, which is output to the preprocessing module and the feature fusion module. Input feature map X2 to branch two. Branch two performs a 1×1 convolution operation on the input feature map. The features after the convolution operation are batch normalized. The batch normalized features are processed by the ReLU activation function. The processed features are then subjected to a 3×3 depthwise convolution operation with a stride of 2. The features after the depthwise convolution operation are batch normalized. The batch normalized features are then subjected to a 1×1 convolution operation. The features after the convolution operation are batch normalized. The batch normalized features are then processed by the ReLU activation function to obtain feature map F2, which is output to the preprocessing module 2 and the feature fusion module. The preprocessing module performs a 1×1 convolution operation on the feature map F1. The convolutional features are then batch normalized, and the batch normalized features are processed by the LeakyReLU activation function to obtain the preprocessed features. Output to adaptive gating; The second preprocessing module performs a 1×1 convolution operation on the feature map F2. The convolutional features are then batch normalized, and the batch-normalized features are processed using the LeakyReLU activation function to obtain the preprocessed features. Output to adaptive gating; Adaptive gated splicing preprocessed feature one and preprocessed feature two are combined. After performing a 1×1 convolution operation on the spliced ​​features, the convolutional features are processed by the LeakyReLU activation function to obtain gate weight one, which is then output to the feature fusion module. The feature fusion module multiplies the first gate weight and the feature map F1 element by element, multiplies the second gate weight and the feature map F2 element by element, and concatenates the multiplied features to obtain the fused feature F3, which is then output to the channel shuffling module. The channel shuffling module performs channel shuffling by fusing features, promoting cross-channel information exchange, and outputting the features. .

9. The UAV visual inspection method based on improved YOLOv5s according to claim 7, characterized in that, The formula for spatial downsampling and channel expansion using the Shuffle downsampling module is as follows: ； Where F1 represents the output features of branch one, Conv is the convolution operation, DWConv is the depthwise convolution operation, ReLU is the ReLU activation function, BN is batch normalization, X1 and X2 are the feature maps output by the channel segmentation module, and F2 is the output features of branch two. For preprocessing feature one, LeakyReLU is the LeakyReLU activation function. For preprocessing feature 2, Concat is used for concatenation, W1 is the first gate weight, W2 is the second gate weight, and F3 is the fused feature. For element-wise multiplication, For channel mixed washing operation, For output features.

10. The UAV visual inspection method based on improved YOLOv5s according to claim 1, characterized in that, The iterative training of the UAV visual detection model through the total loss function includes the following sub-steps: The training dataset is input into the UAV visual detection model to perform multi-level feature extraction and complex feature reasoning to obtain recognition results. By comparing the detection results with the real label data using the total loss function, the gradient descent algorithm is used for backpropagation to dynamically adjust the weight parameters of each layer of the UAV visual detection model. Repeated training continuously optimizes the parameters of the lightweight UAV visual detection model until the training loss converges to a stable state or reaches the preset maximum number of iterations.

Images