A method and system for unmanned aerial vehicle micro-target identification and inspection

Abstract

The present application relates to a kind of unmanned aerial vehicle micro target identification inspection method and system, belong to computer vision technical field.The present application utilizes the powerful computing power of cloud computing, based on DETR original model constructs micro target detection model, micro target detection model utilizes the query guide feature sequence generated by dynamic anchor frame module to initialize the position query of decoder, simultaneously, the present application will static learnable query vector as the input of decoder, so that model can utilize accurate position information, guide decoder to extract more pure, more comprehensive micro target content features, effectively improve the flexibility and accuracy of target detection.Finally, the class information and position information are output by prediction head.The present application fully releases the potential of cloud computing power while solving the problem of missed detection in dynamic scene, suitable for high-precision intelligent supervision task in complex scene, provides effective technical support for safety hazard management.

Description

Technical Field

[0001] This invention relates to a method and system for identifying and inspecting small targets using unmanned aerial vehicles (UAVs), belonging to the field of computer vision technology. Background Technology

[0002] The construction industry has long relied on manual safety supervision. In recent years, the introduction of drone inspections has greatly improved the efficiency of inspections across all scenarios. With the development of computer vision technology, the identification of medium- to large-scale compliant behaviors with significant colors and fixed shapes, such as "safety helmets" or "reflective vests," has become relatively mature. However, in actual safety management, the source of accidents often lurks in more subtle violations. For example, smoking in violation of regulations at a construction site can easily ignite flammable materials and cause a fire, but a cigarette butt occupies only a few pixels in the wide field of view of a drone, making it extremely difficult to detect.

[0003] In traditional detection models, due to the extremely wide range of target scales, the strong semantic features of prominent targets such as helmets and reflective vests often dominate the gradient updates of the network. This leads to the suppression or even complete "submergence" of tiny targets with extremely small pixel proportions (such as cigarette butts / smoke features) during the feature extraction process of deep networks, resulting in an extremely high false negative rate. Furthermore, unlike conventional protective equipment with prominent features, judging minute illegal smoking actions places extremely stringent requirements on the network's ability to preserve high-frequency textures and its resolution of microscopic features.

[0004] There is an urgent need for an identification and inspection method that can accurately identify prominent targets while also effectively identifying small targets at the construction site. Summary of the Invention

[0005] The purpose of this invention is to provide a method and system for identifying and inspecting small targets using unmanned aerial vehicles (UAVs). This invention improves the detection model of small targets based on the original DETR (Detection Transformer) model, which can accurately identify significant targets while effectively improving the detection accuracy of small targets, so as to achieve accurate identification and early warning of various violations at construction sites.

[0006] In a first aspect, the present invention provides a method for identifying and inspecting small targets using a drone, comprising:

[0007] Acquire inspection images of the construction site from the perspective of a drone;

[0008] The inspection image is input into a pre-trained small target detection model to obtain the inspection recognition result;

[0009] The small target detection model is an improvement on the original DETR model. The improvements include:

[0010] The original DETR model's CNN backbone network is replaced with a main network and a feature fusion network connected in sequence; the main network is built based on a fusion-type residual convolutional model, and the feature fusion network is built based on an adaptive bidirectional feature pyramid module.

[0011] In the original DETR model, a query selection module is set at the connection position between the top layer of the encoder and the decoder. The fused feature map output by the feature fusion network is combined with the position encoding matrix and input into the encoder to obtain the first self-attention feature sequence. The first self-attention feature sequence is input into the query selection module to filter out the feature vectors of key spatial locations and map them back to the continuous spatial coordinate system corresponding to the inspection image to obtain the spatial location prior feature sequence.

[0012] In the original DETR model, a dynamic anchor box module is set at the connection position between the decoder and the encoder at the bottom layer. The spatial location prior feature sequence is input into the dynamic anchor box module to generate a query-guided feature sequence in which the spatial location of the reference point is given by the dynamic anchor box. The spatial location of the reference point corresponds to the key spatial location. The query-guided feature sequence is combined with the learnable query vector and input into the decoder to obtain the second self-attention feature sequence. The second self-attention feature sequence and the first self-attention feature sequence are combined with the learnable query vector and the position encoding matrix respectively and then cross-attention processing is performed to obtain the third self-attention feature sequence. The third self-attention feature sequence is processed by the top layer of the decoder to generate the inspection and recognition query vector used to input the prediction head.

[0013] Optionally, the backbone network includes a convolutional layer 1, a fused residual convolutional module 1, a fused residual convolutional module 2, a fused residual convolutional module 3, and a residual convolutional module connected in series.

[0014] The three fusion residual convolution modules I, II, and III all include a fusion residual convolution module structure.

[0015] The feature map of the input fusion residual convolutional module structure is passed sequentially through a 1×1 convolutional layer, a 3×3 convolutional layer, an improved CBAM attention mechanism layer, a 1×1 convolutional layer, and a random deactivation layer, and then the residual is added to itself to obtain the output;

[0016] The improved CBAM attention mechanism layer includes a cascaded CAM-G Gaussian channel attention module and a SAM-G Gaussian spatial attention module;

[0017] The feature fusion network includes a 1×1 convolutional layer three, a 1×1 convolutional layer four, a 1×1 convolutional layer five, and an adaptive bidirectional feature pyramid module;

[0018] The outputs of the fusion residual convolution module II, fusion residual convolution module III, and residual convolution module are used as multi-scale feature maps. After channel unification through 1×1 convolution layer III, 1×1 convolution layer IV, and 1×1 convolution layer V, they are input in parallel into the adaptive bidirectional feature pyramid module.

[0019] The adaptive bidirectional feature pyramid module includes a global pooling layer, a fully connected layer five, a normalization layer, a BiFPN module, and a 3×3 convolutional layer two, connected in sequence.

[0020] Optionally, the encoder includes multiple encoder layers, each encoder layer including: a multi-head self-attention mechanism layer, a residual layer one, a fully connected layer six, a residual layer two, and a query selection module;

[0021] The fused feature map output by the feature fusion network is unified in terms of the number of channels to obtain the input feature sequence of the encoder layer. The input feature sequence is linearly mapped to obtain query matrix 1, key matrix 1, and value matrix 1. Query matrix 1 and key matrix 1 are added to the position encoding matrix respectively, and then fed together with value matrix 1 into the multi-head self-attention mechanism layer to obtain self-attention weight matrix 1. Self-attention weight matrix 1 and the input feature sequence of the encoder layer are weighted and normalized in residual layer 1 to obtain the self-attention feature extraction sequence.

[0022] The self-attention feature extraction sequence is passed through a fully connected layer six, and then weighted and normalized with itself in a residual layer two to obtain the first self-attention feature sequence;

[0023] The first self-attention feature sequence is processed by the query selection module to obtain the spatial location prior feature sequence.

[0024] Optionally, the expression for a single attention head in a multi-head self-attention mechanism layer is:

[0025] ;

[0026] in, This represents a feature dimension scaling factor of 1; Represents the position encoding matrix; , and These represent query matrix one, key matrix one, and value matrix one, respectively. This represents the normalized exponential function; This represents the self-attention weight matrix for a single attention head. The self-attention weight matrix is ​​obtained by concatenating the outputs of all attention heads in the multi-head self-attention mechanism layer along the feature dimension.

[0027] Optionally, the query selection module includes a linear projection network, a sorting and truncation unit, an anchor box mapping layer, and an activation function mapping layer;

[0028] The first self-attention feature sequence of the input query selection module is first input into the linear projection network. The linear projection network performs binary classification on the feature vectors corresponding to each spatial location in the first self-attention feature sequence to evaluate the target confidence and give the probability score that the spatial location belongs to the foreground target.

[0029] After the target confidence assessment, the first self-attention feature sequence is input into the sorting and truncation unit. The sorting and truncation unit selects the feature vectors corresponding to the spatial positions with the highest probability scores as the feature vectors of the key spatial positions.

[0030] The feature vectors of key spatial locations are downsampled and their size restored by using anchor box mapping layer and activation function mapping layer to obtain the prior feature sequence of spatial location.

[0031] Alternatively, the expression for the linear projection network is:

[0032] ;

[0033] in, This indicates that the first self-attention feature sequence of the input query selection module corresponds to the first... Feature vectors of spatial locations and Let represent the weight matrix and bias parameters, respectively, for feature learning in the linear projection network. Represents a sigmoid function. The output of the linear projection network represents the first... The probability score of a spatial location belonging to a foreground target.

[0034] Optionally, the decoder includes multiple decoder layers, each decoder layer including a dynamic anchor frame module, a deformable multi-head attention mechanism layer 1, a residual layer 3, a deformable multi-head attention mechanism layer 2, a residual layer 4, a fully connected layer 7, and a residual layer 5;

[0035] The prior spatial location feature sequence is input into the dynamic anchor box module to generate a query-guided feature sequence in which the spatial location of the reference point is given by the dynamic anchor box. The query-guided feature sequence is linearly mapped to obtain query matrix two, key matrix two, and value matrix two. Query matrix two and key matrix two are added to the learnable query vector, and then together with value matrix two are input into deformable multi-head attention mechanism layer one to obtain self-attention weight matrix two. Self-attention weight matrix two and query-guided feature sequence are weighted and normalized in residual layer three to obtain the second self-attention feature sequence.

[0036] The second self-attention feature sequence is linearly mapped to obtain query matrix three. The first self-attention feature sequence is linearly mapped to obtain key matrix three and value matrix three. Key matrix three and query matrix three are added to the position encoding matrix and the learnable query vector, respectively, and then input together with value matrix three into deformable multi-head attention mechanism layer two to obtain self-attention weight matrix three. Self-attention weight matrix three and the second self-attention feature sequence are weighted and normalized in residual layer four to obtain the third self-attention feature sequence.

[0037] The third self-attention feature sequence is processed by the fully connected layer seven and then weighted and normalized with itself in the residual layer five to obtain the inspection and identification query vector.

[0038] Optionally, the expression for a single deformable attention head in Layer 1 of the deformable multi-head attention mechanism is:

[0039] ;

[0040] in, This represents the normalized exponential function; , and These represent query matrix two, key matrix two, and value matrix two, respectively. Represents a static, learnable query vector; Indicates the spatial position of the reference point given by the dynamic anchor frame module; Indicates the first The offset of each key point in dynamic sampling; This represents the total number of keypoints dynamically sampled near the spatial location of the reference point. This indicates a feature dimension scaling factor of two; The first layer represents the self-attention weight matrix of the output of a single deformable attention head in the deformable multi-head attention mechanism layer 1. The outputs of multiple deformable attention heads in the first layer of the deformable multi-head attention mechanism layer 1 are concatenated along the feature dimension to obtain the second self-attention weight matrix.

[0041] The expression for a single deformable attention head in Layer 2 of the deformable multi-head attention mechanism is:

[0042] ;

[0043] in, This represents the normalized exponential function; , and These represent query matrix three, key matrix three, and value matrix three, respectively. Represents a static, learnable query vector; Represents the position encoding matrix; Indicates the spatial position of the reference point given by the dynamic anchor frame module; Indicates the first The offset of each key point in dynamic sampling is two; This indicates that the total number of key points dynamically sampled near the spatial location of the reference point is two; This indicates a feature dimension scaling factor of three; The self-attention weight matrix represents the output of a single deformable attention head in the second layer of the deformable multi-head attention mechanism. The outputs of multiple deformable attention heads in the second layer of the deformable multi-head attention mechanism are concatenated along the feature dimension to obtain the third self-attention weight matrix.

[0044] Optionally, the prediction head includes a fully connected layer eight and a fully connected layer nine, and the inspection and identification results include category information and location information;

[0045] The inspection and identification query vector is input in parallel to the fully connected layer 8 and the fully connected layer 9; the fully connected layer 8 outputs category information, and the fully connected layer 9 outputs location information.

[0046] Secondly, the present invention provides a UAV small target identification and inspection system, comprising:

[0047] The acquisition module is used to acquire inspection images of the construction site from the perspective of a drone.

[0048] The detection module is used to input the inspection image into a pre-trained small target detection model to obtain the inspection recognition result. The small target detection model is an improvement on the original DETR model. The improvements include: replacing the CNN backbone network of the original DETR model with a main backbone network and a feature fusion network connected in sequence; the main backbone network is built based on a fusion-type residual convolutional model, and the feature fusion network is built based on an adaptive bidirectional feature pyramid module; a query selection module is set at the connection position between the encoder and decoder in the original DETR model. The fused feature map output by the feature fusion network is combined with the position encoding matrix and input into the encoder to obtain the first self-attention feature sequence. The first self-attention feature sequence is input into the query selection module to filter out the feature vectors of key spatial locations and map them back to the inspection. The image is used to obtain a spatial location prior feature sequence in a continuous spatial coordinate system. In the original DETR model, a dynamic anchor box module is set at the connection position between the decoder and the encoder. The spatial location prior feature sequence is input into the dynamic anchor box module to generate a query-guided feature sequence in which the spatial location of the reference point is given by the dynamic anchor box. The spatial location of the reference point corresponds to the key spatial location. The query-guided feature sequence is combined with a learnable query vector and input into the decoder to obtain a second self-attention feature sequence. The second self-attention feature sequence and the first self-attention feature sequence are combined with the learnable query vector and the position encoding matrix, respectively, and then cross-attention processing is performed to obtain a third self-attention feature sequence. The third self-attention feature sequence is processed by the top layer of the decoder to generate an inspection and recognition query vector for input into the prediction head.

[0049] Compared with existing technologies, the beneficial effects achieved by this invention are as follows: This invention proposes a method for identifying and inspecting small targets using unmanned aerial vehicles (UAVs). This invention utilizes the powerful computing power of cloud computing to construct a small target detection model designed for small targets based on the original DETR model. While accurately identifying significant targets, it effectively improves the detection accuracy of small targets, thereby enabling accurate identification and early warning of various violations at construction sites. This method first acquires inspection images of the construction site from the perspective of an unmanned aerial vehicle (UAV). The inspection images are then input into a micro-target detection model. In this invention, a query selection module is set at the connection point between the encoder and decoder in the original DETR model. The fused feature map output by the feature fusion network, combined with the position encoding matrix, is input into the encoder to obtain a first self-attention feature sequence. This first self-attention feature sequence is input into the query selection module to filter out feature vectors of key spatial locations and map them back to the continuous spatial coordinate system corresponding to the inspection image to obtain a priori spatial location feature sequence. Furthermore, this invention also sets a dynamic anchor frame module at the connection point between the decoder and encoder in the original DETR model. The priori spatial location feature sequence is input into the dynamic anchor frame module to generate a query-guided feature sequence that provides the spatial location of a reference point. The reference point spatial location corresponds to a key spatial location. The query-guided feature sequence, combined with a learnable query vector, is input into the decoder to obtain a second self-attention feature sequence. The second and first self-attention feature sequences, combined with the learnable query vector and the position encoding matrix respectively, undergo cross-attention processing to obtain a third self-attention feature sequence. The third self-attention feature sequence is then processed by the decoder's top layer to generate an inspection recognition query vector for inputting into the prediction head. The inspection and identification query vector is input into the prediction head to obtain the inspection and identification result, which includes the location information and category information of the target object.

[0050] This invention utilizes a query-guided feature sequence generated by a dynamic anchor box module to initialize the decoder's location query. Simultaneously, it uses a static, learnable query vector as input to the decoder, enabling the model to leverage precise location information to guide the decoder in extracting purer and more comprehensive features of minute target content, effectively improving the flexibility and accuracy of target detection. Finally, the prediction head outputs category and location information. This invention fully leverages the potential of cloud computing power while solving the problem of missed detections in dynamic scenarios, making it suitable for high-precision intelligent monitoring tasks in complex environments and providing effective technical support for security hazard management. Attached Figure Description

[0051] Figure 1 This is a flowchart illustrating the UAV micro-target identification and inspection method provided in an embodiment of the present invention.

[0052] Figure 2This is a schematic diagram of the structure of the small target detection model in an embodiment of the present invention;

[0053] Figure 3 This is a schematic diagram of the structure of the CAM-G Gaussian channel attention module in an embodiment of the present invention;

[0054] Figure 4 This is a schematic diagram of the SAM-G Gaussian spatial attention module in an embodiment of the present invention;

[0055] Figure 5 This is a schematic diagram of the feature fusion network in an embodiment of the present invention. Detailed Implementation

[0056] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0057] Combination Figure 1 This invention provides a method for identifying and inspecting small targets using a drone, comprising the following steps: Step S1, acquiring inspection images of the construction site from the drone's perspective.

[0058] In this embodiment, the drone establishes a connection with a 5G high-speed base station via a built-in high-gain antenna. A cloud server cluster is deployed in a remote data center, running high-performance deep learning models using Docker containerization technology. The drone maintains low-latency communication with the cloud via real-time streaming protocols (RTMP and WebRTC) to transmit uncompressed ultra-high-definition video streams. The ground remote control terminal receives data from the cloud via the network, receiving the inspection and identification results inferred from the cloud in real time.

[0059] This embodiment acquires real-time on-site inspection video streams from the camera equipment mounted on the drone, extracts each frame of the image, and then sequentially scales the extracted images to unify the input size and normalizes the pixel values ​​to obtain the inspection image.

[0060] Step S2: Input the inspection image into the pre-trained small target detection model to obtain the inspection recognition result.

[0061] This embodiment uses inspection images as input and performs detection based on a small target detection model to obtain inspection recognition results. In this embodiment, the inspection recognition results include the location information and category information of the target object. The location information includes the width of the target bounding box, the height of the target bounding box, the x-coordinate of the center position of the target bounding box, and the y-coordinate of the center position of the target bounding box. The category information includes both salient targets (such as safety helmets and reflective vests) and small targets (such as cigarette butts). The specific category information can be set according to actual safety requirements.

[0062] like Figure 2 As shown, the small target detection model is an improvement on the original DETR model. The improvements include: replacing the CNN backbone network of the original DETR model with a main backbone network and a feature fusion network connected in series; the main backbone network is built based on a fusion-type residual convolutional model, and the feature fusion network is built based on an adaptive bidirectional feature pyramid module; a query selection module is set at the connection position between the encoder top layer and the decoder in the original DETR model; and a dynamic anchor box module is set at the connection position between the decoder bottom layer and the encoder in the original DETR model.

[0063] Specifically, the small target detection model in this embodiment includes a backbone network, a feature fusion network, an encoder, a decoder, and a prediction head. The backbone network performs multi-layer convolutional operations on the input inspection image to extract multi-scale feature maps containing rich texture details. The feature fusion network receives the multi-scale feature maps output by the backbone network and outputs a fused feature map. Through weighted fusion and a pyramid structure, the feature fusion network deeply integrates low-level high-resolution details with high-level semantic context, enhancing the perception of small targets. The encoder uses a multi-head self-attention mechanism to perform global context modeling on the fused feature map, correlating spatial and channel information to obtain a more discriminative image representation. The decoder interacts with the learnable query vector through a deformable multi-head attention mechanism, focusing on sparser or critical locations to gradually refine the target information corresponding to each query and reconstruct the inspection and recognition query vector. The prediction head performs classification and bounding box regression on the inspection and recognition query vector output by the decoder, ultimately outputting the target object's category information and location information (predicted bounding box coordinates).

[0064] Specifically, in combination Figure 2 The backbone network includes a first convolutional layer, a first fused residual convolutional module, a second fused residual convolutional module, a third fused residual convolutional module, and a residual convolutional module connected in sequence. In this embodiment, fused residual convolutional modules are used in the shallow layers of the backbone network to improve the running speed, while ordinary residual convolutional modules are used in the deeper layers. The first, second, and third fused residual convolutional modules all include a fused residual convolutional module structure. Figure 2As shown, the feature map of the input fusion residual convolutional module structure passes through a 1×1 convolutional layer, a 3×3 convolutional layer, an improved CBAM (Convolutional Block Attention Module) attention mechanism layer, a 1×1 convolutional layer, and a random deactivation layer in sequence, and is then added to itself as a residual to obtain the output. The fusion residual convolutional module structure can map complex high-dimensional feature maps into low-dimensional feature vectors, thereby effectively capturing the global dependencies between channels and different spatial locations. The 1×1 convolutional layer 1 expands the number of channels in the input feature map, significantly increasing its depth. The output of the 1×1 convolutional layer 1 is processed by a normalized activation function and then input into the 3×3 convolutional layer 1. The output of the 3×3 convolutional layer 1 is processed by a normalized activation function and then input into the improved CBAM attention mechanism layer. The output of the improved CBAM attention mechanism layer is compressed back to match the original input of the fusion residual convolutional module structure by the 1×1 convolutional layer 2. Finally, a random deactivation layer is used to prevent overfitting and improve the model's generalization ability. The output of the random deactivation layer is added to the original input of the fusion residual convolutional module structure as a residual to promote gradient flow and improve model performance.

[0065] The improved CBAM attention mechanism layer in this embodiment employs Gaussian error units, which can better balance spatial feature information, increase nonlinearity and statistical properties, and improve the performance of the backbone network in acquiring target features while maintaining the stability between the backbone network, the encoder, and the decoder. The improved CBAM attention mechanism layer includes a cascaded CAM-G Gaussian channel attention module and a SAM-G Gaussian spatial attention module.

[0066] The structure of the CAM-G Gaussian channel attention module is as follows: Figure 3 As shown, the feature map input to the CAM-G Gaussian channel attention module is processed in parallel through the channel average pooling branch and the channel max pooling branch, and then processed by the activation function to obtain the channel weights. The channel weights are used to perform channel weighting processing on the feature map input to the improved CBAM attention mechanism layer. The channel average pooling branch includes an average pooling layer 1, a nonlinear fully connected layer 1, a Gaussian error unit 1, and a nonlinear fully connected layer 2 connected in sequence. The channel max pooling branch includes a max pooling layer 1, a fully connected layer 3, a Gaussian error unit 1, and a fully connected layer 4 connected in sequence. The channel average pooling branch and the channel max pooling branch share a single Gaussian error unit 1. The CAM-G Gaussian channel attention module improves the model's attention to important information and enhances its ability to extract target information.

[0067] The expressions for average pooling layer 1 and max pooling layer 1 are as follows:

[0068] ;

[0069] in, This represents the feature map of the input CAM-G Gaussian channel attention module; This represents the channel index of the feature map, used to locate which specific feature channel it is; and These represent the row and column coordinate indices of the feature map in the spatial dimension, respectively; and These represent the height and width of the feature map input to the CAM-G Gaussian channel attention module, respectively. This represents the vector output by the average pooling layer. This represents the vector output by the first max pooling layer.

[0070] The expressions for fully connected layer 1 and fully connected layer 3 are:

[0071] ;

[0072] in, and Let represent the weight matrices of fully connected layer 1 and fully connected layer 3, respectively. and These represent the biases of fully connected layer 1 and fully connected layer 3, respectively. This represents the intermediate feature vector obtained by passing the vector output by the average pooling layer 1 through the fully connected layer 1. This represents the intermediate feature vector 2 obtained by passing the vector output by the first max pooling layer through the third fully connected layer; This represents the Gaussian error eigenvector obtained after the intermediate eigenvector is processed by Gaussian error unit one. This represents the Gaussian error eigenvector obtained after the intermediate eigenvector passes through Gaussian error unit one; This represents the Gaussian error function.

[0073] This embodiment uses the Gaussian error function (GELU module) to process the intermediate feature vector. Intermediate feature vector two To activate the Gaussian error function, the expression is:

[0074] ;

[0075] in, Indicates the independent variable. Represent the independent variable The processing function; This represents the error function.

[0076] The expressions for fully connected layer 2 and fully connected layer 4 are:

[0077] ;

[0078] in, and These represent the weight matrices of the second and fourth fully connected layers, respectively. and These represent the biases of fully connected layer 2 and fully connected layer 4, respectively. Represents the Gaussian error eigenvector. The first output feature vector obtained after the second fully connected layer Represents the second Gaussian error eigenvector. The output feature vector two is obtained after passing through the fully connected layer four.

[0079] Output feature vector 1 and output feature vector 2 Channels and vectors obtained by element-wise addition Channel weights are obtained by processing with the sigmoid function. The expression for the sigmoid function is:

[0080] ;

[0081] in, This represents the natural exponential function.

[0082] Channel weights for each channel The weighted output feature map is obtained by applying the weighted input feature map to the corresponding channel of the original input feature map of the CAM-G Gaussian channel attention module, and the expression is:

[0083] ;

[0084] in, This represents the weighted feature map output by the CAM-G Gaussian channel attention module.

[0085] The structure of the SAM-G Gaussian spatial attention module is as follows: Figure 4 As shown, the feature map input to the SAM-G Gaussian spatial attention module passes through the second average pooling layer and the second max pooling layer in parallel before entering the stitching layer. The outputs of the second average pooling layer and the second max pooling layer are stitched together in the stitching layer. The output of the stitching layer passes through the second Gaussian error unit and a 7×7 convolutional layer, and then is activated by an activation function to generate spatial weights. These spatial weights are used to perform spatial weighting processing on the feature map input to the SAM-G Gaussian spatial attention module.

[0086] The SAM-G Gaussian spatial attention module maintains the consistency of the feature space dimensions through a 7×7 convolutional layer with padding of 3, thereby enhancing the model's attention to the location information in the feature space. The expression for the 7×7 convolutional layer is as follows:

[0087] ;

[0088] in, This represents the current spatial position after processing by a 7×7 convolutional layer, where 1 indicates the number of channels. and These represent the row and column coordinate indices of the feature map output by the 7×7 convolutional layer in the spatial dimension. This represents the channel index of the feature map input to the 7×7 convolutional layer. and This represents the spatial local coordinate index inside the kernel of a 7×7 convolutional layer; This represents the bias term of a 7×7 convolutional layer. This represents the weight tensor of the convolution kernel in a 7×7 convolutional layer. The feature map output from the stitching layer of the SAM-G Gaussian spatial attention module is passed through the output of Gaussian error unit two.

[0089] The data processing procedure of a fused residual convolutional module structure is illustrated using a specific example. The feature map input to the fused residual convolutional module structure is 320×320×3, i.e., an RGB image. After passing through a 1×1 convolutional layer, the input feature map is expanded to 32 channels, resulting in a size of 320×320×32. The output of the 1×1 convolutional layer is processed by a normalized activation function and then passed through a 3×3 convolutional layer to extract spatial features, without changing the image size; the output feature map of the 3×3 convolutional layer remains 320×320×32. The output of the 3×3 convolutional layer is then processed by a normalized activation function and input to the improved CBAM attention mechanism layer. The output of the improved CBAM attention mechanism layer is then passed through a 1×1 convolutional layer. The number of channels in the feature map is compressed back to match the original input of the fusion residual convolutional module structure, thereby reducing the dimensionality of the feature map. The feature map output from the second 1×1 convolutional layer is restored to 320×320×3. After normalization, the feature map output from the second 1×1 convolutional layer is input to a random deactivation layer. The random deactivation layer prevents overfitting and improves the model's generalization ability. The output of the random deactivation layer is added to the original feature map input to the fusion residual convolutional module structure using residuals to obtain the output feature map, which is 320×320×3 in size. Multiple fusion residual convolutional module structures can extract feature maps of different scales. When the feature map is processed by a fusion residual convolutional module structure with a stride of 2, the width and height of the feature map are reduced to half of their original size, while the number of channels increases. This embodiment sets up three fusion-type residual convolutional module structures (fusion-type residual convolutional module one, fusion-type residual convolutional module two, and fusion-type residual convolutional module three). In this embodiment, feature maps P3, P4, and P5 output by fusion-type residual convolutional module two, fusion-type residual convolutional module three, and residual convolutional module three are used for feature fusion operation. That is, feature maps P3, P4, and P5 with sizes of 160×160×64, 80×80×128, and 40×40×256 respectively are fed into the feature fusion network.

[0090] The feature fusion network includes a 1×1 convolutional layer three, a 1×1 convolutional layer four, a 1×1 convolutional layer five, and an adaptive bidirectional feature pyramid (Adaptive BiFPN) module. The feature fusion network first needs to unify the number of channels of feature maps P3, P4, and P5 to 160 through the 1×1 convolutional layer three, 1×1 convolutional layer four, and 1×1 convolutional layer five. That is, the size of feature maps P3, P4, and P5 is adjusted to 160×160×160, 80×80×160, and 40×40×160, respectively.

[0091] like Figure 5As shown, the adaptive bidirectional feature pyramid module includes a global pooling layer, a fully connected layer five, a normalization layer, a BiFPN module, and a 3×3 convolutional layer two connected in sequence.

[0092] Combination Figure 5 This paper describes the fusion operation of the adaptive bidirectional feature pyramid module. The image processing of the adaptive bidirectional feature pyramid module includes: First, applying global average pooling (GAP) to feature maps P3, P4, and P5 after unifying the number of channels to obtain channel description vectors whose spatial dimension is compressed to the number of channels. Next, each channel description vector is input into the fifth fully connected layer (a shared two-layer fully connected neural network, i.e., a multilayer perceptron, MLP) to learn the unnormalized fusion weights of that input feature branch. Then, normalization is applied to all unnormalized fusion weights to obtain the final fusion weights. Before calculating the fusion, it is necessary to align the dimensions of all feature maps (including spatial resolution and number of channels). For feature maps with inconsistent spatial dimensions, upsampling (e.g., bilinear interpolation) or downsampling (e.g., 1×1 convolution with a stride of 2) is used to unify them to the target size; when the number of channels is inconsistent, it is uniformly adjusted to a unified number of channels through 1×1 convolution, thus obtaining aligned feature maps. Taking feature map P4 (80×80×160) as an example: feature map P3 is downsampled to 80×80×160; feature map P5 is upsampled to 80×80×160. At this point, the aligned feature maps P3, P4, and P5 all have a size of 80×80×160. After alignment, the feature maps are weighted and summed according to the fusion weights to obtain the fused features. Finally, the fused features are enhanced through a 3×3 convolutional layer to obtain the final feature map output by the adaptive bidirectional feature pyramid module.

[0093] The expression for the adaptive bidirectional feature pyramid module is:

[0094] ;

[0095] in, The input adaptive bidirectional feature pyramid module represents the first... Each feature map and These represent the horizontal and vertical coordinates of the feature map in spatial dimensions, respectively. This indicates a global average pooling operation; and They represent the first The height and width of each feature map; Indicates the first Channel description vectors obtained by global average pooling of each feature map; This indicates multilayer perceptron processing; This represents the modified linear unit used as an activation function; and These represent the weight matrices of the first and second fully connected layers in the five-layer fully connected system, respectively. and These represent the biases of the first and second fully connected layers in the fifth fully connected layer system, respectively. Represents the natural exponential function; Indicates the first Unnormalized fusion weights of each feature map; This represents the traversal index of the input feature branch of the fusion node in the BiFPN module. This represents the total number of input feature branches. Indicates the first Unnormalized fusion weight score of each input feature branch Indicates the first The fusion weights after normalizing the feature maps This represents the total number of input feature maps; This represents the fused feature obtained by weighted fusion of all aligned feature maps. Indicates the first The feature map after spatial size alignment; Indicates the first The fusion weights after normalization of each input feature branch; Indicates the learning rate; This represents the feature map output by the adaptive bidirectional feature pyramid module; This indicates a 3×3 convolution process.

[0096] like Figure 2 As shown, the encoder includes multiple encoder layers. Each encoder layer includes a multi-head self-attention mechanism layer, residual layer one, fully connected layer six, residual layer two, and a query selection module. The fused feature map output by the feature fusion network is passed through a 1×1 convolution to unify the number of channels, resulting in the input feature sequence for the encoder layer. The input feature sequence is linearly mapped to obtain query matrix one, key matrix one, and value matrix one. Query matrix one and key matrix one are added to the position encoding matrix, and then together with value matrix one, they are input into the multi-head self-attention mechanism layer to obtain self-attention weight matrix one. Self-attention weight matrix one and the input feature sequence for the encoder layer are weighted and normalized in residual layer one to obtain the self-attention feature extraction sequence.

[0097] Specifically, in this embodiment, the fused feature map output by the feature fusion network is first converted into the input feature sequence by a 1×1 convolution to unify the number of channels. Input feature sequence The query matrix is ​​obtained through three sets of independent linear mapping transformations. Key matrix Sum Matrix 1 The corresponding expression is:

[0098] ;

[0099] In this multi-head self-attention mechanism layer, each attention head has a corresponding set of mapping parameters. Represents query matrix one The weight matrix, Represents query matrix one The bias vector, Key matrix one The weight matrix, Key matrix one The bias vector; Value matrix 1 The weight matrix, Value matrix 1 The bias vector can be obtained from the query matrix. Key matrix Sum Matrix 1 The value of is calculated for the self-attention weight matrix of the divided attention heads. The expression for a single attention head in the multi-head self-attention mechanism layer is:

[0100] ;

[0101] in, This represents the normalized exponential function; This represents a feature dimension scaling factor of 1; This represents the location encoding matrix, which is used to provide the model with spatial location information of each element in the feature sequence; This represents the attention scoring matrix, after... The function is then converted to a weight distribution, and then compared with the value matrix. Multiplying them yields the self-attention weight matrix of a single attention head that aggregates global information. After all attention heads in the multi-head self-attention mechanism layer have completed their parallel computations, their outputs are concatenated along the feature dimension to obtain the first self-attention weight matrix. This first self-attention weight matrix, along with the input feature sequence from the input encoder layer, is then weighted and normalized in the first residual layer to obtain the self-attention feature extraction sequence. This self-attention feature extraction sequence passes through a fully connected layer six and is then weighted and normalized again with itself in the second residual layer to obtain the first self-attention feature sequence. Multiple attention heads can capture global dependencies from different subspaces in parallel. Combined with residuals and normalization, this ultimately generates "enhanced features" that retain the original features while being rich in contextual relationships.

[0102] This embodiment aims to enhance the spatial localization and semantic representation capabilities of the query mechanism in a small target detection model by adding a query selection module at the connection point between the encoder and decoder, based on a mixed query selection method. This query selection module can more effectively guide the features output by the encoder to the decoder, enhancing the model's location awareness while maintaining the learnability of content information.

[0103] The first self-attention feature sequence is input into the query selection module to filter out feature vectors of key spatial locations and map them back to the continuous spatial coordinate system corresponding to the original image input to the small target detection model to obtain the spatial location prior feature sequence. The query selection module includes a linear projection network, a sorting and cropping unit, an anchor box mapping layer, and an activation function mapping layer connected in series. The specific processing of the first self-attention feature sequence by the query selection module is as follows: First, the first self-attention feature sequence is input into the linear projection network. Since the background of the construction site under the drone aerial view is extremely complex, the linear projection network learns the global context features extracted by the encoder, performs binary classification of the target confidence of the feature vectors corresponding to each spatial location in the first self-attention feature sequence, and gives a probability score that the spatial location belongs to the foreground target. Specifically, the linear projection network outputs a probability score between 0 and 1, which is used to accurately quantify the probability that the spatial location of the feature vector belongs to the foreground target, thereby achieving preliminary filtering of large areas of invalid background at the feature level. The expression of the linear projection network is:

[0104] ;

[0105] in, This indicates that the first self-attention feature sequence of the input query selection module corresponds to the first... Feature vectors of spatial locations and Let represent the weight matrix and bias parameters, respectively, for feature learning in the linear projection network. This represents a sigmoid function used to limit and map the output between 0 and 1. The output of the linear projection network represents the first... The probability score of a spatial location belonging to a foreground target.

[0106] The sorting and cropping unit receives the probability scores of all spatial locations belonging to the foreground target from the linear projection network. The sorting and cropping unit first sorts all spatial locations in descending order according to the probability scores, and then directly crops the feature vectors of the top K (Top-K) spatial locations with the highest probability scores according to the preset hyperparameter K, as the feature vectors of the key spatial locations.

[0107] Subsequently, the anchor box mapping layer, combined with the activation function mapping layer, transforms the selected K feature vectors into a spatial location prior feature sequence that can be directly used by the decoder. The specific processing steps of the anchor box mapping layer and the activation function mapping layer include: extracting the two-dimensional grid coordinates of these K feature vectors on the current scale feature map; predicting the scale offset of the two-dimensional grid coordinates relative to the grid center through a linear mapping layer; the linear mapping layer can accurately convert deep, high-dimensional semantic features into specific geometric spatial offset values; and combining the downsampling step size of the feature map relative to the high-resolution original image of the input small target detection model, using the anchor box mapping layer and the activation function mapping layer to downsample and resize the K feature vectors to obtain the spatial location prior feature sequence. The expressions for downsampling and resize are as follows:

[0108] ;

[0109] ;

[0110] ;

[0111] in, and These represent the size of the original image input to the small target detection model and the size of the feature map containing the current feature vector, respectively. This indicates the downsampling step size of the feature map relative to the original image; This represents the two-dimensional grid coordinates of the selected feature vectors on the feature map at the current scale; This represents the scale offset of the predicted two-dimensional grid coordinates relative to the center of the grid from the linear mapping layer. The feature vector representing the selected spatial location; and These represent the weight matrix and bias parameters for feature transformation performed in the first linear mapping layer, respectively. This indicates that the selected feature vectors are mapped back to their actual coordinate positions in the continuous spatial coordinate system corresponding to the original image.

[0112] like Figure 2 As shown, the decoder comprises multiple decoder layers. Each decoder layer includes a dynamic anchor box module, a deformable multi-head attention mechanism layer 1, a residual layer 3, a deformable multi-head attention mechanism layer 2, a residual layer 4, a fully connected layer 7, and a residual layer 5. The spatial location prior feature sequence is input to the dynamic anchor box module, generating a query-guided feature sequence that provides the spatial location of a reference point. This reference point corresponds to a key spatial location. The query-guided feature sequence is used to initialize the decoder's location query to guide the subsequent spatial focusing of the deformable multi-head attention mechanism in the key region. Specifically, the dynamic anchor box adopts a four-dimensional anchor box format. Each dynamic anchor box consists of position coordinates and scale information, denoted as (x...).anchor y anchor w anchor h anchor ), where x anchor y anchor w represents the normalized coordinates of the anchor frame center point. anchor and h anchor These are the width and height of the dynamic anchor frame, respectively.

[0113] The query-guided feature sequence is linearly mapped to obtain query matrix two, key matrix two, and value matrix two. Query matrix two and key matrix two are added to the learnable query vector, and then input together with value matrix two into the deformable multi-head attention mechanism layer one to obtain self-attention weight matrix two. In this embodiment, the decoder's position query is initialized through the query-guided feature sequence, while the static learnable query vector is used as the decoder's input. The learnable query vector remains a static and learnable parameter, not directly dependent on the selected features of the query selection module and the dynamic anchor box module. This avoids semantic interference introduced by the selected features covering multiple targets or parts of targets, enabling the model to use precise spatial location information to guide the decoder to extract purer and more comprehensive features of small target content.

[0114] The expression for a single deformable attention head in Layer 1 of the deformable multi-head attention mechanism is:

[0115] ;

[0116] in, This represents the normalized exponential function; , and These represent query matrix two, key matrix two, and value matrix two, respectively. Represents a static, learnable query vector; This indicates the spatial position of the reference point corresponding to the key spatial position, given by the dynamic anchor frame module; The second linear mapping layer predicts the first... The offset of each key point in dynamic sampling; This represents the total number of keypoints dynamically sampled near the spatial location of the reference point. This represents a feature dimension scaling factor of 2, used to prevent gradient vanishing due to excessively large dot product values. The first layer represents the self-attention weight matrix of the output of a single deformable attention head in the first layer of the deformable multi-head attention mechanism. The second layer represents the self-attention weight matrix obtained by concatenating the outputs of multiple deformable attention heads in the feature dimension.

[0117] The second self-attention weight matrix and the query-guided feature sequence are weighted and normalized in the third residual layer to obtain the second self-attention feature sequence. The second self-attention feature sequence is linearly mapped to obtain the third query matrix, and the first self-attention feature sequence is linearly mapped to obtain the third key matrix and the third value matrix. The third key matrix and the third query matrix are added to the position encoding matrix and the learnable query vector, respectively, and then input together with the third value matrix into the second deformable multi-head attention mechanism layer to obtain the third self-attention weight matrix. In this embodiment, the second self-attention feature sequence is linearly mapped to obtain the third query matrix, and the first self-attention feature sequence is linearly mapped to obtain the third key matrix and the third value matrix for cross-attention processing to obtain the third self-attention feature sequence. The learnable query vector can read the multi-scale semantic information aggregated by the encoder.

[0118] The expression for a single deformable attention head in Layer 2 of the deformable multi-head attention mechanism is:

[0119] ;

[0120] in, This represents the normalized exponential function; , and These represent query matrix three, key matrix three, and value matrix three, respectively. Represents a static, learnable query vector; Represents the position encoding matrix; This indicates the spatial position of the reference point corresponding to the key spatial position, given by the dynamic anchor frame module; The third prediction of the linear mapping layer represents the first... The offset of each key point in dynamic sampling is two; This indicates that the total number of key points dynamically sampled near the spatial location of the reference point is two; This indicates a feature dimension scaling factor of 3, used to prevent gradient vanishing due to excessively large dot product values. The self-attention weight matrix represents the output of a single deformable attention head in the second layer of the deformable multi-head attention mechanism. The outputs of multiple deformable attention heads in the second layer of the deformable multi-head attention mechanism are concatenated along the feature dimension to obtain the third self-attention weight matrix.

[0121] The third self-attention feature sequence is obtained by weighting and normalizing the third self-attention weight matrix and the second self-attention feature sequence in the fourth residual layer. The third self-attention feature sequence is then weighted and normalized with itself in the fifth residual layer after passing through the seventh fully connected layer to obtain the inspection and identification query vector.

[0122] The inspection and identification query vector output from the top layer of the decoder is fed into the prediction head, which includes a fully connected layer 8 and a fully connected layer 9. The inspection and identification query vector is input into the fully connected layer 8 and the fully connected layer 9 in parallel. The fully connected layer 8 outputs category information, and the fully connected layer 9 outputs position information.

[0123] In this process, both fully connected layer 8 and fully connected layer 9 perform two linear mappings and one nonlinear transformation on the input inspection and identification query vector. The expressions for fully connected layer 8 and fully connected layer 9 are as follows:

[0124] ;

[0125] in, This represents a feedforward neural network. This represents the inspection and identification query vector input to the prediction head. and Let represent the parameter matrices of linear layer one and linear layer two, respectively. and These represent the bias parameters for linear layer one and linear layer two, respectively. This represents the ReLU normalization function.

[0126] In this embodiment, the constructed small target detection model still needs to be trained. The training method includes: real-time acquisition of on-site inspection video stream from the camera equipment mounted on the UAV, extraction of each frame of the image, and then sequential scaling of the extracted images to unify the input size and normalize the pixel values ​​to obtain the inspection image, and constructing the original dataset.

[0127] The original dataset is labeled to obtain a dataset containing labeled information, and then divided into training set, validation set and test set in a 7:2:1 ratio.

[0128] After initializing the model, the training set is used as input to train the small object detection model, and the model is validated using the validation set. During training, the loss function is calculated. Its expression is:

[0129] ;

[0130] in, , and represent the weight coefficients of the classification loss function, the bounding box regression loss function, and the generalized IoU loss function, respectively. It is a classification loss function. It is the bounding box regression loss function. It is the generalized IoU loss function; This represents the total number of prediction layers involved in calculating the loss. This indicates the traversal index of the prediction layer.

[0131] The test set is then used as input to test the small object detection model and obtain the test results. In this embodiment, precision, recall, and mean average precision (mAP50, IoU=0.5) are used as evaluation metrics for the test results. If the evaluation metrics reach the preset values, training is complete; otherwise, training is repeated until the evaluation metrics reach the preset values, at which point the trained small object detection model is obtained.

[0132] To verify the effectiveness of the small target detection model proposed in this embodiment, a comparative experiment was conducted with the original DETR (DetectionTransformer) model on the same dataset. Compared to the original DETR model, the average accuracy of the small target detection model for detecting cigarette butts increased from 52.3% to 63.7%, the average accuracy for detecting safety helmets increased from 62.2% to 73.4%, and the average accuracy for detecting reflective vests increased from 84.4% to 87.6%. This embodiment's small target detection model effectively improves the detection accuracy of small targets (cigarette butts) while ensuring accurate identification of salient targets (safety helmets and reflective vests).

[0133] Another specific embodiment provides a drone-based small target identification and inspection system, which includes:

[0134] The acquisition module is used to acquire inspection images of the construction site from the perspective of a drone.

[0135] The detection module is used to input the inspection image into a pre-trained small target detection model to obtain the inspection recognition result. The small target detection model is an improvement on the original DETR model. The improvements include: replacing the CNN backbone network of the original DETR model with a main backbone network and a feature fusion network connected in sequence; the main backbone network is built based on a fusion-type residual convolutional model, and the feature fusion network is built based on an adaptive bidirectional feature pyramid module; a query selection module is set at the connection position between the encoder and decoder in the original DETR model. The fused feature map output by the feature fusion network is combined with the position encoding matrix and input into the encoder to obtain the first self-attention feature sequence. The first self-attention feature sequence is input into the query selection module to filter out the feature vectors of key spatial locations and map them back to the inspection. The image is used to obtain a spatial location prior feature sequence in a continuous spatial coordinate system. In the original DETR model, a dynamic anchor box module is set at the connection position between the decoder and the encoder. The spatial location prior feature sequence is input into the dynamic anchor box module to generate a query-guided feature sequence in which the spatial location of the reference point is given by the dynamic anchor box. The spatial location of the reference point corresponds to the key spatial location. The query-guided feature sequence is combined with a learnable query vector and input into the decoder to obtain a second self-attention feature sequence. The second self-attention feature sequence and the first self-attention feature sequence are combined with the learnable query vector and the position encoding matrix, respectively, and then cross-attention processing is performed to obtain a third self-attention feature sequence. The third self-attention feature sequence is processed by the top layer of the decoder to generate an inspection and recognition query vector for input into the prediction head.

[0136] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for identifying and inspecting small targets using a drone, characterized in that, include: Acquire inspection images of the construction site from the perspective of a drone; The inspection image is input into a pre-trained small target detection model to obtain the inspection recognition result; The small target detection model is an improvement on the original DETR model. The improvements include: The original DETR model's CNN backbone network is replaced with a main network and a feature fusion network connected in sequence; the main network is built based on a fusion-type residual convolutional model, and the feature fusion network is built based on an adaptive bidirectional feature pyramid module. In the original DETR model, a query selection module is set at the connection position between the top layer of the encoder and the decoder. The fused feature map output by the feature fusion network is combined with the position encoding matrix and input into the encoder to obtain the first self-attention feature sequence. The first self-attention feature sequence is input into the query selection module to filter out the feature vectors of key spatial locations and map them back to the continuous spatial coordinate system corresponding to the inspection image to obtain the spatial location prior feature sequence. In the original DETR model, a dynamic anchor box module is set at the connection position between the decoder and the encoder at the bottom layer. The spatial location prior feature sequence is input into the dynamic anchor box module to generate a query guidance feature sequence in which the spatial location of the reference point is given by the dynamic anchor box. The spatial location of the reference point corresponds to the key spatial location. The query guidance feature sequence is combined with the learnable query vector and input into the decoder to obtain the second self-attention feature sequence. The second self-attention feature sequence and the first self-attention feature sequence are combined with the learnable query vector and the position encoding matrix respectively and then cross-attention processing is performed to obtain the third self-attention feature sequence. The third self-attention feature sequence is processed by the top layer of the decoder to generate the inspection and recognition query vector used to input the prediction head. The backbone network comprises a convolutional layer 1, a fused residual convolutional module 1, a fused residual convolutional module 2, a fused residual convolutional module 3, and a residual convolutional module connected in series. The three fusion residual convolution modules I, II, and III all include a fusion residual convolution module structure. The feature map of the input fusion residual convolutional module structure is passed sequentially through a 1×1 convolutional layer, a 3×3 convolutional layer, an improved CBAM attention mechanism layer, a 1×1 convolutional layer, and a random deactivation layer, and then the residual is added to itself to obtain the output; The improved CBAM attention mechanism layer includes a cascaded CAM-G Gaussian channel attention module and a SAM-G Gaussian spatial attention module; The feature fusion network includes a 1×1 convolutional layer three, a 1×1 convolutional layer four, a 1×1 convolutional layer five, and an adaptive bidirectional feature pyramid module; The outputs of the fusion residual convolution module II, fusion residual convolution module III, and residual convolution module are used as multi-scale feature maps. After channel unification through 1×1 convolution layer III, 1×1 convolution layer IV, and 1×1 convolution layer V, they are input in parallel into the adaptive bidirectional feature pyramid module. The adaptive bidirectional feature pyramid module includes a global pooling layer, a fully connected layer five, a normalization layer, a BiFPN module, and a 3×3 convolutional layer two, connected in sequence.

2. The UAV small target identification and inspection method according to claim 1, characterized in that, The encoder includes multiple encoder layers, each encoder layer including: a multi-head self-attention mechanism layer, residual layer one, fully connected layer six, residual layer two, and a query selection module; The fused feature map output by the feature fusion network is unified in terms of the number of channels to obtain the input feature sequence of the encoder layer. The input feature sequence is linearly mapped to obtain query matrix 1, key matrix 1, and value matrix 1. Query matrix 1 and key matrix 1 are added to the position encoding matrix respectively, and then fed together with value matrix 1 into the multi-head self-attention mechanism layer to obtain self-attention weight matrix 1. Self-attention weight matrix 1 and the input feature sequence of the encoder layer are weighted and normalized in residual layer 1 to obtain the self-attention feature extraction sequence. The self-attention feature extraction sequence is passed through a fully connected layer six, and then weighted and normalized with itself in a residual layer two to obtain the first self-attention feature sequence; The first self-attention feature sequence is processed by the query selection module to obtain the spatial location prior feature sequence.

3. The UAV small target identification and inspection method according to claim 2, characterized in that, The expression for a single attention head in a multi-head self-attention mechanism layer is: ； in, This represents the feature dimension scaling factor as one. Represents the position encoding matrix; , and These represent query matrix one, key matrix one, and value matrix one, respectively. This represents the normalized exponential function; This represents the self-attention weight matrix for a single attention head. The self-attention weight matrix is ​​obtained by concatenating the outputs of all attention heads in the multi-head self-attention mechanism layer along the feature dimension.

4. The UAV small target identification and inspection method according to claim 1, characterized in that, The query selection module includes a linear projection network, a sorting and truncation unit, an anchor box mapping layer, and an activation function mapping layer. The first self-attention feature sequence of the input query selection module is first input into the linear projection network. The linear projection network performs binary classification on the feature vectors corresponding to each spatial location in the first self-attention feature sequence to evaluate the target confidence and give the probability score that the spatial location belongs to the foreground target. After the target confidence assessment, the first self-attention feature sequence is input into the sorting and truncation unit. The sorting and truncation unit selects the feature vectors corresponding to the spatial positions with the highest probability scores as the feature vectors of the key spatial positions. The feature vectors of key spatial locations are downsampled and their size restored by using anchor box mapping layer and activation function mapping layer to obtain the prior feature sequence of spatial location.

5. The UAV small target identification and inspection method according to claim 4, characterized in that, The expression for a linear projection network is: ； in, This indicates that the first self-attention feature sequence of the input query selection module corresponds to the first... Feature vectors of spatial locations and Let represent the weight matrix and bias parameters, respectively, for feature learning in the linear projection network. Represents a sigmoid function. The first line represents the output of the linear projection network. The probability score of a spatial location belonging to a foreground target.

6. The UAV small target identification and inspection method according to claim 1, characterized in that, The decoder includes multiple decoder layers, each of which includes a dynamic anchor frame module, a deformable multi-head attention mechanism layer 1, a residual layer 3, a deformable multi-head attention mechanism layer 2, a residual layer 4, a fully connected layer 7, and a residual layer 5. The prior spatial location feature sequence is input into the dynamic anchor box module to generate a query-guided feature sequence in which the spatial location of the reference point is given by the dynamic anchor box. The query-guided feature sequence is linearly mapped to obtain query matrix two, key matrix two, and value matrix two. Query matrix two and key matrix two are added to the learnable query vector, and then together with value matrix two are input into deformable multi-head attention mechanism layer one to obtain self-attention weight matrix two. Self-attention weight matrix two and query-guided feature sequence are weighted and normalized in residual layer three to obtain the second self-attention feature sequence. The second self-attention feature sequence is linearly mapped to obtain query matrix three. The first self-attention feature sequence is linearly mapped to obtain key matrix three and value matrix three. Key matrix three and query matrix three are added to the position encoding matrix and the learnable query vector, respectively, and then input together with value matrix three into deformable multi-head attention mechanism layer two to obtain self-attention weight matrix three. Self-attention weight matrix three and the second self-attention feature sequence are weighted and normalized in residual layer four to obtain the third self-attention feature sequence. The third self-attention feature sequence is processed by the fully connected layer seven and then weighted and normalized with itself in the residual layer five to obtain the inspection and identification query vector.

7. The UAV small target identification and inspection method according to claim 6, characterized in that, The expression for a single deformable attention head in Layer 1 of the deformable multi-head attention mechanism is: ； in, This represents the normalized exponential function; , and These represent query matrix two, key matrix two, and value matrix two, respectively. Represents a static, learnable query vector; This indicates that the spatial position of the reference point is given by the dynamic anchor frame module; Indicates the first The offset of each key point in dynamic sampling; This represents the total number of keypoints dynamically sampled near the spatial location of the reference point. This indicates a feature dimension scaling factor of two; The first layer represents the self-attention weight matrix of the output of a single deformable attention head in the deformable multi-head attention mechanism layer 1. The outputs of multiple deformable attention heads in the first layer of the deformable multi-head attention mechanism layer 1 are concatenated along the feature dimension to obtain the second self-attention weight matrix. The expression for a single deformable attention head in Layer 2 of the deformable multi-head attention mechanism is: ； in, This represents the normalized exponential function; , and These represent query matrix three, key matrix three, and value matrix three, respectively. Represents a static, learnable query vector; Represents the position encoding matrix; Indicates the spatial position of the reference point given by the dynamic anchor frame module; Indicates the first The offset of each key point in dynamic sampling is two; This indicates that the total number of key points dynamically sampled near the spatial location of the reference point is two; This indicates a feature dimension scaling factor of three; The self-attention weight matrix represents the output of a single deformable attention head in the second layer of the deformable multi-head attention mechanism. The outputs of multiple deformable attention heads in the second layer of the deformable multi-head attention mechanism are concatenated along the feature dimension to obtain the third self-attention weight matrix.

8. The UAV small target identification and inspection method according to claim 1, characterized in that, The prediction head includes a fully connected layer eight and a fully connected layer nine, and the inspection and identification results include category information and location information. The inspection and identification query vector is input in parallel to the fully connected layer 8 and the fully connected layer 9; the fully connected layer 8 outputs category information, and the fully connected layer 9 outputs location information.

9. A UAV micro-target identification and inspection system, characterized in that, For performing the UAV small target identification and inspection method according to any one of claims 1-8, the UAV small target identification and inspection system includes: The acquisition module is used to acquire inspection images of the construction site from the perspective of a drone. The detection module is used to input the inspection image into a pre-trained small target detection model to obtain the inspection recognition result. The small target detection model is an improvement on the original DETR model. The improvements include: replacing the CNN backbone network of the original DETR model with a main backbone network and a feature fusion network connected in sequence; the main backbone network is built based on a fusion-type residual convolutional model, and the feature fusion network is built based on an adaptive bidirectional feature pyramid module; a query selection module is set at the connection position between the encoder and decoder in the original DETR model. The fused feature map output by the feature fusion network is combined with the position encoding matrix and input into the encoder to obtain the first self-attention feature sequence. The first self-attention feature sequence is input into the query selection module to filter out the feature vectors of key spatial locations and map them back to the inspection. The image is used to obtain a spatial location prior feature sequence in a continuous spatial coordinate system. In the original DETR model, a dynamic anchor box module is set at the connection position between the decoder and the encoder. The spatial location prior feature sequence is input into the dynamic anchor box module to generate a query-guided feature sequence in which the spatial location of the reference point is given by the dynamic anchor box. The spatial location of the reference point corresponds to the key spatial location. The query-guided feature sequence is combined with a learnable query vector and input into the decoder to obtain a second self-attention feature sequence. The second self-attention feature sequence and the first self-attention feature sequence are combined with the learnable query vector and the position encoding matrix, respectively, and then cross-attention processing is performed to obtain a third self-attention feature sequence. The third self-attention feature sequence is processed by the top layer of the decoder to generate an inspection and recognition query vector for input into the prediction head.

Images