Small target detection method and system for UAV inspection based on improved YOLOv7 model

Abstract

The present invention discloses a method and system for detecting small targets in unmanned aerial vehicle inspection based on an improved YOLOv7 model, comprising the following steps: collecting aerial images of unmanned aerial vehicles, selecting small sample images therein and marking them as small target images; performing normalization preprocessing on small target images to enhance the model's ability to identify small target images; constructing a data set for unmanned aerial vehicle inspection of small targets based on the small target images after normalization preprocessing; inputting the data set for unmanned aerial vehicle inspection of small targets into a preset improved YOLOv7 neural network detection model for feature extraction and training; optimizing the preset YOLOv7 neural network detection model; carrying the optimized and trained detection model on the unmanned aerial vehicle, performing image acquisition and detection according to the inspection task and judging the accuracy of the model, and adjusting and optimizing according to the actual situation. This scheme effectively solves the probability of missed detection and false detection in small target image detection of traditional methods, and improves the lightweight and accuracy of the unmanned aerial vehicle inspection model.

Description

Technical Field

[0001] The present invention relates to the field of intelligent detection, and in particular to a method and system for detecting small targets during unmanned aerial vehicle inspection based on an improved YOLOv7 model. Background Art

[0002] In recent years, drones have been widely used in the operation and maintenance of modern industries and infrastructure as an efficient and safe inspection method to collect image data in areas that are difficult to reach or dangerous for human labor. However, during drone power inspections, the images collected by drones often have small image size and unclear details due to the low-altitude field of view of drones. These subtle features are easily overlooked or misjudged. Therefore, accurately detecting and identifying small target objects has always been a challenging task.

[0003] Due to factors such as the high-speed movement of drones and blurred images, the detection algorithm needs to be real-time and robust to adapt to complex inspection environments. Traditional target detection methods have problems such as low recognition rate and high computational complexity when dealing with small targets. With the rapid development of computer vision and artificial intelligence, target detection methods based on deep neural networks have made great breakthroughs, providing new ideas and technical means for small target detection. By enhancing the perception ability of small target detection models, the quality of drone inspection applications in complex environments can be improved, thereby ensuring the safety and maintenance response capabilities of the power system. Summary of the invention

[0004] The purpose of the invention is to provide a method and system for detecting small targets during drone inspection based on an improved YOLOv7 model to solve the above-mentioned problems existing in the prior art.

[0005] The technical solution is a small target detection method for drone inspection based on the improved YOLOv7 model, which includes the following steps:

[0006] Step S1, collecting aerial images from a drone, selecting small sample images from them and marking them as small target images;

[0007] Step S2: performing normalization preprocessing on the small target image to enhance the model's ability to identify the small target image;

[0008] Step S3: constructing a drone inspection small target image dataset based on the normalized preprocessed small target image;

[0009] Step S4, inputting the drone inspection small target image data set into the preset improved YOLOv7 neural network detection model for feature extraction and training;

[0010] Step S5: optimizing the preset improved YOLOv7 neural network detection model, including:

[0011] Step S51: Combined with the multi-scale feature extraction method of small target images, a dynamic sparse attention mechanism is introduced to flexibly allocate weight ratios. At the same time, the detection module specifically adds a small target image detection head to adjust the model's perception field of view and enhance the detection capability of the context information of small target images;

[0012] Step S52: redesign the IOU loss function based on the characteristics of the small target image positioning deviation to improve the accuracy of small target image detection;

[0013] Step S6: The UAV is equipped with an optimized and trained detection model, collects and detects images according to the inspection task, determines the accuracy of the model, and makes adjustments and optimizations according to the actual situation.

[0014] According to another aspect of the present application, the step S1 is further:

[0015] Step S11, selecting sample images containing small target images from the collected aerial image data, including at least parts of buildings and details of equipment;

[0016] Step S12, small target image labeling: labeling the selected small sample image, the labeling process includes the following steps:

[0017] Target box annotation: Use a rectangular box to mark the position of the small target image in the aerial image, and use coordinates to represent the position of the box to obtain the small target box;

[0018] Category labeling: assign corresponding category labels to each small object box so that the model can recognize different types of small object images;

[0019] Occlusion annotation: Mark whether the small target image is blocked by other objects or occluders to improve the model's adaptability to complex scenes.

[0020] According to another aspect of the present application, step S2 is further:

[0021] Step S21, size unification: the selected small target images are uniformly adjusted to the same size by scaling or cropping to ensure that the images input to the model have a consistent size;

[0022] Step S22, color space conversion: converting the color space of the small target image, converting the RGB color space into a grayscale image, to reduce the complexity of the data;

[0023] Step S23, brightness and contrast adjustment: adjust the brightness and contrast of the small target image to make the brightness and contrast of the image more balanced, so as to enhance the model's recognition of the small target;

[0024] Step S24, mean normalization: performing mean normalization processing on the small target image, that is, subtracting the mean of the image and dividing it by the standard deviation, so as to eliminate the deviation between images, which is beneficial to the training and convergence of the model;

[0025] Step S25, data enhancement: Data enhancement operations can be performed during the preprocessing process, including random rotation, flipping, and scaling, to increase data diversity and improve the generalization ability of the model;

[0026] Step S26, border filling: fill the borders of the small target image with black or white borders to meet the model's requirements for inputs of different sizes;

[0027] Step S27, normalization processing: normalize the preprocessed small target image and scale the pixel values ​​to a specific range so that the model can better process the image data.

[0028] According to another aspect of the present application, the feature extraction in step S4 is based on the Backbone network of the improved YOLO-V7 to perform feature extraction of different scales, and the snake convolution is used to replace the ordinary convolution in ELAN, further comprising:

[0029] Step S41, adding a feature layer with a downsampling multiple of 4 to separately obtain feature information of the small target image: adding a feature extraction layer with a downsampling multiple of 4 to the YOLOv7 backbone network, the input small target image is subjected to multi-scale feature extraction by the backbone network, and feature maps of 160×160, 80×80, 40×40, and 20×20 are obtained in sequence, wherein the feature map of 160×160 is extracted from the feature layer of the small target image;

[0030] S42. Use snake convolution to replace ordinary convolution in ELAN: Use dilated convolution to adaptively adjust the size of the perception field according to the different positions of the feature map, thereby reducing the number of model parameters and calculations and adapting to the diversity of inspection targets and the real-time requirements of the detection model.

[0031] According to another aspect of the present application, in step S51, a dynamic sparse attention mechanism is introduced to perform feature enhancement in view of the particularity that the small target image area accounts for a small proportion, further comprising:

[0032] S511. According to the characteristics of small target image feature information being easily lost and small variance between small targets, a dynamic sparse attention mechanism BRA is used to adaptively allocate the weight value of feature content when extracting feature maps in each layer of convolution and pooling operations, and focus on the target area in a key pair query mode to achieve effective allocation of computational workload and improve the model's perception ability and detection speed for small target images. The specific calculation formula is as follows:

[0033] P=Attention(Q,K t ,V t )+LCE(V);

[0034] Among them, P is the enhanced feature information, is the BRA attention mechanism, Q, K, V are the key-value pairs obtained after feature map mapping, query, key, value, t is the tensor of key and value after aggregation, and LCE(~) function is the parameterized formula of deep convolution;

[0035] S512, Neck network performs multi-scale feature fusion, according to the characteristics of small target images, improves the original equal weight fusion method, combines the shallow features with more small target image location and context information, sets a larger weight for fusion ratio, and improves the utilization rate of feature information of different scales. The specific method is as follows:

[0036] x=α1P1+α2P2+α3P3+α4P4;

[0037] α1≥α2≥α3≥α4;

[0038] Among them, x is the feature image for feature mechanism fusion, α1, α2, α3, and α4 are the weights of features of different scales, and the feature layers are from shallow to deep, and P1, P2, P3, and P4 are feature images extracted at different scales;

[0039] S513. The detection branch adds a small target image detection head to process the position, shape and context information of shallow features, thereby enhancing the precise positioning of small target images.

[0040] According to another aspect of the present application, the step S52 further improves the loss function for the small target position feature as follows:

[0041] Step S521, loss function L of the detection head loss Including object confidence loss L conf , object classification loss L cls , prediction box position loss L bbox Three parts, as follows:

[0042] L loss =L conf +L cls +L bbox

[0043] Among them, the object confidence loss and object classification loss functions both use the original cross entropy function of YOLOV7;

[0044] Step S522, the prediction box position loss function GIOU will degenerate into the IOU function when the prediction boxes overlap, and the IOU function is sensitive to the size of small objects and cannot accurately predict the regression results. The Focal-WIOU loss function is used to enhance the convergence speed of the regression gradient and the weight ratio of high and low quality prediction boxes, as shown in the following formula:

[0045] L bbox =L WIoU ×IoU γ

[0046] Among them, γ is the adjusted hyperparameter.

[0047] According to another aspect of the present application, the step 6 is further:

[0048] S61. Record the inspection results, including the number of small targets detected, location information, and confidence, and use the comprehensive indicators of accuracy, recall, F1 score, and average precision to evaluate the model performance;

[0049] Step S62: Determine the expressiveness of the detection model under different detection targets and inspection requirements based on the inspection results, and gradually improve the accuracy and lightness of the small target image detection model through continuous feedback and iteration to obtain more reliable inspection results.

[0050] According to another aspect of the present application, a UAV inspection small target detection system based on an improved YOLOv7 model is also provided, comprising:

[0051] at least one processor; and

[0052] a memory communicatively connected to at least one of the processors; wherein,

[0053] The memory stores instructions that can be executed by the processor, and the instructions are used to be executed by the processor to implement the drone inspection small target detection method based on the improved YOLOv7 model described in any of the above technical solutions.

[0054] Beneficial effects: 1. In the backbone network of the detection model, snake convolution is used to replace the ordinary convolution in the ELAN module, simplifying the network structure and the number of parameters, thereby enhancing the lightweight of the network; at the same time, in order to solve the problem of insufficient feature information of small target images, a small target image detection layer is added, and the generalization ability of the model is enhanced by multi-scale feature fusion and capturing the detail information of small target images;

[0055] 2. To address the problem that deep features are prone to losing semantic information, a dynamic sparse attention mechanism is added to increase the attention paid to the feature areas of small target images; and a small target image detection head is added to the shallow feature network to improve the model's perception of small target images;

[0056] 3. By optimizing the loss function through Focal-WIOU, the model detection accuracy is ensured while the regression speed is improved, ensuring the detection efficiency and accuracy of small target images. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 It is a logic flow chart of the present invention.

[0058] Figure 2 It is a network structure diagram of the present invention. DETAILED DESCRIPTION

[0059] According to one aspect of the present application, a laboratory information management method based on homomorphic encryption and Internet of Things technology comprises the following steps:

[0060] Step S1, collecting aerial images from a drone, selecting small sample images from them and marking them as small target images;

[0061] Step S2: performing normalization preprocessing on the small target image to enhance the model's ability to identify the small target image;

[0062] Step S3: constructing a drone inspection small target image dataset based on the normalized preprocessed small target image;

[0063] Step S4, inputting the drone inspection small target image data set into the preset improved YOLOv7 neural network detection model for feature extraction and training;

[0064] Step S5: optimizing the preset improved YOLOv7 neural network detection model, including:

[0065] Step S51: Combined with the multi-scale feature extraction method of small target images, a dynamic sparse attention mechanism is introduced to flexibly allocate weight ratios. At the same time, the detection module specifically adds a small target image detection head to adjust the model's perception field of view and enhance the detection capability of the context information of small target images;

[0066] Step S52: redesign the IOU loss function based on the characteristics of the small target image positioning deviation to improve the accuracy of small target image detection;

[0067] Step S6: The UAV is equipped with an optimized and trained detection model, collects and detects images according to the inspection task, determines the accuracy of the model, and makes adjustments and optimizations according to the actual situation.

[0068] In this embodiment, by selecting small sample images and marking them as small target images, and performing normalization preprocessing on the small target images, the quality and consistency of the data can be improved, which is beneficial to model training and accuracy; by combining the improved YOLOv7 neural network model for feature extraction and training, and optimizing the multi-scale feature extraction method and attention mechanism of the model, the detection performance and accuracy of the model can be improved; by adding a small target image detection head to adjust the model's perception field of view, the detection ability of the contextual information of the small target image is enhanced, which helps to improve the model's adaptability to complex scenes; the IOU loss function is redesigned, and the characteristics of the small target image positioning deviation are optimized, which can improve the accuracy and stability of small target image detection; the optimized and trained detection model can be carried on a drone for real-time image acquisition and detection, thereby improving the efficiency and accuracy of inspection tasks; the model is adjusted and optimized according to actual conditions to adapt to different scenarios and task requirements, and has certain flexibility and adjustability.

[0069] In summary, through meticulous data processing, model optimization and real-time application, this solution can effectively improve the accuracy, efficiency and adaptability of small target image detection in UAV inspections, providing strong support for the smooth progress of UAV inspection tasks.

[0070] According to another aspect of the present application, the step S1 is further:

[0071] Step S11, selecting sample images containing small target images from the collected aerial image data, including at least parts of buildings and details of equipment;

[0072] Step S12, small target image labeling: labeling the selected small sample image, the labeling process includes the following steps:

[0073] Target box annotation: Use a rectangular box to mark the position of the small target image in the aerial image, and use coordinates to represent the position of the box to obtain the small target box;

[0074] Category labeling: assign corresponding category labels to each small object box so that the model can recognize different types of small object images;

[0075] Occlusion annotation: Mark whether the small target image is blocked by other objects or occluders to improve the model's adaptability to complex scenes.

[0076] Obtain sample images of small targets inspected by drones. The sample data comes from the inspection images taken by drones. Select small sample images and mark the small targets in the samples. By selecting sample images containing small target images and marking them, the quality and accuracy of the data set can be improved, providing a more reliable data basis for model training; marking the position, category and occlusion of small target images can help the model accurately identify different types of small target images and improve the recognition ability and accuracy of the model; through category labeling and occlusion labeling, the model can learn the characteristics of small target images in different categories and complex scenes, and improve the generalization ability and adaptability of the model; labeling the position and occlusion of small target images can help the model accurately identify targets and avoid misjudgment, and improve the accuracy and stability of the model; by marking small target images, the labeling quality of the data set can be improved, the training process of the model can be accelerated, the convergence time of the model can be shortened, and the training efficiency can be improved; occlusion labeling can help the model identify occluded small target images, improve the adaptability of the model in complex scenes, and enhance the robustness of the model.

[0077] In summary, this solution can improve data quality, enhance model recognition capability, improve generalization ability, reduce misjudgment rate, accelerate the training process, and adapt to complex scenarios by selecting sample images and marking the position, category, and occlusion of small target images, thus providing effective support and guarantee for UAV inspection tasks.

[0078] According to another aspect of the present application, step S2 is further:

[0079] Step S21, size unification: the selected small target images are uniformly adjusted to the same size by scaling or cropping to ensure that the images input to the model have a consistent size;

[0080] Step S22, color space conversion: converting the color space of the small target image, converting the RGB color space into a grayscale image, to reduce the complexity of the data;

[0081] Step S23, brightness and contrast adjustment: adjust the brightness and contrast of the small target image to make the brightness and contrast of the image more balanced, so as to enhance the model's recognition of the small target;

[0082] Step S24, mean normalization: performing mean normalization processing on the small target image, that is, subtracting the mean of the image and dividing it by the standard deviation, so as to eliminate the deviation between images, which is beneficial to the training and convergence of the model;

[0083] Step S25, data enhancement: Data enhancement operations can be performed during the preprocessing process, including random rotation, flipping, and scaling, to increase data diversity and improve the generalization ability of the model;

[0084] Step S26, border filling: fill the borders of the small target image with black or white borders to meet the model's requirements for inputs of different sizes;

[0085] Step S27, normalization processing: normalize the preprocessed small target image and scale the pixel values ​​to a specific range so that the model can better process the image data.

[0086] In this embodiment, refinement steps such as size unification, color space conversion, brightness and contrast adjustment, mean normalization, data enhancement, and boundary filling are introduced in the data preprocessing step to perform more refined processing on small target images to improve the training effect and accuracy of the model; a normalization processing step is added to the data preprocessing to scale the pixel values ​​to a specific range, which is conducive to the model to better process image data and improve the generalization ability and training effect of the model. Through refined data preprocessing steps, including size unification, color space conversion, brightness and contrast adjustment, etc., the model's ability to identify small target images can be enhanced, and the model's robustness and accuracy can be improved; the introduction of data enhancement steps can increase the diversity of data, including random rotation, flipping, scaling and other operations, which is conducive to the model learning more diverse features and improving the model's generalization ability and adaptability; the addition of steps such as normalization and boundary filling can eliminate the deviation between images, adapt to the requirements of inputs of different sizes, and improve the training effect and convergence speed of the model; through normalization and refined data preprocessing steps, the performance of the model can be optimized, the model's detection accuracy and stability for small target images can be improved, and the model can be more suitable for complex scenarios and task requirements.

[0087] In summary, compared with traditional solutions, this solution is more sophisticated in the refinement and optimization of data preprocessing, which can improve the training effect, robustness and generalization ability of the model, so as to better adapt to the needs of actual application scenarios.

[0088] According to another aspect of the present application, the feature extraction in step S4 is based on the Backbone network of the improved YOLO-V7 to perform feature extraction of different scales, and the snake convolution is used to replace the ordinary convolution in ELAN, further comprising:

[0089] Step S41, adding a feature layer with a downsampling multiple of 4 to separately obtain feature information of the small target image: adding a feature extraction layer with a downsampling multiple of 4 to the YOLOv7 backbone network, the input small target image is subjected to multi-scale feature extraction by the backbone network, and feature maps of 160×160, 80×80, 40×40, and 20×20 are obtained in sequence, wherein the feature map of 160×160 is extracted from the feature layer of the small target image;

[0090] S42. Use snake convolution to replace ordinary convolution in ELAN: Use dilated convolution to adaptively adjust the size of the perception field according to the different positions of the feature map, thereby reducing the number of model parameters and calculations and adapting to the diversity of inspection targets and the real-time requirements of the detection model.

[0091] In another embodiment of the present application, it also includes:

[0092] Introduce attention mechanisms, such as self-attention mechanisms or attention mechanism networks, to dynamically adjust feature weights at different positions, improve the model’s attention to small target images, and enhance detection capabilities;

[0093] Adding residual connections to connect feature maps of different scales is helpful for information transmission and feature fusion, improving the perception ability and accuracy of the model;

[0094] Multi-scale fusion: In the feature extraction stage, a multi-scale fusion mechanism is introduced to fuse feature maps of different scales, comprehensively utilize multi-scale information, and improve the model's detection effect on small target images;

[0095] Add side branches to the backbone network to extract more detailed feature information, assist the model in detecting and recognizing small target images, and improve the performance and accuracy of the model.

[0096] The use of methods such as snake convolution and attention mechanism can reduce the number of model parameters and calculations, improve the model's operating efficiency and speed, and adapt to real-time requirements; the introduction of residual connection, multi-scale fusion and other mechanisms can enhance the model's perception and feature extraction capabilities, and improve the model's recognition and detection accuracy for small target images; the use of methods such as hole convolution and multi-scale fusion can adapt to the detection needs of targets of different scales and diversity, and improve the generalization ability and adaptability of the model; by adding feature layers, optimizing convolution methods and introducing attention mechanisms, the model's detection accuracy and stability for small target images can be improved, and the model performance and effect can be improved.

[0097] In summary, by adopting different implementation methods and optimization strategies, the feature extraction ability and detection effect of the YOLO-V7 neural network can be further improved, making the model more adaptable to the needs of actual inspection tasks and improving the performance and accuracy of the model.

[0098] According to another aspect of the present application, in step S51, a dynamic sparse attention mechanism is introduced to perform feature enhancement in view of the particularity that the small target image area accounts for a small proportion, further comprising:

[0099] S511. According to the characteristics of small target image feature information being easily lost and small variance between small targets, a dynamic sparse attention mechanism BRA is used to adaptively allocate the weight value of feature content when extracting feature maps in each layer of convolution and pooling operations, and focus on the target area in a key pair query mode to achieve effective allocation of computational workload and improve the model's perception ability and detection speed for small target images. The specific calculation formula is as follows:

[0100] P=Attention(Q,K t ,V t )+LCE(V);

[0101] Among them, P is the enhanced feature information, is the BRA attention mechanism, Q, K, V are the key-value pairs obtained after feature map mapping, query, key, value, t is the tensor of key and value after aggregation, and LCE(~) function is the parameterized formula of deep convolution;

[0102] S512, Neck network performs multi-scale feature fusion, according to the characteristics of small target images, improves the original equal weight fusion method, combines the shallow features with more small target image location and context information, sets a larger weight for fusion ratio, and improves the utilization rate of feature information of different scales. The specific method is as follows:

[0103] x=α1P1+α2P2+α3P3+α4P4;

[0104] α1≥α2≥α3≥α4;

[0105] Among them, x is the feature image for feature mechanism fusion, α1, α2, α3, and α4 are the weights of features of different scales, and the feature layers are from shallow to deep, and P1, P2, P3, and P4 are feature images extracted at different scales;

[0106] S513. The detection branch adds a small target image detection head to process the position, shape and context information of shallow features, thereby enhancing the precise positioning of small target images.

[0107] In this embodiment, a dynamic sparse attention mechanism BRA is introduced to adaptively allocate weight values ​​of feature content, focus on target areas, improve the model's perception and detection speed of small target images, and effectively improve model performance; when multi-scale feature fusion is performed in the Neck network, the original equal weight method is improved according to the characteristic of small target images, so as to improve the utilization rate of feature information of different scales and enhance the model's recognition and positioning capabilities of small target images; by setting a larger weight for fusion ratio, combined with the situation where shallow features are rich in more small target image location and context information, the model's feature extraction and localization of small target images are enhanced. Utilization can improve detection precision and accuracy; adding a small target image detection head to process the position, shape and context information of shallow features can enhance the precise positioning capability of small target images and improve the detection accuracy and stability of the model; through the introduction of dynamic sparse attention mechanism, multi-scale feature fusion and small target image detection head, the model's perception ability, feature extraction ability and positioning accuracy can be improved, and the performance and effect of the model can be improved; this solution aims at the particularity that small target images account for a small area, and by optimizing feature extraction and fusion methods, it can better adapt to the characteristics and requirements of small target images and improve the model's processing effect on small target images.

[0108] In summary, this scheme can effectively improve the performance and accuracy of the model and enhance the model's ability to recognize, locate and detect small target images by introducing optimization strategies such as dynamic sparse attention mechanism, multi-scale feature fusion and small target image detection head.

[0109] According to another aspect of the present application, the step S52 further improves the loss function for the small target position feature as follows:

[0110] Step S521, loss function L of the detection head loss Including object confidence loss L conf , object classification loss L cls , prediction box position loss L bbox Three parts, as follows:

[0111] L loss =L conf +L cls +L bbox

[0112] Among them, the object confidence loss and object classification loss functions both use the original cross entropy function of YOLOV7;

[0113] Step S522, the prediction box position loss function GIOU will degenerate into the IOU function when the prediction boxes overlap, and the IOU function is sensitive to the size of small objects and cannot accurately predict the regression results. The Focal-WIOU loss function is used to enhance the convergence speed of the regression gradient and the weight ratio of high and low quality prediction boxes, as shown in the following formula:

[0114] L bbox =L WIoU ×IoU γ

[0115] Among them, γ is the adjusted hyperparameter.

[0116] In this embodiment, the prediction box position is improved by introducing the Focal-WIOU loss function, the convergence speed of the regression gradient and the weight ratio of high-quality and low-quality prediction boxes are enhanced, the model's ability to accurately predict small target position features is improved, the loss function design is optimized, and the model performance is improved; the Focal-WIOU loss function is used instead of the IOU function to reduce the sensitivity to small target size, improve the robustness of the model to changes in small target size, and make the model more stable and reliable; the Focal-WIOU loss function can enhance the convergence speed of the regression gradient, speed up the model training process, improve the training efficiency and convergence speed of the model, and save training time; by adjusting the hyperparameters, the weight ratio of high-quality and low-quality prediction boxes can be optimized, so that the model pays more attention to high-quality prediction boxes and improves the accuracy and stability of the model for small target positions; optimizing the prediction box position loss function can improve the model's prediction precision and accuracy for small target positions and enhance the model's detection capability and positioning accuracy.

[0117] In summary, by introducing the Focal-WIOU loss function to improve the prediction box position, this scheme can optimize the loss function design, reduce size sensitivity, accelerate convergence speed, optimize weight ratio, and improve prediction accuracy and model performance.

[0118] According to another aspect of the present application, the step 6 is further:

[0119] S61. Record the inspection results, including the number of small targets detected, location information, and confidence, and use the comprehensive indicators of accuracy, recall, F1 score, and average precision to evaluate the model performance;

[0120] Step S62: Determine the expressiveness of the detection model under different detection targets and inspection requirements based on the inspection results, and gradually improve the accuracy and lightness of the small target image detection model through continuous feedback and iteration to obtain more reliable inspection results.

[0121] In this embodiment, by recording the inspection results, including the number of small targets, location information and confidence, and using comprehensive indicators such as accuracy, recall, F1 score and average precision mean to evaluate the model performance, the detection effect and accuracy of the model can be comprehensively and objectively evaluated; the expressiveness of the detection model under different detection targets and inspection requirements is judged according to the inspection results, and through continuous feedback and iterative optimization, the accuracy and lightweight of the small target image detection model are gradually improved, and the model performance can be continuously improved to obtain more reliable inspection results; through continuous feedback and iterative optimization, model performance problems can be discovered and adjusted in time, the real-time and reliability of the model can be improved, and the accuracy and stability of the inspection results can be ensured; by evaluating the model performance through comprehensive indicators and adjusting the model expressiveness according to actual inspection requirements, the accuracy and precision of the small target image detection model can be gradually improved, and the inspection effect and result reliability can be improved; through continuous feedback and iterative optimization, the model performance can be continuously improved, and the expressiveness and lightweight of the small target image detection model can be continuously improved, so as to provide better services and results for inspection tasks.

[0122] According to another aspect of the present application, a UAV inspection small target detection system based on an improved YOLOv7 model is also provided, comprising:

[0123] at least one processor; and

[0124] a memory communicatively connected to at least one of the processors; wherein,

[0125] The memory stores instructions that can be executed by the processor, and the instructions are used to be executed by the processor to implement the drone inspection small target detection method based on the improved YOLOv7 model described in any of the above technical solutions.

[0126] It should be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present invention will not further describe various possible combinations.

Claims

1. A small target detection method for UAV inspection based on the improved YOLOv7 model, characterized in that: The steps include: Step S1, collecting aerial images from a drone, selecting small sample images from them and marking them as small target images; Step S2: performing normalization preprocessing on the small target image to enhance the model's ability to identify the small target image; Step S3: constructing a drone inspection small target image dataset based on the normalized preprocessed small target image; Step S4: input the drone inspection small target image data set into the preset improved YOLOv7 neural network detection model for feature extraction and training; Step S5: optimizing the preset improved YOLOv7 neural network detection model, including: Step S51: Combined with the multi-scale feature extraction method of small target images, a dynamic sparse attention mechanism is introduced to flexibly allocate weight ratios. At the same time, the detection module specifically adds a small target image detection head to adjust the model's perception field of view and enhance the detection capability of the context information of small target images; Step S52: redesign the IOU loss function based on the characteristics of the small target image positioning deviation to improve the accuracy of small target image detection; Step S6: The UAV is equipped with an optimized and trained detection model, collects and detects images according to the inspection task, determines the accuracy of the model, and makes adjustments and optimizations according to the actual situation.

2. The method for detecting small targets in unmanned aerial vehicle inspection based on the improved YOLOv7 model as claimed in claim 1, characterized in that: The step S1 is further as follows: Step S11, selecting sample images containing small target images from the collected aerial image data, including at least parts of buildings and details of equipment; Step S12, small target image labeling: labeling the selected small sample image, the labeling process includes the following steps: Target box annotation: Use a rectangular box to mark the position of the small target image in the aerial image, and use coordinates to represent the position of the box to obtain the small target box; Category labeling: assign corresponding category labels to each small object box so that the model can recognize different types of small object images; Occlusion annotation: Mark whether the small target image is blocked by other objects or occluders to improve the model's adaptability to complex scenes.

3. The method for detecting small targets in unmanned aerial vehicle inspection based on the improved YOLOv7 model as claimed in claim 1, characterized in that: The step S2 is further as follows: Step S21, size unification: the selected small target images are uniformly adjusted to the same size by scaling or cropping to ensure that the images input to the model have a consistent size; Step S22, color space conversion: converting the color space of the small target image, converting the RGB color space into a grayscale image, to reduce the complexity of the data; Step S23, brightness and contrast adjustment: adjust the brightness and contrast of the small target image to make the brightness and contrast of the image more balanced, so as to enhance the model's recognition of the small target; Step S24, mean normalization: performing mean normalization processing on the small target image, that is, subtracting the mean of the image and dividing it by the standard deviation, so as to eliminate the deviation between images, which is beneficial to the training and convergence of the model; Step S25, data enhancement: performing data enhancement operations during the preprocessing process, including random rotation, flipping, and scaling, to increase data diversity and improve the generalization ability of the model; Step S26, border filling: fill the borders of the small target image with black or white borders to meet the model's requirements for inputs of different sizes; Step S27, normalization processing: normalize the preprocessed small target image and scale the pixel values ​​to a specific range so that the model can better process the image data.

4. The method for detecting small targets in unmanned aerial vehicle inspection based on the improved YOLOv7 model as claimed in claim 1, characterized in that: The feature extraction in step S4 is based on the Backbone network of the improved YOLO-V7 to extract features of different scales, and the snake convolution is used to replace the ordinary convolution in ELAN, further as follows: Step S41, adding a feature layer with a downsampling multiple of 4 to separately obtain feature information of the small target image: adding a feature extraction layer with a downsampling multiple of 4 to the YOLOv7 backbone network, the input small target image is subjected to multi-scale feature extraction by the backbone network, and feature maps of 160×160, 80×80, 40×40, and 20×20 are obtained in sequence, wherein the feature map of 160×160 is extracted from the feature layer of the small target image; S42. Use snake convolution to replace ordinary convolution in ELAN: Use dilated convolution to adaptively adjust the size of the perception field according to the different positions of the feature map, thereby reducing the number of model parameters and calculations and adapting to the diversity of inspection targets and the real-time requirements of the detection model.

5. The method for detecting small targets in unmanned aerial vehicle inspection based on the improved YOLOv7 model as claimed in claim 1, characterized in that: In step S51, a dynamic sparse attention mechanism is introduced to enhance features in view of the fact that the small target image area accounts for a small proportion, and further comprises: S511. According to the characteristics of small target image feature information being easily lost and small variance between small targets, a dynamic sparse attention mechanism BRA is used to adaptively allocate the weight value of feature content when extracting feature maps in each layer of convolution and pooling operations, and focus on the target area in a key pair query mode to achieve effective allocation of computational workload and improve the model's perception ability and detection speed for small target images. The specific calculation formula is as follows: P=Attention(Q,K t ,V t )+LCE(V); Among them, P is the enhanced feature information, is the BRA attention mechanism, Q, K, V are the key-value pairs obtained after feature map mapping, query, key, value, t is the tensor of key and value after aggregation, and LCE(~) function is the parameterized formula of deep convolution; S512, Neck network performs multi-scale feature fusion, according to the characteristics of small target images, improves the original equal weight fusion method, combines the shallow features with more small target image location and context information, sets a larger weight for fusion ratio, and improves the utilization rate of feature information of different scales. The specific method is as follows: x=α1P1+α2P2+α3P3+α4P4; α1≥α2≥α3≥α4； Among them, x is the feature image for feature mechanism fusion, α1, α2, α3, and α4 are the weights of features of different scales, and the feature layers are from shallow to deep, and P1, P2, P3, and P4 are feature images extracted at different scales; S513. The detection branch adds a small target image detection head to process the position, shape and context information of shallow features, thereby enhancing the precise positioning of small target images.

6. The method for detecting small targets in unmanned aerial vehicle inspection based on the improved YOLOv7 model as claimed in claim 1, characterized in that: The step S52 further improves the loss function for the small target position feature as follows: Step S521, loss function L of the detection head loss Including object confidence loss L conf , object classification loss L cls , prediction box position loss L bbox Three parts, as follows: L loss =L conf +L cls +L bbox Among them, the object confidence loss and object classification loss functions both use the original cross entropy function of YOLOV7; Step S522, the prediction box position loss function GIOU will degenerate into the IOU function when the prediction boxes overlap, and the IOU function is sensitive to the size of small objects and cannot accurately predict the regression results. The Focal-WIOU loss function is used to enhance the convergence speed of the regression gradient and the weight ratio of high and low quality prediction boxes, as shown in the following formula: L bbox =L WIoU ×IoU γ Among them, γ is the adjusted hyperparameter.

7. The method for detecting small targets in unmanned aerial vehicle inspection based on the improved YOLOv7 model as claimed in claim 1, characterized in that: The step 6 is further as follows: S61. Record the inspection results, including the number of small targets detected, location information, and confidence level, and use the comprehensive indicators of accuracy, recall, F1 score, and average precision to evaluate the model performance; Step S62: Determine the expressiveness of the detection model under different detection targets and inspection requirements based on the inspection results, and gradually improve the accuracy and lightness of the small target image detection model through continuous feedback and iteration to obtain more reliable inspection results.

8. A small target detection system for drone inspection based on an improved YOLOv7 model, characterized in that: include: at least one processor; as well as a memory communicatively connected to at least one of the processors; wherein, The memory stores instructions that can be executed by the processor, and the instructions are used to be executed by the processor to implement the drone inspection small target detection method based on the improved YOLOv7 model as described in any one of claims 1 to 7.

Images