Unmanned aerial vehicle bolt detection model construction method based on image feature guide parameters

Abstract

The invention relates to an unmanned aerial vehicle bolt detection model construction method based on image feature guide parameters, and belongs to the field of computer model construction. The method comprises the steps of obtaining a bolt image sample through shooting of an unmanned aerial vehicle, and constructing a sample library under multiple scenes; on the basis of the YOLOv8 model, establishing a function to analyze the correlation between model parameters and bolt image features so as to establish an improved YOLOv8 model; and based on the sample library in multiple scenes, performing guided training on the improved YOLOv8 model through a weighted parameter adjustment loss function to obtain an unmanned aerial vehicle bolt detection model. According to the model construction method, an operator can visually understand an evaluation rule of the model, the interpretability of the model and the reliability of engineering application are enhanced, and the overall detection performance of the model is comprehensively enhanced.

Description

Technical Field

[0001] This invention belongs to the field of computer model building, specifically relating to a method for building a UAV bolt detection model based on image feature-guided parameters. Background Technology

[0002] With technological advancements, machine inspection technology has gradually become a research focus, significantly improving the efficiency and accuracy of inspections through automation. The widespread application of drone technology in inspection has provided a new solution for bolt inspection. The portability, operability, and flight capabilities of drones in complex environments make them particularly crucial for inspections under extreme weather conditions. Compared to traditional manual and machine inspections, drone inspection demonstrates significant advantages in terms of personnel safety, maintenance costs, and inspection efficiency. Therefore, research on drone target detection algorithms has substantial practical application value.

[0003] Traditional object detection algorithms are mainly categorized into two-stage and single-stage algorithms. Two-stage algorithms, to improve accuracy, perform classification and location regression after generating candidate regions. However, this approach is relatively slow and unsuitable for real-time applications. Single-stage algorithms, on the other hand, integrate the object detection task into a single neural network model, eliminating the need for a region proposal generation stage and directly outputting the object's category and location information. Therefore, they offer faster inference speed and real-time performance. YOLOv8, with its high accuracy, fast inference speed, and small parameter count, has become a focus of current research and provides a new solution for UAV object detection.

[0004] Adding an attention mechanism to the original model enhances its ability to extract feature information, allowing for better utilization of this information and reducing feature loss during sampling. This effectively improves the model's detection capabilities while maintaining a lightweight design. However, existing parameter optimization methods have significant shortcomings: they fail to quantify the relationship between model parameters and bolt image features, and the loss functions often employ fixed weights, failing to dynamically guide the model to focus on core bolt features. This not only leads to insufficient accuracy in bounding box localization and slow model training convergence but also, due to the lack of distinction between core and irrelevant parameters in global parameter adjustments, causes irrelevant parameters to interfere with the learning of effective features, further reducing detection efficiency in complex scenarios. These deficiencies make existing bolt detection models unsuitable for the safe operation and maintenance needs of large equipment such as cranes, necessitating the development of detection models adapted to UAV scenarios and optimizing the correlation between features and parameters to address these issues. Summary of the Invention

[0005] The purpose of this invention is to address the problems existing in the background technology and provide a method for constructing a UAV bolt detection model based on image feature-guided parameters.

[0006] To achieve the above objectives, the technical solution of the present invention is: a method for constructing a UAV bolt detection model based on image feature-guided parameters, comprising:

[0007] Bolt image samples were acquired by taking pictures with drones, and a sample library for multiple scenarios was built;

[0008] Based on the YOLOv8 model, the correlation between the parameters of the function analysis model and the features of the bolt image is constructed to build an improved YOLOv8 model;

[0009] Based on a sample library from multiple scenarios, the improved YOLOv8 model was trained by adjusting the loss function with weighted parameters to obtain a UAV bolt detection model.

[0010] Furthermore, the step of acquiring bolt image samples by taking pictures with a drone includes: using a drone to fly at high altitude to take pictures of the standard section of the large equipment, using the minimum safe distance supported by the drone's optical obstacle avoidance system as the preferred shooting distance, controlling the drone to the acquisition point so that the camera on the drone is facing the standard section of the large equipment, acquiring images of the connection between the two standard sections in each direction, and taking pictures of the standard section from three angles in each direction: top view, eye view, and bottom view, to obtain bolt image samples.

[0011] Furthermore, the step of constructing a sample library for multiple scenarios includes: processing bolt image samples using image generation methods including random rotation, background replacement, and exposure adjustment, and cross-expanding them using sample expansion methods to construct a bolt image sample library based on the original image and the expanded image.

[0012] Furthermore, the construction of the sample library also includes a labeling step: using Labelimg data labeling software, the bolt image samples in the sample library are selected one by one to define the range of the bolt images and assign corresponding working condition labels. After the labeling is completed, the label files corresponding to the bolt image samples are saved. The label files include the bolt working condition category number, the normalized coordinates of the label center, and the width and height information of the label area. Then, based on the bolt image samples in the bolt image sample library and the corresponding label files, a sample dataset is constructed and randomly divided into a training set and a test set at a ratio of 4:1.

[0013] Furthermore, the construction of the association between the function analysis model parameters and bolt image features based on the YOLOv8 model includes: incorporating an attention mechanism at the c2f module of the neck of the YOLOv8 model and constructing the association between the function analysis model parameters and image features; the construction of the association between the function analysis model parameters and image features includes:

[0014] Define the model parameter set F = {x1, x2, ..., x} n}, where each x i This represents a model parameter in the model, i=1,2,…,N;

[0015] Construct a model prediction evaluation function g(F) to evaluate the model's prediction results under the model parameter set F;

[0016] For model parameters x i Construct all possible arrays that do not contain x i The model parameter subset F i ; Use the constructed model to predict the evaluation function for each subset of model parameters F i The computational model includes and does not include model parameters x. i The degree of bolt feature capture and prediction results are compared with the calculation of different parameter configurations, i.e., different model parameter subsets F. i The contribution of the bolt image feature prediction box to the prediction result is calculated using the following formula:

[0017]

[0018]

[0019]

[0020]

[0021] Where K is the number of real bolts in a single sample in the sample library; For the complete parameter set F S Below, the bounding box of the k-th real bolt; Let F be the predicted bounding box of the k-th bolt under the model parameter set F; For bolt feature mask, i.e., the contribution of bolt-related parameters; g(F S ) Use the full parameter set F for the model S Bolt feature capture results at that time, g(F) i Use a subset of model parameters F for the model i The bolt feature capture results at that time; the sign function is the sign function, and the exp function is the exponential operation. A value greater than 0 indicates a positive contribution, while a value less than 0 indicates a negative contribution.

[0022] Furthermore, the construction function analysis of the correlation between model parameters and image features also includes: based on a subset F of model parameters. i Based on the size and bolt image features, construct parameter contribution values. Calculate and construct the parameter contribution matrix The specific calculation formula is as follows:

[0023]

[0024]

[0025]

[0026] Among them, |F i | is a subset of model parameters F i Size; |F S | is the set of all parameters F S Size; To account for the degree of overlap between subsets in the bolt image feature capture results, the degree of overlap between subsets and the whole set is quantized; For the model parameter subset F i The degree of contribution to bolt feature capture; For the complete parameter set F S The degree of contribution to bolt feature capture; For the subset of model parameters F i The degree of contribution of the bolt image feature prediction box to the prediction result; The sum of feature masks for all sample bolt images; parameter contribution values. Represents the model parameters x i The average marginal contribution of the bolt feature capture results indicates that the higher the parameter contribution value, the better the model parameter x. i The greater the impact on the model's prediction results; A matrix vector consisting of the contribution values ​​of the corresponding parameters.

[0027] Furthermore, during the training of the improved YOLOv8 model, parameter contribution values ​​are used to calculate the parameter contribution values ​​of different model parameters to bolt image samples. Analysis reveals the model parameter components that have the greatest impact on bolt image sample evaluation, namely the attention layer. Then, parameter contribution value analysis is performed on different attention heads within the attention layer. Based on the magnitude of the parameter contribution values, attention heads whose parameter contribution values ​​for different bolt identifications are greater than a set value are identified. A dynamic output weight is assigned to each attention head, and the output weight matrix Q for all attention heads is constructed.

[0028]

[0029] in It is the attention head output weight matrix of the i-th model parameter, i=1,2,…,N; Assign weights to the j-th attention head for the i-th model parameter, where j = 1, 2, ..., N;

[0030] During the training of the improved YOLOv8 model, a dynamic output weight is assigned to each attention head. The weight of attention heads that contribute significantly to bolt image samples is increased, while the weight of attention heads that contribute little is decreased. A comprehensive loss function is then constructed. During training, the model parameters are optimized to output weights, calculated as follows:

[0031]

[0032]

[0033]

[0034] Where A is the actual bolt position bounding area; B is the predicted bolt position bounding area; It is the weight of the j-th attention head of the i-th model parameter during the t-th training round; For each round of training, make a small increment of adjustments; To optimize the logical factor, used to determine whether optimization should be performed; is the contribution value of the bolt feature capture parameters of the j-th attention head; Q is the output weight matrix of bolt feature capture parameters for all attention heads; Dr() is the operation to achieve diagonal dominance of matrix Q, so as to optimize the output of model parameters for the corresponding bolt features, thereby achieving accurate capture of bolt features; k b This represents the set of attention heads corresponding to the bolt features; the sign function is the sign function. It is the average weight of the j-th attention head for the i-th model parameter.

[0035] Furthermore, a GUI library including PyQt5 was used to build the UI interface of the bolt detection system, and the trained UAV bolt detection model was loaded.

[0036] The present invention also provides an electronic device, including a memory, a processor, and computer program instructions stored in the memory and executable by the processor, wherein when the processor executes the computer program instructions, it can implement the steps of the method described above.

[0037] The present invention also provides a computer-readable storage medium having stored thereon computer program instructions that can be executed by a processor, wherein when the processor executes the computer program instructions, it can implement the steps of the method described above.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] 1. This invention innovatively constructs a novel parameter contribution value quantification system by deeply analyzing the intrinsic relationship between model parameters and bolt features. It deeply integrates bolt features into the parameter contribution evaluation, accurately quantifying the specific contribution of each attention point to bolt feature capture. This design breaks down the "black box" barrier of traditional models, making the model's internal decision-making logic clearer and more transparent. This not only helps operators intuitively understand the model's evaluation rules but also enhances the model's interpretability and credibility in engineering applications.

[0040] 2. This invention specifically optimizes the weighted parameter adjustment loss function and innovatively incorporates a model parameter output weight guidance mechanism. This improvement constructs a more refined model optimization system. On the one hand, it allows the model to independently and accurately optimize the bolt prediction box, significantly reducing localization and recognition errors. On the other hand, through the directional guidance of parameter output weights, the model can actively focus on the errors of key bolt positions, effectively improving the accuracy of bolt localization and state recognition in complex scenarios, and comprehensively enhancing the overall detection performance of the model.

[0041] 3. This invention embeds a dedicated attention mechanism into the neck of the YOLOv8 model. This mechanism can accurately focus on the bolt area through adaptive weight allocation, effectively filtering interference information such as equipment background and environmental noise. At the same time, it strengthens the transmission and preservation of the bolt's subtle defect features, significantly improving the targeting and accuracy of feature extraction. Moreover, the embedding of this attention mechanism adopts a lightweight design, without adding too much additional computational burden. While greatly improving the model's ability to identify small targets and subtle defects in bolts, it perfectly retains the lightweight advantage of the original model, making it fully adaptable to engineering application scenarios of real-time UAV inspection, achieving a balance between detection accuracy and operational efficiency. Attached Figure Description

[0042] Figure 1 This is a flowchart of the model construction method of the present invention.

[0043] Figure 2 This is a schematic diagram of a drone sampling flight.

[0044] Figure 3 This is a schematic diagram of image sample acquisition.

[0045] Figure 4 Schematic diagram of sample expansion.

[0046] Figure 5 Example diagram of bolt positioning based on LabelImg.

[0047] Figure 6 The UI interface of the bolt inspection system.

[0048] Figure 7 This is a detection diagram of the model constructed in this invention applied to bolt inspection.

[0049] Figure 8 This is a diagram showing the detection results of the model constructed in this invention applied to bolt detection. Detailed Implementation

[0050] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings.

[0051] This invention provides a method for constructing a UAV bolt detection model based on image feature-guided parameters, comprising:

[0052] Bolt image samples were acquired by taking pictures with drones, and a sample library for multiple scenarios was built;

[0053] Based on the YOLOv8 model, the correlation between the parameters of the function analysis model and the features of the bolt image is constructed to build an improved YOLOv8 model;

[0054] Based on a sample library from multiple scenarios, the improved YOLOv8 model was trained by adjusting the loss function with weighted parameters to obtain a UAV bolt detection model.

[0055] The following is a detailed implementation process of the present invention.

[0056] The overall process of this invention is as follows: Figure 1 This invention provides a method for constructing a UAV bolt detection model based on image feature-guided parameters, mainly including the following steps:

[0057] Step 1: Using a drone as a carrier, according to Figure 2 , Figure 3 The flight trajectory was used to photograph the physical model of the standard section of the tower crane within a safe distance. Image samples of high-strength bolts of the normal standard section were collected from different angles to construct an experimental dataset. At the same time, the common tower crane bolt conditions in actual situations were selected, with normal, loose, and missing bolts as the main working conditions. The bolts were also photographed under different lighting conditions and background areas.

[0058] Step 2: Process the images taken by the drone using image generation methods, such as random rotation, background replacement, and exposure adjustment. Figure 4 To further expand the image samples, the original image samples were cross-expanded using the sample expansion method described above, constructing a sample library based on the original images and the expanded images. The dataset includes multiple drone-captured image samples labeled with the type and location of the bolts to be detected.

[0059] Step 3: Use Labelimg data annotation software to annotate the image samples taken by the UAV. Manually select the image range of each bolt with different working conditions in the standard section images of the sample library and assign corresponding working condition labels. The annotation process is as follows: Figure 5 After the annotation is completed, the program will automatically save a label file corresponding to each image. This file contains the bolt condition category number, the normalized coordinates of the label center, and the width and height information of the label area.

[0060] Step 4: Randomly select samples for model training and the rest for model testing. Finally, divide the training set and test set into a 4:1 ratio.

[0061] To further analyze the improvement in model interpretability brought about by the proposed method, the rules learned by the model are analyzed and adjusted using its output attention distribution. Considering the relationship between model modules and different bolt feature datasets, the model modules are evaluated using different bolt feature datasets. The model components are: data embedding layer, position encoding layer, encoder layer (attention layer, feedforward network layer), and fully connected layer. An ablation method is used, replacing the ablation module with a linear layer of the same dimension, and then testing is performed using the same test set. The evaluation results are shown in Table 1.

[0062] Table 1 Evaluation results of different ablation model modules

[0063]

[0064] Based on the correspondence between modules and evaluation results, the attention layer is found to be most relevant to the evaluation results.

[0065] During training, the epoch was set to 400 rounds, the batch size to 64, and the input image resolution to 640*640. After adding a coordinate attention module, the bolt position features in the image were utilized, and the model's recognition of bolt position features was enhanced by constructing a loss function. To verify the effectiveness of the method of this invention, four improved models (V8-0-V8-3) were trained according to the patent method, and ablation experiments were conducted on the same dataset. Recall (R), precision (P), and mean AP (mAP) were used as model evaluation metrics. The experiments all used mAP@0.5 and mAP@0.5-0.95, that is, when the IoU threshold was set to 0.5 and a series of IoU thresholds in the range of 0.5 to 0.95, the average of each class was calculated before calculating mAP. The performance of the models with different improved strategies is shown in Table 2.

[0066] Table 2. Model performance of different improvement strategies

[0067]

[0068] Table 2 shows that experiment v8-0 represents the original model, with mAP0.5 and mAP0.5-0.95 at 92.7% and 64.7%, respectively; experiments v8-1 and v8-2 respectively added the CA attention mechanism and replaced it with The comparative experiments on the loss function show improvements in both mAP 0.5 and mAP 0.5-0.95. Each improvement proposed in this paper contributes to improved model performance, demonstrating the effectiveness and scientific validity of the proposed method. Experiment v8-3 involves adding a CA attention mechanism and replacing it with... The improved loss function model outperforms the traditional model by adding a single term. This leads to the UAV-based bolt detection model guided by image features, as described in this invention.

[0069] Step 5: Apply the model construction method of this invention to the bolt inspection system. Build the UI interface of the bolt inspection system based on GUI libraries such as PyQt5 to improve the operability of the inspection system. The system interface includes: instructions for use, sample set expansion, image detection, video detection, and output reports. The bolt inspection system UI interface is shown below. Figure 6 By loading a pre-trained improved YOLOv8 model, bolt image samples can be identified and detected, resulting in the detection results and allowing for the repair of faulty bolts. The detection process and results are as follows: Figure 7 , Figure 8 .

[0070] The present invention also provides an electronic device, including a memory, a processor, and computer program instructions stored in the memory and executable by the processor, wherein when the processor executes the computer program instructions, it can implement the steps of the method described above.

[0071] The present invention also provides a computer-readable storage medium having stored thereon computer program instructions that can be executed by a processor, wherein when the processor executes the computer program instructions, it can implement the steps of the method described above.

[0072] The above are preferred embodiments of the present invention. Any changes made to the technical solution of the present invention that do not exceed the scope of the technical solution of the present invention shall fall within the protection scope of the present invention.

Claims

1. A method for constructing a UAV bolt detection model based on image feature-guided parameters, characterized in that, include: Bolt image samples were acquired by taking pictures with drones, and a sample library for multiple scenarios was built; Based on the YOLOv8 model, an association between the function analysis model parameters and bolt image features is constructed to build an improved YOLOv8 model. This association includes: incorporating an attention mechanism at the c2f module in the neck of the YOLOv8 model and constructing the association between the function analysis model parameters and image features. Define the model parameter set F = {x1, x2, ..., x} n }, where each x i This represents a model parameter in the model, i=1,2,…,N; Construct a model prediction evaluation function g(F) to evaluate the model's prediction results under the model parameter set F; For model parameters x i Construct all possible arrays that do not contain x i The model parameter subset F i ; Use the constructed model to predict the evaluation function for each subset of model parameters F i The computational model includes and does not include model parameters x. i The degree of bolt feature capture and prediction results are compared with the calculation of different parameter configurations, i.e., different model parameter subsets F. i The contribution of the bolt image feature prediction box to the prediction result is calculated using the following formula: Where K is the number of real bolts in a single sample in the sample library; For the complete parameter set F S Below, the bounding box of the k-th real bolt; Let F be the predicted bounding box of the k-th bolt under the model parameter set F; For bolt feature mask, i.e., the contribution of bolt-related parameters; g(F S ) Use the full parameter set F for the model S Bolt feature capture results at that time, g(F) i Use a subset of model parameters F for the model i The bolt feature capture results at that time; the sign function is the sign function, and the exp function is the exponential operation. A value greater than 0 indicates a positive contribution, while a value less than 0 indicates a negative contribution. Based on a sample library from multiple scenarios, the improved YOLOv8 model was trained by adjusting the loss function with weighted parameters to obtain a UAV bolt detection model.

2. The method for constructing a UAV bolt detection model based on image feature-guided parameters according to claim 1, characterized in that, The steps of obtaining bolt image samples by taking pictures with a drone include: using a drone to fly at high altitude to take pictures of the standard section of the large equipment, using the minimum safe distance supported by the drone's optical obstacle avoidance system as the preferred shooting distance, controlling the drone to the acquisition point so that the camera on the drone is facing the standard section of the large equipment, acquiring images of the connection between the two standard sections in each direction, and taking pictures of the standard section from three angles in each direction: top view, eye view, and bottom view, to obtain bolt image samples.

3. The method for constructing a UAV bolt detection model based on image feature-guided parameters according to claim 1, characterized in that, The steps for constructing a sample library under multiple scenarios include: processing bolt image samples using image generation methods including random rotation, background replacement, and exposure adjustment, and cross-expanding them using sample expansion methods to construct a bolt image sample library based on the original images and expanded images.

4. The method for constructing a UAV bolt detection model based on image feature-guided parameters according to claim 1, characterized in that, The construction of the sample library also includes a labeling step: using Labelimg data labeling software, the bolt image samples in the sample library are selected one by one and the corresponding working condition labels are assigned. After the labeling is completed, the label files corresponding to the bolt image samples are saved. The label files include the bolt working condition category number, the normalized coordinates of the label center, and the width and height information of the label area. Then, based on the bolt image samples in the bolt image sample library and the corresponding label files, a sample dataset is constructed and randomly divided into a training set and a test set at a ratio of 4:

1.

5. The method for constructing a UAV bolt detection model based on image feature-guided parameters according to claim 1, characterized in that, The construction function analyzes the correlation between model parameters and image features, and also includes: based on a subset F of model parameters. i Based on the size and bolt image features, construct parameter contribution values. Calculate and construct the parameter contribution matrix The specific calculation formula is as follows: Among them, |F i | is a subset of model parameters F i Size; |F S | is the set of all parameters F S Size; To account for the degree of overlap between subsets in the bolt image feature capture results, the degree of overlap between subsets and the whole set is quantized; For the model parameter subset F i The degree of contribution to bolt feature capture; For the complete parameter set F S The degree of contribution to bolt feature capture; For the subset of model parameters F i The degree of contribution of the bolt image feature prediction box to the prediction result; The sum of feature masks for all sample bolt images; parameter contribution values. Represents the model parameters x i The average marginal contribution of the bolt feature capture results indicates that the higher the parameter contribution value, the better the model parameter x. i The greater the impact on the model's prediction results; A matrix vector consisting of the contribution values ​​of the corresponding parameters.

6. The method for constructing a UAV bolt detection model based on image feature-guided parameters according to claim 5, characterized in that, In the training process of the improved YOLOv8 model, parameter contribution values ​​are used to calculate the parameter contribution values ​​of different model parameters to bolt image samples. Analysis reveals the model parameter components that have the greatest impact on bolt image sample evaluation, namely the attention layer. Then, parameter contribution value analysis is performed on different attention heads within the attention layer. Based on the magnitude of the parameter contribution values, attention heads whose parameter contribution values ​​for different bolt identifications are greater than a set value are identified. A dynamic output weight is assigned to each attention head, and the output weight matrix Q of all attention heads is constructed. in It is the attention head output weight matrix of the i-th model parameter, i=1,2,…,N; Assign weights to the j-th attention head for the i-th model parameter, where j = 1, 2, ..., N; During the training of the improved YOLOv8 model, a dynamic output weight is assigned to each attention head. The weight of attention heads that contribute significantly to bolt image samples is increased based on the dynamic adjustment of parameter contribution values. For bolt image samples, reduce the weight of attention heads that contribute little to the image sample. Construct a comprehensive loss function During training, the model parameters are optimized to output weights, calculated as follows: Where A is the actual bolt position bounding area; B is the predicted bolt position bounding area; It is the weight of the j-th attention head of the i-th model parameter during the t-th training round; For each round of training, make a small increment of adjustments; To optimize the logical factor, used to determine whether optimization should be performed; is the contribution value of the bolt feature capture parameters of the j-th attention head; Q is the output weight matrix of bolt feature capture parameters for all attention heads; Dr() is the operation to achieve diagonal dominance of matrix Q, so as to optimize the output of model parameters for the corresponding bolt features, thereby achieving accurate capture of bolt features; k b This represents the set of attention heads corresponding to the bolt features; the sign function is the sign function. It is the average weight of the j-th attention head for the i-th model parameter.

7. The method for constructing a UAV bolt detection model based on image feature-guided parameters according to claim 1, characterized in that, The UI interface of the bolt detection system is built using a GUI library including PyQt5, and the trained UAV bolt detection model is loaded.

8. An electronic device, characterized in that, It includes a memory, a processor, and computer program instructions stored in the memory and executable by the processor, which, when executed by the processor, enable the implementation of the steps of the method as described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, It stores computer program instructions that can be executed by a processor, and when the processor executes the computer program instructions, it can implement the steps of the method as described in any one of claims 1-7.

Images