Unmanned aerial vehicle spacer installation error detection method and system based on visual intelligence

Abstract

The application discloses a visual intelligence-based unmanned aerial vehicle spacer installation error detection method and system, and relates to the technical field of visual intelligence. The method comprises the following steps: acquiring multiple frames of visual images of a spacer installation area collected by an unmanned aerial vehicle, inputting the visual images into a spacer detection model, obtaining a target detection frame and a visual positioning confidence, screening an optimal detection frame from the target detection frame, evaluating a quality evaluation parameter, obtaining a time sequence consistency confidence based on a position change of the target detection frame, fusing the visual positioning confidence corresponding to the optimal detection frame, the quality evaluation parameter and the time sequence consistency confidence, calculating a comprehensive confidence score, switching a visual dominant mode or a tactile auxiliary mode, acquiring a spatial deviation amount through a mechanical arm contact measurement of the unmanned aerial vehicle when the tactile auxiliary mode is switched, obtaining a final deviation amount after processing, and outputting a detection result. The application effectively improves the spacer installation error detection precision in a complex environment.

Description

Technical Field

[0001] This invention relates to the field of visual intelligence technology, and more specifically to a method and system for detecting installation errors of spacers in unmanned aerial vehicles based on visual intelligence. Background Technology

[0002] With the rapid development of drone technology and computer vision, drone inspection has become an important means of power transmission line operation and maintenance. In the scenario of spacer installation error detection, existing technologies typically use drones equipped with high-definition cameras to acquire images of the spacer installation area, identify the position of the spacer in the image through image processing algorithms, and analyze the installation status of the spacer based on image features or comparison with preset templates.

[0003] However, when drones operate in complex outdoor environments, factors such as changes in lighting, rain, fog, and target occlusion can severely affect the quality of visual images, leading to a significant decrease in the positioning accuracy of target detection algorithms. Existing methods output detection results in a fixed pattern regardless of image quality, which can easily result in false positives or false negatives when image quality is poor, leading to insufficient detection accuracy and failing to guarantee the stability and reliability of the detection results. Summary of the Invention

[0004] This invention provides a method and system for detecting spacer installation errors in unmanned aerial vehicles (UAVs) based on visual intelligence, aiming to solve the technical problem of insufficient accuracy in detecting spacer installation errors in complex environments in the prior art.

[0005] In view of the above problems, the present invention provides a method and system for detecting installation errors of spacers in unmanned aerial vehicles based on visual intelligence.

[0006] In a first aspect, the present invention provides a method for detecting installation errors of spacers in unmanned aerial vehicles based on visual intelligence, comprising:

[0007] Acquire multiple frames of visual images of the spacer installation area collected by the UAV, input the multiple frames of visual images into the pre-trained spacer detection model, and obtain the target detection box and visual positioning confidence of each frame of visual images.

[0008] The optimal detection box is selected from the target detection boxes. The optimal detection box is then subjected to visual quality assessment and structural verification to obtain quality assessment parameters. These parameters include target space ratio, texture clarity attenuation, illumination zone contrast value, and structural integrity matching degree.

[0009] The temporal consistency confidence score is obtained based on the positional changes of the target detection box;

[0010] The system integrates the visual location confidence, quality assessment parameters, and temporal consistency confidence corresponding to the optimal detection box to calculate a comprehensive confidence score, and switches between the visual dominant mode and the tactile auxiliary mode based on the comprehensive confidence score.

[0011] When switching to tactile-assisted mode, the spatial deviation is obtained by contact measurement through the robotic arm of the drone. The spatial deviation is processed according to preset rules to obtain the final deviation. Based on the final deviation, it is determined whether there is a positioning error in the installation position of the spacer and the detection result is output.

[0012] Secondly, the present invention provides a visual intelligence-based unmanned aerial vehicle (UAV) spacer installation error detection system, comprising:

[0013] The target detection module is used to acquire multiple frames of visual images of the spacer installation area collected by the UAV. The multiple frames of visual images are input into the pre-trained spacer detection model to obtain the target detection box and visual positioning confidence of each frame of visual images.

[0014] The quality assessment module is used to select the optimal detection box from the target detection box, perform visual quality assessment and structural verification on the optimal detection box, and obtain quality assessment parameters, including target space ratio, texture clarity attenuation, illumination zone contrast value and structural integrity matching degree.

[0015] The temporal consistency evaluation module is used to obtain the temporal consistency confidence score based on the positional changes of the target detection box;

[0016] The comprehensive confidence calculation module is used to fuse the visual location confidence, quality assessment parameters and temporal consistency confidence corresponding to the optimal detection box, calculate the comprehensive confidence score, and switch between the visual dominant mode and the tactile auxiliary mode based on the comprehensive confidence score.

[0017] The tactile-assisted detection module is used to obtain spatial deviation through contact measurement by the robotic arm of the drone when switching to tactile-assisted mode. The spatial deviation is processed according to preset rules to obtain the final deviation. Based on the final deviation, it is determined whether there is a positioning error in the installation position of the spacer and the detection result is output.

[0018] One or more technical solutions provided in this invention have at least the following technical effects or advantages:

[0019] This invention provides a method and system for detecting spacer installation errors in unmanned aerial vehicles (UAVs) based on visual intelligence. By acquiring multi-frame visual images of the spacer installation area collected by the UAV, the system inputs these images into a pre-trained spacer detection model to obtain target detection boxes and visual positioning confidence scores. The optimal detection box is then selected from these boxes, ensuring the reliability of the visual data used in subsequent evaluations. Visual quality assessment and structural verification are performed on the optimal detection box to obtain quality assessment parameters. Furthermore, temporal consistency confidence scores are obtained by combining the positional changes of the target detection box, achieving multi-dimensional quantification of the visual detection results. A comprehensive confidence score is calculated by fusing the visual positioning confidence score, quality assessment parameters, and temporal consistency confidence scores corresponding to the optimal detection box. The system adaptively switches between a vision-dominant modality and a tactile-assisted modality, enabling dynamic adjustment of the detection strategy. When switching to the tactile-assisted modality, spatial deviation is obtained through contact measurement using a robotic arm and processed according to preset rules to obtain the final deviation. The final deviation is used to determine whether there is a positioning error in the spacer installation position and outputs the detection result, providing effective supplementary verification when visual information is insufficient. This invention effectively improves the accuracy and reliability of spacer installation error detection in complex environments. Attached Figure Description

[0020] Figure 1 A flowchart illustrating the method for detecting installation errors of spacers in unmanned aerial vehicles based on visual intelligence, as provided in an embodiment of the present invention.

[0021] Figure 2 A schematic diagram of the overall logic of the UAV spacer installation error detection method based on visual intelligence provided in an embodiment of the present invention;

[0022] Figure 3 A schematic diagram of the structure of a visual intelligence-based UAV spacer installation error detection system provided in an embodiment of the present invention;

[0023] The components represented by each number in the attached diagram are explained below:

[0024] Target detection module 11, quality assessment module 12, temporal consistency assessment module 13, comprehensive confidence calculation module 14, and tactile assist detection module 15. Detailed Implementation

[0025] This invention provides a method and system for detecting spacer installation errors in unmanned aerial vehicles (UAVs) based on visual intelligence, which addresses the technical problem of insufficient accuracy in detecting spacer installation errors in complex environments in existing technologies.

[0026] Example 1, as Figure 1 , Figure 2 As shown, this invention provides a method for detecting installation errors of spacers in unmanned aerial vehicles (UAVs) based on visual intelligence. The method includes:

[0027] S100: Acquire multiple frames of visual images of the spacer installation area collected by the UAV, input the multiple frames of visual images into the pre-trained spacer detection model, and obtain the target detection box and visual positioning confidence of each frame of visual images.

[0028] In this embodiment of the invention, due to airflow disturbances during the flight of the UAV, single-frame visual images may suffer from motion blur, target occlusion, or sudden changes in illumination, leading to unstable target detection results. By acquiring multiple frames of visual images and performing target detection on each frame, the optimal detection box can be selected from the multi-frame detection results for subsequent processing, improving the reliability of the detection. Simultaneously, the positional change information of the target detection boxes in each frame of visual images can be used to calculate the subsequent temporal consistency confidence score. Therefore, it is necessary to continuously acquire multiple frames of visual images using the UAV and perform target detection on each frame of visual images using a pre-trained spacer bar detection model, providing the corresponding target detection boxes and visual positioning confidence scores for each frame in subsequent steps.

[0029] Step S100 in the method provided in this embodiment of the invention includes:

[0030] The drone is controlled to continuously acquire multiple frames of visual images of the spacer bar installation area at a preset frame rate. Preprocessing is performed on each frame of visual images, including scaling the visual images to the input size of the spacer bar detection model and normalizing the pixels.

[0031] The preprocessed visual images of multiple frames are input into the pre-trained spacer bar detection model. After inference by the spacer bar detection model, the target detection box and visual localization confidence of each frame of visual image are output.

[0032] First, the drone continuously acquires multiple frames of visual images of the spacer bar installation area at a preset frame rate. The drone's high-definition visible light camera continuously acquires images of the spacer bar installation area at the preset frame rate. The preset frame rate refers to the number of images acquired by the camera per unit of time. The frame rate setting needs to balance detection accuracy and data transmission bandwidth: too high a frame rate will result in excessive data volume and increase the computational burden on the onboard processor; too low a frame rate may miss key frames and reduce the effectiveness of multi-frame filtering. For example, the preset frame rate is set to 10 frames per second (fps). The number of visual images continuously acquired by the drone is denoted as N frames, where N ranges from 5 to 15 frames. After acquisition, a set of multiple visual images containing N frames is obtained, denoted as... ,in This represents the visual image of the i-th frame.

[0033] For example, at the construction site of a 220kV transmission line project, a drone equipped with a high-definition visible light camera hovered 5 meters above the conductor and continuously captured images of the spacer installation area at a preset frame rate of 10fps, collecting a total of 8 visual images, denoted as . Each frame of the visual image has a resolution of 1920×1080 pixels.

[0034] Secondly, preprocessing is performed on multiple frames of visual images. This preprocessing includes scaling the visual images to the input size of the spacer bar detection model and performing pixel normalization. The pre-trained spacer bar detection model requires input images to have a fixed size and a normalized range of pixel values. The original resolution of different frames of visual images may vary depending on camera settings, and the original pixel value range is 0 to 255, which is inconsistent with the data distribution during model training. Therefore, preprocessing operations need to be performed on each frame of visual images separately.

[0035] Image scaling refers to adjusting the resolution of the original visual image to the fixed input size required by the spacer bar detection model. The spacer bar detection model uses a YOLOv5n network structure, with a standard input size of 640×640 pixels. For a visual image with an original resolution of 1920×1080 pixels, the scaling operation uses bilinear interpolation to scale the image width from 1920 pixels to 640 pixels and the height from 1080 pixels to 640 pixels, resulting in a scaled image size of 640×640 pixels. The aspect ratio is maintained during scaling, and any insufficient areas are filled with a preset fill color.

[0036] Pixel normalization refers to mapping the pixel value of each pixel in the scaled image from its original integer range to the numerical range used during model training. In the original visual image, the pixel value range for each pixel is an integer from 0 to 255, while deep learning models typically require the numerical range of the input data to be consistent with the training data. Pixel normalization divides the pixel value of each pixel by 255, mapping the pixel value from the integer range of 0 to 255 to the floating-point range of 0 to 1. The normalized image data format is a floating-point tensor with RGB channels, and a size of 640×640×3. Simultaneously, the channel order of the image data needs to be converted from the original BGR order to RGB order to maintain consistency with the channel order used during model training.

[0037] For example, for the first frame of 8 visual images The original resolution was 1920×1080 pixels. This frame was scaled to 640×640 pixels using bilinear interpolation, with any insufficient pixels padded with 114 pixels. Then, the pixel value of each pixel in the scaled image was divided by 255, mapping the pixel values ​​from 0 to 255 to 0 to 1. Simultaneously, the image channel order was converted from BGR to RGB. After this preprocessing, the first frame of the visual image was converted into a floating-point tensor of size 640×640×3. The remaining 7 frames of the visual image... to Perform the same preprocessing operations sequentially.

[0038] Next, the preprocessed multi-frame visual images are input into the pre-trained spacer bar detection model. After inference by the spacer bar detection model, the model outputs the target detection bounding box and visual location confidence score for each frame of visual image. The preprocessed visual images are then input into the pre-trained spacer bar detection model for inference. The spacer bar detection model is a target detection model based on a deep convolutional neural network. It receives an input image, performs forward propagation calculations, and outputs the location information of all detected target objects in the image and their corresponding confidence scores.

[0039] In this context, the target detection box refers to the rectangular region containing the spacers predicted by the spacer bar detection model in the input image. The position of the target detection box is represented by the pixel coordinates of the top-left and bottom-right corners of the rectangle, denoted as . ,in The x-coordinate of the top left corner. The top-left ordinate is the vertical coordinate. The x-coordinate of the bottom right corner. The vertical coordinate is the bottom right corner, and the unit of coordinates is pixels. For each frame of the input image, the spacer bar detection model may output one or more target detection boxes, each corresponding to a detected spacer bar target.

[0040] Wherein, visual location reliability refers to the probability that the spacer bar detection model actually has a spacer bar within the output target detection box, denoted as . The visual location reliability score ranges from 0 to 1. The closer the value is to 1, the higher the confidence level of the spacer bar detection model in the detection result. The closer the value is to 0, the lower the confidence level of the model in the detection result.

[0041] Specifically, for each preprocessed visual image frame, the spacer bar detection model outputs one or more target detection boxes and the visual localization confidence score for each target detection box after inference. To simplify subsequent processing, for each visual image frame, only the target detection box with the highest visual localization confidence score and its corresponding visual localization confidence score are retained as the target detection result for that frame.

[0042] For example, the preprocessed 8 frames of visual images are input into a pre-trained spacer bar detection model. The model processes the first frame image... After inference, three object detection boxes are output, with corresponding visual localization confidence scores of 0.92, 0.45, and 0.21, respectively. Only the object detection box with the highest confidence score is retained as the object detection result for the first frame. This process is repeated sequentially... to The same reasoning and filtering operations were performed. Specifically, the visual location reliability of the first frame was 0.92, with the top-left corner coordinates of the detection box being (720, 405) and the bottom-right corner being (1200, 675); the visual location reliability of the second frame was 0.88, with the top-left corner coordinates of the detection box being (718, 406) and the bottom-right corner being (1198, 676); ...; the visual location reliability of the eighth frame was 0.90, with the top-left corner coordinates of the detection box being (720, 406) and the bottom-right corner being (1200, 676).

[0043] The pre-training of the spacer detection model includes:

[0044] Multiple historical spacer visual images captured by drones under different lighting conditions and shooting distances during historical inspections are collected and preprocessed. The spacer targets in each preprocessed historical spacer visual image are labeled, and the target detection box labels corresponding to each historical spacer visual image are obtained. All labeled historical spacer visual images are divided into training set and validation set according to a preset ratio.

[0045] An initial spacer detection model is constructed based on the improved YOLOv5n network structure.

[0046] The initial spacer detection model is iteratively trained using the training set. After each training round, the model is validated using the validation set. Training stops when the detection accuracy on the validation set reaches a preset accuracy threshold and the loss function value on the training set tends to stabilize, thus obtaining the pre-trained spacer detection model.

[0047] First, multiple historical visual images of spacers taken by drones under different lighting conditions and shooting distances during historical inspections are collected and preprocessed. The spacer detection model needs to be trained on a large number of labeled historical images to accurately identify spacers from visual images. The construction of the training dataset is the foundation of model training; the diversity of the dataset and the quality of the annotation directly affect the model's detection accuracy and generalization ability.

[0048] Specifically, multiple historical visual images of spacers were collected from drones during historical inspections, taken under varying lighting conditions and at different shooting distances. Lighting conditions included shooting scenarios with different intensities such as bright sunlight, soft light on cloudy days, and low illumination at dawn and dusk. Shooting distances included shooting scenarios at distances of 3 meters, 5 meters, and 8 meters from the guide wire. These diverse acquisition conditions ensured that the training dataset covered the appearance changes of spacers under different environments, improving the model's generalization ability. The resolution of the collected historical spacer visual images was consistent with the resolution of the visual images acquired by the drones, at 1920×1080 pixels. Each historical spacer visual image underwent preprocessing, namely scaling each image to 640×640 pixels and performing pixel normalization, while converting the channel order from BGR to RGB.

[0049] For example, 5000 historical visual images of spacers are collected from a historical inspection database, including 2000 images taken under sunny conditions, 2000 images taken under cloudy conditions, and 1000 images taken under low-light conditions at dawn and dusk; 1500 images taken at a shooting distance of 3 meters, 2000 images taken at 5 meters, and 1500 images taken at 8 meters. All 5000 historical images are then preprocessed sequentially, including scaling to 640×640 pixels, pixel normalization, and channel conversion.

[0050] Secondly, the spacer targets in each preprocessed historical spacer visual image are labeled, and the corresponding target detection box labels for each historical spacer visual image are obtained. All labeled historical spacer visual images are then divided into training and validation sets according to a preset ratio. The target detection box label refers to the position coordinates of a precise rectangular region containing the spacer target, manually marked in the image, serving as a supervision signal during model training. During the labeling process, the labelers use rectangular bounding boxes to precisely outline all visible spacer targets in each historical spacer visual image, recording the coordinates of the top-left and bottom-right corners of each target's bounding box. Each historical spacer visual image may contain one or more spacer targets, and each target corresponds to one target detection box label. After labeling, each historical image corresponds to one or more target detection box labels, which are defined by... The labeled historical spacer visual images were randomly divided into training and validation sets in an 8:2 ratio.

[0051] For example, 5000 preprocessed historical spacer visual images were manually labeled one by one. One historical image contained two spacer targets, and the labelers annotated the detection boxes for each target: the detection box label for the first spacer was (710, 400, 1210, 680), and the detection box label for the second spacer was (1500, 420, 1800, 700). After labeling all 5000 images, the corresponding target detection box labels for each of the 5000 images were obtained. The 5000 labeled historical spacer visual images were randomly divided in an 8:2 ratio, with 4000 images used as the training set and 1000 images used as the validation set.

[0052] Secondly, based on the improved YOLOv5n network structure, an initial spacer detection model is constructed. YOLOv5n is a lightweight version of the YOLOv5 object detection algorithm series, specifically designed for mobile and embedded devices; its full name is YOLOv5Nano. The YOLOv5n network structure adopts a three-segment architecture consisting of a backbone, a neck, and a head.

[0053] The backbone network is responsible for multi-scale feature extraction from the input image. It employs an improved CSP-Darknet53 structure, achieving efficient feature extraction through depthwise separable convolutions and cross-stage local connections (CSPNet). The backbone network is configured as follows: the input layer receives a 640×640×3 normalized image tensor; the first layer is a convolutional layer (Conv) with 64 output channels, a 6×6 kernel size, a stride of 2, and padding of 2; the second layer is a convolutional layer (Conv) with 128 output channels, a 3×3 kernel size, and a stride of 2; the third layer is a C3 module (containing 3 Bottleneck cross-stage local connections) with 128 output channels; and the fourth layer is a convolutional layer (Conv) with 256 output channels and a stride of 2. The kernel size is 3×3 with a stride of 2; the fifth layer is a C3 module with 256 output channels; the sixth layer is a convolutional layer (Conv) with 512 output channels, a 3×3 kernel size, and a stride of 2; the seventh layer is a C3 module with 512 output channels; the eighth layer is a convolutional layer (Conv) with 1024 output channels, a 3×3 kernel size, and a stride of 2; the ninth layer is a C3 module with 1024 output channels; and the tenth layer is a Spatial Pyramid Pooling Fast Module (SPPF) with 1024 output channels and a pooling kernel size of 5. All convolutional layers use the SiLU activation function.

[0054] The neck network is responsible for fusing multi-scale feature maps extracted by the backbone network, enhancing the detection capability for targets of different sizes. The neck network employs a structure combining a Feature Pyramid Network (FPN) and a Path Aggregation Network (PAN). Specifically, the configuration is as follows: Layer 11 is a convolutional layer (Conv) with 512 output channels, a 1×1 kernel size, and a stride of 1; Layer 12 is an upsampling layer (Upsample) with an upsampling factor of 2, using nearest neighbor interpolation; Layer 13 is a concatenation layer (Concat), concatenating the upsampled result with the feature map of Layer 7 of the backbone network along the channel dimension; Layer 14 is a C3 module with 512 output channels; Layer 15 is a convolutional layer (Conv) with 256 output channels, a 1×1 kernel size, and a stride of 1; Layer 16 is an upsampling layer (Upsample) with an upsampling factor of 2; and Layer 17 is a concatenation layer (Concat), concatenating the upsampled result with the feature map of Layer 7 of the backbone network along the channel dimension; Layer 14 is a C3 module with 512 output channels; Layer 15 is a convolutional layer (Conv) with 256 output channels, a 1×1 kernel size, and a stride of 1; Layer 16 is an upsampling layer (Upsample) with an upsampling factor of 2; and Layer 17 is a concatenation layer (Concat), concatenating the upsampled result with the feature map of Layer 7 of the backbone network. The feature maps of the five layers are concatenated along the channel dimension; the eighteenth layer is the C3 module with 256 output channels; the nineteenth layer is a convolutional layer (Conv) with 256 output channels, a kernel size of 3×3, and a stride of 2; the twentieth layer is a concatenation layer (Concat) that concatenates the convolution result with the feature map of the fourteenth layer of the neck network along the channel dimension; the twenty-first layer is the C3 module with 512 output channels; the twenty-second layer is a convolutional layer (Conv) with 512 output channels, a kernel size of 3×3, and a stride of 2; the twenty-third layer is a concatenation layer (Concat) that concatenates the convolution result with the feature map of the eleventh layer of the neck network along the channel dimension; the twenty-fourth layer is the C3 module with 1024 output channels.

[0055] The detection head is responsible for performing final class prediction and bounding box regression on feature maps at three different scales. The detection head employs three detection layers, corresponding to the prediction of small-scale (P3 / 8), medium-scale (P4 / 16), and large-scale (P5 / 32) targets, respectively. Each detection layer outputs the target class probability and bounding box offset for all anchor positions at that scale. The anchor box size used by the detection head is a set of anchor boxes pre-obtained from the training dataset using a clustering algorithm, adapted to the size of the spacer bars.

[0056] The depth multiplier (depth_multiple) is set to 0.33, and the width multiplier (width_multiple) is set to 0.25. The depth multiplier controls the number of repetitions of the Bottleneck in the network module, and the width multiplier controls the number of channels in each convolutional kernel. With the above configuration of depth and width multipliers, the initial spacer bar detection model has approximately 1.9 million parameters (M) and requires approximately 4.5 billion floating-point operations (FLOPs).

[0057] Next, the initial spacer detection model is iteratively trained using the training set. After each training round, the model is validated using the validation set. When the detection accuracy on the validation set reaches the preset accuracy threshold and the loss function value on the training set tends to stabilize, training is stopped, and the pre-trained spacer detection model is obtained.

[0058] During training, each iteration randomly selects a batch of images from the training set and inputs them into the model. The model outputs the predicted class probability and predicted bounding box position of the target in each image. The model's prediction results are compared with the manually labeled target detection boxes, and the difference between the predicted and true values ​​is calculated using a loss function. The loss function consists of two parts: classification loss and regression loss. The classification loss uses the cross-entropy loss function to measure the accuracy of the target class prediction; the regression loss uses the CIoU loss function to measure the degree of matching between the predicted bounding box and the ground truth bounding box in terms of position, size, and shape. The gradient of the loss function with respect to the network parameters of each layer is calculated using the backpropagation algorithm, and the network parameters are updated along the gradient descent direction using the optimizer. This process is repeated to gradually reduce the value of the loss function.

[0059] Specifically, after each complete training round, images from the validation set are input into the current model for validation, and the model's detection accuracy on the validation set is calculated. Detection accuracy is evaluated using mean precision (mAP); a higher mAP value indicates better model performance. Training stops when the detection accuracy on the validation set reaches a preset accuracy threshold and the loss function value on the training set stabilizes. The preset accuracy threshold is set to an mAP value of at least 85%. The loss function value stabilizing on the training set means that the decrease in the loss function value is less than 1% over 10 consecutive training rounds. When both conditions are met, it indicates that the model has converged sufficiently, having learned enough features on the training set without overfitting. After training stops, the network parameters of the current model are saved, resulting in the pre-trained spacer bar detection model.

[0060] For example, the model is trained using 4000 training images and 1000 validation images. During training, 8 images are input per batch, with an initial learning rate of 0.001, a stochastic gradient descent optimizer with a momentum of 0.937 and a weight decay coefficient of 0.0005. The model is trained iteratively for 300 epochs on the training set. After each epoch, the 1000 images from the validation set are input into the model, and the mAP value is calculated. When training reaches epoch 280, the mAP value on the validation set reaches 86.2%, exceeding the preset accuracy threshold of 85%, and the loss function value on the training set has decreased by less than 1% for 12 consecutive epochs, meeting the stopping condition for training. At this point, training is stopped, and the trained model parameters are saved, resulting in the pre-trained spacer bar detection model.

[0061] In this embodiment of the invention, a drone is controlled to continuously acquire multiple frames of visual images of the spacer bar installation area at a preset frame rate. Scaling and pixel normalization preprocessing are then performed to ensure the input data conforms to the standard input format requirements of the spacer bar detection model. By inputting the preprocessed multiple frames of visual images into a pre-trained spacer bar detection model for inference, the target detection box and visual location confidence corresponding to each frame of visual image are obtained. This provides a data foundation for subsequently selecting the optimal detection box from the multi-frame detection results and calculating the temporal consistency confidence score. The pre-trained spacer bar detection model, through thorough training on diverse historical datasets, possesses stable detection capabilities for spacer bar targets under different lighting conditions and shooting distances, ensuring the accuracy and reliability of the target detection box and visual location confidence score.

[0062] S200: Select the optimal detection box from the target detection boxes, perform visual quality assessment and structural verification on the optimal detection box, and obtain quality assessment parameters, including target space ratio, texture clarity attenuation, illumination zone contrast value and structural integrity matching degree.

[0063] In this embodiment of the invention, due to airflow disturbances during the drone's flight, the visual image quality varies across different frames. Some frames may suffer from motion blur, target occlusion, or sudden changes in illumination, leading to inaccurate target detection box positions. By selecting the target detection box with the highest visual positioning confidence from multiple detection results as the optimal detection box, the positional accuracy of the detection boxes participating in subsequent quality assessment can be maximized. However, visual positioning confidence only reflects the detection model's certainty about the existence of the target and does not fully consider factors such as the reasonable proportion of the target in the image, the clarity of the image texture, the uniformity of illumination, and the integrity of the target structure. Therefore, visual quality assessment and structural verification of the optimal detection box are necessary to quantify the reliability of the detection box from multiple dimensions, providing multi-dimensional quality assessment parameters for subsequent calculation of the comprehensive confidence score.

[0064] Step S200 in the method provided in this embodiment of the invention includes:

[0065] Based on the target detection bounding boxes and visual localization confidence of each frame of visual image, the target detection bounding box with the highest visual localization confidence is selected as the optimal detection box;

[0066] Obtain the pixel area of ​​the optimal detection box, and calculate the proportion of the pixel area of ​​the optimal detection box to the total pixel area of ​​the visual image of the frame in which the optimal detection box is located, as the target space proportion.

[0067] Edge detection is performed on the image region within the optimal detection box to obtain the average edge intensity. The ratio of the average edge intensity to the preset reference edge intensity is calculated to obtain the texture clarity attenuation.

[0068] The visual image of the frame containing the optimal detection box is uniformly divided into multiple grid regions. The average brightness of each grid region is obtained, and the ratio of the standard deviation to the average value of the average brightness of each grid region is calculated as the illumination partition contrast value.

[0069] The image region within the optimal detection box is binarized and its contour is extracted. Based on the extracted contour skeleton, the number of clamps and the geometry of the frame are identified. The structural integrity matching degree is obtained according to the degree of matching between the number of clamps and the number of preset standard clamps, and the degree of matching between the geometry of the frame and the preset standard frame template.

[0070] First, based on the target detection bounding boxes and visual location confidence scores of each frame of visual images, the target detection bounding box with the highest visual location confidence score is selected as the optimal detection bounding box. Each frame of visual images is inferred by the spacer bar detection model, and each frame outputs a corresponding target detection bounding box and visual location confidence score. The visual location confidence score ranges from 0 to 1, with a higher value indicating greater confidence in the detection result. Due to differences in image quality between different frames, their corresponding visual location confidence scores also vary. By comparing the visual location confidence scores of each frame, the frame with the highest confidence score is selected as the optimal detection bounding box.

[0071] Specifically, in step S100, the target detection boxes and visual localization confidence scores corresponding to the N frames of visual images are obtained, denoted as set. ,in This represents the target detection box in the i-th frame. This represents the visual localization confidence score of the i-th frame. From this, select... The target detection box corresponding to the maximum value. As the optimal detection box, let it be denoted as Optimal detection box The visual image of the frame is denoted as .

[0072] For example, the visual positioning confidence scores corresponding to the eight visual images are as follows: Frame 1: 0.92, Frame 2: 0.88, Frame 3: 0.85, Frame 4: 0.91, Frame 5: 0.87, Frame 6: 0.79, Frame 7: 0.76, and Frame 8: 0.90. The highest confidence score is 0.92, corresponding to the first visual image. Target detection bounding box The target detection box As the optimal detection box Its position is at the top left corner coordinates (720, 405) and the bottom right corner coordinates (1200, 675). The visual image of the frame containing this detection box is... .

[0073] Secondly, the pixel area of ​​the optimal detection box is obtained, and the ratio of the pixel area of ​​the optimal detection box to the total pixel area of ​​the visual image of the frame containing the optimal detection box is calculated as the target space ratio. The target space ratio refers to the proportion of pixel area occupied by the optimal detection box in the visual image of its frame, used to measure whether the proportion of the space bar target in the image is reasonable. A target space ratio that is too small means the target is too far from the camera or the focal length is insufficient, resulting in unclear target details; a target space ratio that is too large means the target is too close to the camera or the focal length is too large, which may cause the target to exceed the image boundary or result in the loss of local features. By calculating the target space ratio, the reasonableness of the target's proportion in the image can be quantitatively evaluated.

[0074] Specifically, the pixel area of ​​the optimal detection box Calculate using the following formula: ,in , The optimal detection box The x and y coordinates of the top left corner, , The optimal detection box The x and y coordinates of the bottom right corner. The total pixel area of ​​the visual image of the frame containing the optimal detection box. This is calculated by multiplying the width in pixels by the height in pixels for the image frame. For an image with a resolution of 1920×1080 pixels, Pixels. Target space occupancy Calculate using the following formula: Target space percentage The value ranges from 0 to 1, and the larger the value, the larger the proportion of the target in the picture.

[0075] For example, the optimal detection box The top-left corner coordinates are (720, 405), and the bottom-right corner coordinates are (1200, 675). The pixel area of ​​the optimal detection box... Pixels. Visual image of the frame containing the optimal detection box. Total pixel area Pixels. Target space occupancy .

[0076] Next, edge detection is performed on the image region within the optimal detection box to obtain the average edge intensity. The ratio of the average edge intensity to the preset reference edge intensity is then calculated to obtain the texture sharpness attenuation. The texture sharpness attenuation measures the texture sharpness of the image region within the optimal detection box. Sharp images have higher edge intensity, while blurry images have lower edge intensity. By performing edge detection on the image region within the optimal detection box, the texture sharpness of that region can be quantitatively evaluated.

[0077] Edge detection refers to identifying locations in an image where pixel values ​​change drastically, i.e., the edges of objects. Edge strength refers to the gradient magnitude of each edge point in the edge detection result; a larger gradient magnitude indicates a more pronounced edge. The average edge strength is the arithmetic mean of the edge strengths of all pixels within the optimal detection box, denoted as . It is used to characterize the overall texture clarity of the area.

[0078] Among them, the preset reference edge intensity refers to the standard value of edge intensity of the same type of spacer bar at the same shooting distance under ideal imaging conditions, obtained in advance through experimental calibration, and is denoted as . The method for obtaining the preset reference edge intensity is as follows: For spacers of different types and specifications, multiple standard images with clear focus and uniform illumination are acquired at the same shooting distance under standard laboratory environment or ideal on-site lighting conditions. Edge detection is performed on the image area within the target detection box in each standard image, the average edge intensity is calculated, and the statistical median of the average edge intensity of all standard images is taken as the reference edge intensity of that type of spacer at that shooting distance.

[0079] Specifically, texture sharpness attenuation Calculate using the following formula: .when Greater than or equal to At that time, texture clarity attenuation A value of 0 indicates that the texture sharpness of the area meets or exceeds the reference standard, with no attenuation. For example, for the optimal detection box... Sobel edge detection is performed on the image region within the region, and the arithmetic mean of the edge intensities of all pixels within the region is calculated to obtain the current edge intensity. Based on the preset reference edge strength of this type of spacer at a shooting distance of 5 meters. Texture clarity attenuation .

[0080] Furthermore, the visual image of the frame containing the optimal detection box is uniformly divided into multiple grid regions. The average brightness of each grid region is obtained, and the ratio of the standard deviation to the average value of the average brightness of each grid region is calculated as the illumination partition contrast value. The illumination partition contrast value is used to measure the uniformity of illumination distribution between different regions of the image. Images with uniform illumination have smaller brightness differences between regions, while images with uneven illumination have some regions that are too bright or too dark. By uniformly dividing the image into multiple grid regions and statistically analyzing the average brightness of each grid region, the illumination uniformity of the image can be quantitatively evaluated. A grid region refers to a rectangular sub-region formed by uniformly dividing the visual image in the width and height directions. In this embodiment of the invention, the visual image is uniformly divided into 10 rows × 10 columns, totaling 100 grid regions. The size of each grid region is determined by dividing the total image size by the number of divisions.

[0081] The average brightness of each grid region refers to the arithmetic mean of the brightness values ​​of all pixels within that region. Brightness refers to the grayscale value after converting an RGB color image to a grayscale image, ranging from 0 to 255; a higher value indicates a brighter region. The average brightness of each grid region is denoted as the set. , where M is the total number of grid regions, M=100.

[0082] Specifically, the arithmetic mean of the average brightness of each grid region is denoted as... The standard deviation of the average brightness of each grid region is denoted as . Contrast value of illumination zones Calculate using the following formula: Contrast value of illumination zones The value of is typically between 0 and 1. A larger value indicates a greater difference in brightness between different areas of the image, and more uneven lighting; a smaller value indicates that the brightness of different areas of the image is more similar, and the lighting is more uniform. Greater than hour, If the value is greater than 1, then take... .

[0083] For example, the optimal detection box Visual image of the frame The area was evenly divided into 100 grid regions, each 10 rows x 10 columns, with each grid region measuring 192 x 108 pixels. The average brightness of each grid region was calculated, resulting in 100 brightness values. to Calculate the arithmetic mean of the average brightness across all grid regions. Calculate the standard deviation of the average brightness of all grid regions. Contrast value of illumination zones .

[0084] Finally, the image region within the optimal detection box is binarized and its contour is extracted. Based on the extracted contour skeleton, the number of clamps and the geometry of the frame are identified. The structural integrity matching degree is obtained according to the degree of matching between the number of clamps and the number of preset standard clamps, and the degree of matching between the geometry of the frame and the preset standard frame template.

[0085] The structural integrity matching degree is used to measure the degree of matching between the detected spacer structure and the preset standard structure. As a supporting hardware for split conductors, the spacer has fixed geometric structural features: a double-split conductor spacer contains two clamps and one rectangular frame, while a quadruple-split conductor spacer contains four clamps and one rectangular or hexagonal frame. By detecting the number of clamps and the geometry of the frame of the spacer and comparing it with the preset standard structure, the structural integrity of the spacer can be verified.

[0086] Binarization refers to the process of converting the grayscale image within the optimal detection box into a binary image. This involves comparing the grayscale value of each pixel with a preset binarization threshold. Pixels with grayscale values ​​greater than the threshold are set to white (255), and pixels with grayscale values ​​less than or equal to the threshold are set to black (0), thus separating the target region from the background region. Contour extraction refers to extracting the boundary curves of the target region from the binary image, i.e., the contour skeleton of the spacer bar.

[0087] After extracting the contour skeleton, the number of wire clamps and the geometry of the frame are identified from the contour skeleton. A wire clamp is a holding component that connects the spacer bar to the wire; in the contour skeleton, it is represented by a protruding structure extending outward from the frame. The frame is the main structure of the spacer bar, and in the contour skeleton, it is represented by a closed geometry. The number of wire clamps is identified by counting the protruding structures in the contour skeleton, denoted as . The recognition of the frame geometry is achieved by performing polygon fitting on the closed region of the contour skeleton, and the frame shape is determined to be one of rectangle, hexagon or octagon based on the fitting result.

[0088] The preset standard clamp quantity refers to the standard quantity of clamps predetermined according to the design specifications of this type of spacer bar, denoted as For double-split conductor spacers, For four-split conductor spacers, Quantity and integrity rate of wire clamps Calculate using the following formula: .when and When they are equal, This indicates that the number of wire clamps is complete; when and Not waiting, Less than 1.

[0089] The preset standard frame template refers to a frame geometry template pre-established according to the design specifications of this type of spacer, stored in the UAV's onboard memory. Frame geometry matching is achieved by calculating the similarity between the detected frame shape and the preset standard frame template. The normalized correlation coefficient of Hu moment invariants is used as the measure of shape similarity, denoted as [missing information]. Hu moment invariants are a set of moment features that remain unchanged under translation, rotation, and scale changes, effectively describing the shape characteristics of objects. Shape similarity. The value ranges from 0 to 1. The closer the value is to 1, the more consistent the detected frame shape is with the standard template.

[0090] Specifically, structural integrity matching degree Calculate using the following formula: .in This is a weighting coefficient for the integrity rate of the wire clamp quantity. These are the weighting coefficients for shape similarity. and All are preset positive real numbers and satisfy In this embodiment of the invention, , .

[0091] For example, for the optimal detection box The image region within the spacer was binarized and its contour extracted. Two wire clamps and one rectangular frame were identified in the extracted contour skeleton. This spacer is a double-split conductor spacer, with a preset standard number of wire clamps. Number of wire clamps detected Quantity of wire clamps and integrity rate The detected frame geometry is rectangular. The Hu moment normalized correlation coefficient is calculated between the detected frame and a preset standard rectangular template to obtain the shape similarity. Structural integrity matching degree .

[0092] In this embodiment of the invention, the target detection box with the highest visual positioning confidence is selected as the optimal detection box from the multi-frame detection results, ensuring the positional accuracy of the detection boxes participating in subsequent quality assessment. The reasonableness of the target's proportion in the image is quantitatively assessed by calculating the target's spatial proportion. The texture clarity of the image is quantitatively assessed by performing edge detection on the image region within the optimal detection box and calculating the texture clarity attenuation. The illumination uniformity of the image is quantitatively assessed by uniformly dividing the image into multiple grid regions and calculating the illumination partition contrast value. The structural integrity of the spacer is quantitatively assessed by binarizing and contour extracting the image region within the optimal detection box, and calculating the structural integrity matching degree based on the number of clamps and the frame geometry. These four dimensions of quality assessment parameters together constitute a multi-dimensional quality assessment of the optimal detection box, providing comprehensive and quantitative quality assessment parameters for subsequent calculation of the overall confidence score.

[0093] S300: Based on the positional changes of the target detection box, the temporal consistency confidence score is obtained.

[0094] In this embodiment of the invention, during drone hovering or low-speed flight, the position of the same spaced-bar target should remain relatively stable across multiple consecutive frames, meaning the center point coordinates of the target detection boxes in each frame should not fluctuate drastically. If the position of the target detection box changes significantly across multiple consecutive frames, it indicates that the visual detection result is unstable, possibly due to degraded image quality, target occlusion, or false detections by the detection model. Therefore, by analyzing the dispersion of the center point coordinates of the target detection boxes in each frame, the temporal consistency of the visual detection results can be quantitatively evaluated. The lower the dispersion of the center point coordinates, the more consistent the detection results across frames, and the higher the reliability of the visual detection; the higher the dispersion, the more unstable the detection results across frames, and the lower the reliability of the visual detection. By calculating the inter-frame localization variance of the center point coordinates and comparing it with the preset maximum allowable inter-frame localization variance, the temporal consistency confidence level can be obtained, serving as a quantitative indicator for measuring the temporal stability of the visual detection results.

[0095] Step S300 in the method provided in this embodiment of the invention includes:

[0096] Obtain the coordinates of the center point of the target detection box corresponding to each frame of visual image;

[0097] Calculate the inter-frame positioning variance of the center point coordinates;

[0098] Obtain the preset maximum allowable inter-frame positioning variance, and obtain the timing consistency confidence level based on the ratio of the inter-frame positioning variance to the maximum allowable inter-frame positioning variance.

[0099] First, obtain the center point coordinates of the target detection box corresponding to each frame of the visual image. After inference by the spacer bar detection model, the target detection box corresponding to that frame is output. The target detection box is represented by the coordinates of the top left corner of the rectangle. and the coordinates of the bottom right corner The center point coordinates of the target detection bounding box refer to the pixel coordinates of the geometric center of the rectangle, denoted as . ,in The coordinates of the center in the horizontal direction are These are the coordinates of the center point in the vertical direction. The coordinates of the center point are calculated using the following formula: For N frames of visual images, the center point coordinates of the target detection boxes in each frame constitute a sequence of center point coordinates. , where N is the total number of frames in the multi-frame visual image.

[0100] For example, the target detection bounding boxes corresponding to the 8 frames of visual images are: Frame 1 (720, 405, 1200, 675), Frame 2 (718, 406, 1198, 676), ..., Frame 8 (720, 406, 1200, 676). The center point coordinates of the target detection bounding boxes in each frame are calculated: Frame 1: , Frame 2: , ;...; Frame 8: , .

[0101] Secondly, calculate the inter-frame localization variance of the center point coordinates. Inter-frame localization variance refers to the dispersion of the center point coordinates of the target detection box in each frame relative to the average of the center point coordinates of all frames. It measures the magnitude of positional fluctuations in the detection results across frames. A larger inter-frame localization variance indicates greater positional differences between the detection results in each frame, resulting in poorer temporal stability of visual detection; a smaller inter-frame localization variance indicates more concentrated positions of the detection results in each frame, resulting in better temporal stability of visual detection.

[0102] Specifically, first, the arithmetic mean of the x-coordinates of the center points of all frames is calculated. arithmetic mean of the ordinate and the vertical axis Calculate using the following formula: ; Then calculate the inter-frame positioning variance. Calculate using the following formula: , where i represents the frame number, ranging from 1 to N, and N is the total number of frames in the multi-frame visual image.

[0103] For example, the x-coordinates of the center points of the 8 frames are 960, 958, 962, 959, 961, 957, 963, and 960, respectively, and the y-coordinates are 540, 541, 539, 542, 540, 543, 538, and 541, respectively. The average x-coordinate... Average value on the ordinate The sum of squared deviations of the center point coordinates of each frame from the average value was calculated, yielding a total deviation squared = 46.0, and the inter-frame positioning variance... .

[0104] Next, the preset maximum permissible inter-frame positioning variance is obtained. Based on the ratio of the inter-frame positioning variance to the maximum permissible inter-frame positioning variance, the timing consistency confidence score is calculated. The maximum permissible inter-frame positioning variance refers to the preset upper limit threshold of the inter-frame positioning variance, denoted as... When the inter-frame localization variance reaches or exceeds this threshold, it indicates that the position fluctuations of the target detection boxes in each frame are too large, and the temporal consistency is reduced to a minimum. The maximum permissible inter-frame localization variance is set based on the following: when the UAV is hovering or flying at low speed, the position change of the same target in consecutive frames is mainly caused by the attitude disturbance of the UAV itself, and its position fluctuation usually does not exceed 5 pixels. Therefore, the maximum permissible inter-frame localization variance is set to the variance value corresponding to a fluctuation of 5 pixels, i.e. .

[0105] Time series consistency confidence This value measures the temporal consistency of detection results across frames, ranging from 0 to 1. A value closer to 1 indicates greater consistency in the positions of detection results across frames, signifying better temporal stability of the visual detection; a value closer to 0 indicates greater fluctuation in the positions of detection results across frames, signifying poorer temporal stability of the visual detection. The temporal consistency confidence score is calculated using the following formula: ,when Greater than or equal to hour, The value is 0. For example, inter-frame positioning variance. Preset maximum allowable inter-frame positioning variance Time-series consistency confidence level .

[0106] In this embodiment of the invention, by obtaining the center point coordinates of the target detection box corresponding to each frame of visual image, a positional data basis is provided for temporal consistency analysis; by calculating the inter-frame localization variance of the center point coordinates, the degree of dispersion of the detection results in each frame in terms of position is quantitatively evaluated, and the smaller the inter-frame localization variance, the more consistent the detection results in each frame are; by obtaining the preset maximum allowable inter-frame localization variance and calculating the ratio of the inter-frame localization variance to the maximum allowable inter-frame localization variance, the temporal consistency confidence score is obtained, and the inter-frame localization variance is mapped to a confidence score in the range of 0 to 1, providing a quantitative indicator for measuring the temporal stability of visual detection for subsequent calculation of the comprehensive confidence score, and enriching the dimensions of comprehensive confidence evaluation.

[0107] S400: It integrates the visual location confidence, quality assessment parameters and temporal consistency confidence corresponding to the optimal detection box, calculates the comprehensive confidence score, and switches between the visual dominant mode and the tactile auxiliary mode based on the comprehensive confidence score.

[0108] In this embodiment of the invention, in step S200, four quality assessment parameters have been obtained: target space occupancy, texture sharpness attenuation, illumination zone contrast, and structural integrity matching. In step S300, temporal consistency confidence has been obtained. Simultaneously, the optimal detection box selected in step S200 corresponds to a visual localization confidence score. These six parameters reflect the reliability of the visual detection results from different dimensions: visual localization confidence reflects the detection model's certainty about the target's existence; target space occupancy reflects the reasonableness of the target's proportion in the image; texture sharpness attenuation reflects the image's texture sharpness; illumination zone contrast reflects the image's illumination uniformity; structural integrity matching reflects the integrity of the spacer structure; and temporal consistency confidence reflects the temporal stability of the detection results for each frame. However, these six parameters have different dimensions, different numerical ranges, and different contributions to the reliability of the detection results. Therefore, it is necessary to first convert each parameter into a quality factor with a unified dimension, then fuse them into a single comprehensive confidence score through weighted summation, and adaptively switch the detection mode based on the comparison result of the comprehensive confidence score and the preset threshold.

[0109] Step S400 in the method provided in this embodiment of the invention includes:

[0110] The comprehensive confidence score is calculated by fusing the visual location confidence, quality assessment parameters, and temporal consistency confidence of the optimal detection box, including:

[0111] The target space occupancy is compared with the preset ideal occupancy range to obtain the occupancy quality factor. The texture clarity attenuation is converted into the texture quality factor, and the lighting partition contrast value is converted into the lighting quality factor.

[0112] Obtain the preset first weight coefficient, second weight coefficient, third weight coefficient, fourth weight coefficient, fifth weight coefficient and sixth weight coefficient, wherein the first weight coefficient corresponds to the visual localization confidence of the optimal detection box, the second weight coefficient corresponds to the occupancy ratio quality factor, the third weight coefficient corresponds to the texture quality factor, the fourth weight coefficient corresponds to the illumination quality factor, the fifth weight coefficient corresponds to the structural integrity matching degree, and the sixth weight coefficient corresponds to the temporal consistency confidence.

[0113] Based on the first, second, third, fourth, fifth, and sixth weighting coefficients, the visual location confidence, occupancy ratio quality factor, texture quality factor, illumination quality factor, structural integrity matching degree, and temporal consistency confidence of the optimal detection box are weighted and summed to obtain the comprehensive confidence score.

[0114] First, because the three parameters—target space occupancy, texture sharpness attenuation, and illumination zone contrast—have inconsistent numerical meanings and directions, the target space occupancy has an ideal range; deviations from this range, whether large or small, indicate a decrease in quality. Similarly, for texture sharpness attenuation and illumination zone contrast, larger values ​​indicate worse quality. Therefore, it is necessary to convert each of these three parameters into a unified quality factor with a value range of 0 to 1 for subsequent weighted fusion. A quality factor value closer to 1 indicates better quality in that dimension, while a value closer to 0 indicates worse quality.

[0115] This includes comparing the target space occupancy with a preset ideal occupancy range to obtain the occupancy quality factor, converting texture sharpness attenuation into a texture quality factor, and converting illumination zone contrast values ​​into illumination quality factors, including:

[0116] Obtain the preset ideal occupancy ratio range, which consists of the minimum ideal occupancy ratio and the maximum ideal occupancy ratio;

[0117] When the target space occupancy falls within the preset ideal occupancy ratio range, the occupancy ratio quality factor is set to 1.

[0118] When the target space occupancy is less than the minimum ideal occupancy ratio, the difference between the minimum ideal occupancy ratio and the target space occupancy is divided by the minimum ideal occupancy ratio, and 1 is subtracted from the resulting ratio to obtain the occupancy ratio quality factor.

[0119] When the target space occupancy ratio is greater than the maximum ideal occupancy ratio, the difference between the target space occupancy ratio and the maximum ideal occupancy ratio is divided by the maximum ideal occupancy ratio, and the resulting ratio is subtracted from 1 to obtain the occupancy ratio quality factor.

[0120] Subtract the texture sharpness attenuation from 1 to obtain the texture quality factor;

[0121] Subtract the contrast value of the illumination zone from 1 to obtain the illumination quality factor.

[0122] First, a preset ideal occupancy ratio range is obtained, which consists of a minimum ideal occupancy ratio and a maximum ideal occupancy ratio. When the target space occupancy ratio falls within the preset ideal occupancy ratio range, the occupancy ratio quality factor is set to 1. When the target space occupancy ratio is less than the minimum ideal occupancy ratio, the difference between the minimum ideal occupancy ratio and the target space occupancy ratio is divided by the minimum ideal occupancy ratio, and 1 is subtracted from the resulting ratio to obtain the occupancy ratio quality factor. When the target space occupancy ratio is greater than the maximum ideal occupancy ratio, the difference between the target space occupancy ratio and the maximum ideal occupancy ratio is divided by the maximum ideal occupancy ratio, and 1 is subtracted from the resulting ratio to obtain the occupancy ratio quality factor.

[0123] The target space occupancy has an ideal range of values. Within this range, the target's proportion in the image is moderate; it is neither too small, resulting in unclear details, nor too large, causing it to overflow the image or lose local features. The preset ideal occupancy range refers to a pre-defined reasonable range of target space occupancy values, starting from the minimum ideal occupancy ratio. and the maximum ideal occupation ratio Composition, denoted as Minimum ideal occupancy ratio and the maximum ideal occupation ratio It is necessary to ensure that the spacer bar targets occupy a suitable proportion in the image, which is sufficient to guarantee clear and discernible details without exceeding the frame or obscuring key structures. This is usually preset based on the statistical distribution of effective detection samples in a large number of historical inspection images. For example, , .

[0124] Specifically, the percentage of quality factor This occupancy rate quality factor is used to measure the degree to which the target space occupancy deviates from the ideal range. The value ranges from 0 to 1; the closer the value is to 1, the more reasonable the occupancy rate; the closer the value is to 0, the more severely the occupancy rate deviates from the ideal range. The rule for determining the occupancy rate quality factor is: when... Falling into the preset ideal occupancy ratio range, i.e. At that time, the occupied ratio of quality factor The value is 1. Less than the minimum ideal occupancy ratio, i.e. At that time, the occupied ratio of quality factor Calculate using the following formula: .when Greater than the maximum ideal occupancy ratio, i.e. At that time, the occupied ratio of quality factor Calculate using the following formula: When the calculation results for the two deviation scenarios mentioned above are less than 0, The value is 0.

[0125] For example, the proportion of the target space The preset ideal occupancy ratio range is [0.05, 0.25], where... , Since 0.05 ≤ 0.0625 ≤ 0.25, If the occupancy ratio falls within the preset ideal occupancy ratio range, the occupancy ratio quality factor is... .

[0126] Next, subtract the texture sharpness attenuation from 1 to obtain the texture quality factor. Texture sharpness attenuation The value ranges from 0 to 1; a larger value indicates a blurrier texture and lower sharpness. Texture quality factor The texture quality factor is an inverse indicator of texture sharpness attenuation; a higher texture sharpness attenuation corresponds to a lower texture quality factor. The texture quality factor measures the sharpness of image textures, ranging from 0 to 1. A value closer to 1 indicates sharper textures, while a value closer to 0 indicates blurrier textures. Calculate using the following formula: For example, texture sharpness attenuation. Texture quality factor .

[0127] Next, subtract the contrast value of the illumination zone from 1 to obtain the illumination quality factor. Illumination zone contrast value The value typically ranges from 0 to 1; a larger value indicates greater brightness differences between different areas of the image and more uneven illumination. Illumination Quality Factor The illumination quality factor is the inverse indicator of the contrast value of different lighting zones; a higher contrast value corresponds to a lower illumination quality factor. The illumination quality factor measures the uniformity of illumination in an image, ranging from 0 to 1. A value closer to 1 indicates more uniform illumination, while a value closer to 0 indicates less uniform illumination. Calculate using the following formula: When the contrast value of the illumination zone When it is greater than 1, The value is 0. For example, the contrast value of illumination zones. Light quality factor .

[0128] Furthermore, preset weighting coefficients—the first, second, third, fourth, fifth, and sixth—are obtained. The first weighting coefficient corresponds to the visual location confidence of the optimal detection box; the second weighting coefficient corresponds to the occupancy ratio quality factor; the third weighting coefficient corresponds to the texture quality factor; the fourth weighting coefficient corresponds to the illumination quality factor; the fifth weighting coefficient corresponds to the structural integrity matching degree; and the sixth weighting coefficient corresponds to the temporal consistency confidence degree. These weighting coefficients characterize the importance of each parameter in the calculation of the overall confidence score; a larger weighting coefficient indicates a greater impact of that parameter on the overall confidence score.

[0129] Wherein, the first weighting coefficient is denoted as The visual localization confidence corresponding to the optimal detection box Visual location reliability is the direct prediction probability of the detection model regarding the existence of a target. It is the most direct indicator of the reliability of the detection results and therefore receives the highest weight. The second weight coefficient is denoted as... The corresponding occupancy ratio quality factor The target space occupancy reflects the reasonableness of the target's proportion in the image and is an important factor affecting subsequent quality assessment; therefore, it is given a high weight. The third weighting coefficient is denoted as... Corresponding texture quality factor Texture sharpness directly affects the accuracy of edge detection and structure recognition, and is a key factor influencing subsequent quality assessment; therefore, it is given a high weight. The fourth weight coefficient is denoted as... Corresponding to light quality factor Illumination uniformity affects the overall image quality and the difficulty of target recognition, and is an important factor influencing subsequent quality assessment; therefore, it is given a medium weight. The fifth weight coefficient is denoted as... Corresponding structural integrity matching degree Structural integrity matching degree is a direct verification result of the structural integrity of the spacer bars within the optimal detection frame. It is an important indicator for measuring the integrity of the detection target, and therefore is given a high weight. The sixth weight coefficient is denoted as... Corresponding to the time series consistency confidence level Temporal consistency confidence reflects the temporal stability of detection results in each frame and is an indirect verification of the reliability of visual detection; therefore, it is given a moderate weight.

[0130] All six weighting coefficients mentioned above are preset positive real numbers, and satisfy the following conditions: The specific values ​​of each weight coefficient can be preset according to the application scenario. In this embodiment of the invention, the preset values ​​of the six weight coefficients are as follows: , , , , , .

[0131] Finally, based on the first, second, third, fourth, fifth, and sixth weighting coefficients, the visual location confidence, occupancy ratio quality factor, texture quality factor, illumination quality factor, structural integrity matching degree, and temporal consistency confidence of the optimal detection box are weighted and summed to obtain the comprehensive confidence score. Calculate using the following formula: Overall confidence score The value ranges from 0 to 1. The closer the value is to 1, the higher the overall credibility of the visual detection result. The closer the value is to 0, the lower the overall credibility of the visual detection result.

[0132] For example, the visual localization confidence corresponding to the optimal detection box Occupying a proportion of quality factor Texture quality factor Light quality factor Structural integrity matching degree Time-series consistency confidence The six weighting coefficients are as follows: , , , , , Overall confidence score .

[0133] The switching between visual-dominant and tactile-assisted modalities based on the overall confidence score includes:

[0134] Obtain the preset high confidence threshold and low confidence threshold, and compare the comprehensive confidence score with the high confidence threshold and low confidence threshold respectively;

[0135] When the overall confidence score is greater than or equal to the high confidence threshold, switch to the visual-dominant modality;

[0136] When the overall confidence score is less than the low confidence threshold, switch to the haptic-assisted mode;

[0137] When the overall confidence score is greater than or equal to the low confidence threshold but less than the high confidence threshold, the visual dominant modality is maintained and marked as requiring tactile review.

[0138] The overall confidence score is a continuous value from 0 to 1. A higher score indicates higher overall reliability of the visual detection result, making it more suitable for detection using a purely visual method. A lower score indicates lower overall reliability of the visual detection result, requiring tactile assistance for verification. By setting high and low confidence thresholds, the overall confidence score can be divided into three intervals, corresponding to three different detection modalities.

[0139] The high confidence threshold is denoted as . , is a preset threshold parameter, ranging from 0 to 1. When the overall confidence score is greater than or equal to the high confidence threshold, it indicates that the overall credibility of the visual detection result is high enough, and the visual dominant modality can be directly used for detection. In this embodiment of the invention, the high confidence threshold is preset to . .

[0140] The low confidence threshold is denoted as . , is a preset threshold parameter, with a value ranging from 0 to 1, and When the overall confidence score is less than the low confidence threshold, it indicates that the overall reliability of the visual detection result is too low, and it is necessary to switch to the tactile-assisted modality for detection. The low confidence threshold is preset in this embodiment of the invention as follows: .

[0141] The vision-dominated mode refers to the working mode that relies solely on visual images for spacer installation error detection. In this mode, the position of the spacer is directly determined based on the location information of the optimal detection box to determine whether there is a positioning error. The tactile-assisted mode refers to the working mode that, based on visual detection, activates a robotic arm mounted on a drone to perform contact measurement, using physical contact information from tactile sensors to verify the visual detection results. "Requires tactile verification" refers to a situation where the current overall confidence score is in the middle range, meaning the visual detection results are not fully trusted, but not low enough to necessitate a complete switch to the tactile-assisted mode. In this case, the vision-dominated mode is maintained, but the detection result is marked as requiring tactile verification, reminding subsequent processes to prioritize tactile verification when conditions permit.

[0142] Specifically, the overall confidence score Respectively with high confidence threshold and low confidence threshold Comparison: When When switching to the vision-dominated mode. When switching to haptic-assisted mode. At the same time, maintain the visually dominant modality and mark it as requiring tactile verification. For example, the overall confidence score. High confidence threshold Low confidence threshold .because Switch to the vision-dominated mode.

[0143] In this embodiment of the invention, the occupancy ratio quality factor is obtained by comparing the target space occupancy with a preset ideal occupancy ratio range. The texture clarity attenuation is converted into a texture quality factor, and the illumination partition contrast value is converted into an illumination quality factor. Quality evaluation parameters of different dimensions and directions are uniformly converted into quality factors in the range of 0 to 1 with consistent directions, laying the foundation for subsequent weighted fusion. By obtaining preset weight coefficients and weighting and summing the visual positioning confidence, occupancy ratio quality factor, texture quality factor, illumination quality factor, structural integrity matching degree, and temporal consistency confidence of the optimal detection box, a comprehensive confidence score is obtained, realizing a multi-dimensional comprehensive quantitative evaluation of the credibility of visual detection results. By comparing the comprehensive confidence score with preset high and low confidence thresholds, adaptive switching between the visual dominant mode and the tactile auxiliary mode is realized. When the visual detection credibility is high, a lightweight pure visual detection mode is used to improve efficiency. When the visual detection credibility is low, a tactile auxiliary detection mode is activated to ensure accuracy. A tactile verification mark is retained in the intermediate range to prompt subsequent verification, thus balancing detection efficiency and detection accuracy.

[0144] S500: When switching to tactile-assisted mode, the spatial deviation is obtained by contact measurement through the robotic arm of the drone. The spatial deviation is processed according to preset rules to obtain the final deviation. Based on the final deviation, it is determined whether there is a positioning error in the installation position of the spacer and the detection result is output.

[0145] In this embodiment of the invention, if detection relies solely on visual images, the accuracy of visual positioning significantly decreases under adverse conditions such as changes in lighting, rain, fog, and target occlusion. By using a robotic arm mounted on a drone for contact measurement, the actual three-dimensional spatial coordinates of the contact point between the robotic arm's end effector and the spacer can be directly obtained. The actual contact point is then compared with the visually recognized target contact point to obtain the spatial alignment deviation. This spatial alignment deviation reflects the discrepancy between the visual positioning result and the actual physical position, including both visual positioning error and the installation position error of the spacer itself. By decomposing and filtering the spatial alignment deviation, the final deviation can be obtained and compared with a preset installation error allowable threshold to determine whether there is a positioning error in the spacer's installation position. The robotic arm's end effector integrates a multimodal tactile sensor array, which can collect the three-dimensional force vector when the robotic arm's end effector contacts the spacer in real time.

[0146] Step S500 in the method provided in this embodiment of the invention includes:

[0147] Obtain the pixel coordinates of the target contact point corresponding to the optimal detection box, and perform coordinate transformation based on the pixel coordinates to obtain the three-dimensional spatial coordinates of the target contact point;

[0148] Control the robotic arm to move toward the target contact point. When the end of the robotic arm contacts the spacer bar, record the three-dimensional spatial coordinates of the actual contact point. Based on the three-dimensional spatial coordinates of the actual contact point and the three-dimensional spatial coordinates of the target contact point, obtain the spatial alignment deviation.

[0149] Multiple contact measurements were performed at the clamp position of the spacer bar and the center position of the frame to obtain multiple spatial alignment deviations.

[0150] Calculate the angle between the contact force vector direction at the end of the robotic arm and the normal direction of the preset contact surface at the corresponding key position in each contact measurement. Discard contact measurement results with an angle greater than the preset angle threshold, and retain the remaining contact measurement results as valid measurement results.

[0151] The spatial alignment deviation corresponding to each valid measurement result is decomposed into a component along the traverse direction and a component perpendicular to the traverse direction. The median of the component along the traverse direction of all valid measurement results is taken as the final deviation.

[0152] Obtain the preset installation error allowable threshold, compare the final deviation with the preset installation error allowable threshold, and determine that there is no positioning error in the spacer installation position when the final deviation is less than or equal to the preset installation error allowable threshold; otherwise, determine that there is a positioning error in the spacer installation position.

[0153] The output includes the final deviation and the judgment result of the detection.

[0154] First, the pixel coordinates of the target contact point corresponding to the optimal detection box are obtained. Coordinate transformation is then performed based on these pixel coordinates to obtain the three-dimensional spatial coordinates of the target contact point. The target contact point refers to the specific location within the optimal detection box that needs to make physical contact with the end effector of the robotic arm. In this embodiment, the target contact point is selected as the center position of the spacer clamp and the center position of the frame. The center position of the clamp is the geometric center of the clamping component connecting the spacer to the wire, and the center position of the frame is the geometric center of the main frame of the spacer. The pixel coordinates of the target contact point refer to the two-dimensional pixel position of the target contact point in the visual image of the frame containing the optimal detection box, denoted as (u, v), where u is the horizontal pixel coordinate and v is the vertical pixel coordinate. The pixel coordinates of the target contact point are extracted from the optimal detection box: the pixel coordinates of the clamp center position are obtained by identifying the clamp region and calculating its geometric center, and the pixel coordinates of the frame center position are obtained by identifying the frame region and calculating its geometric center.

[0155] Coordinate transformation refers to converting the two-dimensional pixel coordinates of the target contact point in the image coordinate system to three-dimensional spatial coordinates in the world coordinate system. The world coordinate system is a reference coordinate system describing the positions of the drone, robotic arm, and spacer in real three-dimensional space. Coordinate transformation requires the use of the drone's current position and attitude parameters and camera intrinsic parameters. The drone's current position and attitude parameters include its GPS coordinates, IMU attitude angle, and camera gimbal angle. Camera intrinsic parameters include the camera's focal length, principal point coordinates, and distortion coefficients. Coordinate transformation is achieved through the following steps: first, using the camera intrinsic parameters, the pixel coordinates are converted to normalized coordinates in the camera coordinate system; then, using the drone's position and attitude parameters, the coordinates in the camera coordinate system are converted to three-dimensional spatial coordinates in the world coordinate system. After the coordinate transformation is complete, the three-dimensional spatial coordinates of the target contact point are: ,in The horizontal coordinate is . The coordinates are the horizontal longitudinal coordinates (along the direction of the traverse). These are the vertical height coordinates.

[0156] For example, the target contact point corresponding to the optimal detection box is selected as the center position of the clamp, and its pixel coordinates in the visual image of the frame containing the optimal detection box are (u,v)=(960,540). Based on the current GPS coordinates, IMU attitude angle, camera gimbal angle, and camera intrinsic parameters of the UAV, the three-dimensional spatial coordinates of the target contact point in the world coordinate system are calculated through coordinate transformation. The unit is meters.

[0157] Secondly, control the robotic arm to move towards the target contact point. When the end of the robotic arm contacts the spacer bar, record the three-dimensional spatial coordinates of the actual contact point. Based on the three-dimensional spatial coordinates of the actual contact point and the three-dimensional spatial coordinates of the target contact point, obtain the spatial alignment deviation.

[0158] The six-DOF robotic arm mounted on the drone is controlled to move towards the three-dimensional spatial coordinates P_target of the target contact point. The movement of the robotic arm is achieved through inverse kinematics control, that is, calculating the angle values ​​of each joint of the robotic arm based on the target position, and driving the motors of each joint to bring the end effector of the robotic arm to the target position. During the movement of the robotic arm, a multimodal tactile sensor array integrated on the surface of the end effector continuously collects contact force data. The contact force data includes a three-dimensional force vector (Fx, Fy, Fz), where Fx is the horizontal lateral force component, Fy is the horizontal longitudinal force component, and Fz is the vertical force component. The sampling frequency of the tactile sensors is 100 Hz.

[0159] Specifically, when the tactile sensor detects that the magnitude of the contact force vector (Fx, Fy, Fz) exceeds a preset contact force threshold... At that point, it is determined that the end effector of the robotic arm has made contact with the spacer bar. The magnitude of the contact force vector is calculated using the following formula: Among them, the preset contact force threshold In this embodiment of the invention, the value is set to 0.5 Newtons (N). When |F| ≥ 0.5 N, it is determined that the end effector of the robotic arm has made contact with the spacer bar. At this time, the actual three-dimensional spatial coordinates of the end effector of the robotic arm in the world coordinate system are recorded and denoted as . .

[0160] Specifically, the spatial alignment deviation ΔP between the actual contact point's three-dimensional spatial coordinates and the target contact point's three-dimensional spatial coordinates is calculated using the following formula: The three components of the spatial alignment deviation ΔP represent the deviations between the actual contact point and the target contact point in the horizontal, vertical, and transverse directions, respectively. The magnitude of the spatial alignment deviation... .

[0161] For example, controlling the robotic arm to move towards the target contact point Movement. When the tactile sensor detects a contact force vector magnitude |F| = 0.6N, exceeding the preset contact force threshold of 0.5N, it is determined that the end effector of the robotic arm has made contact with the spacer bar, and the actual contact point coordinates are recorded. Spatial alignment deviation Spatial alignment deviation magnitude Meters, approximately 5.7 millimeters.

[0162] Secondly, multiple contact measurements were performed at the clamp position of the spacer and the center position of the frame to obtain multiple spatial alignment deviations. A single contact measurement may not yield accurate results due to factors such as robotic arm positioning errors, contact position deviations, or sensor noise. By performing multiple contact measurements at multiple key positions of the spacer, multiple spatial alignment deviations can be obtained, and the reliability of the measurement results can be improved through subsequent screening and statistical processing.

[0163] The clamp position refers to the location of the clamping component connecting the spacer bar and the conductor. For a double-split conductor spacer bar, there are two clamp positions. The frame center position refers to the geometric center of the main frame of the spacer bar. In this embodiment of the invention, three contact measurements are performed at each of the left clamp center position, the right clamp center position, and the frame center position. For each contact measurement at each position, the pixel coordinates of the target contact point at that position are first obtained and converted into three-dimensional spatial coordinates. Then, the robotic arm is controlled to move towards the target contact point and the actual contact point coordinates are recorded. Finally, the spatial alignment deviation is calculated. There are three key positions: the clamp position and the frame center position. Each position is measured three times, resulting in a total of nine spatial alignment deviations.

[0164] For example, three contact measurements were performed at the center positions of the left and right wire clamps and the frame. The spatial alignment deviation obtained from the first measurement at the center position of the left wire clamp was (0.002, 0.005, -0.002); the spatial alignment deviation obtained from the second measurement at the same position was (0.003, 0.004, -0.003); and the spatial alignment deviation obtained from the third measurement was (0.001, 0.006, -0.001). Three measurements were then performed sequentially at the center positions of the right wire clamp and the frame, resulting in a total of nine spatial alignment deviations.

[0165] Furthermore, the angle between the contact force vector direction of the robotic arm's end effector and the normal direction of the preset contact surface at the corresponding key position is calculated for each contact measurement. Contact measurement results with an angle greater than a preset angle threshold are discarded, and the remaining contact measurement results are retained as valid measurement results. When the robotic arm's end effector contacts the spacer, ideally, the contact force vector direction should be consistent with the surface normal direction at the contact point. If the angle between the contact force vector direction and the preset contact surface normal direction is too large, it indicates that the robotic arm's end effector has contacted the edge, slope, or foreign object of the spacer, and the contact point position of this measurement deviates from the target contact point, making the measurement result unreliable. Therefore, it is necessary to evaluate the reliability of each measurement result by verifying the consistency between the contact force vector direction and the preset contact surface normal direction, and discard unreliable measurement results.

[0166] The preset contact surface normal direction refers to a pre-defined direction vector perpendicular to the surface of the spacer bar at the target contact point, denoted as . Where nx, ny, and nz are the components of the normal direction along the three axes of the world coordinate system, respectively. The preset contact surface normal directions may differ at different key locations and are preset and stored in the UAV's onboard memory. The contact force vector direction refers to the direction of the contact force vector detected by the tactile sensor when the robotic arm's end effector contacts the spacer bar, i.e., the unit direction vector of the contact force vector (Fx, Fy, Fz), denoted as . .

[0167] Specifically, the angle between the direction of the contact force vector and the direction of the normal to the preset contact surface is denoted as . Calculate using the following formula: .in It is the dot product of two direction vectors. The preset angle threshold is denoted as... In this embodiment of the invention, the temperature is set to 30 degrees. If the angle between the contact force vector direction and the preset contact surface normal direction is too large, the contact measurement result is unreliable and should be discarded. When the direction of the contact force vector is consistent with the direction of the normal to the preset contact surface, the contact measurement result is reliable and is retained as a valid measurement result.

[0168] For example, the normal direction of the preset contact surface at the center position of the left clamp is... Along the direction of the conductor. The contact force vector measured in the first measurement is (0.2, 1.8, 0.1), with a modulus of... Contact force vector direction included angle If the angle is less than the preset angle threshold of 30 degrees, the measurement result is retained as a valid measurement result. If the contact force vector of a certain measurement is (0.8, 0.5, 0.3), the modulus... Contact force vector direction included angle If the angle is greater than the preset angle threshold of 30 degrees, the measurement result will be discarded.

[0169] Subsequently, the spatial alignment deviation corresponding to each valid measurement result is decomposed into components along the conductor direction and perpendicular to the conductor direction. The median of the components along the conductor direction of all valid measurement results is taken as the final deviation. The core indicator of spacer installation error is the installation distance deviation along the conductor direction, that is, the difference between the actual installation position and the designed installation position of the spacer along the conductor direction. Deviation in the perpendicular to the conductor direction does not affect the measurement result of conductor spacing. Therefore, it is necessary to decompose the spatial alignment deviation into components along the conductor direction and perpendicular to the conductor direction, and only the component along the conductor direction is used for installation error determination.

[0170] Here, "along the conductor direction" refers to the direction in which the transmission line conductor extends, denoted as the direction vector in the world coordinate system. The spatial alignment deviation ΔP, along the conductor direction, has its component ΔY calculated using the following formula: .in This represents the spatial alignment deviation. This is the unit direction vector for the conductor direction. The vertical conductor direction component includes both horizontal and vertical components and is not used for installation error determination.

[0171] Specifically, after obtaining the components along the traverse direction corresponding to all valid measurement results, the components along the traverse direction are arranged in ascending order, and the median of the arrangement is taken as the final deviation. Using the median effectively avoids the influence of extreme values ​​on the final result, improving the robustness of the measurement results. For example, after screening, two unreliable measurements were removed from nine measurements, leaving seven valid measurements. The components along the traverse direction of each valid measurement were 0.005m, 0.004m, 0.006m, 0.005m, 0.003m, 0.004m, and 0.005m. Arranged from smallest to largest, they are 0.003m, 0.004m, 0.004m, 0.005m, 0.005m, 0.005m, and 0.006m, with the median being the fourth value, 0.005m. Final deviation. .

[0172] Further, a preset installation error allowable threshold is obtained, and the final deviation is compared with the preset installation error allowable threshold. When the final deviation is less than or equal to the preset installation error allowable threshold, it is determined that there is no positioning error in the spacer installation position; otherwise, it is determined that there is a positioning error in the spacer installation position. The preset installation error allowable threshold refers to the maximum allowable value of the spacer installation distance deviation preset according to industry standards, denoted as […]. . According to industry standards, the installation distance deviation of the first spacer on both sides of the tower shall not be greater than ±1.5% of the end sub-span, and the installation distance deviation of the remaining spacers shall not be greater than ±3% of the sub-span.

[0173] Specifically, the final deviation amount is compared with the preset installation error allowable threshold . When , it is determined that there is no positioning error in the installation position of the spacer, that is, the installation is qualified. When , it is determined that there is a positioning error in the installation position of the spacer, that is, the installation is unqualified. For example, the currently detected spacer is the first spacer on both sides of the tower, the end sub-span is 45 meters, and the preset installation error allowable threshold meters. The final deviation amount . Since 0.005m ≤ 0.675m, it is determined that there is no positioning error in the installation position of this spacer, that is, the installation is qualified.

[0174] Finally, the detection result including the final deviation amount and the determination result is output. The detection result refers to the finally output detection report containing the installation error determination conclusion and related data. The detection result shall at least include the final deviation amount and the determination result of whether there is a positioning error in the installation position, so that operation and maintenance personnel can understand the detection conclusion and carry out subsequent processing.

[0175] Specifically, the detection result is output in a structured data format, including the following information fields: line name, detection position, final deviation amount, preset installation error allowable threshold, determination result, detection modality, comprehensive confidence score, and detection timestamp. The detection position includes the tower number and the spacer serial number, the final deviation amount is along the conductor direction, the determination result includes qualified or unqualified, and the detection modality includes a vision-dominant modality or a tactile assistance modality. For example, the output detection result is as follows: the line name is "×× 220kV Transmission Line", the detection position is "the first spacer on the large side of Tower N12", the final deviation amount is 0.005m, the preset installation error allowable threshold is 0.675m, the determination result is "qualified", the detection modality is "tactile assistance modality", the comprehensive confidence score is 0.8666, and the detection time is "2026-08-14-10:32:15".

[0176] In this embodiment of the invention, the visual recognition results are mapped to the world coordinate system through coordinate transformation, providing a target position reference for the robotic arm; the spatial alignment deviation between the actual contact point and the target contact point is obtained through contact measurement by the robotic arm, realizing tactile verification of the visual positioning results; by performing multiple contact measurements at multiple key locations, and after eliminating unreliable measurements with abnormal contact force directions, the median of the component along the conductor direction is taken as the final deviation, improving the accuracy and robustness of the measurement data; by comparing the final deviation with a preset installation error allowable threshold, the automatic determination of the installation position positioning error is realized; by outputting the detection results containing the final deviation and the determination result, clear and complete detection conclusions and data support are provided to maintenance personnel.

[0177] Through the specific implementation methods described above, the embodiments of the present invention achieve the following technical effects:

[0178] This invention provides a method and system for detecting spacer installation errors in unmanned aerial vehicles (UAVs) based on visual intelligence. It acquires multiple frames of visual images for target detection and selects the optimal detection box, ensuring the reliability of the visual data used in subsequent evaluations. By performing visual quality assessment and structural verification on the optimal detection box, quality assessment parameters are obtained, including target spatial proportion, texture clarity attenuation, illumination zone contrast, and structural integrity matching. The temporal consistency confidence score is obtained by combining the positional changes of the target detection boxes in each frame, achieving multi-dimensional quantification of the visual detection results. A comprehensive confidence score is calculated by fusing the visual positioning confidence score, quality assessment parameters, and temporal consistency confidence score corresponding to the optimal detection box. Based on the comprehensive confidence score, the system adaptively switches between a vision-dominant mode and a tactile-assisted mode, achieving dynamic adjustment of the detection strategy. When switching to the tactile-assisted mode, spatial deviation is obtained through contact measurement using a robotic arm and processed according to preset rules to obtain the final deviation. The final deviation is used to determine whether there is a positioning error in the spacer installation position and outputs the detection result, achieving effective supplementary verification when visual information is insufficient. This invention effectively improves the accuracy and reliability of spacer installation error detection in complex environments.

[0179] Example 2, as Figure 3 As shown, this invention provides a visual intelligence-based UAV spacer installation error detection system, the system comprising:

[0180] The target detection module 11 is used to acquire multiple frames of visual images of the spacer installation area collected by the UAV, input the multiple frames of visual images into the pre-trained spacer detection model, and obtain the target detection box and visual positioning confidence of each frame of visual images.

[0181] The quality assessment module 12 is used to select the optimal detection box from the target detection box, perform visual quality assessment and structural verification on the optimal detection box, and obtain quality assessment parameters, including target space ratio, texture clarity attenuation, illumination partition contrast value and structural integrity matching degree.

[0182] The temporal consistency evaluation module 13 is used to obtain the temporal consistency confidence based on the positional changes of the target detection box;

[0183] The comprehensive confidence calculation module 14 is used to fuse the visual location confidence, quality assessment parameters and temporal consistency confidence corresponding to the optimal detection box, calculate the comprehensive confidence score, and switch between the visual dominant mode and the tactile auxiliary mode according to the comprehensive confidence score.

[0184] The tactile-assisted detection module 15 is used to obtain the spatial deviation through contact measurement by the robotic arm of the UAV when switching to the tactile-assisted mode. After processing the spatial deviation according to the preset rules, the final deviation is obtained. Based on the final deviation, it is determined whether there is a positioning error in the installation position of the spacer and the detection result is output.

[0185] In one embodiment, the target detection module 11 is further configured to:

[0186] The drone is controlled to continuously acquire multiple frames of visual images of the spacer bar installation area at a preset frame rate. Preprocessing is performed on each frame of visual images, including scaling the visual images to the input size of the spacer bar detection model and normalizing the pixels.

[0187] The preprocessed visual images of multiple frames are input into the pre-trained spacer bar detection model. After inference by the spacer bar detection model, the target detection box and visual localization confidence of each frame of visual image are output.

[0188] The pre-training of the spacer detection model includes:

[0189] Multiple historical spacer visual images captured by drones under different lighting conditions and shooting distances during historical inspections are collected and preprocessed. The spacer targets in each preprocessed historical spacer visual image are labeled, and the target detection box labels corresponding to each historical spacer visual image are obtained. All labeled historical spacer visual images are divided into training set and validation set according to a preset ratio.

[0190] An initial spacer detection model is constructed based on the improved YOLOv5n network structure.

[0191] The initial spacer detection model is iteratively trained using the training set. After each training round, the model is validated using the validation set. Training stops when the detection accuracy on the validation set reaches a preset accuracy threshold and the loss function value on the training set tends to stabilize, thus obtaining the pre-trained spacer detection model.

[0192] In one embodiment, the quality assessment module 12 is further configured to:

[0193] Based on the target detection bounding boxes and visual localization confidence of each frame of visual image, the target detection bounding box with the highest visual localization confidence is selected as the optimal detection box;

[0194] Obtain the pixel area of ​​the optimal detection box, and calculate the proportion of the pixel area of ​​the optimal detection box to the total pixel area of ​​the visual image of the frame in which the optimal detection box is located, as the target space proportion.

[0195] Edge detection is performed on the image region within the optimal detection box to obtain the average edge intensity. The ratio of the average edge intensity to the preset reference edge intensity is calculated to obtain the texture clarity attenuation.

[0196] The visual image of the frame containing the optimal detection box is uniformly divided into multiple grid regions. The average brightness of each grid region is obtained, and the ratio of the standard deviation to the average value of the average brightness of each grid region is calculated as the illumination partition contrast value.

[0197] The image region within the optimal detection box is binarized and its contour is extracted. Based on the extracted contour skeleton, the number of clamps and the geometry of the frame are identified. The structural integrity matching degree is obtained according to the degree of matching between the number of clamps and the number of preset standard clamps, and the degree of matching between the geometry of the frame and the preset standard frame template.

[0198] In one embodiment, the timing consistency evaluation module 13 is further configured to:

[0199] Obtain the coordinates of the center point of the target detection box corresponding to each frame of visual image;

[0200] Calculate the inter-frame positioning variance of the center point coordinates;

[0201] Obtain the preset maximum allowable inter-frame positioning variance, and obtain the timing consistency confidence level based on the ratio of the inter-frame positioning variance to the maximum allowable inter-frame positioning variance.

[0202] In one embodiment, the comprehensive confidence calculation module 14 is further configured to:

[0203] The comprehensive confidence score is calculated by fusing the visual location confidence, quality assessment parameters, and temporal consistency confidence of the optimal detection box, including:

[0204] The target space occupancy is compared with the preset ideal occupancy range to obtain the occupancy quality factor. The texture clarity attenuation is converted into the texture quality factor, and the lighting partition contrast value is converted into the lighting quality factor.

[0205] Obtain the preset first weight coefficient, second weight coefficient, third weight coefficient, fourth weight coefficient, fifth weight coefficient and sixth weight coefficient, wherein the first weight coefficient corresponds to the visual localization confidence of the optimal detection box, the second weight coefficient corresponds to the occupancy ratio quality factor, the third weight coefficient corresponds to the texture quality factor, the fourth weight coefficient corresponds to the illumination quality factor, the fifth weight coefficient corresponds to the structural integrity matching degree, and the sixth weight coefficient corresponds to the temporal consistency confidence.

[0206] Based on the first, second, third, fourth, fifth, and sixth weighting coefficients, the visual location confidence, occupancy ratio quality factor, texture quality factor, illumination quality factor, structural integrity matching degree, and temporal consistency confidence of the optimal detection box are weighted and summed to obtain the comprehensive confidence score.

[0207] This includes comparing the target space occupancy with a preset ideal occupancy range to obtain the occupancy quality factor, converting texture sharpness attenuation into a texture quality factor, and converting illumination zone contrast values ​​into illumination quality factors, including:

[0208] Obtain the preset ideal occupancy ratio range, which consists of the minimum ideal occupancy ratio and the maximum ideal occupancy ratio;

[0209] When the target space occupancy falls within the preset ideal occupancy ratio range, the occupancy ratio quality factor is set to 1.

[0210] When the target space occupancy is less than the minimum ideal occupancy ratio, the difference between the minimum ideal occupancy ratio and the target space occupancy is divided by the minimum ideal occupancy ratio, and 1 is subtracted from the resulting ratio to obtain the occupancy ratio quality factor.

[0211] When the target space occupancy ratio is greater than the maximum ideal occupancy ratio, the difference between the target space occupancy ratio and the maximum ideal occupancy ratio is divided by the maximum ideal occupancy ratio, and the resulting ratio is subtracted from 1 to obtain the occupancy ratio quality factor.

[0212] Subtract the texture sharpness attenuation from 1 to obtain the texture quality factor;

[0213] Subtract the contrast value of the illumination zone from 1 to obtain the illumination quality factor.

[0214] The switching between visual-dominant and tactile-assisted modalities based on the overall confidence score includes:

[0215] Obtain the preset high confidence threshold and low confidence threshold, and compare the comprehensive confidence score with the high confidence threshold and low confidence threshold respectively;

[0216] When the overall confidence score is greater than or equal to the high confidence threshold, switch to the visual-dominant modality;

[0217] When the overall confidence score is less than the low confidence threshold, switch to the haptic-assisted mode;

[0218] When the overall confidence score is greater than or equal to the low confidence threshold but less than the high confidence threshold, the visual dominant modality is maintained and marked as requiring tactile review.

[0219] In one embodiment, the tactile assist detection module 15 is further configured to:

[0220] Obtain the pixel coordinates of the target contact point corresponding to the optimal detection box, and perform coordinate transformation based on the pixel coordinates to obtain the three-dimensional spatial coordinates of the target contact point;

[0221] Control the robotic arm to move toward the target contact point. When the end of the robotic arm contacts the spacer bar, record the three-dimensional spatial coordinates of the actual contact point. Based on the three-dimensional spatial coordinates of the actual contact point and the three-dimensional spatial coordinates of the target contact point, obtain the spatial alignment deviation.

[0222] Multiple contact measurements were performed at the clamp position of the spacer bar and the center position of the frame to obtain multiple spatial alignment deviations.

[0223] Calculate the angle between the contact force vector direction at the end of the robotic arm and the normal direction of the preset contact surface at the corresponding key position in each contact measurement. Discard contact measurement results with an angle greater than the preset angle threshold, and retain the remaining contact measurement results as valid measurement results.

[0224] The spatial alignment deviation corresponding to each valid measurement result is decomposed into a component along the traverse direction and a component perpendicular to the traverse direction. The median of the component along the traverse direction of all valid measurement results is taken as the final deviation.

[0225] Obtain the preset installation error allowable threshold, compare the final deviation with the preset installation error allowable threshold, and determine that there is no positioning error in the spacer installation position when the final deviation is less than or equal to the preset installation error allowable threshold; otherwise, determine that there is a positioning error in the spacer installation position.

[0226] The output includes the final deviation and the judgment result of the detection.

[0227] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting installation errors of spacers in unmanned aerial vehicles based on visual intelligence, characterized in that, The method includes: Acquire multiple frames of visual images of the spacer installation area collected by the UAV, input the multiple frames of visual images into a pre-trained spacer detection model, and obtain the target detection box and visual positioning confidence of each frame of visual images. The optimal detection box is selected from the target detection box, and the optimal detection box is subjected to visual quality assessment and structural verification to obtain quality assessment parameters, wherein the quality assessment parameters include target space ratio, texture clarity attenuation, illumination zone contrast value and structural integrity matching degree. Based on the positional changes of the target detection box, the temporal consistency confidence level is obtained; The visual location confidence score, the quality assessment parameters, and the temporal consistency confidence score corresponding to the optimal detection box are integrated to calculate a comprehensive confidence score, and the visual dominant mode or tactile auxiliary mode is switched according to the comprehensive confidence score. When switching to the tactile-assisted mode, the spatial deviation is obtained by contact measurement through the robotic arm of the drone. The spatial deviation is processed according to a preset rule to obtain the final deviation. Based on the final deviation, it is determined whether there is a positioning error in the installation position of the spacer and the detection result is output.

2. The method for detecting installation error of UAV spacer bars based on visual intelligence as described in claim 1, characterized in that, Acquire multiple frames of visual images of the spacer installation area captured by a drone, input these multiple frames of visual images into a pre-trained spacer detection model, and obtain the target detection bounding box and visual localization confidence for each frame of visual images, including: The drone is controlled to continuously acquire multiple frames of visual images of the spacer bar installation area at a preset frame rate. Preprocessing is performed on each of the multiple frames of visual images. The preprocessing includes scaling the visual images to the input size of the spacer bar detection model and normalizing the pixels. The preprocessed multi-frame visual images are input into the pre-trained spacer bar detection model. After inference by the spacer bar detection model, the target detection box and visual location confidence corresponding to each frame of visual images are output.

3. The method for detecting installation error of UAV spacer bars based on visual intelligence as described in claim 2, characterized in that, The pre-training of the spacer detection model includes: Multiple historical spacer visual images captured by drones under different lighting conditions and shooting distances during historical inspections are collected and preprocessed. The spacer targets in each preprocessed historical spacer visual image are labeled, and the target detection box labels corresponding to each historical spacer visual image are obtained. All labeled historical spacer visual images are divided into training set and validation set according to a preset ratio. An initial spacer detection model is constructed based on the improved YOLOv5n network structure. The initial spacer detection model is iteratively trained using the training set. After each training round, the model is validated using the validation set. Training stops when the detection accuracy on the validation set reaches a preset accuracy threshold and the loss function value on the training set tends to stabilize, thus obtaining the pre-trained spacer detection model.

4. The method for detecting installation error of UAV spacer bars based on visual intelligence as described in claim 1, characterized in that, The optimal detection box is selected from the target detection boxes, and visual quality assessment and structural verification are performed on the optimal detection box to obtain quality assessment parameters, including: Based on the target detection bounding boxes and visual localization confidence of each frame of visual image, the target detection bounding box with the highest visual localization confidence is selected as the optimal detection box; Obtain the pixel area of ​​the optimal detection box, and calculate the ratio of the pixel area of ​​the optimal detection box to the total pixel area of ​​the visual image of the frame in which the optimal detection box is located, as the target space ratio; Edge detection is performed on the image region within the optimal detection box to obtain the average edge intensity. The ratio of the average edge intensity to the preset reference edge intensity is calculated to obtain the texture clarity attenuation. The visual image of the frame containing the optimal detection box is uniformly divided into multiple grid regions. The average brightness of each grid region is obtained, and the ratio of the standard deviation to the average value of the average brightness of each grid region is calculated as the illumination partition contrast value. The image region within the optimal detection box is binarized and its contour is extracted. Based on the extracted contour skeleton, the number of clamps and the frame geometry are identified. The structural integrity matching degree is obtained according to the degree of matching between the number of clamps and the number of clamps in the preset standard, and the degree of matching between the frame geometry and the preset standard frame template.

5. The method for detecting installation error of UAV spacer bars based on visual intelligence as described in claim 1, characterized in that, Based on the positional changes of the target detection box, a temporal consistency confidence score is obtained, including: Obtain the coordinates of the center point of the target detection box corresponding to each frame of visual image; Calculate the inter-frame positioning variance of the center point coordinates; Obtain the preset maximum allowable inter-frame positioning variance, and obtain the timing consistency confidence level based on the ratio of the inter-frame positioning variance to the maximum allowable inter-frame positioning variance.

6. The method for detecting installation error of UAV spacer bars based on visual intelligence as described in claim 1, characterized in that, By integrating the visual location confidence score corresponding to the optimal detection box, the quality assessment parameters, and the temporal consistency confidence score, a comprehensive confidence score is calculated, including: The target space ratio is compared with the preset ideal occupancy ratio range to obtain the occupancy ratio quality factor. The texture clarity attenuation is converted into a texture quality factor, and the illumination partition contrast value is converted into an illumination quality factor. Obtain preset first weight coefficient, second weight coefficient, third weight coefficient, fourth weight coefficient, fifth weight coefficient and sixth weight coefficient, wherein the first weight coefficient corresponds to the visual location confidence of the optimal detection box, the second weight coefficient corresponds to the occupancy ratio quality factor, the third weight coefficient corresponds to the texture quality factor, the fourth weight coefficient corresponds to the illumination quality factor, the fifth weight coefficient corresponds to the structural integrity matching degree, and the sixth weight coefficient corresponds to the temporal consistency confidence. Based on the first weighting coefficient, the second weighting coefficient, the third weighting coefficient, the fourth weighting coefficient, the fifth weighting coefficient, and the sixth weighting coefficient, the visual location confidence, the occupancy ratio quality factor, the texture quality factor, the illumination quality factor, the structural integrity matching degree, and the temporal consistency confidence corresponding to the optimal detection box are weighted and summed to obtain a comprehensive confidence score.

7. The method for detecting installation error of UAV spacer bars based on visual intelligence as described in claim 6, characterized in that, The target space occupancy is compared with a preset ideal occupancy range to obtain an occupancy quality factor. The texture sharpness attenuation is converted into a texture quality factor, and the illumination partition contrast value is converted into an illumination quality factor, including: Obtain a preset ideal occupancy ratio range, which consists of a minimum ideal occupancy ratio and a maximum ideal occupancy ratio; When the target space ratio falls within the preset ideal occupancy ratio range, the occupancy ratio quality factor is set to 1. When the target space occupancy is less than the minimum ideal occupancy ratio, the difference between the minimum ideal occupancy ratio and the target space occupancy is divided by the minimum ideal occupancy ratio, and the resulting ratio is subtracted from 1, which is used as the occupancy ratio quality factor. When the target space occupancy ratio is greater than the maximum ideal occupancy ratio, the difference between the target space occupancy ratio and the maximum ideal occupancy ratio is divided by the maximum ideal occupancy ratio, and the resulting ratio is subtracted from 1, which is used as the occupancy ratio quality factor. Subtract the texture sharpness attenuation from 1 to obtain the texture quality factor; The illumination quality factor is obtained by subtracting the illumination zone contrast value from 1.

8. The method for detecting installation error of UAV spacer bars based on visual intelligence as described in claim 6, characterized in that, Switching between the visual-dominant modality and the tactile-assisted modality based on the overall confidence score includes: Obtain preset high confidence thresholds and low confidence thresholds, and compare the comprehensive confidence score with the high confidence thresholds and low confidence thresholds respectively; When the overall confidence score is greater than or equal to the high confidence threshold, switch to the visual-dominant mode; When the overall confidence score is less than the low confidence threshold, switch to the haptic-assisted mode; When the overall confidence score is greater than or equal to the low confidence threshold and less than the high confidence threshold, the visual dominant modality is maintained and marked as requiring tactile verification.

9. The method for detecting installation error of UAV spacer bars based on visual intelligence as described in claim 1, characterized in that, When switching to the tactile-assisted mode, the spatial deviation is obtained through contact measurement by the drone's robotic arm. This spatial deviation is then processed according to preset rules to obtain the final deviation. Based on this final deviation, it is determined whether there is a positioning error in the spacer's installation position, and the detection result is output, including: Obtain the pixel coordinates of the target contact point corresponding to the optimal detection box, and perform coordinate transformation based on the pixel coordinates to obtain the three-dimensional spatial coordinates of the target contact point; Control the robotic arm to move toward the target contact point. When the end of the robotic arm contacts the spacer, record the three-dimensional spatial coordinates of the actual contact point. Based on the three-dimensional spatial coordinates of the actual contact point and the three-dimensional spatial coordinates of the target contact point, obtain the spatial alignment deviation. Multiple contact measurements were performed at the clamp position of the spacer bar and the center position of the frame to obtain multiple spatial alignment deviations. Calculate the angle between the contact force vector direction at the end of the robotic arm and the normal direction of the preset contact surface at the corresponding key position in each contact measurement. Discard contact measurement results with an angle greater than a preset angle threshold, and retain the remaining contact measurement results as valid measurement results. The spatial alignment deviation corresponding to each valid measurement result is decomposed into a component along the traverse direction and a component perpendicular to the traverse direction. The median of the component along the traverse direction of all valid measurement results is taken as the final deviation. A preset installation error allowable threshold is obtained, and the final deviation is compared with the preset installation error allowable threshold. When the final deviation is less than or equal to the preset installation error allowable threshold, it is determined that there is no positioning error in the installation position of the spacer; otherwise, it is determined that there is a positioning error in the installation position of the spacer. The output includes the final deviation and the judgment result of the detection.

10. A visual intelligence-based UAV spacer installation error detection system, characterized in that, The system is used to implement the visual intelligence-based UAV spacer installation error detection method according to any one of claims 1 to 9, the system comprising: The target detection module is used to acquire multiple frames of visual images of the spacer installation area collected by the UAV, and input the multiple frames of visual images into the pre-trained spacer detection model to obtain the target detection box and visual positioning confidence of each frame of visual images. The quality assessment module is used to select the optimal detection box from the target detection box, perform visual quality assessment and structural verification on the optimal detection box, and obtain quality assessment parameters, wherein the quality assessment parameters include target space ratio, texture clarity attenuation, illumination partition contrast value and structural integrity matching degree. The temporal consistency evaluation module is used to obtain the temporal consistency confidence level based on the positional changes of the target detection box; The comprehensive confidence calculation module is used to integrate the visual location confidence corresponding to the optimal detection box, the quality assessment parameters, and the temporal consistency confidence to calculate the comprehensive confidence score, and switch between the visual dominant mode and the tactile auxiliary mode according to the comprehensive confidence score. The tactile-assisted detection module is used to obtain the spatial deviation through contact measurement by the robotic arm of the UAV when switching to the tactile-assisted mode, process the spatial deviation according to the preset rules to obtain the final deviation, determine whether there is a positioning error in the installation position of the spacer based on the final deviation, and output the detection result.

Images