Auxiliary string falling control method and system for insulator detection robot

Abstract

The invention discloses an auxiliary string falling control method and system of an insulator detection robot. A self-adaptive rotation convolution mechanism is adopted, the convolution kernel direction is dynamically adjusted according to the attitude of a target in the image, stable identification of the insulator string under multi-angle and multi-attitude conditions is realized, and the structural consistency and robustness of target feature extraction are enhanced; a Transform feature fusion and form adaptive mask prediction method based on prompt vector guidance is introduced, and each insulator disc is accurately segmented; in combination with center line fitting and number mapping strategies, space structure mapping is established, and disk-level fault positioning is achieved; and constructing a string falling offset analysis mechanism taking the numbered disk as a reference, sensing the spatial difference between the target hanging point and the current hanging state in real time, and superposing the spatial difference to a flight control interface in a visual manner for guiding string falling action and performing error early warning. According to the invention, disc-level insulator image segmentation is realized, and the accuracy and efficiency of insulator unmanned aerial vehicle operation are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of power equipment fault detection, and in particular relates to an auxiliary string drop control method and system for an insulator detection robot. Background Art

[0002] Insulators, as interconnecting components of overhead transmission lines, are widely used in transmission networks of all voltage levels. Under complex operating conditions, they are subjected to long-term exposure to factors such as power frequency electric fields, mechanical stress, and temperature and humidity fluctuations, leading to gradual degradation of insulation performance and threatening the safe operation of the power grid. With the continued expansion of my country's ultra-high voltage transmission network, traditional inspection methods are no longer able to meet the efficient operation and maintenance requirements of modern power grids.

[0003] Given this situation, intelligent methods are gradually being introduced to fault monitoring of transmission line insulators. Some operational processes attempt to use drones equipped with line-operating robots to perform insulator fault inspections. However, practical applications still present several challenges. First, current drone operations still rely primarily on manual control by ground operators. Highly variable wind speeds at high altitudes, coupled with the complex environment of transmission lines, make precise drone control challenging. Second, existing drones primarily serve as "lifting tools." Specifically, drones equipped with visual inspection devices capture images of transmission lines, detect faulty insulator strings using image processing algorithms, and then employ drone-mounted robots to replace or otherwise repair the faulty insulators.

[0004] Given the unique safety requirements of UHV transmission networks, these issues mean that the efficiency of insulator fault detection using drones in conjunction with robots remains to be improved. Once an insulator failure occurs, a drone must be activated to control the robot to drop down the string for repair. However, when processing transmission line images, the image processing algorithms typically only identify the overall outline of the insulator string and are unable to precisely identify the specific faulty insulator. This inability to accurately and quickly locate the faulty insulator results in inaccurate string drop, which in turn limits the robot's ability to perform precise repairs. Summary of the Invention

[0005] In order to solve the deficiencies in the prior art, the present invention provides an auxiliary string drop control method and system for an insulator detection robot. An auxiliary string drop control method and system for an insulator detection robot. Adopting an adaptive rotational convolution mechanism, the direction of the convolution kernel is dynamically adjusted according to the posture of the target in the image, so as to achieve robust recognition of insulator strings under multiple angles and multiple postures, and enhance the structural consistency and robustness of target feature extraction; introducing a Transformer feature fusion and morphological adaptive mask prediction method based on prompt vector guidance to accurately segment each insulator disk; combining centerline fitting and number mapping strategies to establish spatial structure mapping and achieve disk-level fault location; constructing a string drop offset analysis mechanism with numbered disks as reference, to perceive the spatial difference between the target hanging point and the current mounting state in real time, and visually superimposing it on the flight control interface to guide string drop action and error warning.

[0006] A first aspect of the present invention provides an auxiliary string drop control method for an insulator inspection robot, which adopts the following technical solution:

[0007] An image sequence including insulator strings is collected along the transmission line path, and each frame of the original image in the image sequence is preprocessed to obtain a standard image frame;

[0008] Sequentially extracting initial feature maps from standard image frames, each of which contains multiple target response regions; eliminating image distortion by dynamically adjusting the direction of the convolution kernel, and outputting a rotated feature map;

[0009] Performing target detection on the rotated feature map, and constructing a minimum point distance fusion loss function and a repulsion factor joint mechanism to optimize the target detection loss function, and outputting a candidate bounding box;

[0010] Generate a target perception feature map based on the candidate bounding boxes whose confidence is greater than a threshold, perform structural dynamic adaptive mask segmentation on the target perception feature map, and obtain an independent mask map corresponding to each insulator disk;

[0011] The insulator disks are spatially positioned and numbered according to the independent mask image, and the geometric centers of the disks are associated with the spatial sorting results by combining geometric fitting and visual enhancement.

[0012] Furthermore, the step of eliminating image distortion includes:

[0013] The standard image frames are sequentially input into the basic convolution module of the YOLOv7 detection network front end to extract the initial feature map corresponding to each frame, each of which contains multiple target corresponding areas;

[0014] The routing function is used to predict the optimal convolution kernel rotation angle and combination weight for each target response area:

[0015] θ=α·Softsign(W θ x pool );

[0016] λ=Sigmoid(W λ x pool );

[0017] θ,λ=f(x);

[0018] Among them, α is the scaling factor, θ is the predicted optimal convolution kernel rotation angle; λ is the predicted combination weight, W θ and W λ are the weight matrices of θ and λ respectively; x pool is the initial feature map after global average pooling, Softsign() and Sigmoid() are activation functions;

[0019] Use bilinear interpolation to achieve sum rotation and output the rotation feature map:

[0020] W i '=Rotate(W i θ i )i=1,2,...,n;

[0021] y=λ1(W1'*x)+λ2(W2'*x)+...λ n (W n '*x);

[0022] Where: W i ' is the rotated kernel; Rotate() is the rotation operation; y is the output feature map.

[0023] Furthermore, the optimization step of the target detection loss function includes:

[0024] Using the rotated feature map as input, perform object detection using the backbone module of the YOLOv7 network, and output a set of preliminary candidate bounding boxes, each of which includes its center coordinates, width and height dimensions, and a confidence score;

[0025] Taking the four corner points of the predicted box and the ground-truth box as references, the Euclidean distance between the corresponding corner points is calculated as a measure of the degree of geometric deformation of the bounding box. The degree of geometric deformation of the bounding box is expressed as:

[0026]

[0027] Among them, D MP is the degree of geometric deformation, P i p and P i tThey represent the i-th corner point of the predicted box and the ground-truth box, respectively, and ||·||2 represents the Euclidean distance.

[0028] Furthermore, the IoU overlap between each candidate negative sample frame and the corresponding positive sample frame in the preliminary candidate bounding box is calculated, and the candidate negative sample frame whose overlap exceeds a first threshold and does not meet the positive sample judgment condition is marked as a high-risk sample;

[0029] The positive sample determination condition is that the overlap degree is greater than a second threshold, and the second threshold is greater than the first threshold;

[0030] A penalty term based on exponential decay of overlap is introduced, which is expressed as:

[0031]

[0032] Among them, L RF is the penalty term, which represents the repulsion factor space suppression loss; γ is the exponential growth coefficient, the higher the IoU, the greater the penalty; IoU is the loss function, B i is the prediction box, B pos is the positive sample frame.

[0033] Furthermore, the optimized target detection loss function is expressed as:

[0034] L total =τ1·L MPDIoU +τ2·L RF +τ3·L cls +τ4·L obj

[0035] Among them, L MPDIoU The geometric difference loss of the corner points is used to measure the degree of geometric deformation; L RF is the repulsive factor steric inhibition loss; L cls is the category cross entropy loss; L obj is the target confidence loss; τ is the weight coefficient of each sub-loss in the formula.

[0036] Furthermore, the target perception feature map generation process includes:

[0037] Filter out candidate bounding boxes with confidence greater than the confidence threshold, perform feature concatenation on their spatial position information and confidence scores, and perform feature projection to generate a coding hint vector;

[0038] The encoded hint vector is dimensionally expanded and input into a two-layer Transformer encoder module for fusion, and the internal dependency of the hint features is modeled through the self-attention mechanism;

[0039] The target perception feature map is output through the ReLU nonlinear activation function.

[0040] Furthermore, the process of the structure dynamic adaptive mask segmentation includes:

[0041] The extended dimension prompt vector processed by the two-layer Transformer structure and the self-attention mechanism is used as input, and the convolution kernel weight parameters are generated through multi-layer perceptron network mapping;

[0042] Performing an adaptive convolution operation on the target perception feature map based on the convolution kernel weight parameters;

[0043] A progressive upsampling strategy is used to gradually restore the initial resolution of 64×64 to 256×256, and a residual link is embedded to output an independent mask map corresponding to each insulator disk.

[0044] Furthermore, the spatial positioning and numbering of the insulator discs includes:

[0045] For each independent mask image, the average coordinates of the pixel points in the mask area are calculated to obtain the centroid position of each insulator disk;

[0046] According to the arrangement direction of the insulator strings, all centroid positions are sorted and a number mapping relationship is established.

[0047] Furthermore, auxiliary string dropping is performed by the spatial position of the insulator disc;

[0048] The centroid coordinates arranged in sequence are used as the fitting point set, and the centerline function of the insulator string is obtained by fitting using a fitting algorithm.

[0049] Calculate the offset between the drone's launch center position and the fitted centerline, and overlay the offset as a colored straight line on the drone view interface;

[0050] The number mapping relationship of the insulator disks is combined with the centroid position of each insulator disk, and gradient tone encoding is used to assign a unique color value to each insulator to form a gradient color strip.

[0051] A second aspect of the present invention provides an auxiliary string drop control system for an insulator inspection robot, which runs the auxiliary string drop control method described in the first aspect of the present invention. The system includes:

[0052] An image acquisition module is used to acquire an image sequence containing insulator strings along the transmission line path, and pre-process each frame of the original image in the image sequence to obtain a standard image frame;

[0053] An image distortion removal module is used to sequentially extract initial feature maps from standard image frames, each of which contains multiple target response regions; eliminate image distortion by dynamically adjusting the direction of the convolution kernel, and output a rotated feature map;

[0054] A loss function joint optimization module is used to perform target detection on the rotated feature map, and to optimize the target detection loss function by constructing a minimum point distance fusion loss function and a repulsion factor joint mechanism to output a candidate bounding box;

[0055] A mask generation and segmentation module generates a target perception feature map based on candidate bounding boxes with confidence greater than a threshold, performs structural dynamic adaptive mask segmentation on the target perception feature map, and obtains an independent mask map corresponding to each insulator disk;

[0056] The auxiliary string drop module is used to spatially locate and number the insulator disks according to the independent mask image, and to associate the disk geometric center with the spatial sorting result by combining geometric fitting and visual enhancement.

[0057] The beneficial effect of the present invention is that, compared with the prior art,

[0058] 1. This invention introduces adaptive rotational convolution to address the recognition failure issues caused by changes in the UAV's flight attitude and angular distortion at the transmission site. By dynamically matching the rotational convolution kernel to the target attitude, stable recognition of insulator strings is achieved under multiple angles and attitudes. This improves the structural consistency and recognition robustness of target features in UHV transmission line images, providing a reliable feature foundation for subsequent detection and segmentation.

[0059] 2. This invention employs a hint-vector-guided Transformer fusion and morphologically adaptive mask prediction mechanism to accurately segment and extract the cluster center of each insulator disk. Combining centerline fitting with spatial number mapping, a spatial arrangement model with clear numbering and sequence is constructed, enabling disk-level fault identification and precise repair instruction generation.

[0060] 3. This invention incorporates a string drop offset analysis mechanism using numbered discs as a reference. This mechanism calculates the difference between the target attachment point and the current attachment state in real time, visually overlaying it on the pilot's control interface as colored lines, providing directional guidance and error warnings for string drop. This mechanism helps operators achieve more precise string drop maneuvers. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 A flow chart of the string drop control method of the insulator inspection robot provided by the present invention;

[0062] Figure 2The target recognition part of the present invention is provided as a flowchart;

[0063] Figure 3 The original SAM image segmentation model structure diagram provided by the present invention;

[0064] Figure 4 The workflow diagram of the semantic segmentation part provided by the present invention;

[0065] Figure 5 This is a schematic diagram of the mask-based positioning and automatic labeling process provided by the present invention. DETAILED DESCRIPTION

[0066] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. The embodiments described in this application are only part of the embodiments of the present invention, not all of them. Based on the spirit of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0067] Example 1

[0068] This embodiment describes a specific implementation method of a string drop control method of an insulator detection robot proposed by the present invention. The method flow chart is shown in FIG. Figure 1 The method comprises the following steps:

[0069] Step 1: Collect transmission line images using drones;

[0070] A drone equipped with a visible light camera is used to cruise along the transmission line path, photographing the target area and obtaining a video stream image sequence containing insulator strings; this image serves as the input basis for the subsequent target detection and segmentation process.

[0071] Step 2: Locate the insulator string and eliminate image distortion;

[0072] For each original image frame in the video stream image sequence, standardization preprocessing including normalization, size scaling and illumination equalization is performed to obtain a standard image frame with consistent clarity and contrast;

[0073] The standard image frames are sequentially input into the basic convolutional module of the YOLOv7 detection network front-end to extract the corresponding initial feature map sequence; each initial feature map contains multiple target response regions; in this scheme, the target response region corresponds to the insulator target region that may exist in the original image;

[0074] When a drone is working over live power lines, the strong electromagnetic field surrounding the transmission tower causes the angle of the drone to constantly change during deployment, resulting in multi-angle photography of insulators. This causes lens distortion at different angles, thus affecting the accuracy of target identification.

[0075] Therefore, a rotation-aware convolution structure is embedded in the basic convolution module at the front end of the YOLOv7 detection network. This structure predicts the optimal convolution kernel rotation angle for each target response area through a routing function, and uses a bilinear interpolation method to dynamically rotate the convolution kernel in the basic convolution module so that its direction adapts to the posture changes of the target in the current image.

[0076] The image distortion caused by angle transformation is eliminated by dynamically adjusting the direction of the convolution kernel. The process is as follows:

[0077] For each target response area in the initial feature map of each frame, the routing function is used to predict the optimal convolution kernel rotation angle and the combined weights of convolution kernels in different directions for each target response area. This processing step includes:

[0078] θ=α·Softsign(W θ x pool );

[0079] λ=Sigmoid(W λ x pool );

[0080] θ,λ=f(x);

[0081] Among them, α is the scaling factor, θ is the predicted convolution kernel rotation angle; λ is the predicted combination weight, W θ and W λ are the weight matrices of θ and λ respectively; x pool is the initial feature map after global average pooling, and Softsign() and Sigmoid() are both activation functions.

[0082] Finally, bilinear interpolation is used to realize kernel rotation so that it adapts to the rotation deformation of the target. The calculation is as follows:

[0083] W i '=Rotate(W i θ i )i=1,2,...,n;

[0084] y=λ1(W1'*x)+λ2(W2'*x)+...λ n (W n '*x);

[0085] Where: W i' is the rotated kernel; Rotate() is the rotation operation; y is the output feature map.

[0086] The rotated adaptive convolution kernel is applied to the initial feature map corresponding to the target response area, performing a directionally aligned local convolution operation and outputting a rotated feature map with direction normalization. In this feature map, targets of the same category will exhibit consistent spatial feature distribution under different shooting angles, providing greater angular robustness.

[0087] Ultimately, this rotated feature map serves as a unified structural representation and serves as the input foundation for the subsequent bounding box prediction module. This rotated feature map possesses rotation normalization capabilities, maintaining structural stability and semantic consistency under conditions of multi-angle shooting and object pose variations, providing a unified feature foundation for subsequent bounding box generation and object detection.

[0088] Step 3: Improve the accuracy of the bounding box and the rationality of its spatial distribution;

[0089] Using the rotated feature map output from step 2 as input, the YOLOv7 network backbone module performs object detection and regresses to output a set of preliminary candidate bounding boxes. Each bounding box contains its center coordinates, width and height dimensions, and a confidence score, representing the possible insulator target area in the image.

[0090] To improve the positioning accuracy and spatial distribution quality of the bounding box, the minimum point distance fusion loss function (MPDIoU) and the repulsion factor (RF) are introduced to optimize the bounding box regression loss function of the object detection module. Specifically:

[0091] Taking the four corner points (upper left, upper right, lower right, and lower left) of the predicted box and the ground-truth box as references, we calculate the Euclidean distance difference between the corresponding corner points and use it as a measure of the geometric deformation of the bounding box. This metric is expressed as:

[0092]

[0093] Among them, D MP is the degree of geometric deformation, P i p and P i t They represent the i-th corner point of the predicted box and the ground-truth box, respectively, and ||·||2 represents the Euclidean distance.

[0094] This distance term is introduced as a geometric error term in the total loss function. The closer it is to the true corner distribution, the smaller the loss, so that the regression process focuses on the shape and rotation adaptability of the box.

[0095] To solve the problem of candidate box aliasing, repeated detection, or missed detection caused by high-overlapping negative samples in target detection, a repulsion factor joint mechanism is introduced. Specifically:

[0096] Calculate the IoU overlap between each candidate negative sample frame and its corresponding positive sample frame. If the overlap exceeds the preset threshold (such as IoU>0.5) but does not meet the positive sample judgment condition (such as IoU>0.7), the candidate negative sample frame is marked as a "high-risk sample". At this time, an additional penalty term based on the exponential decay of the overlap is introduced into the total loss function to form a "spatial repulsion field". This penalty term is expressed as:

[0097]

[0098] Among them, L RF is the penalty term, which represents the repulsion factor space suppression loss; γ is the exponential growth coefficient, the higher the IoU, the greater the penalty; IoU is the loss function, B i is the prediction box, B pos is the positive sample frame.

[0099] The RF mechanism in this application forms a "repulsive field" in the loss function, forcing high-overlap negative samples to stay away from the positive sample area, thereby improving the spatial distribution uniformity of the detection box and the target discrimination.

[0100] MPDIoU and RF mechanisms are embedded in the total loss function of target detection in a linearly weighted manner. The structure is as follows:

[0101] L total =τ1·L MPDIoU +τ2·L RF +τ3·L cls +τ4·L obj

[0102] Among them, L MPDIoU To measure the degree of geometric deformation, the corner geometric difference loss is used to compare the Euclidean distance difference between the four corner points of the predicted box and the real box to measure the parameters such as shape, rotation, aspect ratio, etc. RF is the repulsive factor steric inhibition loss; L cls is the category cross entropy loss; L obj is the target confidence loss; τ is the weight coefficient of each sub-loss in the formula, and the hyperparameter is adjusted according to the task requirements.

[0103] Finally, a set of high-quality candidate bounding boxes with high accuracy and uniform distribution is generated. These high-quality candidate bounding boxes will be used as prompt information in the subsequent steps to input the prompt encoder to guide the semantic segmentation module to perform the mask generation task.

[0104] Step 4: Reference Figure 4,Based on the above high-quality candidate bounding boxes, the semantic segmentation module is guided to perform fine mask segmentation on each disk in the insulator string, and output a disk-level target mask map with clear structure and smooth edges;

[0105] Step 4.1: The high-quality candidate bounding boxes obtained in step 3 are screened according to the confidence threshold (derived by the object detection network, such as 0.5), and the low-confidence candidate bounding boxes are eliminated, while the high-confidence candidate bounding boxes are retained as the basis for subsequent processing;

[0106] Step 4.2: For each high-confidence candidate bounding box, its spatial location information (including center coordinates, width and height) is concatenated with the confidence score to generate a preliminary hint feature vector. This hint feature vector is then fed into a set of fully connected layers for feature projection, thereby generating an encoded hint vector with object-aware capabilities.

[0107] Step 4.3: In order to enhance the expressiveness and context-awareness of the encoded hint vector, a two-layer Transformer structure is used to guide its deep fusion with the image backbone features. Specifically:

[0108] Concatenate the encoding proposal vector with the learnable output category label Embedding to expand the prompt vector dimension;

[0109] The extended dimensional hint vector is input into a two-layer Transformer encoder module for fusion, and the internal dependency of the hint features is modeled through the self-attention mechanism. The extended hint vector with enhanced dependency is used as the query vector, and the image embedding feature (i.e., the rotated feature map in step 2) is used as the key-value pair to perform the cross-attention mechanism. The output length is kept consistent with the input hint vector to preserve its structural consistency.

[0110] The optimized target perception feature map is output through the ReLU nonlinear activation function, which has stronger regional indicative and structural perception capabilities.

[0111] Step 4.4: The expanded cue vector processed by the two-layer Transformer structure and the self-attention mechanism is used as input. The target perception cue vector is mapped into a set of learnable convolution kernel parameters through the multi-layer perceptron (MLP) network. That is, a set of learnable convolution kernel weight parameters is generated for each insulator disk. These parameters will be used to construct an adaptive convolution kernel to dynamically adjust the convolution strategy of the subsequent feature decoding process, enabling the subsequent feature decoding module to have morphological adaptability in the spatial dimension.

[0112] Step 4.5: The convolution kernel weight parameters generated in step 4.4 are used to perform an adaptive convolution operation on the target perception feature map. This dynamic convolution parameter adjustment mechanism allows the convolution kernel to automatically adjust to the structural differences of different insulator targets, thereby more accurately extracting the target morphological features and improving the accuracy of single-piece insulator segmentation. The segmentation quality remains stable even when the target undergoes scale changes or rotational perturbations.

[0113] A progressive upsampling strategy is adopted to gradually restore the initial resolution of 64×64 to 256×256 through two transposed convolutional layers, and residual connections are embedded to retain the edge details and high-frequency texture information of the image, achieving smoothness and coherence of the mask boundary.

[0114] Ultimately, this stage outputs an independent mask image corresponding to each insulator disk. Each pixel area in the mask accurately represents the position, shape, and size information of the target disk in the original image, providing high-quality image structure input for subsequent geometric analysis, automatic numbering, and positioning control.

[0115] This step improves the accuracy and robustness of insulator string disk-level segmentation through a dynamic bounding box screening mechanism, cue vector generation, and image feature fusion. Screening high-confidence bounding boxes and generating target-aware cue vectors ensures accurate target location. Using a Transformer architecture and self-attention mechanism to contextually enhance cue vectors optimizes target recognition and semantic consistency during segmentation. Progressive upsampling and residual linking further enhance target segmentation resolution and edge detail preservation. The application of dynamic convolution kernels enables morphological adaptability in the segmentation process, effectively addressing the complex morphologies of insulator disks at varying shooting angles, scales, and occlusion conditions, ensuring the accuracy and visual quality of mask prediction.

[0116] Step 5: Reference Figure 5 , locate and number the insulators;

[0117] In step 4, the mask prediction results generated by disk-level segmentation provide the image region of each insulator, based on which:

[0118] First, calculate the average coordinates of the pixel points in each mask area to obtain its centroid position, which represents the spatial position of each insulator.

[0119] According to the arrangement direction of the insulator string, all the centroid positions are sorted and a number mapping relationship is established. In this way, each insulator is assigned a unique number, ensuring that each target is uniquely identified in space;

[0120] The sequentially arranged centroid coordinates are then used as a set of fitting points. A fitting algorithm (least squares fitting or spline curve) is then used to fit these centroid coordinates to obtain a centerline function representing the insulator string. This fitted centerline characterizes the principal orientation of the insulator string's actual spatial arrangement, providing a spatial reference for subsequent precise positioning, installation, and adjustment.

[0121] Next, the offset between the drone's launch center and the fitted centerline is calculated. This offset represents the spatial error between the drone's current position and the centerline. This offset is overlaid on the drone's view interface as a colored line to help pilots clearly identify the gap between the drone's current position and the target centerline, allowing for fine-tuning or automatic correction.

[0122] Finally, the insulator number mapping relationship is combined with the center of mass position of each insulator, and gradient color encoding is used to assign a unique color value to each insulator, forming a gradient color strip. This helps pilots clearly observe their number and sorting position at a glance.

[0123] Step 6: Assist in controlling the robot to drop the string;

[0124] After completing the segmentation and geometric positioning of the insulator disks, this step further associates the coordinates of the geometric center points of each disk extracted from the segmentation mask with the spatial sorting results to establish a numbered target position sequence to assist the pilot or robot in achieving precise string landing operations.

[0125] The system first calculates the real-time offset based on the center coordinates of the target insulator disk and the geometric reference point of the robot's current hanging point. The offset is displayed in the form of a directional arrow superimposed on the image transmission screen of the flight control remote controller through a visual interface to indicate the pilot's current deviation direction and offset amplitude.

[0126] When the offset exceeds the set threshold, the interface will trigger a prompt warning mechanism, for example: highlighting the current incorrect direction in the image, displaying the words "Misaligned, please fine-tune the attitude", or providing sound / vibration feedback to remind the pilot to pay attention to correction.

[0127] It should be noted that the method of the present invention mainly serves an auxiliary purpose. As an ultra-high voltage live operation, it still requires professionals with relevant knowledge thresholds to perform real-time operations. This method only liberates the operator from most of the content in the flight control, so that the operator can focus on the important node of assisting close-range string dropping.

[0128] Example 2

[0129] This embodiment describes a specific implementation of an auxiliary string drop control system for an insulator inspection robot. The auxiliary string drop control method described in the first embodiment is executed. The system includes:

[0130] An image acquisition module is used to acquire an image sequence containing insulator strings along the transmission line path, and pre-process each frame of the original image in the image sequence to obtain a standard image frame;

[0131] An image distortion removal module is used to sequentially extract initial feature maps from standard image frames, each of which contains multiple target response regions; eliminate image distortion by dynamically adjusting the direction of the convolution kernel, and output a rotated feature map;

[0132] A loss function joint optimization module is used to perform target detection on the rotated feature map, and to optimize the target detection loss function by constructing a minimum point distance fusion loss function and a repulsion factor joint mechanism to output a candidate bounding box;

[0133] A mask generation and segmentation module generates a target perception feature map based on candidate bounding boxes with confidence greater than a threshold, performs structural dynamic adaptive mask segmentation on the target perception feature map, and obtains an independent mask map corresponding to each insulator disk;

[0134] The auxiliary string drop module is used to spatially locate and number the insulator disks according to the independent mask image, and to associate the disk geometric center with the spatial sorting result by combining geometric fitting and visual enhancement.

[0135] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered by the scope of protection of the claims of the present invention.

Claims

1. An auxiliary string drop control method for an insulator inspection robot, characterized in that: include: An image sequence including insulator strings is collected along the transmission line path, and each frame of the original image in the image sequence is preprocessed to obtain a standard image frame; Sequentially extracting initial feature maps from standard image frames, each of the initial feature maps containing multiple target response areas; Eliminate image distortion by dynamically adjusting the direction of the convolution kernel and output a rotated feature map; Performing target detection on the rotated feature map, and constructing a minimum point distance fusion loss function and a repulsion factor joint mechanism to optimize the target detection loss function, and outputting a candidate bounding box; Generate a target perception feature map based on the candidate bounding boxes whose confidence is greater than a threshold, perform structural dynamic adaptive mask segmentation on the target perception feature map, and obtain an independent mask map corresponding to each insulator disk; The insulator disks are spatially positioned and numbered according to the independent mask image, and the geometric centers of the disks are associated with the spatial sorting results by combining geometric fitting and visual enhancement.

2. The auxiliary string drop control method of the insulator inspection robot according to claim 1, characterized in that: The steps to eliminate image distortion include: The standard image frames are sequentially input into the basic convolution module of the YOLOv7 detection network front end to extract the initial feature map corresponding to each frame, each of which contains multiple target corresponding areas; The routing function is used to predict the optimal convolution kernel rotation angle and combination weight for each target response area: θ=α·Softsign(W θ x pool ); λ=Sigmoid(W λ x pool ); θ,λ=f(x); Among them, α is the scaling factor, θ is the predicted optimal convolution kernel rotation angle; λ is the predicted combination weight, W θ and W λ are the weight matrices of θ and λ respectively; x pool is the initial feature map after global average pooling, Softsign() and Sigmoid() are activation functions; Use bilinear interpolation to achieve sum rotation and output the rotation feature map: W′ i =Rotate(W i ;θ i )i=1,2,...,n; y=λ1(W′1*x)+λ2(W′2*x)+...λ n (W′ n *x); Where: W′ i is the rotated kernel; Rotate() is the rotation operation; y is the output feature map.

3. The auxiliary string drop control method of the insulator inspection robot according to claim 1, characterized in that: The optimization steps of the target detection loss function include: Using the rotated feature map as input, perform object detection using the backbone module of the YOLOv7 network, and output a set of preliminary candidate bounding boxes, each of which includes its center coordinates, width and height dimensions, and a confidence score; Taking the four corner points of the predicted box and the ground-truth box as references, the Euclidean distance between the corresponding corner points is calculated as a measure of the degree of geometric deformation of the bounding box. The degree of geometric deformation of the bounding box is expressed as: Among them, D MP is the degree of geometric deformation, and They represent the i-th corner point of the predicted box and the ground-truth box, respectively, and ||·||2 represents the Euclidean distance.

4. The auxiliary string drop control method of the insulator inspection robot according to claim 3, characterized in that: Calculate the IoU overlap between each candidate negative sample frame and the corresponding positive sample frame in the preliminary candidate bounding box, and mark the candidate negative sample frame whose overlap exceeds a first threshold and does not meet the positive sample judgment condition as a high-risk sample; The positive sample determination condition is that the overlap degree is greater than a second threshold, and the second threshold is greater than the first threshold; A penalty term based on exponential decay of overlap is introduced, which is expressed as: Among them, L RF is the penalty term, which represents the repulsion factor space suppression loss; γ is the exponential growth coefficient, the higher the IoU, the greater the penalty; IoU is the loss function, B i is the prediction box, B pos is the positive sample frame.

5. The auxiliary string drop control method of the insulator inspection robot according to claim 3, characterized in that: The optimized target detection loss function is expressed as: L total =τ1·L MPDIoU +τ2·L RF +τ3·L cls +τ4·L obj Among them, L MPDIoU The geometric difference loss of the corner points is used to measure the degree of geometric deformation; L RF is the repulsive factor steric inhibition loss; L cls is the category cross entropy loss; L obj is the target confidence loss; τ is the weight coefficient of each sub-loss in the formula.

6. The auxiliary string drop control method of an insulator inspection robot according to claim 1, characterized in that: The generation process of the target perception feature map includes: Filter out candidate bounding boxes with confidence greater than the confidence threshold, perform feature concatenation on their spatial position information and confidence scores, and perform feature projection to generate a coding hint vector; The encoded hint vector is dimensionally expanded and input into a two-layer Transformer encoder module for fusion, and the internal dependency of the hint features is modeled through the self-attention mechanism; The target perception feature map is output through the ReLU nonlinear activation function.

7. The auxiliary string drop control method of an insulator inspection robot according to claim 1, characterized in that: The process of structured dynamic adaptive mask segmentation includes: The extended dimension prompt vector processed by the two-layer Transformer structure and the self-attention mechanism is used as input, and the convolution kernel weight parameters are generated through multi-layer perceptron network mapping; Performing an adaptive convolution operation on the target perception feature map based on the convolution kernel weight parameters; A progressive upsampling strategy is used to gradually restore the initial resolution of 64×64 to 256×256, and a residual link is embedded to output an independent mask map corresponding to each insulator disk.

8. The auxiliary string drop control method of an insulator inspection robot according to claim 1, characterized in that: The spatial positioning and numbering of the insulator discs includes: For each independent mask image, the average coordinates of the pixel points in the mask area are calculated to obtain the centroid position of each insulator disk; According to the arrangement direction of the insulator strings, all centroid positions are sorted and a number mapping relationship is established.

9. The auxiliary string drop control method of an insulator inspection robot according to claim 1, characterized in that: Auxiliary string dropping is performed through the spatial position of the insulator disc; The centroid coordinates arranged in sequence are used as the fitting point set, and the centerline function of the insulator string is obtained by fitting using a fitting algorithm. Calculate the offset between the drone's launch center position and the fitted centerline, and overlay the offset as a colored straight line on the drone view interface; The number mapping relationship of the insulator disks is combined with the centroid position of each insulator disk, and gradient tone encoding is used to assign a unique color value to each insulator to form a gradient color strip.

10. An auxiliary string drop control system for an insulator inspection robot, running the auxiliary string drop control method according to any one of claims 1 to 9, characterized in that: The system comprises: An image acquisition module is used to acquire an image sequence containing insulator strings along the transmission line path, and pre-process each frame of the original image in the image sequence to obtain a standard image frame; An image distortion removal module is used to sequentially extract initial feature maps from standard image frames, each of which contains multiple target response regions; eliminate image distortion by dynamically adjusting the direction of the convolution kernel, and output a rotated feature map; A loss function joint optimization module is used to perform target detection on the rotated feature map, and to optimize the target detection loss function by constructing a minimum point distance fusion loss function and a repulsion factor joint mechanism to output a candidate bounding box; A mask generation and segmentation module generates a target perception feature map based on candidate bounding boxes with confidence greater than a threshold, performs structural dynamic adaptive mask segmentation on the target perception feature map, and obtains an independent mask map corresponding to each insulator disk; The auxiliary string drop module is used to spatially locate and number the insulator disks according to the independent mask image, and to associate the disk geometric center with the spatial sorting result by combining geometric fitting and visual enhancement.

Images