Unmanned aerial vehicle mechanical arm grabbing control method and system for line inspection

Abstract

The present application relates to the technical field of unmanned aerial vehicle control and mechanical arm servo control, in particular to a method and system for unmanned aerial vehicle mechanical arm grabbing control for line inspection, which effectively eliminates the body shaking interference caused by wind load and restricts the target jump from the time sequence by constructing the theoretical reference pixel coordinates based on the relative offset condition of adjacent sampling time and the historical reference pixel coordinates and determining the mutation distance parameter; the discrete distribution span feature corresponding to the morphological divergence parameter in the direction of the linear profile of the power transmission line is calculated, and the vertical distance parameter of the center pixel point from the linear profile of the power transmission line is calculated, so as to accurately separate the discrete distribution of the insulator highlight reflection point from the real hanging line foreign matter from the space; the interference evaluation index is determined by comprehensively considering the texture confidence and the calculated parameters, and the optimal pixel points are screened out for grabbing control, so that the accuracy of the unmanned aerial vehicle mechanical arm for guiding the abnormal target grabbing control is higher.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) control and robotic arm servo control technology, specifically to a UAV robotic arm grasping control method and system for line inspection. Background Technology

[0002] In the operation and maintenance of overhead power transmission lines, removing foreign objects such as kite strings or plastic films hanging from the conductors is a routine task. Currently, multi-rotor drones equipped with multi-degree-of-freedom robotic arms are commonly used to perform this type of foreign object removal. Existing technology typically utilizes a monocular vision sensor onboard the drone to acquire images of the site, and then uses a target detection network to directly extract the two-dimensional pixel coordinates of the foreign object in the image. Based on these original coordinates, control commands are calculated to guide the drone's servo mechanism and robotic arm to approach and grasp the object.

[0003] However, in actual close-range inspection scenarios, drones are highly susceptible to wind loads, causing fuselage vibrations that result in relative translation of the camera image. Furthermore, power line insulators are prone to producing bright reflective spots under sunlight. Current technologies do not consider the overall image shift interference caused by fuselage vibrations, nor do they account for the tendency of monocular target detection networks to mistake localized reflections from insulators for foreign objects hanging on the line, leading to abrupt coordinate jumps between the actual object and adjacent insulators. Because it is impossible to effectively separate the false visual signals generated by the superposition of complex lighting and fuselage vibrations from spatial morphology and temporal motion patterns, the underlying servo mechanism interprets these as a sudden, large-span displacement of the target. This results in low accuracy for the robotic arm to grasp objects by directly relying on the raw coordinates output by the target detection network. Summary of the Invention

[0004] To address the low accuracy issue in existing technologies that rely directly on the raw coordinates output by target detection networks to guide robotic arm grasping, this invention aims to provide a robotic arm grasping control method and system for line inspection, the specific technical solution of which is as follows:

[0005] The first aspect of this invention provides a method for controlling the grasping of a robotic arm of a drone for line inspection, comprising:

[0006] During the process of UAV inspection of power transmission lines, the line sampling image and the corresponding historical reference pixel coordinates are acquired at each sampling time; target detection is performed on the line sampling image to determine the texture confidence of each bounding box and the corresponding center pixel.

[0007] Based on the relative offset of the line sampling images between adjacent sampling times and the historical reference pixel coordinates, the theoretical reference pixel coordinates for each sampling time are determined; and the corresponding abrupt change distance parameter is determined according to the positional deviation of each center pixel from the theoretical reference pixel coordinates.

[0008] Based on the vertical offset of each center pixel relative to the straight profile of the transmission line in the line sampling image, the corresponding vertical distance parameter for detachment from suspension is determined; based on the spatial discrete distribution span characteristics of each center pixel in the direction of the straight profile of the transmission line, the morphological divergence parameter of each center pixel is determined.

[0009] By combining the texture confidence, the mutation distance parameter, the vertical distance from the detachment from the suspension, and the morphological divergence parameter, an interference evaluation index is determined for each central pixel; preferred pixels are selected based on the interference evaluation index; and the UAV robotic arm is used for grasping control based on the preferred pixels and the corresponding interference evaluation index.

[0010] Furthermore, the process of obtaining the theoretical reference pixel coordinates includes:

[0011] The system acquires the horizontal and vertical focal length constants of the UAV camera, as well as the yaw and pitch angular velocities of the UAV between each sampling time and the previous sampling time. Based on the horizontal focal length constant, the yaw angular velocity, and the sampling period, it calculates the distance to determine the horizontal pixel offset. Based on the vertical focal length constant, the pitch angular velocity, and the sampling period, it calculates the distance to determine the vertical pixel offset. Finally, it combines the horizontal and vertical pixel offsets to construct a pixel translation vector.

[0012] The historical reference pixel coordinates are vector-added with the pixel translation vector to determine the theoretical reference pixel coordinates at each sampling time.

[0013] Furthermore, the process of obtaining the mutation distance parameter includes:

[0014] The dynamic spatial scale length for each sampling moment is determined based on the diagonal pixel length of the bounding box in the line sampling image of the previous sampling moment where the historical reference pixel coordinates at each sampling moment are located.

[0015] Based on the Euclidean distance between each center pixel and the coordinates of the theoretical reference pixel, the corresponding absolute deviation distance is determined.

[0016] The absolute deviation distance is standardized based on the length of the dynamic spatial scale to determine the mutation distance parameter.

[0017] Furthermore, the process of obtaining the vertical distance parameter for detachment from suspension includes:

[0018] Perform Hough line detection on the sampled image of the line, and extend the longest detected Hough line to both ends of the image to determine the straight line of the transmission line;

[0019] By drawing a perpendicular line from each center pixel to the straight line of the transmission line, the orthogonal projection point of each center pixel is determined; based on the Euclidean distance between each center pixel and the corresponding orthogonal projection point, the corresponding absolute vertical distance is determined.

[0020] The absolute vertical distance is standardized based on the length of the dynamic spatial scale to determine the vertical distance parameter of each center pixel that is detached from the air.

[0021] Furthermore, the process of obtaining the morphological divergence parameter includes:

[0022] Cluster analysis is performed on all orthogonal projection points to determine all orthogonal projection point clusters; the Euclidean distance between the two farthest orthogonal projection points in each orthogonal projection point cluster is calculated to determine the projection distance range; the absolute discrete span is determined based on the projection distance range of the orthogonal projection point cluster to which the orthogonal projection point corresponding to each center pixel is located.

[0023] The absolute discrete span is standardized based on the length of the dynamic spatial scale to determine the morphological divergence parameter of each center pixel.

[0024] Furthermore, the process of obtaining the interference assessment indicators includes:

[0025] The feature outlier penalty score is determined by weighted summation of the mutation distance parameter, the vertical distance from the detachment from the suspension, and the morphological divergence parameter.

[0026] A positive correlation mapping is performed on the texture confidence scores to determine the confidence reward score;

[0027] Based on the deviation between the feature outlier penalty score and the credibility reward score, an interference evaluation index is determined for each center pixel.

[0028] Furthermore, the process of selecting preferred pixels based on the interference evaluation index includes:

[0029] At each sampling time, the center pixel corresponding to the smallest interference evaluation index is selected as the preferred pixel.

[0030] Furthermore, the process of controlling the UAV robotic arm to grasp based on the preferred pixels and the corresponding interference evaluation index includes:

[0031] When the interference evaluation index of the preferred pixel is greater than the preset loss judgment threshold, it is determined that the grasped target has been lost and the movement of the robotic arm is stopped.

[0032] When the interference evaluation index of the preferred pixel is less than or equal to the preset loss determination threshold:

[0033] The center pixel of the bounding box with the highest texture confidence is taken as the native decision pixel; the final execution reference point is determined based on the spatial position deviation between the preferred pixel and the native decision pixel.

[0034] The pixel offsets between the final execution reference point and the center point of the line sampling image are calculated in the horizontal and vertical directions, and servo compensation commands are generated; the servo compensation commands are used to control the UAV robotic arm to grasp the object.

[0035] Furthermore, the process of obtaining the servo compensation command includes:

[0036] The horizontal pixel deviation value is determined based on the difference in horizontal coordinates between the pixel coordinates of the center point of the line sampling image and the pixel coordinates of the final execution reference point; the target yaw rate at each sampling moment is determined based on the product of the horizontal pixel deviation value and the preset proportional gain coefficient.

[0037] The vertical pixel deviation value is determined based on the difference in vertical coordinate values ​​between the pixel coordinates of the center point of the line sampling image and the pixel coordinates of the final execution reference point; the target pitch angular velocity at each sampling moment is determined based on the product of the vertical pixel deviation value and the preset proportional gain coefficient.

[0038] Servo compensation commands are generated based on the target yaw rate and the target pitch rate.

[0039] Secondly, the present invention provides a drone robotic arm grasping control system for line inspection, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements any of the steps of a drone robotic arm grasping control method for line inspection.

[0040] The present invention has the following beneficial effects:

[0041] This invention constructs theoretical reference pixel coordinates based on the relative offset between adjacent sampling times and historical baseline pixel coordinates, and determines abrupt change distance parameters. This effectively eliminates interference from airframe sway caused by wind loads and constrains the instantaneous large-span jumps of the target in terms of time sequence. By calculating the vertical distance parameter of the center pixel relative to the straight profile of the power transmission line, and the morphological divergence parameter corresponding to the spatial discrete distribution span characteristics in the direction of the straight profile of the power transmission line, the discretely arranged bright reflective points of the insulators are accurately separated from the real hanging foreign objects from the physical spatial topology. Furthermore, by comprehensively considering the texture confidence and the above-mentioned multi-dimensional distance and divergence parameters, interference evaluation indicators are determined, and preferred pixels are selected for grasping control. This blocks the transmission of false visual signals caused by complex lighting and airframe sway, making the accuracy of guiding the UAV robotic arm to grasp abnormal targets higher. Attached Figure Description

[0042] Figure 1 This is a flowchart of a drone robotic arm grasping control method for line inspection, provided as an embodiment of the present invention. Detailed Implementation

[0043] The following description, in conjunction with the accompanying drawings, details a specific solution for a drone robotic arm grasping and control method and system for line inspection provided by the present invention.

[0044] This invention provides a method for controlling the grasping action of a drone robotic arm for line inspection. Please refer to [link / reference]. Figure 1 The diagram illustrates a flowchart of a UAV robotic arm grasping control method for line inspection according to an embodiment of the present invention. The method includes:

[0045] Step S101: During the process of UAV inspection of power transmission lines, acquire the line sampling image at each sampling time and the corresponding historical reference pixel coordinates; perform target detection on the line sampling image, and determine the texture confidence of each bounding box and the corresponding center pixel.

[0046] Before conducting power transmission line inspections and capturing and controlling abnormal targets, real-time working environment data needs to be acquired through relevant hardware deployed on the drone. Specifically, a monocular camera is rigidly mounted on the fuselage or end of the robotic arm of the multi-rotor drone, and an inertial measurement unit is rigidly fixed inside or around the monocular camera's casing. An edge computing motherboard for data processing is also installed inside the drone's fuselage. When the multi-rotor drone flies to the work area of ​​the overhead power transmission line to be inspected and enters a close-range hovering alignment state, the monocular camera continuously captures images of the power transmission line ahead at a fixed sampling frequency, determining the line sampling image at each sampling moment, and transmitting it in real time to the edge computing motherboard via the internal data bus. During this synchronization process, the inertial measurement unit senses the minute vibrations generated by the camera body under environmental wind loads in real time, continuously measures and outputs the three-axis relative rotational angular velocity of the UAV between each sampling moment and the previous sampling moment, and specifically extracts the yaw angular velocity that causes horizontal displacement of the image and the pitch angular velocity that causes vertical displacement of the image, providing underlying physical sensing data support for subsequent anti-shake offset compensation; the sampling frequency in this embodiment of the invention is set to 20Hz, that is, the sampling period is 0.05s, which can be adjusted according to the specific implementation environment.

[0047] After acquiring real-time line sampling images, the edge computing motherboard calls its internally pre-deployed object detection neural network to perform feature extraction and forward inference calculations on the current line sampling image. Since transmission line insulators are prone to producing bright reflective points under sunlight, interfering with vision, the object detection neural network identifies multiple suspected objects hanging on the line in the image and outputs a two-dimensional bounding box for each suspected object. Subsequently, the edge computing motherboard calculates the average of the maximum and minimum horizontal and vertical coordinates occupied by each bounding box, thereby accurately determining the center pixel of each bounding box. Simultaneously, the classification module within the object detection neural network compares the extracted local texture features with the real object features pre-trained in the model. After mapping by the activation function at the network's end, it directly outputs a probability value representing the degree of recognition confidence. The edge computing motherboard directly reads this value and uses it as the texture confidence score of the corresponding bounding box. In a specific implementation of this invention, the object detection neural network uses the highly real-time YOLOv5 model, which can be adjusted according to the specific implementation environment.

[0048] Furthermore, during the initial startup or target loss period when no bounding box is detected, the edge computing motherboard temporarily suspends position calculation. When the target detection neural network detects a valid bounding box for the first time after the edge computing motherboard stops performing position calculation, due to the lack of reference from the previous moment, the edge computing motherboard sets the abrupt change distance parameter of all center pixels to 0, sets the dynamic space scale length to the diagonal length of the detected maximum valid bounding box, and combines the calculated texture confidence, detachment vertical distance parameter, and morphological divergence parameter to derive the interference evaluation index for each center pixel when it first detects a valid bounding box. Subsequently, the coordinates of the center pixel corresponding to the minimum interference evaluation index are selected as the output and stored as the historical reference pixel coordinates for the next sampling moment. In subsequent consecutive sampling moments, since the historical reference has been established, the edge computing motherboard will calculate the complete interference evaluation index in conjunction with the abrupt change distance parameter, and roll over and store the final execution reference point calculated at the current sampling moment as the historical reference pixel coordinates for the next sampling moment. At the same time, it extracts and retains the diagonal pixel length of the bounding box corresponding to the final execution reference point and uses it as the historical reference for determining the dynamic space scale length for the next sampling moment. It should be noted that all coordinates in the embodiments of the present invention belong to the pixel coordinates in the image coordinate system of the line sampling image. The specific origin position and coordinate axis division method can adopt the conventional setting method in the field, and are not further limited here.

[0049] Step S102: Based on the relative offset of the line sampling images between adjacent sampling times and the historical reference pixel coordinates, determine the theoretical reference pixel coordinates for each sampling time; determine the corresponding abrupt change distance parameter based on the positional deviation between each center pixel and the theoretical reference pixel coordinates.

[0050] Considering that drones are prone to fuselage vibration due to wind loads, resulting in overall relative translation of the line sampling images, this invention aims to eliminate fuselage sway errors and construct an absolutely stable observation origin. Based on the relative offset of the line sampling images between adjacent sampling times and historical baseline pixel coordinates, the theoretical reference pixel coordinates for each sampling time are determined. Furthermore, based on the principle that the actual trajectory of a hanging object in physical space should exhibit smooth and continuous temporal characteristics, the corresponding abrupt change distance parameter is determined according to the positional deviation of each central pixel from the theoretical reference pixel coordinates. This allows for accurate measurement of the severity of the instantaneous shift of each central pixel from the expected stationary origin, effectively eliminating calculation distortion caused by fuselage sway and accurately identifying false coordinate jumps caused by high-brightness reflection interference from a temporal perspective.

[0051] Preferably, in some possible implementations of the embodiments of the present invention, the process of obtaining the theoretical reference pixel coordinates includes:

[0052] The process involves acquiring the horizontal and vertical focal length constants of the UAV camera, as well as the yaw and pitch angular velocities of the UAV between each sampling time and the previous sampling time. Distance calculations are performed based on the horizontal focal length constant, yaw angular velocity, and sampling period to determine the horizontal pixel offset. Similarly, distance calculations are performed based on the vertical focal length constant, pitch angular velocity, and sampling period to determine the vertical pixel offset. Finally, the horizontal and vertical pixel offsets are combined to construct a pixel translation vector. This is based on the principle of telecentric imaging geometric approximation mapping, using the integral of the product of focal length and relative rotation angular velocity in the horizontal and vertical directions to determine the two-dimensional pixel translation vector, excluding depth parameter limitations. The historical reference pixel coordinates are then vector-added with the pixel translation vector to determine the theoretical reference pixel coordinates for each sampling time. These theoretical reference pixel coordinates represent the expected absolutely stationary imaging position of the target on the image sensor in the current period, after deducting compensation for the aircraft's spin oscillation displacement, providing a vibration-free spatiotemporal coordinate origin for subsequent measurement of abnormal displacement.

[0053] It should be noted that the specific methods for obtaining the horizontal focal length constant and the vertical focal length constant include: directly obtaining the lens physical focal length from the camera's manufacturer's manual, dividing the physical focal length by the horizontal physical width and vertical physical height of a single pixel on the image sensor, and the quotient values ​​are the corresponding horizontal focal length constant and vertical focal length constant.

[0054] Preferably, in some possible implementations of the embodiments of the present invention, the process of obtaining the mutation distance parameter includes:

[0055] The dynamic spatial scale length for each sampling moment is determined based on the diagonal pixel length of the bounding box in the line sampling image of the previous sampling moment, corresponding to the historical reference pixel coordinates at each sampling moment. The corresponding absolute deviation distance is determined based on the Euclidean distance between each center pixel and the theoretical reference pixel coordinates. A larger absolute deviation distance indicates a greater deviation of the center pixel from the expected stationary origin, increasing the likelihood of abnormal large jumps and the probability of it being a false target caused by high-brightness noise. Considering that the continuous shortening of the line-of-sight during the UAV's approach to the target can cause perspective magnification of the pixel size of hanging foreign objects in the image, a single absolute pixel deviation cannot serve as a unified evaluation standard. Therefore, to counteract the geometric perspective dilation effect, a dynamic spatial scale length is introduced, and the absolute deviation distance is standardized based on the dynamic spatial scale length to determine the abrupt change distance parameter. This enables the system to adaptively eliminate dimensional distortion caused by camera approach motion, ensuring consistency in distance evaluation standards between sampling moments. In this embodiment, the abrupt change distance parameter is determined based on the ratio between the absolute deviation distance and the dynamic spatial scale length, such that a larger abrupt change distance parameter increases the probability of it being a false target caused by high-brightness noise. Furthermore, it should be noted that if the target detection neural network detects only one bounding box in the current line sampling image (i.e., only one center pixel is extracted), then the center pixel is directly taken as the preferred pixel, and the subsequent multi-dimensional interference evaluation and comparison analysis process for multiple center pixels is stopped, so as to avoid the clustering analysis operation crash or redundant consumption of computing resources due to a single sample.

[0056] Step S103: Based on the vertical offset of each center pixel relative to the straight profile of the transmission line in the line sampling image, determine the corresponding vertical distance parameter for detachment from suspension; based on the spatial discrete distribution span characteristics of each center pixel in the direction of the straight profile of the transmission line, determine the morphological divergence parameter of each center pixel.

[0057] Since real foreign objects (such as kite strings or plastic films) are physically attached to or wrapped around the power transmission line, and line components such as insulators are also installed along the power transmission line, the center pixels of the real targets are close to the area where the straight outline of the power transmission line is located in the image space. Therefore, this embodiment of the invention determines the corresponding vertical distance parameter of detachment from suspension based on the vertical offset of each center pixel relative to the straight outline of the power transmission line in the line sampling image; thereby effectively identifying and eliminating false bright background features that are detached from the support of the line body in the topological geometry dimension.

[0058] Preferably, in some possible implementations of the embodiments of the present invention, the process of obtaining the vertical distance parameter of detachment from suspension includes:

[0059] Transmission lines are typically the longest linear contour features in line sampling images. Therefore, Hough line detection is performed on the line sampling images, and the longest detected Hough line is extended to both ends of the image to determine the transmission line line. A perpendicular line is drawn from each center pixel to the transmission line line to determine the orthogonal projection point of each center pixel. Based on the Euclidean distance between each center pixel and its corresponding orthogonal projection point, the corresponding absolute vertical distance is determined. The absolute vertical distance directly reflects the physical distance of the suspected target from the transmission line entity. The smaller the absolute vertical distance, the more it conforms to the physical topological constraint that a real foreign object hanging on the line must be closely attached to the line body, and the lower the probability that it is a suspended speck or background reflection.

[0060] Similarly, a dynamic spatial scale length is introduced here as a metric reference. The absolute vertical distance is standardized based on the dynamic spatial scale length to determine the vertical distance parameter of each center pixel. In this embodiment of the invention, the vertical distance parameter is determined based on the ratio between the absolute vertical distance and the dynamic spatial scale length. This results in a larger vertical distance parameter, which increases the probability of the target being a false target caused by high-brightness noise.

[0061] Furthermore, considering that power insulators are component structures composed of multiple sheds connected in series and arranged at equal intervals, the multiple strong reflective points generated under sunlight exposure will inevitably exhibit a discrete and discontinuous linear distribution pattern along the transmission line direction. Based on this, it can be deduced that real foreign objects hanging on the line (such as kite strings or agricultural mulch film) are usually wrapped and compressed into a clump, appearing as tightly clustered dots or interwoven clusters along the transmission line direction. Therefore, this embodiment of the invention further determines the morphological divergence parameter of each central pixel based on the spatial discrete distribution span characteristics of each central pixel along the straight outline direction of the transmission line; thereby transforming the physical structure divergence difference of the target into a quantifiable feature parameter. Under highly similar local texture appearances, the scattered insulator reflective array and the real foreign object clumps are accurately separated from the aggregation tightness of the one-dimensional topological morphology.

[0062] Preferably, in some possible implementations of the embodiments of the present invention, the process of obtaining the morphological divergence parameter includes:

[0063] Cluster analysis is performed on all orthogonal projection points to determine all orthogonal projection point clusters. The Euclidean distance between the two farthest orthogonal projection points in each cluster is calculated to determine the projection distance range. Based on the projection distance range of the orthogonal projection point cluster to which the orthogonal projection point corresponding to each center pixel belongs, the absolute discrete span is determined. The smaller the absolute discrete span, the more it conforms to the objective morphology of real foreign objects clustering and intertwining, and the lower the probability that it belongs to a false target due to the scattered state of the insulator structure. It should be noted that if the number of orthogonal projection points in the corresponding orthogonal projection point cluster is 1, the corresponding projection distance range is directly set to 0 to avoid calculation problems.

[0064] In one specific implementation of this invention, the method for clustering all orthogonal projection points employs the mean-shift clustering algorithm. The bandwidth parameter (Bandwidth), which determines the search radius of the kernel function, is not a fixed constant but is set as a dynamic adaptive parameter. Specifically, it is obtained by multiplying the dynamic spatial scale length at the current sampling time by a preset bandwidth ratio coefficient (empirically set to 1.5), thus adaptively covering the radiated range of insulator high-gloss reflections at different proximity distances. It should be noted that mean-shift clustering is a technique well-known to those skilled in the art and will not be further elaborated upon here.

[0065] Similarly, the dynamic spatial scale length is also introduced here as a measurement reference. The absolute discrete span is standardized based on the dynamic spatial scale length to determine the morphological divergence parameter of each center pixel. In this embodiment of the invention, the morphological divergence parameter is determined based on the ratio between the absolute discrete span and the dynamic spatial scale length. This makes the probability of a false target caused by high brightness noise greater when the morphological divergence parameter is larger.

[0066] Step S104: Combine texture confidence, mutation distance parameter, vertical distance from suspension parameter, and morphological divergence parameter to determine the interference evaluation index for each center pixel; select preferred pixels based on the interference evaluation index; and perform UAV robotic arm grasping control based on the preferred pixels and the corresponding interference evaluation index.

[0067] Texture confidence is the recognition confidence probability output by the target detection neural network based on local visual features. Therefore, the texture confidence of real hanging foreign objects is usually high. The mutation distance parameter, the vertical distance parameter of detachment from suspension, and the morphological divergence parameter are all positively correlated with the probability that the target belongs to false noise such as high brightness reflection. Therefore, by further combining texture confidence, mutation distance parameter, vertical distance parameter of detachment from suspension, and morphological divergence parameter, the interference evaluation index of each center pixel is determined. Thus, on the basis of local texture features, a multi-dimensional mutual balance between temporal motion and physical spatial topology is introduced, making the final evaluation result more robust and reliable.

[0068] Preferably, in some possible implementations of the embodiments of the present invention, the process of obtaining the interference evaluation index includes:

[0069] The mutation distance parameter, the vertical distance from the detachment from the overhang parameter, and the morphological divergence parameter are weighted and summed to determine the feature outlier penalty score; a positive correlation mapping is applied to the texture confidence score to determine the credibility reward score. In a specific implementation of this invention, the weights of the mutation distance parameter, the vertical distance from the detachment from the overhang parameter, and the morphological divergence parameter are all set to 0.3 for weighted summation. The product of the texture confidence score and the preset confidence weight is used as the corresponding credibility reward score; the preset confidence weight is set to 0.1. That is, in this embodiment of the invention, the weights of the mutation distance parameter, the vertical distance from the detachment from the overhang parameter, and the morphological divergence parameter are all the same, and the sum of the weights of the four parameters is 1. This weight allocation method ensures that the penalty factor for physical structure and temporal continuity can have an overwhelming suppressive advantage against the local visual blind confidence of the neural network. It can be adjusted according to the frequency of occurrence and suppression priority of different interference sources in the specific implementation environment, and will not be further limited or elaborated here.

[0070] Finally, based on the offsetting relationship between punishment and reward, and based on the deviation between the feature outlier penalty score and the credibility reward score, the interference evaluation index of each central pixel is determined. In this embodiment of the invention, the credibility reward score is subtracted from the feature outlier penalty score to determine the interference evaluation index. The larger the interference evaluation index, the deeper the interference of the central pixel is caused by ambient light jumps and loose structural noise, and the higher the probability that it is a false visual signal. Conversely, the smaller the index, the more likely it is to be a real and reliable grasping target.

[0071] Furthermore, at each sampling moment, the center pixel corresponding to the smallest interference evaluation index is selected as the preferred pixel. The smallest interference evaluation index indicates that the center pixel is least affected by the combined interference of complex lighting and body vibration, and best matches the physical continuity and structural form of the real foreign object. This provides the most reliable correction benchmark for the generation of subsequent servo commands. The embodiments of the present invention further perform UAV robotic arm grasping control based on the preferred pixel and the corresponding interference evaluation index.

[0072] Preferably, in some possible implementations of the embodiments of the present invention, the process of performing UAV robotic arm grasping control based on preferred pixels and corresponding interference evaluation indicators includes:

[0073] When the interference evaluation index of the preferred pixel exceeds the preset loss judgment threshold, it is determined that the grasped target has detached, and the robotic arm stops moving. This indicates that even the best candidate target in the current image has exceeded the safety threshold in terms of overall deterioration, and it is highly likely that the image is completely contaminated by reflections or the target has been completely lost. Forcibly cutting off the control link at this time can effectively prevent the robotic arm from performing dangerous blind swinging actions.

[0074] When the interference evaluation index of the preferred pixel is less than or equal to the preset loss judgment threshold:

[0075] The center pixel of the bounding box with the highest texture confidence is used as the native judgment pixel. Based on the spatial positional deviation between the preferred pixel and the native judgment pixel, the final execution reference point is determined. Specifically, the trust assessment distance is determined based on the Euclidean distance between the preferred pixel and the native judgment pixel. The trust assessment distance is standardized based on the dynamic spatial scale length to determine the trust assessment parameter. In this embodiment, the ratio between the trust assessment distance and the dynamic spatial scale length is used as the corresponding trust assessment parameter. The dynamic spatial scale length is introduced to counteract the perspective magnification effect, allowing the trust assessment parameter to objectively reflect the severity of the relative deviation between two pixels at a uniform scale. If the trust assessment parameter is greater than a preset jump tolerance threshold, the preferred pixel is used as the final execution reference point; if the trust assessment parameter is less than or equal to the preset jump tolerance threshold, the native judgment pixel is used as the final execution reference point. When the deviation is within the tolerance range, it indicates that the original output of the network has not suffered from severe high light pollution, and the original judgment pixels are directly allowed to maintain the smoothness of tracking; however, once the deviation exceeds the limit, it indicates that the network has been captured by the reflective umbrella skirt and a vicious change has occurred. The system triggers a forced interception and replaces the execution with the preferred pixels that have undergone multi-dimensional screening, thereby actively resolving the servo risk induced by the erroneous indication points.

[0076] The pixel offsets between the final execution reference point and the center point of the line sampling image are calculated in the horizontal and vertical directions to generate servo compensation commands. Specifically: the horizontal pixel deviation value is determined based on the difference in horizontal coordinates between the center point of the line sampling image and the final execution reference point; the target yaw rate at each sampling moment is determined based on the product of the horizontal pixel deviation value and a preset proportional gain coefficient; the vertical pixel deviation value is determined based on the difference in vertical coordinates between the center point of the line sampling image and the final execution reference point; and the target pitch rate at each sampling moment is determined based on the product of the vertical pixel deviation value and a preset proportional gain coefficient. Here, the classic image-based visual servo control law (IBVS) proportional drive model is adopted, directly mapping the horizontal and vertical pixel offsets of the image plane linearly to the spatial angular velocity drive of the multi-rotor aircraft and the robotic arm, avoiding complex multibody dynamics inverse kinematics calculations.

[0077] Servo compensation commands are generated based on the target's yaw and pitch angular velocities, and these commands are used to control the UAV's robotic arm for grasping. It should be noted that, to prevent extreme large-scale instantaneous movement of the target in the frame that could cause angular velocity overload (such as motor burnout or aircraft instability and crash), the edge computing motherboard determines whether the absolute values ​​of the target's yaw and pitch angular velocities exceed a preset mechanical safety angular velocity threshold (set to 45 degrees / second in this embodiment, but adjustable based on the specific implementation environment). If they exceed this threshold, the exceeding angular velocities are forcibly clamped back to the upper limit. This linear deviation amplification and safety clamping control method ensures that the final correction action is both gentle and timely, as well as absolutely stable and safe.

[0078] In one specific implementation of this invention, the preferred range of the preset loss judgment threshold is set to [0.5, 0.7]. It can be adjusted according to the specific implementation environment. When the light reflection in the working environment is extremely strong and the background features are more complex (easily producing a large number of highlight artifacts), the preset loss judgment threshold should be larger. In this embodiment of the invention, it is set to 0.6. The preset proportional gain coefficient is obtained by controlling the drone to hover and recording the fastest convergence angular velocity response curve corresponding to different pixel deviations in a standard indoor windless laboratory environment, and fitting the slope of the linear interval by the least squares method. The unit of the preset proportional gain coefficient is degrees per second per pixel, and its specific engineering value example can be set to 0.05.

[0079] In summary, a method for controlling the grasping of a drone robotic arm for power line inspection effectively eliminates interference from wind-induced swaying and constrains the instantaneous large-span jumps of targets by constructing theoretical reference pixel coordinates based on the relative offset between adjacent sampling times and historical baseline pixel coordinates, and determining abrupt change distance parameters. By calculating the vertical distance parameter of the center pixel relative to the straight profile of the power line and the morphological divergence parameter corresponding to the spatial discrete distribution span characteristics in the direction of the straight profile of the power line, the method accurately separates the discretely arranged bright reflective points of insulators from real hanging foreign objects from the physical spatial topology. Furthermore, by comprehensively considering texture confidence and the above-mentioned multi-dimensional distance and divergence parameters, interference evaluation indicators are determined, and preferred pixels are selected for grasping control. This blocks the transmission of false visual signals caused by complex lighting and superposition of body sway, resulting in higher accuracy in guiding the drone robotic arm to grasp abnormal targets.

[0080] The present invention also provides a drone robotic arm grasping control system for line inspection, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements any of the steps of a drone robotic arm grasping control method for line inspection.

Claims

1. A method for controlling the grasping of a robotic arm by a drone for line inspection, characterized in that, The method includes: During the process of UAV inspection of power transmission lines, the line sampling image and the corresponding historical reference pixel coordinates are acquired at each sampling time; target detection is performed on the line sampling image to determine the texture confidence of each bounding box and the corresponding center pixel. Based on the relative offset of the line sampling images between adjacent sampling times and the historical reference pixel coordinates, the theoretical reference pixel coordinates for each sampling time are determined; and the corresponding abrupt change distance parameter is determined according to the positional deviation of each center pixel from the theoretical reference pixel coordinates. Based on the vertical offset of each center pixel relative to the straight profile of the transmission line in the line sampling image, the corresponding vertical distance parameter for detachment from suspension is determined; based on the spatial discrete distribution span characteristics of each center pixel in the direction of the straight profile of the transmission line, the morphological divergence parameter of each center pixel is determined. By combining the texture confidence, the mutation distance parameter, the vertical distance from suspension, and the morphological divergence parameter, an interference evaluation index for each center pixel is determined; preferred pixels are selected based on the interference evaluation index; and the drone robotic arm grasping control is performed based on the preferred pixels and the corresponding interference evaluation index. The process of obtaining the vertical distance parameter for detachment from suspension includes: The dynamic spatial scale length for each sampling moment is determined based on the diagonal pixel length of the bounding box in the line sampling image of the previous sampling moment where the historical reference pixel coordinates at each sampling moment are located. Perform Hough line detection on the sampled image of the line, and extend the longest detected Hough line to both ends of the image to determine the straight line of the transmission line; By drawing a perpendicular line from each center pixel to the straight line of the transmission line, the orthogonal projection point of each center pixel is determined; based on the Euclidean distance between each center pixel and the corresponding orthogonal projection point, the corresponding absolute vertical distance is determined. The absolute vertical distance is standardized based on the length of the dynamic spatial scale to determine the detachment vertical distance parameter of each center pixel. The process of obtaining the morphological divergence parameter includes: Cluster analysis is performed on all orthogonal projection points to determine all orthogonal projection point clusters; the Euclidean distance between the two farthest orthogonal projection points in each orthogonal projection point cluster is calculated to determine the projection distance range; the absolute discrete span is determined based on the projection distance range of the orthogonal projection point cluster to which the orthogonal projection point corresponding to each center pixel is located. The absolute discrete span is standardized based on the length of the dynamic spatial scale to determine the morphological divergence parameter of each center pixel.

2. The method for controlling the grasping of a drone robotic arm for line inspection according to claim 1, characterized in that, The process of obtaining the theoretical reference pixel coordinates includes: The system acquires the horizontal and vertical focal length constants of the UAV camera, as well as the yaw and pitch angular velocities of the UAV between each sampling time and the previous sampling time. Based on the horizontal focal length constant, the yaw angular velocity, and the sampling period, it calculates the distance to determine the horizontal pixel offset. Based on the vertical focal length constant, the pitch angular velocity, and the sampling period, it calculates the distance to determine the vertical pixel offset. Finally, it combines the horizontal and vertical pixel offsets to construct a pixel translation vector. The historical reference pixel coordinates are vector-added with the pixel translation vector to determine the theoretical reference pixel coordinates at each sampling time.

3. The method for controlling the grasping of a drone robotic arm for line inspection according to claim 1, characterized in that, The process of obtaining the mutation distance parameter includes: Based on the Euclidean distance between each center pixel and the coordinates of the theoretical reference pixel, the corresponding absolute deviation distance is determined. The absolute deviation distance is standardized based on the length of the dynamic spatial scale to determine the mutation distance parameter.

4. The method for controlling the grasping of a drone robotic arm for line inspection according to claim 1, characterized in that, The process of obtaining the interference assessment indicators includes: The feature outlier penalty score is determined by weighted summation of the mutation distance parameter, the vertical distance from the detachment from the suspension, and the morphological divergence parameter. A positive correlation mapping is performed on the texture confidence scores to determine the confidence reward score; Based on the deviation between the feature outlier penalty score and the credibility reward score, an interference evaluation index is determined for each center pixel.

5. The method for controlling the grasping of a drone robotic arm for line inspection according to claim 3, characterized in that, The process of selecting preferred pixels based on the interference evaluation index includes: At each sampling time, the center pixel corresponding to the smallest interference evaluation index is selected as the preferred pixel.

6. A method for controlling the grasping of a drone robotic arm for line inspection according to claim 5, characterized in that, The process of controlling the UAV robotic arm to grasp based on the preferred pixels and the corresponding interference evaluation index includes: When the interference evaluation index of the preferred pixel is greater than the preset loss determination threshold, it is determined that the grasped target has been detached and the movement of the robotic arm is stopped. When the interference evaluation index of the preferred pixel is less than or equal to the preset loss determination threshold: The center pixel of the bounding box with the highest texture confidence is taken as the native decision pixel; the final execution reference point is determined based on the spatial position deviation between the preferred pixel and the native decision pixel. The pixel offsets between the final execution reference point and the center point of the line sampling image are calculated in the horizontal and vertical directions, and servo compensation commands are generated; the servo compensation commands are used to control the UAV robotic arm to grasp the object.

7. A method for controlling the grasping of a drone robotic arm for line inspection according to claim 6, characterized in that, The process of obtaining the servo compensation command includes: The horizontal pixel deviation value is determined based on the difference in horizontal coordinates between the pixel coordinates of the center point of the line sampling image and the pixel coordinates of the final execution reference point; the target yaw rate at each sampling moment is determined based on the product of the horizontal pixel deviation value and the preset proportional gain coefficient. The vertical pixel deviation value is determined based on the difference in vertical coordinate values ​​between the pixel coordinates of the center point of the line sampling image and the pixel coordinates of the final execution reference point; the target pitch angular velocity at each sampling moment is determined based on the product of the vertical pixel deviation value and the preset proportional gain coefficient. Servo compensation commands are generated based on the target yaw rate and the target pitch rate.

8. A robotic arm grasping control system for line inspection, the system comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of a UAV robotic arm grasping control method for line inspection as described in any one of claims 1 to 7.

Images