A multi-arm cooperative control method of a power transmission line maintenance robot

Abstract

The application relates to the technical field of robot control, in particular to a power transmission line maintenance robot multi-mechanical arm cooperative control method, which comprises the following steps: dividing images collected by the robot at each collection time to obtain image blocks, extracting highlight areas in the image blocks, representing overexposure internal and external influences according to edge distribution characteristics in the highlight areas, evaluating distortion difference conditions of the image blocks, and then obtaining interference degree characteristic values; evaluating the texture differentiation degree of each image block at each collection time, analyzing frame-to-frame jump conditions, obtaining vibration interference degree characteristics, calculating disturbance possibility evaluation values, obtaining the unavailability evaluation values of the images, screening out usable images, and controlling the mechanical arm of the power transmission line maintenance robot in combination with a three-dimensional model. The application can effectively select usable images with actual spatial information and high accuracy, and then improve the multi-mechanical arm cooperative control precision of the power transmission line maintenance robot.

Description

Technical Field

[0001] This application relates to the field of robot control technology, specifically to a multi-arm collaborative control method for a power transmission line maintenance robot. Background Technology

[0002] Transmission lines are exposed to complex natural environments for extended periods, inevitably suffering from lightning strikes, icing, wind deflection, and external damage, leading to faults such as loose fittings, conductor damage, and insulator flashover. To ensure the reliability of the power grid, regular inspections and timely maintenance are crucial. Traditional maintenance methods, especially live-line work, typically involve specialized technicians wearing heavy protective suits in high-altitude, strong electromagnetic fields. This is not only labor-intensive and inefficient but also carries significant safety risks of falls and electric shock. However, with the development of line maintenance robots, transmission line maintenance is gradually being replaced by these robots, greatly improving the efficiency of line maintenance.

[0003] Due to the complexity of the maintenance environment and the numerous maintenance tasks and items, multiple robotic arms are typically deployed for collaborative maintenance to improve efficiency. However, in the process of collaborative control of multiple robotic arms, to ensure that they operate without collisions and accurately reach designated positions for their tasks, it is usually necessary to combine various sensory data. For example, a high-precision environmental model can be built using visual and depth information. This 3D model is then used to constrain and plan the movement of the robotic arms, achieving collision avoidance and precise movement. However, in actual operation, the maintenance robot's perception of the environment is affected by various interference sources. For instance, power lines or robotic arms under strong sunlight can cause localized overexposure, or backlighting can result in extremely low contrast, or high current-carrying wires can cause uneven air refractive index due to thermal convection, leading to distortion. All of these can distort visual or depth information, resulting in a decrease in the accuracy of the 3D model and affecting the accuracy of subsequent multi-robotic arm collaborative control. Summary of the Invention

[0004] To address the aforementioned technical problems, this application provides a multi-robotic arm collaborative control method for a power transmission line maintenance robot, thereby resolving the existing issues.

[0005] The multi-robotic arm collaborative control method for a power transmission line maintenance robot proposed in this application adopts the following technical solution: One embodiment of this application provides a multi-robotic arm collaborative control method for a power transmission line maintenance robot, including the following steps: The images acquired by the robot at each acquisition time are divided into image blocks, where each image is divided into a preset number of image blocks; The highlight areas in each image block are extracted. Based on the edge distribution characteristics in the highlight areas, the internal and external effects of overexposure are characterized. The differences in grayscale distribution and edge change trends of image blocks at the same position between frames are analyzed. The distortion differences of each image block are evaluated, and the feature value of the degree of interference is obtained. The degree of texture distortion of each image block at each acquisition time is evaluated, and the difference in edge texture of image blocks at the same position in the inter-frame images is analyzed to characterize the inter-frame jump situation, thereby obtaining the characteristics of the degree of vibration interference. Based on the interference degree feature value and vibration interference degree evaluation value of each image block at each acquisition time, the disturbance probability assessment value of each image block at each acquisition time is obtained, and then the unavailability assessment value of each image at each acquisition time is obtained, so as to filter the available images. The robotic arm of the power transmission line maintenance robot is controlled by combining available images with a 3D model.

[0006] Preferably, pixels with gray values ​​higher than a threshold are designated as highlights, and a region growing algorithm is used to extract the highlight areas in each image block, with each highlight as the initial point. The ratio between the total number of edges in the highlighted area of ​​the image block and the total number of edges in the image block is calculated, and the difference between the value 1 and the ratio is calculated. The smallest bounding rectangle of each highlighted area is taken as the neighboring area of ​​each highlighted area. The mean of the total number of edge pixels in each neighboring area is calculated. The difference and the mean are multiplied by a preset minimum positive number. The result of the multiplication operation is used as a characterization value of the internal and external effects of overexposure.

[0007] Preferably, the absolute difference between the grayscale mean of the image block at the same position at each acquisition time and the previous acquisition time is calculated, and the slope of the pixel at each edge in the image block is calculated. The standard deviation of all slopes in each image block is used as the distortion coefficient of each image block, and the absolute difference between the distortion coefficient of the image block at the same position at each acquisition time and the previous acquisition time is calculated. The ratio of the absolute difference between the distortion coefficients to the absolute difference between the grayscale mean values ​​is used as the evaluation value of the distortion difference of each image block at each acquisition time.

[0008] Preferably, the interference degree characteristic value is positively correlated with the characterization value of the internal and external effects of overexposure and the evaluation value of the distortion difference.

[0009] Preferably, the mean distance between any two edge midpoints in each image block is calculated, the total number of corner points in each image block is detected, and the variance of gray values ​​of all pixels in each image block is calculated, which together form the texture feature vector of each image block. The mean value of the difference vector magnitude between the texture feature vectors of image blocks at the same location at each acquisition time and at multiple previous acquisition times is calculated as an evaluation value of the degree of texture alienation of the image blocks at the same location.

[0010] Preferably, the line segments in each image block are detected, and the ratio of the number of line segments in the image block to the total number of edges is calculated. The absolute difference between the ratio at each acquisition time and the image block at the same position at the previous acquisition time is calculated. Calculate the LBP value of each pixel in each image block, obtain the mean distance between the LBP values ​​of all pixels at the same position in the image block at the previous acquisition time, and use the product of the absolute difference and the mean value as the characterization value of the inter-frame jump situation of the image block at the same position at each acquisition time.

[0011] Preferably, the vibration interference level feature is positively correlated with the characterization value of inter-frame jump and the evaluation value of texture alienation level.

[0012] Preferably, the disturbance probability assessment value is positively correlated with the disturbance degree characteristic value and the vibration disturbance degree evaluation value, respectively.

[0013] Preferably, the average value of the disturbance probability assessment of all image blocks in a single acquisition time image is used as the unavailability assessment value of the single acquisition time image.

[0014] Preferably, if the unavailability assessment value is less than a preset threshold, the corresponding image is used as an available image. Based on multiple available images prior to the current acquisition time, a 3D model of the surrounding environment of the maintenance robot at the current acquisition time is obtained using a 3D model, which is then used to control the robot's robotic arm.

[0015] This application has at least the following beneficial effects: This application analyzes the disturbance level of the image data collected during the operation of the power transmission line maintenance robot, and further combines the vibration interference during the operation of the maintenance robot to correct and compensate for the disturbance of the image data. Ultimately, it achieves accurate evaluation of the quality of the collected image data, improves the accuracy of subsequent 3D modeling of the environment around the maintenance robot, and ensures the accuracy of multi-robotic arm collaborative control. This application addresses the problem that in the current process of using traditional visual perception data for 3D modeling of maintenance robots, visual data is easily disturbed by environmental factors and robot vibrations, leading to distortion of visual or depth information, which in turn affects the accuracy of the 3D model and reduces the accuracy of multi-arm collaborative control. Attached Figure Description

[0016] Figure 1A flowchart illustrating the steps of a multi-robotic arm collaborative control method for a power transmission line maintenance robot provided in this application. Detailed Implementation

[0017] The following description, in conjunction with the accompanying drawings, details a specific scheme for a multi-robotic arm collaborative control method for a power transmission line maintenance robot provided in this application.

[0018] This application provides an embodiment of a multi-robotic arm collaborative control method for a power transmission line maintenance robot. For details, please refer to [link to relevant documentation]. Figure 1 This includes the following steps: Step 1: Divide the images acquired by the robot at each acquisition time into image blocks. Each image is divided into a preset number of image blocks.

[0019] In this embodiment, the visual perception device on the maintenance robot acquires omnidirectional image data of the surrounding environment of the maintenance robot at intervals T, and transmits it back to the control system of the maintenance robot, and divides the image equally into n. For n image blocks, the acquired image data is further converted into grayscale image data using the grayscale averaging method, and all edges in the image are detected using the Canny edge detection algorithm and the dilation erosion algorithm.

[0020] In this embodiment, T is set to 0.5s, and n is set to 15. It should be noted that each acquisition time corresponds to one image, and each image corresponds to n. n image blocks.

[0021] Step 2: Extract the highlight areas from each image block. Based on the edge distribution characteristics in the highlight areas, characterize the internal and external effects of overexposure. Analyze the differences in grayscale distribution and edge change trends of image blocks at the same position between frames, evaluate the distortion differences of each image block, and then obtain the feature value of the degree of interference.

[0022] During the operation of the maintenance robot, located in an open, high-altitude area with little shade, it experiences high levels of sunlight during the day. The robot and its wiring typically have corresponding metal and insulating shells, which are prone to strong reflections under intense light. This can lead to localized overexposure in the acquired image data, distorting details within these areas and affecting the accuracy of subsequent 3D reconstruction. Furthermore, short-term changes in ambient light (such as temporary cloud cover causing a significant decrease followed by an increase in light intensity) can cause the visual sensing equipment to lose focus during image acquisition, resulting in image distortion and increased blurring of details, further impacting modeling accuracy. Therefore, it is necessary to analyze the interference affecting the images beforehand.

[0023] Specifically, regarding overexposure interference caused by ambient light, the degree of interference to the image typically varies depending on the location and extent of the overexposure. For example, if the overexposure occurs on a small area of ​​the robot's outer shell or a component, the strong reflection results in a small overexposure area with minimal impact on overall detail. This usually manifests as a small highlight in a localized image area, with relatively intact details outside the highlighted area and some residual texture within the highlighted area. However, in cases of large-area overexposure, not only is most of the detail lost in the overexposed area, but some detail loss also occurs around it. This is mainly characterized by large areas of highlight in a localized region, with very little or no texture within the highlighted area, and localized loss of texture edges in the surrounding area, leaving only scattered short edges. Furthermore, for image distortion and blurring caused by focus issues, the main characteristic is increased distortion of texture edges in the localized area, but with minimal overall color change, and the degree of distortion and blurring varies across multiple frames.

[0024] Based on the above analysis, the characteristics of interference in local areas can be represented by the characteristics of overexposure and internal and external influences, as well as the characteristics of distortion difference. That is, the more severe the loss of internal details in the highlight position of a single image block at a single acquisition time, the less detail in the external neighboring area. In the case of multiple frames, the difference in texture and color is small, but the difference in distortion is large, indicating that there is more likely to be environmental interference in the current image block.

[0025] In this embodiment, to characterize the internal and external effects of overexposure, for the image data acquired at the i-th acquisition time, histogram binarization is used to obtain the grayscale value segmentation threshold, and this threshold is used as the highlight segmentation threshold K (it should be noted that, to prevent the threshold from being too low when there is no overexposure in the image, when the segmentation threshold is less than or equal to 230, then 230 is used). Taking a single image block at the i-th acquisition time as an example, all pixels in the image block with grayscale values ​​greater than or equal to the threshold are taken as highlights. Using a region growing algorithm, each highlight is taken as the initial point, and the pixels in the eight-neighborhood of the initial point are taken as the target points. The growth condition is that the absolute difference between the grayscale values ​​of the initial point and the target point is less than or equal to 2. The stopping condition is that there are no points in the eight-neighborhood of the initial point that meet the growth condition. The output is multiple growing regions, and each growing region is taken as the highlight area of ​​the image block.

[0026] Furthermore, the ratio between the total number of edges in the highlighted areas of the image patch and the total number of edges in the image patch is calculated. To avoid a zero denominator during the ratio calculation, a very small positive number is added to the denominator in this embodiment. In this embodiment, the value of this very small positive number is 0.00001. The implementer can also set its own value to prevent calculation errors caused by a zero denominator. The difference between the value 1 and the ratio result is calculated. The smallest bounding rectangle of each highlighted area is taken as the neighboring region of each highlighted area. The average number of edge pixels in each neighboring region is calculated. The difference is multiplied by the sum of the above difference and the above average, plus the preset very small positive number (in this embodiment, the very small positive number is 0.001 to avoid a zero product). The result of this product operation is used as a characterization value for the internal and external effects of overexposure. The larger the characterization value for the internal and external effects of overexposure, the more severe the loss of texture in the evaluated highlighted area, requiring a higher unusability rating.

[0027] To characterize the distortion difference, this embodiment calculates the absolute difference between the mean grayscale values ​​of image blocks at the same position at the i-th acquisition time and the previous acquisition time. Further, the slope at each pixel on each edge of the image block is calculated. To prevent the slope of pixels at local edges from being positive infinity, the maximum slope value at each edge pixel is first removed. Finally, the standard deviation of all slopes within a single image block is used as the distortion coefficient of that image block. The absolute difference between the distortion coefficients of image blocks at the same position at the i-th acquisition time and the previous acquisition time is calculated. The ratio of the absolute difference between the distortion coefficients to the absolute difference between the mean grayscale values ​​is used as the evaluation value of the distortion difference of the j-th image block at the i-th acquisition time. It should be noted that when the absolute difference of the mean grayscale values ​​is 0, the absolute difference of the distortion coefficients is directly used as the evaluation value of the distortion difference of the j-th image block at the i-th acquisition time.

[0028] Therefore, by combining the characterization values ​​of the internal and external effects of overexposure and the evaluation values ​​of distortion differences, the interference degree feature values ​​of each image block are obtained. These interference degree feature values ​​are positively correlated with both the characterization values ​​of the internal and external effects of overexposure and the evaluation values ​​of distortion differences.

[0029] It should be noted that the positive correlation described in this embodiment is used to characterize the same trend of change between variables, where one variable increases (decreases) as the other variable increases (decreases). The specific calculation implementer can choose according to the actual application scenario, and no special limitation is made in this embodiment.

[0030] Preferably, in this embodiment, the calculation formula for the interference level feature value of the j-th image block at the i-th acquisition time is as follows: In the formula, Let be the feature value of the interference level of the j-th image patch at the i-th acquisition time. Let be the characterization value representing the internal and external effects of overexposure on the j-th image block at the i-th acquisition time. The feature evaluation value of the distortion difference of the j-th image block at the i-th acquisition time.

[0031] in, This feature is used to characterize the severity of environmental interference affecting the j-th image patch at the i-th acquisition time. This is used to reflect the changes in the inside and outside of the overexposed area of ​​the j-th image block under overexposure interference at the i-th acquisition time. It is reflected by analyzing the edge ratio inside the highlight area and the edge length in the adjacent area of ​​the highlight area. It is similar to the calculation principle of texture retention and detail loss in existing characterization images, which is reflected by analyzing the edge ratio and edge length in the local image. It is used to reflect the difference in image distortion between the j-th image block at the i-th acquisition time and the corresponding image block at the previous acquisition time. It is analyzed and calculated by the difference in edge distortion and gray value between adjacent frames.

[0032] It should be noted that the more severe the loss of details in the highlighted areas of a single image block at a single acquisition time, the less detail there is in the surrounding areas. In multi-frame states, the difference in texture and color is small, but the difference in distortion is large, indicating that there is more likely to be environmental interference in the current image block.

[0033] Step 3: Evaluate the degree of texture distortion of each image block at each acquisition time, and characterize the inter-frame jump by analyzing the edge texture differences of image blocks at the same position in the inter-frame images, thereby obtaining the characteristics of the degree of vibration interference.

[0034] In the actual operation of the maintenance robot, in addition to the interference of environmental changes on the acquired image data, the robot itself will experience slight vibrations due to the acceleration and deceleration of the joint motors or motion motors when moving or working on the power transmission line. This will cause the visual sensing equipment to jitter during the acquisition process, resulting in a certain degree of jitter in the acquired image data. Depending on the degree of jitter, some areas or even the entire image data will be blurred or drifted, leading to deviations in the actual spatial relationships represented by the image data. Therefore, further analysis is needed.

[0035] Specifically, the mechanical vibration of the inspection robot will cause different interferences to the acquired images depending on the degree of vibration. In the case of relatively slight vibration, the overall image is less affected, mainly the edge blurring of local areas in the image increases, the edge density decreases, and the gray-scale gradient and corner number change significantly. As the vibration intensity increases, linear trailing will appear in the image, the image blurring will further increase, and finally the content between frames will jump, that is, the texture distribution between adjacent frames will be very different.

[0036] Based on the above analysis, vibration interference characteristics can be represented by texture distortion characteristics and inter-frame jump characteristics. That is, the greater the change in edge density, gray value variance and corner number in a single image block between a single acquisition time and the previous acquisition time, the more obvious the linear trailing, and the greater the texture jump between adjacent frames, the more likely the current image block is to have vibration interference.

[0037] In this embodiment, in order to characterize the texture heterogeneity features, taking a single image block at the i-th acquisition time as an example, the robot's current motion pose data is further obtained. Based on the above-mentioned reference information, the historical images at the m acquisition times before the i-th acquisition time (the value is 10 in this embodiment) are registered and aligned in the spatial coordinate system. The specific process is existing technology and will not be described in detail in this embodiment. Furthermore, the feature vectors of image blocks at the same location are calculated. Specifically, the distance between any two edge midpoints in a single image block is calculated. In this embodiment, Euclidean distance is used. In this embodiment, the Harris corner detection algorithm is used to obtain the corners in the image block and count their total number. Then, the variance of the gray values ​​of all pixels in the image block is calculated. Since the gray value variance is significantly different from the Euclidean distance and the number of corners, the gray value variance of all image blocks in the image at a single acquisition time is normalized using min-max normalization to normalize it to between 0 and 1. Correspondingly, the Euclidean distance and the total number of corners are also subjected to min-max normalization. Finally, the mean of the normalized Euclidean distance, the total number of corners, and the variance of the pixel gray values ​​are used to construct the texture feature vector of the image block at the i-th acquisition time. Similarly, the texture feature vector of each image block at each acquisition time is constructed.

[0038] Furthermore, the magnitude of the difference vector between the image texture feature vectors of the image patch at the same location at the i-th acquisition time and those at the previous m acquisition times is calculated. Finally, the mean of this magnitude is used as an evaluation value to characterize the degree of texture alienation of the j-th image patch at the i-th acquisition time. The larger the mean of the above magnitude, the more discrete the edge distribution in the current image patch is, the relatively low density, the decrease in gray-level gradient in the image, and the significant change in the number of corner points, which is more likely to be an image change caused by the vibration of the maintenance robot.

[0039] Meanwhile, in order to characterize the inter-frame jump features, taking a single image block at the i-th acquisition time as an example, the LSD (Line Segment Detection) algorithm is first used to obtain all line segments in the image block, and the ratio of the number of line segments in the image block to the total number of edges is calculated. The absolute difference between the above ratio at the i-th acquisition time and the image block at the same position at the previous acquisition time is calculated. Further, preferably, in this embodiment, to facilitate calculation, pixels in a single image block are randomly sampled to obtain g representative pixels (in this embodiment, g is 50; however, the implementer can also use all pixels in the image block for calculation, and this embodiment does not impose any special restrictions). The LBP value of these representative pixels is calculated using the LBP algorithm. Simultaneously, pixels in the same position in the image block at the same location at the i-th acquisition time are obtained, and their corresponding LBP values ​​are calculated. The average distance between the LBP values ​​of each representative pixel in the same position in the image block at the i-th acquisition time and the previous acquisition time is calculated. Preferably, the dtw distance is used in this embodiment. Then, the product of the absolute difference and the average dtw distance is used as the characterization value of the inter-frame jump situation of a single image block at the i-th acquisition time. The larger the characterization value of the inter-frame jump situation, the more severe the linear trailing situation in the current image, and the more obvious the feature difference of the overall image between adjacent frames. This is more likely to be due to the excessive vibration of the maintenance robot causing the image feature change.

[0040] In summary, based on the characterization values ​​of the inter-frame jump situation and the evaluation values ​​of the texture distortion degree of each image patch, the vibration interference degree feature is obtained. The vibration interference degree feature is positively correlated with both the characterization value of the inter-frame jump situation and the evaluation value of the texture distortion degree.

[0041] Preferably, in this embodiment, the characteristic representing the degree of vibration interference to each image block at a single acquisition moment can be calculated using the following formula: In the formula, Let be the evaluation value of the degree of vibration interference of the j-th image block at the i-th acquisition time. It is an evaluation value used to characterize the degree of texture alienation of the j-th image patch at the i-th acquisition time. This represents the inter-frame jump condition of the j-th image block at the i-th acquisition time. It should be noted that, in other implementations, the implementer can also use the sum of the inter-frame jump condition representation value and the texture distortion assessment value to obtain the vibration interference level evaluation value.

[0042] In the above formula, This represents the image texture change characteristics of the j-th image block at the i-th acquisition time when it is subjected to vibration disturbance. The first part of the relation is... This method reflects the degree of texture alienation by analyzing the differences in edge density, grayscale variance, and number of corner points within the same image block at different acquisition times. This is similar to the existing calculation principle for representing the degree of texture feature alienation of the same image block at different acquisition times, which is reflected by analyzing the differences in edge, grayscale, and corner point conditions within the image block.

[0043] The second part of the above formula is , represents the inter-frame transition characteristics of the j-th image block at the i-th acquisition time and the adjacent acquisition time. It is reflected by analyzing the differences in the proportion of straight line segments and the differences in LBP values ​​of multiple pixels between different frames. It is similar to the existing calculation principle for representing the differences in image features between different frames, which is reflected by analyzing the proportion of straight lines and the differences in local features between different frames.

[0044] Among them, the greater the change in edge density, gray value variance, and number of corner points in a single image block between a single acquisition time and the previous acquisition time, the more obvious the linear trailing, and the greater the texture jump between adjacent frames, the more likely the current image block is to have vibration interference.

[0045] Step 4: Based on the interference level feature value and vibration interference level evaluation value of each image block at each acquisition time, obtain the disturbance probability assessment value of each image block at each acquisition time, and then obtain the unavailability assessment value of each image at each acquisition time, so as to filter the available images.

[0046] Based on the interference degree feature value and vibration interference degree evaluation value of each image block at each acquisition time, the disturbance probability assessment value of each image block at each acquisition time is obtained. The disturbance probability assessment value is positively correlated with the interference degree feature value and the vibration interference degree evaluation value, respectively.

[0047] Preferably, in this embodiment, the formula for obtaining the disturbance probability assessment value of the j-th image block at the i-th acquisition time is as follows: In the formula, Let be the disturbance probability assessment value of the j-th image block at the i-th acquisition time. Let be the feature value of the interference level of the j-th image patch at the i-th acquisition time. Let be the evaluation value of the degree of vibration interference of the j-th image block at the i-th acquisition time. , These are the weighting coefficients for the characteristic value of the degree of interference and the evaluation value of the degree of vibration interference, respectively, and the sum of the two is always 1. In this embodiment, they are both taken as 0.5.

[0048] It is understandable that the higher the degree of interference from the environment and vibration in the j-th image block at the i-th acquisition time, the worse the image quality of the current image block is, the lower its ability to represent the information features within the image, and the less likely it is to be used for subsequent 3D model construction.

[0049] At this point, the probability assessment value of disturbance for each image block at a single acquisition time can be calculated. Further, historical image data from multiple acquisition times is acquired, and the probability assessment value of disturbance for each image block in all historical acquisition time image data is calculated. This value is then used to perform max-min normalization on the currently calculated probability assessment value, normalizing it to between 0 and 1. Furthermore, the average probability assessment value of disturbance for all image blocks in an image at a single acquisition time is used as the unavailability assessment value for that image.

[0050] Furthermore, each captured image frame is evaluated. When the unavailability evaluation value of the captured image at a single acquisition moment is greater than or equal to a preset threshold U (0.9 in this embodiment), it indicates that the overall image quality of the current image is poor, the actual spatial information that can be represented is less and less accurate, and therefore the image frame is marked as an unusable image. Conversely, if the unavailability evaluation value is less than or equal to a preset threshold U, it is considered that the overall image quality of the current image is good, the actual spatial information that can be represented is more and more accurate, and it should be used in subsequent 3D model construction, and the image frame is marked as a usable image.

[0051] Step 5: Use available images and 3D models to control the robotic arm of the power transmission line maintenance robot.

[0052] Furthermore, a three-dimensional model of the environment surrounding the maintenance robot is constructed using the available images obtained through screening. In this embodiment, all available images from the 10 acquisition times prior to the current acquisition time are input into the three-dimensional model, and the output is a three-dimensional model of the environment surrounding the maintenance robot at the current acquisition time. Based on the currently output three-dimensional model, the robot's robotic arm is controlled. The specific robotic arm control can adopt existing control methods, and no special restrictions are placed on this in this embodiment.

[0053] Preferably, this embodiment includes: based on the constructed three-dimensional model, obtaining the current position information and model parameters of the robotic arm in the three-dimensional model, performing simulation in the three-dimensional model space, and generating the working path of each robotic arm; when the working path of the robotic arm coincides with the working path of other robotic arms at the same time point or the distance between them is less than a preset safety threshold (0.05m in this embodiment), it is determined that the corresponding robotic arm has an interference risk.

[0054] Furthermore, in this embodiment, the position information and model parameters of the robotic arm with interference risk are adjusted according to the preset priority of the robotic arm (for example, the movement speed of the robotic arm with higher priority is increased and iteratively adjusted in a loop, and the specific adjustment is made according to the robotic arm control method in the prior art, and this embodiment does not impose any special restrictions on this), so as to determine the position information and model parameters of the robotic arm without interference risk.

[0055] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A multi-arm collaborative control method for a power transmission line maintenance robot, characterized in that, Includes the following steps: The images acquired by the robot at each acquisition time are divided into image blocks, where each image is divided into a preset number of image blocks; The highlight areas in each image block are extracted. Based on the edge distribution characteristics in the highlight areas, the internal and external effects of overexposure are characterized. The differences in grayscale distribution and edge change trends of image blocks at the same position between frames are analyzed. The distortion differences of each image block are evaluated, and the feature value of the degree of interference is obtained. The degree of texture distortion of each image block at each acquisition time is evaluated, and the difference in edge texture of image blocks at the same position in the inter-frame images is analyzed to characterize the inter-frame jump situation, thereby obtaining the characteristics of the degree of vibration interference. Based on the interference degree feature value and vibration interference degree evaluation value of each image block at each acquisition time, the disturbance probability assessment value of each image block at each acquisition time is obtained, and then the unavailability assessment value of each image at each acquisition time is obtained, so as to filter the available images. The robotic arm of the power transmission line maintenance robot is controlled by combining available images with a 3D model.

2. The multi-arm collaborative control method for a power transmission line maintenance robot as described in claim 1, characterized in that, Pixels with gray values ​​higher than a threshold are designated as highlights, and a region growing algorithm is used to extract the highlight areas in each image block, starting from each highlight. The ratio between the total number of edges in the highlighted area of ​​the image block and the total number of edges in the image block is calculated, and the difference between the value 1 and the ratio is calculated. The smallest bounding rectangle of each highlighted area is taken as the neighboring area of ​​each highlighted area. The mean of the total number of edge pixels in each neighboring area is calculated. The difference and the mean are multiplied by a preset minimum positive number. The result of the multiplication operation is used as a characterization value of the internal and external effects of overexposure.

3. The multi-arm collaborative control method for a power transmission line maintenance robot as described in claim 2, characterized in that, Calculate the absolute difference between the mean grayscale values ​​of image blocks at the same position at each acquisition time and the previous acquisition time, calculate the slope of pixels on each edge of the image block, and use the standard deviation of all slopes in each image block as the distortion coefficient of each image block. Calculate the absolute difference between the distortion coefficients of image blocks at the same position at each acquisition time and the previous acquisition time. The ratio of the absolute difference between the distortion coefficients to the absolute difference between the grayscale mean values ​​is used as the evaluation value of the distortion difference of each image block at each acquisition time.

4. The multi-arm collaborative control method for a power transmission line maintenance robot as described in claim 3, characterized in that, The interference level characteristic values ​​are positively correlated with the characterization values ​​of the internal and external effects of overexposure and the evaluation values ​​of distortion differences.

5. The multi-arm collaborative control method for a power transmission line maintenance robot as described in claim 1, characterized in that, The mean distance between any two edge midpoints in each image patch is calculated, the total number of corner points in each image patch is detected, and the variance of the gray values ​​of all pixels in each image patch is calculated. Together, these are used to form the texture feature vector of each image patch. The mean value of the difference vector magnitude between the texture feature vectors of image blocks at the same location at each acquisition time and at multiple previous acquisition times is calculated as an evaluation value of the degree of texture alienation of the image blocks at the same location.

6. The multi-arm collaborative control method for a power transmission line maintenance robot as described in claim 5, characterized in that, Detect straight line segments in each image block and calculate the ratio of the number of straight line segments to the total number of edges in the image block. Calculate the absolute difference between the ratio at each acquisition time and the image block at the same position at the previous acquisition time. Calculate the LBP value of each pixel in each image block, obtain the mean distance between the LBP values ​​of all pixels at the same position in the image block at the previous acquisition time, and use the product of the absolute difference and the mean value as the characterization value of the inter-frame jump situation of the image block at the same position at each acquisition time.

7. The multi-arm collaborative control method for a power transmission line maintenance robot as described in claim 6, characterized in that, The vibration interference level characteristics are positively correlated with the characterization value of inter-frame jump and the evaluation value of texture alienation.

8. The multi-arm collaborative control method for a power transmission line maintenance robot as described in claim 1, characterized in that, The disturbance probability assessment value is positively correlated with the disturbance degree characteristic value and the vibration disturbance degree evaluation value, respectively.

9. The multi-arm collaborative control method for a power transmission line maintenance robot as described in claim 1, characterized in that, The average value of the disturbance probability assessment of all image blocks in a single acquisition time image is used as the unavailability assessment value of the single acquisition time image.

10. The multi-arm collaborative control method for a power transmission line maintenance robot as described in claim 9, characterized in that, If the unavailability assessment value is less than the preset threshold, the corresponding image is used as an available image. Based on multiple available images before the current acquisition time, a 3D model of the surrounding environment of the maintenance robot at the current acquisition time is obtained using a 3D model, which is used to control the robot's robotic arm.

Images