Wire recognition and broken wire control method and system based on multi-sensor information fusion

Abstract

The application discloses a kind of based on multi-sensor information fusion's wire identification and broken wire control method and system, it is related to transmission line identification and broken wire control technical field.The method includes: the two-dimensional image data and three-dimensional point cloud data of operation environment are collected;Multi-sensor fusion model based on space-time registration is constructed, the three-dimensional point cloud data is mapped to the two-dimensional image data, and the three-dimensional space pose of target wire is solved;The end trajectory of mechanical arm is planned based on the three-dimensional space pose, controls cutting execution mechanism to align and clamps target wire;Real-time monitoring the electromechanical state parameter of cutting execution mechanism in cutting process, calculates cutting impedance based on state observer model, according to the change characteristics of cutting impedance, identifies wire fracture critical point, and executes adaptive broken wire control strategy.The application is used to solve the problem of low wire identification accuracy and mechanical arm rebound instability in high tension broken wire instant under complex operation background.

Description

Technical Field

[0001] This invention relates to the field of power transmission line identification and disconnection control technology, and more specifically, to a method and system for conductor identification and disconnection control based on multi-sensor information fusion. Background Technology

[0002] To ensure power supply reliability and reduce the risk of electric shock to workers, the use of high-voltage live-line working robots to replace manual labor in high-risk operations such as cutting power lines at heights and connecting lead wires has become an inevitable trend in the industry. These robots are typically mounted on aerial work platform trucks or autonomously walk along power lines. They use hydraulic shearing tools at the end of their robotic arms to physically cut high-tensile power lines that have been in service for many years. They are key equipment for achieving uninterrupted maintenance and automated upgrades of power transmission lines.

[0003] In existing technological systems, the identification and control of wires in robotic robots mainly rely on a single vision sensor or simple position control logic. Specifically, at the perception level, images are acquired using a visible light camera, and image processing algorithms such as edge detection or Hough transform are used to extract straight line features to locate the wire. At the motion level, the robotic arm typically employs teach-and-playback or position-based visual servo control. Once aligned with the wire, it controls the hydraulic shear actuator to close the blade at a preset speed or maximum power until the wire is completely cut.

[0004] However, the aforementioned existing technologies have significant drawbacks when dealing with the complex and unstructured operating environments of aging power lines. First, aging power lines often cross tree barriers, buildings, or intersect with other power lines. Single vision sensors are highly susceptible to interference from changes in lighting and background textures (such as tree branches and building edges), resulting in low conductor recognition rates or even false or missed grasps. Furthermore, the lack of precise depth information makes it difficult for the robotic arm to achieve high-precision alignment. Second, existing wire breakage control methods are mostly open-loop or constant-speed shearing, unable to sense the stress state of the conductor. Since aging conductors are usually under high tension, they accumulate enormous elastic potential energy within the milliseconds of being instantly cut. Once broken, this can easily trigger a violent "whipping effect" or rebound, which may not only cause instability of the robotic arm and damage to precision sensors, but in severe cases, may even lead to safety accidents such as the broken wire striking towers or nearby live conductors. Summary of the Invention

[0005] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide a wire identification and wire breakage control method based on multi-sensor information fusion, which solves the problems of low wire identification accuracy and mechanical arm rebound instability at the moment of high-tension wire breakage under complex operation backgrounds.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A wire identification and breakage control method based on multi-sensor information fusion includes the following steps: acquiring two-dimensional image data and three-dimensional point cloud data of the working environment; mapping the three-dimensional point cloud data onto the two-dimensional image data to calculate the three-dimensional spatial pose of the target wire; planning the end effector trajectory of the robotic arm based on the three-dimensional spatial pose, controlling the shearing actuator to align with and clamp the target wire; monitoring the electromechanical state parameters of the shearing actuator in real time during the shearing process, calculating the shearing impedance, identifying the wire breakage critical point based on the change characteristics of the shearing impedance, and executing an adaptive breakage control strategy.

[0008] In a preferred embodiment, the adaptive wire breakage control strategy actively adjusts the shear torque output when a critical breakage point is detected, in order to suppress the energy impact at the moment of wire breakage.

[0009] In a preferred embodiment, before calculating the three-dimensional spatial pose of the target conductor, the method further includes: generating a dynamic mask based on the spatial features of the point cloud, and filtering out the background of the two-dimensional image data.

[0010] In a preferred embodiment, the step of generating a dynamic mask based on the spatial features of point clouds includes: obtaining the laser point cloud depth value corresponding to each pixel projected onto a two-dimensional image, and calculating the average point cloud depth value in the local neighborhood of the pixel; determining whether the pixel satisfies the depth continuity condition by comparing the absolute value of the difference between the laser point cloud depth value and the average point cloud depth value with a preset depth discreteness threshold; performing spatial linear fitting on the point cloud data in the local neighborhood of the pixel, and determining whether the pixel satisfies the linear feature condition by comparing the obtained spatial linear fitting degree with a preset linearity threshold; and generating a mask to distinguish between the retained area and the background area based on whether the pixel simultaneously satisfies the depth continuity condition and the linear feature condition.

[0011] In a preferred embodiment, the calculation of shear impedance based on the state observer model specifically involves constructing an inverse dynamics model of the shear actuator using the state observer and estimating the shear impedance using the inverse dynamics model.

[0012] In a preferred embodiment, the step of identifying the critical point of conductor breakage based on the change characteristics of shear impedance and executing an adaptive wire breakage control strategy includes: calculating the time change rate of shear impedance; when the shear impedance exceeds the yield threshold and the change rate shows a negative abrupt change, determining that the conductor has entered the final stage of plastic deformation, i.e., the critical point of breakage; and executing active energy dissipation control, controlling the shear motor to output a reverse damping torque, thereby absorbing the bouncing kinetic energy of the robotic arm end at the moment of wire breakage.

[0013] In a preferred embodiment, after background filtering of the two-dimensional image data, a line detection algorithm based on Hough transform is used to perform secondary fitting on the pixels in the mask area, extract the equation of the conductor centerline, and back-project the equation into three-dimensional space to correct the end trajectory of the robotic arm.

[0014] This invention provides a wire identification and breakage control system based on multi-sensor information fusion, comprising: a data acquisition module for acquiring two-dimensional image data and three-dimensional point cloud data of the working environment; a fusion perception module for mapping the three-dimensional point cloud data onto the two-dimensional image data and calculating the three-dimensional spatial pose of the target wire; and a breakage control module for planning the end effector trajectory of a robotic arm based on the three-dimensional spatial pose, controlling the shearing actuator to align and clamp the target wire, monitoring the electromechanical state parameters of the shearing actuator in real time during the shearing process, calculating the shearing impedance, identifying the wire breakage critical point based on the change characteristics of the shearing impedance, and executing an adaptive breakage control strategy.

[0015] The technical effects and advantages of the wire identification and wire breakage control method and system based on multi-sensor information fusion of this invention are as follows:

[0016] This invention constructs a multi-sensor fusion model based on spatiotemporal registration of vision and lidar, and utilizes the spatial features of point clouds to generate dynamic masks to effectively remove unstructured background interference in two-dimensional images, achieving high-precision calculation of the three-dimensional pose of the target conductor in complex working environments. At the same time, it calculates the shearing impedance by combining the electromechanical state parameters of the shearing actuator, and executes an adaptive wire breakage control strategy when the critical point of conductor breakage is identified, actively adjusting the shearing torque to suppress the energy impact at the moment of wire breakage. This effectively solves the technical problems of poor robustness of traditional single vision recognition and easy rebound instability of robotic arms caused by the breakage of high-tension conductors. Attached Figure Description

[0017] Figure 1 This is a schematic flowchart of a wire identification and wire breakage control method based on multi-sensor information fusion provided in an embodiment of the present invention;

[0018] Figure 2 This is a schematic diagram of the mask generation process provided in an embodiment of the present invention;

[0019] Figure 3 A fifth-order polynomial interpolation trajectory curve provided for an embodiment of the present invention;

[0020] Figure 4 The shear resistance change and fracture identification curve provided in the embodiments of the present invention;

[0021] Figure 5 This is a block diagram of the wire identification and disconnection control system based on multi-sensor information fusion provided in an embodiment of the present invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0023] Example 1, Figure 1 The present invention provides a wire identification and wire breakage control method based on multi-sensor information fusion, comprising the following steps:

[0024] S1 collects two-dimensional image data and three-dimensional point cloud data of the working environment through a visible light camera and a lidar, respectively.

[0025] It should be noted that this step is the physical foundation for subsequent multi-sensor information fusion, and its core purpose is to acquire complementary visual texture information and spatial geometric information in the work scene. In specific implementation, to ensure the accuracy of subsequent spatial registration, it is first necessary to construct a perception hardware system. Specifically, a high-resolution visible light camera and a high-precision lidar (such as a mechanical multi-line radar or solid-state radar) are fixed together on the end effector or gimbal base of the work robot using a high-strength rigid bracket. The installation positions of the two are adjusted so that the scanning field of view of the lidar completely covers the imaging field of view of the visible light camera. This rigid connection structure ensures that the relative pose between the two sensors remains constant during the robot's movement, providing a physical prerequisite for subsequent extrinsic parameter calibration.

[0026] To meet the stringent time consistency requirements of "spatiotemporal registration" in step S2, "time slippage" errors caused by different sensor sampling frequencies or transmission delays must be eliminated during the acquisition process. Specifically, a hardware-triggered synchronous acquisition strategy is adopted. This involves simultaneously sending trigger commands to the visible light camera and the LiDAR using the same pulse signal generator, or performing nearest-neighbor matching at the software level based on precise system timestamps. This ensures that each frame of 2D image data and its corresponding frame of 3D point cloud data are acquired at the same time or within a millisecond-level time threshold, thereby avoiding dynamic distortion caused by the movement of mechanical equipment or wire swaying.

[0027] The final acquired 2D image data is specifically represented as an RGB pixel matrix containing red, green, and blue channels, used to provide color and texture features of the working environment; the acquired 3D point cloud data is specifically represented as a set of spatial coordinate points. The corresponding reflection intensity values ​​are used to provide precise depth and geometric contour information of the operating environment. These two types of heterogeneous data are transmitted in real-time to the memory of the airborne controller via a communication bus, serving as input data for subsequent steps.

[0028] S2, Construct a multi-sensor fusion model based on spatiotemporal registration, map the three-dimensional point cloud data to the two-dimensional image data, and calculate the three-dimensional spatial pose of the target conductor.

[0029] In this embodiment, step S2 includes:

[0030] S201, a rigid transformation relationship between the lidar coordinate system and the camera coordinate system is established using joint calibration parameters. The discrete 3D point cloud data obtained in step S1 is projected onto the 2D image pixel plane, assigning depth information to the image pixels. Specifically, a projection transformation from point cloud to image is performed using the joint calibration matrix. Through this transformation, the corresponding pixel position of each spatial point cloud can be found on the image. The calculation formula for this projection transformation is as follows:

[0031] ,

[0032] In the formula, These are pixel coordinates in a two-dimensional image coordinate system. The coordinates of the laser point cloud in the world coordinate system. As a scale factor, This is the intrinsic parameter matrix of the camera. The extrinsic parameter matrix of the LiDAR relative to the camera includes the rotation matrix. Translation vector In a typical practical calibration implementation, the calibration parameters obtained are shown in Table 1. The table displays the principal point coordinates of the camera's intrinsic parameter matrix, the focal length, and the rotation and translation components of the LiDAR relative to the camera. The system uses these parameters to achieve high-precision pixel-level alignment. These represent the equivalent focal lengths of the camera in the horizontal (x-axis) and vertical (y-axis) directions of the image plane, respectively. This represents the coordinates of the intersection of the camera's optical axis and the image plane in the pixel coordinate system. These are rotation parameters expressed in Rodrigues vector form, used to describe the attitude rotation relationship between the LiDAR coordinate system and the camera coordinate system. They can be converted into a 3×3 rotation matrix using the Rodrigues formula. ; These represent the x, y, and z axis components of the lidar coordinate system origin in the camera coordinate system. This parameter describes the relative positional offset of the optical centers of the two sensors in physical space, i.e., the translation vector. .

[0033] Table 1

[0034]

[0035] S202, after completing the projection mapping, a dynamic region of interest (ROI) mask needs to be generated to accurately extract conductor features from the complex unstructured background. In practice, instead of simply retaining all projected points, the unstructured background is removed based on the dual features of the point cloud: "depth continuity condition" and "linear feature condition." The system iterates through each projected pixel, calculates the laser point cloud depth value of that point and its local neighborhood points, calculates the average depth value of the point cloud within that local neighborhood, and takes the absolute value of the difference between the two as the depth determination criterion. It also performs a linear fit on the point cloud within that local neighborhood, calculating the spatial linearity degree as the linearity determination criterion.

[0036] Next, a logical judgment is made: if the absolute value of the difference is less than the preset depth discrete threshold, it is determined that the depth continuity condition is met (representing a smooth depth change); if the spatial linearity fit is greater than the preset linearity threshold, it is determined that the linear feature condition is met (representing a linear arrangement).

[0037] When generating the mask, the following filtering strategy is implemented: a pixel is marked as a reserved region (i.e., a potential guide region) if and only if it simultaneously satisfies the depth continuity condition and the linear feature condition; if the pixel does not satisfy the depth continuity condition (e.g., abrupt depth changes due to leaf edges) or the linear feature condition (e.g., cluttered point clouds inside the tree canopy), it is determined to be unstructured background interference and filtered out. This feature-based filtering method generates dynamic region of interest (ROI) masks. The logical judgment expression is as follows:

[0038] ,

[0039] In the formula, For image pixels The mask value at the location is 1, which means that the pixel is retained as a candidate point for the wire, and 0 means that it is filtered out as background. This represents the depth value of the projected point cloud at that pixel. The absolute value of the difference between the two values ​​is the "depth determination criterion". A preset depth discrete threshold is used to remove background edges with abrupt depth changes; The spatial linearity fit of the point cloud in this region, i.e., the "linearity criterion" (can be obtained by calculating the contribution rate of the first principal component through principal component analysis (PCA)). A preset linearity threshold is used to remove nonlinear scattered noise. To visually illustrate the background filtering effect, this embodiment describes the mask generation process under typical complex backgrounds as follows: Figure 2 As shown in the figure. From left to right, the figures are: (a) the original RGB image, showing the wire hidden in the background of trees; (b) the laser point cloud depth projection heatmap, where warmer colors represent closer distances; (c) the spatial linearity feature map, where bright areas represent significant linear features; and (d) the final generated binarized dynamic ROI mask, where background interference is effectively filtered out, retaining only the wire area.

[0040] S203, after generating the denoised binarized mask image, in order to further improve the recognition accuracy and obtain the geometric equation of the wire, a line detection algorithm based on Hough Transform is used to perform a secondary fitting on the effective pixels within the mask area. Specifically:

[0041] 1) Extracting the 2D centerline equation: Perform a Hough transform on the binarized mask image, in the Hough transform parameter space ( Find the local peak of the accumulator voting value in ) where and Let x be the perpendicular distance from the origin to the line and y be the angle between the perpendicular from the origin to the line and the positive x-axis, respectively, to determine the two-dimensional equation of the conductor on the image plane. This process utilizes the integral property to effectively connect discontinuous pixels caused by sparse or occluded laser point clouds, thus repairing the visual breaks in the lines on the image.

[0042] 2) Perform a 2D to 3D back projection operation: Depth cannot be obtained using only the two-dimensional straight line equation, so it is necessary to combine it with the effective point cloud depth information retained in step S2. Specifically, select a number of sampling points on the aforementioned two-dimensional straight line according to a preset step size. And retrieve the depth values ​​corresponding to these sampling points in the registered depth map. Using the inverse of the camera intrinsic parameter matrix Each sampling point is mapped back from the pixel coordinate system to the camera's 3D coordinate system, and then the result is obtained through the extrinsic parameter matrix. Transform to the world coordinate system (or robot base coordinate system). The back projection calculation formula for a single sampling point is as follows:

[0043] ,

[0044] In the formula, For the first The three-dimensional coordinates of each sampling point in the world coordinate system This represents the depth value corresponding to that point. These are pixel coordinates.

[0045] 3) Three-dimensional line fitting: After back projection, a set of discrete points approximately arranged on a straight line in three-dimensional space is obtained. To eliminate measurement noise, three-dimensional least squares or principal component analysis (PCA) is used to fit a spatial straight line to the point set. Specifically, the geometric center of the point set is calculated as a point on the straight line, and the eigenvector corresponding to the largest eigenvalue of the point set's covariance matrix is ​​calculated as the direction vector of the straight line. This constructs an accurate three-dimensional straight line equation for the target traverse (including spatial position coordinates and direction vector), thus completing the solution for the three-dimensional spatial pose.

[0046] This embodiment achieves the technical effect of "filtering out" invalid backgrounds like the human eye in unstructured environments such as tree occlusion and complex lighting, while retaining and accurately calculating the spatial position of slender wires, through the multi-sensor fusion and feature filtering method described above. This significantly improves the robustness of subsequent robotic arm motion planning.

[0047] S3, based on the three-dimensional spatial pose, plan the end-effector trajectory of the robotic arm, and control the shearing actuator to align and clamp the target wire.

[0048] It should be noted that this step is a crucial step in converting environmental perception results into entity motion control. Its core lies in solving the problem of high-precision alignment between the tool center point (TCP) of the robotic arm's end effector and the target guideline in unstructured space, while simultaneously avoiding collisions during approach. In specific implementation, the target pose matrix of the robotic arm's end effector needs to be constructed first based on the three-dimensional linear equation of the target guideline (including the coordinates of spatial points on the guideline and the guideline direction vector) calculated in step S2. Specifically, the three-dimensional position of the target guideline is set as the position target of the end effector. Simultaneously, based on the opening direction of the shearing actuator (such as hydraulic shears), a rotation matrix is ​​constructed so that the feed direction of the shearing actuator is perpendicular to the guideline direction vector, and the normal vector of the shearing plane is parallel to it, thereby ensuring that the cutting edge can perpendicularly engage with the guideline.

[0049] To ensure operational safety and prevent short circuits or wobbling caused by the robotic arm accidentally touching the guide wire due to inertia or control errors when moving directly to the guide wire position, a "hierarchical approximation" strategy is adopted in the trajectory planning process. Specifically, a "pre-grasp point" is first set at a certain safe distance (e.g., 10-15 cm) from the guide wire surface along the radial direction. The system calls the inverse kinematics algorithm, combined with the link parameters (DH parameters) of the robotic arm, to calculate the target angles of each joint required for the robotic arm to move from the current posture to the pre-grasp point.

[0050] In the trajectory generation stage, to ensure the smoothness of the robotic arm's movement and avoid jitter caused by sudden stops and starts, fifth-order polynomial interpolation is used to perform time planning for the angles of each joint, generating a motion trajectory where position, velocity, and acceleration are all continuous. The trajectory curve of a joint from its initial pose to the pre-grasp point, planned based on fifth-order polynomial interpolation, is shown below. Figure 3 As shown in the figure, the horizontal axis represents the normalized time period, and the vertical axis, from top to bottom, represents the angular position, angular velocity, and angular acceleration of the joint, respectively. It can be seen that the velocity and acceleration curves are smooth and continuous, and zero at both the start and end points, with no abrupt changes, thus ensuring the stability of the robotic arm's operation. The controller drives the servo motors of each joint according to this trajectory, causing the shearing actuator to move smoothly to the pre-grabbing point. Subsequently, after confirming that the alignment error meets the preset accuracy requirements (e.g., less than 2 mm), the controller controls the robotic arm to feed radially at a low speed until the wire is completely inside the jaws of the shearing actuator. Finally, the controller controls the clamping motor of the shearing actuator to close the jaws until they contact and pre-tighten the wire, completing the clamping operation and preparing the body for subsequent controlled wire cutting.

[0051] S4 monitors the electromechanical state parameters of the shearing actuator in real time during the shearing process, calculates the shearing impedance based on the state observer model, identifies the critical point of wire breakage based on the change characteristics of the shearing impedance, and executes an adaptive wire breakage control strategy.

[0052] In this embodiment, step S4 specifically includes:

[0053] S401, throughout the entire process of the shearing actuator closing its blade to cut the wire, the high-frequency current sensor and rotary encoder built into the motor driver are used to collect the three-phase current, rotor position, and speed data of the shearing motor in real time with a millisecond-level sampling period. Traditional shearing control usually relies only on position commands and cannot sense the interaction force between the blade and the wire, which can easily lead to "overcutting" or wire breakage and rebound. To address this, this embodiment constructs a state observer based on dynamic equations in the controller. This observer can isolate the energy consumed by the motor to overcome the system's own friction and inertia, thereby accurately estimating the equivalent load impedance used solely for cutting the wire. Specifically, the real-time shearing impedance is calculated based on the state observer model. The calculation model for this shear resistance is as follows:

[0054] ,

[0055] In the formula, Characterized in The equivalent physical impedance (including stiffness and damping characteristics) of a conductor resisting shear at any given time. The electromagnetic output torque of the shear motor is calculated based on the real-time current. The equivalent moment of inertia of the transmission system (including reducer and connecting rod) referred to the motor shaft; The motor angular acceleration is obtained by the second derivative of the encoder data; The intrinsic viscous friction coefficient of the system; This refers to the angular velocity of the motor. This is the linear feed rate of the tool edge calculated using forward kinematics. This formula allows the system to quantify changes in the hardness and stress state of the wire material.

[0056] S402, the system performs real-time analysis of the calculated shear impedance data stream, identifies the critical point of conductor breakage based on the changing characteristics of the shear impedance, and executes an adaptive wire breakage control strategy. Specifically, the system continuously calculates the first derivative (i.e., rate of change) of the shear impedance with respect to time. In the early and middle stages of shearing, as the cutting edge penetrates the metal layer of the conductor, the impedance... It will show an upward or high-level trend; when the metal enters the final stage of plastic deformation (i.e., when "necking" occurs), the resistance provided by the wire will decrease sharply even if the tool continues to feed. Therefore, when the system detects... Exceeding the preset material yield threshold, and its rate of change When a significant negative abrupt change occurs, the current moment is determined to be the "breakage critical point" at which the conductor is about to completely break.

[0057] S403 identifies the critical fracture point. The controller immediately interrupts the conventional constant speed or constant force shearing command and instead executes active energy dissipation control. This means that in the extremely short time before the wire is completely severed, the shearing motor outputs a braking torque in the opposite direction of motion to preemptively dissipate the elastic potential energy accumulated at the end of the robotic arm. This reverse damping torque... The calculation formula is as follows:

[0058] ,

[0059] In the formula, This is the damping gain coefficient, used to adjust the magnitude of the reverse torque; Given the current motor angular velocity, ensure that the damping force is proportional to the velocity (simulating physical viscous damping). It is a decay factor used to control the compliance of damping intervention; The time difference between the estimated distance based on the current feed rate and the remaining conductor diameter is used to predict the fracture time. This control law determines the time difference between the estimated fracture time and ... Approaching zero), the damping force increases exponentially, thus catching the robotic arm like a "soft landing" at the moment the conductor breaks, preventing it from experiencing violent forward thrust or whiplash due to the sudden loss of load. In a measured steel-cored aluminum stranded wire shearing experiment, the system recorded the shearing impedance change and the control intervention timing as follows: Figure 4 As shown in the figure. The solid blue line represents the shear impedance calculated in real time. The red dashed line represents the rate of change of impedance. At the critical point of breakage, the impedance reaches its peak and then begins to decrease, with a significant negative spike in the rate of change. Based on this, the system determines that it is the critical point of breakage and triggers the reverse damping torque (green shaded area in the figure), successfully suppressing the bouncing of the robotic arm at the moment of wire breakage.

[0060] This embodiment achieves a technological leap from "blind hard cutting" to "perceptive soft cutting" by establishing a shear impedance model and an active damping control strategy. It effectively solves the problem of robot arm instability and equipment damage caused by the instantaneous energy release when a high-tension conductor breaks, and greatly improves the safety and operational stability of the live-line working robot.

[0061] Example 2, Figure 5 A wire identification and wire breakage control system based on multi-sensor information fusion is presented, including:

[0062] The data acquisition module is used to collect two-dimensional image data and three-dimensional point cloud data of the working environment;

[0063] The fusion perception module is used to map the three-dimensional point cloud data onto the two-dimensional image data and calculate the three-dimensional spatial pose of the target conductor.

[0064] The wire breakage control module is used to plan the end effector trajectory of the robotic arm based on the three-dimensional spatial pose, control the shearing actuator to align and clamp the target wire, monitor the electromechanical state parameters of the shearing actuator in real time during the shearing process, calculate the shearing impedance, identify the wire breakage critical point based on the change characteristics of the shearing impedance, and execute an adaptive wire breakage control strategy.

[0065] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0066] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, in the form of a computer program product.

[0067] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0068] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0069] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0070] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for wire identification and wire breakage control based on multi-sensor information fusion, characterized in that, Includes the following steps: Collect two-dimensional image data and three-dimensional point cloud data of the working environment; The three-dimensional point cloud data is mapped onto the two-dimensional image data using a joint calibration parameter matrix to calculate the three-dimensional spatial pose of the target conductor; the parameter matrix includes the intrinsic parameter matrix of the camera and the extrinsic parameter matrix of the lidar relative to the camera; Based on the three-dimensional spatial pose, the end effector trajectory of the robotic arm is planned, and the shearing actuator is controlled to align and clamp the target wire. The electromechanical status parameters of the shearing actuator are monitored in real time during the shearing process, and the shearing impedance is calculated. Based on the characteristics of the shearing impedance change, the critical point of wire breakage is identified, and an adaptive wire breakage control strategy is executed. The method of identifying the critical point of conductor breakage based on the change characteristics of shear impedance includes: calculating the time change rate of shear impedance, and determining that a critical point of breakage has been identified when the time change rate exceeds the yield threshold and a negative abrupt change occurs. The adaptive wire breakage control strategy includes actively adjusting the shearing torque output when the critical breakage point is detected. Specifically, it involves interrupting the conventional constant speed or constant force shearing command and instead executing active energy dissipation control, controlling the shearing motor to output a braking torque opposite to the direction of motion, so as to consume the elastic potential energy accumulated at the end of the robotic arm in advance and suppress the energy impact at the moment of wire breakage.

2. The wire identification and wire breakage control method based on multi-sensor information fusion according to claim 1, characterized in that, Before calculating the three-dimensional spatial pose of the target conductor, the method further includes: generating a dynamic mask based on the spatial features of the point cloud, and filtering out the background of the two-dimensional image data.

3. The wire identification and wire breakage control method based on multi-sensor information fusion according to claim 1, characterized in that, The projection transformation expression for mapping the three-dimensional point cloud data to the two-dimensional image data is as follows: In the formula, These are pixel coordinates in a two-dimensional image coordinate system. The coordinates of the laser point cloud in the world coordinate system. As a scale factor, This is the intrinsic parameter matrix of the camera. The extrinsic parameter matrix of the LiDAR relative to the camera includes the rotation matrix. Translation vector .

4. The wire identification and wire breakage control method based on multi-sensor information fusion according to claim 2, characterized in that, The dynamic mask generation based on the spatial features of point clouds includes obtaining depth determination criteria and linearity determination criteria: Obtain the laser point cloud depth value corresponding to the pixel point projected onto the two-dimensional image, and calculate the average point cloud depth value in the local neighborhood of the pixel point; Calculate the absolute value of the difference between the laser point cloud depth value and the average point cloud depth value, and use it as the basis for depth determination; Spatial linear fitting is performed on the point cloud data within the local neighborhood of the pixel, and the spatial linearity fitting degree is calculated and used as the criterion for linearity determination.

5. The wire identification and wire breakage control method based on multi-sensor information fusion according to claim 4, characterized in that, The method for generating a dynamic mask based on the spatial features of point clouds also includes: The absolute value of the difference is compared with a preset depth discrete threshold. If it is less than the threshold, the depth continuity condition is determined to be met. The spatial linearity fit is compared with a preset linearity threshold. If it is greater than the threshold, it is determined that the linearity feature condition is met. A pixel is marked as a reserved region and a mask is generated if and only if the pixel simultaneously satisfies the depth continuity condition and the linear feature condition. If a pixel does not meet the depth continuity condition or the linear feature condition, then the pixel is marked as a background region and filtered out.

6. The wire identification and wire breakage control method based on multi-sensor information fusion according to claim 1, characterized in that, The calculation of shear impedance specifically involves constructing an inverse dynamics model of the shear actuator using the electromechanical state parameters, and estimating the shear impedance using this inverse dynamics model. The inverse dynamics model is as follows: In the formula, for The equivalent shear resistance of the wire to the tool at any given time. This is the output torque of the shear motor. The moment of inertia of the transmission system. For the angular acceleration of the motor, The system's viscous friction coefficient, The angular velocity of the motor. This represents the feed rate of the cutting edge.

7. The wire identification and wire breakage control method based on multi-sensor information fusion according to claim 6, characterized in that, The control shear motor outputs a braking torque opposite to the direction of motion, and its reverse damping torque... The calculation formula is: In the formula, This is the damping gain coefficient. As the attenuation factor, This represents the time difference between the current moment and the predicted moment of fracture.

8. The wire identification and wire breakage control method based on multi-sensor information fusion according to claim 5, characterized in that, After generating the dynamic mask, a line detection algorithm based on Hough transform is used to perform a second fitting on the pixels within the mask area, extract the equation of the conductor centerline, and back-project the equation into three-dimensional space to correct the end trajectory of the robotic arm.

9. A system using the wire identification and wire breakage control method based on multi-sensor information fusion as described in any one of claims 1-8, characterized in that, include: The data acquisition module is used to collect two-dimensional image data and three-dimensional point cloud data of the working environment; The fusion perception module is used to map the three-dimensional point cloud data to the two-dimensional image data using a joint calibration parameter matrix, and calculate the three-dimensional spatial pose of the target conductor. The parameter matrix includes the camera's intrinsic parameter matrix and the lidar's extrinsic parameter matrix relative to the camera; The wire breakage control module is used to plan the end trajectory of the robotic arm based on the three-dimensional spatial pose, control the shearing actuator to align and clamp the target wire, monitor the electromechanical state parameters of the shearing actuator in real time during the shearing process, calculate the shearing impedance, identify the wire breakage critical point based on the change characteristics of the shearing impedance, and execute an adaptive wire breakage control strategy. The method of identifying the critical point of conductor breakage based on the change characteristics of shear impedance includes: calculating the time change rate of shear impedance, and determining that a critical point of breakage has been identified when the time change rate exceeds the yield threshold and a negative abrupt change occurs. The adaptive wire breakage control strategy includes actively adjusting the shearing torque output when the critical breakage point is detected. Specifically, it involves interrupting the conventional constant speed or constant force shearing command and instead executing active energy dissipation control, controlling the shearing motor to output a braking torque opposite to the direction of motion, so as to consume the elastic potential energy accumulated at the end of the robotic arm in advance and suppress the energy impact at the moment of wire breakage.

Images