Hot-line work mechanical arm remote control method and system based on virtual reality

Abstract

The invention relates to the technical field of mechanical arm control, in particular to a live working mechanical arm remote control method and system based on virtual reality. According to the method, the space occupation result of the grabbed target object on the workable area is judged in real time, and the form complex parameters of the target object are obtained; and then analyzing the operation instruction information of the hot-line work mechanical arm and the state transition condition of the response motion information of the hot-line work mechanical arm, obtaining the response effective coefficient of the hot-line work mechanical arm, and then controlling the hot-line work mechanical arm in combination with the form complex parameters and the space occupation result. The space occupation condition of the grabbed target object is analyzed in real time so that a protection mechanism can be triggered, meanwhile, the response deviation is evaluated in combination with the instruction and the state transition condition of the mechanical arm movement, then the mechanical arm movement is regulated and controlled in real time, the collision risk is reduced, and the remote control effect on the hot-line work mechanical arm is improved.

Description

Technical Field

[0001] This invention relates to the field of robotic arm control technology, specifically to a remote control method and system for a live-line working robotic arm based on virtual reality. Background Technology

[0002] A live-line working robot is a robotic device specifically designed for performing live-line work on high-voltage electrical equipment. It employs force feedback and master-slave control technology to replace manual labor in performing dangerous or high-precision tasks. However, due to potential perceptual errors in the actual working environment of the robot, current technology typically uses Virtual Reality (VR) to provide operators with a three-dimensional virtual environment that closely resembles the live-line working scenario of the robot. This enhances the operator's environmental perception, while sensors are used for distance and collision monitoring to assist in risk warnings, thereby enabling remote control of the robot.

[0003] However, when the robotic arm moves to its physical limit or is unable to continue moving due to environmental limitations, it may become unresponsive to operating commands. Furthermore, if the operator does not adjust the operating commands in time after the robotic arm stops due to obstacles or restrictions and instead inputs commands to continue moving, it will not only affect the user experience but may also cause unpredictable movements after the robotic arm resumes movement due to the accumulation of operating commands, increasing safety risks such as collisions. This results in poor remote control of the robotic arm for live-line work. Summary of the Invention

[0004] To address the technical problem of poor remote control performance for live-line working robotic arms, the present invention aims to provide a virtual reality-based remote control method and system for live-line working robotic arms. The specific technical solution adopted is as follows: A method for remote control of a live-line working robotic arm based on virtual reality, the method comprising: The system can acquire images of the live-line working robot arm grasping the target object and the workable area in real time, and also acquire operation command information and response motion information of the live-line working robot arm in real time. Determine the spatial encroachment of the captured target object in the captured image on the workable area; obtain the morphological complexity parameters of the target object based on the morphological features of the region corresponding to the target object in the captured image; Based on the state transition of the operation instruction information and the response motion information, a significant transition vector of the operation instruction information and a significant transition vector of the response motion information are constructed respectively; based on the similarity between the significant transition vector of the operation instruction information and the significant transition vector of the response motion information, the effective response coefficient of the live-line working robot is obtained; based on the effective response coefficient and the morphological complexity parameter, combined with the spatial encroachment result, the live-line working robot is remotely controlled.

[0005] Furthermore, the methods for determining the results of space encroachment include: All segmented regions in the captured image are obtained based on a threshold segmentation algorithm. The segmented region containing the center pixel of the captured image is taken as the target object region. When the area of ​​the target object region is greater than the area of ​​the workable region, it is determined that the captured target object has spatial encroachment; otherwise, it is determined that it does not exist.

[0006] Furthermore, the method for obtaining the morphologically complex parameters includes: In the captured image, any boundary pixel on the boundary of the target object region is taken as the target pixel. Starting from the target pixel, the system traverses along the boundary of the region from any direction to obtain the coordinate distances between the corresponding position coordinates of all adjacent boundary pixels. The coordinate distances are then sorted according to the traversal order to construct a coordinate distance sequence. Based on the coordinate distance sequence and its first-order difference sequence, the morphological complexity parameters of the target object are obtained.

[0007] Furthermore, the method for obtaining the morphological complexity parameters of the target object based on the coordinate distance sequence and its first-order difference sequence includes: The variance of all sequence elements in the coordinate distance sequence is used as the first morphological complexity parameter, and the total number of non-zero elements in the first-order difference sequence is used as the second morphological complexity parameter. The morphological complexity parameters of the target object are obtained by fusing the first and second morphological complexity parameters.

[0008] Furthermore, the operation instruction information is an operation instruction vector with displacement, linear acceleration, and angular velocity as vector elements set for each axis of the live-line working robot, and the response motion information is a work motion vector with displacement, linear acceleration, and angular velocity as vector elements when each axis of the live-line working robot moves.

[0009] Furthermore, the method for obtaining the instruction significant transfer vector and the motion significant transfer vector includes: Based on the extended Kalman filter algorithm, the instruction state transition matrix of the operation instruction vector and the motion state transition matrix of the job motion vector are obtained respectively; the largest matrix element in each row of the instruction state transition matrix is ​​selected and a row vector is constructed to obtain the instruction significant transition vector; the largest matrix element in each row of the motion state transition matrix is ​​selected and a row vector is constructed to obtain the motion significant transition vector.

[0010] Furthermore, the method for obtaining the response validity coefficient includes: The negative correlation normalization result of the magnitude of the difference vector between the command significant transfer vector and the motion significant transfer vector is used as the response effective coefficient of the live-line working robot.

[0011] Furthermore, methods for controlling a live-line working robotic arm include: The normalized result of the morphological complexity parameter of the target object is used as the operation complexity weight; the response effectiveness coefficient is weighted using the operation complexity weight, and the weighted result is used as the response control coefficient of the live-line working robot. The control level symbol of the live-line working robot is determined based on the space encroachment result; the negative correlation normalization result of the response control coefficient is weighted using a preset gain coefficient, and the weighted result is limited by adding a preset basic control level; combined with the control level symbol, the control level is determined to control the live-line working robot.

[0012] Furthermore, the method for determining the control level symbol includes: When the target object occupies space, the control level sign is set to negative; when the target object does not occupy space, the control level sign is set to positive.

[0013] The system includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the virtual reality-based remote control method for live-line working robotic arms.

[0014] The present invention has the following beneficial effects: This invention acquires real-time images of a live-line working robot grasping a target object and the workable area, as well as real-time operation commands and response motion information of the robot, providing data preparation for subsequent analysis. It determines the spatial encroachment of the grasped target object on the workable area in the image to decide whether a protection mechanism needs to be triggered to adjust the control level. Then, based on the morphological characteristics of the corresponding area of ​​the target object in the image, it acquires the morphological complexity parameters of the target object, preparing for subsequent control. Based on the state transitions of the operation command information and response motion information, it constructs significant transition vectors for both the operation command information and the response motion information, respectively, to measure response deviation based on the similarity between the two vectors and obtain the effective response coefficient of the live-line working robot. Finally, it combines the morphological complexity parameters and spatial encroachment results to control the live-line working robot. This invention analyzes the spatial encroachment of the target object in real time to trigger a protection mechanism. At the same time, it assesses the response deviation by combining the command and the state transition of the robotic arm movement, and then adjusts the robotic arm movement in real time to reduce the risk of collision and improve the remote control effect of the robotic arm for live-line work. Attached Figure Description

[0015] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 A flowchart illustrating a method for remote control of a live-line working robotic arm based on virtual reality, as provided in one embodiment of the present invention; Figure 2 This is a flowchart illustrating a method for controlling a live-line working robotic arm, as provided in one embodiment of the present invention. Detailed Implementation

[0017] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of a virtual reality-based remote control method and system for live-line working robotic arms proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0018] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0019] The following description, in conjunction with the accompanying drawings, details a specific solution for a virtual reality-based remote control method and system for live-line working robotic arms provided by this invention.

[0020] Please see Figure 1 The diagram illustrates a flowchart of a remote control method for a live-line working robotic arm based on virtual reality, according to an embodiment of the present invention, which specifically includes: Step S1: Real-time acquisition of the grasping image and workable area of ​​the live-line working robot when it grasps the target object, and real-time acquisition of the operation command information and response motion information of the live-line working robot.

[0021] In one embodiment of the present invention, the live-line working robot arm is first powered on and started, and a VR head-mounted device is configured for the operator of the live-line working robot arm. Then, the live-line working robot arm and the VR device are initialized, configured and calibrated, that is, converted by a pre-calibrated scale, and the scale difference between the actual human operating range and the working range of the robot arm is solved by spatial scaling, so as to ensure that the visual feedback in the VR device is consistent with the actual movement direction of the robot arm. This is a prior art and operation method known to those skilled in the art, and will not be described in detail here. The live-line working robotic arm mainly consists of a base, a large arm, a small arm, and a gripper. A camera is integrated on the small arm, and weight sensors are installed at the gripper and small arm positions. When the sensor data is zero, the gripper is unloaded; when the sensor data is not zero, the robotic arm is determined to be in a gripping state. The camera on the small arm then captures real-time images of the gripper gripping the target object. The center of the gripping image is aligned with the center of the gripper, meaning the centers coincide. Further preprocessing, such as grayscale conversion and noise reduction, is performed on the gripping image; these are common techniques and will not be elaborated further. Meanwhile, the operable area of ​​the live-line working robot arm when grasping the target object is obtained in real time based on the panoramic image in the VR device. The operable area is not only the non-obstacle area during the operation, but also the remaining maximum motion area of ​​the live-line working robot arm. The intersection area of ​​the non-obstacle area and the remaining maximum motion area is taken as the operable area of ​​the live-line working robot arm. The acquisition of the operable area is also a well-known existing technical means in the art, and will not be described in detail here.

[0022] Then, in a preferred embodiment of the present invention, the operator's operation instruction information is obtained based on the VR device. The operation instruction information includes the displacement, linear acceleration and angular velocity set for each axis of the live-line working robot, and this information is used as vector elements to construct the operation instruction vector; the response motion information of the live-line working robot to the operation instruction is obtained based on the attitude sensor built into the live-line working robot, that is, the displacement, linear acceleration and angular velocity of each axis are collected as vector elements to construct the operation motion vector; It should be noted that the acquisition of operation command information and response motion information are all existing technical means, and will not be elaborated further; the order of each vector element in the operation command vector and the operation motion vector is consistent to ensure the effectiveness of subsequent comparative analysis.

[0023] Step S2: Determine the spatial encroachment of the captured target object on the workable area in the captured image; obtain the morphological complexity parameters of the target object based on the morphological features of the corresponding area of ​​the target object in the captured image.

[0024] Due to limitations such as confined working space, the surrounding facilities may interfere with the operation of the robotic arm, leading to collision risks or affecting operational stability. The size of the target object grasped by the robotic arm and its encroachment on the workable area can help determine the redundancy of the robotic arm's working space, allowing for adjustments to operational commands to avoid collisions.

[0025] Preferably, in one embodiment of the present invention, considering that the area corresponding to the target object in the captured image initially reflects the size of the target object, and the difference in area between the target object and the workable area initially reflects the spatial redundancy of the target object within the workable area, the spatial encroachment result of the captured target object is determined, and the existence of collision risk is assessed, in preparation for triggering the protection mechanism and controlling the robotic arm to avoid obstacles and stop in time; furthermore, considering that the center of the captured image coincides with the center of the gripper, and the target object appears as a connected region in the captured image, the target object in the image can be determined by the image center; furthermore, when the area corresponding to the target object is greater than the area of ​​the workable area, no redundant space can be assessed; therefore, the method for determining the spatial encroachment result includes: All segmented regions in the captured image are obtained based on the threshold segmentation algorithm. The segmented region containing the center pixel of the captured image is taken as the target object region. When the area of ​​the target object region is larger than the area of ​​the workable region, it is determined that the captured target object has spatial encroachment; otherwise, it is determined that it does not exist.

[0026] As an example, the algorithm first uses the Otsu threshold segmentation algorithm to obtain several segmented regions in the captured image, which can then be used to obtain the target object region and determine the spatial encroachment result. When encroachment is determined, obstacle avoidance and stopping control are required. When there is no encroachment, the operation command can continue to be responded to.

[0027] In other examples, implementers may also use other threshold segmentation methods or other pre-trained annotation models to determine the target object region. Both of these, along with Otsu threshold segmentation, are existing technologies well known to those skilled in the art and will not be described further.

[0028] Furthermore, considering that when an irregularly structured target object is grasped and moved by a live-line working robot arm, its irregular protrusions in various directions may lead to greater spatial interference, which is less conducive to the robot arm's response and movement, this embodiment of the invention will analyze the dimensional uniformity of the target object structure based on the boundary morphological characteristics of the corresponding region of the target object in the grasping image, obtain the morphological complexity parameters of the target object, and then facilitate subsequent control of the robot arm by combining its interference with the robot arm's movement.

[0029] Preferably, in one embodiment of the present invention, considering that the boundary contour changes of the target object region can indirectly reflect its morphological complexity, and the changes in the coordinate distance between adjacent boundary pixels on the boundary contour roughly reflect the complexity of the boundary contour changes, the more consistent the coordinate distance between all adjacent boundary pixels, the more regular or smooth the boundary is; otherwise, the more rugged and complex it is. Based on this, the boundary can first be traversed to obtain the sequence of changes in the coordinate distance between all adjacent boundary pixels, and then the changes in the sequence can be analyzed to evaluate the morphological complexity parameters of the target object; therefore, the method for obtaining the morphological complexity parameters includes: In the image capture process, any boundary pixel on the boundary of the target object region is taken as the target pixel. Starting from the target pixel, the system traverses along the boundary of the region from any direction to obtain the coordinate distances between the corresponding positions of all adjacent boundary pixels. The coordinate distances are then sorted according to the traversal order to construct a coordinate distance sequence. Based on the coordinate distance sequence and its first-order difference sequence, the complex morphological parameters of the target object are obtained.

[0030] Furthermore, considering that the fluctuation of coordinate distance in the coordinate distance sequence can reflect the change or ruggedness of the boundary contour, the greater the fluctuation of the sequence elements, the more rugged the boundary contour; variance can help assess the intensity of fluctuation of the sequence elements in the sequence; and considering that if adjacent sequence elements in the coordinate distance sequence are consistent, it means that their corresponding boundary pixels are more likely to be smooth boundaries, while if adjacent sequence elements are different, it means that the boundary is rugged, possibly with depressions or convexities, and the boundary shape is more complex. Therefore, in a preferred embodiment of the present invention, the variance of all sequence elements in the coordinate distance sequence is used as the first morphological complexity parameter, and the total number of non-zero elements in the first-order difference sequence is used as the second morphological complexity parameter; the first morphological complexity parameter and the second morphological complexity parameter are fused to obtain the morphological complexity parameter of the target object.

[0031] As an example, a coordinate system is first constructed with the center of the captured image as the origin to determine the position coordinates of each pixel. Then, a target pixel is determined on the boundary of the region, and the system is traversed counterclockwise along the boundary. The Euclidean distance between adjacent boundary pixels, i.e., the coordinate distance, is evaluated based on the position coordinates. The coordinate distances are then sorted according to the traversal order to obtain a coordinate distance sequence, and a first-order difference sequence of the coordinate distance sequence is further obtained. Then, the first morphological complexity parameter and the second morphological complexity parameter are obtained, and the first morphological complexity parameter and the second morphological complexity parameter are multiplied and fused to obtain the morphological complexity parameter.

[0032] In another embodiment of the present invention, the implementer can also assess the ruggedness of the boundary by the change in boundary curvature. Specifically, the boundary curve is fitted based on a curve fitting algorithm, and the local curvature at each boundary pixel is calculated. Boundary pixels with curvature greater than 0 correspond to protruding parts, i.e., convex points, and boundary pixels with curvature less than 0 correspond to concave parts, i.e., concave points. Considering that the more numerous and closer the convex and concave points are, the greater the possibility of the boundary ruggedness, the absolute value of the difference between the number of convex and concave points is negatively correlated. For example, after adding a very small non-zero positive parameter of 0.1, the reciprocal operation is performed to obtain the first morphological complexity parameter. Considering that the greater the distance between the convex points and the center of the captured image, and the closer the distance between the concave points and the center of the captured image, the greater the degree of boundary ruggedness, the difference between the average distance of all convex points and the average distance of the concave points and the center of the captured image is used as the second morphological complexity parameter. Then, the first morphological complexity parameter and the second morphological complexity parameter are multiplied and fused to obtain the morphological complexity parameter.

[0033] In another embodiment of the present invention, morphological operations can be performed on the target object region to further determine the number of eroded regions and expanded regions, and then the eroded regions are regarded as raised regions and the expanded regions are regarded as recessed regions. The greater the total number of recessed regions and expanded regions, the greater the morphological complexity parameter.

[0034] It should be noted that in other embodiments, the implementer may also use other fusion methods such as addition or weighted summation to fuse the first morphological complex parameters and the second morphological complex parameters. The boundary line segment angle method may also be used to evaluate the boundary concavity and convexity of the target object region. These, along with the fitting boundary curve, curvature acquisition and morphological operations, are existing technologies well known to those skilled in the art and will not be described in detail here.

[0035] Step S3: Based on the state transition of the operation command information and the response motion information, construct the command significant transition vector of the operation command information and the motion significant transition vector of the response motion information respectively; based on the similarity between the command significant transition vector and the motion significant transition vector, obtain the response effectiveness coefficient of the live-line working robot; based on the response effectiveness coefficient and the morphological complexity parameter, control the live-line working robot in combination with the space encroachment result.

[0036] Considering the operator's operating instructions and the response movements of the live-line working robot, certain state transitions will occur during remote control operations. By analyzing the state transition information, we can help understand the differences between the instructions and the response, thereby accurately locating the response delay and deviation. This will help optimize the performance of the robot control system and reduce delay and deviation. Therefore, this embodiment of the invention will first construct the instruction significant transition vector of the operating instruction information and the motion significant transition vector of the response motion information, respectively. The two significant transition vectors reflect the main state transition situations.

[0037] Preferably, in one embodiment of the present invention, considering that the Extended Kalman Filter (EKF) is a recursive algorithm for state estimation of nonlinear systems, which can help estimate state transitions, the command state transition matrix and the motion state transition matrix can be obtained first based on the EKF algorithm. Furthermore, considering that each element in each row of the state transition matrix represents the probability distribution of transitions from the initial state to different states, and the maximum state transition probability in each row can indirectly reflect the significant transition intention or maximum transition information, a significant transition vector can be constructed to prioritize variables that have a greater impact on system state changes, thereby better designing control strategies. Therefore, the method for obtaining the command significant transition vector and the motion significant transition vector includes: Based on the extended Kalman filter algorithm, the instruction state transition matrix of the operation instruction vector and the motion state transition matrix of the job motion vector are obtained respectively. The largest matrix element in each row of the instruction state transition matrix is ​​selected and a row vector is constructed to obtain the instruction significant transition vector. The largest matrix element in each row of the motion state transition matrix is ​​selected and a row vector is constructed to obtain the motion significant transition vector.

[0038] It should be noted that obtaining the instruction state transition matrix and motion state transition matrix based on the extended Kalman filter algorithm, as well as constructing new row vectors, are existing technologies well known to those skilled in the art, and will not be elaborated further.

[0039] Considering that if the main state transition of the operation command is similar to the main state transition of the robotic arm's response motion, it indicates that the current execution action of the robotic arm can accurately respond to the operator's operation command, and the response effect is better; therefore, this embodiment of the invention will obtain the response effectiveness coefficient of the live-line working robotic arm based on the similarity between the command significant transition vector and the motion significant transition vector. The response effectiveness coefficient reflects the degree of deviation of the robotic arm's response to the control command, providing a certain reference value for subsequent control of the robotic arm's motion, so as to improve the control effect.

[0040] Preferably, in one embodiment of the present invention, considering that the smaller the magnitude of the difference vector between two vectors, the more similar the vectors are, which further indicates a better response of the motion to the command; and that the logical relationship can be adjusted through negative correlation mapping; therefore, the method for obtaining the effective response coefficient includes: The negative correlation normalization result of the magnitude of the difference vector between the command significant transfer vector and the motion significant transfer vector is used as the effective response coefficient of the live-line working robot.

[0041] As an example, negative correlation normalization can be achieved by taking the reciprocal. To avoid the difference vector having a magnitude of 0, which would render the denominator meaningless, a very small positive parameter, such as 0.0001, can be added to the magnitude before taking the reciprocal to obtain the effective response coefficient. Implementers can also use other negative correlation normalization methods, which are existing technologies and will not be elaborated further.

[0042] In another embodiment of the present invention, the implementer may also measure the Pearson correlation coefficient between the instruction significant transfer vector and the motion significant transfer vector, and then normalize the Pearson correlation coefficient to obtain the response efficiency coefficient; the Pearson correlation coefficient is also prior art and will not be described in detail here.

[0043] Considering that when the target object encroaches on space, i.e., there is a risk of collision and interference with other objects in the work scene, the object needs to be stopped; and considering that the complex shape parameters of the target object have a certain interference effect on the response to the operation command, that is, the more complex the shape and structure of the target object and the greater the possibility of spatial interference, the slower the response speed of the robotic arm should be during the operation to reduce the possibility of interference and collision accidents; therefore, after obtaining the effective response coefficient in this embodiment of the invention, the live-line working robotic arm can be further controlled based on the effective response coefficient and the complex shape parameters, combined with the spatial encroachment result.

[0044] Preferably, in one embodiment of the present invention, the method for controlling a live-line working robotic arm includes: Please see Figure 2 The diagram illustrates a method for controlling a live-line working robotic arm according to an embodiment of the present invention, specifically including: Step S301: The normalized result of the shape complexity parameters of the target object is used as the operation complexity weight; the response effectiveness coefficient is weighted using the operation complexity weight, and the weighted result is used as the response control coefficient of the live-line working robot.

[0045] Considering that the gripper may rotate after grasping the target object, the target object region in the grasping image acquired at different times may represent the state of the target object from different perspectives. The relative magnitude of the morphological complexity parameter of the target object region in the grasping image currently acquired in real time reflects its relative operational complexity at real time. Furthermore, the response control coefficient of the robotic arm is determined by combining the effective response coefficient of the robotic arm.

[0046] As an example, the morphological complexity parameters of the target object currently acquired in real time are used as the numerator, and the maximum value of the morphological complexity parameters of the target object acquired at all historical moments up to the current moment is used as the denominator. Since the denominator is always greater than or equal to the numerator, normalization can be performed, and the ratio of the fractions is used as the task complexity weight. Then, the task complexity weight is multiplied by the response effectiveness coefficient to obtain the response control coefficient.

[0047] Step S302: Determine the control level symbol of the live-line working robot based on the space encroachment result; use a preset gain coefficient to weight the negative correlation normalization result of the response control coefficient, add the preset basic control level to the weighted result and then limit it; combine the control level symbol to determine the control level to control the live-line working robot.

[0048] Considering that when there is space encroachment, the robotic arm needs to be controlled to stop in time to avoid obstacles, a protection mechanism can be triggered by outputting a negative control level to stop the robotic arm from avoiding obstacles; while when there is no space encroachment, a positive control level can be output to allow the robotic arm to continue executing operation commands; therefore, in a preferred embodiment of the present invention, the method for determining the control level sign includes: when the target object has space encroachment, the control level sign is set to a negative sign; when the target object does not have space encroachment, the control level sign is set to a positive sign.

[0049] Furthermore, considering that a larger response control coefficient indicates a greater spatial interference effect on the target object being grasped, and a smaller response deviation of the robotic arm, it is necessary to adjust the control level in a timely manner to reduce the risk of collision, while ensuring the control level required for normal grasping motion. At the same time, it is also necessary to ensure that the adjusted control level is within the normal working range of the robotic arm, and further amplitude limiting processing is required.

[0050] As an example, since the response control coefficient ranges from 0 to 1, negative correlation normalization can be performed directly by subtracting the effective control coefficient from 1. Then, the preset gain coefficient is multiplied by the negative correlation normalization result, and the product is added to the preset basic control level to obtain the control level. Then, the clamp function is used to limit the control level within the normal operating level range, thereby assigning the control level sign to the limited control level to control the robotic arm.

[0051] It should be noted that the preset gain coefficient is set to 0.7 and the preset basic control level is set to 7.4V. The normal operating level of the robotic arm needs to be determined according to the design parameters of the robotic arm. Implementers can also determine the preset gain coefficient and preset basic control level according to the actual application, which will not be elaborated further.

[0052] The present invention also proposes a virtual reality-based remote control system for a live-line working robotic arm. The system includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the virtual reality-based remote control method for a live-line working robotic arm described in steps S1-S3 above.

[0053] In summary, this invention determines the spatial encroachment of the grasped target object on the workable area in real time and obtains the object's morphological complexity parameters. Then, based on the operation command information and the state transition of the live-line working robot's response motion information, it constructs a significant transition vector for the operation command information and a significant transition vector for the response motion information. Furthermore, based on the similarity between the two vectors, it obtains the effective response coefficient of the live-line working robot. Finally, it controls the live-line working robot by combining the morphological complexity parameters and the spatial encroachment results. This invention analyzes the spatial encroachment of the grasped target object in real time to trigger a protection mechanism, and simultaneously evaluates the response deviation by combining the state transition of the command and robot movement, thereby adjusting the robot's movement in real time, reducing collision risks, and improving the remote control effect of the live-line working robot.

[0054] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0055] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. A remote control method for a live-line working robotic arm based on virtual reality, characterized in that, The method includes: The system can acquire images of the live-line working robot arm grasping the target object and the workable area in real time, and also acquire operation command information and response motion information of the live-line working robot arm in real time. Determine the spatial encroachment of the captured target object in the captured image on the workable area; obtain the morphological complexity parameters of the target object based on the morphological features of the region corresponding to the target object in the captured image; Based on the state transition of the operation instruction information and the response motion information, a significant transition vector of the operation instruction information and a significant transition vector of the response motion information are constructed respectively; based on the similarity between the significant transition vector of the operation instruction information and the significant transition vector of the response motion information, the effective response coefficient of the live-line working robot is obtained; based on the effective response coefficient and the morphological complexity parameter, combined with the spatial encroachment result, the live-line working robot is remotely controlled.

2. The remote control method for a live-line working robotic arm based on virtual reality according to claim 1, characterized in that, Methods for determining the results of space encroachment include: All segmented regions in the captured image are obtained based on a threshold segmentation algorithm. The segmented region containing the center pixel of the captured image is taken as the target object region. When the area of ​​the target object region is greater than the area of ​​the workable region, it is determined that the captured target object has spatial encroachment; otherwise, it is determined that it does not exist.

3. The remote control method for a live-line working robotic arm based on virtual reality according to claim 2, characterized in that, The methods for obtaining the morphologically complex parameters include: In the captured image, any boundary pixel on the boundary of the target object region is taken as the target pixel. Starting from the target pixel, the system traverses along the boundary of the region from any direction to obtain the coordinate distances between the corresponding position coordinates of all adjacent boundary pixels. The coordinate distances are then sorted according to the traversal order to construct a coordinate distance sequence. Based on the coordinate distance sequence and its first-order difference sequence, the morphological complexity parameters of the target object are obtained.

4. The remote control method for a live-line working robotic arm based on virtual reality according to claim 3, characterized in that, The method for obtaining the complex morphological parameters of the target object based on the coordinate distance sequence and its first-order difference sequence includes: The variance of all sequence elements in the coordinate distance sequence is used as the first morphological complexity parameter, and the total number of non-zero elements in the first-order difference sequence is used as the second morphological complexity parameter. The morphological complexity parameters of the target object are obtained by fusing the first and second morphological complexity parameters.

5. The method for remote control of a live-line working robotic arm based on virtual reality according to claim 1, characterized in that, The operation instruction information is an operation instruction vector with displacement, linear acceleration, and angular velocity as vector elements set for each axis of the live-line working robot arm, and the response motion information is an operation motion vector with displacement, linear acceleration, and angular velocity as vector elements when each axis of the live-line working robot arm moves.

6. The method for remote control of a live-line working robotic arm based on virtual reality according to claim 5, characterized in that, The methods for obtaining the instruction significant transfer vector and the motion significant transfer vector include: Based on the extended Kalman filter algorithm, the instruction state transition matrix of the operation instruction vector and the motion state transition matrix of the job motion vector are obtained respectively; the largest matrix element in each row of the instruction state transition matrix is ​​selected and a row vector is constructed to obtain the instruction significant transition vector; the largest matrix element in each row of the motion state transition matrix is ​​selected and a row vector is constructed to obtain the motion significant transition vector.

7. The remote control method for a live-line working robotic arm based on virtual reality according to claim 1, characterized in that, The method for obtaining the effective response coefficient includes: The negative correlation normalization result of the magnitude of the difference vector between the command significant transfer vector and the motion significant transfer vector is used as the response effective coefficient of the live-line working robot.

8. The method for remote control of a live-line working robotic arm based on virtual reality according to claim 1, characterized in that, Methods for controlling a live-line working robotic arm include: The normalized result of the morphological complexity parameter of the target object is used as the operation complexity weight; the response effectiveness coefficient is weighted using the operation complexity weight, and the weighted result is used as the response control coefficient of the live-line working robot. The control level symbol of the live-line working robot is determined based on the space encroachment result; the negative correlation normalization result of the response control coefficient is weighted using a preset gain coefficient, and the weighted result is limited by adding a preset basic control level; combined with the control level symbol, the control level is determined to control the live-line working robot.

9. The method for remote control of a live-line working robotic arm based on virtual reality according to claim 8, characterized in that, The method for determining the control level symbol includes: When the target object occupies space, the control level sign is set to negative; when the target object does not occupy space, the control level sign is set to positive.

10. A remote control system for a live-line working robotic arm based on virtual reality, the system comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the virtual reality-based remote control method for a live-line working robotic arm as described in any one of claims 1 to 9.

Images