Visual and tactile intelligent adoption control method for mechanical arm of extraterrestrial non-cooperative target

Abstract

According to the visual and tactile intelligent adoption control method for the mechanical arm of the extraterrestrial non-cooperative target, information obtained through camera observation and tactile information obtained after approaching calibration are fused; a target point cloud model of a non-cooperative target, a mechanical arm telemetry data pose model and a mechanical arm telemetry data correction model are established respectively, intelligent fusion of contact type remote sensing information and image information is utilized, model parameters are iteratively updated, accurate control over the mechanical arm relative to the non-cooperative target is achieved, and the mechanical arm telemetry accuracy is improved. And the error direction of the follow-up mechanical arm on the non-cooperative operation condition is evaluated, and the control precision is further improved.

Description

Technical Field

[0001] The present invention relates to a visual-tactile intelligent control method for a robot arm of an extraterrestrial non-cooperative target, and belongs to the field of information fusion intelligent control solution in a non-terrestrial gravity environment. Background Art

[0002] For aerospace missions with strict weight constraints, the flexibility of large robotic arms cannot be ignored under the constraints of weight reduction design. However, in typical extraterrestrial environments such as lunar sampling, large flexible robotic arms have the following difficulties in high-precision operations on non-cooperative targets:

[0003] (1) The accuracy of modeling the unknown non-cooperative target position in an unknown lighting environment is uncertain. Due to the uncertainty of the lighting environment and the unknown target texture information, there is uncertainty in the information of the target shape and position determined by image 3D reconstruction.

[0004] (2) The deformation of a large multi-degree-of-freedom flexible manipulator in a non-Earth gravity environment cannot be ignored due to the gravity difference. Therefore, the end-of-arm posture obtained by analyzing only the joint angle telemetry information will be biased. At the same time, due to the complex configuration of the multi-degree-of-freedom manipulator and the uncertainty of the spacecraft's relative posture to the operating target, the deformation compensation analysis of the manipulator is more complicated.

[0005] (3) The backlash of each joint motion of the multi-degree-of-freedom robot and the uncertainty of the interaction force after the robot contacts the target also have a significant impact on high-precision control. Summary of the invention

[0006] The technical problem solved by the present invention is: with respect to the existing technology, a method is proposed.

[0007] The present invention solves the above technical problems by the following technical solutions:

[0008] A visual-tactile intelligent control method for a robot arm of an extraterrestrial non-cooperative target, comprising:

[0009] Establish a non-cooperative target point cloud model and a robotic arm execution model, collect stereo image information of the non-cooperative target and substitute it into the non-cooperative target point cloud model, obtain the observed point cloud data of the non-cooperative target, and calculate the real point cloud data of the non-cooperative target based on the observed point cloud data;

[0010] Collect the stereoscopic image information of the robot arm and substitute it into the robot arm execution model to obtain the telemetry position and posture data of the robot arm, and calculate the real position and posture data of the robot arm according to the robot arm execution model;

[0011] A contact constraint function is established based on the real position and posture data of the robot arm to judge the contact relationship. If a touch occurs, a position and posture model of the robot arm in the touch state is established. Otherwise, the stereo image information of the non-cooperative target is re-collected and the tactile sensor is readjusted.

[0012] Using the actual position and posture data of the robot arm actuator output by the position and posture model of the robot arm in the contact state, the deviation between the telemetry data and the actual data of the robot arm during the contact process is calculated. According to the deviation value, a relationship model between the robot arm's visual posture and the actual posture, a relationship model between the robot arm's telemetry posture and the visual posture, and a model for calculating the force and deformation of the robot arm are established.

[0013] The real position of the target point is selected from the real point cloud data of the non-cooperative target, and the moving target position and target posture of the robotic arm are predicted based on the force and deformation calculation model of the robotic arm;

[0014] The target position and target posture of the robot arm are substituted into the relationship model between the robot arm's telemetry posture and visual posture, and the reference position and reference posture of the robot arm are solved. The reference position and reference posture of the robot arm are used as the control target to control the robot arm to execute.

[0015] There is a position deviation between the real point cloud data of the non-cooperative target and the observed point cloud data. The calculation method of the real point cloud data of the non-cooperative target is:

[0016]

[0017] In the formula, To obtain the observation point cloud data of non-cooperative targets after stereo observation and 3D reconstruction, is the real point cloud data of the non-cooperative target, I is the unit matrix, I+M is the transformation matrix, which represents the influence of the binocular camera calibration error and modeling uncertainty, ΔX target is the position deviation.

[0018] There is a deviation between the real position and posture data of the robotic arm and the calculated position and posture data. The calculation method of the real position and posture data of the robotic arm is:

[0019] X Real =(I+M)X photo +ΔX arm =X photo +(MX photo +ΔX arm )

[0020] Θ Real =Θ photo +ΔΘ arm

[0021] In the formula, X photoThe calculated data position of the robotic arm obtained by substituting the stereoscopic image information of the robotic arm into the robotic arm execution model, Θ photo is the calculated posture data of the robot arm, X Real is the real position data of the robot arm, Θ Real is the real posture data of the robot; I is the third-order unit matrix, I+M is the transformation matrix, ΔX arm With ΔΘ arm It is the constant term deviation value of the robot arm position and posture.

[0022] The method for judging the contact relationship is:

[0023] Establish the contact constraint function between the robot arm tactile sensor and the non-cooperative target during the touch process;

[0024] The contact constraint function is used to judge the contact force between the tactile sensor and the target. If the contact constraint is satisfied, the actual contact position between the tactile sensor and the non-cooperative target is calculated and a contact signal is triggered. Otherwise, no calculation is performed and the next contact is waited for.

[0025] When the contact signal is triggered, the robot arm movement is stopped, the stereoscopic image information of the robot arm and the actual contact position and the collected telemetry information are obtained, and the telemetry position and telemetry posture of the robot arm actuator, and the real position and real posture of the robot arm actuator are determined;

[0026] Among them, the position and posture model of the robot arm in the touching state is established according to the telemetry posture and real posture of the robot arm actuator.

[0027] The contact constraint function between the manipulator tactile sensor and the non-cooperative target is:

[0028]

[0029] In the formula, is the actual position of the reference point of the robot arm actuator when the touch occurs, The actual posture of the reference point of the robot arm actuator when the touch occurs, A is the relative position between the known tactile sensor installation position and the tactile sensor measurement point, f is the known relative relationship between the reference point of the robot arm actuator and the tactile sensor installation position, is the actual contact position, ΔX(F) is the sensor deformation, ΔX target (F) is the deformation of the non-cooperative target, which is determined based on the contact force between the tactile sensor and the target.

[0030] The position and posture model of the robot arm in the contact state is:

[0031]

[0032] In the formula, Δphoto is the position data determined by the tactile sensor according to the stereo image information and the real data during the contact process, and the deviation of the real position data, Δ Tel The deviation between the posture data and the real posture data determined by the non-cooperative target during the contact process based on the telemetry information and the real data. and In the figure, they are the real position and real posture of the robot arm actuator measured by the image under the touch state. and They are the position and posture of the robot arm according to the telemetry data in the touch state, ΔX Tel (F) and ΔΘ Tel (F) The contact force F is the correction amount for the calculated position and calculated posture after the robot arm is deformed.

[0033] The relationship model between the visual posture and the real posture of the robot arm is:

[0034]

[0035] Where M is the transformation matrix describing the position based on the image solution and the actual deviation, representing the influence of the binocular camera calibration error and the uncertainty of the 3D modeling algorithm, ΔX target is the position deviation of the camera calibration when observing a non-cooperative target and the constant term deviation caused by the uncertainty of the target optical characteristics, ΔX arm is the position deviation of the camera calibration for the observed manipulator and the constant term deviation caused by the uncertainty of the target optical characteristics. Considering the difference between the optical characteristics of the manipulator and the non-cooperative target, ΔX arm With ΔX target To distinguish, M, ΔX target With ΔX arm The numerical values ​​are obtained by solving the touch test, and the initial values ​​are all 0.

[0036] The relationship model between the telemetry posture and visual posture of the robot arm is:

[0037]

[0038] Where M T To describe the transformation matrix based on the image-based position and the telemetry position deviation, To solve the image to get the constant deviation of the robot arm position and the robot arm position obtained by telemetry, The constant term deviation between the robot arm posture calculated from the image and the robot arm posture obtained by telemetry; It is the telemetry value of the position and posture of the robot arm in the non-contact state; The position and attitude of the robot arm are calculated from the images at the same time as the telemetered position and attitude in the non-contact state.

[0039] The force deformation calculation model of the robotic arm is:

[0040]

[0041] In the formula, is the telemetry value of the position and posture of the robot arm in the touch state, The position and attitude of the robot arm are calculated by the image at the same time as the telemetry position and attitude in the touch state, Δ photo , Δ Tel The deviation of the touch process is calculated for the image data and the telemetry data, which can be calculated according to the touch model. 1 、F 2 is the geometric relationship coefficient between the robot arm and the non-cooperative target during the touch process, which can be calculated based on the relative position of the robot arm and the tactile sensor and the touch posture, and finally the position and posture deformation ΔX of the robot arm relative to the telemetry value is solved. Tel (F) and ΔΘ Tel (F).

[0042] After selecting the actual position of the target point, a weight value is assigned to the selected target point according to the historical touch control test data. After performing a specified number of touch control tests, all test data are statistically analyzed according to the corresponding weight value of the actual position of the selected target point to obtain statistical results for subsequent analysis.

[0043] The advantages of the present invention compared with the prior art are:

[0044] The present invention provides a visual-tactile intelligent control method for a robotic arm of an extraterrestrial non-cooperative target. The method is not affected by gravity differences and deformation of the robotic arm. There is uncertainty in visual measurement of uncertain targets in complex lighting environments. By fusing visual measurements and tactile information of non-cooperative targets and gradually carrying out intelligent corrections during the task, the method can assist in achieving relatively high-precision control of the robotic arm relative to the non-cooperative target. The method is not affected by multiple error sources such as gravity differences, has high reliability, and has high control accuracy. The method can assist in high-precision control of the robotic arm for non-cooperative targets in extraterrestrial gravity environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A schematic diagram of the deformation traces of the lunar surface caused by touching the moon during the sampling process provided by the present invention;

[0046] Figure 2 A schematic diagram of the relationship between intelligent adjustment parameters of a robotic arm based on the fusion of vision and touch provided by the present invention;

[0047] Figure 3 A schematic diagram of establishing a terrain model based on binocular camera images provided by the present invention;

[0048] Figure 4 A schematic diagram of the modeling of the relative target position of the robot arm and the movement amount of the robot arm provided by the present invention;

[0049] Figure 5 Schematic diagram of intelligent sampling auxiliary control of the robotic arm provided by the present invention. DETAILED DESCRIPTION

[0050] A visual-tactile intelligent control method for a robotic arm of an extraterrestrial non-cooperative target is adopted. By fusing camera visual information and tactile information obtained after proximity calibration, a target point cloud model of the non-cooperative target, a position model of the robotic arm, and a telemetry data correction model of the robotic arm are respectively established in a high-precision operation scenario in a non-terrestrial gravity environment. By intelligently fusing image information and contact remote sensing information, the model parameters are iteratively updated to achieve precise control of the robotic arm relative to the non-cooperative target, and the error direction of the subsequent robotic arm's evaluation of the non-cooperative operation situation is further improved.

[0051] A high-precision intelligent sampling auxiliary control method for the robotic arm of an extraterrestrial non-cooperative target intelligently integrates visual and tactile information to achieve high-precision control of the robotic arm.

[0052] For a multi-DOF manipulator with n joints, the joint angle telemetry data θ 1 ,θ 2 ...θ n And the geometric parameters of the robot arm l 1 …… k The calculated position X represents the end actuator position of the robot arm. TEM With posture Θ TEM Reference point data:

[0053] X Tel =f X (θ 1 ,...,θ n ,l 1 ,...l k )

[0054] Θ Tel =f Θ (θ 1 ,...,θ n ,l 1 ,...l k )

[0055] Here the superscript tel indicates that the position X and the posture Θ are obtained through telemetry of the robot arm joint.

[0056] In the general robot arm control task, the target position X and posture θ are calculated by the robot arm planning algorithm, which is the corresponding joint angle data θ 1,θ 2 ...θ n , and then control the robotic arm to reach the target position and posture.

[0057] In tasks involving non-cooperative target manipulation, it is necessary to measure the spatial position information of the non-cooperative target during the task. Here, the point cloud set {X i} target To describe, the manipulation of the robot arm for non-cooperative targets is essentially to adjust the robot arm relative to the target area {X i} target In order to obtain the position X and attitude Θ of the reference point of the robot arm, and the target area {X i} target In order to obtain information and ensure the accuracy and stability of the results, the following issues need to be addressed during the implementation of the on-orbit mission:

[0058] 1) Establish the relationship between the point cloud of the target area and the coordinate system of the robot arm control.

[0059] The point cloud information of the target area in the camera coordinate system can be obtained through visual measurement and three-dimensional reconstruction. The conversion relationship between the robot coordinate system and the camera coordinate system is obtained through prior calibration on the ground, and then the coordinates of the target area point cloud in the robot coordinate system are calculated, and the relationship between the target point cloud and the position and posture of the robot reference point is established.

[0060] Therefore, when configuring the binocular camera, it is necessary to calibrate the camera relative to the robot coordinate system on the ground, and then obtain the coordinate value of the target point in the robot coordinate system. All position and posture information in the following text uses the robot coordinate system.

[0061] 2) Non-cooperative target position measurement uncertainty

[0062] The location and shape information of the non-cooperative target area (described by point cloud) needs to be obtained through binocular camera stereo measurement + 3D reconstruction technology. The sources of error in this process include:

[0063] ① Camera calibration parameter error. The camera’s internal and external parameter errors will lead to systematic deviations in the reconstruction results, which can be controlled through high-precision ground calibration.

[0064] ② Occlusion caused by lighting uncertainty, target feature uncertainty, and complex shapes of the target area will bring uncertainty to 3D modeling. Advanced 3D reconstruction algorithms can reduce reconstruction errors, but cannot specifically eliminate the errors caused by uncertain lighting and targets.

[0065] 3) Uncertainty of the reference point position and posture of the robot arm actuator

[0066] In an extraterrestrial environment, sources of position and attitude errors of the reference point of the manipulator actuator relative to the telemetry data prediction include:

[0067] ① Errors caused by differences in gravity environments in non-ground environments. This includes errors caused by deformation of joints and robotic arm rods due to changes in gravity environments; errors caused by uncertainty in the direction of gravity due to attitude uncertainties such as spacecraft landing.

[0068] ② The return difference of the mechanism movement. For a multi-joint robot, the return difference of each joint is considered, and the actual state of the robot is also related to the movement history of the robot.

[0069] ③ Other force effects on the manipulator. The deformation caused by the force when the manipulator interacts with the target is mainly a recoverable elastic deformation, which can be restored by removing the external force. Consider the impact of the manipulator on the joint return difference state after the force is applied to the manipulator. If the return difference state changes, the local deformation may not be completely restored after the external force is removed.

[0070] In order to eliminate the modeling error of non-cooperative targets and the error of position and attitude control of the flexible robotic arm calculated based on joint angle telemetry parameters, the visual and tactile information are integrated, the relationship between multi-source data is established, the error of each link is eliminated, and the precise control of the robotic arm relative to the non-cooperative target is achieved.

[0071] The actual position and posture of the robot arm are recorded as X Real With Θ Real The reference point position and posture of the robot arm obtained based on telemetry are recorded as X Tel With Θ Tel , which differs from the actual state as follows:

[0072] X Real =X Tel +ΔX(X Tel ,Θ Tel ,gravity)+ΔX(path)

[0073] Θ Real =Θ Tel +Δθ(X Tel ,Θ Tel ,gravity)+ΔΘ(path)

[0074] Where ΔX(X Real ,Θ Real , gravity) and ΔΘ(X Real ,Θ Real , gravity) is used to describe the state of the robot and the error caused by the gravity environment, ΔX(path) and ΔΘ(path) describe the return error with the motion history of the multi-joint robot. Based on the above technology, a control method using telemetry data is proposed.

[0075] The following is further described in conjunction with the accompanying drawings and preferred embodiments of the specification:

[0076] In the current embodiment, the process of the non-cooperative target robotic arm telemetry data sampling control method is as follows: Figure 5 As shown:

[0077] (I) First, establish the target point cloud model of the non-cooperative target and the end of the robotic arm:

[0078] The target area point cloud model of the non-cooperative target is obtained through image 3D modeling. The non-cooperative model modeling example is as follows Figure 3 As shown, the deviation between modeling and reality can be described as:

[0079]

[0080] (1) In the formula, To obtain the point cloud position information of non-cooperative targets after stereo observation and 3D reconstruction; is the true position of the point cloud data of the non-cooperative target; I is the third-order unit matrix, I+M is the transformation matrix, which represents the influence of the binocular camera calibration error and the uncertainty of the 3D modeling algorithm, ΔX target It is the position deviation of the camera calibration when observing a non-cooperative target and the constant term deviation caused by the uncertainty of the target's optical characteristics.

[0081] After the target point cloud model of the non-cooperative target is established, the stereo image information of the non-cooperative target obtained by stereo observation is used as input to obtain the real point cloud data of the non-cooperative target;

[0082] Similarly, if the robotic arm appears in the camera field of view, the three-dimensional model of the robotic arm can be established through the image information and the position and posture of the robotic arm can be solved. The example of modeling the robotic arm actuator is as follows: Figure 4 As shown, the image solution results are as follows:

[0083] X Real =(I+M)X photo +ΔX arm =X photo +(MX photo +ΔX arm ) (2)

[0084] Θ Real =Θ photo +ΔΘ arm (3)

[0085] In formulas (2) and (3), X photo The coordinates of the reference point of the robot arm are obtained after stereo observation and 3D reconstruction, Θ photoX is the posture angle of the actuator of the robot arm after solving; Real is the real coordinate of the robot reference point, Θ Real is the actual posture angle of the robot arm actuator; I, I+M are the same as (1) ΔX arm With ΔΘ arm is the constant term deviation of the position and posture of the robot arm. Considering the difference between the optical characteristics of the robot arm and the non-cooperative target, ΔX arm With ΔX target Make a distinction.

[0086] (ii) Secondly, establish the posture model of the robot arm in the contact state;

[0087] During the touch process, the tactile sensor can be triggered only when the sensor contacts the target and the contact force exceeds F. Without considering the deformation, when the tactile sensor touches the non-cooperative target and is triggered, the geometric constraint function G is used to express it:

[0088]

[0089] in, is the actual position of the reference point of the robot arm actuator when the touch occurs, The actual posture of the reference point of the robot arm actuator when the touch occurs, The actual position of the non-cooperative target at the touch location. The form of the geometric constraint function G is determined by the actual shape of the tactile sensor and the geometric shape of the non-cooperative target surface. After considering the deformation factor, the above formula can be modified to obtain:

[0090]

[0091] Where, the contact force F is determined by the sensor threshold and is a known quantity; ΔX(F) is the sensor deformation caused by the contact force F. Whether it can be ignored depends on the hardness of the sensor. Usually, it can be ignored. target (F) is the target deformation caused by the contact force F. The target deformation can be referred to Figure 1 The ground is touched by the robot arm and produces an indentation. Here, the sensor deformation variable ΔX(F) can be calculated by the sensor deformation model. It is usually small and can be ignored. The target deformation variable ΔX target (F) can be obtained through actual measurement through image 3D modeling.

[0092] The geometric constraint G is essentially the contact relationship between the two surfaces of the tactile sensor and the non-cooperative target. After solving the contact point (touch point, marked with the subscript tp) between the tactile sensor and the non-cooperative target:

[0093]

[0094] Where A represents the relative position between the installation position of the tactile sensor and the measurement point of the tactile sensor, which is a known quantity; f is used to calculate the relative relationship between the reference point of the robot arm actuator and the installation position of the tactile sensor, which is a known relationship; is the actual contact position. After combining the two, we can get:

[0095]

[0096] During the tactile sensor proximity calibration process, the docking signal is monitored in real time. When the contact signal is triggered, the robot arm movement is stopped immediately, and the graphic information and telemetry information of the robot arm and the contact are obtained. The image information is used to obtain the reference point position of the robot arm actuator With posture The reference point position of the manipulator actuator obtained from telemetry information With posture Since the force F at the time of contact causes the robot arm to deform, the telemetry needs to be corrected, which is recorded as and

[0097] In actual tasks, the real position and posture of the robot arm are unknown. Considering that the image measurement results and telemetry results are different from the real values, After substituting into (4), the right side of the equation is not 0, and we can get:

[0098]

[0099] Through (5) and (6), we can calculate the deviation term Δ generated by the visual measurement pose and the telemetry pose during the contact process after the tactile sensor contacts and is triggered by the non-cooperative target: photo With Δ Tel , the visual measurement error and telemetry error are small relative to the measurement target, and (5) and (6) can be linearized to obtain:

[0100]

[0101] Visual measurement error and telemetry error are small relative to the measurement target and are ignored in further simplification. and The difference is that the visual measurement results are closer to the actual situation, so the unified and make: Simplified to:

[0102]

[0103] (iii) Again, based on the binocular image, the robot arm information, the non-cooperative target information (including deformation information after touch), the robot arm telemetry information and the deformation model described in (ii) after the tactile sensor is triggered (described by (5) to (8)), establish a relationship model between the posture in the visual information and the actual posture, a relationship model between the telemetry and visual information, and the force deformation of the robot arm.

[0104] 1) Model of the relationship between the pose in visual information and the actual pose

[0105] Substituting (1)(2)(3) into (7), we can obtain:

[0106]

[0107] In (9), F 1 、F 2 , As known numbers, M and ΔX can be solved by multiple touch tests. target With ΔX target If only a few touch tests are performed, it is considered that M, ΔX target With ΔΘ arm 0, prioritize calculation of ΔX arm ; As the number of touches increases, ΔX is calculated step by step target , ΔΘ arm With M.

[0108] 2) Telemetry and visual information relationship model

[0109] According to (2) and (3), the reference point telemetry of the robot arm actuator and the image information satisfy the relationship:

[0110]

[0111] Since the robot arm has no additional force in this process, the process must be carried out in a non-contact process to ensure that the robot arm is not subjected to additional force. The corresponding Substituting it into (10) and (11), we can calculate M T and The priority calculation and As the number of trials increases, M is calculated step by step. T .

[0112] 3) Calculation of force and deformation of the robotic arm

[0113] Subtracting (8) from (7) yields:

[0114]

[0115] Among them, F 1 、F 2 , All are known, and ΔX can be solved Tel (F) and ΔΘ Tel (F).

[0116] (iv) Finally, based on the relationship model between the robot arm posture in the visual information and the real posture, the relationship model between the telemetry and visual information, and the force deformation of the robot arm, the control amount of the robot arm arriving at the new target point is predicted;

[0117] For new target points (The target point is selected based on the point non-cooperative target cloud model, so it is the result of visual modeling, not the real position). Using the relationship between the target point cloud and the actual deviation (1) in (I), the real position of the target point is calculated. According to the mechanical shape and the force deformation of the robot arm (the calculation result of the force deformation of the robot arm in (III)), the target position of the robot arm is predicted. With posture According to the telemetry and visual information relationship model in (III), it is solved as the telemetry target position With posture This is the target value required for the robot arm control. Figure 2 As shown. Control the robotic arm to complete the modeling of the relative position of the robotic arm to the target and the movement of the robotic arm as shown Figure 4 shown.

[0118] It should be noted that the model in this paper is linearly simplified to a certain extent. The model has good characteristics, so the data of the area near the target location is more valuable for reference. Therefore, a weight coefficient for the target point is added: Assume that the history has conducted k tests, and the distance between these test points and the target point is r k , then the weight of this test is This coefficient improves the contribution value of the neighbor area test results and maximizes the use of previous experience.

[0119] Visual measurement technology is not affected by gravity differences and deformation of the robotic arm, but there is uncertainty in the visual measurement of uncertain targets in complex lighting environments. By integrating the visual measurement and tactile information of non-cooperative targets and gradually carrying out intelligent corrections during the task, it can assist in achieving relatively high-precision control of the robotic arm relative to non-cooperative targets. This solution has the advantages of being unaffected by multiple error sources such as gravity differences, high reliability, and high control accuracy. It can assist in high-precision control of the robotic arm for non-cooperative targets in extraterrestrial gravity environments.

[0120] Although the present invention has been disclosed as above in the form of a preferred embodiment, it is not intended to limit the present invention. Any person skilled in the art may make possible changes and modifications to the technical solution of the present invention by using the methods and technical contents disclosed above without departing from the spirit and scope of the present invention. Therefore, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention shall fall within the protection scope of the technical solution of the present invention.

[0121] The contents not described in detail in the specification of the present invention belong to the common knowledge of those skilled in the art.

Claims

1. A visual tactile intelligent control method for a robot arm of an extraterrestrial non-cooperative target, characterized in that include: Establish a non-cooperative target point cloud model and a robotic arm execution model, collect stereo image information of the non-cooperative target and substitute it into the non-cooperative target point cloud model, obtain the observed point cloud data of the non-cooperative target, and calculate the real point cloud data of the non-cooperative target based on the observed point cloud data; Collect the stereoscopic image information of the robot arm and substitute it into the robot arm execution model to obtain the telemetry position and posture data of the robot arm, and calculate the real position and posture data of the robot arm according to the robot arm execution model; A contact constraint function is established based on the real position and posture data of the robot arm to judge the contact relationship. If a touch occurs, a position and posture model of the robot arm in the touch state is established. Otherwise, the stereo image information of the non-cooperative target is re-collected and the tactile sensor is readjusted. Using the actual position and posture data of the robot arm actuator output by the position and posture model of the robot arm in the contact state, the deviation between the telemetry data and the actual data of the robot arm during the contact process is calculated. According to the deviation value, a relationship model between the robot arm's visual posture and the actual posture, a relationship model between the robot arm's telemetry posture and the visual posture, and a model for calculating the force and deformation of the robot arm are established. The real position of the target point is selected from the real point cloud data of the non-cooperative target, and the moving target position and target posture of the robotic arm are predicted based on the force and deformation calculation model of the robotic arm; The target position and target posture of the robot arm are substituted into the relationship model between the robot arm's telemetry posture and visual posture, and the reference position and reference posture of the robot arm are solved. The reference position and reference posture of the robot arm are used as the control target to control the robot arm to execute.

2. The visual tactile intelligent control method for a robot arm of an extraterrestrial non-cooperative target according to claim 1, characterized in that: There is a position deviation between the real point cloud data of the non-cooperative target and the observed point cloud data. The calculation method of the real point cloud data of the non-cooperative target is: In the formula, To obtain the observation point cloud data of non-cooperative targets after stereo observation and 3D reconstruction, is the real point cloud data of the non-cooperative target, I is the unit matrix, I+M is the transformation matrix, which represents the influence of the binocular camera calibration error and modeling uncertainty, ΔX target is the position deviation.

3. The visual tactile intelligent control method for a robot arm of an extraterrestrial non-cooperative target according to claim 1, characterized in that: There is a deviation between the real position and posture data of the robotic arm and the calculated position and posture data. The calculation method of the real position and posture data of the robotic arm is: X Real =(I+M)X photo +ΔX arm =X photo +(MX photo +ΔX arm ) I Real =Θ photo +DTH arm Where, X photo The calculated data position of the robotic arm obtained by substituting the stereoscopic image information of the robotic arm into the robotic arm execution model, Θ photo is the calculated posture data of the robot arm, X Real is the real position data of the robot arm, Θ Real is the real posture data of the robotic arm; I is the third-order unit matrix, I+M is the transformation matrix, ΔX arm With ΔΘ arm It is the constant term deviation value of the robot arm position and posture.

4. The visual tactile intelligence control method for a robot arm of an extraterrestrial non-cooperative target according to claim 1, characterized in that: The method for judging the contact relationship is: Establish the contact constraint function between the robot arm tactile sensor and the non-cooperative target during the touch process; The contact constraint function is used to judge the contact force between the tactile sensor and the target. If the contact constraint is satisfied, the actual contact position between the tactile sensor and the non-cooperative target is calculated and a contact signal is triggered. Otherwise, no calculation is performed and the next contact is waited for. When the contact signal is triggered, the robot arm movement is stopped, the stereoscopic image information of the robot arm and the actual contact position and the collected telemetry information are obtained, and the telemetry position and telemetry posture of the robot arm actuator, and the real position and real posture of the robot arm actuator are determined; Among them, the position and posture model of the robot arm in the touching state is established according to the telemetry posture and real posture of the robot arm actuator.

5. The visual tactile intelligence control method for a robot arm of an extraterrestrial non-cooperative target according to claim 4, characterized in that: The contact constraint function between the manipulator tactile sensor and the non-cooperative target is: In the formula, is the actual position of the reference point of the robot arm actuator when the touch occurs, The actual posture of the reference point of the robot arm actuator when the touch occurs, A is the known relative position between the tactile sensor installation position and the tactile sensor measurement point, f is the known relative relationship between the reference point of the robot arm actuator and the tactile sensor installation position, is the actual contact position, ΔX(F) is the sensor deformation, ΔX target (F) is the deformation of the non-cooperative target, which is determined based on the contact force between the tactile sensor and the target.

6. The visual tactile intelligence control method for a robot arm of an extraterrestrial non-cooperative target according to claim 4, characterized in that: The position and posture model of the robot arm in the touch state is: In the formula, Δ photo is the position data determined by the tactile sensor according to the stereo image information and the real data during the contact process, and the deviation of the real position data, Δ Tel The deviation between the posture data and the real posture data determined by the non-cooperative target during the contact process based on the telemetry information and the real data. and In the figure, they are the real position and real posture of the robot arm actuator measured by the image under the touch state. and They are the position and posture of the robot arm according to the telemetry data in the touch state, ΔX Tel (F) and ΔΘ Tel (F) The contact force F is the correction amount for the calculated position and calculated posture after the robot arm is deformed.

7. The visual tactile intelligence control method for a robot arm of an extraterrestrial non-cooperative target according to claim 4, characterized in that: The relationship model between the visual posture and the real posture of the robot arm is: Where M is the transformation matrix describing the position based on the image solution and the actual deviation, representing the influence of the binocular camera calibration error and the uncertainty of the 3D modeling algorithm, ΔX target is the position deviation of the camera calibration when observing a non-cooperative target and the constant term deviation caused by the uncertainty of the target optical characteristics, ΔX arm is the position deviation of the camera calibration for the observed manipulator and the constant term deviation caused by the uncertainty of the target optical characteristics. Considering the difference between the optical characteristics of the manipulator and the non-cooperative target, ΔX arm With ΔX target To distinguish, M, ΔX target With ΔX arm The numerical values ​​are obtained by solving the touch test, and the initial values ​​are all 0.

8. The visual tactile intelligence control method for a robot arm of an extraterrestrial non-cooperative target according to claim 7, characterized in that: The relationship model between the telemetry posture and visual posture of the robot arm is: Where M T To describe the transformation matrix based on the image-based position and the telemetry position deviation, To solve the image to get the constant deviation of the robot arm position and the robot arm position obtained by telemetry, The constant term deviation between the robot arm posture calculated from the image and the robot arm posture obtained by telemetry; It is the telemetry value of the position and posture of the robot arm in the non-contact state; The position and attitude of the robot arm are calculated from the images at the same time as the telemetered position and attitude in the non-contact state.

9. The visual tactile intelligence control method for a robot arm of an extraterrestrial non-cooperative target according to claim 8, characterized in that: The force deformation calculation model of the robotic arm is: In the formula, is the telemetry value of the position and posture of the robot arm in the touch state, The position and attitude of the robot arm are calculated by the image at the same time as the telemetry position and attitude in the touch state, Δ photo , Δ Tel The deviation of the touch process is calculated for the image data and the telemetry data, which can be calculated according to the touch model. F1 and F2 are the correlation coefficients of the geometric relationship between the manipulator and the non-cooperative target during the touch process, which can be calculated according to the relative position of the manipulator and the tactile sensor and the touch posture. Finally, the position and posture deformation ΔX of the manipulator relative to the telemetry value is solved. Tel (F) and ΔΘ Tel (F).

10. The visual tactile intelligence control method for a robot arm of an extraterrestrial non-cooperative target according to claim 1, characterized in that: After selecting the actual position of the target point, a weight value is assigned to the selected target point according to the historical touch control test data. After performing a specified number of touch control tests, all test data are statistically analyzed according to the corresponding weight value of the actual position of the selected target point to obtain statistical results for subsequent analysis.

Images