An extraterrestrial non-cooperative target mechanical arm visual tactile intelligence control method

Abstract

The application discloses a control method for a mechanical arm vision-tactile intelligence of an extraterrestrial non-cooperative target, which is characterized by the following steps: fusing camera observation information and tactile information obtained after approaching calibration; in a high-precision operation occasion in a non-ground gravity environment, establishing a target point cloud model of the non-cooperative target, a mechanical arm remote sensing data pose model and a mechanical arm remote sensing data correction model; intelligently fusing contact remote sensing information and image information; iteratively updating model parameters; and realizing accurate control of the mechanical arm relative to the non-cooperative target, and evaluating an error direction of the operation of the mechanical arm on the non-cooperative target, so as to further improve control precision.

Description

Technical Field

[0001] This invention relates to a visual and tactile intelligent control method for a robotic arm targeting non-cooperative extraterrestrial targets, belonging to the field of information fusion intelligent control and computation in non-terrestrial gravity environments. Background Technology

[0002] For aerospace missions with strict weight constraints, the flexibility of large robotic arms cannot be ignored under the constraints of weight reduction design. In typical extraterrestrial environments such as lunar surface sampling, large flexible robotic arms face the following challenges in high-precision operation against non-cooperative targets:

[0003] (1) The modeling accuracy of unknown non-cooperative target locations under unknown lighting conditions is uncertain. Due to the uncertainty of the lighting environment and the unknown target texture information, the information for determining the target shape and location through image 3D reconstruction is uncertain.

[0004] (2) Due to gravity differences, the deformation of large multi-degree-of-freedom flexible robotic arms in non-Earth gravity environments cannot be ignored. Therefore, relying solely on telemetry information of joint angles to analyze the end pose of the robotic arm will result in deviations. At the same time, due to the complex configuration of multi-degree-of-freedom robotic arms and the uncertainty of the attitude of the spacecraft relative to the target, the deformation compensation analysis of the robotic arm is more complicated.

[0005] (3) The motion backlash of each joint of the multi-degree-of-freedom robotic arm and the uncertainty of the interaction force after the robotic arm comes into contact with the target also have a significant impact on high-precision control. Summary of the Invention

[0006] The technical problem solved by this invention is to address the shortcomings of existing technologies and propose a solution.

[0007] The present invention solves the above-mentioned technical problem through the following technical solution:

[0008] A robotic arm employing visual-tactile intelligence for non-cooperative extraterrestrial targets utilizes a control method, including:

[0009] Establish a point cloud model of a non-cooperative target and a robotic arm execution model. Collect stereo image information of the non-cooperative target and input it into the point cloud model of the non-cooperative target to obtain the observation point cloud data of the non-cooperative target. Calculate the real point cloud data of the non-cooperative target based on the observation point cloud data.

[0010] The robot arm's stereoscopic image information is collected and substituted into the robot arm execution model to obtain the robot arm's telemetry position and attitude data. The robot arm's actual position and attitude data are calculated based on the robot arm execution model.

[0011] A contact constraint function is established based on the actual position and posture data of the robotic arm to determine the contact relationship. If a contact occurs, a position and posture model of the robotic arm under the contact state is established; otherwise, the stereo image information of the non-cooperative target is re-acquired and the tactile sensor is readjusted.

[0012] Using the position and posture model of the robotic arm under the touch state to output the real position and posture data of the robotic arm actuator, the deviation value between the telemetry data and the real data of the robotic arm during the contact process is calculated. Based on the deviation value, a model of the relationship between the visual pose and the real pose of the robotic arm, a model of the relationship between the telemetry pose and the visual pose of the robotic arm, and a model for solving the force deformation of the robotic arm are established.

[0013] The actual location of the target point is selected from the real point cloud data of the non-cooperative target, and the target position and attitude of the robot arm are predicted according to the force deformation calculation model of the robot arm.

[0014] The target position and orientation of the robotic arm are substituted into the relationship model between the telemetry pose and the visual pose of the robotic arm to calculate the reference position and orientation of the robotic arm. The robotic arm is then controlled to execute the movement based on the reference position and orientation of the robotic arm.

[0015] There is a positional discrepancy between the actual point cloud data and the observed point cloud data of the non-cooperative target. The method for calculating the actual point cloud data of the non-cooperative target is as follows:

[0016]

[0017] In the formula, To obtain observation point cloud data of non-cooperative targets after stereo observation and 3D reconstruction, For real point cloud data of non-cooperative targets, I is the identity matrix, I+M is the transformation matrix, representing the influence terms of binocular camera calibration error and modeling uncertainty, ΔX target This is the positional deviation.

[0018] There is a deviation between the actual position and attitude data of the robotic arm and the calculated position and attitude data. The calculation method for the actual position and attitude data of the robotic arm is as follows:

[0019] X Real = (I+M)X photo +ΔX arm =X photo +(MX photo +ΔX arm )

[0020] Θ Real =Θ photo +ΔΘ arm

[0021] In the formula, X photoThe position of the robotic arm obtained by substituting the 3D image information of the robotic arm into the robotic arm execution model, Θ photo X is the solution for the robot arm's attitude data. Real For the actual position data of the robotic arm, Θ Real This represents the actual posture data of the robotic arm; I is the third-order identity matrix, I+M is the transformation matrix, and ΔX... arm With ΔΘ arm This represents the constant deviation value for the position and orientation of the robotic arm.

[0022] The method for determining contact relationships is as follows:

[0023] Establish the contact constraint function between the robotic arm's tactile sensor and the non-cooperative target during the touch process;

[0024] The contact force between the tactile sensor and the target is determined using a contact constraint function. If the contact constraint is satisfied, the actual contact position between the tactile sensor and the non-cooperative target is calculated and a contact signal is triggered. Otherwise, no calculation is performed, and the system waits for the next contact.

[0025] When the contact signal is triggered, the movement of the robotic arm stops, and the stereoscopic image information of the robotic arm and the actual contact position and the collected telemetry information are obtained to determine the telemetry position and telemetry attitude of the robotic arm actuator, as well as the real position and real attitude of the robotic arm actuator.

[0026] The position and posture model of the robotic arm in the touch state is established based on the telemetry posture and the actual posture of the robotic arm actuator.

[0027] The contact constraint function between the robotic arm's tactile sensor and the non-cooperative target is:

[0028]

[0029] In the formula, The actual position of the reference point of the robotic arm actuator at the time of the contact. The actual orientation of the robotic arm's reference point at the time of the touch event, where A is the known relative position between the tactile sensor's mounting location and its measurement point, and f is a known value used to calculate the relative relationship between the robotic arm's reference point and the tactile sensor's mounting location. The actual contact position is given by ΔX(F), where ΔX is the sensor deformation. target (F) represents the deformation of the non-cooperative target, determined based on the tactile sensor and the contact force of the target.

[0030] The position and attitude model of the robotic arm in the touch state is as follows:

[0031]

[0032] In the formula, Δphoto The pose data determined by the tactile sensor during contact based on stereo image information and real data, and the deviation of the actual pose data, Δ. Tel The deviation between the pose data determined by telemetry information and real data for a non-cooperative target during contact and the actual pose data. and The images in the middle show the actual position and posture of the robotic arm actuator as measured by image during the touch state. and The positions and attitudes of the robotic arm, ΔX and ΔX, are calculated based on telemetry data during the touch state. Tel (F) and ΔΘ Tel (F) The contact force F is the correction amount for the solution position and solution attitude after the robot arm is deformed.

[0033] The model relating the robotic arm's visual pose to its actual pose is as follows:

[0034]

[0035] In the formula, M is the transformation matrix describing the position calculated from the image and the actual deviation, representing the influence term of the binocular camera calibration error and the uncertainty of the 3D modeling algorithm, and ΔX target Let ΔX be the constant term deviation caused by the positional deviation of the camera calibration and the uncertainty of the target's optical characteristics when observing non-cooperative targets. arm To account for the constant term deviation caused by the positional deviation of the camera calibration during the observation of the robotic arm and the uncertainty of the target's optical characteristics, and considering the differences between the robotic arm's optical characteristics and those of the non-cooperative target, therefore, for ΔX... arm With ΔX target To distinguish between M and ΔX target With ΔX arm The values ​​were obtained by solving the touch test, and the initial values ​​were all 0.

[0036] The model relating the telemetry pose and visual pose of the robotic arm is as follows:

[0037]

[0038] In the formula, M T To describe the transformation matrix between image-based calculated position and telemetry position deviation, To calculate the constant term deviation between the robot arm position obtained from the image and the robot arm position obtained from telemetry, The constant term is used to calculate the deviation between the robot arm posture obtained from the image and the robot arm posture obtained from telemetry. These are the telemetry values ​​of the robotic arm's position and orientation in a non-contact state; The position and attitude of the robotic arm are calculated from images taken simultaneously with the telemetry position and attitude in a non-touch state.

[0039] The calculation model for the deformation of the robotic arm under force is as follows:

[0040]

[0041] In the formula, These are the telemetry values ​​of the robotic arm's position and orientation during the touch state. To calculate the position and orientation of the robotic arm from images simultaneously with the telemetry position and orientation under the touch state, Δ photo Δ Tel The deviation between image data and telemetry data during the touch process is calculated based on the touch model. F1 and F2 are correlation coefficients with the geometric relationship between the robotic arm and the non-cooperative target during the touch process, which can be calculated based on the relative position of the robotic arm and the tactile sensor, as well as the touch posture. Finally, the position and posture deformation ΔX of the robotic arm relative to the telemetry value is calculated. Tel (F) and ΔΘ Tel (F).

[0042] After selecting the actual location of the target point, a weight value is assigned to the selected target point based on historical touch control test data. After conducting a specified number of touch control tests, all test data are statistically analyzed based on the corresponding weight value of the actual location of the selected target point, and the statistical results are used for subsequent analysis.

[0043] The advantages of this invention compared to the prior art are:

[0044] This invention provides a visual-tactile intelligent control method for robotic arms targeting non-cooperative targets on extraterrestrial surfaces. This method is unaffected by gravity differences or robotic arm deformation. Addressing the uncertainty in visual measurements of uncertain targets under complex lighting conditions, this invention integrates visual measurements and tactile information from the non-cooperative target and performs intelligent corrections progressively during the task. This method assists in achieving relatively high-precision control of the robotic arm relative to the non-cooperative target. It boasts advantages such as being unaffected by multiple error sources including gravity differences, high reliability, and high control precision, enabling high-precision control of robotic arms targeting non-cooperative targets in extraterrestrial gravity environments. Attached Figure Description

[0045] Figure 1 A schematic diagram of the deformation marks on the lunar surface caused by contact with the moon during the sampling process provided by this invention;

[0046] Figure 2 A schematic diagram illustrating the intelligent adjustment parameter relationship of a robotic arm based on the fusion of vision and touch, provided by this invention.

[0047] Figure 3 This is a schematic diagram of a terrain model built based on binocular camera images provided by the present invention;

[0048] Figure 4A schematic diagram illustrating the modeling of the robotic arm's position relative to the target and the amount of movement of the robotic arm, provided by the present invention;

[0049] Figure 5 This is a schematic diagram of the intelligent sampling auxiliary control of the robotic arm provided by the present invention. Detailed Implementation

[0050] A vision-tactile intelligent control method for extraterrestrial non-cooperative targets employs a fusion of camera visual information and tactile information obtained after proximity calibration. In high-precision operation scenarios outside of ground gravity, a target point cloud model of the non-cooperative target, a pose model of the robotic arm, and a correction model of the robotic arm's telemetry data are established. By intelligently fusing image information with contact-based remote sensing information and iteratively updating model parameters, the robotic arm can achieve precise control relative to the non-cooperative target and subsequently evaluate the error direction of the robotic arm's operation on the non-cooperative target, further improving control accuracy.

[0051] A high-precision intelligent sampling-assisted control method for robotic arms targeting non-cooperative extraterrestrial targets is proposed, which intelligently integrates visual and tactile information to achieve high-precision control of the robotic arm.

[0052] For a multi-degree-of-freedom robotic arm with n joints, the joint angle telemetry data θ1, θ2...θ3 can be used to... n And the geometric parameters l1……l of the robotic arm k The position X representing the end effector of the robotic arm is calculated. TEM With posture Θ TEM Reference point data:

[0053] X Tel =f X (θ1,...,θ n ,l1,...l k )

[0054] Θ Tel =f Θ (θ1,...,θ n ,l1,...l k )

[0055] The superscript tel here indicates that the position X and attitude Θ are obtained through telemetry of the robotic arm joints.

[0056] In general robotic arm control tasks, the target position X and attitude Θ are calculated using a robotic arm planning algorithm, which corresponds to the joint angle data θ1, θ2, ... θ3. n This allows the robotic arm to be controlled to reach the target position and orientation.

[0057] In tasks involving non-cooperative targets, it is necessary to measure the spatial location information of the non-cooperative targets during the task. Here, a point cloud set {X} is used.i} target To describe it, the essence of manipulating a robotic arm against a non-cooperative target is adjusting the robotic arm relative to the target area {X}. i} target The relative position and orientation of the robotic arm reference point are determined in order to obtain the position X and orientation Θ of the reference point, as well as the target area {X}. i} target To obtain the necessary information and ensure the accuracy and stability of the results, the following issues need to be addressed during the implementation of on-orbit missions:

[0058] 1) Establish the relationship between the point cloud of the target area and the coordinate system controlled by the robotic arm.

[0059] Visual measurement and 3D reconstruction can be used to obtain point cloud information of the target area in the camera coordinate system. The ground is pre-calibrated to obtain the transformation relationship between the robot arm coordinate system and the camera coordinate system, and then the coordinates of the target area point cloud in the robot arm coordinate system are calculated to establish the relationship between the position and attitude of the target point cloud and the robot arm reference point.

[0060] Therefore, when configuring a stereo camera, it is necessary to calibrate the camera relative to the robotic arm's coordinate system on the ground to obtain the target point's coordinates in the robotic arm's coordinate system. All position and attitude information herein will use the robotic arm's coordinate system.

[0061] 2) Uncertainty in measuring the location of non-cooperative targets

[0062] The location and shape information of the non-cooperative target area needs to be obtained through stereo measurement with a binocular camera and 3D reconstruction technology (using point cloud description). The sources of error in this process include:

[0063] ① Camera calibration parameter errors. Errors in the camera's intrinsic and extrinsic parameters will cause systematic deviations in the reconstruction results, which can be controlled through high-precision ground calibration.

[0064] ② Uncertainty in lighting, uncertainty in target features, and occlusion caused by the complex shape of the target area will bring uncertainty to 3D modeling. Advanced 3D reconstruction algorithms can reduce reconstruction errors, but cannot specifically eliminate the errors caused by uncertain lighting and targets.

[0065] 3) Uncertainty in the position and attitude of the reference point of the robotic arm actuator

[0066] In extraterrestrial environments, the sources of position and attitude errors of the robotic arm actuator's reference point relative to telemetry data predictions include:

[0067] ① Errors arise from differences in gravity in non-terrestrial environments. This includes errors caused by deformation of joints and robotic arms due to changes in the gravity environment; it also includes errors caused by the uncertainty of gravity direction due to attitude uncertainties such as spacecraft landing.

[0068] ② Backstroke difference of the mechanism's motion. For multi-joint robotic arms, the backstroke difference of each joint must be considered. The actual state of the robotic arm is also related to its motion history.

[0069] ③ Other forces affecting the robotic arm. The deformation caused by the forces acting on the robotic arm when interacting with the target is mainly a recoverable elastic deformation, which can be restored upon removal of the external force. However, the impact of forces on the joint return error should be considered. If the return error changes, the local deformation may not be fully recovered after the external force is removed.

[0070] To eliminate errors in modeling non-cooperative targets and errors in the position and attitude control of the flexible robotic arm calculated based on joint angle telemetry parameters, visual and tactile information is integrated to establish relationships between data from multiple sources, thereby eliminating errors in each stage and achieving precise control of the robotic arm relative to non-cooperative targets.

[0071] Let X be the actual pose and orientation of the robotic arm. Real With Θ Real The position and attitude of the robotic arm reference point obtained based on telemetry are denoted as X. Tel With Θ Tel The difference between it and the actual state is as follows:

[0072] X Real =X Tel +ΔX(X Tel ,Θ Tel gravity) + ΔX(path)

[0073] Θ Real =Θ Tel +ΔΘ(X Tel ,Θ Tel ,gravity)+ΔΘ(path)

[0074] Where ΔX(X) Real ,Θ Real gravity) and ΔΘ(X Real ,Θ Real ΔX(path) and ΔΘ(path) are used to describe the state of the robotic arm and the error caused by gravity. ΔX(path) and ΔΘ(path) describe the return error of the multi-joint robotic arm's motion history. Based on the above techniques, a control method is proposed to use telemetry data.

[0075] The following description, in conjunction with the accompanying drawings and preferred embodiments, provides further details:

[0076] In the current embodiment, the process of the robotic arm telemetry data sampling and control method for non-cooperative targets is as follows: Figure 5 As shown:

[0077] (I) First, establish a point cloud model of the non-cooperative target and the end effector of the robotic arm:

[0078] Obtaining the point cloud model of the target region for a non-cooperative target through 3D image modeling; an example of non-cooperative model modeling is shown below. Figure 3 As shown, the deviation between the model and reality can be described as:

[0079]

[0080] (1) In the formula, To obtain the point cloud location information of non-cooperative targets after stereoscopic observation and 3D reconstruction; The true location of the point cloud data for non-cooperative targets; I is the third-order identity matrix, I+M is the transformation matrix, representing the influence terms of binocular camera calibration error and 3D modeling algorithm uncertainty, ΔX target This is the constant term deviation caused by the positional deviation of the camera calibration and the uncertainty of the target's optical characteristics when observing non-cooperative targets.

[0081] After establishing the target point cloud model of the non-cooperative target, the real point cloud data of the non-cooperative target is obtained by using the stereo image information of the non-cooperative target obtained from stereo observation as input.

[0082] Similarly, if the robotic arm appears in the camera's field of view, a 3D model of the robotic arm can be created using image information, and its position and orientation can be calculated. An example of modeling the robotic arm's actuator is shown below. Figure 4 As shown, the image processing results are as follows:

[0083] X Real = (I+M)X photo +ΔX arm =X photo +(MX photo +ΔX arm (2)

[0084] Θ Real =Θ photo +ΔΘ arm (3)

[0085] In equations (2) and (3), X photo To obtain the coordinates of the robotic arm's reference point after stereoscopic observation and 3D reconstruction, Θ photo X represents the attitude angle of the robotic arm actuator after calculation. Real Θ represents the true coordinates of the robotic arm's reference point. Real The actual posture angle of the robotic arm actuator; I, I+M are the same as (1)ΔX arm With ΔΘ arm For the constant term deviation of the robotic arm's position and attitude, considering the differences between the robotic arm's optical characteristics and the non-cooperative target, therefore, ΔXarm With ΔX target To make distinctions.

[0086] (ii) Secondly, establish a pose model of the robotic arm in the touch state;

[0087] During the touch process, the tactile sensor can only be triggered when the sensor comes into contact with the target and the contact force exceeds F. Ignoring deformation, when the tactile sensor touches a non-cooperative target and is just triggered, the geometric constraint function G is used to represent this:

[0088]

[0089] in, The actual position of the reference point of the robotic arm actuator at the time of the contact. The actual orientation of the reference point of the robotic arm actuator at the time of the contact. The actual position of the non-cooperative target at the point of contact is given. Here, the form of the geometric constraint function G is determined by the actual shape of the tactile sensor and the surface geometry of the non-cooperative target. After considering deformation factors, the above equation can be modified to obtain:

[0090]

[0091] In the formula, the contact force F is determined by the sensor threshold and is a known quantity; ΔX(F) is the sensor deformation caused by the contact force F, which depends on the sensor's hardness and flexibility and is usually negligible. target (F) represents the target deformation caused by the contact force F. The target deformation can be referenced. Figure 1 When the robotic arm contacts the ground, it creates an indentation. The sensor deformation ΔX(F) can be calculated using the sensor deformation model; it is usually small and can be ignored. The target deformation ΔX target (F) can be obtained through actual measurement using three-dimensional image modeling.

[0092] The geometric constraint G is essentially the contact relationship between two curved surfaces of a tactile sensor and a non-cooperative target. The solution involves determining the position of the contact point (marked by the subscript tp) between the tactile sensor and the non-cooperative target.

[0093]

[0094] Where A represents the relative position between the installation location of the tactile sensor and the measurement point of the tactile sensor, which is a known quantity; f is used to calculate the relative relationship between the reference point of the robotic arm actuator and the installation location of the tactile sensor, which is a known relationship; Given the actual contact location, by combining the two equations, we can obtain:

[0095]

[0096] During the tactile sensor's proximity calibration, the docking signal is monitored in real time. Upon triggering the contact signal, the robotic arm immediately stops moving and acquires graphic and telemetry information about the robotic arm and the contact point. The image information is then used to determine the reference point position of the robotic arm's actuator. with posture The reference point position of the robotic arm actuator obtained from telemetry information with posture Because the force F applied at the moment of contact causes deformation of the robotic arm, the telemetry needs to be corrected, denoted as . and

[0097] In actual tasks, the true position and attitude of the robotic arm are unknown. Considering that there are deviations between image measurement results, telemetry results, and true values, After substituting into (4), the right side of the equation is not 0, so we get:

[0098]

[0099] By using (5) and (6), the deviation term Δ generated by the visual measurement pose and the telemetry pose during the contact process after the tactile sensor comes into contact with and is triggered by the non-cooperative target can be calculated. photo With Δ Tel Since the visual measurement error and telemetry error are relatively small compared to the measurement target, (5) and (6) can be linearized to obtain:

[0100]

[0101] Visual measurement error and telemetry error are small relative to the measurement target, and are therefore ignored in further simplification. and The differences were considered to mean that visual measurement results were closer to reality, and a unified approach was adopted. and make: Simplified to:

[0102]

[0103] (iii) Again, based on the information provided by the binocular images, the information of the robotic arm, the information of the non-cooperative target (including the deformation information after touch), the telemetry information of the robotic arm, and the deformation model after the tactile sensor is triggered as described in (ii) (described by (5) to (8), establish the relationship model between the pose in the visual information and the real pose, the relationship model between telemetry and visual information, and the deformation of the robotic arm under force.

[0104] 1) Model of the relationship between pose and true pose in visual information

[0105] Substituting (1), (2), and (3) into (7), we get:

[0106]

[0107] In (9), F1, F2, Given the quantities, M and ΔX can be solved using multiple touch experiments. target With ΔX target The numerical values, if only a few touch tests are performed, are considered to be M and ΔX. target With ΔΘ arm If the value is 0, calculate ΔX first. arm As the number of touches increases, ΔX is calculated sequentially. target , ΔΘ arm With M.

[0108] 2) Relationship model between telemetry and visual information

[0109] Following (2) and (3), the telemetry of the reference point of the robotic arm actuator and the image information satisfy the following relationship:

[0110]

[0111] Since the robotic arm experiences no additional force during this process, it must be performed in a non-contact manner to ensure that the robotic arm is not subjected to any additional force. This process can then obtain the corresponding... Substituting these into (10) and (11), we can calculate the result. M T and Among them, priority calculation and As the number of trials increases, M is calculated step by step. T .

[0112] 3) Calculation of the deformation of the robotic arm under stress

[0113] Subtracting (8) from (7) gives us:

[0114]

[0115] Among them, F1, F2, Given that all the information is known, ΔX can be solved. Tel (F) and ΔΘ Tel (F).

[0116] (iv) Finally, based on the established model of the relationship between the robot arm pose and the real pose in the visual information, the model of the relationship between telemetry and visual information, and the force deformation of the robot arm, the prediction of the robot arm control quantity to reach the new target point is completed.

[0117] For the new target location (The target point is selected based on the point non-cooperative target cloud model, so it is a visual modeling result rather than the actual location.) Using the deviation relationship between the target point cloud and the actual location in (I) (1), the actual location of the target point is calculated. Based on the mechanical shape and the deformation of the robotic arm under force (the calculation results of the deformation of the robotic arm under force in section (III)), the target position of the robotic arm's movement is predicted. with posture Based on the telemetry and visual information relationship model in section (III), it is calculated as the telemetry target position. with posture This is the target value required for the robotic arm control. The above process is as follows: Figure 2 As shown. The robotic arm is controlled to complete the modeling of its position relative to the target and the amount of movement, as shown below. Figure 4 As shown.

[0118] It should be noted that the model in this paper has undergone a certain degree of linearization simplification. This model possesses good characteristics, therefore, data from the vicinity of the target location is more valuable for reference. Thus, a weighting coefficient for the target point is added: assuming k tests were conducted historically, and the distance between these test points and the target point is r... k The weight of this test is . This coefficient increases the contribution value of neighboring area test results, making the most of previous experience.

[0119] Visual measurement technology is unaffected by gravity differences and robotic arm deformation. However, visual measurements of uncertain targets in complex lighting environments exhibit uncertainty. By fusing visual measurements of non-cooperative targets with tactile information and progressively implementing intelligent corrections during the task, relatively high-precision control of the robotic arm relative to non-cooperative targets can be achieved. This solution offers advantages such as being unaffected by multiple error sources including gravity differences, high reliability, and high control precision, enabling high-precision control of robotic arms for non-cooperative targets in extraterrestrial gravity environments.

[0120] Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make possible changes and modifications to the technical solutions of the present invention by utilizing the methods and techniques disclosed above without departing from the spirit and scope of the present invention. Therefore, any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solutions of the present invention shall fall within the protection scope of the technical solutions of the present invention.

[0121] The contents not described in detail in this specification are common knowledge to those skilled in the art.

Claims

1. A high-precision intelligent sampling control method for a robotic arm targeting non-cooperative extraterrestrial targets, characterized in that... include: Establish a point cloud model of a non-cooperative target and a robotic arm execution model. Collect stereo image information of the non-cooperative target and input it into the point cloud model of the non-cooperative target to obtain the observation point cloud data of the non-cooperative target. Calculate the real point cloud data of the non-cooperative target based on the observation point cloud data. The robot arm's stereoscopic image information is collected and substituted into the robot arm execution model to obtain the robot arm's telemetry position and attitude data. The robot arm's actual position and attitude data are calculated based on the robot arm execution model. A contact constraint function is established based on the actual position and posture data of the robotic arm to determine the contact relationship. If a contact occurs, a position and posture model of the robotic arm under the contact state is established; otherwise, the stereo image information of the non-cooperative target is re-acquired and the tactile sensor is readjusted. Using the position and posture model of the robotic arm under the touch state to output the real position and posture data of the robotic arm actuator, the deviation value between the telemetry data and the real data of the robotic arm during the contact process is calculated. Based on the deviation value, a model of the relationship between the visual pose and the real pose of the robotic arm, a model of the relationship between the telemetry pose and the visual pose of the robotic arm, and a model for solving the force deformation of the robotic arm are established. The actual location of the target point is selected from the real point cloud data of the non-cooperative target, and the target position and attitude of the robot arm are predicted according to the force deformation calculation model of the robot arm. The target position and orientation of the robotic arm are substituted into the relationship model between the telemetry pose and the visual pose of the robotic arm to calculate the reference position and orientation of the robotic arm. The robotic arm is then controlled to execute the movement based on the reference position and orientation of the robotic arm.

2. The high-precision intelligent sampling control method for a robotic arm targeting a non-cooperative extraterrestrial target according to claim 1, characterized in that: There is a positional discrepancy between the actual point cloud data and the observed point cloud data of the non-cooperative target. The method for calculating the actual point cloud data of the non-cooperative target is as follows: In the formula, To obtain observation point cloud data of non-cooperative targets after stereo observation and 3D reconstruction, For real point cloud data of non-cooperative targets, It is the identity matrix. The transformation matrix represents the influence terms of binocular camera calibration error and modeling uncertainty. This is the positional deviation.

3. The high-precision intelligent sampling control method for a robotic arm targeting a non-cooperative extraterrestrial target according to claim 1, characterized in that: There is a deviation between the actual position and attitude data of the robotic arm and the calculated position and attitude data. The calculation method for the actual position and attitude data of the robotic arm is as follows: In the formula, The position of the robotic arm is obtained by substituting the 3D image information of the robotic arm into the robotic arm execution model. The calculated attitude data for the robotic arm. This is the actual position data of the robotic arm. This is the actual posture data of the robotic arm; It is a third-order identity matrix. For the transformation matrix, and This represents the constant deviation value for the position and orientation of the robotic arm.

4. The high-precision intelligent sampling control method for a robotic arm targeting a non-cooperative extraterrestrial target according to claim 1, characterized in that: The method for determining contact relationships is as follows: Establish the contact constraint function between the robotic arm's tactile sensor and the non-cooperative target during the touch process; The contact force between the tactile sensor and the target is determined using a contact constraint function. If the contact constraint is satisfied, the actual contact position between the tactile sensor and the non-cooperative target is calculated and a contact signal is triggered. Otherwise, no calculation is performed, and the system waits for the next contact. When the contact signal is triggered, the movement of the robotic arm stops, and the stereoscopic image information of the robotic arm and the actual contact position and the collected telemetry information are obtained to determine the telemetry position and telemetry attitude of the robotic arm actuator, as well as the real position and real attitude of the robotic arm actuator. The position and posture model of the robotic arm in the touch state is established based on the telemetry posture and the actual posture of the robotic arm actuator.

5. The high-precision intelligent sampling control method for a robotic arm targeting a non-cooperative extraterrestrial target according to claim 4, characterized in that: The contact constraint function between the robotic arm's tactile sensor and the non-cooperative target is: In the formula, The actual position of the reference point of the robotic arm actuator at the time of the contact. This refers to the actual contact location. The actual orientation of the reference point of the robotic arm actuator at the time of the contact. Given the known installation location of the tactile sensor and the relative position between the tactile sensor measurement point. Given the known relative relationship between the reference point of the robotic arm actuator and the mounting position of the tactile sensor, For sensor deformation, The deformation of the non-cooperative target is determined based on the tactile sensor and the contact force of the target.

6. The high-precision intelligent sampling control method for a robotic arm targeting a non-cooperative extraterrestrial target according to claim 5, characterized in that: The position and attitude model of the robotic arm in the touch state is as follows: In the formula, The pose data determined by the tactile sensor during contact based on stereo image information and real data, and the deviation of the actual pose data. The deviation between the pose data determined by telemetry information and real data for a non-cooperative target during contact and the actual pose data. and The images in the middle show the actual position and posture of the robotic arm actuator as measured by image during the touch state. and The robotic arm calculates its position and attitude based on telemetry data during the touch state, respectively. and Contact force F This is the amount of correction to the solution position and solution attitude after the robotic arm deforms.

7. The high-precision intelligent sampling control method for a robotic arm targeting a non-cooperative extraterrestrial target according to claim 4, characterized in that: The model relating the robotic arm's visual pose to its actual pose is as follows: In the formula, To describe the transformation matrix between image-derived position and actual deviation, representing the influence of binocular camera calibration error and 3D modeling algorithm uncertainty, This is the constant term deviation caused by the positional deviation of the camera calibration and the uncertainty of the target's optical characteristics when observing non-cooperative targets. To account for the constant term deviation caused by the positional deviation of the camera calibration during the observation of the robotic arm and the uncertainty of the target's optical characteristics, and considering the differences in optical properties between the robotic arm and the non-cooperative target, the following is provided: and To differentiate, , and The values ​​were obtained by solving the touch test, and the initial values ​​were all 0.

8. The high-precision intelligent sampling control method for a robotic arm targeting a non-cooperative extraterrestrial target according to claim 7, characterized in that: The model relating the telemetry pose and visual pose of the robotic arm is as follows: In the formula, To describe the transformation matrix between image-based calculated position and telemetry position deviation, To calculate the constant term deviation between the robot arm position obtained from the image and the robot arm position obtained from telemetry, The constant term is used to calculate the deviation between the robot arm posture obtained from the image and the robot arm posture obtained from telemetry. , These are the telemetry values ​​of the robotic arm's position and orientation in a non-contact state; , The position and attitude of the robotic arm are calculated from images taken simultaneously with the telemetry position and attitude in a non-touch state.

9. A high-precision intelligent sampling control method for a robotic arm targeting a non-cooperative extraterrestrial target according to claim 8, characterized in that: The calculation model for the deformation of the robotic arm under force is as follows: In the formula, , These are the telemetry values ​​of the robotic arm's position and orientation during the touch state. , The position and orientation of the robotic arm are calculated from images taken simultaneously with the telemetry position and orientation under the touch state. , The deviation between image data and telemetry data during the touch process can be calculated based on the touch model. , The correlation coefficient between the robotic arm and the non-cooperative target's geometry during the touch process can be calculated based on the relative position of the robotic arm and the tactile sensor, as well as the touch posture. Ultimately, the position and posture deformation of the robotic arm relative to the telemetry values ​​are calculated. and .

10. A high-precision intelligent sampling control method for a robotic arm targeting a non-cooperative extraterrestrial target according to claim 1, characterized in that: After selecting the actual location of the target point, a weight value is assigned to the selected target point based on historical touch control test data. After conducting a specified number of touch control tests, all test data are statistically analyzed based on the corresponding weight value of the actual location of the selected target point, and the statistical results are used for subsequent analysis.

Images