An autonomous target measurement method for an on-orbit maintenance robot in space

Abstract

The present invention relates to an autonomous measurement method for a space on-orbit maintenance robot target, which belongs to the field of space robots; it includes the following steps: constructing the overall architecture of the space on-orbit maintenance robot; performing hand-eye calibration; establishing an obstacle map and detecting maintenance tools; and performing intelligent detection to detect target category maintenance tools within the field of view, and obtaining the position of the target and sending it to the control part; the end of the mechanical arm moves to an ultra-close distance to the target; high-precision posture estimation; and transmitting the high-precision posture information to the control part, so that the mechanical arm adjusts the end posture and uses a gripper to grab the target. The present invention discloses an autonomous measurement method and device for a space on-orbit maintenance robot target, which can autonomously and effectively enable the robot to autonomously sense and measure the target to facilitate the next step of grabbing, thereby completing the space maintenance task.

Description

Technical Field

[0001] The invention belongs to the field of space robots and relates to an autonomous target measurement method for a space on-orbit maintenance robot. Background Art

[0002] Space robots are important operating tools for humans to explore space and complete space missions, and they play an important role in space operations. With the development of science and technology, aerospace technology has been continuously updated, and space station operations have gradually become more complicated, and the requirements for the tasks to be performed have gradually increased. Faced with the dangerous and complex space environment, space robots can help humans complete many difficult, high-precision, and high-risk operations, thereby helping astronauts complete space operation tasks, improve work efficiency, and increase the success rate and safety of missions.

[0003] When space stations and satellites need to be repaired, space robots are used to intelligently detect target repair tools, obtain high-precision position information of the target tools, control the space robots to grab the repair tools, and use the tools to perform repair operations, thereby achieving autonomous maintenance, saving manpower, and improving efficiency and safety. This has certain significance in practical applications.

[0004] The above operational requirements for high-precision autonomous maintenance using space robots can be achieved based on fixed known spatial relationships. However, in actual applications, the space environment is complex and the model and position of the target cannot be guaranteed to be known. When performing complex maintenance operations, high-precision intelligent operations are required. For non-cooperative targets with unknown models and information, the space robot needs to first detect the target and obtain high-precision position information to ensure the accuracy of the maintenance operation steps.

[0005] Therefore, how to better combine the field of space robots with perception and autonomous operation technologies has become a technical problem that needs to be urgently solved by technical personnel in this field. Summary of the invention

[0006] The technical problem solved by the present invention is: to overcome the deficiencies of the prior art and to propose an autonomous target measurement method for a space on-orbit maintenance robot, which can perform maintenance operations more accurately and precisely, execute precise maintenance tasks, and better save time and labor costs.

[0007] The solution of the present invention is:

[0008] A method for autonomously measuring a target of an on-orbit maintenance robot in space, comprising:

[0009] Step 1: Build the overall architecture of the space on-orbit maintenance robot, including the control part, the perception part and the communication part; the control part is to control the seven-degree-of-freedom robotic arm with grippers through a stepper motor; the perception part is to install a head camera on the robot head and a hand camera at the end of the robotic arm; the communication part is to achieve network connectivity between the control part and the perception part;

[0010] Step 2: Perform hand-eye calibration to determine the relationship between the camera coordinate system, the robotic arm coordinate system, and the earth coordinate system;

[0011] Step 3: Create an obstacle map and detect maintenance tools: The head camera of the space on-orbit maintenance robot senses the information ahead and creates an obstacle map to facilitate the control of some operating manipulators to move; and perform intelligent detection to detect the target type maintenance tools in the field of view and obtain the target's position and posture.

[0012] Step 4: The end of the robotic arm moves to a very close distance to the target: The control part uses the target position information observed by the head camera to make the end of the robotic arm avoid obstacles and move to a very close distance to the target, so that the camera at the end of the robotic arm can perform high-precision pose estimation;

[0013] Step 5: High-precision pose estimation: Use the hand camera at the end of the robotic arm to perform intelligent detection and pose estimation of the target at an ultra-close distance to obtain the high-precision pose of the target;

[0014] Step 6: Execute grasping: The high-precision posture information is transmitted to the control part through LCM, so that the robot arm adjusts the end posture and uses the gripper to grasp the target.

[0015] In the above-mentioned autonomous target measurement method for an on-orbit maintenance robot in space, in step one, both the head camera and the hand camera are RGB-D cameras; the RGB-D camera can obtain the distance information of the target from the camera while obtaining the target RGB image, and perceive the RGB information and depth information at one time, avoiding the calibration problem caused by using the RGB camera and the depth camera separately.

[0016] In the above-mentioned method for autonomous target measurement of an on-orbit maintenance robot in space, in step one, the communication method of the communication part adopts lightweight level communication and grouping LCM.

[0017] In the above-mentioned autonomous target measurement method for an on-orbit maintenance robot in space, in step 2, the hand-eye calibration is to unify all information into the robot arm coordinate system, and the specific method is:

[0018] S21, fix the RGB-D camera, perform hand-eye calibration after fixing the position, and do not change the position of the camera;

[0019] S22. According to the camera's field of view and the length of the robot arm, select a calibration plate of appropriate size to make the information more accurate and effective during calibration;

[0020] S23. Use the eye in hand method to calibrate the hand camera at the end of the robot arm, that is, solve the relationship between the camera and the end of the robot; use the eye to hand method to calibrate the head camera of the robot arm, that is, solve the relationship between the camera and the robot base; move the robot arm multiple times and ensure that the calibration plate is always within the camera field of view;

[0021] S24. Use the Tsai method to solve the problem in the hand-eye calibration problem and obtain the target matrix, which is the relationship between the camera coordinate system and the robotic arm coordinate system. After defining the robotic arm coordinate system, the earth coordinate system is artificially defined according to the robotic arm coordinate system.

[0022] In the above-mentioned method for autonomous target measurement of an on-orbit maintenance robot in space, in step 3, the specific method of establishing an obstacle map and detecting maintenance tools is:

[0023] S31, converting the depth information collected by the head camera of the space on-orbit maintenance robot into a three-dimensional map, and using it as an obstacle map, which is used as a basis for the robot arm to plan the path when it moves, so as to avoid collision of the robot arm; the obstacle map is a grid map;

[0024] S32, performing intelligent detection of the target, including establishing a data set for the target, performing data enhancement, training, and testing; further, using a head camera to perform target recognition and instance segmentation to obtain a point cloud corresponding to the target; target recognition and instance segmentation are implemented using a Mask-RCNN network; in the target recognition and instance segmentation steps, the outer contour of the target in the RGB image is obtained; further, by aligning the RGB information with the depth map, the depth information of the target, that is, the point cloud of the target, is obtained;

[0025] S33, solving the position coordinates of the target under the robotic arm base: obtaining the outer rectangular frame of the target through target recognition and instance segmentation, and further obtaining the pixel coordinates of the center point of the target; combining the intrinsic parameters of the camera, solving the coordinates of the center point of the target in the camera system; finally solving the position coordinates of the target under the robotic arm base through the results obtained by hand-eye calibration;

[0026] S34. Solve the posture information of the target under the base of the robotic arm: obtain the position and posture of the target by aligning the template point cloud of the point cloud registration model with the target point cloud; after the alignment, the target point cloud is obtained according to the rotation and translation relationship of the template point cloud, and further the matrix transformation of the hand-eye calibration result is performed to obtain the posture of the target under the base of the robotic arm.

[0027] In the above-mentioned method for autonomous target measurement of a space on-orbit maintenance robot, in S34, the accuracy of the template point cloud is equal to or higher than the acquisition accuracy of the camera at an ultra-close distance to the target; after the template point cloud is made, the template point cloud is fixed at a certain position with a known posture, so as to obtain the posture relationship between the two point clouds; preferably, the template point cloud is fixed at the origin position of the camera coordinate system to facilitate the transformation and calculation of the coordinate system;

[0028] The point cloud registration model is an improved model based on ICP.

[0029] In the above-mentioned method for autonomously measuring a target of an on-orbit maintenance robot in space, in step 4, the specific method for moving the end of the robotic arm to an ultra-close distance to the target is:

[0030] S41, the target position of the robot arm is set to a coordinate position a certain distance above the target center point coordinate in step S3, and the target posture of the robot arm is the same as the target posture in step S3;

[0031] The distance between the robot arm's moving target position and the actual grasping target is mainly related to the length of the robot arm's gripper. The purpose is to set an ultra-close distance so that the hand RGB-D camera can perform high-precision pose estimation of the grasping target within the nearest working range without the robot arm touching the target object.

[0032] S42, performing path planning according to the target posture set in step S41 and the obstacle map acquired in step S31, and smoothing the path according to the dynamics of the robot arm to obtain a smooth trajectory;

[0033] S43, sending the calculated motion trajectory to the control module through LCM communication, controlling the robotic arm to track the planned trajectory, and moving the end of the robotic arm to an ultra-close distance to the target.

[0034] In the above-mentioned autonomous target measurement method for an on-orbit maintenance robot in space, step five is the same as steps three and four, and a high-precision pose of the target is obtained by aligning the point cloud registration model template point cloud with the target point cloud.

[0035] In the above-mentioned autonomous target measurement method for an on-orbit maintenance robot in space, the difference between step five and step three and step four is that when step three performs point cloud registration, the camera is far away from the target, and the number of two point clouds has a certain difference, and the registration result is not accurate, and only needs to provide the target position for step four;

[0036] In step five, the hand camera is closer to the target, the collected point cloud is more accurate, the registration result is more accurate, and the high-precision pose of the target can be obtained.

[0037] In the above-mentioned method for autonomous target measurement of an on-orbit maintenance robot in space, in step 6, the specific process of performing the grasping is:

[0038] S61, high-precision pose information includes 7 data, namely, the position information (x, y, z) and quaternion of the target center point in the coordinate system of the robot base;

[0039] S62, through LCM communication, high-precision posture information is sent from the computer of the vision and planning part and received by the computer of the control part;

[0040] S63. According to the received position information, the robot arm is controlled to move and the gripper is used to grasp.

[0041] The beneficial effects of the present invention compared with the prior art are:

[0042] (1) In the dangerous space environment, the present invention uses a space on-orbit maintenance robot to replace astronauts to perform dangerous maintenance operations, thereby greatly ensuring the safety of astronauts;

[0043] (2) For high-precision maintenance tasks, the present invention uses an on-orbit maintenance robot to perform maintenance operations more accurately and precisely than manual maintenance;

[0044] (3) For maintenance environments where the space is small and it is inconvenient for astronauts to operate, the present invention uses an on-orbit maintenance robot to better grasp small maintenance tools and perform precise maintenance tasks;

[0045] (4) For large-scale, basic maintenance operations, the present invention uses an on-orbit maintenance robot to better save time and labor costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 This is a flow chart of autonomous target measurement of the space on-orbit maintenance robot of the present invention. DETAILED DESCRIPTION

[0047] The present invention will be further described below in conjunction with the embodiments.

[0048] The present invention provides a method for an on-orbit maintenance robot in space to autonomously perform maintenance tasks, which can perform maintenance operations more accurately and precisely, execute precise maintenance tasks, and better save time and labor costs.

[0049] Autonomous measurement method for on-orbit maintenance robot targets in space, such as Figure 1 As shown, the specific steps include:

[0050] Step 1: Construct the overall architecture of the space on-orbit maintenance robot, including the control part, the perception part and the communication part; the control part controls the seven-degree-of-freedom robotic arm with a gripper through a stepper motor; the perception part installs a head camera on the robot head and a hand camera at the end of the robotic arm; the communication part realizes network connectivity between the control part and the perception part.

[0051] Both the head camera and the hand camera are RGB-D cameras; the RGB-D camera can obtain the distance information of the target from the camera while obtaining the target RGB image, and perceive the RGB information and depth information at one time, avoiding the calibration problem caused by using the RGB camera and depth camera separately.

[0052] The communication part adopts lightweight communication and marshaling LCM.

[0053] Step 2: Perform hand-eye calibration to determine the relationship between the camera coordinate system, the robotic arm coordinate system, and the earth coordinate system.

[0054] Hand-eye calibration is to unify all information into the robot arm coordinate system. The specific method is as follows:

[0055] S21, fix the RGB-D camera, perform hand-eye calibration after fixing the position, and do not change the position of the camera;

[0056] S22. According to the camera's field of view and the length of the robot arm, select a calibration plate of appropriate size to make the information more accurate and effective during calibration;

[0057] S23. Use the eye in hand method to calibrate the hand camera at the end of the robot arm, that is, solve the relationship between the camera and the end of the robot; use the eye to hand method to calibrate the head camera of the robot arm, that is, solve the relationship between the camera and the robot base; move the robot arm multiple times and ensure that the calibration plate is always within the camera field of view;

[0058] S24. Use the Tsai method to solve the problem in the hand-eye calibration problem and obtain the target matrix, which is the relationship between the camera coordinate system and the robotic arm coordinate system. After defining the robotic arm coordinate system, the earth coordinate system is artificially defined according to the robotic arm coordinate system.

[0059] Step 3. Create an obstacle map and detect maintenance tools: The head camera of the space on-orbit maintenance robot senses the information ahead and creates an obstacle map to facilitate the control of some operating robotic arms to move; and perform intelligent detection to detect the target category maintenance tools in the field of view and obtain the target's position and posture.

[0060] The specific method of establishing an obstacle map and testing maintenance tools is as follows:

[0061] S31, converting the depth information collected by the head camera of the space on-orbit maintenance robot into a three-dimensional map, and using it as an obstacle map, which is used as a basis for the robot arm to plan the path when it moves, so as to avoid collision of the robot arm; the obstacle map is a grid map;

[0062] S32, performing intelligent detection of the target, including establishing a data set for the target, performing data enhancement, training, and testing; further, using a head camera to perform target recognition and instance segmentation to obtain a point cloud corresponding to the target; target recognition and instance segmentation are implemented using a Mask-RCNN network; in the target recognition and instance segmentation steps, the outer contour of the target in the RGB image is obtained; further, by aligning the RGB information with the depth map, the depth information of the target, that is, the point cloud of the target, is obtained;

[0063] S33, solving the position coordinates of the target under the robotic arm base: obtaining the outer rectangular frame of the target through target recognition and instance segmentation, and further obtaining the pixel coordinates of the center point of the target; combining the intrinsic parameters of the camera, solving the coordinates of the center point of the target in the camera system; finally solving the position coordinates of the target under the robotic arm base through the results obtained by hand-eye calibration;

[0064] S34. Solve the posture information of the target under the base of the robotic arm: obtain the position and posture of the target by aligning the template point cloud of the point cloud registration model with the target point cloud; after the alignment, the target point cloud is obtained according to the rotation and translation relationship of the template point cloud, and further the matrix transformation of the hand-eye calibration result is performed to obtain the posture of the target under the base of the robotic arm.

[0065] In the S34, the accuracy of the template point cloud is equal to or higher than the acquisition accuracy of the camera at an ultra-close distance to the target; after the template point cloud is made, the template point cloud is fixed at a certain position with a known posture, so as to obtain the posture relationship between the two point clouds; preferably, the template point cloud is fixed at the origin position of the camera coordinate system to facilitate the transformation and calculation of the coordinate system;

[0066] The point cloud registration model is an improved model based on ICP.

[0067] Step 4: The end of the robotic arm moves to a super-close distance to the target: Based on the target position information observed by the head camera, the control part uses the end of the robotic arm to avoid obstacles and move to a super-close distance to the target, so that the camera at the end of the robotic arm can perform high-precision pose estimation.

[0068] The specific method of moving the end of the robotic arm to a super close distance to the target is:

[0069] S41, the target position of the robot arm is set to a coordinate position a certain distance above the target center point coordinate in step S3, and the target posture of the robot arm is the same as the target posture in step S3;

[0070] The distance between the robot arm's moving target position and the actual grasping target is mainly related to the length of the robot arm's gripper. The purpose is to set an ultra-close distance so that the hand RGB-D camera can perform high-precision pose estimation of the grasping target within the nearest working range without the robot arm touching the target object.

[0071] S42, performing path planning according to the target posture set in step S41 and the obstacle map acquired in step S31, and smoothing the path according to the dynamics of the robot arm to obtain a smooth trajectory;

[0072] S43, sending the calculated motion trajectory to the control module through LCM communication, controlling the robotic arm to track the planned trajectory, and moving the end of the robotic arm to an ultra-close distance to the target.

[0073] Step 5: High-precision pose estimation: Use the hand camera at the end of the robotic arm to perform intelligent detection and pose estimation of the target at an ultra-close distance to obtain the target's high-precision pose.

[0074] The step five is the same as the steps three and four, and the high-precision pose of the target is obtained by aligning the point cloud registration model template point cloud with the target point cloud.

[0075] The difference between step 5 and step 3 and step 4 is that when step 3 performs point cloud registration, the camera is far away from the target, and there is a certain difference in the number of point clouds in the two groups. The registration result is not accurate and only needs to provide the target position for step 4.

[0076] In step five, the hand camera is closer to the target, the collected point cloud is more accurate, the registration result is more accurate, and the high-precision pose of the target can be obtained.

[0077] Step 6: Execute grasping: The high-precision posture information is transmitted to the control part through LCM, so that the robot arm adjusts the end posture and uses the gripper to grasp the target.

[0078] The specific process of performing crawling is:

[0079] S61, high-precision pose information includes 7 data, namely, the position information (x, y, z) and quaternion of the target center point in the coordinate system of the robot base;

[0080] S62, through LCM communication, high-precision posture information is sent from the computer of the vision and planning part and received by the computer of the control part;

[0081] S63. According to the received position information, the robot arm is controlled to move and the gripper is used to grasp.

[0082] Example

[0083] The proposed UAV vision-based target recognition and assessment method includes the following steps in sequence:

[0084] S1. Construct the overall architecture of the on-orbit maintenance robot;

[0085] In step S1, the overall architecture of the space on-orbit maintenance robot mainly includes a sensing part, a control part and a communication part;

[0086] S11, wherein the sensing part includes a fixed camera installed on the head and a hand camera installed at the end of the robotic arm (the wrist joint connected to the gripper), and the camera is preferably an RGB-D camera. The RGB-D camera can obtain the distance information of the target from the camera while obtaining the target RGB image, so that the RGB information and the depth information can be sensed at one time, and the calibration problem caused by using the RGB camera and the depth camera separately is avoided;

[0087] S12, the control part mainly controls a seven-degree-of-freedom robotic arm with a gripper, and controls the stepper motor of the robotic arm through a micro operating system;

[0088] S13, the hardware of the communication part includes a router and network cables respectively connecting the sensing part and the control part;

[0089] Preferably, the communication method uses lightweight communications and marshalling (LCM). LCM can be applicable to real-time systems with high bandwidth and low latency, has few dependencies, uses an efficient broadcast mechanism, and is suitable for accurate and fast communications in a space environment.

[0090] S2. Perform hand-eye calibration: In order to make the perception function of the hand-eye camera accurate and effective, hand-eye calibration is required to determine the relationship between the camera coordinate system, the robot arm coordinate system and the earth coordinate system;

[0091] Preferably, in the overall measurement method, all information is unified into the robot arm base coordinate system, which is more intuitive and convenient for the robot arm to operate;

[0092] S21, fix the RGB-D camera, perform hand-eye calibration after fixing the position, and the position of the camera cannot be changed;

[0093] S22, according to the field of view of the camera and the arm length of the robot arm, a calibration plate of appropriate size is selected to make the information more accurate and effective during calibration;

[0094] S23, using the "eye in hand" method to calibrate the hand camera at the end of the robot arm, that is, to solve the relationship between the camera and the end of the robot; using the "eye to hand" method to calibrate the head camera of the robot arm, that is, to solve the relationship between the camera and the base of the robot; the robot arm moves multiple times and ensures that the calibration plate is always within the field of view of the camera;

[0095] S24, using a hand-eye calibration solution method to solve a solution problem in the hand-eye calibration problem, and obtaining a target matrix;

[0096] Preferably, the Tsai method is used, which is more widely used in the solution algorithm of hand-eye calibration and has higher robustness.

[0097] S3. Establishing obstacle map and detecting maintenance tools: The head camera of the space on-orbit maintenance robot senses the information ahead and establishes obstacle map to facilitate the control of some operating manipulators to move; and intelligent detection is performed to detect the target type maintenance tools in the field of view and obtain the position and posture of the target;

[0098] S31, the depth information collected by the head camera of the space on-orbit maintenance robot is converted into a three-dimensional map and used as an obstacle map for the robot arm to plan the path when it moves, so as to avoid collision of the robot arm;

[0099] Preferably, the obstacle map is established as a grid map;

[0100] S32, performing intelligent detection of the target, including establishing a data set for the target, performing data enhancement, training, and testing; further, using a head camera to perform target recognition and instance segmentation to obtain a point cloud corresponding to the target;

[0101] Preferably, object recognition and instance segmentation are implemented using a Mask-RCNN network;

[0102] In the target recognition and instance segmentation steps, the outer contour of the target in the RGB image can be obtained; further, by aligning the RGB information with the depth map, the depth information of the target, that is, the point cloud of the target, can be obtained;

[0103] S33, solving the position coordinates of the target under the robot arm base: the outer rectangular frame of the target can be obtained through target recognition and instance segmentation, and the pixel coordinates of the center point of the target can be further obtained; combined with the intrinsic parameters of the camera, the coordinates of the center point of the target in the camera system can be solved; the result obtained by hand-eye calibration can finally solve the position coordinates of the target under the robot arm base;

[0104] S34, solving the posture information of the target under the robot arm base: by registering the template point cloud of the point cloud registration model with the target point cloud, the position and posture of the target are obtained; after the registration, the rotation and translation relationship of the target point cloud according to the template point cloud can be obtained, and the posture of the target under the robot arm base can be obtained by further matrix transformation of the hand-eye calibration result;

[0105] Preferably, when making the template point cloud, its accuracy should be equal to or higher than the acquisition accuracy of the camera at a very close distance to the target, so as to facilitate high-precision pose estimation when using the hand camera later; after making the template point cloud, fix the template point cloud at a certain position with known pose, so as to obtain the pose relationship between the two point clouds. In particular, it is preferred to fix the template point cloud at the origin of the camera coordinate system to facilitate the transformation and calculation of the coordinate system.

[0106] If the target has a 3D model, it can be directly converted into a uniform point cloud model as a template point cloud. If it is a non-cooperative target without a model, the depth information collected by an RGB-D camera at an ultra-close distance can be used as its template point cloud.

[0107] Preferably, the point cloud registration model is an improved model based on ICP, such as the teaser++ model, which is an open source model derived from the paper TEASER: Fast and Certifiable Point Cloud Registration.

[0108] S4, the end of the robot arm moves to a very close distance to the target: The control part uses the target position information observed by the head camera to make the end of the robot arm avoid obstacles and move to a very close distance to the target, so that the camera at the end of the robot arm can perform high-precision pose estimation;

[0109] S41, the target position of the robot arm is set to a coordinate position a certain distance above the target center point coordinate in step S3, and the target posture of the robot arm is the same as the target posture in step S3;

[0110] The distance between the robot arm's moving target position and the actual grasping target is mainly related to the length of the robot arm's gripper. The purpose is to set an ultra-close distance so that the hand RGB-D camera can perform high-precision pose estimation of the grasping target within the nearest working range without the robot arm touching the target object.

[0111] S42, performing path planning according to the target posture set in step S41 and the obstacle map acquired in step S31, and smoothing the path according to the dynamics of the robot arm to obtain a smooth trajectory;

[0112] S43, sending the calculated motion trajectory to the control module through LCM communication, controlling the robotic arm to track the planned trajectory, and moving the end of the robotic arm to an ultra-close distance to the target.

[0113] S5. High-precision pose estimation: Use the hand camera at the end of the robotic arm to perform intelligent detection and pose estimation of the target at an ultra-close distance to obtain the high-precision pose of the target;

[0114] In step S5, similar to step S33 and step S34, the point cloud registration model template point cloud and the target point cloud are registered to obtain a high-precision pose of the target, and the specific steps are not repeated here;

[0115] The difference is that when performing point cloud registration in S3, the camera is far away from the target, and there is a certain difference in the number of two point clouds. The registration result is not accurate, and it only needs to provide the target position for step S4. In step S5, the hand camera is closer to the target, the collected point cloud is more accurate, and the registration result is more accurate, so the high-precision pose of the target can be obtained.

[0116] S6, execute grasping: transmit high-precision posture information to the control part through LCM, so that the robot arm adjusts the end posture and uses the gripper to grasp the target;

[0117] S61, high-precision pose information includes 7 data, namely, the position information (x, y, z) and quaternion of the target center point in the coordinate system of the robot base;

[0118] S62, through LCM communication, high-precision posture information is sent from the computer of the vision and planning part to the computer of the control part;

[0119] S63, according to the received position information, control the robot arm to move and use the gripper to grasp.

[0120] The present invention realizes the use of on-orbit space maintenance robots to replace astronauts in performing dangerous maintenance operations in dangerous space environments, thereby greatly ensuring the safety of astronauts; for high-precision maintenance tasks, compared with manual maintenance, the use of on-orbit space maintenance robots can perform maintenance operations more accurately and precisely; for maintenance environments with narrow space and inconvenient for astronauts to operate, the use of on-orbit space maintenance robots can better grasp smaller maintenance tools and perform precise maintenance tasks; for large-scale, basic maintenance operations, the use of on-orbit space maintenance robots can better save time and labor costs.

[0121] Although the present invention has been disclosed as above in the form of a preferred embodiment, it is not intended to limit the present invention. Any person skilled in the art may make possible changes and modifications to the technical solution of the present invention by using the methods and technical contents disclosed above without departing from the spirit and scope of the present invention. Therefore, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention shall fall within the protection scope of the technical solution of the present invention.

Claims

1. A method for autonomous target measurement of a space on-orbit maintenance robot, characterized in that: include: Step 1: Build the overall architecture of the space on-orbit maintenance robot, including the control part, the perception part and the communication part; the control part is to control the seven-degree-of-freedom robotic arm with grippers through a stepper motor; the perception part is to install a head camera on the robot head and a hand camera at the end of the robotic arm; the communication part is to achieve network connectivity between the control part and the perception part; Step 2: Perform hand-eye calibration to determine the relationship between the camera coordinate system, the robotic arm coordinate system, and the earth coordinate system; Step 3: Create an obstacle map and detect maintenance tools: The head camera of the on-orbit maintenance robot senses the information ahead and creates an obstacle map to facilitate the control of some operating robotic arms to move; And perform intelligent detection to detect the target category maintenance tools in the field of view and obtain the target's position and posture; Step 4: The end of the robotic arm moves to a very close distance to the target: The control part uses the target position information observed by the head camera to make the end of the robotic arm avoid obstacles and move to a very close distance to the target, so that the camera at the end of the robotic arm can perform high-precision pose estimation; Step 5: High-precision pose estimation: Use the hand camera at the end of the robotic arm to perform intelligent detection and pose estimation of the target at an ultra-close distance to obtain the high-precision pose of the target; Step 6: Execute the grasping: The high-precision posture information is transmitted to the control part through LCM, so that the robot arm adjusts the end posture and uses the gripper to grasp the target; In step 3, the specific method of establishing an obstacle map and detecting maintenance tools is: S31, converting the depth information collected by the head camera of the space on-orbit maintenance robot into a three-dimensional map, and using it as an obstacle map, which is used as a basis for the robot arm to plan the path when it moves, so as to avoid collision of the robot arm; the obstacle map is a grid map; S32, performing intelligent detection of the target, including establishing a data set for the target, performing data enhancement, training, and testing; Use the head camera to perform target recognition and instance segmentation to obtain the point cloud corresponding to the target; Target recognition and instance segmentation are implemented using the Mask-RCNN network. In the target recognition and instance segmentation steps, the outer contour of the target in the RGB image is obtained. By aligning the RGB information with the depth map, the depth information of the target, that is, the point cloud of the target, is obtained. S33, solving the position coordinates of the target under the robot arm base: the outer rectangular frame of the target is obtained through target recognition and instance segmentation, and the pixel coordinates of the center point of the target can be obtained; combined with the intrinsic parameters of the camera, the coordinates of the center point of the target in the camera system are solved; the result obtained by hand-eye calibration is finally solved The position coordinates of the target under the robot arm base; S34. Solve the posture information of the target under the base of the robotic arm: obtain the position and posture of the target by aligning the template point cloud of the point cloud registration model with the target point cloud; after the alignment, the target point cloud is obtained according to the rotation and translation relationship of the template point cloud, and the matrix transformation of the hand-eye calibration result is used to obtain the posture of the target under the base of the robotic arm.

2. The method for autonomous target measurement of an on-orbit maintenance robot in space according to claim 1, characterized in that: In the step 1, both the head camera and the hand camera are RGB-D cameras; the RGB-D camera can obtain the distance information of the target from the camera while obtaining the target RGB image, and perceive the RGB information and depth information at one time, avoiding the calibration problem caused by using the RGB camera and the depth camera separately.

3. The method for autonomous target measurement of an on-orbit maintenance robot in space according to claim 1, characterized in that: In the step 1, the communication part adopts lightweight communication and grouping LCM.

4. The method for autonomous target measurement of an on-orbit maintenance robot in space according to claim 1, characterized in that: In step 2, the hand-eye calibration is to unify all information into the robot arm coordinate system. The specific method is as follows: S21, fix the RGB-D camera, perform hand-eye calibration after fixing the position, and do not change the position of the camera; S22. According to the camera's field of view and the length of the robot arm, select a calibration plate of appropriate size to make the information more accurate and effective during calibration; S23, calibrate the hand camera at the end of the robot arm using the eyes-on-hand method, that is, solve the relationship between the camera and the end of the robot; calibrate the head camera of the robot arm using the eyes-outside-hand method, that is, solve the relationship between the camera and the robot base; move the robot arm multiple times and ensure that the calibration plate is always within the camera field of view; S24. Use the Tsai method to solve the problem in the hand-eye calibration problem and obtain the target matrix, which is the relationship between the camera coordinate system and the robotic arm coordinate system. After defining the robotic arm coordinate system, the earth coordinate system is artificially defined according to the robotic arm coordinate system.

5. The method for autonomous target measurement of an on-orbit maintenance robot in space according to claim 1, characterized in that: In S34, the accuracy of the template point cloud is equal to or higher than the acquisition accuracy of the camera when it is at an extremely close distance to the target; After making the template point cloud, fix the template point cloud at a known position to obtain the pose relationship between the two point clouds; fix the template point cloud at the origin of the camera coordinate system to facilitate the transformation and calculation of the coordinate system; The point cloud registration model is an improved model based on ICP.

6. The method for autonomous target measurement of an on-orbit maintenance robot in space according to claim 1, characterized in that: In step 4, the specific method for the end of the robotic arm to move to the ultra-close distance of the target is: S41, the target position of the robot arm is set to a coordinate position a certain distance above the target center point coordinate in step S3, and the target posture of the robot arm is the same as the target posture in step S3; The distance between the robot arm's moving target position and the actual grasping target is mainly related to the length of the robot arm's gripper. The purpose is to set an ultra-close distance so that the hand RGB-D camera can perform high-precision pose estimation of the grasping target within the nearest working range without the robot arm touching the target object. S42, performing path planning according to the target posture set in step S41 and the obstacle map acquired in step S31, and smoothing the path according to the dynamics of the robot arm to obtain a smooth trajectory; S43, sending the calculated motion trajectory to the control module through LCM communication, controlling the robotic arm to track the planned trajectory, and moving the end of the robotic arm to an ultra-close distance to the target.

7. The method for autonomous target measurement of an on-orbit maintenance robot in space according to claim 1, characterized in that: The step five is the same as the steps three and four, and the high-precision pose of the target is obtained by aligning the point cloud registration model template point cloud with the target point cloud.

8. The method for autonomous target measurement of an on-orbit maintenance robot in space according to claim 7, characterized in that: The difference between step 5 and step 3 and step 4 is that when step 3 performs point cloud registration, the camera is far away from the target, and the number of point clouds in the two groups is different. The registration result is not accurate and only needs to provide the target position for step 4. In step five, the hand camera is closer to the target, the collected point cloud is more accurate, the registration result is more accurate, and the high-precision pose of the target can be obtained.

9. The method for autonomous target measurement of an on-orbit maintenance robot in space according to claim 1, characterized in that: In step 6, the specific process of executing the crawling is: S61, high-precision pose information includes 7 data, namely, the position information (x, y, z) and quaternion of the target center point in the coordinate system of the robot base; S62, through LCM communication, high-precision posture information is sent from the computer of the vision and planning part and received by the computer of the control part; S63. According to the received position information, the robot arm is controlled to move and the gripper is used to grasp.

Images