Robot environment information conversion method and device, storage medium and program product
By detecting the relative pose changes of the sensors, constructing candidate pose information groups and converting environmental information, the accuracy problem of the sensors when their relative pose changes is solved, and efficient and accurate environmental information conversion is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHONGQING PHOENIX TECHNOLOGY CO LTD
- Filing Date
- 2026-03-18
- Publication Date
- 2026-06-12
AI Technical Summary
Existing technologies cannot effectively guarantee the accuracy of environmental information conversion when the relative pose of the sensor changes.
By detecting the relative attitude change of the sensor, the current motion direction and pose change parameters are determined. Interpolation processing is performed using standard pose information and reference pose information to construct a candidate pose information group. Based on the target pose information group, the environmental information is transformed into the coordinate system of the second sensor.
It improves the efficiency and accuracy of environmental information conversion, adapts to the actual pose changes of sensors, and ensures the accuracy of environmental information conversion.
Smart Images
Figure CN122185183A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of sensor calibration technology, and in particular to a method, apparatus, storage medium, and program product for converting environmental information of a robot. Background Technology
[0002] With the continuous development of the robotics field, in order to ensure the coordinated operation of various components within a robot, calibration processing of the robot's sensors has emerged, thereby transforming the environmental information collected by each sensor into the same coordinate system. Generally, this involves using pre-set calibration boards and other tools to determine the pose relationships between the sensors, thus achieving the conversion of environmental information collected by multiple sensors.
[0003] However, the above method can only convert information collected by multiple sensors with fixed relative poses. If the relative pose of any sensor changes, the accuracy of environmental information conversion will be reduced if the original relative pose information between multiple sensors is still used. Summary of the Invention
[0004] Based on this, this application addresses the aforementioned technical problems by providing a method, apparatus, storage medium, and program product for converting environmental information of a robot that can improve the accuracy of sensor calibration.
[0005] Firstly, this application provides a method for converting environmental information of a robot, including:
[0006] When a change in the relative posture of the first sensor and the second sensor in the robot is detected, the current motion direction and current posture change parameters of the first sensor are determined based on the current pose of the first sensor and the standard pose of the first sensor when the robot is in a standard state.
[0007] From the candidate pose information groups corresponding to each candidate motion direction, select the target pose information group corresponding to the current motion direction; wherein, each candidate pose information group corresponding to a candidate motion direction is obtained by processing the standard pose information and the reference pose information corresponding to the candidate motion direction; the standard pose information is the relative pose information between the first sensor and the second sensor when the robot is in a standard state;
[0008] Based on the target pose information group and the current pose change parameters, the environmental information collected by the first sensor is transformed into the coordinate system of the second sensor.
[0009] In the above-mentioned environmental information conversion method for robots, when a change in the relative posture of the first sensor and the second sensor in the robot is detected, the current motion direction and current posture change parameters of the first sensor are determined based on the current pose of the first sensor and the standard pose of the first sensor when the robot is in a standard state. Then, from the candidate pose information groups corresponding to each candidate motion direction, the target pose information group corresponding to the current motion direction is selected, and the environmental information collected by the first sensor is converted to the coordinate system of the second sensor based on the target pose information group and the current pose change parameters. Using the above method, on the one hand, by processing the standard pose information and the reference pose information corresponding to the candidate motion direction, the candidate pose information group corresponding to each candidate motion direction is obtained in advance. In the subsequent environmental information conversion operation, the required pose information can be obtained without real-time calculation, which can effectively improve the efficiency of the subsequent environmental information conversion processing. On the other hand, when the relative posture between the robot sensors changes, by combining the target pose information group in the current motion direction and the current pose change parameters, the environmental information collected by the first sensor is converted to the coordinate system of the second sensor, so that the environmental information conversion operation is adapted to the actual pose change of the sensor, thereby ensuring the accuracy of the environmental information conversion.
[0010] In an optional embodiment of the first aspect, processing the standard pose information and the reference pose information corresponding to the candidate motion direction includes:
[0011] Based on the motion range of the first sensor, determine the candidate calibration position of the first sensor in each candidate motion direction;
[0012] For each candidate motion direction, after adjusting the first sensor to the candidate calibration position of the candidate motion direction, the first sensor and the second sensor are calibrated to obtain the relative pose information between the first sensor and the second sensor, which is used as the reference pose information corresponding to the candidate motion direction.
[0013] Interpolation processing is performed on the standard pose information and the reference pose information corresponding to the candidate motion direction to obtain the candidate pose information group corresponding to the candidate motion direction.
[0014] In the above optional embodiments, by interpolating the standard pose information and the reference pose information, a candidate pose information group is obtained. It is not necessary to calibrate each movable position of the first sensor one by one. This can improve the efficiency of candidate pose information group determination while ensuring the comprehensiveness of the candidate pose information group coverage.
[0015] In an optional embodiment of the first aspect, interpolation processing is performed on the standard pose information and the reference pose information corresponding to the candidate motion direction to obtain a candidate pose information group corresponding to the candidate motion direction, including:
[0016] Interpolation is performed on the standard rotation matrix in the standard pose information and the reference rotation matrix in the reference pose information to obtain the interpolated rotation matrix.
[0017] Interpolation is performed on the standard translation vector in the standard pose information and the reference translation vector in the reference pose information to obtain the interpolated translation vector.
[0018] Based on the interpolation rotation matrix and the interpolation translation vector, the candidate pose information group corresponding to the candidate motion direction is determined.
[0019] In the above optional embodiments, by interpolating the rotation matrix and translation vector respectively, an interpolated rotation matrix and an interpolated translation vector are obtained, thereby constructing a candidate pose information group, which can ensure the rationality and accuracy of the candidate pose information group construction.
[0020] In an optional embodiment of the first aspect, interpolation processing is performed on the standard rotation matrix in the standard pose information and the reference rotation matrix in the reference pose information to obtain an interpolated rotation matrix, including:
[0021] The standard rotation matrix in the standard pose information is converted into the first quaternion;
[0022] The reference rotation matrix in the reference pose information is converted into a second quaternion;
[0023] Interpolate the first quaternion and the second quaternion to obtain the interpolated quaternion, and then convert the interpolated quaternion into an interpolation rotation matrix.
[0024] In the above optional embodiments, by converting the rotation matrix into a quaternion, the interpolation process can be simplified, thereby improving the efficiency and accuracy of determining the interpolation rotation matrix.
[0025] In an optional embodiment of the first aspect, determining the candidate calibration position of the first sensor in each candidate motion direction based on the motion range of the first sensor includes:
[0026] For each candidate motion direction, the motion boundary position of the first sensor in the candidate motion direction is determined based on the motion range of the first sensor.
[0027] The location of the motion boundary is used as the candidate calibration location for the candidate motion direction.
[0028] In the above optional embodiments, by using the motion boundary position as the candidate calibration position, it is possible to ensure the comprehensiveness of the motion position coverage of the subsequently determined candidate pose information group, thereby ensuring the accuracy of sensor calibration.
[0029] In an optional embodiment of the first aspect, transforming the environmental information collected by the first sensor into the coordinate system of the second sensor based on the target pose information group and the current pose change parameters includes:
[0030] Obtain the current pose information corresponding to the current pose change parameters from the target pose information group;
[0031] Using the current pose information, the environmental information collected by the first sensor is transformed into the coordinate system of the second sensor.
[0032] In the above optional embodiments, by obtaining the current pose information from the target pose information group according to the current pose change parameters, the environmental information collected by the first sensor can be converted, thus ensuring the accuracy of the environmental information conversion.
[0033] In an optional embodiment of the first aspect, the first sensor and the second sensor are calibrated to obtain relative pose information between the first sensor and the second sensor, which serves as reference pose information corresponding to the candidate motion direction, including:
[0034] The first calibration position of the standard calibration board is obtained through the first sensor, and the second calibration position of the standard calibration board is obtained through the second sensor.
[0035] Based on the first calibration position, the second calibration position, and the sensor parameters of the first sensor, the first sensor and the second sensor are calibrated to obtain the relative pose information between the first sensor and the second sensor, which serves as the reference pose information corresponding to the candidate motion direction.
[0036] In the above optional embodiments, by using the same standard calibration board to calibrate the first sensor and the second sensor, the accuracy of sensor calibration can be guaranteed.
[0037] Secondly, this application also provides an environmental information conversion device for a robot, comprising:
[0038] The state acquisition module is used to determine the current motion direction and current pose change parameters of the first sensor when the relative posture of the first sensor and the second sensor in the robot changes, based on the current pose of the first sensor and the standard pose of the first sensor when the robot is in a standard state.
[0039] The information acquisition module is used to select the target pose information group corresponding to the current motion direction from the candidate pose information groups corresponding to each candidate motion direction; wherein, the candidate pose information group corresponding to each candidate motion direction is obtained by processing the standard pose information and the reference pose information corresponding to the candidate motion direction; the standard pose information is the relative pose information between the first sensor and the second sensor when the robot is in the standard state;
[0040] The information conversion module is used to convert the environmental information collected by the first sensor to the coordinate system of the second sensor based on the target pose information group and the current pose change parameters.
[0041] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described above.
[0042] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described above.
[0043] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in any of the above aspects.
[0044] Regarding the beneficial effects of any of the technical solutions in the second to fifth aspects mentioned above, refer to the beneficial effects of the corresponding technical solutions in the first aspect; repeated examples will not be listed here. Attached Figure Description
[0045] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0046] Figure 1 This is a schematic diagram of an optional process for a robot's environmental information conversion method in one embodiment;
[0047] Figure 2 This is a schematic diagram of the internal structure of the robot in one embodiment;
[0048] Figure 3 This is a schematic diagram of an optional process for determining a group of candidate pose information in one embodiment;
[0049] Figure 4 This is a schematic diagram of an optional process for determining a group of candidate pose information in another embodiment;
[0050] Figure 5 This is a schematic diagram of an optional process for determining the interpolation rotation matrix in one embodiment;
[0051] Figure 6This is a schematic diagram of an optional process for environmental information conversion in one embodiment;
[0052] Figure 7 This is a schematic diagram of an optional process for determining reference pose information in one embodiment;
[0053] Figure 8 This is a schematic diagram of an optional process for a robot's environmental information conversion method in another embodiment;
[0054] Figure 9 This is a schematic diagram of an optional structure of the robot's environmental information conversion device in one embodiment;
[0055] Figure 10 This is a schematic diagram of an optional internal structure of a computer device in one embodiment. Detailed Implementation
[0056] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of this application.
[0057] With the continuous development of the robotics field, in order to ensure the coordinated operation of various components within a robot, calibration processing of the robot's sensors has emerged. Generally, this involves using pre-set calibration boards and other tools to determine the pose relationships between the sensors, thereby achieving calibration processing for multiple sensors.
[0058] However, the above method can only calibrate multiple sensors with fixed relative poses. If the relative pose of any sensor changes, the accuracy of sensor calibration will be reduced if the original relative pose information between multiple sensors is still used for calibration.
[0059] Based on this, in an exemplary embodiment, a method for converting environmental information of a robot is provided. The method is illustrated using an example of its application to a robot. Figure 1 As shown, it includes the following steps:
[0060] S101, if a change in the relative posture of the first sensor and the second sensor in the robot is detected, the current motion direction and current posture change parameters of the first sensor are determined based on the current pose of the first sensor and the standard pose of the first sensor when the robot is in a standard state.
[0061] The robot includes a first sensor and a second sensor, both of which have information acquisition capabilities. For example, they can be cameras, lidar, and inertial measurement units (IMUs). The relative attitude refers to the attitude difference between the first and second sensors.
[0062] In a robot coordinate system constructed with the robot as the origin, the first sensor is a sensor with a changeable pose, and the second sensor is a sensor with a fixed pose. For example, refer to... Figure 2 The diagram shows the internal structure of the robot. The robot's motion control module sends motion commands to the domain controller. The domain controller then adjusts the positions of the sensors using multi-degree-of-freedom motors. For example, the first sensor could be a camera mounted inside the robot's head; its pose changes in the robot's coordinate system as the head rotates. The second sensor could be a LiDAR or IMU located in the robot's torso. While the robot moves, the LiDAR or IMU in the torso also moves, but its pose in the robot's coordinate system remains unchanged.
[0063] The current pose refers to the position and orientation of the first sensor at the current moment. The standard state refers to the robot's state without any change in orientation; for example, the standard state could be that the robot's head and arm are in their factory settings. The standard pose refers to the robot's pose in the standard state. The current direction of motion refers to the direction of motion of the first sensor in the robot's coordinate system. For example, when the robot's head rotates to the left, the current direction of motion of the first sensor located within the head is left. The current pose change parameter refers to the degree of pose change at the current moment, such as rotation angle or translation amount.
[0064] In one alternative approach, if a change in the relative pose of the first and second sensors in the robot is detected, it indicates that recalibration between the two sensors is necessary. In this case, the standard pose of the first sensor in the robot's standard state can be used as a reference. Based on the pose difference between the current pose and the standard pose, the current motion direction and pose change parameters of the first sensor are determined. For example, if the pose difference between the current pose and the standard pose of the first sensor is a 45° leftward rotation, then the current motion direction is left, and the current pose change parameter is 45°.
[0065] S102, select the target pose information group corresponding to the current motion direction from the candidate pose information groups corresponding to each candidate motion direction.
[0066] The candidate motion direction refers to the direction in which the first sensor can move. The candidate pose information group contains pose information of the first and second sensors under different pose parameters, used for calibration processing. The pose information may include, but is not limited to, rotation matrices and translation vectors.
[0067] The candidate pose information group corresponding to each candidate motion direction is obtained by processing the standard pose information and the reference pose information corresponding to the candidate motion direction. The standard pose information is the relative pose information between the first and second sensors when the robot is in a standard state. The reference pose information is the relative pose information between the first and second sensors when the first sensor is located at a preset candidate calibration position. The candidate calibration positions are different for each candidate motion direction. The target pose information group is the pose information group corresponding to the current motion direction.
[0068] In one alternative approach, before the robot begins operation, corresponding candidate calibration positions can be selected for each candidate motion direction. Then, when the first sensor is located at different candidate calibration positions, calibration processing is performed on both the first and second sensors to obtain reference pose information for each candidate motion direction. For each candidate motion direction, the reference pose information and standard pose information in that direction can be used as boundary values to construct a candidate pose information group for that direction.
[0069] Based on this, after detecting a change in the relative posture of the first and second sensors in the robot, the current direction of motion can be directly used as an index to query the candidate pose information group corresponding to each candidate direction of motion, thereby obtaining the target pose information group corresponding to the current direction of motion.
[0070] S103, based on the target pose information group and the current pose change parameters, convert the environmental information collected by the first sensor to the coordinate system of the second sensor.
[0071] Among them, the so-called environmental information can be the information of the robot's environment collected by the sensors, such as obstacle information.
[0072] In one alternative approach, the relative pose information between the first sensor and the second sensor in the current pose can be determined based on the current pose change parameters and the target pose information set. This relative pose information may include the rotation matrix and translation vector between the first and second sensors. Subsequently, the relative pose information can be used to perform position transformation processing on the environmental information acquired by the first sensor in its own coordinate system to obtain the environmental information located in the coordinate system of the second sensor.
[0073] The environmental information obtained after transformation, located in the coordinate system of the second sensor, and the environmental information collected by the second sensor itself, located in the coordinate system of the second sensor, are fused to obtain fused environmental information. For example, taking the first sensor as a camera and the second sensor as a lidar, the rotation matrix and translation vector between the camera and the lidar can be used to convert the image information collected by the camera in the camera coordinate system into image information in the lidar coordinate system; then, the fused environmental information is generated based on the image information in the lidar coordinate system and the point cloud information collected by the lidar.
[0074] Based on the fused environmental information, the robot is instructed to perform subsequent processing. For example, obstacle avoidance can be performed by the robot based on obstacle information contained in the fused environmental information. Alternatively, the robot can be instructed to pick up the object based on the object information contained in the fused environmental information.
[0075] In the above-mentioned environmental information conversion method for robots, when a change in the relative posture of the first sensor and the second sensor in the robot is detected, the current motion direction and current posture change parameters of the first sensor are determined based on the current pose of the first sensor and the standard pose of the first sensor when the robot is in a standard state. Then, from the candidate pose information groups corresponding to each candidate motion direction, the target pose information group corresponding to the current motion direction is selected, and the environmental information collected by the first sensor is converted to the coordinate system of the second sensor based on the target pose information group and the current pose change parameters. Using the above method, on the one hand, by processing the standard pose information and the reference pose information corresponding to the candidate motion direction, the candidate pose information group corresponding to each candidate motion direction is obtained in advance. In the subsequent environmental information conversion operation, the required pose information can be obtained without real-time calculation, which can effectively improve the efficiency of the subsequent environmental information conversion processing. On the other hand, when the relative posture between the robot sensors changes, by combining the target pose information group in the current motion direction and the current pose change parameters, the environmental information collected by the first sensor is converted to the coordinate system of the second sensor, so that the environmental information conversion operation is adapted to the actual pose change of the sensor, thereby ensuring the accuracy of the environmental information conversion.
[0076] Based on the above embodiments, this application provides an optional method for determining candidate pose information groups, such as... Figure 3 As shown, it includes the following steps:
[0077] S301, Based on the motion range of the first sensor, determine the candidate calibration position of the first sensor in each candidate motion direction.
[0078] The range of motion refers to the area within which the first sensor can move. For example, when the first sensor is deployed on the robot's head, the range of motion can be the rotational range of the robot's head. The candidate calibration position refers to the position in the candidate motion direction where calibration processing is to be performed.
[0079] In one alternative approach, motion locations with a motion frequency greater than a preset threshold and located within the motion range can be selected from the historical motion records of the first sensor as candidate calibration locations of the first sensor in each candidate motion direction.
[0080] In one alternative approach, for each candidate motion direction, the candidate calibration position of the first sensor in each candidate motion direction can be determined based on the motion range of the first sensor in that motion direction and a preset motion interval. For example, if the motion range of the first sensor is a 90° leftward rotation, and the preset motion interval is 45°, then the candidate calibration positions of the first sensor in the leftward direction are 45° and 90°, respectively.
[0081] S302, for each candidate motion direction, after adjusting the first sensor to the candidate calibration position of the candidate motion direction, calibrate the first sensor and the second sensor to obtain the relative pose information between the first sensor and the second sensor, which is used as the reference pose information corresponding to the candidate motion direction.
[0082] In one alternative approach, for each candidate motion direction, after adjusting the first sensor to the candidate calibration position for that motion direction, a calibration reference at a preset position can be used to calibrate the first and second sensors, thereby obtaining the relative pose information between the first and second sensors. For example, the calibration processing of the first and second sensors can be performed by combining the position information of the calibration reference acquired by the first and second sensors respectively. In this case, the obtained relative pose information can be used as the reference pose information corresponding to that candidate motion direction.
[0083] S303, interpolate the standard pose information and the reference pose information corresponding to the candidate motion direction to obtain the candidate pose information group corresponding to the candidate motion direction.
[0084] In one alternative approach, for each candidate motion direction, in order to ensure the comprehensive coverage of relative pose information in that candidate motion direction, the reference pose information and standard pose information corresponding to that candidate motion direction can be interpolated to obtain a candidate pose information group corresponding to that candidate motion direction, thereby supplementing the relative pose information between the first sensor and the second sensor when the first sensor is located at each motion position in that candidate motion direction.
[0085] For example, standard pose information can be used as the processing starting point and reference pose information as the reference processing point. Interpolation processing is performed on other motion positions between the processing starting point and the maximum motion position according to a preset step size to obtain the relative pose information at each motion position. Then, according to the order of motion position changes, the standard pose information, the relative pose information at each motion position, and the reference pose information are combined to obtain a candidate pose information group.
[0086] In the above optional embodiments, by interpolating the standard pose information and the reference pose information, a candidate pose information group is obtained. It is not necessary to calibrate each movable position of the first sensor one by one. This can improve the efficiency of candidate pose information group determination while ensuring the comprehensiveness of the candidate pose information group coverage.
[0087] Based on the above embodiments, this application provides another optional method for determining candidate pose information groups, such as... Figure 4 As shown, it includes the following steps:
[0088] S401, interpolate the standard rotation matrix in the standard pose information and the reference rotation matrix in the reference pose information to obtain the interpolated rotation matrix.
[0089] The standard rotation matrix is the rotation matrix between the first sensor and the second sensor when the first sensor is in its standard state. The reference rotation matrix is the rotation matrix between the first sensor and the second sensor when the first sensor is in its candidate calibration position. The rotation matrix characterizes the difference in rotation direction between the first and second sensors. The interpolated rotation matrix is the rotation matrix obtained through interpolation.
[0090] In one alternative approach, the standard pose of the first sensor when it is in a standard state can be determined, along with the angle between this pose and the reference pose when it is located at a candidate calibration position corresponding to the reference pose information. Then, based on a preset step size and the angle, the standard rotation matrix and the reference rotation matrix are interpolated to obtain an interpolated rotation matrix.
[0091] S402, interpolate the standard translation vector in the standard pose information and the reference translation vector in the reference pose information to obtain the interpolated translation vector.
[0092] The standard translation vector is the translation vector between the first sensor and the second sensor when the first sensor is in its standard state. The reference translation vector is the translation vector between the first sensor and the second sensor when the first sensor is in its candidate calibration position. The translation vector is used to characterize the positional difference between the first and second sensors. The interpolated translation vector is the translation vector obtained through interpolation.
[0093] In one alternative approach, the standard translation vector and the reference translation vector can be interpolated according to a preset step size to obtain the interpolated translation vector. For example, referring to the formula t(t)=(1-t)t1+t×t2, i.e., the linear interpolation formula Lerp, the standard translation vector t1 and the reference translation vector t2 can be interpolated according to the interpolation parameter t to obtain the interpolated translation vector t(t). Where t∈[0,1].
[0094] S403, determine the candidate pose information group corresponding to the candidate motion direction based on the interpolation rotation matrix and the interpolation translation vector.
[0095] In one alternative approach, relative pose information at each motion position can be constructed based on the interpolation rotation matrix and interpolation translation vector obtained from the interpolation process, and the relative pose information at each motion position can be used as a candidate pose information group corresponding to the candidate motion direction.
[0096] For example, referring to the following formula (1), a new relative pose matrix T(t) can be constructed based on the interpolation rotation matrix R(t) and the interpolation translation vector t(t), serving as the candidate pose information group corresponding to the candidate motion direction. In the subsequent real-time calibration process, the current pose change parameters can be converted into the corresponding interpolation parameters, thereby querying the corresponding relative pose information in T(t).
[0097] (1)
[0098] In the above optional embodiments, by interpolating the rotation matrix and translation vector respectively, an interpolated rotation matrix and an interpolated translation vector are obtained, thereby constructing a candidate pose information group, which can ensure the rationality and accuracy of the candidate pose information group construction.
[0099] Based on the above embodiments, this application provides an optional method for determining the interpolation rotation matrix, such as... Figure 5 As shown, it includes the following steps:
[0100] S501 converts the standard rotation matrix in the standard pose information into the first quaternion.
[0101] In this context, a quaternion is a mathematical representation of a rotation state or orientation. The first quaternion is the quaternion corresponding to a standard rotation matrix. A standard rotation matrix can be a 3×3 rotation matrix.
[0102] In one alternative approach, the sum of the matrix parameters along the diagonal of the standard rotation matrix can be calculated first, i.e., the diagonal sum. Then, based on the diagonal sum and the other matrix parameters in the standard rotation matrix, the first quaternion corresponding to the standard rotation matrix can be calculated.
[0103] S502 converts the reference rotation matrix in the reference pose information into a second quaternion.
[0104] The so-called second quaternion is the quaternion corresponding to the reference rotation matrix. The reference rotation matrix can be a 3×3 rotation matrix.
[0105] In one alternative approach, the sum of the matrix parameters along the diagonal of the reference rotation matrix can be calculated first, i.e., the diagonal sum. Then, based on the diagonal sum and the other matrix parameters in the reference rotation matrix, the second quaternion corresponding to the reference rotation matrix can be calculated.
[0106] S503, interpolate the first quaternion and the second quaternion to obtain the interpolated quaternion, and convert the interpolated quaternion into an interpolation rotation matrix.
[0107] The so-called interpolated quaternion is the quaternion obtained after interpolation.
[0108] In one alternative approach, the standard pose of the first sensor when it is in a standard state can be determined, and the angle between this pose and the reference pose when it is located at a candidate calibration position corresponding to the reference pose information can be determined. Then, based on the angle deviation and a preset step size, the first quaternion and the second quaternion are interpolated to obtain the interpolated quaternion. For example, referring to the following formula (2), the spherical linear interpolation (Slerp) formula can be used to interpolate the first quaternion q1 and the second quaternion q2 based on the angle θ and the interpolation parameter t to obtain the interpolated quaternion q(t).
[0109] (2)
[0110] According to the preset transformation formula, the real and imaginary parts of the interpolation quaternion are processed to convert the interpolation quaternion into an interpolation rotation matrix.
[0111] In the above optional embodiments, by converting the rotation matrix into a quaternion, the interpolation process can be simplified, thereby improving the efficiency and accuracy of determining the interpolation rotation matrix.
[0112] Based on the above embodiments, this application provides an optional method for determining candidate calibration positions. Specifically, for each candidate motion direction, the motion boundary position of the first sensor in the candidate motion direction is determined according to the motion range of the first sensor; the motion boundary position is used as the candidate calibration position of the candidate motion direction.
[0113] The so-called motion boundary position is the maximum position that the first sensor can move in the candidate motion direction.
[0114] In one alternative approach, to ensure the comprehensive coverage of the candidate pose information group, for each candidate motion direction, the motion boundary position of the first sensor in that candidate motion direction can be determined based on the motion range of the first sensor, and the motion boundary position can be used as the candidate calibration position for that candidate motion direction.
[0115] For example, when the first sensor is deployed on the robot's head, the direction of movement can include up, down, left, and right. In this case, the candidate calibration positions can include the upper limit movement position, the lower limit movement position, the left limit movement position, and the right limit movement position.
[0116] In the above optional embodiments, by using the motion boundary position as the candidate calibration position, it is possible to ensure the comprehensiveness of the motion position coverage of the subsequently determined candidate pose information group, thereby ensuring the accuracy of sensor calibration.
[0117] Based on the above embodiments, this application provides an optional method for environmental information conversion, such as... Figure 6 As shown, it includes the following steps:
[0118] S601, Obtain the current pose information corresponding to the current pose change parameters from the target pose information group.
[0119] The so-called current pose information refers to the relative pose information corresponding to the current pose change parameters in the target pose information group.
[0120] In one alternative approach, the interpolation parameter corresponding to the current pose change parameter can be determined based on the parameter ratio between the current pose change parameter and the maximum pose change parameter in the current motion direction. Then, the interpolation parameter corresponding to the current pose change parameter is used as an index to query the target pose information group, thereby obtaining the current pose information corresponding to the current pose change parameter.
[0121] S602 uses the current pose information to convert the environmental information collected by the first sensor to the coordinate system of the second sensor.
[0122] In one alternative approach, the position of the environmental information acquired by the first sensor can be transformed based on the sensor parameters of the first sensor and the rotation matrix and translation vector in the current pose information to obtain the environmental information in the coordinate system of the second sensor.
[0123] For example, in a robot navigation scenario, taking a camera as the first sensor and a LiDAR as the second sensor, the pixel information in the two-dimensional coordinates of the image information acquired by the camera can be converted into pixel information in the three-dimensional coordinates of the coordinate system where the LiDAR is located, based on the camera's intrinsic parameter matrix and the rotation matrix and translation vector in the current pose information. Then, based on the three-dimensional point cloud information acquired by the LiDAR and the pixel information in the three-dimensional coordinates, the fused environment information after position alignment is determined. Finally, a three-dimensional model corresponding to the robot navigation scenario is constructed based on the aligned fused environment information, enabling the robot to better plan its navigation path.
[0124] In one alternative approach, candidate pose information groups for each motion direction of the first sensor relative to the robot reference point, and fixed pose information for the second sensor relative to the robot reference point, can be constructed based on the aforementioned construction logic for candidate pose information groups. In practical applications, the current pose information of the first sensor relative to the robot reference point can be obtained from the candidate pose information group for the current motion direction based on the current motion direction and current pose change parameters of the first sensor. Then, the environmental information collected by the first sensor is transformed into the robot coordinate system using the current pose information, while the environmental information collected by the second sensor is transformed into the robot coordinate system using the fixed pose information.
[0125] The two types of environmental information in the robot coordinate system are fused to obtain fused environmental information in the robot coordinate system. Then, based on the fused environmental information in the robot coordinate system, the robot is instructed to perform subsequent processing.
[0126] In the above optional embodiments, by obtaining the current pose information from the target pose information group according to the current pose change parameters, the environmental information collected by the first sensor can be converted, thus ensuring the accuracy of the environmental information conversion.
[0127] Based on the above embodiments, this application provides an optional method for determining reference pose information, such as... Figure 7 As shown, it includes the following steps:
[0128] S701 obtains the first calibration position of the standard calibration board through the first sensor and the second calibration position of the standard calibration board through the second sensor.
[0129] The standard calibration board is a calibration reference used for calibration processing, such as a checkerboard or circular array board. The positional accuracy of the standard calibration board is required to be less than 1 mm. The first calibration position is the positional data of each key point on the standard calibration board acquired by the first sensor. The second calibration position is the positional data of each key point on the standard calibration board acquired by the second sensor.
[0130] In one alternative approach, a robot coordinate system can be constructed. For example, the robot coordinate system can be established with the center of the robot's two feet or the center of the IMU as the origin, where the front of the robot is the X direction, the right side of the robot is the Y direction, and the top of the robot is the Z direction.
[0131] After placing the standard calibration board within the shared field of view of the first and second sensors, the first calibration position of each key point on the standard calibration board is obtained by adjusting the first sensor to the candidate calibration position. Simultaneously, the second calibration position of each key point on the standard calibration board is obtained by the second sensor. For example, the pixel coordinates (u, v) of the key points are obtained from the standard calibration board image acquired by the camera, and the three-dimensional point cloud coordinates (X, v) of the key points in the standard calibration board image are obtained by using the LiDAR. W ,Y W Z W ).
[0132] In practical applications, if the robot has a large range of motion, multiple calibration plates can be set up at the same time.
[0133] S702, based on the first calibration position, the second calibration position, and the sensor parameters of the first sensor, calibrate the first sensor and the second sensor to obtain the relative pose information between the first sensor and the second sensor, which serves as the reference pose information corresponding to the candidate motion direction.
[0134] The sensor parameters refer to the attribute parameters of the first sensor. For example, they could be the camera's intrinsic parameter matrix, which includes focal length and optical center coordinates. In a vehicle obstacle avoidance scenario, the camera's focal length could be the focal length of the vehicle's reversing camera.
[0135] In one alternative approach, the sensor parameters of the first sensor can be combined to calibrate the first and second calibration positions of the same key point, thereby obtaining the relative pose information between the first and second sensors. This relative pose information can then be used as the reference pose information corresponding to the candidate motion direction.
[0136] For example, referring to the following formula (3), based on the sensor parameter matrix K, the pixel coordinates (u,v) and 3D point cloud coordinates (X) of the same key point can be calculated. W ,Y W Z W The calibration process is performed to obtain the relative pose information [RT].
[0137] (3)
[0138] For example, taking vehicle calibration as an example, the vehicle to be calibrated can be driven into the calibration field limiter to lock the distance between the vehicle and the calibration plate in front. Then, a diagnostic tool is connected to the vehicle's On-Board Diagnostics (OBD) system to send a calibration command to the vehicle's Controller Area Network (CAN), triggering the calibration function to begin calibration. Upon completion, the calibration result is returned. If calibration is successful, the diagnostic tool displays "Calibration Successful"; if calibration fails, the diagnostic tool displays a "Calibration Failure Fault Code," and calibration is restarted until completion.
[0139] In the above optional embodiments, by using the same standard calibration board to calibrate the first sensor and the second sensor, the accuracy of sensor calibration can be guaranteed.
[0140] Figure 8 This is a flowchart illustrating a robot environmental information conversion method in another embodiment. Based on the above embodiments, this embodiment provides an optional example of a robot environmental information conversion method. (Combined with...) Figure 8 The specific implementation process is as follows:
[0141] S801, based on the motion range of the first sensor in the robot, determine the candidate calibration position of the first sensor in each candidate motion direction.
[0142] In one alternative approach, for each candidate motion direction, the motion boundary position of the first sensor in the candidate motion direction is determined based on the motion range of the first sensor; the motion boundary position is then used as the candidate calibration position for the candidate motion direction.
[0143] S802, for each candidate motion direction, after adjusting the first sensor to the candidate calibration position of the candidate motion direction, calibrate the first sensor and the second sensor to obtain the relative pose information between the first sensor and the second sensor, which is used as the reference pose information corresponding to the candidate motion direction.
[0144] In one alternative approach, a first calibration position of the standard calibration board is obtained through a first sensor, and a second calibration position of the standard calibration board is obtained through a second sensor; based on the first calibration position, the second calibration position, and the sensor parameters of the first sensor, the first sensor and the second sensor are calibrated to obtain the relative pose information between the first sensor and the second sensor, which serves as the reference pose information corresponding to the candidate motion direction.
[0145] S803 performs interpolation processing on the standard rotation matrix in the standard pose information and the reference rotation matrix in the reference pose information to obtain the interpolated rotation matrix, and performs interpolation processing on the standard translation vector in the standard pose information and the reference translation vector in the reference pose information to obtain the interpolated translation vector.
[0146] Among them, the standard pose information is the relative pose information between the first sensor and the second sensor when the robot is in a standard state.
[0147] In one alternative approach, the standard rotation matrix in the standard pose information is converted into a first quaternion; the reference rotation matrix in the reference pose information is converted into a second quaternion; the first and second quaternions are interpolated to obtain an interpolated quaternion, and the interpolated quaternion is converted into an interpolated rotation matrix.
[0148] S804: Determine the candidate pose information group corresponding to the candidate motion direction based on the interpolation rotation matrix and the interpolation translation vector.
[0149] S805, if a change in the relative posture of the first sensor and the second sensor is detected, the current motion direction and current posture change parameters of the first sensor are determined based on the current pose of the first sensor and the standard pose of the first sensor when the robot is in a standard state.
[0150] S806: Select the target pose information group corresponding to the current motion direction from the candidate pose information groups corresponding to each candidate motion direction.
[0151] S807: Obtain the current pose information corresponding to the current pose change parameters from the target pose information group, and use the current pose information to convert the environmental information collected by the first sensor to the coordinate system of the second sensor.
[0152] The specific processes of S801-S807 described above can be found in the description of the above method embodiments. Their implementation principles and technical effects are similar, and will not be repeated here.
[0153] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.
[0154] Based on the same inventive concept, this application also provides an environmental information conversion device for implementing the above-described robot environmental information conversion method. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more robot environmental information conversion device embodiments provided below can be found in the limitations of the robot environmental information conversion method described above, and will not be repeated here.
[0155] In one exemplary embodiment, such as Figure 9 As shown, an environmental information conversion device 1 for a robot is provided, comprising: a state acquisition module 10, an information acquisition module 20, and an information conversion module 30, wherein:
[0156] The state acquisition module 10 is used to determine the current motion direction and current pose change parameters of the first sensor based on the current pose of the first sensor and the standard pose of the first sensor when the relative pose of the first sensor and the robot in the standard state is detected to have changed.
[0157] The information acquisition module 20 is used to select the target pose information group corresponding to the current motion direction from the candidate pose information groups corresponding to each candidate motion direction; wherein, the candidate pose information group corresponding to each candidate motion direction is obtained by processing the standard pose information and the reference pose information corresponding to the candidate motion direction; the standard pose information is the relative pose information between the first sensor and the second sensor when the robot is in the standard state;
[0158] The information conversion module 30 is used to convert the environmental information collected by the first sensor to the coordinate system of the second sensor based on the target pose information group and the current pose change parameters.
[0159] In an exemplary embodiment, the robot's environmental information conversion device 1 further includes a preprocessing module, wherein the preprocessing module is specifically used for:
[0160] Based on the motion range of the first sensor, the candidate calibration position of the first sensor in each candidate motion direction is determined; for each candidate motion direction, the first sensor is adjusted to the candidate calibration position of the candidate motion direction, and the first sensor and the second sensor are calibrated to obtain the relative pose information between the first sensor and the second sensor, which is used as the reference pose information corresponding to the candidate motion direction; the standard pose information and the reference pose information corresponding to the candidate motion direction are interpolated to obtain the candidate pose information group corresponding to the candidate motion direction.
[0161] In one exemplary embodiment, the preprocessing module is further configured to:
[0162] Interpolate the standard rotation matrix in the standard pose information and the reference rotation matrix in the reference pose information to obtain the interpolated rotation matrix; interpolate the standard translation vector in the standard pose information and the reference translation vector in the reference pose information to obtain the interpolated translation vector; and determine the candidate pose information group corresponding to the candidate motion direction based on the interpolated rotation matrix and the interpolated translation vector.
[0163] In one exemplary embodiment, the preprocessing module is further configured to:
[0164] The standard rotation matrix in the standard pose information is converted into a first quaternion; the reference rotation matrix in the reference pose information is converted into a second quaternion; the first and second quaternions are interpolated to obtain an interpolated quaternion, and the interpolated quaternion is converted into an interpolated rotation matrix.
[0165] In one exemplary embodiment, the preprocessing module is further configured to:
[0166] For each candidate motion direction, the motion boundary position of the first sensor in the candidate motion direction is determined according to the motion range of the first sensor; the motion boundary position is used as the candidate calibration position of the candidate motion direction.
[0167] In one exemplary embodiment, the information conversion module 30 is specifically used for:
[0168] Obtain the current pose information corresponding to the current pose change parameters from the target pose information group; use the current pose information to transform the environmental information collected by the first sensor into the coordinate system of the second sensor.
[0169] In one exemplary embodiment, the preprocessing module is further configured to:
[0170] The first calibration position of the standard calibration board is obtained through the first sensor, and the second calibration position of the standard calibration board is obtained through the second sensor. Based on the first calibration position, the second calibration position, and the sensor parameters of the first sensor, the first sensor and the second sensor are calibrated to obtain the relative pose information between the first sensor and the second sensor, which is used as the reference pose information corresponding to the candidate motion direction.
[0171] The various modules in the aforementioned robot's environmental information conversion device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the computer device's memory as software, so that the processor can call and execute the corresponding operations of each module.
[0172] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 10 As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a method for converting environmental information for a robot. The display unit is used to form a visually visible image and can be a display screen, projection device, or virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.
[0173] Those skilled in the art will understand that Figure 10The structure shown is a block diagram of a partial structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.
[0174] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.
[0175] In one exemplary embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above-described method embodiments.
[0176] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.
[0177] The data involved in this application (including but not limited to sensor pose data) are all data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.
[0178] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program mentioned can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.
[0179] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.
[0180] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.
Claims
1. A method for converting environmental information of a robot, characterized in that, The method includes: When a change in the relative posture of the first sensor and the second sensor in the robot is detected, the current motion direction and current posture change parameters of the first sensor are determined based on the current pose of the first sensor and the standard pose of the first sensor when the robot is in a standard state. From the candidate pose information groups corresponding to each candidate motion direction, select the target pose information group corresponding to the current motion direction; wherein, each candidate pose information group corresponding to a candidate motion direction is obtained by processing the standard pose information and the reference pose information corresponding to the candidate motion direction; the standard pose information is the relative pose information between the first sensor and the second sensor when the robot is in the standard state; Based on the target pose information group and the current pose change parameters, the environmental information collected by the first sensor is converted to the coordinate system of the second sensor.
2. The method according to claim 1, characterized in that, Processing the standard pose information and the reference pose information corresponding to the candidate motion direction includes: Based on the motion range of the first sensor, determine the candidate calibration position of the first sensor in each candidate motion direction; For each candidate motion direction, after adjusting the first sensor to the candidate calibration position of the candidate motion direction, the first sensor and the second sensor are calibrated to obtain the relative pose information between the first sensor and the second sensor, which is used as the reference pose information corresponding to the candidate motion direction. Interpolation processing is performed on the standard pose information and the reference pose information corresponding to the candidate motion direction to obtain the candidate pose information group corresponding to the candidate motion direction.
3. The method according to claim 2, characterized in that, The step of interpolating the standard pose information and the reference pose information corresponding to the candidate motion direction to obtain the candidate pose information group corresponding to the candidate motion direction includes: The standard rotation matrix in the standard pose information and the reference rotation matrix in the reference pose information are interpolated to obtain the interpolated rotation matrix. The standard translation vector in the standard pose information and the reference translation vector in the reference pose information are interpolated to obtain the interpolated translation vector. Based on the interpolation rotation matrix and the interpolation translation vector, the candidate pose information group corresponding to the candidate motion direction is determined.
4. The method according to claim 3, characterized in that, The step of interpolating the standard rotation matrix in the standard pose information and the reference rotation matrix in the reference pose information to obtain the interpolated rotation matrix includes: The standard rotation matrix in the standard pose information is converted into the first quaternion; The reference rotation matrix in the reference pose information is converted into a second quaternion; The first quaternion and the second quaternion are interpolated to obtain an interpolated quaternion, and the interpolated quaternion is converted into an interpolation rotation matrix.
5. The method according to claim 2, characterized in that, The step of determining the candidate calibration position of the first sensor in each candidate motion direction based on the motion range of the first sensor includes: For each candidate motion direction, the motion boundary position of the first sensor in the candidate motion direction is determined based on the motion range of the first sensor. The position of the motion boundary is used as the candidate calibration position for the candidate motion direction.
6. The method according to any one of claims 1-5, characterized in that, The step of converting the environmental information collected by the first sensor to the coordinate system of the second sensor based on the target pose information group and the current pose change parameters includes: Obtain the current pose information corresponding to the current pose change parameter from the target pose information group; Using the current pose information, the environmental information collected by the first sensor is converted to the coordinate system of the second sensor.
7. The method according to claim 2, characterized in that, The calibration of the first sensor and the second sensor to obtain the relative pose information between the first sensor and the second sensor, which serves as the reference pose information corresponding to the candidate motion direction, includes: The first calibration position of the standard calibration plate is obtained through the first sensor, and the second calibration position of the standard calibration plate is obtained through the second sensor. Based on the first calibration position, the second calibration position, and the sensor parameters of the first sensor, the first sensor and the second sensor are calibrated to obtain the relative pose information between the first sensor and the second sensor, which serves as the reference pose information corresponding to the candidate motion direction.
8. An environmental information conversion device for a robot, characterized in that, The device includes: The state acquisition module is used to determine the current motion direction and current pose change parameters of the first sensor based on the current pose of the first sensor and the standard pose of the first sensor when the relative posture of the first sensor and the robot in the standard state is detected to have changed. The information acquisition module is used to select the target pose information group corresponding to the current motion direction from the candidate pose information groups corresponding to each candidate motion direction; wherein, each candidate pose information group corresponding to a candidate motion direction is obtained by processing the standard pose information and the reference pose information corresponding to the candidate motion direction; the standard pose information is the relative pose information between the first sensor and the second sensor when the robot is in the standard state; The information conversion module is used to convert the environmental information collected by the first sensor to the coordinate system of the second sensor based on the target pose information group and the current pose change parameters.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.
10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.