Sensor calibration method, robot, storage medium, device and program product

Abstract

The invention provides a sensor calibration method, a robot, a storage medium, equipment and a program product, which are used for calibrating a plurality of sensors including an image sensor and / or a laser sensor on the robot. The method comprises the following steps: receiving truth value data of a corresponding image sensor and / or a laser sensor of a calibration space under a truth value coordinate system; sensing the calibration space by using a first sensor so as to dynamically calibrate the first sensor, and determining a pose conversion relationship between the body coordinate system and the true value coordinate system through a first sensor coordinate system corresponding to the first sensor; and sensing the calibration space by using the second sensor so as to determine a pose conversion relation between a second sensor coordinate system corresponding to the second sensor and the body coordinate system through the true value coordinate system, thereby realizing static calibration of the second sensor. According to the method, the sensor on the robot can be calibrated without a four-wheel positioning system or a precision clamp, the calibration process is simplified, and the calibration cost is reduced.

Description

Technical Field

[0001] The present application relates to the technical field of robot positioning algorithms, and in particular to sensor calibration methods, robots, storage media, devices and program products. Background Art

[0002] In the field of robotics, accurate calibration of various sensors on robots is very important to ensure the accuracy and consistency of products. Most of the related production line calibration methods rely on the construction of a high-precision calibration room, in which calibration plates and fixtures (e.g., card slots) are installed. During the calibration process, the robot needs to be accurately moved into the card slot to ensure that the robot's body coordinate system is aligned with the known reference coordinate system in the calibration room, so as to achieve high-precision positioning and calibration. This precise positioning often requires the use of a four-wheel alignment system, which can ensure that the robot is accurately positioned at the calibration position for subsequent sensor calibration. However, the four-wheel alignment system is expensive and requires regular maintenance, which increases the cost and complexity of the production line. On production lines with high requirements for the accuracy of mobile equipment, the reliance on the four-wheel alignment system has gradually become a bottleneck, limiting the improvement of production efficiency.

[0003] Based on this, the present application provides a sensor calibration method, a robot, a storage medium, a device and a program product to improve the relevant technology. Summary of the invention

[0004] The purpose of this application is to provide a sensor calibration method, a robot, a storage medium, a device and a program product, which can efficiently and accurately calibrate multiple sensors on a robot without a four-wheel alignment system or a precision fixture.

[0005] The purpose of this application is achieved by the following technical solutions:

[0006] In a first aspect, the present application provides a sensor calibration method for calibrating multiple sensors on a robot, including image sensors and / or laser sensors, wherein the multiple sensors include at least one first sensor and at least one second sensor, wherein the at least one first sensor is a dynamic sensor, and the at least one second sensor includes a dynamic sensor and / or a static sensor, wherein the relative position of the dynamic sensor to the robot body is not fixed, and the relative position of the static sensor to the robot body is fixed; the method comprises: receiving true value data corresponding to the image sensor and / or the laser sensor in a calibration space in a true value coordinate system; the true value data corresponding to the image sensor includes a calibration plate in the calibration space; The three-dimensional coordinate data of the feature points on the object in the true value coordinate system, and the true value data corresponding to the laser sensor include the laser point cloud data of the calibration space in the true value coordinate system; for the at least one first sensor, the first sensor is used to perceive the calibration space to dynamically calibrate the first sensor, and the posture conversion relationship between the body coordinate system and the true value coordinate system is determined through the first sensor coordinate system corresponding to the first sensor; for the at least one second sensor, the second sensor is used to perceive the calibration space to determine the posture conversion relationship between the second sensor coordinate system corresponding to the second sensor and the body coordinate system through the true value coordinate system, so as to realize the static calibration of the second sensor.

[0007] In some embodiments, the three-dimensional coordinate data of the feature points on the calibration plate in the calibration space in the true value coordinate system is obtained by extracting the laser point cloud data of the calibration space in the true value coordinate system.

[0008] In some embodiments, the first sensor is installed on the end joint of the robot, and the dynamic calibration of the first sensor includes: determining the posture conversion relationship between the first sensor coordinate system and the end joint coordinate system through a target hand-eye calibration algorithm; determining the posture conversion relationship between the end joint coordinate system and the body coordinate system through robot kinematics; according to the posture conversion relationship between the first sensor coordinate system and the end joint coordinate system and the posture conversion relationship between the end joint coordinate system and the body coordinate system, calculating the posture conversion relationship between the first sensor coordinate system and the body coordinate system to achieve calibration of the first sensor.

[0009] In some embodiments, determining the posture conversion relationship between the body coordinate system and the true value coordinate system through the first sensor coordinate system corresponding to the first sensor includes: acquiring the first calibration plate data of the calibration plate in the first sensor coordinate system; matching the first calibration plate data with the true value data corresponding to the first sensor through the target matching algorithm corresponding to the first sensor to determine the posture conversion relationship between the first sensor coordinate system and the true value coordinate system; calculating the posture conversion relationship between the body coordinate system and the true value coordinate system according to the posture conversion relationship between the first sensor coordinate system and the body coordinate system and the posture conversion relationship between the first sensor coordinate system and the true value coordinate system.

[0010] In some embodiments, determining the pose conversion relationship between the second sensor coordinate system corresponding to the second sensor and the body coordinate system through the true value coordinate system includes: acquiring the second calibration plate data of the calibration plate in the second sensor coordinate system; matching the second calibration plate data with the true value data corresponding to the second sensor through the target matching algorithm corresponding to the second sensor to determine the pose conversion relationship between the second sensor coordinate system and the true value coordinate system; calculating the pose conversion relationship between the second sensor coordinate system and the body coordinate system according to the pose conversion relationship between the second sensor coordinate system and the true value coordinate system and the pose conversion relationship between the body coordinate system and the true value coordinate system.

[0011] In some embodiments, the target matching algorithm corresponding to the image sensor includes an N-point perspective PnP algorithm, and / or the target matching algorithm corresponding to the laser sensor includes an iterative closest point ICP algorithm.

[0012] In some embodiments, the posture transformation relationship is represented by a posture transformation matrix.

[0013] In some embodiments, the at least one first sensor includes an image sensor mounted on the distal joint of the left hand, and the at least one second sensor includes one or more of a laser sensor, an image sensor mounted on the distal joint of the right hand, an image sensor mounted on the head, and an image sensor mounted on the chassis; or, the at least one first sensor includes an image sensor mounted on the distal joint of the right hand, and the at least one second sensor includes one or more of a laser sensor, an image sensor mounted on the distal joint of the left hand, an image sensor mounted on the head, and an image sensor mounted on the chassis; or, the at least one first sensor includes an image sensor mounted on the distal joint of the left hand and an image sensor mounted on the distal joint of the right hand, and the at least one second sensor includes one or more of a laser sensor, an image sensor mounted on the head, and an image sensor mounted on the chassis.

[0014] In a second aspect, the present application provides a robot comprising a robot body, a plurality of sensors including image sensors and / or laser sensors, and a control module, wherein the control module is used to execute any of the above methods to calibrate the plurality of sensors.

[0015] In a third aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program implements any of the above methods when executed by a processor.

[0016] In a fourth aspect, the present application provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements any one of the above methods when executing the computer program.

[0017] In a fifth aspect, the present application provides a computer program product, wherein the computer program product comprises a computer program, and the computer program implements any one of the above methods when executed by a processor.

[0018] The present application provides a sensor calibration method, a robot, a storage medium, a device and a program product, which can efficiently and accurately calibrate multiple sensors on a robot without a four-wheel positioning system or a precision fixture, simplify the calibration process, reduce equipment costs, and ensure the accuracy and reliability of sensor calibration, so as to meet the dual requirements of high efficiency and high precision of the manufacturing production line. The method obtains true value data in the true value coordinate system in the calibration space, including three-dimensional coordinate data of the characteristic points of the calibration plate for the image sensor and laser point cloud data for the laser sensor. Among them, for at least one dynamic sensor (first sensor), the calibration space is sensed by it and dynamically calibrated. Through the perception result of the dynamic sensor, the posture conversion relationship between the robot's body coordinate system and the true value coordinate system is determined. Then, for at least one second sensor (dynamic sensor and / or static sensor), the calibration space is sensed by it, and the posture conversion relationship between the coordinate system of these sensors and the body coordinate system of the robot is determined through the posture conversion relationship between the body coordinate system and the true value coordinate system, so as to realize the static calibration of the second sensor. The whole process does not rely on the precise physical positioning of the robot and the calibration space, but calculates the external parameters between each sensor and the robot body through the sensor's perception data and known true value data. This method does not rely on expensive and complex four-wheel alignment systems or precision mechanical fixtures, simplifies the sensor calibration process, and reduces the calibration cost. The posture conversion relationship between the robot's body coordinate system and the true value coordinate system is determined by dynamic sensors, so that static calibration of other sensors can be achieved. The sensor's own perception data and the true value data of the calibration space are used to ensure the accuracy and reliability of the calibration. At the same time, this method is applicable to a variety of sensor combinations and has high versatility. It meets the needs of the robotics and manufacturing industries for efficient and high-precision calibration, and improves the efficiency and flexibility of the production line. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The present application is further described below in conjunction with the accompanying drawings and specific implementation methods.

[0020] Figure 1 It is a flow chart of a sensor calibration method provided in an embodiment of the present application.

[0021] Figure 2 It is a schematic diagram (top view) of the layout of a calibration room provided in an embodiment of the present application.

[0022] Figure 3 It is a sensor calibration flow chart provided in an embodiment of the present application.

[0023] Figure 4 It is a structural block diagram of a robot provided in an embodiment of the present application.

[0024] Figure 5 It is a structural block diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in this application to clearly and completely describe the technical solutions in the implementation mode of this application. Obviously, the described implementation mode is only a part of the implementation mode of this application, not all the implementation modes. Based on the implementation mode in this application, all other implementation modes obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0026] In the description of the embodiments of the present application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0027] The relevant production line calibration methods are widely used in the manufacturing industry, especially in the field of robotics. Its main purpose is to ensure that the position and posture of each sensor on the robot can be accurately calibrated, so as to ensure product accuracy and consistency. For example, during the production line calibration process, a high-precision calibration room can be constructed, in which a high-precision calibration plate and a card slot (calibration position) are installed. By pushing the robot into the card slot, it is aligned with the known reference in the calibration room to achieve high-precision positioning and calibration. The calibration room is designed to reduce measurement errors and provide a high-precision reference for subsequent production line operations. Although the production line calibration solution based on the calibration room is now widely used, there are still the following shortcomings. Specifically, for some vehicle or mobile robot application scenarios, when doing production line calibration, the robot needs to be accurately moved to the calibration position, so there is a dependence on the four-wheel alignment system, which is a necessary tool. However, the four-wheel alignment system is usually expensive and requires regular maintenance. On production lines with high precision requirements for mobile equipment, four-wheel alignment has become a bottleneck.

[0028] In addition, the calibration methods that rely on precision fixtures or four-wheel alignment systems have high requirements for the mechanical accuracy and stability of the equipment. Any wear or displacement of the fixture may cause calibration errors, which in turn affects the calibration accuracy of the robot sensor. This not only increases the maintenance cost of the equipment, but also makes the calibration process more complicated.

[0029] Therefore, although the relevant production line calibration method based on the calibration room has guaranteed accuracy, it has problems such as high equipment cost and dependence on the four-wheel alignment system. There is an urgent need for a more simplified and efficient calibration method, especially to improve calibration efficiency without the need for a four-wheel alignment system.

[0030] The implementation methods of the present application will be described in detail below.

[0031] See also Figure 1 , Figure 1 It is a flow chart of a sensor calibration method provided in an embodiment of the present application.

[0032] The embodiment of the present application provides a sensor calibration method for calibrating multiple sensors including image sensors and / or laser sensors on a robot, wherein the multiple sensors include at least one first sensor and at least one second sensor, wherein the at least one first sensor is a dynamic sensor, and the at least one second sensor includes a dynamic sensor and / or a static sensor, wherein the relative position between the dynamic sensor and the robot body is not fixed, and the relative position between the static sensor and the robot body is fixed. The method includes steps S101 to S103.

[0033] Step S101: receiving true value data corresponding to the image sensor and / or the laser sensor in the calibration space in the true value coordinate system. The true value data corresponding to the image sensor includes the three-dimensional coordinate data of the feature points on the calibration plate in the calibration space in the true value coordinate system, and the true value data corresponding to the laser sensor includes the laser point cloud data of the calibration space in the true value coordinate system.

[0034] Step S102: for the at least one first sensor, use the first sensor to perceive the calibration space to dynamically calibrate the first sensor, and determine the posture conversion relationship between the body coordinate system and the true value coordinate system through the first sensor coordinate system corresponding to the first sensor.

[0035] Step S103: For the at least one second sensor, use the second sensor to perceive the calibration space, so as to determine the posture conversion relationship between the second sensor coordinate system corresponding to the second sensor and the body coordinate system through the true coordinate system, so as to realize static calibration of the second sensor.

[0036] In the above embodiments, the robot may be a humanoid robot (also known as a bipedal robot, a humanoid robot), a quadrupedal robot, a wheeled robot (eg, an AGV vehicle), etc.

[0037] In some embodiments, the calibration space may be a calibration room with physical boundaries. In other embodiments, the calibration space may not have physical boundaries but only electronic boundaries, that is, a part of a physical space is used as the calibration room.

[0038] In some embodiments, the posture conversion relationship can be represented by a posture conversion matrix.

[0039] In the above embodiment, the robot includes a robot body and a plurality of sensors, and the plurality of sensors include an image sensor (e.g., a camera) and / or a laser sensor (e.g., a lidar). Since the calibration of these sensors in the above embodiment is divided into two types, dynamic calibration and static calibration, corresponding calibration operations are performed in the dynamic calibration stage and the static calibration stage, respectively, and one or more sensors are calibrated respectively, so the one or more sensors for dynamic calibration are referred to as first sensors, and the one or more sensors for static calibration are referred to as second sensors.

[0040] From the perspective of the positional relationship between the sensor and the robot body, the sensor can be divided into dynamic sensors and static sensors. Static sensors refer to sensors that are fixedly connected to the robot body. The relative position of the static sensor and the robot body will not change, such as laser sensors (for example, lidar) and image sensors (for example, chassis cameras, head cameras) with fixed relative positions. The robot can also include one or more mechanical arms, mechanical legs, etc. Dynamic sensors, for example, refer to sensors installed on the end joints of the mechanical arms. The relative position of such sensors to the end joints of the mechanical arms (for example, the end joints of the left hand and the end joints of the right hand) is fixed, but the relative position to the robot body can be changed, such as image sensors (for example, hand cameras) and laser sensors whose relative positions are not fixed.

[0041] It should be noted that the first sensor for dynamic calibration must be a dynamic sensor, however, the sensor for static calibration can be a dynamic sensor or a static sensor. The "dynamic" in dynamic calibration means that the sensor needs to change its posture during the calibration process; the "dynamic" in dynamic sensor means that the relative position of the sensor and the robot body is not fixed. The "static" in static calibration means that the sensor does not need to change its posture during the calibration process; the "static" in static sensor means that the relative position of the sensor and the robot body is fixed.

[0042] In some embodiments, the at least one first sensor includes an image sensor installed on the end joint of the left hand, and the at least one second sensor includes one or more of a laser sensor, an image sensor installed on the end joint of the right hand, an image sensor installed on the head, and an image sensor installed on the chassis. That is to say, only the image sensor on the end joint of the left hand can be dynamically calibrated, and the remaining sensors can be statically calibrated. The laser sensor is, for example, a laser radar, the image sensor installed on the end joint of the left hand is, for example, a left-hand camera, the image sensor installed on the end joint of the right hand is, for example, a right-hand camera, the image sensor installed on the head is, for example, a head camera, and the image sensor installed on the chassis is, for example, a chassis camera. The left-hand camera and the right-hand camera can be collectively referred to as a hand camera.

[0043] In other embodiments, the at least one first sensor includes an image sensor mounted on the right hand end joint, and the at least one second sensor includes one or more of a laser sensor, an image sensor mounted on the left hand end joint, an image sensor mounted on the head, and an image sensor mounted on the chassis. In other words, only the image sensor on the right hand end joint can be dynamically calibrated, and the remaining sensors can be statically calibrated.

[0044] In some other embodiments, the at least one first sensor includes an image sensor installed on the left hand end joint and an image sensor installed on the right hand end joint, and the at least one second sensor includes one or more of a laser sensor, an image sensor installed on the head, and an image sensor installed on the chassis. In other words, the image sensors on the left hand end joint and the right hand end joint can be dynamically calibrated independently, and the remaining sensors can be statically calibrated.

[0045] In some embodiments, when the multiple sensors on the robot include only image sensors, the true value data of the calibration space corresponding to the image sensor in the true value coordinate system may be received. In other embodiments, when the multiple sensors on the robot include only laser sensors, the true value data of the calibration space corresponding to the laser sensor in the true value coordinate system may be received. In still other embodiments, when the multiple sensors on the robot include both image sensors and laser sensors, the true value data of the calibration space corresponding to the image sensor and the true value data corresponding to the laser sensor in the true value coordinate system are received.

[0046] In the above embodiment, the true value data corresponding to the image sensor may include, for example, the three-dimensional coordinate data of the feature points on the calibration plate in the calibration space in the true value coordinate system; the true value data corresponding to the laser sensor may include, for example, the laser point cloud data of the calibration space in the true value coordinate system. A plurality of feature points may be arranged on the calibration plate, and the feature points may be in the form of, for example, checkerboard corner points, dots or other geometric features.

[0047] In some embodiments, the three-dimensional coordinate data of the feature points on the calibration plate in the calibration space in the true value coordinate system is obtained by extracting the laser point cloud data of the calibration space in the true value coordinate system.

[0048] See also Figure 2 , Figure 2 Schematic diagram (top view) of the layout of a calibration room provided in an embodiment of the present application. Figure 2 A 6m*6m calibration room is shown in the figure. In practical applications, calibration rooms of other specifications may also be selected, and the above embodiment does not limit this.

[0049] For example, assuming that the calibration space adopts a calibration room, for the production line calibration based on the calibration room, the corresponding high-precision true value data of the calibration room can be constructed. The specific steps are as follows.

[0050] First of all, the layout of the calibration room. The overall layout of the calibration room is as follows: Figure 2 As shown, a calibration plate can be placed in the calibration room, but there is no need to install a specific card slot or four-wheel alignment system. During the calibration process, it is sufficient for the robot to be able to sense the calibration plate.

[0051] Then comes the true value acquisition in the calibration room. As an example, the calibration room can be reconstructed with high precision using a high-precision 3D scanning device to obtain the laser point cloud data of the calibration space in the true value coordinate system, and then the three-dimensional coordinate data of each feature point on the calibration plate in the true value coordinate system can be extracted from the laser point cloud data.

[0052] The above-mentioned embodiments can realize the production line calibration of sensors without the need for a four-wheel alignment system, and reduce the dependence on the four-wheel alignment system in related technologies while ensuring the calibration accuracy. Specifically, the true value of the calibration space is first constructed, and then the robot's body coordinate system and the true value coordinate system are aligned (in this process, dynamic calibration of one or more dynamic sensors is achieved), and then static calibration of one or more sensors is performed (in this process, static calibration of one or more dynamic sensors and / or static sensors is achieved). The above-mentioned embodiments can complete the production line calibration of each sensor of the robot without the need for a precise fixture or a four-wheel alignment system, which simplifies the calibration process and reduces the calibration cost.

[0053] The related production line calibration method requires a precise fixture or a four-wheel alignment system to align the robot's body coordinate system with the true value coordinate system, which is time-consuming, labor-intensive and costly. The above embodiment only requires moving the robot to the front of the calibration plate to complete the alignment of the body coordinate system with the true value coordinate system.

[0054] In some embodiments, the first sensor is installed on the end joint of the robot, and the dynamic calibration of the first sensor may include: determining the posture conversion relationship between the first sensor coordinate system and the end joint coordinate system through a target hand-eye calibration algorithm; determining the posture conversion relationship between the end joint coordinate system and the body coordinate system through robot kinematics; according to the posture conversion relationship between the first sensor coordinate system and the end joint coordinate system and the posture conversion relationship between the end joint coordinate system and the body coordinate system, calculating the posture conversion relationship between the first sensor coordinate system and the body coordinate system to achieve calibration of the first sensor.

[0055] In the above embodiment, the first sensor is used to sense the calibration space to dynamically calibrate the first sensor. The target hand-eye calibration algorithm may include solving based on a rotation matrix and a translation vector, and using a least squares method or other optimization algorithms to improve accuracy.

[0056] For example, for a dynamic sensor, the target hand-eye calibration algorithm can be used to obtain the external parameters of the dynamic sensor (e.g., the first sensor) to the end joint (e.g., the pose conversion matrix between the first sensor coordinate system and the end joint coordinate system). Assuming that the first sensor is an image sensor (e.g., a camera) and is installed on the end joint of the left arm or the right arm, taking the hand camera as an example, the robot is moved to the front of the calibration plate, and the hand camera is used to perceive the calibration plate in the calibration space, and the external parameters T of the hand camera relative to the end joint of the arm can be obtained through the target hand-eye calibration algorithm. hc (i.e., the pose transformation matrix between the first sensor coordinate system and the end joint coordinate system). In addition, the pose transformation relationship between the end joint of the arm and the robot body can be directly obtained through robot kinematics to obtain a high-precision result, which is denoted as T bh , represents the pose transformation matrix from the end joint of the arm to the robot body. From this, we can get the external parameter T from the camera to the robot body bc (i.e., the pose transformation matrix between the first sensor coordinate system and the body coordinate system), T bc =T bh ×T hc .

[0057] In some embodiments, determining the posture conversion relationship between the body coordinate system and the true value coordinate system through the first sensor coordinate system corresponding to the first sensor may include: acquiring first calibration plate data of the calibration plate in the first sensor coordinate system; matching the first calibration plate data with the true value data corresponding to the first sensor through a target matching algorithm corresponding to the first sensor to determine the posture conversion relationship between the first sensor coordinate system and the true value coordinate system; calculating the posture conversion relationship between the body coordinate system and the true value coordinate system according to the posture conversion relationship between the first sensor coordinate system and the body coordinate system and the posture conversion relationship between the first sensor coordinate system and the true value coordinate system.

[0058] In some embodiments, when the first sensor is an image sensor, the target matching algorithm corresponding to the image sensor may include an N-point perspective PnP algorithm. In some embodiments, when the first sensor is a laser sensor, the target matching algorithm corresponding to the laser sensor may include an iterative closest point ICP algorithm. Among them, the N-point perspective PnP (Perspective-n-Point) algorithm is a method for solving 3D to 2D point pair motion, which can be used to estimate the camera's pose. The iterative closest point ICP (Iterative Closest Point) algorithm is a point cloud registration algorithm used to align the coordinate systems of two or more point clouds.

[0059] In the above embodiment, the calibration space is sensed by the first sensor, so as to calculate the posture conversion relationship between the robot's body coordinate system and the true coordinate system through the first sensor coordinate system corresponding to the first sensor, thereby aligning the machine body coordinate system and the true coordinate system. The true value data of the corresponding image sensor includes the 3D coordinates of the feature points on the calibration plate in the true value coordinate system (i.e., three-dimensional coordinate data), and the first sensor can also sense the calibration plate. Taking the hand camera as an example, the first calibration plate data of the calibration plate in the first sensor coordinate system can be obtained through the calibration plate detection algorithm, for example, the 2D coordinates of the feature points on the calibration plate in the hand camera image (i.e., two-dimensional coordinate data). Next, through the 3D-2D N-point perspective PnP algorithm, the posture conversion relationship from the hand camera to the true value coordinate can be obtained, which is recorded as T here. gc In addition, it is known in the above that T bc Therefore, the relative relationship between the robot body coordinate system and the true value coordinate system can be obtained. gb :T gb =T gc ×T bc ―1 After obtaining the posture transformation relationship T from the robot's body coordinate system to the true value coordinate system gbAfter that, the robot's body coordinate system and the true value coordinate system can be aligned.

[0060] In some embodiments, determining the pose conversion relationship between the second sensor coordinate system corresponding to the second sensor and the body coordinate system through the true value coordinate system may include: acquiring second calibration plate data of the calibration plate in the second sensor coordinate system; matching the second calibration plate data with the true value data corresponding to the second sensor through a target matching algorithm corresponding to the second sensor to determine the pose conversion relationship between the second sensor coordinate system and the true value coordinate system; calculating the pose conversion relationship between the second sensor coordinate system and the body coordinate system according to the pose conversion relationship between the second sensor coordinate system and the true value coordinate system and the pose conversion relationship between the body coordinate system and the true value coordinate system.

[0061] In the above embodiment, the calibration space is sensed by the second sensor to determine the pose conversion relationship between the second sensor coordinate system corresponding to the second sensor and the body coordinate system through the true value coordinate system. After aligning the body coordinate system and the true value coordinate system, the remaining sensors can be calibrated with external parameters through a static calibration algorithm. In step S101, the true value data of the calibration space is obtained, including the true value point cloud of the calibration space (i.e., the laser point cloud data in the true value coordinate system) and the true value of each feature point on the calibration plate (i.e., the 3D coordinates in the true value coordinate system). In step S102, the alignment relationship T between the body coordinate system and the true value coordinate system is obtained. gb Then the remaining uncalibrated sensors can complete the external parameter calculation through their own static calibration algorithms. The following takes the lidar and camera as examples to illustrate.

[0062] Assume that at least one second sensor includes a laser sensor (e.g., a laser radar). Taking the static calibration of the laser radar as an example, the laser radar senses the calibration space to obtain the laser radar point cloud data. The laser radar point cloud data is aligned with the true value point cloud through the ICP algorithm, and the pose conversion relationship between the laser radar and the true value point cloud (i.e., the pose conversion relationship between the second sensor coordinate system and the true value coordinate system) can be obtained: T gl Through the alignment relationship T between the body coordinate system and the true value coordinate system gb , and then the external parameters of the laser radar to the robot body (i.e., the posture conversion relationship between the second sensor coordinate system and the body coordinate system) can be finally obtained by compensating the laser radar posture: T bl =T gb ―1 ×T gl .

[0063] Assume that at least one second sensor includes an image sensor (e.g., a camera). Taking the static calibration of the camera as an example, the camera senses the calibration space, and the 2D coordinates of the feature points on the calibration plate in the hand camera image are obtained through the calibration plate detection algorithm. This is the data in the camera coordinate system, that is, the current second sensor coordinate system. Since the 3D coordinates of the calibration plate in the true value coordinate system are known, the external parameters of the camera in the true value coordinate system (i.e., the posture transformation relationship between the second sensor coordinate system and the true value coordinate system) can be obtained through the 3D-2D PnP algorithm: T gc Through the alignment relationship T between the body coordinate system and the true value coordinate system gb , by compensating the camera pose, we can finally find the external parameters from the camera to the robot body: T bc =T gb ―1 ×T gc .

[0064] Through the above steps S101 to S103, the external parameter calibration of each sensor on the robot is completed, wherein dynamic calibration is performed for at least one sensor, and static calibration is performed for at least one sensor. The above embodiment implements a combined calibration scheme of dynamic calibration and static calibration of multiple sensors, which not only saves the cost of the four-wheel positioning system, but also simplifies the calibration process, reduces the calibration cost, minimizes the calculation steps and amount of calculation, improves the calibration efficiency, and saves computing resources.

[0065] See also Figure 3 , Figure 3 This is a sensor calibration flow chart provided in an embodiment of the present application. Figure 3 As shown, assuming that the calibration space adopts a calibration room, in step one, the truth value of the calibration room is constructed, in step two, the robot's body coordinate system and the truth coordinate system are aligned (dynamic calibration of one or more dynamic sensors is performed in step two), and in step three, static calibration of each sensor is implemented.

[0066] See also Figure 4 , Figure 4 It is a structural block diagram of a robot provided in an embodiment of the present application.

[0067] An embodiment of the present application also provides a robot, which includes a robot body, multiple sensors including image sensors and / or laser sensors, and a control module, and the control module is used to execute any of the above methods to calibrate the multiple sensors.

[0068] In some embodiments, the robot may further include a left arm, a right arm, a head and a chassis. The left arm and the right arm may be collectively referred to as a robotic arm.

[0069] In some embodiments, the image sensor may include a camera (or a camera module). As an example, cameras may be provided at the end joint of the left arm, the end joint of the right arm, the head, and the chassis, respectively, that is, a left hand camera, a right hand camera, a head camera, and a chassis camera.

[0070] In some embodiments, the laser sensor may include a 2D laser sensor and / or a 3D laser sensor. Taking a laser radar as an example, the laser radar may be set on the head of the robot, however, the above embodiment is not limited to this, and the laser radar may also be set on other parts of the robot.

[0071] In some embodiments, the robot may also include an odometer and / or an IMU.

[0072] In some embodiments, the robot may further include one or more of an angle encoder, a torque sensor, and a PIR sensor.

[0073] In some embodiments, the robot may be a multi-jointed robot.

[0074] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, any of the above methods is implemented.

[0075] An embodiment of the present application further provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, any of the above methods is implemented.

[0076] The computer program product may be a portable compact disk read-only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the computer program product of the present application is not limited thereto, and the computer program product may be any combination of one or more computer readable media.

[0077] An embodiment of the present application further provides a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor implements any of the above methods when executing the computer program.

[0078] See also Figure 5 , Figure 5 It is a structural block diagram of a computer device provided in an embodiment of the present application.

[0079] The embodiments of the present application do not limit the computer device, which may be, for example, a local computer device, a cloud computer device, a distributed computer device, etc.

[0080] The computer device may include: a memory 110, a processor 120, and a communication interface 130. The memory 110, the processor 120, and the communication interface 130 are connected via an internal connection path.

[0081] The memory 110 is used to store computer programs. In some implementations, the computer programs may include codes for implementing the methods of the embodiments of the present application.

[0082] The processor 120 is used to execute the computer program stored in the memory 110 to control the communication interface 130 to receive input data and information and output data such as operation results. In some implementations, when the solution of the embodiment of the present application is implemented by software or firmware, the computer program for implementing the solution of the embodiment of the present application can be stored in the processor 120 and executed by the processor 120.

[0083] The memory 110 may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (Programmable ROM, PROM), an erasable programmable read-only memory (Erasable PROM, EPROM), an electrically erasable programmable read-only memory (EEPROM) or a flash memory. The volatile memory may be a random access memory (RAM). It should be noted that the memory 110 described herein is intended to include, but is not limited to, any memory of these and other suitable types. As an example, the memory 110 includes a random access memory (RAM), a cache memory and a read-only memory (ROM). Among them, the memory 110 stores a computer program, and the computer program can be executed by the processor 120 so that the processor 120 implements the steps of any of the above methods.

[0084] The processor 120 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor, or the processor 120 may be any conventional processor, etc.

[0085] In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor 120 or an instruction in the form of software. The method disclosed in conjunction with the embodiment of the present application can be directly embodied as a hardware processor for execution, or a combination of hardware and software modules in the processor 120 for execution. The software module can be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 110, and the processor 120 reads the information in the memory 110 and completes the steps of the above method in conjunction with its hardware. To avoid repetition, it is not described in detail here.

[0086] In some implementations, in addition to the hardware units described above, the computer device may also include software modules, where the software modules may be, for example, an operating system, a basic input output system (BIOS), application software, etc.

[0087] The operating system is used to manage the hardware and / or software resources of computer devices and is the core and cornerstone of computer devices. The operating system needs to handle basic tasks such as managing and configuring memory, determining the priority of system resource supply and demand, controlling input and output devices, operating networks, and managing file systems. In order to facilitate user operations, most operating systems will provide an operating interface for users to interact with the system.

[0088] BIOS is used to run hardware initialization during the power-on boot phase and provide runtime services for the operating system and application programs. In some implementations, BIOS can also monitor and display processor temperature and execute functions such as adjusting temperature protection strategies.

[0089] Application software is also called application program, which can be understood as software written for a specific application purpose of the user. It is one of the main categories of computer software. For example, application software can be a program used to achieve power control, temperature management, etc.

[0090] It should be noted that although some embodiments of the present application take mobile robots as examples, the present application can be applied to other bionic robots, such as AGVs, drones, etc., and the present application is not limited to this.

[0091] It should be understood that the specific examples in this specification are only intended to help those skilled in the art to better understand the implementation methods of the present application, rather than to limit the scope of protection of the present application.

[0092] It is understood that in the various implementations of this specification, the size of the sequence number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of this application.

[0093] It can be understood that the various implementation modes described in this specification can be implemented individually or in combination, and this application is not limited to this.

[0094] Unless otherwise specified, all technical and scientific terms used in this specification have the same meaning as those generally understood by those skilled in the art of the technical field of this specification. The terms used in this specification are only for the purpose of describing specific embodiments and are not intended to limit the scope of this specification. The term "and / or" used in this specification includes any and all combinations of one or more related listed items. The singular forms of "a", "above", and "the" used in this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0095] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this specification.

[0096] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of each embodiment described above can refer to the corresponding process in other embodiments and will not be repeated here.

[0097] In the several embodiments provided in this specification, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0098] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the technical solution of the present application.

[0099] In addition, each functional unit in each embodiment of the present specification may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0100] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this specification, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of this specification. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0101] The above are only specific implementations of this specification, but the protection scope of this application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in this specification, which should be included in the protection scope of this specification. Therefore, the protection scope of this application should be based on the protection scope of the claims.

Claims

1. A sensor calibration method, characterized in that: Used to calibrate multiple sensors including image sensors and / or laser sensors on the robot, wherein the multiple sensors include at least one first sensor and at least one second sensor, wherein the at least one first sensor is a dynamic sensor, and the at least one second sensor includes a dynamic sensor and / or a static sensor, wherein the relative position of the dynamic sensor to the robot body is not fixed, and the relative position of the static sensor to the robot body is fixed; The method comprises: Receive true value data corresponding to the image sensor and / or the laser sensor in the calibration space in the true value coordinate system; the true value data corresponding to the image sensor includes three-dimensional coordinate data of feature points on the calibration plate in the calibration space in the true value coordinate system, and the true value data corresponding to the laser sensor includes laser point cloud data of the calibration space in the true value coordinate system; For the at least one first sensor, using the first sensor to sense the calibration space to dynamically calibrate the first sensor, and determining a posture conversion relationship between a body coordinate system and the true value coordinate system through a first sensor coordinate system corresponding to the first sensor; For the at least one second sensor, the calibration space is sensed by the second sensor, so as to determine the posture conversion relationship between the second sensor coordinate system corresponding to the second sensor and the body coordinate system through the true value coordinate system, thereby realizing static calibration of the second sensor.

2. The sensor calibration method according to claim 1, characterized in that: The three-dimensional coordinate data of the feature points on the calibration plate in the calibration space in the true value coordinate system are obtained by extracting the laser point cloud data of the calibration space in the true value coordinate system.

3. The sensor calibration method according to claim 1, characterized in that: The first sensor is mounted on the end joint of the robot, and the dynamic calibration of the first sensor includes: Determine the posture conversion relationship between the first sensor coordinate system and the end joint coordinate system through a target hand-eye calibration algorithm; Determine the posture conversion relationship between the end-joint coordinate system and the body coordinate system through robot kinematics; According to the posture conversion relationship between the first sensor coordinate system and the end joint coordinate system and the posture conversion relationship between the end joint coordinate system and the body coordinate system, the posture conversion relationship between the first sensor coordinate system and the body coordinate system is calculated to achieve calibration of the first sensor.

4. The sensor calibration method according to claim 1, characterized in that: The determining of the posture conversion relationship between the body coordinate system and the true value coordinate system through the first sensor coordinate system corresponding to the first sensor includes: Acquire first calibration plate data of the calibration plate in the first sensor coordinate system; Matching the first calibration plate data with the true value data corresponding to the first sensor through a target matching algorithm corresponding to the first sensor to determine a posture conversion relationship between the first sensor coordinate system and the true value coordinate system; The posture conversion relationship between the body coordinate system and the true value coordinate system is calculated according to the posture conversion relationship between the first sensor coordinate system and the body coordinate system and the posture conversion relationship between the first sensor coordinate system and the true value coordinate system.

5. The sensor calibration method according to claim 4, characterized in that: The determining, by using the true value coordinate system, a posture conversion relationship between a second sensor coordinate system corresponding to the second sensor and the body coordinate system comprises: Acquire second calibration plate data of the calibration plate in the second sensor coordinate system; Matching the second calibration plate data with the true value data corresponding to the second sensor through a target matching algorithm corresponding to the second sensor to determine a posture conversion relationship between the second sensor coordinate system and the true value coordinate system; The posture conversion relationship between the second sensor coordinate system and the true value coordinate system is calculated according to the posture conversion relationship between the second sensor coordinate system and the true value coordinate system and the posture conversion relationship between the body coordinate system and the true value coordinate system.

6. The sensor calibration method according to claim 1, characterized in that: The at least one first sensor includes an image sensor mounted on the distal joint of the left hand, and the at least one second sensor includes one or more of a laser sensor, an image sensor mounted on the distal joint of the right hand, an image sensor mounted on the head, and an image sensor mounted on the chassis; or, The at least one first sensor includes an image sensor mounted on the right hand distal joint, and the at least one second sensor includes one or more of a laser sensor, an image sensor mounted on the left hand distal joint, an image sensor mounted on the head, and an image sensor mounted on the chassis; or, The at least one first sensor includes an image sensor installed on the distal joint of the left hand and an image sensor installed on the distal joint of the right hand, and the at least one second sensor includes one or more of a laser sensor, an image sensor installed on the head, and an image sensor installed on the chassis.

7. A robot, characterized in that: The robot comprises a robot body, a plurality of sensors including an image sensor and / or a laser sensor, and a control module, wherein the control module is used to execute the method according to any one of claims 1 to 6 to calibrate the plurality of sensors.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

9. A computer device, characterized in that: The computer device comprises a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 6 when executing the computer program.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images