A robot sensor calibration method, apparatus, robot, and storage medium.

Abstract

This invention discloses a robot sensor calibration method, apparatus, robot, and storage medium. The method includes: determining the transformation relationship between the calibration board coordinate system and the radar coordinate system based on geometric similarity, according to the scanning data of the robot's radar sensor on the calibration board and the attribute information of the calibration board; determining the second position of the target label in the radar coordinate system based on the first position of the target label in the calibration board coordinate system and the transformation relationship; determining the third position of the target label in the image coordinate system based on the image data collected by the robot's image sensor on the calibration board; and calibrating the image sensor and the radar sensor based on the second and third positions. The technical solution of this invention can determine a more accurate transformation relationship between the calibration board coordinate system and the radar coordinate system, improving the performance of the robot's perception system and providing a new scheme for robot sensor calibration.

Description

Technical Field

[0001] This invention relates to the field of robotics, and more particularly to a method, apparatus, robot, and storage medium for calibrating robot sensors. Background Technology

[0002] With the continuous development of robotics technology, robots are becoming more and more comprehensive in their functions, and the number and types of sensors installed on robots are also constantly increasing.

[0003] How to more accurately determine the relationship between the coordinate systems of the robot's various sensors, and more effectively calibrate the robot's sensors to improve the overall perception system of the robot for positioning or obstacle avoidance, is a problem that urgently needs to be solved. Summary of the Invention

[0004] This invention provides a robot sensor calibration method, apparatus, robot, and storage medium.

[0005] According to one aspect of the present invention, a robot sensor calibration method is provided, comprising:

[0006] Based on geometric similarity, the transformation relationship between the calibration plate coordinate system and the radar coordinate system is determined according to the scanning data of the calibration plate by the robot's radar sensor and the attribute information of the calibration plate.

[0007] Based on the first position of the target label on the calibration board in the calibration board coordinate system and the transformation relationship, the second position of the target label in the radar coordinate system is determined.

[0008] Based on the image data collected by the robot's image sensor from the calibration board, the third position of the target label in the image coordinate system is determined, and the image sensor and the radar sensor are calibrated based on the second position and the third position.

[0009] According to another aspect of the present invention, a robot sensor calibration device is provided, comprising:

[0010] The relationship determination module is used to determine the transformation relationship between the calibration board coordinate system and the radar coordinate system based on geometric similarity, according to the scanning data of the calibration board by the robot's radar sensor and the attribute information of the calibration board.

[0011] The position determination module is used to determine the second position of the target tag in the radar coordinate system based on the first position of the target tag on the calibration board in the calibration board coordinate system and the transformation relationship.

[0012] The calibration module is used to determine the third position of the target label in the image coordinate system based on the image data collected by the robot's image sensor from the calibration board, and to calibrate the image sensor and the radar sensor based on the second position and the third position.

[0013] According to another aspect of the present invention, a robot is provided, the robot comprising:

[0014] At least one processor; and

[0015] A memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the robot sensor calibration method according to any embodiment of the present invention.

[0017] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the robot sensor calibration method according to any embodiment of the present invention.

[0018] The technical solution of this invention, based on geometric similarity, determines the transformation relationship between the calibration board coordinate system and the radar coordinate system according to the scanning data of the calibration board by the robot's radar sensor and the attribute information of the calibration board. Based on the first position of the target label on the calibration board in the calibration board coordinate system and the transformation relationship, the second position of the target label in the radar coordinate system is determined. Based on the image data collected by the robot's image sensor from the calibration board, the third position of the target label in the image coordinate system is determined. Based on the second and third positions, the image sensor and the radar sensor are calibrated. This method, based on geometric similarity, can efficiently determine a more accurate transformation relationship between the calibration board coordinate system and the radar coordinate system. Using the calibration board as a medium, calibration can be completed quickly using the label, improving the performance of the robot's perception system and providing a new scheme for calibrating robot sensors.

[0019] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart of a robot sensor calibration method provided in Embodiment 1 of the present invention;

[0022] Figure 2A This is a flowchart of a robot sensor calibration method provided in Embodiment 2 of the present invention;

[0023] Figure 2B This is a schematic diagram of the geometric relationship of the calibration plate provided in Embodiment 2 of the present invention;

[0024] Figure 2C This is a schematic diagram of the calibration plate provided in Embodiment 2 of the present invention;

[0025] Figure 3 This is a flowchart of a robot sensor calibration method provided in Embodiment 3 of the present invention;

[0026] Figure 4 This is a flowchart of a robot sensor calibration method provided in Embodiment 4 of the present invention;

[0027] Figure 5 This is a structural block diagram of a robot sensor calibration device provided in Embodiment 5 of the present invention;

[0028] Figure 6 This is a schematic diagram of the structure of a robot provided in Embodiment Six of the present invention. Detailed Implementation

[0029] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0030] It should be noted that the terms "first," "second," "target," "candidate," "alternative," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0031] Example 1

[0032] Figure 1 This is a flowchart of a robot sensor calibration method provided in Embodiment 1 of the present invention. This embodiment is applicable to the calibration of robot sensors, especially to the calibration of multiple different types of robot sensors. The method can be executed by a robot sensor calibration device, which can be implemented in software and / or hardware and integrated into an electronic device with robot sensor calibration functionality. This electronic device can be a robot, such as a robot with autonomous positioning or obstacle avoidance functions. Figure 1 As shown, the robot sensor calibration method provided in this embodiment specifically includes:

[0033] S101. Based on geometric similarity, determine the transformation relationship between the calibration plate coordinate system and the radar coordinate system according to the scanning data of the calibration plate by the robot's radar sensor and the attribute information of the calibration plate.

[0034] In this context, geometric similarity refers to the similarity relationship corresponding to the geometry of the calibration plate. The main geometry of the calibration plate satisfies the following condition: at different heights, the lengths of the line segments connecting the points on the left and right sides along a direction parallel to the base do not coincide. For example, the left and right sides of the calibration plate are not parallel to each other. Exemplarily, the geometry of the calibration plate can be a triangle, an isosceles trapezoid, a right trapezoid, etc. If the geometry of the calibration plate is triangular, the geometric similarity relationship can be a similarity relationship based on the principle of triangle similarity. Compared to square calibration plates, which require the calibration plate to be directly facing the sensor or whose position relative to the sensor is known in advance, thus limiting operation, calibration based on the geometric similarity relationship of the calibration plate in this application offers greater operational flexibility and convenience.

[0035] The radar sensor can be, for example, a lidar sensor, such as a two-dimensional lidar sensor. Scanning data refers to the data collected by the radar sensor scanning the calibration board. Calibration board attribute information may include the length of its base edge and its height, as well as its geometry. A transformation relationship characterizes the correspondence between the calibration board coordinate system and the radar coordinate system; through this transformation, the calibration board coordinate system and the radar coordinate system can be unified into the same coordinate system. Both the calibration board coordinate system and the radar coordinate system can be three-dimensional world coordinate systems. The calibration board can be pre-placed in the robot's working environment.

[0036] Optionally, the transformation relationship between the calibration plate coordinate system and the radar coordinate system may include a rotation matrix and a translation matrix for transforming the calibration plate coordinate system to the radar coordinate system, or it may include a rotation matrix and a translation matrix for transforming the radar coordinate system to the calibration plate coordinate system.

[0037] Optionally, a radar sensor can be used to emit radar rays towards the calibration board and record the reflection of the radar rays. Where there is a calibration board, the radar rays emitted by the radar will be reflected back to the radar. The recorded reflection data is used as the scanning data of the calibration board, that is, to determine the scanning data of the robot's radar sensor on the calibration board.

[0038] Optionally, after determining the scanning data, the scanning data can be analyzed based on preset rules to determine the calibration plate information detected by the radar sensor. Further, based on geometric similarity, a calibration plate coordinate system can be constructed according to the calibration plate information and attribute information. Finally, the transformation relationship between the calibration plate coordinate system and the radar coordinate system can be determined according to the projection relationship between the constructed calibration plate coordinate system and the radar coordinate system. Alternatively, the scanning data of the robot's radar sensor on the calibration plate and the calibration plate attribute information can be directly input into a pre-trained model, and the rotation matrix and translation matrix of the calibration plate coordinate system and the radar coordinate system can be output to determine the transformation relationship.

[0039] S102. Based on the first position of the target tag on the calibration board in the calibration board coordinate system and the transformation relationship, determine the second position of the target tag in the radar coordinate system.

[0040] The target tag refers to the tag pre-configured on the calibration plate. The first position is the three-dimensional position of the target tag on the calibration plate in the calibration plate coordinate system, which can be determined by measuring the target tag's placement on the calibration plate. The second position is the three-dimensional position of the target tag on the calibration plate in the radar coordinate system.

[0041] Optionally, the rotation and translation matrices between the calibration board coordinate system and the radar coordinate system can be determined based on the transformation relationship between the calibration board coordinate system and the radar coordinate system. Furthermore, based on preset calculation rules, the position information of the target label in the radar coordinate system can be determined according to the first position of the target label in the calibration board coordinate system and the rotation and translation matrices between the calibration board coordinate system and the radar coordinate system, that is, the second position of the target label in the radar coordinate system can be determined. Alternatively, the first position of the target label in the calibration board coordinate system and the transformation relationship can be input into a pre-trained model to directly output the second position of the target label in the radar coordinate system.

[0042] For example, the second position of the target tag in the radar coordinate system can be determined based on the following formula:

[0043] (XG,YG,ZG)=Rx1*(XL,YL,RL)+T1

[0044] Where (XG, YG, ZG) represents the second position of the target tag in the radar coordinate system, (XL, YL, RL) represents the first position of the target tag in the calibration board coordinate system, Rx1 represents the rotation matrix between the calibration board coordinate system and the radar coordinate system, and T1 represents the translation matrix between the calibration board coordinate system and the radar coordinate system.

[0045] S103. Based on the image data collected by the robot's image sensor from the calibration board, determine the third position of the target label in the image coordinate system, and calibrate the image sensor and radar sensor based on the second and third positions.

[0046] Here, the image sensor refers to a sensor capable of acquiring infrared images, such as a stereo vision camera. The third position can be the two-dimensional position of the target label on the calibration board in the image coordinate system.

[0047] Optionally, an image sensor can be used to acquire images of a pre-placed calibration board, determine the target image containing the calibration board, further perform recognition processing on the target image, determine the position information of the target label in the target image, and use it as the third position of the target label in the image coordinate system.

[0048] Optionally, after determining the third position of the target label in the image coordinate system, the transformation relationship between the image coordinate system and the radar coordinate system can be determined based on preset rules and the correspondence between the third and second positions of different target labels in the image coordinate system and the radar coordinate system, thereby achieving the calibration of the image sensor and the radar sensor.

[0049] Optionally, the calibration board can be placed in the overlapping area of ​​the radar sensor and the image sensor's acquisition range, which facilitates simultaneous data acquisition from the radar sensor and the image sensor, improving calibration efficiency and accuracy.

[0050] It should be noted that the robot sensor calibration scheme based on the calibration board provided by the present invention can place the calibration board in a preferred position in advance, which is convenient to use and can effectively improve the calibration efficiency of robot sensors.

[0051] It should be noted that the robot sensor calibration scheme provided by this invention is applicable to many application scenarios and user groups. Specifically, it can be used not only for the calibration of robot sensors before the robot leaves the factory, but also for post-manufacturing correction and recalibration. For example, when a robot operates in a restaurant, hotel, or other similar environment for a period of time and detects abnormalities in its positioning, obstacle avoidance, or other functions, the calibration scheme provided by this invention can be used to recalibrate the sensors, effectively ensuring the robot's performance. This application completes the calibration based on a simple calibration board and tags, making the calibration operation easy for robot users.

[0052] The technical solution of this invention, based on geometric similarity, determines the transformation relationship between the calibration board coordinate system and the radar coordinate system based on the scanning data of the calibration board by the robot's radar sensor and the attribute information of the calibration board. Based on the first position of the target label on the calibration board in the calibration board coordinate system and the transformation relationship, the second position of the target label in the radar coordinate system is determined. Based on the image data collected by the robot's image sensor from the calibration board, the third position of the target label in the image coordinate system is determined. Based on the second and third positions, the image sensor and the radar sensor are calibrated. This method, based on geometric similarity, efficiently determines a more accurate transformation relationship between the calibration board coordinate system and the radar coordinate system, thereby obtaining the precise position of the target label in the radar coordinate system. Further combining this with the position of the target label in the image coordinate system, the calibration of the robot's image sensor and radar sensor is achieved. Through multi-sensor data fusion, the sensors can complement each other's strengths. By using the calibration board as a medium, calibration can be quickly completed using the label, improving the overall effectiveness of the robot's perception system and thus enhancing the accuracy of robot detection.

[0053] Optionally, if the robot has at least two radar sensors and / or image sensors, then the associated sensor groups are determined according to the robot's control rules, and the main sensor and its corresponding coordinate system are determined within each associated sensor group; the transformation relationship between the main sensor coordinate system and other sensor coordinate systems within each group is determined, and the robot sensors are calibrated.

[0054] Among them, control rules refer to preset control rules associated with the robot performing specific functions (such as obstacle avoidance or localization). Associated sensor group refers to a group of sensors involved in the robot's control rules that need to interact and reference each other.

[0055] Optionally, based on the performance of each sensor in the robot and according to pre-defined rules, all sensors of the robot can be pre-sorted. For each group of associated sensors, the sensor ranked first in the associated sensor group is taken as the master sensor according to the sorting of each sensor in the group. The transformation relationship between the corresponding master sensor coordinate system and the coordinate systems of other sensors is determined, and the robot sensors are calibrated.

[0056] Optionally, if the robot has only one radar sensor and one image sensor, then the radar sensor and the image sensor can be directly identified as an associated sensor group, and the radar sensor or the image sensor can be used as the master sensor. The transformation relationship between the corresponding master sensor coordinate system and the coordinate systems of other sensors can be determined to calibrate the robot sensors.

[0057] For example, if the robot's obstacle avoidance function requires the integrated analysis of data from sensor 1 and sensor 2, then sensor 1 and sensor 2 are a set of sensors that need to interact and reference each other, and thus sensor 1 and sensor 2 can be identified as a set of associated sensors. As another example, if the robot's localization requires the integrated analysis of data from sensor 1 and sensor 3, then sensor 1 and sensor 3 can be identified as another set of associated sensors.

[0058] Optionally, based on the position information of the target label in the main sensor coordinate system and other sensor coordinate systems within each group, the transformation relationship between the main sensor coordinate system and other sensor coordinate systems within each group can be determined to calibrate the robot's different sensors. The main sensor can be the sensor that plays the primary role in collecting data when performing the corresponding function, while other sensors can be sensors that play an auxiliary role in collecting data.

[0059] Optionally, if the associated sensor group includes radar sensors and image sensors, the robot sensor calibration method described in S101-S103 of this invention can be executed for each radar sensor and image sensor to calibrate the radar sensors and image sensors. If the number of radar sensors in the associated sensor group is at least two, the transformation relationship between the radar coordinate system of each radar sensor and other radar coordinate systems can also be determined to calibrate the robot radar sensors. Similarly, if the number of image sensors in the associated sensor group is at least two, the transformation relationship between the image coordinate system of each image sensor and other image coordinate systems can also be determined to calibrate the robot image sensors.

[0060] It should be noted that for each group of associated sensors, the conversion relationship between each other sensor in the group (excluding the main sensor) and the main sensor can be determined, avoiding the need to determine the conversion relationship for every two sensors, thus improving the efficiency of robot calibration.

[0061] It should be noted that, considering that when a robot avoids obstacles or locates itself, it may need to control multiple radar sensors for data acquisition and analysis, or use a combination of radar sensors and image sensors for data acquisition and analysis, the technical solution of the present invention calibrates the sensors whose data have mutual reference functions as a group of related sensors. This helps to quickly convert the data of other sensors in a group to the main sensor data when the robot is used after calibration. This makes the robot's judgment and control more comprehensive and accurate when performing tasks, thus ensuring the robot's working efficiency.

[0062] Example 2

[0063] Figure 2A This is a flowchart of a robot sensor calibration method provided in Embodiment 2 of the present invention. Figure 2B This is a schematic diagram of the geometric relationship of the calibration plate provided in Embodiment 2 of the present invention. Figure 2C This is a schematic diagram of the calibration plate provided in Embodiment 2 of the present invention. Based on the above embodiments, this embodiment further explains in detail the process of "determining the transformation relationship between the calibration plate coordinate system and the radar coordinate system based on geometric similarity, according to the scanning data of the robot's radar sensor on the calibration plate and the attribute information of the calibration plate," as follows: Figure 2A As shown, the method specifically includes:

[0064] S201. Obtain the scanning data when the bottom edge of the parallel calibration plate of the robot's radar sensor scans the calibration plate.

[0065] Optionally, the robot's radar sensor can be controlled to send scanning rays parallel to the bottom edge of the calibration plate, and the ray data reflected by the calibration plate within a preset time period can be recorded, and the recorded reflection data can be used as scanning data.

[0066] S202. Determine the detection line segment parallel to the bottom edge of the calibration plate based on the scanning data.

[0067] The detection line segment refers to the line segment on the calibration plate scanned by the radar sensor, that is, the intersection line segment of the radar sensor's scanning ray and the calibration plate. The detection line segment is parallel to the bottom edge of the calibration plate.

[0068] Optionally, the scan data can be analyzed based on preset rules, and the length of the detection line segment parallel to the bottom edge of the calibration plate can be determined based on the relationship between the scan data and the transmitted ray. That is, the detection line segment parallel to the bottom edge of the calibration plate can be determined based on the scan data.

[0069] S203. Determine the transformation relationship between the calibration plate coordinate system and the radar coordinate system based on the bottom edge length of the calibration plate, the detection line segment, and the height of the calibration plate.

[0070] Optionally, based on geometric similarity, the length of the base of the calibration plate, the length of the detection line segment, and the height of the calibration plate can be input into a pre-trained model to output the rotation and translation matrices between the calibration plate coordinate system and the radar coordinate system, thus determining the transformation relationship between the calibration plate coordinate system and the radar coordinate system. Alternatively, based on the geometric similarity satisfied by the geometry of the calibration plate, the geometric ratio between the calibration plate coordinate system and the radar coordinate system can be calculated according to the length of the base of the calibration plate, the detection line segment, and the height of the calibration plate, thereby determining the transformation relationship between the calibration plate coordinate system and the radar coordinate system.

[0071] Optionally, the transformation relationship between the calibration plate coordinate system and the radar coordinate system is determined based on the length of the bottom edge of the calibration plate, the detection line segment, and the height of the calibration plate. This includes: determining the perpendicular line corresponding to the detection line segment based on geometric similarity, according to the length of the detection line segment, the length of the bottom edge of the calibration plate, and the height of the calibration plate; constructing a candidate calibration plate coordinate system based on the bottom edge of the calibration plate; translating the candidate calibration plate coordinate system based on the deviation between the length of the perpendicular line and the height of the calibration plate to determine the target calibration plate coordinate system; and determining the transformation relationship between the target calibration plate coordinate system and the radar coordinate system based on the projection of the detection line segment onto the radar coordinate system.

[0072] Optional, refer to Figure 2B As shown, based on geometric similarity, the proportional relationship between the length d1 of the detection line segment, the length d2 of the bottom edge of the calibration plate, the perpendicular line h1 corresponding to the detection line segment, and the height h2 of the calibration plate can be determined. For example, it can satisfy the formula "d1 / d2=h1 / h2". Further, based on the satisfied proportional relationship, combined with the length of the detection line segment, the length of the bottom edge of the calibration plate, and the height of the calibration plate, the length of the perpendicular line corresponding to the detection line segment can be calculated.

[0073] For example, see Figure 2B If the calibration plate has a triangular geometry, the geometric positional relationship between the corresponding detection line segment d1, the bottom edge d2 of the calibration plate, the height h2 of the calibration plate, and the perpendicular line h1 corresponding to the detection line segment is shown in the figure. Here, d1 can be obtained from the scanning data of the radar sensor, and d2 and h2 can be measured beforehand. Therefore, h1 can be calculated using the formula "d1 / d2 = h1 / h2".

[0074] Optionally, a candidate calibration plate coordinate system can be established with the midpoint of the bottom edge of the calibration plate as the origin of the coordinate system, the line containing the bottom edge of the calibration plate as the Y-axis, the line containing the height of the calibration plate perpendicular to the bottom edge of the calibration plate as the Z-axis, and the line perpendicular to the plane containing the calibration plate and pointing outward as the X-axis.

[0075] Optionally, the difference between the length h1 of the vertical line and the height h2 of the calibration plate can be determined as h3 = h2 - h1. The distance corresponding to the difference is taken as the target translation distance of the candidate calibration plate coordinate system. Based on this target translation distance, the candidate calibration plate coordinate system is translated. Finally, the translated candidate calibration plate coordinate system is determined as the target calibration plate coordinate system.

[0076] It should be noted that the coordinate system of the target calibration plate after translation in the above manner can be based on the center of the detection line segment as the origin, the vertical direction of the detection line segment (i.e., the direction of the vertical line of the calibration plate) as the X-axis, the right-hand coordinate system as the Y-axis (i.e., the line where the detection line segment is located is the Y-axis), and the line where the vertical line corresponding to the detection line segment is located is the Z-axis.

[0077] Optionally, after determining the target calibration plate coordinate system, the counterclockwise deflection angle t1 of the X-axis between the target calibration plate coordinate system and the radar coordinate system can be determined, as well as the projection of the detection line segment onto the radar coordinate system (i.e., the projected line segment). The coordinates (x1, y1, z1) of the midpoint of the projected line segment in the radar coordinate system can then be further determined. Based on the deflection angle, the rotation matrix between the target calibration plate coordinate system and the radar coordinate system can be determined. Based on the coordinates of the midpoint of the projected line segment in the radar coordinate system, the translation matrix between the target calibration plate coordinate system and the radar coordinate system can be determined. Finally, based on the rotation matrix and translation matrix between the target calibration plate coordinate system and the radar coordinate system, the transformation relationship between the target calibration plate coordinate system and the radar coordinate system can be determined.

[0078] Optionally, the radar coordinate system has the center of the radar sensor as the origin, the vertical direction perpendicular to the ground as the Z-axis (i.e., the perpendicular direction to the detected line segment), and the robot's forward direction (i.e., directly in front of the radar) as the X-axis. The Y-axis is determined by a right-handed coordinate system. Based on the established radar coordinate system, the line segment length information d1 seen by the radar's 2D laser on the calibration board can be obtained.

[0079] For example, let t1 be the counterclockwise deflection angle between the X-axis of the target calibration plate coordinate system and the X-axis of the radar coordinate system. That is, when the robot radar sensor and the calibration plate are not placed parallel, the target calibration plate coordinate system, after a counterclockwise deflection t1 around the Z-axis, yields yawZ. The two Z-axis are parallel. When the robot radar sensor and the calibration plate are placed parallel, t1 = 0, yawZ = 0. The coordinates of the midpoint of the projected line segment in the radar coordinate system are (x1, y1, z1). The rotation matrix between the target calibration plate coordinate system and the radar coordinate system can then be determined using the following formula:

[0080] Rz=[0,cos(t1),-sin(t1);0,sin(t1),cos(t1);1,0,0]

[0081] Where Rz represents the rotation matrix between the target calibration plate coordinate system and the radar coordinate system; t1 represents the counterclockwise deflection angle between the X-axis of the target calibration plate coordinate system and the radar coordinate system.

[0082] The translation matrix between the target calibration board coordinate system and the radar coordinate system can be determined using the following formula:

[0083] T = (x1, y1, z1)

[0084] Where T represents the translation matrix between the target calibration board coordinate system and the radar coordinate system. (x1, y1, z1) are the coordinates of the midpoint of the projected line segment in the radar coordinate system.

[0085] It should be noted that by first constructing a candidate coordinate system based on the bottom edge of the calibration board, and then further translating to determine the target calibration board coordinate system, an accurate calibration board coordinate system can be easily constructed. This facilitates the determination of a more precise transformation relationship between the calibration board coordinate system and the radar coordinate system, thereby improving the robot's detection performance.

[0086] Optional, refer to Figure 2C As shown, the calibration board's main geometry can be triangular, with a target label mounted on it. Both the calibration board and the robot can be placed on the ground. The calibration board can be stably mounted on the ground using a base, and it can be placed perpendicular to the ground, within the scanning range of the robot's radar sensor, allowing the radar sensor to scan the calibration board parallel to its bottom edge. The selectable height of the base can be determined based on the radar sensor's scanning height; that is, the radar sensor's scanning height is higher than the base height.

[0087] S204. Based on the first position of the target tag on the calibration board in the calibration board coordinate system and the transformation relationship, determine the second position of the target tag in the radar coordinate system.

[0088] Optionally, the position information of the target tag in the candidate calibration board coordinate system can be determined first. Then, based on the translation relationship between the candidate and target calibration board coordinate systems, this position information can be corrected to obtain the position information of the target tag in the target calibration board coordinate system, i.e., determining the first position of the target tag on the calibration board in the calibration board coordinate system. Then, based on the transformation relationship, the second position of the target tag in the radar coordinate system can be determined.

[0089] S205. Based on the image data collected by the robot's image sensor from the calibration board, determine the third position of the target label in the image coordinate system, and calibrate the image sensor and radar sensor based on the second and third positions.

[0090] The technical solution of this invention involves acquiring scanning data from the bottom edge of a calibration plate when the robot's radar sensor scans the calibration plate. Based on the scanning data, a detection line segment parallel to the bottom edge of the calibration plate is determined. The transformation relationship between the calibration plate coordinate system and the radar coordinate system is determined based on the length of the bottom edge of the calibration plate, the detection line segment, and the height of the calibration plate. Further, a second and third position are determined to calibrate the image sensor and the radar sensor. By introducing the detection line segment determined from the radar sensor's scanning data, a feasible implementation method for determining the transformation relationship between the calibration plate coordinate system and the radar coordinate system is provided. This effectively links the two coordinate systems, facilitating the accurate determination of the target label's position in the radar coordinate system, thereby achieving accurate and effective robot sensor calibration.

[0091] Example 3

[0092] Figure 3 This is a flowchart of a robot sensor calibration method provided in Embodiment 3 of the present invention. Based on the above embodiments, this embodiment further explains in detail the process of "determining the third position of the target label in the image coordinate system based on the image data collected by the robot's image sensor from the calibration board," as follows: Figure 3 As shown, the method specifically includes:

[0093] S301. Based on geometric similarity, determine the transformation relationship between the calibration plate coordinate system and the radar coordinate system according to the scanning data of the calibration plate by the robot's radar sensor and the attribute information of the calibration plate.

[0094] S302. Based on the first position of the target tag on the calibration board in the calibration board coordinate system and the transformation relationship, determine the second position of the target tag in the radar coordinate system.

[0095] S303. Control the robot to the same position and acquire the infrared image of the calibration board obtained by the robot's image sensor.

[0096] In this context, "same position" refers to the same robot position as when scanning the calibration board using a radar sensor. By controlling the robot to be in the same position while using both radar and image sensors to collect data from the calibration board, calibration accuracy can be guaranteed. Infrared radiation is an image formed by an image sensor (such as a stereo vision camera with infrared imaging capabilities) capturing the radiation of the calibration board in the infrared band.

[0097] Optionally, the robot can be controlled to be in the same position as during radar scanning, and an image sensor can be used to acquire images of the calibration board, obtaining an infrared image containing the calibration board.

[0098] Optionally, the calibration board can be fixed in the robot's working environment for image and radar sensor data acquisition, used for initial robot localization / sensor recalibration. For robots using tag-based localization, the calibration board can be placed at the robot's origin, such as a charging station. A unique tag ID is determined by the number and positional relationship of the target tags on the calibration board, used for initial localization during each robot run. Simultaneously, the calibration board can also be used for sensor recalibration, for example, after running for a certain period or distance, the sensor can be calibrated again using the calibration board. This fully utilizes the calibration board's capabilities. Especially when the calibration board has a reflective frame, its shape and the reflective properties of the tags can be used to improve the accuracy of the robot's initial localization. It also addresses the calibration problem when sensors may shift after prolonged robot use, requiring recalibration.

[0099] S304. Perform segmentation processing on the infrared image to determine the two-dimensional position of the reflective label on the calibration plate in the infrared image.

[0100] Reflective tags are tags pre-placed on a calibration plate and are made of reflective material. The geometry of a reflective tag can be, for example, circular. Two-dimensional position refers to the pixel position of the reflective tag in a two-dimensional infrared image.

[0101] Optionally, the characteristics of reflective tags in infrared images can be utilized. Reflective tags can be used as image features, and a preset segmentation algorithm (such as threshold segmentation algorithm) can be used to segment the infrared image to identify all reflective tags on the calibration board and determine the pixel position of the center of the reflective tag, that is, to determine the two-dimensional position of the reflective tag in the pixel coordinate system. Furthermore, based on the preset transformation rules, the two-dimensional position of the reflective tag in the image coordinate system can be determined as the two-dimensional position of the reflective tag in the infrared image.

[0102] It should be noted that by setting the labels on the calibration plate to reflective labels and using them as image features, the efficiency of subsequent image processing can be improved. While ensuring the accuracy of extraction, the two-dimensional position of the reflective labels in the infrared image can be quickly determined, thus improving the efficiency and accuracy of calibration.

[0103] S305. Identify the target label from the reflective labels on the calibration plate, and determine the two-dimensional position of the target label in the infrared image as the third position of the target label in the image coordinate system.

[0104] Among them, the target label refers to the reflective label that meets the screening criteria.

[0105] Optionally, the number of target labels can be preset, such as 4. After determining the number of reflective labels on the calibration plate, if the number of reflective labels on the calibration plate is the same as the number of target labels, all reflective labels can be directly determined as target labels. If the number of reflective labels on the calibration plate is greater than the number of target labels, a preset number of labels can be randomly selected from the reflective labels as target labels. Alternatively, the reflective labels can be filtered based on certain screening principles, such as the target labels not being located on the same straight line, to determine the target labels.

[0106] S306. Based on the second and third positions, calibrate the image sensor and the radar sensor.

[0107] The technical solution of this invention, after determining the second position of the target tag in the radar coordinate system, controls the robot to acquire an infrared image of the calibration board from the same position using the robot's image sensor. The infrared image is segmented to determine the two-dimensional position of the reflective tag on the calibration board within the infrared image. The target tag is then identified from the reflective tags on the calibration board, and its two-dimensional position in the infrared image is determined as its third position in the image coordinate system. Finally, the image sensor and radar sensor are calibrated based on the second and third positions. By identifying the target tag from the reflective tags and determining its third position, the determination of the position of all reflective tags is avoided, improving calibration efficiency while ensuring accuracy. Furthermore, by using an image sensor to acquire infrared images and utilizing the reflective properties of the reflective tags to determine their position in the image, the accurate location of the tag is determined. Compared to the existing detection method using a black-and-white grid calibration board, the calibration board with reflective tags can complete calibration even in dark environments, broadening the robot's application scenarios and improving its detection performance.

[0108] Example 4

[0109] Figure 4 This is a flowchart of a robot sensor calibration method provided in Embodiment 4 of the present invention; based on the above embodiments, this embodiment further explains in detail the "calibration of the robot image sensor and radar sensor according to the second position and the third position", such as... Figure 4 As shown, the method specifically includes:

[0110] S401. Based on geometric similarity, determine the transformation relationship between the calibration plate coordinate system and the radar coordinate system according to the scanning data of the calibration plate by the robot's radar sensor and the attribute information of the calibration plate.

[0111] S402. Based on the first position of the target tag on the calibration board in the calibration board coordinate system and the transformation relationship, determine the second position of the target tag in the radar coordinate system.

[0112] S403. Based on the image data collected by the robot's image sensor from the calibration board, determine the third position of the target label in the image coordinate system.

[0113] S404. Based on the preset two-dimensional image coordinate system and three-dimensional camera coordinate system conversion algorithm, determine the fourth position of the target label in the three-dimensional camera coordinate system according to the third position.

[0114] The transformation algorithm refers to a pre-defined algorithm that can convert the position between the two-dimensional image coordinate system and the three-dimensional camera coordinate system, such as the PNP algorithm (Perspective-n-Point). The fourth position refers to the three-dimensional position of the target label in the three-dimensional camera coordinate system.

[0115] Optionally, based on the preset intrinsic parameter matrix of the image sensor, the PNP algorithm can be used to determine the extrinsic parameters corresponding to the third position of the target label in the image coordinate system. Then, by combining the extrinsic parameters and the third position, the fourth position of the target label in the 3D camera coordinate system can be determined. For example, if the third position of the target label in the image coordinate system is (XC, YC), and the corresponding position in the 3D camera coordinate system is (X'C, Y'C), and the extrinsic parameter corresponding to the third position is ZC, then the fourth position of the target label in the 3D camera coordinate system is (X'C, Y'C, ZC).

[0116] S405. Based on the fourth and second positions, determine the rotation and translation matrices between the target radar coordinate system and the three-dimensional camera coordinate system to calibrate the robot image sensor and radar sensor.

[0117] Optionally, for each target label, the fourth and second positions of the target label can be substituted into a preset calculation formula. Finally, the calculation formulas satisfied by all target labels can be solved together to determine the rotation matrix and translation matrix between the target radar coordinate system and the three-dimensional camera coordinate system, thereby realizing the calibration of the robot image sensor and radar sensor.

[0118] For example, the preset calculation formula can be:

[0119] (XG,YG,ZG)=Rx2*(X'C,Y'C,ZC)+T2

[0120] Where (XG, YG, ZG) represents the second position of the target label in the radar coordinate system, (X'C, Y'C, ZC) represents the fourth position of the target label in the 3D camera coordinate system, Rx2 represents the rotation matrix between the 3D camera coordinate system and the radar coordinate system, and T2 represents the translation matrix between the 3D camera coordinate system and the radar coordinate system.

[0121] Optionally, the number of target tags is at least four; correspondingly, the rotation and translation matrices between the target radar coordinate system and the 3D camera coordinate system are determined, including: determining at least four sets of positional relationships between the target tags in the 3D camera coordinate system and the target radar coordinate system based on the fourth position of the target tag in the 3D camera coordinate system and the second position of the target tag in the target radar coordinate system; and using the PNP algorithm, determining the rotation and translation matrices between the target radar coordinate system and the 3D camera coordinate system based on the determined at least four sets of positional relationships.

[0122] For example, if the number of target tags is at least four, the second and fourth positions of each target tag can be substituted into the above-mentioned preset calculation formula, and the PNP algorithm can be further used to determine the rotation matrix and translation matrix between the target radar coordinate system and the three-dimensional camera coordinate system.

[0123] It should be noted that by transforming the two-dimensional image coordinates of the label points to the three-dimensional camera coordinate system, the conversion relationship between the three-dimensional camera coordinates and the three-dimensional radar coordinates can be determined, thereby achieving robot calibration and ensuring the accuracy of the calibration.

[0124] The technical solution of this invention, after determining the third position of the target label in the image coordinate system, uses a preset transformation algorithm between the two-dimensional image coordinate system and the three-dimensional camera coordinate system to determine the fourth position of the target label in the three-dimensional camera coordinate system based on the third position. Finally, based on the fourth and second positions, the rotation and translation matrices between the target radar coordinate system and the three-dimensional camera coordinate system are determined, thereby achieving the calibration of the robot's image sensor and radar sensor. By transforming the two-dimensional image coordinates of the label point to the three-dimensional camera coordinate system, the transformation relationship between the three-dimensional camera coordinates and the three-dimensional radar coordinates can be determined, achieving accurate calibration of the robot sensor.

[0125] Example 5

[0126] Figure 5 This is a structural block diagram of a robot sensor calibration device provided in Embodiment 5 of the present invention. The robot sensor calibration device provided in this embodiment of the present invention can execute the robot sensor calibration method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0127] like Figure 5 As shown, the device includes:

[0128] The relationship determination module 501 is used to determine the transformation relationship between the calibration plate coordinate system and the radar coordinate system based on geometric similarity, according to the scanning data of the calibration plate by the robot's radar sensor and the attribute information of the calibration plate.

[0129] The position determination module 502 is used to determine the second position of the target tag in the radar coordinate system based on the first position of the target tag on the calibration board in the calibration board coordinate system and the transformation relationship.

[0130] The calibration module 503 is used to determine the third position of the target label in the image coordinate system based on the image data collected by the robot's image sensor from the calibration board, and to calibrate the image sensor and the radar sensor based on the second position and the third position.

[0131] The technical solution of this invention, based on geometric similarity, determines the transformation relationship between the calibration board coordinate system and the radar coordinate system based on the scanning data of the calibration board by the robot's radar sensor and the attribute information of the calibration board. Based on the first position of the target label on the calibration board in the calibration board coordinate system and the transformation relationship, the second position of the target label in the radar coordinate system is determined. Based on the image data collected by the robot's image sensor from the calibration board, the third position of the target label in the image coordinate system is determined. Based on the second and third positions, the image sensor and the radar sensor are calibrated. By determining a more accurate transformation relationship between the calibration board coordinate system and the radar coordinate system based on geometric similarity, the precise position of the target label in the radar coordinate system can be obtained. Further combining this with the position of the target label in the image coordinate system, the calibration of the robot's image sensor and radar sensor is achieved. Through multi-sensor data fusion, the sensors can complement each other, improving the overall effect of the robot's perception system and thus increasing the accuracy of robot detection.

[0132] Furthermore, the relationship determination module 501 may include:

[0133] The acquisition unit is used to acquire the scanning data of the robot's radar sensor when it scans the calibration plate parallel to the bottom edge of the calibration plate;

[0134] A line segment determination unit is used to determine a detection line segment parallel to the bottom edge of the calibration plate based on the scan data.

[0135] The relationship determination unit is used to determine the transformation relationship between the calibration plate coordinate system and the radar coordinate system based on the bottom edge length of the calibration plate, the detection line segment, and the height of the calibration plate.

[0136] Furthermore, the relation determination unit is specifically used for:

[0137] Based on geometric similarity, the perpendicular line corresponding to the detection line segment is determined according to the length of the detection line segment, the length of the bottom edge of the calibration plate, and the height of the calibration plate.

[0138] Based on the bottom edge of the calibration plate, construct a coordinate system for the candidate calibration plate;

[0139] Based on the deviation between the length of the vertical line and the height of the calibration plate, the coordinate system of the candidate calibration plate is translated to determine the coordinate system of the target calibration plate.

[0140] Based on the projection of the detection line segment onto the radar coordinate system, the transformation relationship between the target calibration plate coordinate system and the radar coordinate system is determined.

[0141] Furthermore, the calibration module 503 is specifically used for:

[0142] Control the robot to the same position and acquire infrared images of the calibration board obtained by the robot's image sensor;

[0143] The infrared image is segmented to determine the two-dimensional position of the reflective label on the calibration plate in the infrared image;

[0144] The target label is determined from the reflective labels on the calibration plate, and the two-dimensional position of the target label in the infrared image is determined as the third position of the target label in the image coordinate system.

[0145] Furthermore, the calibration module 503 may include:

[0146] The fourth position determination unit is used to determine the fourth position of the target label in the three-dimensional camera coordinate system based on the third position, according to the preset two-dimensional image coordinate system and the three-dimensional camera coordinate system conversion algorithm.

[0147] The calibration unit is used to determine the rotation and translation matrices between the target radar coordinate system and the three-dimensional camera coordinate system based on the fourth position and the second position, so as to calibrate the robot image sensor and the radar sensor.

[0148] Furthermore, the number of target tags is at least four; the calibration unit is specifically used for:

[0149] Based on the fourth position of the target label in the three-dimensional camera coordinate system and the second position of the target label in the target radar coordinate system, at least four sets of target labels are determined to have positional relationships in the three-dimensional camera coordinate system and the target radar coordinate system, respectively.

[0150] The PNP algorithm is used to determine the rotation and translation matrices between the target radar coordinate system and the 3D camera coordinate system based on at least four determined positional relationships.

[0151] Furthermore, the above-mentioned device is also used for:

[0152] If the number of radar sensors and / or image sensors of the robot is at least two, then the associated sensor group is determined according to the robot's control rules, and the main sensor and the corresponding main sensor coordinate system are determined within each associated sensor group.

[0153] Determine the transformation relationship between the main sensor coordinate system and other sensor coordinate systems within each group, and then calibrate the robot sensors.

[0154] Example 6

[0155] Figure 6 This is a schematic diagram of the structure of a robot provided in Embodiment Six of the present invention. Figure 6 A schematic diagram of a robot 10, which can be used to implement embodiments of the present invention, is shown. The robot is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The robot can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0156] like Figure 6 As shown, robot 10 includes at least one processor 11 and a memory, such as read-only memory (ROM) 12, random access memory (RAM) 13, etc., communicatively connected to at least one processor 11. The memory stores computer programs executable by at least one processor. Processor 11 can perform various appropriate actions and processes based on the computer program stored in ROM 12 or loaded from storage unit 18 into RAM 13. RAM 13 can also store various programs and data required for the operation of robot 10. Processor 11, ROM 12, and RAM 13 are interconnected via bus 14. Input / output (I / O) interface 15 is also connected to bus 14.

[0157] Multiple components in robot 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows robot 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0158] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as robot sensor calibration methods.

[0159] In some embodiments, the robot sensor calibration method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or mounted on the robot 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the robot sensor calibration method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the robot sensor calibration method by any other suitable means (e.g., by means of firmware).

[0160] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0161] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0162] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0163] To provide interaction with a user, the systems and techniques described herein can be implemented on a robot having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the robot. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0164] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0165] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0166] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0167] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A robot sensor calibration method, characterized by, include: Based on geometric similarity, the transformation relationship between the calibration plate coordinate system and the radar coordinate system is determined according to the scanning data of the calibration plate by the robot's radar sensor and the attribute information of the calibration plate. Based on the first position of the target label on the calibration board in the calibration board coordinate system and the transformation relationship, the second position of the target label in the radar coordinate system is determined; Based on the image data collected by the robot's image sensor from the calibration board, the third position of the target label in the image coordinate system is determined, and the image sensor and the radar sensor are calibrated based on the second position and the third position. The calibration plate attribute information includes the base length and height of the calibration plate. The step of determining the transformation relationship between the calibration plate coordinate system and the radar coordinate system based on geometric similarity, according to the scanning data of the robot's radar sensor and the calibration plate attribute information, includes: Acquire scanning data of the robot's radar sensor when it scans the calibration plate parallel to the bottom edge of the calibration plate; Based on the scan data, a detection line segment parallel to the bottom edge of the calibration plate is determined; wherein, the detection line segment refers to the intersection line segment of the scanning ray of the radar sensor and the calibration plate; Based on the geometric similarity relationship satisfied by the geometry of the calibration plate, the perpendicular line corresponding to the detection line segment is determined by proportional calculation according to the length of the detection line segment, the length of the bottom edge of the calibration plate, and the height of the calibration plate. Based on the bottom edge of the calibration plate, construct a coordinate system for the candidate calibration plate; Based on the deviation between the length of the vertical line and the height of the calibration plate, the coordinate system of the candidate calibration plate is translated to determine the coordinate system of the target calibration plate. Based on the projection of the detection line segment onto the radar coordinate system, the transformation relationship between the target calibration plate coordinate system and the radar coordinate system is determined; the target calibration plate coordinate system takes the center of the detection line segment as its origin. The calibration plate has a geometric shape of triangle, isosceles trapezoid, or right trapezoid; the geometric similarity relationship refers to the similarity relationship with the geometric shape of the calibration plate.

2. The method according to claim 1, characterized in that, The step of determining the third position of the target label in the image coordinate system based on the image data collected by the robot's image sensor from the calibration board includes: Control the robot to the same position and acquire infrared images of the calibration board obtained by the robot's image sensor; The infrared image is segmented to determine the two-dimensional position of the reflective label on the calibration plate in the infrared image; The target label is determined from the reflective labels on the calibration plate, and the two-dimensional position of the target label in the infrared image is determined as the third position of the target label in the image coordinate system.

3. The method according to claim 1, characterized in that, Based on the second and third positions, the robot's image sensor and radar sensor are calibrated, including: Based on the preset transformation algorithm between the two-dimensional image coordinate system and the three-dimensional camera coordinate system, the fourth position of the target label in the three-dimensional camera coordinate system is determined according to the third position. Based on the fourth position and the second position, the rotation matrix and translation matrix between the target radar coordinate system and the three-dimensional camera coordinate system are determined to achieve the calibration of the robot image sensor and radar sensor.

4. The method according to claim 3, characterized in that, in, The number of target tags is at least four; Accordingly, the rotation and translation matrices between the target radar coordinate system and the 3D camera coordinate system are determined, including: Based on the fourth position of the target label in the three-dimensional camera coordinate system and the second position of the target label in the target radar coordinate system, at least four sets of target labels are determined to have positional relationships in the three-dimensional camera coordinate system and the target radar coordinate system, respectively. The PNP algorithm is used to determine the rotation and translation matrices between the target radar coordinate system and the 3D camera coordinate system based on at least four determined positional relationships.

5. The method according to claim 1, characterized in that, Also includes: If the number of radar sensors and / or image sensors of the robot is at least two, then the associated sensor group is determined according to the robot's control rules, and the main sensor and the corresponding main sensor coordinate system are determined within each associated sensor group. Determine the transformation relationship between the main sensor coordinate system and other sensor coordinate systems within each group, and then calibrate the robot sensors.

6. A robot sensor calibration device, characterized in that, include: The relationship determination module is used to determine the transformation relationship between the calibration board coordinate system and the radar coordinate system based on geometric similarity, according to the scanning data of the calibration board by the robot's radar sensor and the attribute information of the calibration board. The position determination module is used to determine the second position of the target tag in the radar coordinate system based on the first position of the target tag on the calibration board in the calibration board coordinate system and the transformation relationship. The calibration module is used to determine the third position of the target label in the image coordinate system based on the image data collected by the robot's image sensor from the calibration board, and to calibrate the image sensor and the radar sensor based on the second position and the third position. The calibration plate attribute information includes the bottom edge length and the height of the calibration plate; The relationship determination module includes: The acquisition unit is used to acquire the scanning data of the robot's radar sensor when it scans the calibration plate parallel to the bottom edge of the calibration plate; A line segment determination unit is used to determine a detection line segment parallel to the bottom edge of the calibration plate based on the scanning data; wherein, the detection line segment refers to the intersection line segment of the scanning ray of the radar sensor and the calibration plate; The relationship determination unit is used to determine the transformation relationship between the calibration plate coordinate system and the radar coordinate system based on the bottom edge length of the calibration plate, the detection line segment, and the height of the calibration plate. The relationship determination unit is specifically used for: determining the perpendicular line corresponding to the detection line segment based on the geometric similarity relationship satisfied by the calibration plate's geometry, according to the length of the detection line segment, the length of the base of the calibration plate, and the height of the calibration plate, through proportional calculation; constructing a candidate calibration plate coordinate system based on the base of the calibration plate; translating the candidate calibration plate coordinate system according to the deviation between the length of the perpendicular line and the height of the calibration plate to determine the target calibration plate coordinate system; determining the transformation relationship between the target calibration plate coordinate system and the radar coordinate system based on the projection of the detection line segment onto the radar coordinate system; the target calibration plate coordinate system has the center of the detection line segment as its origin; wherein, the geometry of the calibration plate is a triangle, an isosceles trapezoid, or a right trapezoid; the geometric similarity relationship refers to the similarity relationship corresponding to the geometry of the calibration plate.

7. A robot, characterized in that, The robot includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the robot sensor calibration method according to any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the robot sensor calibration method according to any one of claims 1-5.

Images