Extrinsic parameter calibration method for sensors, and cleaning robot, cleaning system and medium

Abstract

An extrinsic parameter calibration method for sensors. The method comprises: during the motion process of a cleaning robot, acquiring reference local sensing data of a reference sensor for each motion interval, and acquiring auxiliary local sensing data of each sensor to be calibrated for each motion interval (S110); on the basis of the reference local sensing data and the auxiliary local sensing data, determining a first transformation matrix from each sensor to be calibrated to the reference sensor (S120); acquiring a second transformation matrix from the reference sensor to an odometry coordinate system, and on the basis of the first transformation matrix combined with the second transformation matrix, determining a candidate transformation matrix from each sensor to be calibrated to the odometry coordinate system (S130); and optimizing each candidate transformation matrix, in order to obtain a final transformation matrix from each sensor to be calibrated to the odometry coordinate system (S140). Thus, low-cost and accurate calibration of extrinsic parameters of sensors can be achieved. Further disclosed are a cleaning robot, a cleaning system, a computer-readable storage medium, and an electronic device (1600).

Description

Extrinsic calibration method of sensor, cleaning robot, cleaning system and medium Cross-reference to Related Applications

[0001] This application claims priority to the application with the application number 202411338145.2 and the application name "Extrinsic calibration method of sensor, cleaning robot, cleaning system and medium" filed with the State Intellectual Property Office of China on September 24, 2024, the whole content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The present application relates to the field of automatic control technology, in particular to an extrinsic calibration method of sensor, a cleaning robot, a cleaning system, a computer readable storage medium and an electronic device. BACKGROUND

[0003] A robot system usually needs to fuse the data of multiple sensors to work together, and the sensing characteristics of these sensors are different, and are placed at different positions of the robot body, so that the perception of each sensor to the environmental information is different. When the robot system applies these sensor data, it is hoped that these data can be converted into a unified description.

[0004] In view of this, there is an urgent need in the art to develop a new extrinsic calibration method of sensor and a cleaning robot.

[0005] It should be noted that the information disclosed in the above background section is only used to strengthen the understanding of the background of the present application. SUMMARY

[0006] The purpose of the present application is to provide an extrinsic calibration method of sensor, a cleaning robot, a computer readable storage medium and an electronic device.

[0007] Other characteristics and advantages of the present application will become apparent from the following detailed description, or will be learned by practice of the present application.

[0008] According to a first aspect of the present application, an extrinsic calibration method of sensor is provided, the method is applied to a cleaning robot, the cleaning robot is provided with K+1 sensors, the K+1 sensors include a reference sensor and K sensors to be calibrated, K is an integer greater than 1, and the method comprises:

[0009] In the motion process of the cleaning robot, reference local perception data of the reference sensor for each motion interval is obtained, and auxiliary local perception data of each of the sensors to be calibrated for the each motion interval is obtained;

[0010] According to the reference local perception data, the to-be-solved transformation matrix of each of the to-be-calibrated sensors to the reference sensor, and the auxiliary local perception data, a local optimization is performed on the to-be-solved transformation matrix of each of the to-be-calibrated sensors to the reference sensor, to obtain a first transformation matrix of each of the to-be-calibrated sensors to the reference sensor;

[0011] A second transformation matrix of the reference sensor to the odometer coordinate system is obtained, and according to the first transformation matrix and the second transformation matrix, a candidate transformation matrix of each of the to-be-calibrated sensors to the odometer coordinate system is determined;

[0012] According to the global perception data of each of the to-be-calibrated sensors during the movement of the cleaning robot, a global optimization is performed on the candidate transformation matrix of each of the to-be-calibrated sensors to the odometer coordinate system, to obtain a final transformation matrix of each of the to-be-calibrated sensors to the odometer coordinate system.

[0013] In the example embodiments of the present application, the local optimization of the to-be-solved transformation matrix of each of the to-be-calibrated sensors to the reference sensor according to the reference local perception data, the to-be-solved transformation matrix of each of the to-be-calibrated sensors to the reference sensor, and the auxiliary local perception data includes:

[0014] According to the reference local perception data, the to-be-solved transformation matrix of each of the to-be-calibrated sensors to the reference sensor, and the auxiliary local perception data, a local optimization is performed on the to-be-solved transformation matrix of each of the to-be-calibrated sensors to the reference sensor, to obtain a first transformation matrix of each of the to-be-calibrated sensors to the reference sensor;

[0015] The value of the to-be-solved transformation matrix is optimized to minimize the comprehensive perception deviation, to obtain a first transformation matrix of each of the to-be-calibrated sensors to the reference sensor.

[0016] In the example embodiments of the present application, the determination of the comprehensive perception deviation corresponding to N motion intervals according to the reference local perception data, the to-be-solved transformation matrix of each of the to-be-calibrated sensors to the reference sensor, and the auxiliary local perception data includes:

[0017] determining, for each combination of the reference sensor and each to-be-calibrated sensor, a comprehensive perception deviation corresponding to N motion intervals according to the reference local perception data, a to-be-solved transformation matrix of each to-be-calibrated sensor to the reference sensor, and the auxiliary local perception data, wherein the comprehensive perception deviation is determined according to N local perception deviations corresponding to the N motion intervals.

[0018] The comprehensive perception deviation is determined according to N local perception deviations corresponding to M effective motion intervals.

[0019] In the example embodiments of the present application, the determining, for each combination of the reference sensor and each to-be-calibrated sensor, a comprehensive perception deviation corresponding to N motion intervals according to the reference local perception data, a to-be-solved transformation matrix of each to-be-calibrated sensor to the reference sensor, and the auxiliary local perception data, comprises:

[0020] For each combination of the reference sensor and each to-be-calibrated sensor, M effective data groups with successful registration are selected from N data groups corresponding to the N motion intervals; M is less than or equal to N; the M effective data groups are associated with M effective motion intervals; each data group contains the reference local perception data and the auxiliary local perception data, and each effective data group contains effective reference local perception data and effective auxiliary local perception data with successful registration;

[0021] For each combination of the reference sensor and each to-be-calibrated sensor, a local perception deviation of each to-be-calibrated sensor relative to the reference sensor on each effective motion interval is determined according to the effective reference local perception data, a to-be-solved transformation matrix of each to-be-calibrated sensor to the reference sensor, and the effective auxiliary local perception data.

[0022] The comprehensive perception deviation is determined according to M local perception deviations corresponding to the M effective motion intervals.

[0023] In the example embodiments of the present application, the effective reference local perception data comprises an effective reference local map, the effective auxiliary local perception data comprises an effective auxiliary local map, and the local perception deviation comprises a local map deviation.

[0024] For each combination of the reference sensor and each to-be-calibrated sensor, the determining a local perception deviation of each to-be-calibrated sensor relative to the reference sensor on each effective motion interval according to the effective reference local perception data, a to-be-solved transformation matrix of each to-be-calibrated sensor to the reference sensor, and the effective auxiliary local perception data, comprises:

[0025] According to the effective reference local map, the to-be-calibrated sensor-to-reference sensor conversion matrix of each to-be-calibrated sensor, and the effective auxiliary local map, a local map deviation of each to-be-calibrated sensor relative to the reference sensor on each effective motion interval is determined.

[0026] In the example embodiment of the present application, the local map deviation of each to-be-calibrated sensor relative to the reference sensor on each effective motion interval is determined based on the following formula:

[0027]

[0028] wherein, l map represents the local map deviation of each to-be-calibrated sensor relative to the reference sensor on each effective motion interval, for each combination of a reference sensor and a to-be-calibrated sensor, represents an effective reference local map corresponding to the reference sensor for the i-th effective motion interval, represents an effective auxiliary local map corresponding to the to-be-calibrated sensor for the i-th effective motion interval, and T represents the to-be-calibrated sensor-to-reference sensor conversion matrix to be solved.

[0029] In the example embodiment of the present application, the comprehensive perception deviation is determined according to M local perception deviations corresponding to M effective motion intervals, including:

[0030] The comprehensive perception deviation is determined based on the following formula:

[0031]

[0032] wherein, l a1 represents the comprehensive perception deviation.

[0033] In the example embodiment of the present application, the effective reference local perception data includes an effective reference local trajectory, the effective auxiliary local perception data includes an effective auxiliary local trajectory, and the local perception deviation includes a local trajectory deviation;

[0034] For each combination of the reference sensor and each to-be-calibrated sensor, the local perception deviation of each to-be-calibrated sensor relative to the reference sensor on each effective motion interval is determined according to the effective reference local perception data, the to-be-calibrated sensor-to-reference sensor conversion matrix to be solved, and the effective auxiliary local perception data, including:

[0035] According to the effective reference local perception trajectory, the to-be-calibrated sensor-to-reference sensor conversion matrix to be solved, and the effective auxiliary local perception estimation, a local trajectory deviation of each to-be-calibrated sensor relative to the reference sensor on each effective motion interval is determined.

[0036] In the example embodiment of the present application, the local trajectory deviation of each to-be-calibrated sensor relative to the reference sensor on each effective motion interval is determined based on the following formula:

[0037]

[0038] wherein, for each combination of a reference sensor and a to-be-calibrated sensor, l traj represents the local trajectory deviation of each to-be-calibrated sensor relative to the reference sensor on each effective motion interval, represents the effective reference local trajectory corresponding to the reference sensor for the i-th effective motion interval, represents the effective auxiliary local trajectory corresponding to the to-be-calibrated sensor for the i-th effective motion interval, and T represents the to-be-calibrated sensor-to-reference sensor conversion matrix to be solved.

[0039] In the example embodiment of the present application, the determination of the comprehensive perception deviation according to the M local perception deviations corresponding to the M effective motion intervals comprises:

[0040] The comprehensive perception deviation is determined based on the following formula:

[0041]

[0042] wherein, for each combination of a reference sensor and a to-be-calibrated sensor, l a1 represents the comprehensive perception deviation.

[0043] In the example embodiment of the present application, each data group comprises a map group and a trajectory group, each map group is composed of a reference local map and an auxiliary local map corresponding to each motion interval, and each trajectory group is composed of a reference local trajectory and an auxiliary local trajectory corresponding to each motion interval.

[0044] The screening of M effective data groups with successful registration from N data groups corresponding to N motion intervals comprises:

[0045] For each combination of the reference sensor and each of the to-be-calibrated sensors, M valid map groups are selected from N map groups corresponding to N motion intervals, each of the map groups consisting of a reference local map and an auxiliary local map, each of the valid map groups consisting of a valid reference local map and a valid auxiliary local map;

[0046] and M valid trajectory groups are selected from N trajectory groups corresponding to N motion intervals, each of the trajectory groups consisting of a reference local trajectory and an auxiliary local trajectory, each of the valid trajectory groups consisting of a valid reference local trajectory and a valid auxiliary local trajectory.

[0047] In exemplary embodiments of the present application, for each combination of the reference sensor and each of the to-be-calibrated sensors, each of the valid map groups and each of the valid trajectory groups are determined by:

[0048] For each combination of a reference sensor and a to-be-calibrated sensor, a first converted map is determined according to a product between the auxiliary local map for each motion interval and an estimated map conversion matrix, and a first converted trajectory is determined according to a product between the auxiliary local trajectory for each motion interval and an estimated trajectory conversion matrix;

[0049] A specified map deviation between the reference local map corresponding to each motion interval and the first converted map is determined, a first norm corresponding to the specified map deviation and a first square value of the first norm are determined;

[0050] The value of the estimated map conversion matrix is optimized so that the first square value is minimized;

[0051] A specified trajectory deviation between the reference local trajectory corresponding to each motion interval and the first converted trajectory is determined, a second norm corresponding to the specified trajectory deviation and a second square value of the second norm are determined;

[0052] The value of the estimated trajectory conversion matrix is optimized so that the second square value is minimized;

[0053] If the first square value with the smallest value is less than a first preset error threshold and the second square value with the smallest value is less than a second preset error threshold, the auxiliary local map and the reference local map are determined as one valid map group with successful registration and the auxiliary local trajectory and the reference local trajectory are determined as one valid trajectory group with successful registration.

[0054] In exemplary embodiments of the present application, the local perception deviation includes a local map deviation and a local trajectory deviation;

[0055] For the combination of the reference sensor and each of the to-be-calibrated sensors, the local perception deviation of each of the to-be-calibrated sensors relative to the reference sensor on each of the effective motion intervals is determined according to the effective reference local perception data, the to-be-solved transformation matrix of each of the to-be-calibrated sensors to the reference sensor, and the effective auxiliary local perception data, including:

[0056] The local map deviation of each of the to-be-calibrated sensors relative to the reference sensor on each of the effective motion intervals is determined according to the effective reference local map, the to-be-solved transformation matrix of each of the to-be-calibrated sensors to the reference sensor, and the effective auxiliary local map.

[0057] The local trajectory deviation of each of the to-be-calibrated sensors relative to the reference sensor on each of the effective motion intervals is determined according to the effective reference local perception trajectory, the to-be-solved transformation matrix of each of the to-be-calibrated sensors to the reference sensor, and the effective auxiliary local perception estimate.

[0058] In the example embodiment of the present application, the comprehensive perception deviation l a1 is determined according to M

[0059] The comprehensive perception deviation is determined based on the following formula:

[0060]

[0061] Wherein, l a1 represents the comprehensive perception deviation, represents the cumulative value of M represents the cumulative value of M

[0062] In the example embodiment of the present application, the second transformation matrix of the reference sensor to the odometer coordinate system is obtained, including:

[0063] For each of the effective motion intervals, the odometer trajectory of the cleaning robot is obtained;

[0064] According to the odometer trajectory, each of the effective reference local trajectory, and the second to-be-solved transformation matrix of the reference sensor to the odometer coordinate system, each of the standard local trajectory deviations is determined, and the comprehensive trajectory deviation corresponding to M

[0065] optimizing a value of a second to-be-solved conversion matrix of the reference sensor to the odometry coordinate system so as to minimize the comprehensive trajectory deviation, and solving the second conversion matrix of the reference sensor to the odometry coordinate system.

[0066] In an example embodiment of the present application, the determining of each standard local trajectory deviation according to the odometry trajectory, each effective reference local trajectory and the second to-be-solved conversion matrix of the reference sensor to the odometry coordinate system comprises:

[0067] The each standard local trajectory deviation is determined based on the following formula:

[0068]

[0069] wherein, for the i-th effective motion interval, l straj represents the each standard local trajectory deviation, represents the odometry trajectory of the cleaning robot, T 基准2里程计 represents the second to-be-solved conversion matrix of the reference sensor to the odometry coordinate system, represents the effective reference local trajectory.

[0070] In an example embodiment of the present application, the determining of the candidate conversion matrix of each to-be-calibrated sensor to the odometry coordinate system according to the first conversion matrix in combination with the second conversion matrix comprises:

[0071] multiplying the first conversion matrix and the second conversion matrix to obtain the candidate conversion matrix of each to-be-calibrated sensor to the odometry coordinate system.

[0072] In an example embodiment of the present application, the global perception data of each to-be-calibrated sensor in the motion process of the cleaning robot comprises an auxiliary global trajectory of each to-be-calibrated sensor in the motion process of the cleaning robot;

[0073] The global optimization of the candidate conversion matrix of each to-be-calibrated sensor to the odometry coordinate system according to the global perception data of each to-be-calibrated sensor in the motion process of the cleaning robot to obtain the final conversion matrix of each to-be-calibrated sensor to the odometry coordinate system comprises:

[0074] configuring an identification number corresponding to each to-be-calibrated sensor of the K to-be-calibrated sensors, and the K identification numbers are in an increasing order;

[0075] When the maximum identification number is K, an xth to-be-calibrated sensor is selected from the K to-be-calibrated sensors in turn, and an x+1th sensor to a Kth sensor is combined with the xth to-be-calibrated sensor respectively to obtain K-x sensor combinations associated with the xth to-be-calibrated sensor; each of the sensor combinations comprises the xth to-be-calibrated sensor and a designated to-be-calibrated sensor; wherein 1≤x≤K-1;

[0076] For each of the sensor combinations, an error expression between the xth to-be-calibrated sensor and the designated to-be-calibrated sensor is determined according to an auxiliary global trajectory associated with the xth to-be-calibrated sensor, an auxiliary global trajectory associated with the designated to-be-calibrated sensor, a candidate conversion matrix of the xth to-be-calibrated sensor to the odometer coordinate system, and a candidate conversion matrix of the designated to-be-calibrated sensor to the odometer coordinate system;

[0077] An overall error expression between the xth to-be-calibrated sensor and K-x designated to-be-calibrated sensors is determined according to a sum of K-x error expressions;

[0078] A value of the candidate conversion matrix of the xth to-be-calibrated sensor to the odometer coordinate system is optimized so that a value of the overall error expression is minimum, and a final conversion matrix of the xth to-be-calibrated sensor to the odometer coordinate system is obtained.

[0079] In the example embodiment of the present application, the error expression between the xth to-be-calibrated sensor and the designated to-be-calibrated sensor is determined according to the auxiliary global trajectory associated with the xth to-be-calibrated sensor, the auxiliary global trajectory associated with the designated to-be-calibrated sensor, the candidate conversion matrix of the xth to-be-calibrated sensor to the odometer coordinate system, and the candidate conversion matrix of the designated to-be-calibrated sensor to the odometer coordinate system, and comprises:

[0080] The error expression between the xth to-be-calibrated sensor and the designated to-be-calibrated sensor is determined based on the following formula:

[0081]

[0082] Wherein, e x,y represents the error expression between the xth to-be-calibrated sensor and the designated to-be-calibrated sensor, Traj x represents the auxiliary global trajectory associated with the xth to-be-calibrated sensor, Traj y represents the auxiliary global trajectory associated with the designated to-be-calibrated sensor, represents an inverse matrix of the candidate conversion matrix of the xth to-be-calibrated sensor to the odometer coordinate system, T ya candidate transformation matrix of the designated sensor to be calibrated to the odometer coordinate system.

[0083] In the example embodiments of the present application, the determining the overall error expression between the xth sensor to be calibrated and K-x designated sensors to be calibrated according to the sum of K-x error expressions comprises:

[0084] The overall error expression between the xth sensor to be calibrated and K-x designated sensors to be calibrated is determined according to the following formula:

[0085]

[0086] wherein E represents the overall error expression between the xth sensor to be calibrated and K-x designated sensors to be calibrated.

[0087] In the example embodiments of the present application, the motion interval is obtained according to the motion distance or motion angle of the cleaning robot.

[0088] According to a second aspect of the present application, a cleaning robot is provided, wherein K+1 sensors are arranged on the cleaning robot, the K+1 sensors comprising a reference sensor and K sensors to be calibrated, K being an integer greater than 1, and the cleaning robot comprising a perception module and an extrinsic parameter calibration module:

[0089] The perception module is configured to obtain reference local perception data of the reference sensor for each motion interval during the motion process of the cleaning robot, and obtain auxiliary local perception data of each sensor to be calibrated for the each motion interval.

[0090] The extrinsic parameter calibration module is configured to, for each combination of the reference sensor and a sensor to be calibrated, perform local optimization on a to-be-solved transformation matrix of each sensor to be calibrated to the reference sensor according to the reference local perception data, the to-be-solved transformation matrix of each sensor to be calibrated to the reference sensor, and the auxiliary local perception data, to obtain a first transformation matrix of each sensor to be calibrated to the reference sensor.

[0091] obtain a second transformation matrix of the reference sensor to the odometer coordinate system, and determine a candidate transformation matrix of each sensor to be calibrated to the odometer coordinate system according to the first transformation matrix and the second transformation matrix.

[0092] According to global perception data of each to-be-calibrated sensor during movement of the cleaning robot, a candidate transformation matrix of each to-be-calibrated sensor to the odometer coordinate system is globally optimized to obtain a final transformation matrix of each to-be-calibrated sensor to the odometer coordinate system.

[0093] According to a third aspect of the present application, a cleaning system is provided, the cleaning system comprising K+1 sensors, the K+1 sensors comprising a reference sensor and K to-be-calibrated sensors, K being an integer greater than 1, the cleaning system comprising a data collector and an extrinsic parameter calibration processor:

[0094] The data collector is configured to, during movement of the cleaning system, acquire reference local perception data of the reference sensor for each movement interval, and acquire auxiliary local perception data of each to-be-calibrated sensor for each movement interval;

[0095] The extrinsic parameter calibration processor is configured to, for each combination of the reference sensor and a to-be-calibrated sensor, according to the reference local perception data, a to-be-solved transformation matrix of each to-be-calibrated sensor to the reference sensor, and the auxiliary local perception data, locally optimize the to-be-solved transformation matrix of each to-be-calibrated sensor to the reference sensor to obtain a first transformation matrix of each to-be-calibrated sensor to the reference sensor;

[0096] A second transformation matrix of the reference sensor to an odometer coordinate system is acquired, and a candidate transformation matrix of each to-be-calibrated sensor to the odometer coordinate system is determined according to the first transformation matrix in combination with the second transformation matrix;

[0097] According to global perception data of each to-be-calibrated sensor during movement of the cleaning system, a candidate transformation matrix of each to-be-calibrated sensor to the odometer coordinate system is globally optimized to obtain a final transformation matrix of each to-be-calibrated sensor to the odometer coordinate system.

[0098] According to a fourth aspect of the present application, a computer readable storage medium is provided, the computer readable storage medium storing a computer program, the computer program being executed by a processor to implement the extrinsic parameter calibration method of the sensor according to the first aspect.

[0099] According to a fifth aspect of the present application, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the extrinsic parameter calibration method of the sensor according to the first aspect by executing the executable instructions.

[0100] The present application should be understood to be merely exemplary and explanatory, and not limiting to the present application. BRIEF DESCRIPTION OF DRAWINGS

[0101] The drawings incorporated in and forming a part of the specification, illustrate preferred embodiments of the present application and, together with the description, serve to explain the principles of the present application. It is pointed out that the drawings described below are only some embodiments of the present application, and other drawings can also be obtained according to these drawings without creative labor for those skilled in the art.

[0102] Fig. 1 shows a flowchart of a method for calibrating extrinsic parameters of sensors in an embodiment of the present application;

[0103] Fig. 2 shows a schematic diagram of a running path of a cleaning robot in an environment in an embodiment of the present application;

[0104] Fig. 3 shows a schematic diagram of a running path of another cleaning robot in an environment in an embodiment of the present application;

[0105] Fig. 4 shows a flowchart of how to locally optimize the to-be-solved transformation matrix of each to-be-calibrated sensor to the reference sensor according to the reference local perception data, the to-be-solved transformation matrix of each to-be-calibrated sensor to the reference sensor and the auxiliary local perception data, and obtain the first transformation matrix of each to-be-calibrated sensor to the reference sensor for the combination of the reference sensor and each to-be-calibrated sensor in an embodiment of the present application;

[0106] Fig. 5 shows a flowchart of how to determine the comprehensive perception deviation corresponding to N motion intervals according to the reference local perception data, the to-be-solved transformation matrix of each to-be-calibrated sensor to the reference sensor and the auxiliary local perception data for the combination of the reference sensor and each to-be-calibrated sensor in an embodiment of the present application;

[0107] Fig. 6 shows a flowchart of how to determine the comprehensive perception deviation corresponding to N motion intervals according to the reference local perception data, the to-be-solved transformation matrix of each to-be-calibrated sensor to the reference sensor and the auxiliary local perception data for the combination of the reference sensor and each to-be-calibrated sensor in another embodiment of the present application;

[0108] Fig. 7 shows a flowchart of how to select M effective data groups from N data groups in an embodiment of the present application;

[0109] Fig. 8 shows a flowchart of how to select M effective data groups from N data groups in another embodiment of the present application;

[0110] Figure 9 shows a flow diagram illustrating how to select M valid data groups from N data groups according to an embodiment of the present application;

[0111] Figure 10 shows a flow diagram illustrating how to obtain a second conversion matrix of a reference sensor to an odometer coordinate system according to an embodiment of the present application;

[0112] Figure 11 shows a flow diagram illustrating how to globally optimize each candidate conversion matrix of each sensor to be calibrated to the odometer coordinate system and obtain a final conversion matrix of each sensor to be calibrated to the odometer coordinate system according to an embodiment of the present application;

[0113] Figure 12 shows a flow diagram illustrating how to calibrate each candidate conversion matrix of each sensor to be calibrated to the odometer coordinate system by using local perception data according to an embodiment of the present application;

[0114] Figure 13 shows a flow diagram illustrating how to globally optimize each candidate conversion matrix and obtain a final conversion matrix of each sensor to be calibrated to the odometer coordinate system according to an embodiment of the present application;

[0115] Figure 14 shows a schematic diagram illustrating a structure of a cleaning robot according to an example embodiment of the present application;

[0116] Figure 15 shows a schematic diagram illustrating a structure of a cleaning system according to an example embodiment of the present application;

[0117] Figure 16 shows a schematic diagram illustrating a structure of an electronic device according to an example embodiment of the present application. DETAILED DESCRIPTION

[0118] Example implementations will now be described more fully with reference to the accompanying drawings. Example implementations can be implemented in any

[0119] The terms "one", "an", "the", and "said" are used in this specification to convey "at least one" or "one or more"; the terms "include" and "has" are used to indicate an open-ended inclusion of elements or components; the terms "first" and "second" are used as labels, not to limit the quantity of objects.

[0120] In addition, the accompanying drawings are merely schematic and are not necessarily drawn to scale. Like reference numerals designate like or similar parts throughout the drawings and the detailed description, in which changes in the drawings will not be described repeatedly. Some of the block diagrams shown in the drawings are functional entities, which do not necessarily correspond to physically or logically independent entities.

[0121] Based on the sensor characteristics, the application proposes a scheme for online calibration of multiple depth sensor extrinsic parameters in a general home scene (non-standard environment), which has the advantages of low scene requirement, simple action, high precision, high reliability, and can meet the online calibration requirements of the depth sensor.

[0122] In the embodiments of the application, a sensor extrinsic parameter calibration method is first provided, which at least partially overcomes the defect in the related art that high-precision equipment support or strong prior knowledge of the environment is required to obtain high-precision sensor extrinsic parameters.

[0123] FIG. 1 shows a flowchart of a sensor extrinsic parameter calibration method in the embodiments of the application, and the execution subject of the sensor extrinsic parameter calibration method can be a cleaning robot.

[0124] Referring to FIG. 1, the sensor extrinsic parameter calibration method according to one embodiment of the application includes the following steps:

[0125] In step S110, during the movement of the cleaning robot, the reference sensor obtains reference local perception data for each movement interval, and each to-be-calibrated sensor obtains auxiliary local perception data for each movement interval.

[0126] In step S120, for each combination of the reference sensor and each to-be-calibrated sensor, the to-be-solved conversion matrix of each to-be-calibrated sensor to the reference sensor is locally optimized according to the reference local perception data, the auxiliary local perception data, and the to-be-solved conversion matrix of each to-be-calibrated sensor to the reference sensor, to obtain a first conversion matrix of each to-be-calibrated sensor to the reference sensor.

[0127] In step S130, a second conversion matrix of the reference sensor to the odometer coordinate system is obtained, and a candidate conversion matrix of each to-be-calibrated sensor to the odometer coordinate system is determined according to the first conversion matrix and the second conversion matrix.

[0128] Step S140, according to the global perception data of each to-be-calibrated sensor in the movement process of the cleaning robot, globally optimizing the candidate transformation matrix of each to-be-calibrated sensor to the odometer coordinate system, and obtaining the final transformation matrix of each to-be-calibrated sensor to the odometer coordinate system.

[0129] In the technical solution provided by the embodiment shown in FIG. 1, on the one hand, by combining the reference sensor and each to-be-calibrated sensor, according to the reference local perception data, the to-be-solved transformation matrix of each to-be-calibrated sensor to the reference sensor, and the auxiliary local perception data, the to-be-solved transformation matrix of each to-be-calibrated sensor to the reference sensor is locally optimized to obtain the first transformation matrix of each to-be-calibrated sensor to the reference sensor, and the second transformation matrix of the reference sensor to the odometer coordinate system is obtained, and according to the first transformation matrix combined with the second transformation matrix, the candidate transformation matrix of each to-be-calibrated sensor to the odometer coordinate system is determined, which can capture the fine-grained local perception deviation according to the fine-grained data (local perception data of each movement interval) without making special requirements on the running environment, so as to ensure that the determined comprehensive perception deviation is more consistent with the actual situation, and the data is more real and reliable, so as to ensure that the candidate transformation matrix with high precision can be determined by the multiple optimization solving mode. On the other hand, by globally optimizing the candidate transformation matrix of each to-be-calibrated sensor to the odometer coordinate system according to the global perception data of each to-be-calibrated sensor in the movement process of the cleaning robot, the final transformation matrix of each to-be-calibrated sensor to the odometer coordinate system is obtained, which can ensure the consistency and accuracy of the sensor in the entire running environment, reduce the error between different sensors due to installation position, angle difference and other reasons, and also reduce the error caused by insufficient local calibration, so as to ensure the accuracy of the sensor extrinsic parameter calibration without the aid of high-precision equipment and prior environmental knowledge, with low cost and high reliability, which also helps to improve the navigation and obstacle avoidance precision of the subsequent cleaning robot, and improve the performance and working efficiency of the cleaning robot.

[0130] The specific implementation process of each step in FIG. 1 is described in detail as follows:

[0131] Before step S110, first of all, it needs to be pointed out that the sensor extrinsic parameter in the present application refers to the position and attitude parameters of the sensor relative to a certain reference coordinate system. Specifically, the extrinsic parameter can include a translation vector (Translation Vector) and a rotation matrix (Rotation Matrix) of the sensor, which collectively describe the specific position and direction of the sensor in its working environment.

[0132] Secondly, the method in the application is applied to a cleaning robot, and the cleaning robot can be provided with K+1 sensors. For example, a reference sensor can be specified from the K+1 sensors, and each of the remaining K (K is an integer greater than 1) sensors can be called a sensor to be calibrated.

[0133] The sensor can be a depth sensor. By providing a plurality of depth sensors on the cleaning robot, more accurate environment perception and navigation can be achieved, helping the robot to construct a three-dimensional model of the surrounding environment, so as to better plan a path and avoid obstacles, and different types of ground coverings or stains can be detected and identified.

[0134] In addition, it should be noted that, since the application is applied to a general home scene, the operating environment of the cleaning robot is not required to be too high, but in order to ensure the accuracy of the calibration result, objects that can be observed by the depth sensor need to exist in the environment, such as furniture such as tables, chairs, stools, and walls of a room. More specifically, the minimum distance between these objects can be less than the range of the depth sensor, and the specific positions of the objects do not need to be known in advance.

[0135] In addition, after entering the calibration mode (i.e., the working mode of calibrating sensor parameters), the cleaning robot can explore the environment. In order to improve the accuracy of the calibration result, the motion path of the cleaning robot can be configured by itself, so that each depth sensor can observe as many objects as possible during the movement. For example, referring to FIGS. 2-3, FIG. 2 shows a schematic diagram of a motion path of a cleaning robot in an environment according to an embodiment of the application, and FIG. 3 shows a schematic diagram of another motion path of a cleaning robot in an environment according to an embodiment of the application. The circle represents the cleaning robot, and the square represents a random object in the home environment. The specific motion path can be set according to actual conditions, and the application does not make special limitations thereon.

[0136] The selection of the path is not unique and can be generated in real time during the exploration. The strategy of exploration is also not unique and can satisfy the condition that each depth sensor can observe as many objects as possible. For example, one strategy can be that, when a depth sensor detects some objects under the condition that the installation position of the sensor module is known in advance, the cleaning robot can be controlled to move to a suitable position away from the objects, and then the orientation of the cleaning robot can be adjusted so that other depth sensors can also observe the objects.

[0137] Referring to FIG. 1, in step S110, during the movement of the cleaning robot, the reference sensor obtains reference local perception data for each movement interval, and each to-be-calibrated sensor obtains auxiliary local perception data for each movement interval.

[0138] In this step, the cleaning robot can be controlled to enter the calibration mode, so that during the movement of the cleaning robot, the reference sensor can obtain reference local perception data for each movement interval, and each to-be-calibrated sensor can obtain auxiliary local perception data for each movement interval.

[0139] For example, the movement of the cleaning robot can be divided into N (N is an integer greater than 1) movement intervals according to the movement distance or the movement angle. For example, the cleaning robot can move a preset distance (for example, 20 cm) as a movement interval, or the cleaning robot can turn as an interval division basis, that is, the cleaning robot is considered to start a new section each time it turns. This division manner enables the cleaning robot to perceive data in a relatively small and controllable range, thereby improving the data perception accuracy.

[0140] Thus, for each movement interval, the reference sensor can output corresponding reference local perception data, and each to-be-calibrated sensor can output corresponding auxiliary local perception data.

[0141] In a first optional embodiment, the local perception data in the present application can be a local map. The local map refers to the environmental information in a certain specific area obtained by the sensor. For example, the local map can be a point cloud map, a grid map, etc., which can be set according to actual conditions, and the present application does not make special limitations thereon. Thus, the reference sensor can output a reference local map for each movement interval, and each to-be-calibrated sensor can output an auxiliary local map for each movement interval.

[0142] In a second optional embodiment, the local perception data in the present application can be a local trajectory. The local trajectory refers to the moving path or state sequence recorded by the sensor within a period of time. These trajectories reflect the actual movement of the robot in the environment, and usually include changes in position, direction, speed, etc. Thus, the reference sensor can output a reference local trajectory for each movement interval, and each to-be-calibrated sensor can output an auxiliary local trajectory for each movement interval.

[0143] In a third alternative implementation, the local perception data in the present application can contain data of both local map and local trajectory, so that the reference sensor can output a reference local map and a reference local trajectory for each motion section, and each to-be-calibrated sensor can output an auxiliary local map and an auxiliary local trajectory for each motion section.

[0144] In step S120, for each combination of the reference sensor and each to-be-calibrated sensor, the to-be-solved transformation matrix of each to-be-calibrated sensor to the reference sensor is locally optimized according to the reference local perception data, the to-be-solved transformation matrix of each to-be-calibrated sensor to the reference sensor, and the auxiliary local perception data, to obtain a first transformation matrix of each to-be-calibrated sensor to the reference sensor.

[0145] In this step, referring to FIG. 4, FIG. 4 shows a flowchart of how to locally optimize the to-be-solved transformation matrix of each to-be-calibrated sensor to the reference sensor according to the reference local perception data, the to-be-solved transformation matrix of each to-be-calibrated sensor to the reference sensor, and the auxiliary local perception data, to obtain a first transformation matrix of each to-be-calibrated sensor to the reference sensor, for each combination of the reference sensor and each to-be-calibrated sensor in the embodiment of the present application, which contains steps S401-S402:

[0146] In step S401, for each combination of the reference sensor and each to-be-calibrated sensor, the comprehensive perception deviation corresponding to the N motion sections is determined according to the reference local perception data, the to-be-solved transformation matrix of each to-be-calibrated sensor to the reference sensor, and the auxiliary local perception data; N is an integer greater than 1.

[0147] In this step, in the first alternative implementation, referring to FIG. 5, FIG. 5 shows a flowchart of how to determine the comprehensive perception deviation corresponding to the N motion sections according to the reference local perception data, the to-be-solved transformation matrix of each to-be-calibrated sensor to the reference sensor, and the auxiliary local perception data, for each combination of the reference sensor and each to-be-calibrated sensor in the embodiment of the present application, which contains steps S501-S502:

[0148] In step S501, for each combination of the reference sensor and each to-be-calibrated sensor, the comprehensive perception deviation corresponding to the N motion sections is determined according to the reference local perception data, the to-be-solved transformation matrix of each to-be-calibrated sensor to the reference sensor, and the auxiliary local perception data, and the local perception deviation of each to-be-calibrated sensor relative to the reference sensor on each motion section is determined.

[0149] In this step, after obtaining the reference local perception data of the reference sensor for each motion interval and the auxiliary local perception data of each to-be-calibrated sensor for each motion interval, the to-be-solved conversion matrix T of each to-be-calibrated sensor to the reference sensor can be obtained. Then, the local perception deviation of each to-be-calibrated sensor relative to the reference sensor on each motion interval can be determined according to the reference local perception data, the to-be-solved conversion matrix of each to-be-calibrated sensor to the reference sensor, and the auxiliary local perception data.

[0150] Further, for the case that the local perception data only includes a local map or a local trajectory, the local perception deviation of each to-be-calibrated sensor relative to the reference sensor on each motion interval can be calculated according to the calculation method of the reference local perception data-T*auxiliary local perception data.

[0151] For the case that the local perception data includes both a local map and a local trajectory, the local map deviation of each to-be-calibrated sensor relative to the reference sensor on each motion interval can be calculated according to the calculation method of the reference local perception map-T*auxiliary local perception map, and the local trajectory deviation of each to-be-calibrated sensor relative to the reference sensor on each motion interval can be calculated according to the calculation method of the reference local perception trajectory-T*auxiliary local perception trajectory.

[0152] In step S502, the comprehensive perception deviation is determined according to the N local perception deviations corresponding to the N motion intervals.

[0153] For the case that the local perception data only includes a local map or a local trajectory, the comprehensive perception deviation can be directly determined according to the accumulated values of the N local perception deviations corresponding to the N motion intervals, according to the related description of step S501.

[0154] For the case that the local perception data includes both a local map and a local trajectory, the accumulated values of the N local map deviations corresponding to the N motion intervals can be calculated, and the accumulated values of the N local trajectory deviations corresponding to the N motion intervals can be calculated. Then, the two types of accumulated values are added again to determine the comprehensive perception deviation.

[0155] In the second optional embodiment, the application further provides another method for determining the comprehensive perception deviation. Referring to FIG. 6, FIG. 6 shows a flowchart of how to determine the comprehensive perception deviation corresponding to the N motion intervals according to the reference local perception data, the to-be-solved conversion matrix of each to-be-calibrated sensor to the reference sensor, and the auxiliary local perception data for the combination of the reference sensor and each to-be-calibrated sensor in the embodiment of the application, which includes steps S601-S602.

[0156] In step S601, for each combination of the reference sensor and each sensor to be calibrated, M valid data groups with successful registration are selected from N data groups corresponding to N motion intervals.

[0157] In this step, in order to ensure the accuracy of the subsequent sensor calibration result, the N data groups corresponding to N motion intervals obtained in step S110 can be screened to select M (M is less than or equal to N) valid data groups from them, and the M valid data groups are associated with M valid motion intervals.

[0158] Specifically, for each combination of the reference sensor and each sensor to be calibrated, the reference local perception data and the auxiliary local perception data are included in each data group, and the valid reference local perception data and the valid auxiliary local perception data with successful registration are included in each valid data group. Thus, M data groups with successful registration can be selected from the N data groups.

[0159] In the first optional embodiment, for the case where the local perception data only includes a local map, the reference local map and the auxiliary local map are included in each data group. For example, refer to FIG. 7, which shows a flowchart of how to select M valid data groups from N data groups in an embodiment of the present application, including steps S701-S704:

[0160] In step S701, for each combination of the reference sensor and the sensor to be calibrated, a first conversion map is determined according to the product between the auxiliary local map for each motion interval and the estimated map conversion matrix.

[0161] In this step, for each combination of the reference sensor and the sensor to be calibrated, the first conversion map can be determined according to the product between the auxiliary local map for each motion interval i and the estimated map conversion matrix.

[0162] In step S702, a specified map deviation between the reference local map corresponding to each motion interval and the first conversion map is determined, a first norm corresponding to the specified map deviation is determined, and a first square value of the first norm is determined.

[0163] In this step, the specified map deviation between the reference local map corresponding to each motion interval and the first conversion map can be determined, that is, the specified map deviation

[0164] Then, the first norm corresponding to the specified map deviation can be determined, that is, the first norm ​​​and calculate a first square value of the first norm

[0165] In step S703, the value of the estimated map conversion matrix is optimized so that the first square value is minimized.

[0166] In this step, the value of the above-mentioned err i is minimized.

[0167] In step S704, if the first square value with the minimum value is less than a first preset error threshold, the auxiliary local map and the reference local map are determined as a valid map set in which registration is successful.

[0168] In this step, if the first square value with the minimum value i is less than the first preset error threshold, the auxiliary local map and the reference local map are determined as a valid map set in which registration is successful, that is, after registration is successful, the auxiliary local map can be referred to as a valid auxiliary local map, and the reference local map can be referred to as a valid reference local map.

[0169] In the second optional embodiment, for the case in which the local perception data only contains local trajectories, each data set contains a reference local trajectory and an auxiliary local trajectory. For example, reference can be made to FIG. 8, which shows a flowchart of another way of screening M valid data sets from N data sets in the embodiment of the present application, containing steps S801-S804.

[0170] In step S801, for each combination of a reference sensor and a sensor to be calibrated, a first converted trajectory is determined according to the product between the auxiliary local trajectory and the estimated trajectory conversion matrix for each motion interval.

[0171] In this step, for each combination of a reference sensor and a sensor to be calibrated, a first converted trajectory can be determined according to the product between the auxiliary local trajectory and the estimated trajectory conversion matrix for each motion interval i.

[0172] In step S802, a specified trajectory deviation between the reference local trajectory corresponding to each motion interval and the first converted trajectory is determined, a second norm corresponding to the specified trajectory deviation is determined, and a second square value of the second norm is determined.

[0173] In this step, the specified trajectory deviation between the reference local trajectory corresponding to each motion interval and the first converted trajectory can be determined, that is, the specified trajectory deviation​

[0174] Then, the second norm corresponding to the specified trajectory deviation can be determined. And calculate the second square value of the aforementioned second norm.

[0175] In step S803, the value of the estimated trajectory transformation matrix is ​​optimized to minimize the second square value.

[0176] In this step, the above can be optimized. The value of makes the aforementioned second squared value err i2 minimize.

[0177] In step S804, if the smallest second square value is less than the second preset error threshold, then the auxiliary local trajectory and the reference local trajectory are determined as a valid trajectory group that has been successfully registered.

[0178] In this step, if the smallest second squared value is err i2 If the error is less than the second preset error threshold, then the above-mentioned auxiliary local trajectory and the above-mentioned reference local trajectory are determined as a valid trajectory group that has been successfully registered. That is, after successful registration, the above-mentioned auxiliary local trajectory can be called a valid auxiliary local trajectory, and the above-mentioned reference local trajectory can be called a valid reference local trajectory.

[0179] In a third optional implementation, for cases where the local sensing data includes both local maps and local trajectories, each data group includes a map group and a trajectory group. Each map group consists of a reference local map and an auxiliary local map corresponding to each motion interval, and each trajectory group consists of a reference local trajectory and an auxiliary local trajectory corresponding to each motion interval. Therefore, referring to Figure 9, Figure 9 shows a flowchart illustrating another method for selecting M valid data groups from N data groups in this application embodiment. Specifically, it illustrates how, for a combination of a reference sensor and each sensor to be calibrated, M successfully registered valid map groups are selected from N map groups corresponding to N motion intervals, and how M successfully registered valid trajectory groups are selected from N trajectory groups corresponding to N motion intervals, including steps S901-S906:

[0180] In step S901, for each combination of reference sensor and sensor to be calibrated, a first transformed map is determined based on the product of the auxiliary local map for each motion range and the estimated map transformation matrix, and a first transformed trajectory is determined based on the product of the auxiliary local trajectory for each motion range and the estimated trajectory transformation matrix.

[0181] In step S902, a specified map deviation between the reference local map corresponding to each motion section and the first conversion map is determined, a first norm corresponding to the specified map deviation is determined, and a first square value of the first norm is determined.

[0182] In step S903, the value of the estimated map conversion matrix is optimized so as to minimize the first square value.

[0183] In step S904, a specified trajectory deviation between the reference local trajectory corresponding to each motion section and the first conversion trajectory is determined, a second norm corresponding to the specified trajectory deviation is determined, and a second square value of the second norm is determined.

[0184] In step S905, the value of the estimated trajectory conversion matrix is optimized so as to minimize the second square value.

[0185] In step S906, if the first square value with the smallest value is smaller than a first preset error threshold and the second square value with the smallest value is smaller than a second preset error threshold, the auxiliary local map and the reference local map are determined as a valid map group with successful registration, and the auxiliary local trajectory and the reference local trajectory are determined as a valid trajectory group with successful registration.

[0186] Thus, the valid data group comprises the valid map group and the valid trajectory group, the valid map group comprises the valid reference local map and the valid auxiliary local map with successful registration, and the valid trajectory group comprises the valid reference local trajectory and the valid auxiliary local trajectory with successful registration.

[0187] It should be noted that the specific explanations of steps S901-S906 can refer to the corresponding steps in FIG. 7 or FIG. 8, which will not be described herein.

[0188] After the M valid data groups are selected from the N data groups, in step S602, the local perception deviation of each to-be-calibrated sensor relative to the reference sensor on each valid motion section can be determined according to the valid reference local perception data, the to-be-solved conversion matrix of each to-be-calibrated sensor to the reference sensor, and the valid auxiliary local perception data, and the comprehensive perception deviation can be determined according to the M local perception deviations corresponding to the M valid motion sections.

[0189] In the first optional embodiment, after the M valid data groups are selected, the valid reference local perception data can comprise the valid reference local map, the valid auxiliary local perception data can comprise the valid auxiliary local map, and the local perception deviation can comprise the local map deviation, so that the local map deviation of each to-be-calibrated sensor relative to the reference sensor on each valid motion section can be determined according to the valid reference local map, the to-be-solved conversion matrix of each to-be-calibrated sensor to the reference sensor, and the valid auxiliary local map.

[0190] Specifically, the local map deviation of each sensor to be calibrated relative to the reference sensor on each valid motion interval can be determined based on the following formula 1 map :

[0191]

[0192] wherein, l map represents the local map deviation of each sensor to be calibrated relative to the reference sensor on each valid motion interval, for each combination of reference sensor and sensor to be calibrated, represents the valid reference local map corresponding to the reference sensor for the ith valid motion interval, represents the valid auxiliary local map corresponding to the sensor to be calibrated for the ith valid motion interval, and T represents the to-be-solved transformation matrix of the sensor to be calibrated to the reference sensor.

[0193] Then, the above comprehensive perception deviation l a1 :

[0194]

[0195] wherein, l a1 represents the above comprehensive perception deviation.

[0196] In the second optional embodiment, the valid reference local perception data can include a valid reference local trajectory, the valid auxiliary local perception data includes a valid auxiliary local trajectory, and the local perception deviation includes a local trajectory deviation, so that the local trajectory deviation of each sensor to be calibrated relative to the reference sensor on each valid motion interval can be determined according to the valid reference local perception trajectory, the to-be-solved transformation matrix of each sensor to be calibrated to the reference sensor, and the valid auxiliary local perception estimation.

[0197] Specifically, the local trajectory deviation of each sensor to be calibrated relative to the reference sensor on each valid motion interval can be determined based on the following formula 3 traj :

[0198]

[0199] wherein, for each combination of reference sensor and sensor to be calibrated, l traj represents the local trajectory deviation of each sensor to be calibrated relative to the reference sensor on each valid motion interval, represents the valid reference local trajectory corresponding to the reference sensor for the ith valid motion interval, T represents the effective auxiliary local trajectory corresponding to the sensor to be calibrated for the i th effective motion interval, and T represents the to-be-solved conversion matrix of the sensor to be calibrated to the reference sensor.

[0200] Then, the above comprehensive perception deviation l can be determined based on the following formula 4 aa1 :

[0201]

[0202] wherein, l a1 represents the comprehensive perception deviation.

[0203] In the third optional implementation, each data group can include a map group and a trajectory group, each map group is composed of a reference local map and an auxiliary local map corresponding to each motion interval, each trajectory group is composed of a reference local trajectory and an auxiliary local trajectory corresponding to each motion interval, the local perception deviation can include a local map deviation and a local trajectory deviation, so that the local map deviation of each sensor to be calibrated relative to the reference sensor on each effective motion interval can be determined according to the above formula 1; and the local trajectory deviation of each sensor to be calibrated relative to the reference sensor on each effective motion interval can be determined according to the above formula 3.

[0204] Then, the comprehensive perception deviation l can be determined based on the following formula 5 a1 :

[0205]

[0206] wherein, l a1 represents the comprehensive perception deviation, represents the cumulative value of the M local map deviations corresponding to the M effective motion intervals, represents the cumulative value of the M local trajectory deviations corresponding to the M effective motion intervals.

[0207] In step S402, the value of the to-be-solved conversion matrix is optimized to minimize the comprehensive perception deviation, and a first conversion matrix of each sensor to be calibrated to the reference sensor is obtained.

[0208] In this step, the value of the to-be-solved conversion matrix can be continuously optimized to minimize the comprehensive perception deviation, and a first conversion matrix of each sensor to be calibrated to the reference sensor is obtained.

[0209] In the first optional implementation, the T in the above formula 2 can be continuously optimized to minimize the comprehensive perception deviation in formula 2, so as to obtain a first conversion matrix of each sensor to be calibrated to the reference sensor.

[0210] In the second optional embodiment, T in the above formula 4 can be constantly optimized so that the comprehensive perception deviation in the formula 4 is minimum, to obtain the first conversion matrix of each sensor to be calibrated to the reference sensor.

[0211] In the third optional embodiment, T in the above formula 5 can be constantly optimized so that the comprehensive perception deviation in the formula 5 is minimum, to obtain the first conversion matrix of each sensor to be calibrated to the reference sensor.

[0212] In step S130, a second conversion matrix of the reference sensor to the odometer coordinate system is obtained, and a candidate conversion matrix of each sensor to be calibrated to the odometer coordinate system is determined according to the first conversion matrix combined with the second conversion matrix.

[0213] In this step, referring to FIG. 10, FIG. 10 shows a flowchart of how to obtain the second conversion matrix of the reference sensor to the odometer coordinate system in the embodiment of the application, which includes steps S1001-S1003:

[0214] In step S1001, the odometer trajectory of the cleaning robot is obtained for each valid motion interval.

[0215] In this step, the odometer trajectory of the robot can be obtained for each valid motion interval. For example, the odometer trajectory can be obtained by the odometer of the wheel of the robot, or can be determined by the inertial navigation data loaded on the robot, or can be determined by a combination of multiple data.

[0216] In step S1002, each standard local trajectory deviation is determined according to the odometer trajectory, each valid reference local trajectory, and the second conversion matrix to be solved of the reference sensor to the odometer coordinate system, and the comprehensive trajectory deviation corresponding to the M standard local trajectory deviations is obtained.

[0217] In this step, each standard local trajectory deviation l i can be determined based on the following formula 6: straj :

[0218]

[0219] wherein for the i th valid motion interval, l i represents each standard local trajectory deviation, straj represents the odometer trajectory of the cleaning robot, T i represents the first conversion matrix of the i th sensor to be calibrated to the reference sensor, 基准2里程计 represents the second conversion matrix to be solved of the reference sensor to the odometer coordinate system, and represents the valid reference local trajectory.

[0220] Then, the comprehensive trajectory deviation l corresponding to the M standard local trajectory deviations can be determined based on the following formula 7: a2 ​:

[0221]

[0222] In step S1003, the value of the second to-be-solved conversion matrix of the reference sensor to the odometry coordinate system is optimized so as to minimize the comprehensive trajectory deviation, and the second conversion matrix of the reference sensor to the odometry coordinate system is solved.

[0223] In this step, the value of T 基准2里程计 may be continuously optimized so as to minimize the comprehensive trajectory deviation, and the second conversion matrix of the reference sensor to the odometry coordinate system is solved.

[0224] Then, the first conversion matrix and the second conversion matrix can be multiplied to obtain a candidate conversion matrix of each to-be-calibrated sensor to the odometry coordinate system.

[0225] After obtaining the candidate conversion matrix of each to-be-calibrated sensor to the odometry coordinate system, in order to avoid the problem that the precision of the candidate conversion matrix is not enough due to system error, the application can also perform global optimization on each candidate conversion matrix based on step S140 to obtain a final conversion matrix of each to-be-calibrated sensor to the odometry coordinate system.

[0226] Next, referring to FIG. 1, in step S140, the global perception data of each to-be-calibrated sensor during the movement of the cleaning robot is used to perform global optimization on the candidate conversion matrix of each to-be-calibrated sensor to the odometry coordinate system, and a final conversion matrix of each to-be-calibrated sensor to the odometry coordinate system is obtained.

[0227] In this step, the global perception data of each to-be-calibrated sensor during the movement of the cleaning robot can include an auxiliary global trajectory of each to-be-calibrated sensor during the movement of the cleaning robot. Optionally, the auxiliary global trajectory can be obtained by splicing N auxiliary local trajectories corresponding to the N movement intervals, or can be obtained by re-traversing the cleaning robot in the environment once. The actual situation can be self-determined, and the application does not make special limitations thereto.

[0228] Referring to FIG. 11, FIG. 11 shows a flowchart of how to perform global optimization on the candidate conversion matrix of each to-be-calibrated sensor to the odometry coordinate system to obtain a final conversion matrix of each to-be-calibrated sensor to the odometry coordinate system in the embodiment of the application, which includes steps S1101-S1105.

[0229] In step S1101, an identification number corresponding to each to-be-calibrated sensor in the K to-be-calibrated sensors is configured.

[0230] In this step, the identification number corresponding to each of the K to-be-calibrated sensors can be configured. The K identification numbers can be in an increasing order, for example, 1 to K. In the following embodiment, K is taken as 5 for illustration.

[0231] In step S1102, when the maximum identification number is K, the xth to-be-calibrated sensor is selected from the K to-be-calibrated sensors in turn, and the x+1th sensor to the Kth sensor are combined with the xth to-be-calibrated sensor respectively to obtain K-x sensor combinations associated with the xth to-be-calibrated sensor; wherein, 1≤x≤K-1.

[0232] In this step, one xth to-be-calibrated sensor can be selected from the K to-be-calibrated sensors each time, and the value of x can be from 1 to K-1. Thus, when x is 1, the 1st sensor can be combined with the 2nd sensor, the 3rd sensor, the 4th sensor, and the 5th sensor respectively to obtain 4 sensor combinations (i.e., 1-2, 1-3, 1-4, and 1-5); when x is 2, the 2nd sensor can be combined with the 3rd sensor, the 4th sensor, and the 5th sensor respectively to obtain 3 sensor combinations (i.e., 2-3, 2-4, and 2-5); when x is 3, the 3rd sensor can be combined with the 4th sensor and the 5th sensor respectively to obtain 2 sensor combinations (i.e., 3-4 and 3-5); and when x is 4, the 4th sensor can be combined with the 5th sensor to obtain 1 sensor combination (i.e., 4-5).

[0233] Each sensor combination contains the xth to-be-calibrated sensor and a specified to-be-calibrated sensor. For example, in the case where x is 1, taking the sensor combination 1-2 as an example, the 2nd to-be-calibrated sensor is the specified to-be-calibrated sensor.

[0234] In step S1103, for each sensor combination, an error expression between the xth to-be-calibrated sensor and the specified to-be-calibrated sensor is determined according to the auxiliary global trajectory associated with the xth to-be-calibrated sensor, the auxiliary global trajectory associated with the specified to-be-calibrated sensor, the candidate conversion matrix of the xth to-be-calibrated sensor to the odometer coordinate system, and the candidate conversion matrix of the specified to-be-calibrated sensor to the odometer coordinate system.

[0235] In this step, the error expression e between the xth to-be-calibrated sensor and the specified to-be-calibrated sensor can be determined based on the following formula 8. x,y :

[0236]

[0237] wherein, e x,y represents the error expression between the xth to-be-calibrated sensor and the specified to-be-calibrated sensor, Traj xTraj represents an auxiliary global trajectory associated with the xth sensor to be calibrated y Traj represents an auxiliary global trajectory associated with the xth sensor to be calibrated T represents an inverse matrix of the candidate transformation matrix of the xth sensor to be calibrated to the odometry coordinate system y Traj represents an auxiliary global trajectory associated with the xth sensor to be calibrated

[0238] In step S1104, the overall error expression between the xth sensor to be calibrated and the K-xth sensor to be calibrated is determined according to the sum of the K-x error expressions.

[0239] In this step, the overall error expression between the xth sensor to be calibrated and the K-xth sensor to be calibrated can be determined based on the following formula 9:

[0240]

[0241] wherein E represents the overall error expression between the xth sensor to be calibrated and the K-xth sensor to be calibrated.

[0242] In step S1105, the value of the candidate transformation matrix of the xth sensor to be calibrated to the odometry coordinate system is optimized so that the value of the overall error expression is minimized, and the final transformation matrix of the xth sensor to be calibrated to the odometry coordinate system is obtained.

[0243] In this step, the value of the above-mentioned candidate transformation matrix T of the xth sensor to be calibrated to the odometry coordinate system x can be continuously optimized (equivalent to optimizing the value of ), so that the value of the overall error expression is minimized, and the final transformation matrix of the xth sensor to be calibrated to the odometry coordinate system (i.e., the extrinsic parameter of each sensor to be calibrated) is obtained.

[0244] Based on the final transformation matrix, the positions or postures of the various sensors to be calibrated on the cleaning robot can be adjusted to ensure that the data of different sensors are more highly synchronized and consistent in time and space, and the effective fusion and coordination of the entire system are achieved.

[0245] Referring to FIG. 12, FIG. 12 shows the overall flow diagram of how to calibrate the candidate transformation matrix of each sensor to be calibrated to the odometry coordinate system through local perception data in the embodiment of the present application, which includes steps S1201-S1210:

[0246] In step S1201, local calibration starts;

[0247] In step S1202, the reference depth sensor locally maps, and outputs the reference local map and the reference local trajectory;

[0248] In step S1203, the depth sensor to be calibrated is locally mapped, and an auxiliary local map and an auxiliary local trajectory are output;

[0249] In step S1204, it is determined whether the map registration is successful; if not, the data is discarded in step S1205;

[0250] If yes, it is determined whether the trajectory registration is successful in step S1206; if not, the data is discarded in step S1207;

[0251] If both the map and the trajectory are successfully registered, the map data and the trajectory data that are successfully registered are added to the data set in step S1208;

[0252] In step S1209, the comprehensive perception bias corresponding to the data in the data set is calculated, and T (the conversion matrix to be solved of the sensor to be calibrated to the reference sensor) is optimized so as to minimize the comprehensive perception bias;

[0253] In step S1210, the candidate conversion matrix is obtained by successful optimization.

[0254] Referring to FIG. 13, FIG. 13 shows the overall flow diagram of how to globally optimize each candidate conversion matrix to obtain the final conversion matrix of each sensor to be calibrated to the coordinate system of the odometer in the embodiment of the application, which includes steps S1301-S1306:

[0255] In step S1301, the initial value of local optimization, i.e., the value of the candidate conversion matrix, is obtained;

[0256] In step S1302, the global calibration stage is entered;

[0257] In step S1303, each sensor to be calibrated is independently mapped;

[0258] In step S1304, the K sensors to be calibrated are paired two by two;

[0259] In step S1305, global optimization is performed, and the final conversion matrix of each sensor to be calibrated to the coordinate system of the odometer is output;

[0260] In step S1306, the global calibration ends.

[0261] Based on the above technical solution, the application provides a scheme that can online calibrate the external parameters of multiple depth sensors in an ordinary home scene, which has the advantages of low scene requirement, simple action, high precision, high reliability, etc., and can meet the online calibration requirements of the depth sensor without the aid of high-precision equipment.

[0262] The application also provides a cleaning robot, wherein K+1 sensors are arranged on the cleaning robot, the K+1 sensors include a reference sensor and K sensors to be calibrated, K is an integer greater than 1, and FIG. 14 shows a structural schematic diagram of the cleaning robot in an exemplary embodiment of the application; as shown in FIG. 14, the cleaning robot 1400 can include a perception module 1410 and an extrinsic parameter calibration module 1420. Wherein:

[0263] The perception module 1410 is configured to acquire reference local perception data of the reference sensor for each motion interval in the motion process of the cleaning robot, and acquire auxiliary local perception data of each sensor to be calibrated for the each motion interval;

[0264] The extrinsic parameter calibration module 1420 is configured to, for each combination of the reference sensor and a sensor to be calibrated, perform local optimization on a to-be-solved conversion matrix of each sensor to be calibrated to the reference sensor according to the reference local perception data, the to-be-solved conversion matrix of each sensor to be calibrated to the reference sensor, and the auxiliary local perception data, to obtain a first conversion matrix of each sensor to be calibrated to the reference sensor;

[0265] acquire a second conversion matrix of the reference sensor to an odometer coordinate system, and determine a candidate conversion matrix of each sensor to be calibrated to the odometer coordinate system according to the first conversion matrix and the second conversion matrix;

[0266] perform global optimization on the candidate conversion matrix of each sensor to be calibrated to the odometer coordinate system according to global perception data of each sensor to be calibrated in the motion process of the cleaning robot, to obtain a final conversion matrix of each sensor to be calibrated to the odometer coordinate system.

[0267] In the exemplary embodiment of the application, the extrinsic parameter calibration module 1420, for each combination of the reference sensor and a sensor to be calibrated, performs local optimization on a to-be-solved conversion matrix of each sensor to be calibrated to the reference sensor according to the reference local perception data, the to-be-solved conversion matrix of each sensor to be calibrated to the reference sensor, and the auxiliary local perception data, to obtain a first conversion matrix of each sensor to be calibrated to the reference sensor, including:

[0268] for each combination of the reference sensor and a sensor to be calibrated, determining a comprehensive perception deviation corresponding to N motion intervals according to the reference local perception data, the to-be-solved conversion matrix of each sensor to be calibrated to the reference sensor, and the auxiliary local perception data; N is an integer greater than 1;

[0269] Optimizing the value of the to-be-solved transformation matrix so that the comprehensive perception deviation is minimized, to obtain a first transformation matrix of each of the to-be-calibrated sensors to the reference sensor.

[0270] In the example embodiments of the present application, the extrinsic parameter calibration module 1420 determines, for each combination of the reference sensor and each to-be-calibrated sensor, a comprehensive perception deviation corresponding to N motion intervals according to the reference local perception data, the to-be-solved transformation matrix of each of the to-be-calibrated sensors to the reference sensor, and the auxiliary local perception data, including:

[0271] For each combination of the reference sensor and each to-be-calibrated sensor, the comprehensive perception deviation corresponding to N motion intervals is determined according to the reference local perception data, the to-be-solved transformation matrix of each of the to-be-calibrated sensors to the reference sensor, and the auxiliary local perception data, and the local perception deviation of each of the to-be-calibrated sensors relative to the reference sensor in each of the motion intervals is determined.

[0272] The comprehensive perception deviation is determined according to N local perception deviations corresponding to N motion intervals.

[0273] In the example embodiments of the present application, the extrinsic parameter calibration module 1420 determines, for each combination of the reference sensor and each to-be-calibrated sensor, a comprehensive perception deviation corresponding to N motion intervals according to the reference local perception data, the to-be-solved transformation matrix of each of the to-be-calibrated sensors to the reference sensor, and the auxiliary local perception data, including:

[0274] For each combination of the reference sensor and each of the to-be-calibrated sensors, M valid data groups with successful registration are selected from N data groups corresponding to N motion intervals; M is less than or equal to N; the M valid data groups are associated with M valid motion intervals; each of the data groups contains the reference local perception data and the auxiliary local perception data, and each of the valid data groups contains valid reference local perception data and valid auxiliary local perception data with successful registration.

[0275] According to the valid reference local perception data, the to-be-solved transformation matrix of each of the to-be-calibrated sensors to the reference sensor, and the valid auxiliary local perception data, the local perception deviation of each of the to-be-calibrated sensors relative to the reference sensor in each of the valid motion intervals is determined.

[0276] The comprehensive perception deviation is determined according to M local perception deviations corresponding to M valid motion intervals.

[0277] In an example embodiment of the present application, the effective reference local perception data comprises an effective reference local map, the effective auxiliary local perception data comprises an effective auxiliary local map, and the local perception deviation comprises a local map deviation.

[0278] The extrinsic calibration module 1420 determines, for each combination of the reference sensor and each of the sensors to be calibrated, a local perception deviation of each of the sensors to be calibrated relative to the reference sensor over each of the effective motion intervals, according to the effective reference local perception data, a to-be-solved transformation matrix of each of the sensors to be calibrated to the reference sensor, and the effective auxiliary local perception data, including:

[0279] The extrinsic calibration module 1420 determines, for each combination of the reference sensor and each of the sensors to be calibrated, a local perception deviation of each of the sensors to be calibrated relative to the reference sensor over each of the effective motion intervals, according to the effective reference local perception data, a to-be-solved transformation matrix of each of the sensors to be calibrated to the reference sensor, and the effective auxiliary local perception data, including:

[0280] In an example embodiment of the present application, the extrinsic calibration module 1420 determines the local map deviation of each of the sensors to be calibrated relative to the reference sensor over each of the effective motion intervals based on the following formula:

[0281]

[0282] wherein, l map represents the local map deviation of each of the sensors to be calibrated relative to the reference sensor over each of the effective motion intervals, for each combination of the reference sensor and the sensors to be calibrated, represents an effective reference local map corresponding to the reference sensor for the i-th effective motion interval, represents an effective auxiliary local map corresponding to the sensor to be calibrated for the i-th effective motion interval, and T represents a to-be-solved transformation matrix of the sensor to be calibrated to the reference sensor.

[0283] In an example embodiment of the present application, the extrinsic calibration module 1420 determines the comprehensive perception deviation according to M local perception deviations corresponding to M effective motion intervals, including:

[0284] The extrinsic calibration module 1420 determines the comprehensive perception deviation based on the following formula:

[0285]

[0286] wherein, l a1 represents the comprehensive perception deviation.

[0287] In an example embodiment of the present application, the effective reference local perception data comprises effective reference local trajectories, the effective auxiliary local perception data comprises effective auxiliary local trajectories, and the local perception deviation comprises a local trajectory deviation;

[0288] The extrinsic calibration module 1420 determines, for each combination of the reference sensor and each of the sensors to be calibrated, a local perception deviation of each of the sensors to be calibrated relative to the reference sensor over each of the effective motion intervals, according to the effective reference local perception data, the to-be-solved transformation matrix of each of the sensors to be calibrated to the reference sensor, and the effective auxiliary local perception data, comprising:

[0289] The extrinsic calibration module 1420 determines, for each combination of the reference sensor and each of the sensors to be calibrated, a local trajectory deviation of each of the sensors to be calibrated relative to the reference sensor over each of the effective motion intervals, according to the effective reference local perception trajectories, the to-be-solved transformation matrix of each of the sensors to be calibrated to the reference sensor, and the effective auxiliary local perception estimation.

[0290] In an example embodiment of the present application, the extrinsic calibration module 1420 determines the local trajectory deviation of each of the sensors to be calibrated relative to the reference sensor over each of the effective motion intervals based on the following formula:

[0291]

[0292] wherein, for each combination of a reference sensor and a sensor to be calibrated, l traj represents the local trajectory deviation of each of the sensors to be calibrated relative to the reference sensor over each of the effective motion intervals, represents the effective reference local trajectory corresponding to the reference sensor for the i-th effective motion interval, represents the effective auxiliary local trajectory corresponding to the sensor to be calibrated for the i-th effective motion interval, and T represents the to-be-solved transformation matrix of the sensor to be calibrated to the reference sensor.

[0293] In an example embodiment of the present application, the extrinsic calibration module 1420 determines the comprehensive perception deviation according to M local perception deviations corresponding to M effective motion intervals, comprising:

[0294] The extrinsic calibration module 1420 determines the comprehensive perception deviation based on the following formula:

[0295]

[0296] wherein, for each combination of a reference sensor and a sensor to be calibrated, l a1 represents the comprehensive perception deviation.

[0297] In the example embodiments of the present application, each of the data groups comprises a map group and a trajectory group, each of the map groups consists of a reference local map and an auxiliary local map corresponding to each of the motion intervals, and each of the trajectory groups consists of a reference local trajectory and an auxiliary local trajectory corresponding to each of the motion intervals;

[0298] The extrinsic calibration module 1420 screens M valid data groups with successful registration from N data groups corresponding to N motion intervals, comprising:

[0299] For each combination of the reference sensor and the to-be-calibrated sensor, M valid map groups with successful registration are screened from N map groups corresponding to N motion intervals; each of the map groups consists of a reference local map and an auxiliary local map, and each of the valid map groups consists of a valid reference local map and a valid auxiliary local map with successful registration;

[0300] and M valid trajectory groups with successful registration are screened from N trajectory groups corresponding to N motion intervals; each of the trajectory groups consists of a reference local trajectory and an auxiliary local trajectory, and each of the valid trajectory groups consists of a valid reference local trajectory and a valid auxiliary local trajectory with successful registration.

[0301] In the example embodiments of the present application, for each combination of the reference sensor and the to-be-calibrated sensor, each of the valid map groups and each of the valid trajectory groups are determined by:

[0302] For each combination of a reference sensor and a to-be-calibrated sensor, a first converted map is determined according to the product between the auxiliary local map corresponding to each motion interval and the estimated map conversion matrix, and a first converted trajectory is determined according to the product between the auxiliary local trajectory corresponding to each motion interval and the estimated trajectory conversion matrix;

[0303] A specified map deviation between the reference local map corresponding to each motion interval and the first converted map is determined, a first norm corresponding to the specified map deviation and a first square value of the first norm are determined;

[0304] The value of the estimated map conversion matrix is optimized so that the first square value is minimized;

[0305] A specified trajectory deviation between the reference local trajectory corresponding to each motion interval and the first converted trajectory is determined, a second norm corresponding to the specified trajectory deviation and a second square value of the second norm are determined;

[0306] The value of the estimated trajectory conversion matrix is optimized so that the second square value is minimized;

[0307] If the first square value with the smallest value is less than a first preset error threshold and the second square value with the smallest value is less than a second preset error threshold, the auxiliary local map and the reference local map are determined as an effective map group with successful registration, and the auxiliary local trajectory and the reference local trajectory are determined as an effective trajectory group with successful registration.

[0308] In the example embodiments of the present application, the local perception deviation includes a local map deviation and a local trajectory deviation;

[0309] The extrinsic parameter calibration module 1420 determines, for each combination of the reference sensor and each of the to-be-calibrated sensors, a local perception deviation of each of the to-be-calibrated sensors relative to the reference sensor on each of the effective motion intervals, according to the effective reference local perception data, the to-be-solved transformation matrix of each of the to-be-calibrated sensors to the reference sensor, and the effective auxiliary local perception data, including:

[0310] According to the effective reference local map, the to-be-solved transformation matrix of each of the to-be-calibrated sensors to the reference sensor, and the effective auxiliary local map, a local map deviation of each of the to-be-calibrated sensors relative to the reference sensor on each of the effective motion intervals is determined.

[0311] According to the effective reference local perception trajectory, the to-be-solved transformation matrix of each of the to-be-calibrated sensors to the reference sensor, and the effective auxiliary local perception estimate, a local trajectory deviation of each of the to-be-calibrated sensors relative to the reference sensor on each of the effective motion intervals is determined.

[0312] In the example embodiments of the present application, the extrinsic parameter calibration module 1420 determines the comprehensive perception deviation l a1 , according to M local perception deviations corresponding to M effective motion intervals.

[0313] The comprehensive perception deviation is determined based on the following formula:

[0314]

[0315] wherein l a1 represents the comprehensive perception deviation, represents an accumulated value of M local map deviations corresponding to M effective motion intervals, represents an accumulated value of M local trajectory deviations corresponding to M effective motion intervals.

[0316] In the example embodiment of the present application, the extrinsic parameter calibration module 1420 obtains the second conversion matrix of the reference sensor to the odometer coordinate system, comprising:

[0317] For each of the effective motion intervals, the odometer trajectory of the cleaning robot is obtained;

[0318] According to the odometer trajectory, each of the effective reference local trajectory and the second to-be-solved conversion matrix of the reference sensor to the odometer coordinate system, the standard local trajectory deviation of each is determined, and the comprehensive trajectory deviation corresponding to the M standard local trajectory deviations is obtained;

[0319] The value of the second to-be-solved conversion matrix of the reference sensor to the odometer coordinate system is optimized to minimize the comprehensive trajectory deviation, and the second conversion matrix of the reference sensor to the odometer coordinate system is solved.

[0320] In the example embodiment of the present application, the extrinsic parameter calibration module 1420 determines the standard local trajectory deviation of each according to the odometer trajectory, each of the effective reference local trajectory and the second to-be-solved conversion matrix of the reference sensor to the odometer coordinate system, comprising:

[0321] The standard local trajectory deviation of each is determined based on the following formula:

[0322]

[0323] Wherein, for the i-th effective motion interval, l straj represents the standard local trajectory deviation of each, represents the odometer trajectory of the cleaning robot, T 基准2里程计 represents the second to-be-solved conversion matrix of the reference sensor to the odometer coordinate system, represents the effective reference local trajectory.

[0324] In the example embodiment of the present application, the extrinsic parameter calibration module 1420 determines the candidate conversion matrix of each to-be-calibrated sensor to the odometer coordinate system according to the first conversion matrix combined with the second conversion matrix, comprising:

[0325] The first conversion matrix is multiplied by the second conversion matrix to obtain the candidate conversion matrix of each to-be-calibrated sensor to the odometer coordinate system.

[0326] In the example embodiment of the present application, the global perception data of each to-be-calibrated sensor during the motion of the cleaning robot includes an auxiliary global trajectory of each to-be-calibrated sensor during the motion of the cleaning robot;

[0327] The global optimization of the candidate transformation matrix of each to-be-calibrated sensor to the odometer coordinate system according to the global perception data of each to-be-calibrated sensor during the motion of the cleaning robot obtains a final transformation matrix of each to-be-calibrated sensor to the odometer coordinate system, and the global optimization comprises:

[0328] The identification number corresponding to each to-be-calibrated sensor in the K to-be-calibrated sensors is configured, and the K identification numbers are in an increasing order.

[0329] When the maximum identification number is K, the xth to-be-calibrated sensor is sequentially selected from the K to-be-calibrated sensors, the x+1th sensor to the Kth sensor are combined with the xth to-be-calibrated sensor respectively, and K-x sensor combinations associated with the xth to-be-calibrated sensor are obtained. Each sensor combination comprises the xth to-be-calibrated sensor and a specified to-be-calibrated sensor, and 1≤x≤K-1.

[0330] For each sensor combination, an error expression between the xth to-be-calibrated sensor and the specified to-be-calibrated sensor is determined according to the auxiliary global trajectory associated with the xth to-be-calibrated sensor, the auxiliary global trajectory associated with the specified to-be-calibrated sensor, the candidate transformation matrix of the xth to-be-calibrated sensor to the odometer coordinate system, and the candidate transformation matrix of the specified to-be-calibrated sensor to the odometer coordinate system.

[0331] An overall error expression between the xth to-be-calibrated sensor and K-x specified to-be-calibrated sensors is determined according to the sum of K-x error expressions.

[0332] The value of the candidate transformation matrix of the xth to-be-calibrated sensor to the odometer coordinate system is optimized so that the value of the overall error expression is minimized, and a final transformation matrix of the xth to-be-calibrated sensor to the odometer coordinate system is obtained.

[0333] In the exemplary embodiments of the present application, the extrinsic parameter calibration module 1420 determines, for each sensor combination, an error expression between the xth to-be-calibrated sensor and the specified to-be-calibrated sensor according to the auxiliary global trajectory associated with the xth to-be-calibrated sensor, the auxiliary global trajectory associated with the specified to-be-calibrated sensor, the candidate transformation matrix of the xth to-be-calibrated sensor to the odometer coordinate system, and the candidate transformation matrix of the specified to-be-calibrated sensor to the odometer coordinate system, and the error expression comprises:

[0334] The error expression between the xth to-be-calibrated sensor and the specified to-be-calibrated sensor is determined based on the following formula:

[0335]

[0336] wherein, e x,y represents an error expression between the xth sensor to be calibrated and the designated sensor to be calibrated, Traj x represents an auxiliary global trajectory associated with the xth sensor to be calibrated, Traj y represents an auxiliary global trajectory associated with the designated sensor to be calibrated, represents an inverse matrix of a candidate transformation matrix of the xth sensor to be calibrated to the odometry coordinate system, T y represents a candidate transformation matrix of the designated sensor to be calibrated to the odometry coordinate system.

[0337] In an example embodiment of the present application, the extrinsic calibration module 1420 determines an overall error expression between the xth sensor to be calibrated and K-x designated sensors to be calibrated according to a sum of K-x error expressions, including:

[0338] determines an overall error expression between the xth sensor to be calibrated and K-x designated sensors to be calibrated based on the following formula:

[0339]

[0340] wherein, E represents an overall error expression between the xth sensor to be calibrated and K-x designated sensors to be calibrated.

[0341] In an example embodiment of the present application, the motion interval is obtained according to a motion distance or a motion angle of the cleaning robot.

[0342] The specific details of the modules in the above cleaning robot have been described in detail in the method for calibrating the extrinsic parameters of the corresponding sensor, and thus will not be described here again.

[0343] The present application also provides a cleaning system, which includes K+1 sensors, the K+1 sensors including a reference sensor and K sensors to be calibrated, K being an integer greater than 1, and FIG. 15 shows a structural schematic diagram of the cleaning system in an example embodiment of the present application; as shown in FIG. 15, the cleaning system 1500 can include a data collector 1510 and an extrinsic calibration processor 1520. Wherein:

[0344] The data collector 1510 is configured to acquire reference local perception data of the reference sensor for each motion interval during a motion process of the cleaning system, and acquire auxiliary local perception data of each sensor to be calibrated for the each motion interval;

[0345] The extrinsic parameter calibration processor 1520 is configured to, for each combination of the reference sensor and each to-be-calibrated sensor, perform local optimization on a to-be-solved transformation matrix of each to-be-calibrated sensor to the reference sensor according to the reference local perception data, the to-be-solved transformation matrix of each to-be-calibrated sensor to the reference sensor, and the auxiliary local perception data, to obtain a first transformation matrix of each to-be-calibrated sensor to the reference sensor.

[0346] The extrinsic parameter calibration processor 1520 is configured to, for each combination of the reference sensor and each to-be-calibrated sensor, perform local optimization on a to-be-solved transformation matrix of each to-be-calibrated sensor to the reference sensor according to the reference local perception data, the to-be-solved transformation matrix of each to-be-calibrated sensor to the reference sensor, and the auxiliary local perception data, to obtain a first transformation matrix of each to-be-calibrated sensor to the reference sensor.

[0347] The extrinsic parameter calibration processor 1520 is configured to, for each combination of the reference sensor and each to-be-calibrated sensor, perform local optimization on a to-be-solved transformation matrix of each to-be-calibrated sensor to the reference sensor according to the reference local perception data, the to-be-solved transformation matrix of each to-be-calibrated sensor to the reference sensor, and the auxiliary local perception data, to obtain a first transformation matrix of each to-be-calibrated sensor to the reference sensor.

[0348] In an example embodiment of the present application, the extrinsic parameter calibration processor 1520 is configured to, for each combination of the reference sensor and each to-be-calibrated sensor, perform local optimization on a to-be-solved transformation matrix of each to-be-calibrated sensor to the reference sensor according to the reference local perception data, the to-be-solved transformation matrix of each to-be-calibrated sensor to the reference sensor, and the auxiliary local perception data, to obtain a first transformation matrix of each to-be-calibrated sensor to the reference sensor, including:

[0349] In an example embodiment of the present application, the extrinsic parameter calibration processor 1520 is configured to, for each combination of the reference sensor and each to-be-calibrated sensor, perform local optimization on a to-be-solved transformation matrix of each to-be-calibrated sensor to the reference sensor according to the reference local perception data, the to-be-solved transformation matrix of each to-be-calibrated sensor to the reference sensor, and the auxiliary local perception data, to obtain a first transformation matrix of each to-be-calibrated sensor to the reference sensor, including:

[0350] In an example embodiment of the present application, the extrinsic parameter calibration processor 1520 is configured to, for each combination of the reference sensor and each to-be-calibrated sensor, perform local optimization on a to-be-solved transformation matrix of each to-be-calibrated sensor to the reference sensor according to the reference local perception data, the to-be-solved transformation matrix of each to-be-calibrated sensor to the reference sensor, and the auxiliary local perception data, to obtain a first transformation matrix of each to-be-calibrated sensor to the reference sensor, including:

[0351] In an example embodiment of the present application, the extrinsic parameter calibration processor 1520 is configured to, for each combination of the reference sensor and each to-be-calibrated sensor, perform local optimization on a to-be-solved transformation matrix of each to-be-calibrated sensor to the reference sensor according to the reference local perception data, the to-be-solved transformation matrix of each to-be-calibrated sensor to the reference sensor, and the auxiliary local perception data, to obtain a first transformation matrix of each to-be-calibrated sensor to the reference sensor, including:

[0352] For each combination of the reference sensor and each to-be-calibrated sensor, the extrinsic calibration processor 1520 determines, according to the reference local perception data, a to-be-solved transformation matrix of each to-be-calibrated sensor to the reference sensor, and the auxiliary local perception data, a comprehensive perception deviation corresponding to N motion intervals, and determines a local perception deviation of each to-be-calibrated sensor relative to the reference sensor on each motion interval.

[0353] The extrinsic calibration processor 1520 determines the comprehensive perception deviation according to N local perception deviations corresponding to the N motion intervals.

[0354] In the exemplary embodiments of the present application, for each combination of the reference sensor and each to-be-calibrated sensor, the extrinsic calibration processor 1520 determines, according to the reference local perception data, a to-be-solved transformation matrix of each to-be-calibrated sensor to the reference sensor, and the auxiliary local perception data, a comprehensive perception deviation corresponding to N motion intervals, and includes:

[0355] For each combination of the reference sensor and each to-be-calibrated sensor, the extrinsic calibration processor 1520 selects M valid data groups with successful registration from N data groups corresponding to the N motion intervals; M is less than or equal to N; the M valid data groups are associated with M valid motion intervals; each data group contains the reference local perception data and the auxiliary local perception data, and each valid data group contains valid reference local perception data and valid auxiliary local perception data with successful registration;

[0356] The extrinsic calibration processor 1520 determines, according to the valid reference local perception data, a to-be-solved transformation matrix of each to-be-calibrated sensor to the reference sensor, and the valid auxiliary local perception data, a local perception deviation of each to-be-calibrated sensor relative to the reference sensor on each valid motion interval.

[0357] The extrinsic calibration processor 1520 determines the comprehensive perception deviation according to M local perception deviations corresponding to the M valid motion intervals.

[0358] In the exemplary embodiments of the present application, the valid reference local perception data includes a valid reference local map, the valid auxiliary local perception data includes a valid auxiliary local map, and the local perception deviation includes a local map deviation.

[0359] For each combination of the reference sensor and each to-be-calibrated sensor, the extrinsic calibration processor 1520 determines, according to the valid reference local perception data, a to-be-solved transformation matrix of each to-be-calibrated sensor to the reference sensor, and the valid auxiliary local perception data, a local perception deviation of each to-be-calibrated sensor relative to the reference sensor on each valid motion interval.

[0360] According to the effective reference local map, the local map deviation of each of the to-be-calibrated sensors relative to the reference sensor on each of the effective motion intervals is determined in combination with the to-be-solved transformation matrix of each of the to-be-calibrated sensors to the reference sensor.

[0361] In an example embodiment of the present application, the extrinsic parameter calibration processor 1520 determines the local map deviation of each of the to-be-calibrated sensors relative to the reference sensor on each of the effective motion intervals based on the following formula:

[0362]

[0363] wherein l map represents the local map deviation of each of the to-be-calibrated sensors relative to the reference sensor on each of the effective motion intervals, for each combination of a reference sensor and a to-be-calibrated sensor, represents the effective reference local map corresponding to the i-th effective motion interval of the reference sensor, represents the effective auxiliary local map corresponding to the i-th effective motion interval of the to-be-calibrated sensor, and T represents the to-be-solved transformation matrix of the to-be-calibrated sensor to the reference sensor.

[0364] In an example embodiment of the present application, the extrinsic parameter calibration processor 1520 determines the comprehensive perception deviation according to M local perception deviations corresponding to M effective motion intervals, including:

[0365] The comprehensive perception deviation is determined based on the following formula:

[0366]

[0367] wherein l a1 represents the comprehensive perception deviation.

[0368] In an example embodiment of the present application, the effective reference local perception data includes an effective reference local trajectory, the effective auxiliary local perception data includes an effective auxiliary local trajectory, and the local perception deviation includes a local trajectory deviation;

[0369] The extrinsic parameter calibration processor 1520, for each combination of a reference sensor and each of the to-be-calibrated sensors, determines the local perception deviation of each of the to-be-calibrated sensors relative to the reference sensor on each of the effective motion intervals in combination with the to-be-solved transformation matrix of each of the to-be-calibrated sensors to the reference sensor and the effective auxiliary local perception data, including:

[0370] According to the effective reference local perception trajectory, the to-be-calibrated sensor-to-reference sensor conversion matrix to be solved in combination with the effective auxiliary local perception estimation, a local trajectory deviation of each to-be-calibrated sensor relative to the reference sensor on each effective motion interval is determined.

[0371] In the example embodiment of the present application, the external parameter calibration processor 1520 determines the local trajectory deviation of each to-be-calibrated sensor relative to the reference sensor on each effective motion interval based on the following formula:

[0372]

[0373] Wherein, for each combination of a reference sensor and a to-be-calibrated sensor, l traj represents the local trajectory deviation of each to-be-calibrated sensor relative to the reference sensor on each effective motion interval, represents the effective reference local trajectory corresponding to the reference sensor for the i-th effective motion interval, represents the effective auxiliary local trajectory corresponding to the to-be-calibrated sensor for the i-th effective motion interval, and T represents the to-be-calibrated sensor-to-reference sensor conversion matrix to be solved.

[0374] In the example embodiment of the present application, the external parameter calibration processor 1520 determines the comprehensive perception deviation according to M local perception deviations corresponding to M effective motion intervals, including:

[0375] The comprehensive perception deviation is determined based on the following formula:

[0376]

[0377] Wherein, l a1 represents the comprehensive perception deviation.

[0378] In the example embodiment of the present application, each data group includes a map group and a trajectory group, each map group is composed of a reference local map and an auxiliary local map corresponding to each motion interval, and each trajectory group is composed of a reference local trajectory and an auxiliary local trajectory corresponding to each motion interval;

[0379] The external parameter calibration processor 1520 screens M effective data groups with successful registration from N data groups corresponding to N motion intervals, including:

[0380] For each combination of the reference sensor and each of the to-be-calibrated sensors, M valid map groups are selected from N map groups corresponding to N motion intervals, each of the map groups consisting of a reference local map and an auxiliary local map, each of the valid map groups consisting of a valid reference local map and a valid auxiliary local map;

[0381] and M valid trajectory groups are selected from N trajectory groups corresponding to N motion intervals, each of the trajectory groups consisting of a reference local trajectory and an auxiliary local trajectory, each of the valid trajectory groups consisting of a valid reference local trajectory and a valid auxiliary local trajectory.

[0382] In the exemplary embodiments of the present application, for each combination of the reference sensor and each of the to-be-calibrated sensors, each of the valid map groups and each of the valid trajectory groups are determined by:

[0383] For each combination of a reference sensor and a to-be-calibrated sensor, a first converted map is determined according to a product between the auxiliary local map for each motion interval and an estimated map conversion matrix, and a first converted trajectory is determined according to a product between the auxiliary local trajectory for each motion interval and an estimated trajectory conversion matrix;

[0384] A specified map deviation between the reference local map corresponding to each motion interval and the first converted map is determined, a first norm corresponding to the specified map deviation and a first square value of the first norm are determined;

[0385] The value of the estimated map conversion matrix is optimized so that the first square value is minimized;

[0386] A specified trajectory deviation between the reference local trajectory corresponding to each motion interval and the first converted trajectory is determined, a second norm corresponding to the specified trajectory deviation and a second square value of the second norm are determined;

[0387] The value of the estimated trajectory conversion matrix is optimized so that the second square value is minimized;

[0388] If the first square value with the smallest value is less than a first preset error threshold and the second square value with the smallest value is less than a second preset error threshold, the auxiliary local map and the reference local map are determined as one valid map group with successful registration and the auxiliary local trajectory and the reference local trajectory are determined as one valid trajectory group with successful registration.

[0389] In the exemplary embodiments of the present application, the local perception deviation includes a local map deviation and a local trajectory deviation;

[0390] The extrinsic calibration processor 1520 determines, for each of the to-be-calibrated sensors, a local perception deviation of the to-be-calibrated sensor relative to the reference sensor on each of the effective motion intervals, according to the effective reference local perception data, the to-be-solved transformation matrix of each of the to-be-calibrated sensors relative to the reference sensor, and the effective auxiliary local perception data, including:

[0391] determines, for each of the to-be-calibrated sensors, a local map deviation of the to-be-calibrated sensor relative to the reference sensor on each of the effective motion intervals, according to the effective reference local map, the to-be-solved transformation matrix of each of the to-be-calibrated sensors relative to the reference sensor, and the effective auxiliary local map;

[0392] determines, for each of the to-be-calibrated sensors, a local trajectory deviation of the to-be-calibrated sensor relative to the reference sensor on each of the effective motion intervals, according to the effective reference local perception trajectory, the to-be-solved transformation matrix of each of the to-be-calibrated sensors relative to the reference sensor, and the effective auxiliary local perception estimate.

[0393] In an example embodiment of the present application, the extrinsic calibration processor 1520 determines the comprehensive perception deviation l a1 , according to M local perception deviations corresponding to M effective motion intervals.

[0394] The comprehensive perception deviation is determined based on the following formula:

[0395]

[0396] wherein l a1 represents the comprehensive perception deviation, represents an accumulated value of M local map deviations corresponding to M effective motion intervals, represents an accumulated value of M local trajectory deviations corresponding to M effective motion intervals.

[0397] In an example embodiment of the present application, the extrinsic calibration processor 1520 obtains a second transformation matrix of the reference sensor relative to an odometer coordinate system, including:

[0398] obtains, for each of the effective motion intervals, an odometer trajectory of the cleaning system;

[0399] determines each standard local trajectory deviation according to the odometer trajectory, each effective reference local trajectory, and the second to-be-solved transformation matrix of the reference sensor relative to the odometer coordinate system, and obtains a comprehensive trajectory deviation corresponding to M standard local trajectory deviations;

[0400] optimizing a value of the second to-be-solved conversion matrix of the reference sensor to the odometry coordinate system so as to minimize the comprehensive trajectory deviation, and solving the second conversion matrix of the reference sensor to the odometry coordinate system.

[0401] In an example embodiment of the present application, the extrinsic calibration processor 1520 determines each standard local trajectory deviation according to the odometry trajectory, each effective reference local trajectory and the second to-be-solved conversion matrix of the reference sensor to the odometry coordinate system, including:

[0402] The each standard local trajectory deviation is determined based on the following formula:

[0403]

[0404] wherein, for the i-th effective motion interval, l straj represents the each standard local trajectory deviation, represents the odometry trajectory of the cleaning system, T 基准2里程计 represents the second to-be-solved conversion matrix of the reference sensor to the odometry coordinate system, represents the effective reference local trajectory.

[0405] In an example embodiment of the present application, the extrinsic calibration processor 1520 determines the candidate conversion matrix of each to-be-calibrated sensor to the odometry coordinate system according to the first conversion matrix in combination with the second conversion matrix, including:

[0406] multiplying the first conversion matrix and the second conversion matrix to obtain the candidate conversion matrix of each to-be-calibrated sensor to the odometry coordinate system.

[0407] In an example embodiment of the present application, the global perception data of each to-be-calibrated sensor during the motion of the cleaning system includes an auxiliary global trajectory of each to-be-calibrated sensor during the motion of the cleaning system;

[0408] The global optimization of the candidate conversion matrix of each to-be-calibrated sensor to the odometry coordinate system according to the global perception data of each to-be-calibrated sensor during the motion of the cleaning system to obtain the final conversion matrix of each to-be-calibrated sensor to the odometry coordinate system, including:

[0409] configuring an identification number corresponding to each to-be-calibrated sensor of the K to-be-calibrated sensors, and the K identification numbers are in an increasing order;

[0410] When the maximum identification number is K, an xthto-be-calibrated sensor is selected from the K to-be-calibrated sensors in turn, and an x+1thsensor to a Kthsensor are combined with the xthto-be-calibrated sensor respectively to obtain K-x sensor combinations associated with the xthto-be-calibrated sensor; each of the sensor combinations comprises the xthto-be-calibrated sensor and a designated to-be-calibrated sensor; wherein 1≤x≤K-1;

[0411] For each of the sensor combinations, an error expression between the xthto-be-calibrated sensor and the designated to-be-calibrated sensor is determined according to an auxiliary global trajectory associated with the xthto-be-calibrated sensor, an auxiliary global trajectory associated with the designated to-be-calibrated sensor, a candidate conversion matrix of the xthto-be-calibrated sensor to the odometer coordinate system, and a candidate conversion matrix of the designated to-be-calibrated sensor to the odometer coordinate system.

[0412] An overall error expression between the xthto-be-calibrated sensor and K-x designated to-be-calibrated sensors is determined according to a sum of K-x error expressions.

[0413] A value of the candidate conversion matrix of the xthto-be-calibrated sensor to the odometer coordinate system is optimized so that a value of the overall error expression is minimized, and a final conversion matrix of the xthto-be-calibrated sensor to the odometer coordinate system is obtained.

[0414] In the exemplary embodiments of the present application, the external parameter calibration processor 1520 determines, for each of the sensor combinations, an error expression between the xthto-be-calibrated sensor and the designated to-be-calibrated sensor according to an auxiliary global trajectory associated with the xthto-be-calibrated sensor, an auxiliary global trajectory associated with the designated to-be-calibrated sensor, a candidate conversion matrix of the xthto-be-calibrated sensor to the odometer coordinate system, and a candidate conversion matrix of the designated to-be-calibrated sensor to the odometer coordinate system, including:

[0415] The error expression between the xthto-be-calibrated sensor and the designated to-be-calibrated sensor is determined based on the following formula:

[0416]

[0417] wherein e x,y represents the error expression between the xthto-be-calibrated sensor and the designated to-be-calibrated sensor, Traj x represents the auxiliary global trajectory associated with the xthto-be-calibrated sensor, Traj y represents the auxiliary global trajectory associated with the designated to-be-calibrated sensor, represents an inverse matrix of the candidate conversion matrix of the xthto-be-calibrated sensor to the odometer coordinate system, T yrepresenting a candidate transformation matrix of the specified to-be-calibrated sensor to the odometry coordinate system.

[0418] In the exemplary embodiments of the present application, the extrinsic calibration processor 1520 determines an overall error expression between the xth to-be-calibrated sensor and K-x specified to-be-calibrated sensors according to a sum of K-x error expressions, including:

[0419] The overall error expression between the xth to-be-calibrated sensor and K-x specified to-be-calibrated sensors is determined based on the following formula:

[0420]

[0421] wherein E represents the overall error expression between the xth to-be-calibrated sensor and K-x specified to-be-calibrated sensors.

[0422] In the exemplary embodiments of the present application, the motion interval is obtained according to a motion distance or a motion angle of the cleaning system.

[0423] The specific details of the modules in the above cleaning system have been described in detail in the extrinsic calibration method of the corresponding sensor, and thus will not be described here again.

[0424] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, such division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into embodied by multiple modules or units.

[0425] In addition, although the steps of the method in the present application are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps can be omitted, multiple steps can be combined into one step, and / or one step can be divided into multiple steps, etc.

[0426] Those skilled in the art can clearly understand that the example embodiments described in the specification can be implemented by software, or by a combination of software and necessary hardware. Therefore, the technical solutions of the embodiments of the present application can be embodied in a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, a USB, a hard disk, etc.) or a network, and includes a number of instructions to enable a computing device (such as a personal computer, a server, a mobile terminal, or a network device) to perform the methods according to the embodiments of the present application.

[0427] The present application also provides a computer readable storage medium, which can be included in the electronic device described in the above embodiments, or can exist independently without being assembled into the electronic device.

[0428] The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this application, the computer readable storage medium can be any tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device.

[0429] The computer readable storage medium can send, propagate, or transmit programs for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer readable storage medium can be transmitted using any suitable medium, including but not limited to wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0430] The computer readable storage medium carries one or more programs, which, when executed by the electronic device, enable the electronic device to implement the methods described in the above embodiments.

[0431] In addition, an electronic device capable of implementing the above method is also provided in the embodiments of the present application.

[0432] Those skilled in the art can understand that various aspects of the present application can be implemented as a system, a method or a program product. Therefore, various aspects of the present application can be embodied in a form of entirely hardware, entirely software (including firmware, microcode, etc.), or a combination of hardware and software, which can be generically referred to as "circuitry", "module" or "system".

[0433] The electronic device 1600 according to this embodiment of the present application will be described below with reference to FIG. 16. FIG. 16 shows the electronic device 1600 only as an example, and should not bring any limitation to the functions and usage scope of the embodiments of the present application.

[0434] As shown in FIG. 16, the electronic device 1600 is in the form of a general computing device. The components of the electronic device 1600 can include, but are not limited to, at least one processor 1610, at least one memory 1620, a bus 1630 connecting different system components including the memory 1620 and the processor 1610, and a display 1640.

[0435] The memory stores program codes which can be executed by the processor 1610, so that the processor 1610 performs the steps according to various exemplary embodiments of the present application described in the above "Exemplary Method" section of the present specification. For example, the processor 1610 can perform the steps as shown in FIG. 1, i.e., step S110, acquiring the reference local perception data of the reference sensor for each motion section during the movement of the cleaning robot, and acquiring the auxiliary local perception data of each of the to-be-calibrated sensors for the each motion section; step S120, for each combination of the reference sensor and the to-be-calibrated sensor, performing local optimization on the to-be-solved conversion matrix of each of the to-be-calibrated sensors to the reference sensor according to the reference local perception data, the to-be-solved conversion matrix of each of the to-be-calibrated sensors to the reference sensor, and the auxiliary local perception data, to obtain the first conversion matrix of each of the to-be-calibrated sensors to the reference sensor; step S130, acquiring the second conversion matrix of the reference sensor to the odometry coordinate system, and determining the candidate conversion matrix of each of the to-be-calibrated sensors to the odometry coordinate system according to the first conversion matrix and the second conversion matrix; and step S140, performing global optimization on the candidate conversion matrix of each of the to-be-calibrated sensors to the odometry coordinate system according to the global perception data of each of the to-be-calibrated sensors during the movement of the cleaning robot, to obtain the final conversion matrix of each of the to-be-calibrated sensors to the odometry coordinate system.

[0436] Memory 1620 can include a readable medium in the form of volatile memory, such as random access memory (RAM) 1621 and / or cache memory 1622, and can further include read only memory (ROM) 1623.

[0437] Memory 1620 can also include a program / utility 1624 having a set of programs / modules 1625, each of which performs one or more of the tasks of the electronic device 1600, as will be understood by those skilled in the art. The programs / utility 1624, can include both system and application programs, e.g., that can be used to implement the methods of the present application. The programs / utility 1624, can also include other programs of use in operations relating to the application, e.g., programs for network communications, programs for implementing network environments, etc.

[0438] Bus 1630 can represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration bus, a processor or local bus using any of a variety of bus architectures, etc.

[0439] The electronic device 1600 can also communicate with one or more external devices 1700 such as a keyboard or pointing device, a Bluetooth device, etc.; one or more devices that enable a user to interact with the electronic device 1600; and / or one or more devices that enable the electronic device 1600 to communicate with one or more other computing devices. Such communication can be via input / output (I / O) interface 1650. Still yet, the electronic device 1600 can communicate with one or more networks, such as a local area network (LAN), a general wide area network (WAN), and / or a public network such as the Internet, via network adapter 1660. As depicted, network adapter 1660 can communicate with the other components of the electronic device 1600 via bus 1630. It should be appreciated that although not shown, other hardware and / or software modules could be used in conjunction with the electronic device 1600. Such modules can include, but are not limited to, microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.

[0440] Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. It is intended that the specification and examples be considered as exemplary only, with the true scope and spirit of the application being indicated by the following claims.

Claims

1. A method for calibrating extrinsic parameters of a sensor, wherein, The method is applied to a cleaning robot, the cleaning robot is provided with K+1 sensors, the K+1 sensors include a reference sensor and K sensors to be calibrated, K is an integer greater than 1, and the method comprises the following steps: During the movement of the cleaning robot, reference local perception data of the reference sensor for each movement interval is acquired, and auxiliary local perception data of each sensor to be calibrated for each movement interval is acquired; For each combination of the reference sensor and each sensor to be calibrated, a to-be-solved conversion matrix of each sensor to be calibrated to the reference sensor is locally optimized according to the reference local perception data, the to-be-solved conversion matrix of each sensor to be calibrated to the reference sensor, and the auxiliary local perception data, to obtain a first conversion matrix of each sensor to be calibrated to the reference sensor; A second conversion matrix of the reference sensor to an odometer coordinate system is acquired, and a candidate conversion matrix of each sensor to be calibrated to the odometer coordinate system is determined according to the first conversion matrix and the second conversion matrix; Global optimization is performed on the candidate conversion matrix of each sensor to be calibrated to the odometer coordinate system according to global perception data of each sensor to be calibrated during the movement of the cleaning robot, to obtain a final conversion matrix of each sensor to be calibrated to the odometer coordinate system.

2. The method of claim 1, wherein, The method is applied to a cleaning robot, the cleaning robot is provided with K+1 sensors, the K+1 sensors include a reference sensor and K sensors to be calibrated, K is an integer greater than 1, and the method comprises the following steps: During the movement of the cleaning robot, reference local perception data of the reference sensor for each movement interval is acquired, and auxiliary local perception data of each sensor to be calibrated for each movement interval is acquired; For each combination of the reference sensor and each sensor to be calibrated, a to-be-solved conversion matrix of each sensor to be calibrated to the reference sensor is locally optimized according to the reference local perception data, the to-be-solved conversion matrix of each sensor to be calibrated to the reference sensor, and the auxiliary local perception data, to obtain a first conversion matrix of each sensor to be calibrated to the reference sensor; 3. The method of claim 2, wherein, A second conversion matrix of the reference sensor to an odometer coordinate system is acquired, and a candidate conversion matrix of each sensor to be calibrated to the odometer coordinate system is determined according to the first conversion matrix and the second conversion matrix; Global optimization is performed on the candidate conversion matrix of each sensor to be calibrated to the odometer coordinate system according to global perception data of each sensor to be calibrated during the movement of the cleaning robot, to obtain a final conversion matrix of each sensor to be calibrated to the odometer coordinate system. The method is applied to a cleaning robot, the cleaning robot is provided with K+1 sensors, the K+1 sensors include a reference sensor and K sensors to be calibrated, K is an integer greater than 1, and the method comprises the following steps: During the movement of the cleaning robot, reference local perception data of the reference sensor for each movement interval is acquired, and auxiliary local perception data of each sensor to be calibrated for each movement interval is acquired; For each combination of the reference sensor and each sensor to be calibrated, a to-be-solved conversion matrix of each sensor to be calibrated to the reference sensor is locally optimized according to the reference local perception data, the to-be-solved conversion matrix of each sensor to be calibrated to the reference sensor, and the auxiliary local perception data, to obtain a first conversion matrix of each sensor to be calibrated to the reference sensor; A second conversion matrix of the reference sensor to an odometer coordinate system is acquired, and a candidate conversion matrix of each sensor to be calibrated to the odometer coordinate system is determined according to the first conversion matrix and the second conversion matrix; Global optimization is performed on the candidate conversion matrix of each sensor to be calibrated to the odometer coordinate system according to global perception data of each sensor to be calibrated during the movement of the cleaning robot, to obtain a final conversion matrix of each sensor to be calibrated to the odometer coordinate system. The comprehensive perception bias is determined according to N local perception biases corresponding to N motion intervals.

4. The method of claim 2, wherein, The comprehensive perception bias corresponding to each of the N motion intervals is determined according to the reference local perception data, a to-be-solved transformation matrix of each of the to-be-calibrated sensors to the reference sensor, and the auxiliary local perception data. For each combination of the reference sensor and each of the to-be-calibrated sensors, M valid data groups with successful registration are selected from N data groups corresponding to the N motion intervals; M is less than or equal to N; the M valid data groups are associated with M valid motion intervals; each of the data groups includes the reference local perception data and the auxiliary local perception data, and each of the valid data groups includes valid reference local perception data and valid auxiliary local perception data with successful registration; The local perception bias of each of the to-be-calibrated sensors relative to the reference sensor on each of the valid motion intervals is determined according to the valid reference local perception data, the to-be-solved transformation matrix of each of the to-be-calibrated sensors to the reference sensor, and the valid auxiliary local perception data. The comprehensive perception bias is determined according to M local perception biases corresponding to M valid motion intervals.

5. The method of claim 4, wherein, The valid reference local perception data includes a valid reference local map, the valid auxiliary local perception data includes a valid auxiliary local map, and the local perception bias includes a local map bias; The local perception bias of each of the to-be-calibrated sensors relative to the reference sensor on each of the valid motion intervals is determined according to the valid reference local perception data, the to-be-solved transformation matrix of each of the to-be-calibrated sensors to the reference sensor, and the valid auxiliary local perception data. The local map bias of each of the to-be-calibrated sensors relative to the reference sensor on each of the valid motion intervals is determined according to the valid reference local map, the to-be-solved transformation matrix of each of the to-be-calibrated sensors to the reference sensor, and the valid auxiliary local map.

6. The method of claim 5, wherein, The local map deviation of each of the to-be-calibrated sensors relative to the reference sensor on each of the effective motion intervals is determined based on the following formula: wherein, l map represents the local map bias of each said to-be-calibrated sensor relative to said reference sensor on each said valid motion section, for each combination of reference sensor and to-be-calibrated sensor, a representative effective reference local map corresponding to the reference sensor for the i-th effective motion interval, T represents the to-be-solved transformation matrix of the to-be-calibrated sensor to the reference sensor.

7. The method of claim 6, wherein, The comprehensive perception bias is determined according to M local perception biases corresponding to M valid motion intervals. The integrated perceptual bias is determined based on the following equation: wherein, l a1 represents the comprehensive perceptual bias.

8. The method of any one of claims 4 to 7, wherein, The valid reference local perception data includes a valid reference local trajectory, the valid auxiliary local perception data includes a valid auxiliary local trajectory, and the local perception bias includes a local trajectory bias; According to the effective reference local perception data, the to-be-solved transformation matrix of each to-be-calibrated sensor to the reference sensor, and the effective auxiliary local perception data, the local perception deviation of each to-be-calibrated sensor relative to the reference sensor on each effective motion interval is determined, including: According to the effective reference local perception trajectory, the to-be-solved transformation matrix of each to-be-calibrated sensor to the reference sensor, and the effective auxiliary local perception estimation, the local trajectory deviation of each to-be-calibrated sensor relative to the reference sensor on each effective motion interval is determined.

9. The method of claim 8, wherein, The local trajectory deviation of each of the to-be-calibrated sensors relative to the reference sensor on each of the effective motion intervals is determined based on the following formula: wherein, for each combination of a reference sensor and a sensor to be calibrated, l traj representing a local trajectory deviation of each of the sensors to be calibrated relative to the reference sensor over each of the effective motion intervals, representing an effective reference local trajectory corresponding to the reference sensor for the i-th effective motion interval, T represents the effective auxiliary local trajectory corresponding to the to-be-calibrated sensor for the i-th effective motion interval, and T represents the to-be-solved transformation matrix of the to-be-calibrated sensor to the reference sensor.

10. The method of claim 9, wherein, The determination of the comprehensive perception deviation according to the M local perception deviations corresponding to the M effective motion intervals includes: The integrated perceptual bias is determined based on the following equation: wherein, l a1 represents the comprehensive perceptual bias.

11. The method of any one of claims 4-10, wherein, Each data group includes a map group and a trajectory group, each map group is composed of a reference local map and an auxiliary local map corresponding to each motion interval, and each trajectory group is composed of a reference local trajectory and an auxiliary local trajectory corresponding to each motion interval. The screening of M effective data groups with successful registration from N data groups corresponding to N motion intervals includes: For the combination of the reference sensor and each to-be-calibrated sensor, M effective map groups with successful registration are screened from N map groups corresponding to N motion intervals, each map group is composed of a reference local map and an auxiliary local map, and each effective map group is composed of an effective reference local map and an effective auxiliary local map with successful registration. And, M effective trajectory groups with successful registration are screened from N trajectory groups corresponding to N motion intervals, each trajectory group is composed of a reference local trajectory and an auxiliary local trajectory, and each effective trajectory group is composed of an effective reference local trajectory and an effective auxiliary local trajectory with successful registration.

12. The method of claim 11, wherein, For the combination of the reference sensor and each to-be-calibrated sensor, each effective map group and each effective trajectory group is determined by: For each combination of a reference sensor and a to-be-calibrated sensor, a first conversion map is determined according to the product between the auxiliary local map for each motion interval and the estimated map conversion matrix, and a first conversion trajectory is determined according to the product between the auxiliary local trajectory for each motion interval and the estimated trajectory conversion matrix; The specified map deviation between the reference local map corresponding to each motion interval and the first conversion map is determined, the first norm corresponding to the specified map deviation and the first square value of the first norm are determined; The value of the estimated map conversion matrix is optimized so that the first square value is minimized; and The value of the estimated trajectory conversion matrix is optimized so that the second square value is minimized. determining a specified trajectory deviation between the reference local trajectory corresponding to each motion section and the first conversion trajectory, determining a second norm corresponding to the specified trajectory deviation and a second square value of the second norm; optimizing the value of the estimated trajectory conversion matrix so that the second square value is minimum; if the first square value with the minimum value is less than a first preset error threshold and the second square value with the minimum value is less than a second preset error threshold, determining that the auxiliary local map and the reference local map are an effective map group with successful registration and the auxiliary local trajectory and the reference local trajectory are an effective trajectory group with successful registration.

13. The method of claim 11, wherein, The local perception deviation includes a local map deviation and a local trajectory deviation. For each combination of the reference sensor and the to-be-calibrated sensor, the local perception deviation of each to-be-calibrated sensor relative to the reference sensor in each effective motion section is determined according to the effective reference local perception data, the to-be-solved conversion matrix of each to-be-calibrated sensor to the reference sensor, and the effective auxiliary local perception data, including: determining the local map deviation of each to-be-calibrated sensor relative to the reference sensor in each effective motion section according to the effective reference local map, the to-be-solved conversion matrix of each to-be-calibrated sensor to the reference sensor, and the effective auxiliary local map; determining the local trajectory deviation of each to-be-calibrated sensor relative to the reference sensor in each effective motion section according to the effective reference local perception trajectory, the to-be-solved conversion matrix of each to-be-calibrated sensor to the reference sensor, and the effective auxiliary local perception estimate.

14. The method of claim 13, wherein, determining the comprehensive perceptual bias l according to M local perceptual biases corresponding to M effective motion intervals a1 comprising: The integrated perceptual bias is determined based on the following equation: wherein, l a1 represents the comprehensive perception bias, accumulated values of M local map biases corresponding to M said effective motion intervals, The accumulated value of M local trajectory deviations corresponding to M effective motion sections.

15. The method of claim 8, wherein, The acquisition of the second conversion matrix of the reference sensor to the odometer coordinate system includes: acquiring the odometer trajectory of the cleaning robot for each effective motion section; determining each standard local trajectory deviation according to the odometer trajectory, each effective reference local trajectory, and the second to-be-solved conversion matrix of the reference sensor to the odometer coordinate system, and acquiring a comprehensive trajectory deviation corresponding to M standard local trajectory deviations; optimizing the value of the second to-be-solved conversion matrix of the reference sensor to the odometer coordinate system so that the comprehensive trajectory deviation is minimum, and solving the second conversion matrix of the reference sensor to the odometer coordinate system.

16. The method of claim 15, wherein, The determination of each standard local trajectory deviation according to the odometer trajectory, each effective reference local trajectory, and the second to-be-solved conversion matrix of the reference sensor to the odometer coordinate system includes: The each standard local trajectory deviation is determined based on the following equation: wherein, for the ith valid motion interval, l xtraj representing said each standard local trajectory deviation, representing an odometry trajectory of a cleaning robot, T 基准2里程计 representing a second conversion matrix to be solved of the reference sensor to the odometry coordinate system, representing the effective reference local trajectory.

17. The method of claim 1, wherein, The determination of the candidate conversion matrix of each to-be-calibrated sensor to the odometer coordinate system according to the first conversion matrix and the second conversion matrix includes: multiplying the first conversion matrix and the second conversion matrix to obtain the candidate conversion matrix of each to-be-calibrated sensor to the odometer coordinate system.

18. The method of claim 1, wherein, The global perception data of each sensor to be calibrated during the motion of the cleaning robot comprises an auxiliary global trajectory of each sensor to be calibrated during the motion of the cleaning robot; The global optimization of the candidate conversion matrix of each sensor to be calibrated to the odometer coordinate system according to the global perception data of each sensor to be calibrated during the motion of the cleaning robot, to obtain the final conversion matrix of each sensor to be calibrated to the odometer coordinate system, comprises: An identification number corresponding to each of the K sensors to be calibrated is configured, and the K identification numbers are in an increasing order; When the maximum identification number is K, an xth sensor to be calibrated is selected from the K sensors to be calibrated in turn, and an x+1th sensor to a Kth sensor are combined with the xth sensor to be calibrated respectively to obtain K-x sensor combinations associated with the xth sensor to be calibrated; each sensor combination comprises the xth sensor to be calibrated and a specified sensor to be calibrated; wherein, 1≤x≤K-1; For each sensor combination, an error expression between the xth sensor to be calibrated and the specified sensor to be calibrated is determined according to an auxiliary global trajectory associated with the xth sensor to be calibrated, an auxiliary global trajectory associated with the specified sensor to be calibrated, a candidate conversion matrix of the xth sensor to be calibrated to the odometer coordinate system, and a candidate conversion matrix of the specified sensor to be calibrated to the odometer coordinate system; An overall error expression between the xth sensor to be calibrated and K-x specified sensors to be calibrated is determined according to a sum of K-x error expressions; The value of the candidate conversion matrix of the xth sensor to be calibrated to the odometer coordinate system is optimized so that the value of the overall error expression is minimized, and a final conversion matrix of the xth sensor to be calibrated to the odometer coordinate system is obtained.

19. The method of claim 18, wherein, The global optimization of the candidate conversion matrix of each sensor to be calibrated to the odometer coordinate system according to the global perception data of each sensor to be calibrated during the motion of the cleaning robot, to obtain the final conversion matrix of each sensor to be calibrated to the odometer coordinate system, comprises: An error expression between the xth sensor to be calibrated and the designated sensor to be calibrated is determined based on the following equation: wherein e x,y represents an error expression between the xth sensor to be calibrated and the designated sensor to be calibrated, Traj x represents an auxiliary global trajectory associated with the xth sensor to be calibrated, Traj y represents an auxiliary global trajectory associated with the designated sensor to be calibrated, an inverse of a candidate transformation matrix representing the xth sensor to be calibrated to the odometry coordinate system, T y a candidate transformation matrix representing the designated sensor to be calibrated to the odometry coordinate system.

20. The method of claim 19, wherein, The global optimization of the candidate conversion matrix of each sensor to be calibrated to the odometer coordinate system according to the global perception data of each sensor to be calibrated during the motion of the cleaning robot, to obtain the final conversion matrix of each sensor to be calibrated to the odometer coordinate system, comprises: An overall error expression between the x-th sensor to be calibrated and K-x specified sensors to be calibrated is determined based on the following equation: Wherein, E represents the overall error expression between the xth sensor to be calibrated and K-x specified sensors to be calibrated.

21. The method of any one of claims 1 to 20, wherein, The motion interval is obtained according to the motion distance or the motion angle of the cleaning robot.

22. A cleaning robot, wherein, The cleaning robot is provided with K+1 sensors, the K+1 sensors comprising a reference sensor and K sensors to be calibrated, K being an integer greater than 1, and the cleaning robot comprising a perception module and an external parameter calibration module: The perception module is configured to obtain reference local perception data of the reference sensor for each motion interval during motion of the cleaning robot, and obtain auxiliary local perception data of each of the sensors to be calibrated for the each motion interval; The external parameter calibration module is configured to: for each combination of the reference sensor and each sensor to be calibrated, according to the reference local perception data, a to-be-solved conversion matrix of each of the sensors to be calibrated to the reference sensor, and the auxiliary local perception data, locally optimize the to-be-solved conversion matrix of each of the sensors to be calibrated to the reference sensor to obtain a first conversion matrix of each of the sensors to be calibrated to the reference sensor; obtain a second conversion matrix of the reference sensor to an odometer coordinate system, and determine a candidate conversion matrix of each of the sensors to be calibrated to the odometer coordinate system according to the first conversion matrix in combination with the second conversion matrix; globally optimize the candidate conversion matrix of each of the sensors to be calibrated to the odometer coordinate system according to global perception data of each of the sensors to be calibrated during motion of the cleaning robot, and obtain a final conversion matrix of each of the sensors to be calibrated to the odometer coordinate system.

23. The cleaning robot of claim 22, wherein, The external parameter calibration module is configured to: for each combination of the reference sensor and each sensor to be calibrated, according to the reference local perception data, a to-be-solved conversion matrix of each of the sensors to be calibrated to the reference sensor, and the auxiliary local perception data, determine a comprehensive perception deviation corresponding to N motion intervals; N is an integer greater than 1; optimize a value of the to-be-solved conversion matrix to minimize the comprehensive perception deviation, and obtain a first conversion matrix of each of the sensors to be calibrated to the reference sensor.

24. The cleaning robot of claim 23, wherein, The external parameter calibration module is configured to: for each combination of the reference sensor and each sensor to be calibrated, according to the reference local perception data, a to-be-solved conversion matrix of each of the sensors to be calibrated to the reference sensor, and the auxiliary local perception data, determine a comprehensive perception deviation corresponding to N motion intervals, and determine a local perception deviation of each of the sensors to be calibrated relative to the reference sensor in each of the motion intervals; determine the comprehensive perception deviation according to N local perception deviations corresponding to N motion intervals.

25. The cleaning robot of claim 23, wherein, The external parameter calibration module is configured to: for each combination of the reference sensor and each of the sensors to be calibrated, filter out M valid data groups with successful registration from N data groups corresponding to N motion intervals; M is less than or equal to N; the M valid data groups are associated with M valid motion intervals; each of the data groups includes the reference local perception data and the auxiliary local perception data, and each of the valid data groups includes valid reference local perception data and valid auxiliary local perception data with successful registration; determining, according to the effective reference local perception data, the to-be-solved transformation matrix of each of the to-be-calibrated sensors to the reference sensor, in combination with the effective auxiliary local perception data, a local perception deviation of each of the to-be-calibrated sensors relative to the reference sensor on each of the effective motion intervals; determining the comprehensive perception deviation according to M local perception deviations corresponding to M of the effective motion intervals.

26. The cleaning robot of claim 25, wherein, The effective reference local perception data comprises an effective reference local map, the effective auxiliary local perception data comprises an effective auxiliary local map, and the local perception deviation comprises a local map deviation. The extrinsic parameter calibration module is further configured to: determining, according to the effective reference local map, the to-be-solved transformation matrix of each of the to-be-calibrated sensors to the reference sensor, in combination with the effective auxiliary local map, a local map deviation of each of the to-be-calibrated sensors relative to the reference sensor on each of the effective motion intervals.

27. The cleaning robot of claim 26, wherein, The external parameter calibration module determines the local map deviation of each of the sensors to be calibrated relative to the reference sensor on each of the effective motion intervals based on the following formula: wherein, l map represents the local map bias of each said to-be-calibrated sensor relative to said reference sensor on each said valid motion section, for each combination of reference sensor and to-be-calibrated sensor, a representative effective reference local map corresponding to the reference sensor for the i-th effective motion interval, T represents the to-be-solved transformation matrix of the to-be-calibrated sensor to the reference sensor.

28. The cleaning robot of claim 27, wherein, The extrinsic parameter calibration module is configured to: The integrated perceptual bias is determined based on the following equation: wherein, l a1 represents the comprehensive perceptual bias.

29. The cleaning robot of any one of claims 25 to 28, wherein, The effective reference local perception data comprises an effective reference local trajectory, the effective auxiliary local perception data comprises an effective auxiliary local trajectory, and the local perception deviation comprises a local trajectory deviation. The extrinsic parameter calibration module is further configured to: determining, according to the effective reference local perception trajectory, the to-be-solved transformation matrix of each of the to-be-calibrated sensors to the reference sensor, in combination with the effective auxiliary local perception estimate, a local trajectory deviation of each of the to-be-calibrated sensors relative to the reference sensor on each of the effective motion intervals.

30. The cleaning robot of claim 29, wherein, The external parameter calibration module is configured to determine a local trajectory deviation of each of the sensors to be calibrated relative to the reference sensor on each of the effective motion intervals based on the following formula: wherein, for each combination of a reference sensor and a sensor to be calibrated, l traj representing a local trajectory deviation of each of the sensors to be calibrated relative to the reference sensor over each of the effective motion intervals, representing an effective reference local trajectory corresponding to the reference sensor for the i-th effective motion interval, T represents the to-be-solved transformation matrix of the to-be-calibrated sensor to the reference sensor.

31. The cleaning robot of claim 30, wherein, The extrinsic parameter calibration module is configured to: The integrated perceptual bias is determined based on the following equation: wherein, l a1 represents the comprehensive perceptual bias.

32. The cleaning robot of any one of claims 25 to 31, wherein, Each of the data groups comprises a map group and a trajectory group, each of the map groups is composed of a reference local map and an auxiliary local map corresponding to each of the motion intervals, and each of the trajectory groups is composed of a reference local trajectory and an auxiliary local trajectory corresponding to each of the motion intervals. The extrinsic parameter calibration module is configured to: for each combination of the reference sensor and each of the to-be-calibrated sensors, screening M effective map groups with successful registration from N map groups corresponding to N of the motion intervals; each of the map groups is composed of a reference local map and an auxiliary local map, and each of the effective map groups is composed of an effective reference local map and an effective auxiliary local map with successful registration; and screening M effective trajectory groups with successful registration from N trajectory groups corresponding to N of the motion intervals; each of the trajectory groups is composed of a reference local trajectory and an auxiliary local trajectory, and each of the effective trajectory groups is composed of an effective reference local trajectory and an effective auxiliary local trajectory with successful registration.

33. The cleaning robot of claim 32, wherein, The extrinsic parameter calibration module is configured to: determining a first converted map according to a product between the auxiliary local map and the estimated map conversion matrix for each motion section corresponding to each combination of the reference sensor and the sensor to be calibrated, and determining a first converted trajectory according to a product between the auxiliary local trajectory and the estimated trajectory conversion matrix for each motion section; determining a specified map deviation between the reference local map corresponding to each motion section and the first converted map, determining a first norm corresponding to the specified map deviation and a first square value of the first norm; optimizing the value of the estimated map conversion matrix so that the first square value is minimized; determining a specified trajectory deviation between the reference local trajectory corresponding to each motion section and the first converted trajectory, determining a second norm corresponding to the specified trajectory deviation and a second square value of the second norm; optimizing the value of the estimated trajectory conversion matrix so that the second square value is minimized; if the first square value with the minimum value is less than a first preset error threshold and the second square value with the minimum value is less than a second preset error threshold, determining the auxiliary local map and the reference local map as an effective map group with successful registration and determining the auxiliary local trajectory and the reference local trajectory as an effective trajectory group with successful registration.

34. The cleaning robot of claim 32, wherein, The local perception deviation includes a local map deviation and a local trajectory deviation. The extrinsic parameter calibration module is configured to: determine a local map deviation of each sensor to be calibrated relative to the reference sensor on each effective motion section according to the effective reference local map, the to-be-solved conversion matrix of each sensor to be calibrated to the reference sensor and the effective auxiliary local map; determine a local trajectory deviation of each sensor to be calibrated relative to the reference sensor on each effective motion section according to the effective reference local perception trajectory, the to-be-solved conversion matrix of each sensor to be calibrated to the reference sensor and the effective auxiliary local perception estimate.

35. The cleaning robot of claim 34, wherein, The extrinsic parameter calibration module is configured to: The integrated perceptual bias is determined based on the following equation: wherein, l a1 representing the comprehensive perceptual bias, accumulated values of M local map biases corresponding to M said effective motion intervals, represent an accumulated value of M local trajectory deviations corresponding to M effective motion sections.

36. The cleaning robot of claim 29, wherein, The extrinsic parameter calibration module is further configured to: for each effective motion section, obtain an odometer trajectory of the cleaning robot; determine each standard local trajectory deviation according to the odometer trajectory, each effective reference local trajectory and a second to-be-solved conversion matrix of the reference sensor to the odometer coordinate system, and obtain a comprehensive trajectory deviation corresponding to M standard local trajectory deviations; optimize the value of the second to-be-solved conversion matrix of the reference sensor to the odometer coordinate system so that the comprehensive trajectory deviation is minimized, and solve the second conversion matrix of the reference sensor to the odometer coordinate system.

37. The cleaning robot of claim 36, wherein, The extrinsic parameter calibration module is configured to: The each standard local trajectory deviation is determined based on the following equation: wherein, for the ith valid motion interval, l straj representing said each standard local trajectory deviation, representing an odometry trajectory of a cleaning robot, T 基准2里程计 representing a second conversion matrix to be solved of the reference sensor to the odometry coordinate system, represent the effective reference local trajectory.

38. The cleaning robot of claim 22, wherein, The extrinsic parameter calibration module is configured to: multiply the first conversion matrix and the second conversion matrix to obtain a candidate conversion matrix of each sensor to be calibrated to the odometer coordinate system.

39. The cleaning robot of claim 22, wherein, The global perception data of each to-be-calibrated sensor during the motion of the cleaning robot comprises an auxiliary global trajectory of each to-be-calibrated sensor during the motion of the cleaning robot; The extrinsic calibration module is configured to: An identification number corresponding to each to-be-calibrated sensor in the K to-be-calibrated sensors is configured, and the K identification numbers are in an increasing order; When the maximum identification number is K, an xth to-be-calibrated sensor is sequentially selected from the K to-be-calibrated sensors, and an x+1th sensor to a Kth sensor are combined with the xth to-be-calibrated sensor respectively to obtain K-x sensor combinations associated with the xth to-be-calibrated sensor; each sensor combination comprises the xth to-be-calibrated sensor and a designated to-be-calibrated sensor; wherein 1≤x≤K-1; For each sensor combination, an error expression between the xth to-be-calibrated sensor and the designated to-be-calibrated sensor is determined according to an auxiliary global trajectory associated with the xth to-be-calibrated sensor, an auxiliary global trajectory associated with the designated to-be-calibrated sensor, a candidate conversion matrix of the xth to-be-calibrated sensor to the odometer coordinate system, and a candidate conversion matrix of the designated to-be-calibrated sensor to the odometer coordinate system; An overall error expression between the xth to-be-calibrated sensor and K-x designated to-be-calibrated sensors is determined according to a sum of K-x error expressions. The value of the candidate conversion matrix of the xth to-be-calibrated sensor to the odometer coordinate system is optimized so that the value of the overall error expression is minimized, and a final conversion matrix of the xth to-be-calibrated sensor to the odometer coordinate system is obtained.

40. The cleaning robot of claim 39, wherein, The extrinsic calibration module is configured to: An error expression between the xth sensor to be calibrated and the designated sensor to be calibrated is determined based on the following equation: wherein e x,y represents an error expression between the xth sensor to be calibrated and the designated sensor to be calibrated, Traj x represents an auxiliary global trajectory associated with the xth sensor to be calibrated, Traj y represents an auxiliary global trajectory associated with the designated sensor to be calibrated, an inverse of a candidate transformation matrix representing the xth sensor to be calibrated to the odometry coordinate system, T y a candidate transformation matrix representing the designated sensor to be calibrated to the odometry coordinate system.

41. The cleaning robot of claim 40, wherein, The extrinsic calibration module is configured to: An overall error expression between the x-th sensor to be calibrated and K-x specified sensors to be calibrated is determined based on the following equation: Wherein E represents the overall error expression between the xth to-be-calibrated sensor and K-x designated to-be-calibrated sensors.

42. The cleaning robot of any one of claims 22 to 41, wherein, The motion interval is obtained according to the motion distance or the motion angle of the cleaning system.

43. A cleaning system wherein, The cleaning system comprises K+1 sensors, the K+1 sensors comprising a reference sensor and K to-be-calibrated sensors, K being an integer greater than 1, and the cleaning system comprising a data collector and an extrinsic calibration processor: The data collector is configured to, during the motion of the cleaning system, acquire reference local perception data of the reference sensor for each motion interval, and acquire auxiliary local perception data of each to-be-calibrated sensor for each motion interval; The extrinsic calibration processor is configured to: For a combination of the reference sensor and each to-be-calibrated sensor, each to-be-calibrated sensor to the reference sensor is locally optimized according to the reference local perception data, the auxiliary local perception data, and a to-be-solved conversion matrix of each to-be-calibrated sensor to the reference sensor, to obtain a first conversion matrix of each to-be-calibrated sensor to the reference sensor. obtain a second conversion matrix of the reference sensor to the odometer coordinate system, and determine a candidate conversion matrix of each to-be-calibrated sensor to the odometer coordinate system according to the first conversion matrix in combination with the second conversion matrix; perform global optimization on the candidate conversion matrix of each to-be-calibrated sensor to the odometer coordinate system according to global perception data of each to-be-calibrated sensor during movement of the cleaning system, and obtain a final conversion matrix of each to-be-calibrated sensor to the odometer coordinate system.

44. The cleaning system of claim 43, wherein, The extrinsic calibration processor is configured to: for each combination of the reference sensor and a to-be-calibrated sensor, determine a comprehensive perception deviation corresponding to N motion intervals according to the reference local perception data, a to-be-solved conversion matrix of each to-be-calibrated sensor to the reference sensor in combination with the auxiliary local perception data; N is an integer greater than 1; optimize the value of the to-be-solved conversion matrix to minimize the comprehensive perception deviation, and obtain a first conversion matrix of each to-be-calibrated sensor to the reference sensor.

45. The cleaning system of claim 44, wherein, The extrinsic calibration processor is configured to: for each combination of the reference sensor and a to-be-calibrated sensor, determine a comprehensive perception deviation corresponding to N motion intervals according to the reference local perception data, a to-be-solved conversion matrix of each to-be-calibrated sensor to the reference sensor in combination with the auxiliary local perception data; N is an integer greater than 1; determine a local perception deviation of each to-be-calibrated sensor relative to the reference sensor in each motion interval according to N local perception deviations corresponding to N motion intervals.

46. The cleaning system of claim 44, wherein, The extrinsic calibration processor is configured to: for each combination of the reference sensor and a to-be-calibrated sensor, select M valid data groups with successful registration from N data groups corresponding to N motion intervals; M is less than or equal to N; the M valid data groups are associated with M valid motion intervals; each data group includes the reference local perception data and the auxiliary local perception data, and each valid data group includes valid reference local perception data and valid auxiliary local perception data with successful registration; determine a local perception deviation of each to-be-calibrated sensor relative to the reference sensor in each valid motion interval according to the valid reference local perception data, a to-be-solved conversion matrix of each to-be-calibrated sensor to the reference sensor in combination with the valid auxiliary local perception data; determine the comprehensive perception deviation according to M local perception deviations corresponding to M valid motion intervals.

47. The cleaning system of claim 46, wherein, The valid reference local perception data includes a valid reference local map, the valid auxiliary local perception data includes a valid auxiliary local map, and the local perception deviation includes a local map deviation; The extrinsic calibration processor is configured to: determine a local map deviation of each to-be-calibrated sensor relative to the reference sensor in each valid motion interval according to the valid reference local map, a to-be-solved conversion matrix of each to-be-calibrated sensor to the reference sensor in combination with the valid auxiliary local map.

48. The cleaning system of claim 47, wherein, The external parameter calibration processor determines the local map deviation of each of the sensors to be calibrated relative to the reference sensor on each of the effective motion intervals based on the following formula: wherein, l map represents the local map bias of each said to-be-calibrated sensor relative to said reference sensor on each said valid motion section, for each combination of reference sensor and to-be-calibrated sensor, a representative effective reference local map corresponding to the reference sensor for the i-th effective motion interval, T represents the to-be-calibrated sensor corresponding to the effective auxiliary local map of the i-th effective motion interval, and T represents the to-be-calibrated sensor to the reference sensor.

49. The cleaning system of claim 48, wherein, The external parameter calibration processor is configured to: The integrated perceptual bias is determined based on the following equation: wherein, l a1 represents the overall perceptual bias.

50. The cleaning system of any one of claims 46 to 49, wherein, The effective reference local perception data includes an effective reference local trajectory, and the effective auxiliary local perception data includes an effective auxiliary local trajectory, and the local perception deviation includes a local trajectory deviation. The external parameter calibration processor is configured to: According to the effective reference local perception trajectory, the to-be-calibrated sensor to the reference sensor, and the effective auxiliary local perception estimation, the local trajectory deviation of each to-be-calibrated sensor relative to the reference sensor on each effective motion interval is determined.

51. The cleaning system of claim 50, wherein, The external parameter calibration processor determines the local trajectory deviation of each of the sensors to be calibrated relative to the reference sensor on each of the effective motion intervals based on the following formula: wherein, for each combination of a reference sensor and a sensor to be calibrated, l traj representing a local trajectory deviation of each of the sensors to be calibrated relative to the reference sensor over each of the effective motion intervals, representing an effective reference local trajectory corresponding to the reference sensor for the i-th effective motion interval, T represents the to-be-calibrated sensor corresponding to the effective auxiliary local trajectory of the i-th effective motion interval, and T represents the to-be-calibrated sensor to the reference sensor.

52. The cleaning system of claim 51, wherein, The external parameter calibration processor is configured to: The integrated perceptual bias is determined based on the following equation: wherein, l a1 represents the overall perceptual bias.

53. The cleaning system of any one of claims 46 to 52, wherein, Each data group includes a map group and a trajectory group, each map group is composed of a reference local map and an auxiliary local map corresponding to each motion interval, and each trajectory group is composed of a reference local trajectory and an auxiliary local trajectory corresponding to each motion interval. The external parameter calibration processor is configured to: For the combination of the reference sensor and each to-be-calibrated sensor, M effective map groups with successful registration are selected from N map groups corresponding to N motion intervals; each map group is composed of a reference local map and an auxiliary local map, and each effective map group is composed of an effective reference local map and an effective auxiliary local map with successful registration. And from N trajectory groups corresponding to N motion intervals, M effective trajectory groups with successful registration are selected; each trajectory group is composed of a reference local trajectory and an auxiliary local trajectory, and each effective trajectory group is composed of an effective reference local trajectory and an effective auxiliary local trajectory with successful registration.

54. The cleaning system of claim 53, wherein, The external parameter calibration processor is configured to: For each combination of a reference sensor and a to-be-calibrated sensor, a first conversion map is determined according to the product of the auxiliary local map for each motion interval and the estimated map conversion matrix, and a first conversion trajectory is determined according to the product of the auxiliary local trajectory for each motion interval and the estimated trajectory conversion matrix. A specified map deviation between the reference local map corresponding to each motion interval and the first conversion map is determined, a first norm corresponding to the specified map deviation and a first square value of the first norm are determined; The value of the estimated map conversion matrix is optimized so that the first square value is minimized; A specified trajectory deviation between the reference local trajectory corresponding to each motion interval and the first conversion trajectory is determined, a second norm corresponding to the specified trajectory deviation and a second square value of the second norm are determined; The value of the estimated trajectory conversion matrix is optimized so that the second square value is minimized; If the first square value with the minimum value is less than a first preset error threshold and the second square value with the minimum value is less than a second preset error threshold, the auxiliary local map and the reference local map are determined as an effective map group with successful registration, and the auxiliary local trajectory and the reference local trajectory are determined as an effective trajectory group with successful registration.

55. The cleaning system of claim 53, wherein, The local perception deviation includes a local map deviation and a local trajectory deviation; The extrinsic parameter calibration processor is configured to: According to the effective reference local map, the to-be-solved conversion matrix of each to-be-calibrated sensor to the reference sensor, and the effective auxiliary local map, a local map deviation of each to-be-calibrated sensor relative to the reference sensor in each effective motion interval is determined. According to the effective reference local perception trajectory, the to-be-solved conversion matrix of each to-be-calibrated sensor to the reference sensor, and the effective auxiliary local perception estimation, a local trajectory deviation of each to-be-calibrated sensor relative to the reference sensor in each effective motion interval is determined.

56. The cleaning system of claim 55, wherein, The extrinsic parameter calibration processor is configured to: The integrated perceptual bias is determined based on the following equation: wherein, l a1 representing the integrated perceptual bias, an accumulated value of M local map biases corresponding to M said effective motion intervals, An accumulated value of M local trajectory deviations corresponding to M effective motion intervals.

57. The cleaning system of claim 50, wherein, The extrinsic parameter calibration processor is configured to: For each effective motion interval, an odometer trajectory of the cleaning system is obtained. According to the odometer trajectory, each effective reference local trajectory, and a second to-be-solved conversion matrix of the reference sensor to the odometer coordinate system, a standard local trajectory deviation is determined, and a comprehensive trajectory deviation corresponding to M standard local trajectory deviations is obtained. The value of the second to-be-solved conversion matrix of the reference sensor to the odometer coordinate system is optimized so that the comprehensive trajectory deviation is minimized, and a second conversion matrix of the reference sensor to the odometer coordinate system is solved.

58. The cleaning system of claim 57, wherein, The extrinsic parameter calibration processor is configured to: The each standard local trajectory deviation is determined based on the following equation: wherein, for the ith valid motion interval, l straj representing said each standard local trajectory deviation, a second odometry trajectory T representing the odometry trajectory of the cleaning system, 基准2里程计 a second conversion matrix to be solved T representing the conversion of the reference sensor to the odometry coordinate system, The effective reference local trajectory is represented.

59. The cleaning system of claim 43, wherein, The extrinsic parameter calibration processor is configured to: The first conversion matrix is multiplied by the second conversion matrix to obtain a candidate conversion matrix of each to-be-calibrated sensor to the odometer coordinate system.

60. The cleaning system of claim 43, wherein the global perception data of each to-be-calibrated sensor during the motion of the cleaning system comprises an auxiliary global trajectory of each to-be-calibrated sensor during the motion of the cleaning system. The extrinsic parameter calibration processor is configured to: An identification number corresponding to each to-be-calibrated sensor of the K to-be-calibrated sensors is configured, and the K identification numbers are in an increasing order. When the maximum identification number is K, an xth to-be-calibrated sensor is selected from the K to-be-calibrated sensors in turn, and an x+1st sensor to a Kth sensor is combined with the xth to-be-calibrated sensor respectively to obtain K-x sensor combinations associated with the xth to-be-calibrated sensor; each of the sensor combinations comprises the xth to-be-calibrated sensor and a designated to-be-calibrated sensor; wherein, 1≤x≤K-1; For each sensor combination, according to the auxiliary global trajectory associated with the xth to-be-calibrated sensor, the auxiliary global trajectory associated with the specified to-be-calibrated sensor, the candidate conversion matrix of the xth to-be-calibrated sensor to the odometer coordinate system, and the candidate conversion matrix of the specified to-be-calibrated sensor to the odometer coordinate system, an error expression between the xth to-be-calibrated sensor and the specified to-be-calibrated sensor is determined. determine an overall error expression between the xth sensor to be calibrated and K-x specified sensors to be calibrated according to a sum of K-x error expressions; optimize a value of a candidate conversion matrix of the xth sensor to be calibrated to the odometry coordinate system so that a value of the overall error expression is minimized, and obtain a final conversion matrix of the xth sensor to be calibrated to the odometry coordinate system.

61. The cleaning system of claim 60, wherein, The extrinsic calibration processor is configured to: An error expression between the xth sensor to be calibrated and the designated sensor to be calibrated is determined based on the following equation: wherein e z,y represents an error expression between the xth sensor to be calibrated and the designated sensor to be calibrated, Traj x represents an auxiliary global trajectory associated with the xth sensor to be calibrated, Traj y represents an auxiliary global trajectory associated with the designated sensor to be calibrated, an inverse of a candidate transformation matrix representing the xth sensor to be calibrated to the odometry coordinate system, T y a candidate transformation matrix representing the specified sensor to be calibrated to the odometry coordinate system.

62. The cleaning system of claim 61, wherein, The extrinsic calibration processor is configured to: An overall error expression between the x-th sensor to be calibrated and K-x specified sensors to be calibrated is determined based on the following equation: wherein E represents the overall error expression between the xth sensor to be calibrated and K-x specified sensors to be calibrated.

63. The cleaning system of any one of claims 43 to 62, wherein, The motion interval is obtained according to a motion distance or a motion angle of the cleaning system.

64. A computer readable storage medium having stored thereon a computer program, wherein, The computer program, when executed by a processor, implements the extrinsic calibration method of the sensor according to any one of claims 1 to 21.

65. An electronic device, comprising: comprise: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the extrinsic calibration method of the sensor according to any one of claims 1 to 21 by executing the executable instructions.

Images