Sensor calibration method, apparatus, device and storage medium for a self-moving device

Abstract

The invention relates to a sensor calibration method, device and equipment of self-moving equipment, a storage medium and a computer program product. The method comprises the following steps: acquiring operation data of the self-moving equipment in a current time window; optimizing a numerical value of a preset calibration parameter corresponding to a specified sensor in the self-moving equipment by utilizing the operation data; the preset calibration parameter is an observable parameter screened from the calibration parameters of the specified sensor, and the observable parameter is a parameter which can be estimated based on the operation data of the self-moving device; and outputting a calibration result of the specified sensor according to the optimized numerical value of the preset calibration parameter. By adopting the method, the calibration efficiency of the sensor in the self-moving equipment can be greatly improved.

Description

(19) State Intellectual Property Office (12) Invention Patent Application (10) Application Publication Number (43) Application Publication Date (21) Application Number 202511724872.7 (22) Application Date 2022.08.09 (62) Divisional Application Data 202210952170.4 2022.08.09 (71) Applicant: Chase Innovation Technology (Suzhou) Co., Ltd. Address: Units 1, 2, and 3, Building 8, No. 1688, Songwei Road, Guoxiang Street, Wuzhong Economic Development Zone, Suzhou City, Jiangsu Province, 215000 (72) Inventors: Xu Feng, Yang Sheng, Sheng Tengfei (74) Patent Agency: Huajin United Patent & Trademark Agency Co., Ltd. 44224 Patent Attorney: Lai Xiaojun (51) Int.Cl. G01D 18 / 00 (2006.01) G01C 1 / 00 ​​(2006.01) G06T 7 / 80 (2017.01) G06T 7 / 33 (2017.01) G06V 10 / 75 (2022.01) (54) Invention Title: Sensor Calibration Method, Apparatus, Device, and Storage Medium for Self-Moving Devices (57) Abstract: This application relates to a sensor calibration method, apparatus, device, storage medium, and computer program product for self-moving devices. The method includes: collecting operating data of the self-moving device in the current time window; using the operating data to optimize the values ​​of preset calibration parameters corresponding to a specified sensor in the self-moving device; the preset calibration parameters are observable parameters selected from the calibration parameters of the specified sensor, and the observable parameters refer to parameters that can be estimated based on the operating data of the self-moving device; and outputting the calibration result of the specified sensor according to the optimized values ​​of the preset calibration parameters. This method can greatly improve the calibration efficiency of sensors in self-moving devices. Claims 2 pages, Description 16 pages, Drawings 4 pages, CN 121655595 A 2026.03.13 CN 1 21 65 55 95 A 1. A sensor calibration method for a self-moving device, characterized in that the method includes: collecting operating data of the self-moving device in the current time window, the operating data containing feature points; when the reprojection error of the feature points is greater than a preset error, or the ratio between the number of matching feature points and the number of feature points is less than a preset ratio, optimizing the value of a preset calibration parameter corresponding to a specified sensor in the self-moving device using the operating data; the feature points are feature points in a target keyframe, and the matching feature points are feature points in other keyframes that match the feature points; outputting the calibration result of the specified sensor according to the optimized value of the preset calibration parameter. 2. The method according to claim 1, characterized in that the method further includes:The method comprises: acquiring image information of the self-moving device during operation; determining the pixel positions of feature points in the image information; determining the projection position of the feature points based on the depth information between the self-moving device and the feature points; and calculating the reprojection error of the feature points based on the pixel positions and the projection positions. 3. The method according to claim 1, further comprising: selecting at least two feature points in the target keyframe of the running data; and searching for matching feature points in other keyframes of the running data based on the at least two feature points. 4. The method according to any one of claims 1 to 3, characterized in that, before collecting the operating data of the mobile device in the current time window, the method further includes: performing observability analysis on the calibration parameters of the designated sensor based on the operating data of the mobile device in different operating modes to obtain observable parameters in different operating modes; wherein the observable parameters are preset calibration parameters; associating and storing the preset calibration parameters with the operating modes; the step of optimizing the value of the preset calibration parameters corresponding to the designated sensor in the mobile device using the operating data includes: optimizing the value of the associated and stored preset calibration parameters using the operating data belonging to the operating mode in the current time window. 5. The method according to any one of claims 1 to 3, wherein the self-moving device includes a robotic vacuum cleaner, and the designated sensor includes a camera and an odometer; before collecting the operating data of the self-moving device in the current time window, the method further includes: selecting calibration parameters that can be estimated based on the operating data of the robotic vacuum cleaner from the calibration parameters of the camera and the odometer; wherein the selected calibration parameters are preset calibration parameters; associating the preset calibration parameters with the camera and the odometer, and storing the preset calibration parameters; optimizing the value of the preset calibration parameters corresponding to the designated sensor in the self-moving device using the operating data includes: optimizing the value of the stored preset calibration parameters using the operating data. 6. The method according to claim 5, characterized in that, the step of selecting calibration parameters that can be estimated based on the operating data of the sweeping robot from the calibration parameters of the camera and the odometer includes: performing observability analysis on the rotation parameters and translation of the camera and the odometer based on the operating data of the sweeping robot, and obtaining observability analysis results; if the observability analysis results indicate that the rotation parameter is observable, then the rotation parameter is selected; wherein, the rotation parameter is a calibration parameter that can be estimated based on the operating data of the sweeping robot.7. A sensor calibration device for a self-moving device, characterized in that the device comprises: a data acquisition module, configured to acquire operating data of the self-moving device in a current time window, the operating data including feature points; an optimization module, configured to optimize the value of a preset calibration parameter corresponding to a specified sensor in the self-moving device using the operating data when the reprojection error of the feature points is greater than a preset error, or the ratio between the number of matching feature points and the number of feature points is less than a preset ratio; wherein the feature points are feature points in a target keyframe, and the matching feature points are feature points in other keyframes that match the feature points; and a determination module, configured to output the calibration result of the specified sensor based on the optimized value of the preset calibration parameter. 8. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, implements the steps of the method according to any one of claims 1 to 6. 9. A computer-readable storage medium storing a computer program thereon, characterized in that the computer program, when executed by a processor, implements the steps of the method according to any one of claims 1 to 6. 10. A computer program product, comprising a computer program, characterized in that, when executed by a processor, the computer program implements the steps of the method described in any one of claims 1 to 6. Claims 2 / 2 Page 3 CN 121655595 A Sensor calibration method, apparatus, device and storage medium for self-moving devices

[0001] This application is a divisional application filed with the Chinese Patent Office on August 9, 2022, with application number 2022109521704, entitled "Sensor calibration method, apparatus, device and storage medium for self-moving devices", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of data processing technology for self-moving devices, and in particular to a sensor calibration method, apparatus, device and storage medium for self-moving devices. Background Art

[0003] A robotic vacuum cleaner is a self-moving device used in homes to clean indoor environments. It can automatically clean without supervision and plans its cleaning path based on the indoor environment. This involves using a camera on the self-moving device to collect indoor image information. The central processing unit (CPU) of the self-moving device can calculate the geometric information of objects in the three-dimensional space of the room, thereby reconstructing or recognizing objects, realizing the understanding of the indoor environment, and thus completing the cleaning work.

[0004] In order for the self-moving device to accurately perform cleaning work, its sensors need to be calibrated. Traditional calibration methods usually require cumbersome manual operations to complete the calibration process, greatly reducing the efficiency of self-moving device calibration. Summary of the Invention

[0005] Based on this, it is necessary to provide a sensor calibration method, apparatus, device, and storage medium for a self-moving device to address the above-mentioned technical problems, which can greatly improve the calibration efficiency of sensors in self-moving devices.

[0006] In a first aspect, this application provides a sensor calibration method for a self-moving device, the method comprising:

[0007] collecting operating data of the self-moving device in the current time window;

[0008] using the operating data to optimize the value of a preset calibration parameter corresponding to a specified sensor in the self-moving device; the preset calibration parameter is an observable parameter selected from the calibration parameters of the specified sensor, the observable parameter referring to a parameter that can be estimated based on the operating data of the self-moving device;

[0009] outputting the calibration result of the specified sensor according to the optimized value of the preset calibration parameter.

[0010] In one embodiment, optimizing the value of the preset calibration parameter corresponding to the specified sensor in the self-moving device comprises:

[0011] optimizing the value of the preset calibration parameter corresponding to the specified sensor in the self-moving device when the feature points in the operating data in the current time window meet preset conditions.

[0012] In one embodiment, the feature points in the running data under the current time window are determined to meet the preset conditions by at least the following methods:

[0013] Image information of the self-moving device when it is working is collected;

[0014] The pixel position of the feature points in the image information is determined;

[0015] The projection position of the feature points is determined based on the depth information between the self-moving device and the feature points;

[0016] The reprojection error of the feature points is calculated based on the pixel position and the projection position;

[0017] When the reprojection error is greater than the preset error, the feature points meet the preset conditions.

[0018] In one embodiment, the feature points in the running data under the current time window are determined to meet a preset condition using at least the following methods:

[0019] Selecting at least two feature points in the target keyframe of the running data;

[0020] Searching for matching feature points in other keyframes of the running data based on the at least two feature points;

[0021] When the ratio between the number of matching feature points and the number of at least two feature points is less than a preset ratio, the feature points meet the preset condition.

[0022] In one embodiment, the method further includes:

[0023] Real-time acquisition of the yaw angle of the self-moving device during the optimization process;

[0024] Calculating the difference between a preset number of yaw angles to obtain the difference in the yaw angle of the self-moving device during the optimization process;

[0025] When the difference satisfies the difference condition, the optimized value of the preset calibration parameter is obtained.

[0026] In one embodiment, the difference includes variance or standard deviation; the step of outputting the calibration result of the specified sensor based on the optimized value of the preset calibration parameter includes:

[0027] When the variance of the yaw angle of the self-moving device during the optimization process is not greater than the preset variance, or the standard deviation of the yaw angle of the self-moving device during the optimization process is not greater than the preset standard deviation, the optimization of the value of the preset calibration parameter is stopped, and the optimized value of the preset calibration parameter is obtained;

[0028] The step of outputting the calibration result of the specified sensor based on the optimized value of the preset calibration parameter includes:

[0029] The optimized value of the preset calibration parameter is used as the calibration result of the specified sensor.

[0030] In one embodiment, before optimizing the value of the preset calibration parameter corresponding to the specified sensor in the self-moving device, the method further includes:

[0031] obtaining a calibration parameter sample dataset associated with the specified sensor; wherein, the calibration parameter sample dataset contains the calibration result of the specified sensor in a sample device; the sample device refers to a device with the same batch identifier as the self-moving device;

[0032] initializing the value of the preset calibration parameter corresponding to the specified sensor in the self-moving device using the calibration parameter sample dataset associated with the specified sensor, to obtain the initialization parameter value of the preset calibration parameter;

[0033] optimizing the value of the preset calibration parameter corresponding to the specified sensor in the self-moving device using the running data includes: optimizing the initialization parameter value of the preset calibration parameter using the running data.

[0034] In one embodiment, optimizing the value of the preset calibration parameter corresponding to the specified sensor in the self-moving device using the running data includes:

[0035] Optimizing the value of the preset calibration parameter corresponding to the specified sensor in the self-moving device according to the running data and the objective function; wherein, the objective function is used to characterize the correlation between the value of the preset calibration parameter and the running data of the self-moving device; the objective function is constructed under the constraint of the least squares function, and the least squares function is used to characterize the error between the predicted value of the preset calibration parameter and the parameter label value. Specification 2 / 16 pages 5 CN 121655595 A

[0036] In a second aspect, this application also provides a sensor calibration device for a self-moving device, the device including:

[0037] a data acquisition module, used to acquire running data of the self-moving device in the current time window;

[0038] an optimization module, used to optimize the value of the preset calibration parameter corresponding to the specified sensor in the self-moving device using the running data.The preset calibration parameters are optimized; the preset calibration parameters are observable parameters selected from the calibration parameters of the specified sensor, and the observable parameters refer to parameters that can be estimated based on the operating data of the self-moving device;

[0039] The determining module is used to output the calibration result of the specified sensor according to the optimized preset calibration parameters.

[0040] In one embodiment, the optimization module is further used to optimize the values ​​of the preset calibration parameters corresponding to the specified sensor in the self-moving device when the feature points in the operating data under the current time window meet the preset conditions.

[0041] In one embodiment, the device further includes:

[0042] the acquisition module, further configured to acquire image information of the self-moving device during operation;

[0043] the determination module, further configured to determine the pixel position of the feature point in the image information; and determine the projection position of the feature point based on the depth information between the self-moving device and the feature point;

[0044] a first calculation module, configured to calculate the reprojection error of the feature point based on the pixel position and the projection position; and when the reprojection error is greater than a preset error, the feature point satisfies the preset condition.

[0045] In one embodiment, the device further includes:

[0046] a selection module, configured to select at least two feature points in the target keyframe of the running data;

[0047] a search module, configured to search for matching feature points in other keyframes of the running data based on the at least two feature points; and when the ratio between the number of matching feature points and the number of at least two feature points is less than a preset ratio, the feature point satisfies the preset condition.

[0048] In one embodiment, the device further includes:

[0049] the acquisition module, further configured to acquire the yaw angle of the self-moving device in real time during the optimization process;

[0050] the second calculation module, configured to perform difference calculation on a preset number of yaw angles to obtain the difference of the yaw angle of the self-moving device during the optimization process; when the difference satisfies the difference condition, obtain the value of the optimized preset calibration parameter.

[0051] In one embodiment, the difference includes variance or standard deviation;

[0052] the second calculation module, further configured to stop optimizing the value of the preset calibration parameter when the variance of the yaw angle of the self-moving device during the optimization process is not greater than a preset variance, or the standard deviation of the yaw angle of the self-moving device during the optimization process is not greater than a preset standard deviation, and obtain the value of the optimized preset calibration parameter;

[0053] the determination module, further configured to use the value of the optimized preset calibration parameter as the calibration result of the specified sensor.

[0054] In one embodiment, the device further includes:

[0055] An acquisition module is used to acquire a calibration parameter sample dataset associated with the specified sensor; wherein the calibration parameter sample dataset contains the calibration results of the specified sensor in a sample device; the sample device refers to a device with the same batch identifier as the self-moving device;

[0056] An initialization module is used to initialize the value of the preset calibration parameter corresponding to the specified sensor in the self-moving device using the calibration parameter sample dataset associated with the specified sensor, to obtain the initialization parameter value of the preset calibration parameter; Specification 3 / 16 page 6 CN 121655595 A

[0057] The optimization module is further used to optimize the initialization parameter value of the preset calibration parameter using the running data.

[0058] In one embodiment, the optimization module is further configured to optimize the value of a preset calibration parameter corresponding to a specified sensor in the self-moving device according to the running data and the objective function; wherein the objective function is used to characterize the correlation between the value of the preset calibration parameter and the running data of the self-moving device; the objective function is constructed under the constraint of a least squares function, and the least squares function is used to characterize the error between the predicted value of the preset calibration parameter and the parameter label value.

[0059] In a third aspect, this application also provides a computer device. The computer device includes a memory and a processor, and the processor executes the computer program to implement the steps in the sensor calibration method for the self-moving device provided in the above aspects.

[0060] In a fourth aspect, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, and the computer program, when executed by a processor, implements the steps in the sensor calibration method for the self-moving device provided in the above aspects.

[0061] In a fifth aspect, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps in the sensor calibration method for the self-moving device provided in the above aspects.

[0062] The sensor calibration method for the self-moving device provided in this application embodiment, by pre-selecting calibration parameters that can be estimated based on operating data from the calibration parameters of the sensor as preset calibration parameters, and then using the operating data of the self-moving device to perform online calibration of the pre-selected preset calibration parameters, can achieve rapid online calibration of at least some calibration parameters, improving the simplicity and efficiency of sensor calibration; at the same time, it can also realize timely correction of the preset calibration parameters based on the real-time operating data of the self-moving device, ensuring the real-time accurate representation of the relative position relationship of the sensors, thereby improving the accuracy of self-moving device control. Brief Description of the Drawings

[0063] Figure 1 is an application environment diagram of the sensor calibration method for the self-moving device in one embodiment;

[0064] Figure 2 is a flowchart illustrating a sensor calibration method for a self-moving device in one embodiment;

[0065] Figure 3 is a flowchart illustrating the initialization and optimization of preset calibration parameters corresponding to a specified sensor in a self-moving device in one embodiment;

[0066] Figure 4 is a structural block diagram of a sensor calibration device for a self-moving device in one embodiment;

[0067] Figure 5 is a structural block diagram of a sensor calibration device for a self-moving device in one embodiment;

[0068] Figure 6 is an internal structural diagram of a computer device in one embodiment. Detailed Description

[0069] To make the objectives, technical solutions, and advantages of this application clearer, the following description, in conjunction with the accompanying drawings and embodiments, will provide a more detailed explanation of this application. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0070] The sensor calibration method for a self-moving device provided in this application embodiment can be applied to the application environment shown in Figure 1. The self-moving device 102 communicates with the server 104 and the terminal 106 via a network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on server 104, or it can be placed on cloud manual page 4 / 16 7 CN 121655595 A or other network servers.

[0071] The self-moving device 102 can be a cleaning device or other mobile device that automatically cleans the indoor environment. The cleaning device can be a sweeping robot, mopping robot, vacuum cleaner, or sweeping and mopping robot.

[0072] When a system update is required, server 104 can establish a network connection between self-moving device 102 and server 104 based on user operation, and then download the updated version of the system for update. The server 104 can be an independent physical server or a server cluster composed of multiple physical servers. It can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery network (CDN), and big data and artificial intelligence platforms.

[0073] The terminal 106 can be a smartphone, tablet, laptop, desktop computer, smartwatch, IoT device, or portable wearable device, etc. Users can use the terminal 106 to control the self-moving device to clean the entire indoor environment or a specific area within that environment.

[0074] In one scenario example provided in this specification, the calibration of the camera and odometer of a robotic vacuum cleaner is illustrated below. During operation, the robotic vacuum cleaner typically needs to use both the camera and odometer to estimate the movement of the machine.Human movement is a crucial factor, therefore, accurate calibration between the camera and the odometer is essential for improving the stability of the robot vacuum's operation. Traditional calibration methods typically require complex manual operations to complete the calibration of the camera and odometer, resulting in low efficiency and cumbersome procedures.

[0075] In this scenario example, based on observability analysis, calibration parameters that can be estimated from the robot vacuum's operating data can be selected as preset calibration parameters. These preset calibration parameters can be associated with and stored with the camera and odometer.

[0076] Subsequently, the preset calibration parameters can be calibrated online using the robot vacuum's operating data. This online calibration can be performed in real-time as the robot vacuum operates. If the online calibration is performed in real-time as the robot vacuum operates, the real-time operating data of the robot vacuum can be used to optimize the preset calibration parameters online, further ensuring the accuracy of the sensor calibration results upon which the robot vacuum's operation depends, thereby improving the stability of the robot vacuum's operation. Of course, the online calibration can also be performed at a specified time point or according to the user's instructions, and this specification does not limit it.

[0077] During the online calibration of the camera and odometer, the operating stability of the sweeping robot under the corresponding time window can be judged based on the operating data of the sweeping robot. If the operation of the self-moving device is relatively stable, the preset calibration parameters do not need to be optimized; while if the operating stability of the self-moving device is poor, the preset calibration parameters are optimized. For example, the reprojection error of the feature points of the sweeping robot under a certain time window can be used to determine whether the preset calibration parameters need to be optimized. When it is determined that optimization is needed, the operating data under that time window is used to optimize the preset calibration parameters. The data processing volume involved in sensor calibration is usually large. In this scenario example, by analyzing the actual operation of the self-moving device under a certain time window, it is determined whether the preset calibration parameters need to be optimized under that time window, which can effectively reduce the data processing volume while ensuring the operating stability of the self-moving device.

[0078] When it is determined that the preset calibration parameters need to be optimized, the operating data such as the robot's running position, obstacle position, and distance value within the time window can be used to optimize the values ​​of the preset calibration parameters corresponding to the camera and odometer, obtaining the optimized preset calibration parameter values ​​as the calibration results of the odometer and camera. Afterwards, the robot can perform operation control based on the calibration results of the camera and odometer, thereby improving the stability of the robot's operation. (Instruction manual, page 5 / 16, 8 CN 121655595 A)

[0079] In this scenario example, by pre-selecting the calibration parameters of the camera and odometer that can be estimated based on the operating data...The calibration parameters of the device are used as preset calibration parameters. Then, the operating data of the self-moving device is used to perform online calibration of the preset calibration parameters, which can realize the rapid online calibration of at least some calibration parameters, improve the simplicity and efficiency of camera and odometer parameter calibration; at the same time, the data of the preset calibration parameters can be corrected in a timely manner based on the real-time operating data of the self-moving device, ensuring the real-time accurate representation of the relative position relationship of the sensors, thereby improving the accuracy of self-moving device control.

[0080] Based on the above scenario example, this specification embodiment provides a sensor calibration method for a self-moving device. The method can be applied to the server that establishes a communication connection with the self-moving device, or to the terminal that establishes a communication connection with the self-moving device, or to the self-moving device. Taking the application of this method to the self-moving device in Figure 1 as an example, as shown in Figure 2, the method includes at least the following steps:

[0081] S202, collect the operating data of the self-moving device under the current time window.

[0082] The operating data of the self-moving device over time can be extracted using the time window to obtain the operating data under the corresponding time window. The time length covered by the time window can be set as needed. For ease of description, the length of time covered by the time window can be described as the window length. When the running data is a video data stream, the window length can also be measured by the number of keyframes. For example, if the number of keyframes is 5, then the window length is 5. A keyframe can refer to a video frame sampled at fixed intervals in a video frame sequence, such as sampling one video frame every 2 frames as a keyframe. Of course, when the running data is other types of data, the window length of the time window can be characterized in other ways, which are not limited in this specification.

[0083] The current time window can be the time window on which the current sensor calibration is based. The running data extracted based on the current time window can be the real-time running data of the self-moving device or the historical running data.

[0084] The running data can include the data collected by the built-in sensors of the self-moving device during operation, and can also include the status data, relative position data, etc. of the self-moving device collected by other devices during operation. Such as the position, running time, attitude of the self-moving device, and video frames collected by the camera, etc. The types of running data used for calibration of different calibration parameters usually differ to some extent. The types of running data required for calibrating a specific calibration parameter can be analyzed in advance, and the running data types can be associated and stored with the calibration parameters. Correspondingly, during online calibration, only the data of the running data type corresponding to the calibration parameter is extracted for calibration, reducing the amount of running data collected and further improving calibration processing efficiency.

[0085] The collection of running data can be performed using a sliding time window or randomly. Of course,Other methods can also be used, and this specification does not limit them. For example, a time window of the target window length can be extracted from a certain time point as the current time window, and the running data of the self-moving device under the current time window can be collected. For example, during the operation of the sweeping robot, the position information, running time information, posture information and video frame information of the sweeping robot under the time window with a window length of 5 are collected; among them, the video frame is the video key frame, since the window length is represented by the key frame. Then, the time window is slid, such as sliding the time window once every 3 seconds, and the time window is used as the current time window in turn, and the running data of the self-moving device under the current time window is collected.

[0086] S204, using the running data, the values ​​of the preset calibration parameters corresponding to the specified sensors in the self-moving device are optimized.

[0087] The specified sensor can refer to the sensor that needs to be calibrated, such as the odometer, camera, obstacle detector, distance detector, and the speed sensor and angle sensor of the self-moving device. The specified sensor can be a single sensor, that is, the calibration of the relative position relationship of a single sensor with respect to the body of the self-moving device. For example, when calibrating the position of the odometer relative to the device body, the designated sensor refers to the odometer. Alternatively, the designated sensor can also be the calibration of two or more sensors, i.e., the calibration of the relative positional relationship between two or more sensors. For example, when calibrating the relative positional relationship between the camera and the odometer, the designated sensor refers to both the camera and the odometer.

[0088] The preset calibration parameters are observable parameters selected from the calibration parameters of the designated sensors. Observable parameters refer to parameters that can be estimated based on the operating data of the self-moving device. For example, observability analysis can be used to analyze the calibration parameters of the designated sensors and select calibration parameters that can be estimated based on the operating data of the self-moving device as preset calibration parameters. The observability analysis can be performed using methods such as nonlinear system observability analysis, VI-SLAM system observability analysis, or INS system observability analysis based on geometric features. For example, for the odometer of a robotic vacuum cleaner, the preset calibration parameters may include at least one of extrinsic parameters (such as rotation parameters), map points, and attitude. The map points may refer to the position of the robotic vacuum cleaner on a map, which is a map of the indoor environment.

[0089] For example, during the operation of the robotic vacuum cleaner, the observability analysis of the calibration parameters of a specified sensor is performed by adding global coordinate system X-direction positioning. At this time, it can be determined that the global positioning X-direction and yaw angle are observable parameters; similarly, the observability analysis of the calibration parameters of a specified sensor is performed by adding global coordinate system Y-direction positioning. At this time, it can be determined that the global positioning Y-direction...The Z-direction and yaw angle are observable parameters. When the global coordinate system Z-direction positioning is added, an observability analysis is performed on the calibration parameters of the specified sensor. At this point, it can be determined that the global positioning Z-direction is an observable parameter, and the yaw angle is an unobservable parameter. Furthermore, when the robot vacuum cleaner performs pure rotational motion, an observability analysis of the calibration parameters of the specified sensor determines that the scale of the feature points is unobservable; when the robot vacuum cleaner performs constant acceleration motion, an observability analysis of the calibration parameters of the specified sensor determines that the scale of the feature points is unobservable. Through the above observability analysis, observable parameters can be selected from the calibration parameters of the specified sensor as preset calibration parameters.

[0090] For example, regarding the calibration of the robot vacuum cleaner's odometer and camera, the intrinsic parameters of the odometer and camera generally do not change significantly and can be calibrated offline. The online calibration process can mainly calibrate the extrinsic parameters of the odometer and camera. The extrinsic parameters of the odometer and camera include rotation parameters and translation parameters. The rotation parameters characterize the rotational transformation between the camera coordinate system and the odometer coordinate system, while the translation parameters characterize the translational transformation between the camera coordinate system and the odometer coordinate system.

[0091] The observability of the rotation and translation parameters of the odometer and camera of the robotic vacuum cleaner can be analyzed based on the operating data. Since the robotic vacuum cleaner mostly moves in a straight line, the observability of the rotation and translation parameters of the odometer and camera can be analyzed based on the operating data of the robotic vacuum cleaner during straight-line movement. Ultimately, it is determined that the translational transformations of the X, Y, and Z axes are not observable, while the rotation parameters are observable. Accordingly, the rotation parameters can be used as preset calibration parameters for online calibration. Alternatively, by analyzing the robotic vacuum cleaner in other operating modes, the operating mode for which the translation parameters can be calibrated can be determined, and the translation parameters can be calibrated online based on the operating data in that operating mode.

[0092] Accordingly, observability analysis can be performed on the calibration parameters of a specified sensor in advance based on the operating data under different operating modes to determine the observable parameters of the calibration parameters under different operating modes, and the selected observable parameters can be used as preset calibration parameters and associated with the corresponding operating modes for storage. For example, for the above example, the linear motion, odometer, and camera rotation parameters can be associated and stored. Subsequently, the preset calibration parameters of the specified sensor can be calibrated online based on the operating data of the sweeping robot in the corresponding operating mode. For the above example, the operating data corresponding to the linear motion of the sweeping robot can be obtained to calibrate the odometer and camera rotation parameters online. By distinguishing the operating modes to select observable parameters, the selection of observable parameters can be made more accurate and comprehensive; at the same time, calibrating the observable parameters based on the operating data under the corresponding operating modes can also make the observable parameter calibration more accurate.

[0093] Of course, if it is determined through analysis that the selection of observable parameters in the calibration parameters is unrelated to the operating mode, then the selection of observable parameters can be performed online based on the operating data under the corresponding operating mode.The selected preset calibration parameters are specially labeled. During online calibration, the preset calibration parameters can be calibrated without distinguishing the operating mode.

[0094] During the online calibration process, the pre-extracted preset calibration parameters can be obtained, and the values ​​of the preset calibration parameters corresponding to the specified sensor can be optimized using the collected operating data to ensure the real-time accurate representation of the relative position relationship of the sensors, thereby improving the accuracy of the self-moving device control.

[0095] For example, the values ​​of the preset calibration parameters can be optimized using a pre-constructed objective function and operating data. The objective function can be used to characterize the correlation between the values ​​of the preset calibration parameters and the operating data of the self-moving device.

[0096] The preset calibration parameters of the specified sensor can be calibrated offline first, and the calibration results can be stored as the parameter label values ​​of the preset calibration parameters. Then, the operating data of the sweeping robot under stable operation based on the above parameter label values ​​can be obtained, and the objective function can be constructed using the operating data and parameter label values. In the process of constructing the objective function, the least squares function can also be used for constraints to accelerate the convergence of the objective function. The least squares function can be used to characterize the error between the value predicted by the preset calibration parameter based on the running data and the parameter label value. For ease of description, the value predicted by the preset calibration parameter based on the running data can be simplified as the predicted value of the preset calibration parameter in the following text.

[0097] Based on the predicted value of the preset calibration parameter and the parameter label value, the objective function is updated until the value of the pre-constructed least squares function meets the preset value condition, and the objective function is obtained. Since the least squares function can characterize the error between the predicted value of the preset calibration parameter and the parameter label value, this error continuously decreases during the update process of the objective function, which can make the finally constructed objective function more accurately characterize the correlation between the running data and the parameter label value.

[0098] For example, the least squares function can be the sum of squares of the differences between the predicted values ​​of the preset calibration parameters and the parameter label values ​​(true values):

[0099]

[0100] Where, represents the predicted values ​​of the preset calibration parameters, and represents the parameter label values ​​of the preset calibration parameters.

[0101] The expression of the objective function can be, or can be, where represents each running data, and , , , ..., are the function coefficients respectively. In subsequent embodiments, let the expression of the objective function be.

[0102] Converting the above objective function into matrix form, we have X, therefore the expression of the least squares function is: .

[0103] Therefore, the least squares function can be regarded as a system of quadratic equations about , solving for the minimum value of the least squares function L, because the minimum value of L corresponds to the optimal value, so the value is continuously adjusted to make L the minimum.

[0104] The least squares function is considered as a function of L with respect to the variable. The first derivative of the least squares function is calculated to obtain:

[0105]

[0106] The above formula is simplified to obtain: Specification 8 / 16 page 11 CN 121655595 A

[0107]

[0108] Therefore, according to the above simplified formula, the value of the input to the target function is calculated, and then the value of the least squares function L is calculated. When the value of the least squares function L meets the preset value condition, the target function is obtained. Then, the constructed target function and running data can be used to optimize the value of the preset calibration parameters.

[0109] Of course, the above optimization method of the preset calibration parameters is a preferred example. The preset calibration parameters can also be optimized in other ways, such as referring to the calibration parameter data optimization method used in the traditional offline calibration process to optimize the preset calibration parameters. This is not limited here.

[0110] In one embodiment, the value of a preset calibration parameter corresponding to a specified sensor in the self-moving device can be optimized if the feature points in the running data under the current time window meet preset conditions.

[0111] After obtaining the preset calibration parameter, it can be determined whether the preset calibration parameter should be optimized based on whether the feature points in the running data under the current time window meet the preset conditions. When the feature points meet the preset conditions, the preset calibration parameter needs to be optimized; when the feature points do not meet the preset conditions, the preset calibration parameter does not need to be optimized.

[0112] The feature points can also be data points extracted from the running data that meet certain feature requirements. For example, when the running data includes image information, the feature points can refer to pixels in the image that meet certain feature requirements. For example, the feature points can be points where the image grayscale value changes drastically or points with large curvature on the image edge. Feature points in the image can be extracted using methods such as SIFT, SURF, and ORB. The preset conditions correspond to the analysis method used when analyzing the feature points of the running data. When the running data is laser point cloud data, the feature points can be data points extracted from the point cloud data that meet certain feature requirements.

[0113] Sensor calibration involves a large amount of data, especially for real-time calibration, which will occupy too much of the operating resources of the self-moving device. Generally, if the self-moving device operates relatively stably, the calibration parameter data of the relative position relationship between the sensors of the self-moving device usually will not have a big problem; if the self-moving device operates poorly in the current time window, there may be a problem that the calibration parameter data cannot accurately reflect the relative position relationship between the sensors of the self-moving device. By analyzing the actual operation of the self-moving device in the current time window, it can be determined whether it is necessary to calibrate the sensor.Sensor parameter calibration can effectively reduce data processing volume while ensuring the stability of self-moving device operation. Determining the self-moving device's operating status based on feature points can further reduce the data processing volume and resource consumption; simultaneously, feature points can more accurately reflect the characteristics of operating data, reducing noise interference, thereby further improving the accuracy of self-moving device operating status determination.

[0114] In some embodiments, the determination of whether preset calibration parameters need optimization can be made by using the reprojection error of feature points, or by judging the observability of feature points in other key frames, as described below:

[0115] Method 1: Determine whether preset calibration parameters need optimization by using whether the reprojection error of feature points meets preset conditions.

[0116] The reprojection error of feature points can be calculated first, and then compared with the preset error. If the reprojection error is less than or equal to the preset error, the preset calibration parameter does not need optimization; if the reprojection error is greater than the preset error, the preset calibration parameter needs optimization.

[0117] For example, the calculation steps for reprojection error may include: acquiring image information collected by the built-in camera of the self-moving device during operation; determining the pixel position of the feature point in the image information; determining the projection position of the feature point based on the depth information between the self-moving device and the feature point; calculating the reprojection error of the feature point based on the pixel position and the projection position; when the reprojection error is greater than a preset error, the feature point meets the preset condition, indicating that the preset calibration parameter needs to be optimized, and at this time the preset calibration parameter is used as the preset calibration parameter for optimization.

[0118] Wherein, the image information can be a key frame based on the camera, that is, a video key frame. The depth information can refer to the distance between the self-moving device and the feature point in the world coordinate system. It should be noted that the number of feature points can be one or more.

[0119] For example, suppose the relationship between the pixel position of a feature point and the spatial point position corresponding to the feature point is as follows:

[0120]

[0121] Where, is depth information, is the pixel position (i.e., pixel coordinates) of the feature point, represents the position of the feature point in the world coordinate system, is the projection position (i.e., projection coordinates) of the feature point, representing the transformation of the feature point from the world coordinate system to the camera coordinate system, and K is a coefficient.

[0122] Writing the above relationship in matrix form, we can obtain: .

[0123] Due to noise issues, the above relationship has an error. Therefore, by transforming the above relationship, we can obtain the following error relationship:

[0124]

[0125] Based on the above error relationship, the error can be calculated, which is the reprojection error of the feature point.The reprojection error not only considers the calculation error of the homography matrix, but also the measurement error of the feature points, so its accuracy is high. Therefore, it can accurately determine whether to optimize the preset calibration parameters, which is beneficial to improving the optimization effect.

[0126] Method 2: Determine whether to optimize the preset calibration parameters based on whether the observability of the feature points of a certain key frame in other key frames meets the preset conditions.

[0127] Select at least two feature points in the target key frame of the running data; based on the at least two feature points, search for matching feature points in other key frames of the running data; when the ratio between the number of matching feature points and the number of at least two feature points is greater than or equal to the preset ratio, the feature points meet the preset conditions.

[0128] Specifically, the running data may include video frames captured by the camera. Therefore, key frames are sampled from the captured video frames, and then feature points are selected in a certain target key frame. Then, it is queried in other key frames whether there are feature points that match the feature points (i.e., matching feature points). If matching feature points are found in other key frames, it means that the feature points in the target key frame are observable in other key frames. When the ratio between the number of matched feature points and the number of extracted feature points is less than a preset ratio, such as when the feature point ratio is less than 80%, it indicates that the feature points meet the preset conditions, and it is determined that the preset calibration parameters of this specification (page 10 / 16, CN 121655595 A) need to be optimized; when the ratio between the number of matched feature points and the number of extracted feature points is greater than or equal to the preset ratio, such as when the feature point ratio is greater than 80%, it is determined that the preset calibration parameters do not need to be optimized.

[0129] For example, for keyframe a and keyframe b, assuming that feature points 1 to 10 are selected in keyframe a, and then in keyframe b, it is queried whether there are feature points that match feature points 1 to 10. If 9 matching feature points are found, it is determined that the preset calibration parameters do not need to be optimized; if only 7 matching feature points are found, it is determined that the preset calibration parameters need to be optimized. When determining whether to optimize the preset calibration parameters, it is not necessary to use the entire running data for judgment. It is only necessary to judge the feature point matching of the key frames in the running data. Therefore, even if there is a lot of data in the running data, it is possible to accurately determine whether the optimization result is satisfactory, and it can also effectively reduce the amount of computation.

[0130] S206, output the calibration result of the specified sensor according to the value of the optimized preset calibration parameters.

[0131] The value of the optimized preset calibration parameters can be output as the calibration result of the specified sensor. Alternatively, during the optimization process of the preset calibration parameters, it can be determined whether the value of the optimized preset calibration parameters meets the optimization conditions. When the optimization conditions are met, the optimization process is stopped, and the value of the optimized preset calibration parameters is obtained as the calibration result of the specified sensor.

[0132] For example, for the calibration of rotation parameters of cameras and odometers, the three-axis rotation is usually calibrated together. Therefore, the error of any one axis will affect the calibration of other axes. The accuracy of the three-axis rotation calibration can be determined by analyzing the running direction of the robot vacuum cleaner.

[0133] In some embodiments, the process of judging whether the value of the optimized preset calibration parameter meets the optimization conditions can be judged by the difference of the yaw angle. Specifically, it is as follows:

[0134] During the optimization process, the yaw angle of the self-moving device is collected in real time; the difference of the preset number of yaw angles is calculated to obtain the difference of the yaw angle of the self-moving device during the optimization process; when the difference meets the difference condition, the value of the optimized preset calibration parameter is obtained.

[0135] For example, during the optimization process, the yaw angle of the self-moving device can be collected in real time and then stored in a buffer. When the number of stored yaw angles reaches the preset number, the collection of yaw angles is stopped and the difference calculation is started, thereby avoiding the collection of too many yaw angles and prolonging the optimization time.

[0136] The difference can be variance or standard deviation. When the variance of the yaw angle of the self-moving device during the optimization process is not greater than the preset variance, or the standard deviation of the yaw angle of the self-moving device during the optimization process is not greater than the preset standard deviation, the optimization of the preset calibration parameter value is stopped, and the optimized preset calibration parameter value is obtained; then, the optimized preset calibration parameter value is used as the calibration result of the specified sensor.

[0137] For example, after obtaining a preset number of yaw angles of the sweeping robot, the variance or standard deviation of the yaw angle of the sweeping robot during the optimization process is calculated, and then the variance of the yaw angle is compared with the preset variance, or the standard deviation of the yaw angle is compared with the preset standard deviation, so as to determine whether to stop the optimization process based on the comparison result. During the optimization of the preset calibration parameters, the yaw angle is greatly affected by the preset calibration parameters. Using a preset number of yaw angles to evaluate whether to stop the optimization process can accurately determine whether the preset calibration parameters have reached the optimal value, thereby avoiding excessive yaw angle collection and extending the optimization time, and also improving the accuracy of parameter calibration.

[0138] Alternatively, in some other embodiments, the attitude of the self-moving device can be collected in real time during the optimization process; the difference between a preset number of attitudes can be calculated to obtain the difference in the attitude of the self-moving device during the optimization process; when the difference meets the difference condition, the value of the optimized preset calibration parameters can be obtained. When optimizing the value of the preset calibration parameters, the attitude of the self-moving device can be optimized. Therefore, directly using the difference between a preset number of attitudes to evaluate whether to stop the optimization process can avoid excessive attitude collection and extending the optimization time, and can also ensure the accuracy of attitude optimization.Accuracy.

[0139] For example, during the optimization process, the posture of the sweeping robot can be collected in real time and stored in a buffer. When the number of stored postures reaches a preset number, the posture collection stops and the difference calculation begins, thereby avoiding excessive posture collection and extending the optimization time.

[0140] In the above embodiments, by pre-selecting calibration parameters that can be estimated based on the running data from the calibration parameters of the sensor as preset calibration parameters, and then using the running data of the self-moving device to perform online calibration of the pre-selected preset calibration parameters, at least some calibration parameters can be quickly calibrated online, improving the simplicity and efficiency of sensor calibration; at the same time, the preset calibration parameters can be corrected in a timely manner based on the real-time running data of the self-moving device, ensuring the real-time accurate representation of the relative position relationship of the sensor, thereby improving the accuracy of the self-moving device control.

[0141] In some other embodiments, as shown in FIG3, before S204, the method further includes:

[0142] S302, obtaining the calibration parameter sample dataset associated with the specified sensor.

[0143] The calibration parameter sample dataset contains the calibration results of the specified sensors in the sample device; the sample device refers to the device with the same batch identifier as the self-moving device. The batch identifier refers to the information that identifies the production batch of the device. Typically, devices in the same production batch have similar device parameter information. For example, a sweeping robot from the same batch as the target sweeping robot can be used as a sample to obtain the preset calibration parameters corresponding to the camera, odometer, and other sensors in the sample, and the preset calibration parameters can be associated and stored with the sensors; at the same time, the calibration results of the preset calibration parameters can also be stored. Since the parameters of devices in the same batch are similar, when optimizing, the preset characterization parameters corresponding to the specified sensors in the same batch of devices can be reused, without the need to filter the preset calibration parameters again, which can greatly reduce the calibration complexity. The calibration results of the preset calibration parameters can also be used as the initialization data of the self-moving device to be calibrated, which can further accelerate the data convergence speed in the initial calibration process and improve the calibration efficiency.

[0144] S304, using the calibration parameter sample dataset associated with the specified sensor, initialize the values ​​of the preset calibration parameters corresponding to the specified sensor in the self-moving device to obtain the initialization parameter values ​​of the preset calibration parameters.

[0145] Using the values ​​of the preset calibration parameters in the calibration parameter sample dataset, initialize the values ​​of the corresponding preset calibration parameters corresponding to the specified sensor in the self-moving device to obtain the initialization parameter values ​​of the preset calibration parameters. For example, obtain the rotation parameters of the camera in a sample of a robotic vacuum cleaner from the same batch as robotic vacuum cleaner a, and then use the values ​​of the rotation parameters of that camera to initialize the values ​​of the rotation parameters of the camera in robotic vacuum cleaner a to obtain the initialization parameter values ​​of the rotation parameters of the camera in robotic vacuum cleaner a.

[0146] S306, Optimize the initial parameter values ​​of the preset calibration parameters using running data.

[0147] As described in the above embodiments, the initial parameter values ​​of the preset calibration parameters can be optimized using an objective function, and the optimization process can be constrained using a least squares function. The objective function can be constructed based on the sample device; by reusing the objective function, the complexity of calibrating devices in the same batch can be further reduced. Alternatively, the reused objective function can be updated using running data from the stable operation of the self-moving device to be calibrated, so that the updated objective function can be used to perform online calibration of the preset calibration parameters, further improving the accuracy of online calibration. Of course, other methods can also be used to optimize the initial parameter values; this specification does not limit this.

[0148] In the above embodiments, the calibration parameter sample dataset associated with the specified sensor is used to initialize the value of the preset calibration parameter corresponding to the specified sensor in the self-moving device. Since the calibration parameter sample dataset contains the calibration results of sample devices in the same batch as the self-moving device, a better initialization parameter value can be obtained. Therefore, when optimizing the initialization parameter value of the preset calibration parameter using the running data, the optimization speed can be accelerated, the optimization time can be shortened, and the optimization efficiency can be effectively improved, thereby effectively improving the parameter calibration efficiency. In addition, before optimizing the parameter value of the initialization specification 12 / 16 pages 15 CN 121655595 A, the objective function is first optimized using the least squares function, and then the optimized objective function is used to optimize the initialization parameter value of the preset calibration parameter, thereby reducing the complexity of online calibration and making the sensor calibration more accurate.

[0149] It should be understood that although the steps in the flowcharts involved in the above embodiments are shown sequentially according to the arrow indication, these steps are not necessarily executed sequentially according to the order indicated by the arrow. Unless otherwise expressly stated herein, there is no strict order restriction on the execution of these steps, and these steps may be executed in other orders. Moreover, at least some of the steps in the flowcharts involved in the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but may be executed at different times. The execution order of these steps or stages is not necessarily sequential, but may be executed alternately or in turn with other steps or at least some of the steps or stages in other steps.

[0150] Based on the same inventive concept, the embodiments of this application also provide a sensor calibration device for a self-moving device for implementing the sensor calibration method of the self-moving device involved above. The solution to the problem provided by this device is similar to the implementation solution described in the above method. Therefore, the sensor calibration method of one or more self-moving devices provided below is similar to the solution to the problem described in the above method.Specific limitations in the embodiments of the calibration device can be found in the limitations of the sensor calibration method for self-moving devices described above, and will not be repeated here.

[0151] In one embodiment, as shown in FIG4, a sensor calibration device for a self-moving device is provided, including: a data acquisition module 402, an optimization module 404, and a determination module 406, wherein:

[0152] The data acquisition module 402 is used to acquire the running data of the self-moving device in the current time window;

[0153] The optimization module 404 is used to optimize the value of the preset calibration parameter corresponding to the specified sensor in the self-moving device using the running data; the preset calibration parameter is an observable parameter selected from the calibration parameters of the specified sensor, and the observable parameter refers to the parameter that can be estimated based on the running data of the self-moving device;

[0154] The determination module 406 is used to output the calibration result of the specified sensor according to the value of the optimized preset calibration parameter.

[0155] In one embodiment, the optimization module 404 is further configured to optimize the value of the preset calibration parameter corresponding to the specified sensor in the self-moving device when the feature points in the running data under the current time window meet the preset conditions;

[0156] In one embodiment, as shown in FIG5, the device further includes:

[0157] an acquisition module 402, further configured to acquire image information of the self-moving device when it is working;

[0158] a determination module 406, further configured to determine the pixel position of the feature points in the image information; and determine the projection position of the feature points based on the depth information between the self-moving device and the feature points;

[0159] a first calculation module 408, configured to calculate the reprojection error of the feature points based on the pixel position and the projection position; when the reprojection error is greater than the preset error, the feature points meet the preset conditions.

[0160] In one embodiment, as shown in FIG5, the device further includes:

[0161] a selection module 410, used to select at least two feature points in the target key frame of the running data;

[0162] a search module 412, used to search for matching feature points in other key frames of the running data based on the at least two feature points; when the ratio between the number of matching feature points and the number of at least two feature points is less than a preset ratio, the feature points satisfy the preset conditions.

[0163] In one embodiment, as shown in FIG5, the device further includes:

[0164] a collection module 402, also used to collect the yaw angle of the self-moving device in real time during the optimization process; Specification 13 / 16 pages 16 CN 121655595 A

[0165] a second calculation module 414, used to perform difference calculation on a preset number of yaw angles to obtain the difference of the yaw angle of the self-moving device during the optimization process; when the difference satisfies the difference condition, the value of the optimized preset calibration parameter is obtained.

[0166] In one embodiment, the difference includes variance or standard deviation;

[0167] The second calculation module 414 is further configured to stop optimizing the value of the preset calibration parameter when the variance of the yaw angle of the self-moving device during the optimization process is not greater than the preset variance, or the standard deviation of the yaw angle of the self-moving device during the optimization process is not greater than the preset standard deviation, and obtain the optimized value of the preset calibration parameter;

[0168] The determination module 406 is further configured to use the optimized value of the preset calibration parameter as the calibration result of the specified sensor.

[0169] In the above embodiments, by pre-selecting calibration parameters that can be estimated based on operating data from the calibration parameters of the sensor as preset calibration parameters, and then using the real-time operating data of the self-moving device to perform online calibration of the pre-selected preset calibration parameters, it is possible to achieve rapid online calibration of at least some calibration parameters, improve the simplicity and efficiency of camera and odometer parameter calibration; at the same time, it is also possible to promptly correct the data of the preset calibration parameters based on the real-time operating data of the self-moving device, ensure the real-time accurate representation of the relative position relationship of the sensors, and thus improve the accuracy of self-moving device control.

[0170] In one embodiment, as shown in FIG5, the device further includes:

[0171] an acquisition module 416, configured to acquire a calibration parameter sample dataset associated with the specified sensor; wherein the calibration parameter sample dataset includes the calibration results of the specified sensor in a sample device; the sample device refers to a device with the same batch identifier as the self-moving device;

[0172] an initialization module 418, configured to initialize the values ​​of the preset calibration parameters corresponding to the specified sensor in the self-moving device using the calibration parameter sample dataset associated with the specified sensor, to obtain the initialization parameter values ​​of the preset calibration parameters;

[0173] an optimization module 404, further configured to optimize the initialization parameter values ​​of the preset calibration parameters using running data.

[0174] In one embodiment, the optimization module 404 is further configured to optimize the value of the preset calibration parameter corresponding to the specified sensor in the self-moving device according to the running data and the objective function; wherein, the objective function is used to characterize the correlation between the value of the preset calibration parameter and the running data of the self-moving device; the objective function is constructed under the constraint of the least squares function, and the least squares function is used to characterize the error between the predicted value of the preset calibration parameter and the parameter label value.

[0175] In the above embodiment, by using the same batch of sample devices and the calibration parameter sample dataset corresponding to the specified sensor, the value of the preset calibration parameter corresponding to the specified sensor in the self-moving device is initialized, thereby obtaining a better initial parameter value. Therefore, when using the running data to optimize the initial parameter value of the preset calibration parameter, the optimization speed can be accelerated, the optimization time can be shortened, the optimization efficiency can be effectively improved, and thus the parameter calibration efficiency can be effectively improved.

[0176] Each module in the sensor calibration device of the aforementioned self-moving device can be implemented entirely or partially through software, hardware, or a combination thereof. Each module can be embedded in or independent of the processor in the computer device in hardware form, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0177] In one embodiment, a computer device is provided, which can be a self-moving device (such as a robotic vacuum cleaner), and its internal structure diagram can be shown in Figure 6. The computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus (see page 14 / 16 of the specification, CN 121655595 A), and the communication interface, display unit, and input device are connected to the system bus via the input / output interface. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The input / output interface of the computer device is used for exchanging information between the processor and external devices. The communication interface of the computer device is used for wired or wireless communication with external terminals. Wireless communication can be achieved through WIFI, mobile cellular networks, NFC (Near Field Communication) or other technologies. When the computer program is executed by the processor, it implements a sensor calibration method for a self-moving device. The display unit of the computer device is used to form a visually visible image. It can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covering the display screen, or buttons or touchpads set on the casing of the computer device.

[0178] Those skilled in the art will understand that the structure shown in FIG6 is only a block diagram of a part of the structure related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0179] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program. When the processor executes the computer program, it implements the steps of a sensor calibration method for a self-moving device.

[0180] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of a sensor calibration method for a self-moving device.

[0181] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of a sensor calibration method for a self-moving device.

[0182] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data need to comply with the relevant laws, regulations and standards of relevant countries and regions.

[0183] Those skilled in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Any reference to memory, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the various embodiments provided in this application may include at least one of relational databases and non-relational databases. Non-relational databases may include blockchain-based distributed databases, etc., but are not limited thereto. The processors involved in the various embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited thereto. Specification 15 / 16 pages 18 CN 121655595 A

[0184] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, the above embodiments are not described in detail.All possible combinations of the various technical features described herein are described; however, as long as there is no contradiction in the combination of these technical features, they should all be considered within the scope of this specification.

[0185] The embodiments described above only illustrate several implementation methods of this application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of this application. It should be noted that for those skilled in the art, several modifications and improvements can be made without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims. Instruction Manual 16 / 16 Page 19 CN 121655595 A Figure 1 Instruction Manual Appendix 1 / 4 Page 20 CN 121655595 A Figure 2 Figure 3 Instruction Manual Appendix 2 / 4 Page 21 CN 121655595 A Figure 4 Figure 5 Instruction Manual Appendix 3 / 4 Page 22 CN 121655595 A Figure 6 Instruction Manual Appendix 4 / 4 Page 23 CN 121655595 A SENSOR CALIBRATION METHOD, APPARATUS, DEVICE AND STORAGE MEDIUM FOR A SELF-MOVING DEVICE Abstract The present application relates to a sensor calibration method, apparatus, device, storage medium and computer program product for a self-mobile device. The method includes: collecting operation data of the self-mobile device within a current time window; using the operation data to optimize values ​​of preset calibration parameters corresponding to a designated sensor in the self-mobile device, wherein the preset calibration parameters are observable parameters selected from calibrationparameters of the designated sensor, and the observable parameters refer to parameters that can be estimated based on the operation data of the self -mobile device; and outputting a calibration result of the designated sensor according to the optimized values of the preset calibration parameters. By adopting the method, the calibration efficiency of the sensor in the self-mobile device can be greatly improved.

Claims

1. A sensor calibration method for a self-moving device, characterized in that, The method includes: The data collected from the mobile device within the current time window includes feature points. If the reprojection error of the feature point is greater than a preset error, or if the ratio between the number of matching feature points and the number of feature points is less than a preset ratio, the values ​​of the preset calibration parameters corresponding to the specified sensor in the self-moving device are optimized using the running data; the feature point is the feature point in the target keyframe, and the matching feature point is the feature point in other keyframes that matches the feature point. Based on the optimized values ​​of the preset calibration parameters, the calibration result of the specified sensor is output.

2. The method according to claim 1, characterized in that, The method further includes: The self-moving device is used to collect image information during operation. Determine the pixel positions of feature points in the image information; Based on the depth information between the self-moving device and the feature point, the projection position of the feature point is determined; The reprojection error of the feature point is calculated based on the pixel position and the projection position.

3. The method according to claim 1, characterized in that, The method further includes: Select at least two of the feature points in the target keyframe of the running data; Based on the at least two of the aforementioned feature points, the matching feature points are searched in other keyframes of the running data.

4. The method according to any one of claims 1 to 3, characterized in that, Before collecting the running data from the mobile device within the current time window, the method further includes: Based on the operating data of the self-moving device under different operating modes, the observability analysis is performed on the calibration parameters of the specified sensor to obtain observable parameters under different operating modes; wherein, the observable parameters are preset calibration parameters; The preset calibration parameters are associated with and stored in relation to the operating mode; The step of optimizing the values ​​of preset calibration parameters corresponding to a specified sensor in the self-moving device using the operational data includes: The values ​​of the preset calibration parameters stored in the associated storage are optimized using the running data belonging to the running mode under the current time window.

5. The method according to any one of claims 1 to 3, characterized in that, The self-moving device includes a robotic vacuum cleaner, and the designated sensors include a camera and an odometer; Before collecting the running data from the mobile device within the current time window, the method further includes: From the calibration parameters of the camera and the odometer, calibration parameters that can be estimated based on the operating data of the sweeping robot are selected; wherein, the selected calibration parameters are preset calibration parameters; After associating the preset calibration parameters with the camera and the odometer, the preset calibration parameters are stored. The step of optimizing the values ​​of preset calibration parameters corresponding to a specified sensor in the self-moving device using the operational data includes: The stored values ​​of the preset calibration parameters are optimized using the operational data.

6. The method according to claim 5, characterized in that, The step of selecting calibration parameters that can be estimated based on the operating data of the sweeping robot from the calibration parameters of the camera and the odometer includes: Based on the operating data of the sweeping robot, an observability analysis was performed on the rotation parameters and translation of the camera and the odometer to obtain the observability analysis results. If the observability analysis results indicate that the rotation parameter is observable, then the rotation parameter is selected; wherein the rotation parameter is a calibration parameter that can be estimated based on the operating data of the sweeping robot.

7. A sensor calibration device for a self-moving device, characterized in that, The device includes: The acquisition module is used to acquire running data from the mobile device within the current time window, and the running data includes feature points; An optimization module is used to optimize the values ​​of preset calibration parameters corresponding to a specified sensor in the self-moving device using the running data when the reprojection error of the feature point is greater than a preset error, or the ratio between the number of matching feature points and the number of feature points is less than a preset ratio; the feature point is a feature point in the target keyframe, and the matching feature point is a feature point in other keyframes that matches the feature point. The determination module is used to output the calibration result of the specified sensor based on the optimized values ​​of the preset calibration parameters.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the method described in any one of claims 1 to 6.

Images