Sensor calibration method, apparatus, device and storage medium for self-moving device

Abstract

The invention relates to a sensor calibration method, device and equipment of self-moving equipment, a storage medium and a computer program product. The method comprises the following steps: acquiring operation data of the self-moving equipment in a current time window; optimizing a numerical value of a preset calibration parameter corresponding to a specified sensor in the self-moving equipment by utilizing the operation data; the preset calibration parameter is an observable parameter screened from the calibration parameters of the specified sensor, and the observable parameter is a parameter which can be estimated based on the operation data of the self-moving device; and outputting a calibration result of the specified sensor according to the optimized numerical value of the preset calibration parameter. By adopting the method, the calibration efficiency of the sensor in the self-moving equipment can be greatly improved.

Description

(19) State Intellectual Property Office (12) Invention Patent Application (10) Application Publication Number (43) Application Publication Date (21) Application Number 202511720425.4 (22) Application Date 2022.08.09 (62) Divisional Application Data 202210952170.4 2022.08.09 (71) Applicant: ZhuiMi Innovation Technology (Suzhou) Co., Ltd. Address: Units 1, 2, and 3, Building 8, No. 1688, Songwei Road, Guoxiang Street, Wuzhong Economic Development Zone, Suzhou City, Jiangsu Province, 215000 (72) Inventors: Xu Feng, Yang Sheng, Sheng Tengfei (74) Patent Agency: Huajin United Patent & Trademark Agency Co., Ltd. 44224 Patent Attorney: Chen Qifang (51) Int.Cl. G01D 18 / 00 (2006.01) G01C 1 / 00 ​​(2006.01) G06T 7 / 80 (2017.01) G06T 7 / 33 (2017.01) G06V 10 / 75 (2022.01) (54) Invention Title: Sensor Calibration Method, Apparatus, Device, and Storage Medium for Self-Moving Devices (57) Abstract: This application relates to a sensor calibration method, apparatus, device, storage medium, and computer program product for self-moving devices. The method includes: collecting operating data of the self-moving device in the current time window; using the operating data to optimize the values ​​of preset calibration parameters corresponding to a specified sensor in the self-moving device; the preset calibration parameters are observable parameters selected from the calibration parameters of the specified sensor, and the observable parameters refer to parameters that can be estimated based on the operating data of the self-moving device; and outputting the calibration result of the specified sensor according to the optimized values ​​of the preset calibration parameters. This method can greatly improve the calibration efficiency of sensors in self-moving devices. Claims 2 pages, Description 16 pages, Drawings 4 pages, CN 121655594 A 2026.03.13 CN 1 21 65 55 94 A 1. A sensor calibration method for a self-moving device, characterized in that the method includes: collecting operating data of the self-moving device in the current time window; using the operating data to optimize the value of a preset calibration parameter corresponding to a specified sensor in the self-moving device, and during the optimization process, calculating the difference in the real-time attitude of the self-moving device to obtain the difference in attitude during the optimization process, and obtaining the optimized value of the preset calibration parameter when the difference satisfies the difference condition; outputting the calibration result of the specified sensor according to the optimized value of the preset calibration parameter. 2. The method according to claim 1, characterized in that optimizing the value of the preset calibration parameter corresponding to the specified sensor in the self-moving device includes:The method involves: acquiring image information of the self-moving device during operation; determining the pixel positions of feature points in the image information; determining the projection position of the feature points based on the depth information between the self-moving device and the feature points; calculating the reprojection error of the feature points based on the pixel positions and the projection positions; and optimizing the value of the preset calibration parameter corresponding to the specified sensor in the self-moving device when the reprojection error is greater than a preset error. 3. The method according to claim 1, wherein optimizing the value of the preset calibration parameter corresponding to the specified sensor in the self-moving device includes: selecting at least two feature points in the target keyframe of the running data; searching for matching feature points in other keyframes of the running data based on the at least two feature points; and optimizing the value of the preset calibration parameter corresponding to the specified sensor in the self-moving device when the ratio between the number of matching feature points and the number of at least two feature points is less than a preset ratio. 4. The method according to claim 1, wherein the step of calculating the difference in attitude of the self-moving device in real time during the optimization process to obtain the difference in attitude during the optimization process includes: acquiring the attitude of the self-moving device in real time during the optimization process; storing the attitude in a buffer; and when the number of attitudes stored in the buffer reaches a preset number, calculating the difference in attitude stored in the buffer to obtain the difference in attitude during the optimization process. 5. The method according to claim 1, wherein the attitude includes a yaw angle; the step of calculating the difference in attitude of the self-moving device in real time during the optimization process to obtain the difference in attitude during the optimization process includes: acquiring the yaw angle of the self-moving device in real time during the optimization process; storing the yaw angle in a buffer; and when the number of yaw angles stored in the buffer reaches a preset number, calculating the difference in yaw angle stored in the buffer to obtain the difference in yaw angle during the optimization process. 6. The method according to claim 5, wherein the difference includes variance or standard deviation; obtaining the optimized value of the preset calibration parameter when the difference satisfies the difference condition includes: stopping the optimization of the value of the preset calibration parameter when the variance of the yaw angle of the self-moving device during the optimization process is not greater than the preset variance, or the standard deviation of the yaw angle of the self-moving device during the optimization process is not greater than the preset standard deviation, and obtaining the optimized value of the preset calibration parameter; outputting the calibration result of the specified sensor according to the optimized value of the preset calibration parameter includes: using the optimized value of the preset calibration parameter as the calibration result of the specified sensor.7. The method according to any one of claims 1 to 6, characterized in that, before collecting the operating data of the mobile device in the current time window, the method further includes: performing observability analysis on the calibration parameters of the designated sensor based on the operating data of the mobile device in different operating modes to obtain observable parameters in different operating modes; wherein the observable parameters are preset calibration parameters; associating and storing the preset calibration parameters with the operating modes; the step of optimizing the value of the preset calibration parameters corresponding to the designated sensor in the mobile device using the operating data includes: optimizing the value of the associated and stored preset calibration parameters using the operating data belonging to the operating mode in the current time window. 8. The method according to any one of claims 1 to 6, wherein the self-moving device includes a robotic vacuum cleaner, and the designated sensor includes a camera and an odometer; before collecting the operating data of the self-moving device in the current time window, the method further includes: selecting calibration parameters that can be estimated based on the operating data of the robotic vacuum cleaner from the calibration parameters of the camera and the odometer; wherein the selected calibration parameters are preset calibration parameters; associating the preset calibration parameters with the camera and the odometer, and storing the preset calibration parameters; optimizing the value of the preset calibration parameters corresponding to the designated sensor in the self-moving device using the operating data includes: optimizing the value of the stored preset calibration parameters using the operating data. 9. A sensor calibration device for a self-moving device, characterized in that the device comprises: a data acquisition module, configured to acquire operating data of the self-moving device within a current time window; an optimization module, configured to optimize the values ​​of preset calibration parameters corresponding to a specified sensor in the self-moving device using the operating data, and during the optimization process, to calculate the difference in the real-time attitude of the self-moving device to obtain the difference in attitude during the optimization process, and to obtain the optimized value of the preset calibration parameter when the difference satisfies the difference condition; and a determination module, configured to output the calibration result of the specified sensor based on the optimized value of the preset calibration parameter. 10. A computer device, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that the processor, when executing the computer program, implements the steps of the method according to any one of claims 1 to 8. 11. A computer-readable storage medium storing a computer program thereon, characterized in that the computer program, when executed by a processor, implements the steps of the method according to any one of claims 1 to 8. 12. A computer program product, comprising a computer program, characterized in that the computer program, when executed by a processor, implements the steps of the method according to any one of claims 1 to 8. Claims (page 2 / 2)3 CN 121655594 A Sensor Calibration Method, Apparatus, Device and Storage Medium for Self-Moving Devices

[0001] This application is a divisional application filed with the Chinese Patent Office on August 9, 2022, with application number 2022109521704, entitled "Sensor Calibration Method, Apparatus, Device and Storage Medium for Self-Moving Devices", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of data processing technology for self-moving devices, and in particular to a sensor calibration method, apparatus, device and storage medium for self-moving devices. Background Art

[0003] A robotic vacuum cleaner is a household self-moving device for cleaning indoor environments. It can automatically clean without supervision and automatically plan its cleaning path according to the indoor environment. That is, it uses a camera on the self-moving device to collect indoor image information. The central processing unit (CPU) of the self-moving device can calculate the geometric information of objects in the three-dimensional space of the room, and thereby reconstruct or identify objects, realize the recognition of the indoor environment, and thus complete the cleaning work.

[0004] In order for self-moving devices to accurately perform cleaning work, it is necessary to calibrate the sensors of the self-moving devices. In traditional calibration schemes, the calibration process of the sensors of self-moving devices usually requires complicated manual operations, which greatly reduces the efficiency of self-moving device calibration. Summary of the Invention

[0005] Based on this, it is necessary to provide a sensor calibration method, apparatus, device and storage medium for self-moving devices to address the above-mentioned technical problems, which can greatly improve the calibration efficiency of sensors in self-moving devices.

[0006] In a first aspect, this application provides a sensor calibration method for self-moving devices, the method comprising:

[0007] collecting operating data of the self-moving device in the current time window;

[0008] using the operating data to optimize the value of a preset calibration parameter corresponding to a specified sensor in the self-moving device; the preset calibration parameter is an observable parameter selected from the calibration parameters of the specified sensor, and the observable parameter refers to a parameter that can be estimated based on the operating data of the self-moving device;

[0009] outputting the calibration result of the specified sensor according to the optimized value of the preset calibration parameter.

[0010] In one embodiment, optimizing the value of the preset calibration parameter corresponding to the designated sensor in the self-moving device includes:

[0011] Optimizing the value of the preset calibration parameter corresponding to the designated sensor in the self-moving device when the feature points in the running data under the current time window meet preset conditions.

[0012] In one embodiment, the feature points in the running data under the current time window that meet the preset conditions are determined at least using the following methods:

[0013]

[0014] Collect image information of the self-moving device during operation;

[0015] Determine the pixel position of the feature point in the image information;

[0016] Determine the projection position of the feature point based on the depth information between the self-moving device and the feature point; Specification 1 / 16 Page 4 CN 121655594 A

[0017] Calculate the reprojection error of the feature point based on the pixel position and the projection position;

[0018] When the reprojection error is greater than a preset error, the feature point satisfies the preset condition.

[0019] In one embodiment, the feature point in the running data under the current time window is determined to satisfy the preset condition by at least the following method:

[0020] Select at least two feature points in the target key frame of the running data;

[0021] Based on the at least two feature points, find matching feature points in other key frames of the running data;

[0022] When the ratio between the number of matching feature points and the number of at least two feature points is less than a preset ratio, the feature point satisfies the preset condition.

[0022] In one embodiment, the method further includes:

[0023] during the optimization process, acquiring the yaw angle of the self-moving device in real time;

[0024] calculating the difference between a preset number of yaw angles to obtain the difference in the yaw angle of the self-moving device during the optimization process;

[0025] when the difference satisfies the difference condition, obtaining the value of the optimized preset calibration parameter.

[0026] In one embodiment, the difference includes variance or standard deviation; the step of outputting the calibration result of the specified sensor based on the optimized value of the preset calibration parameter includes:

[0027] When the variance of the yaw angle of the self-moving device during the optimization process is not greater than the preset variance, or the standard deviation of the yaw angle of the self-moving device during the optimization process is not greater than the preset standard deviation, the optimization of the value of the preset calibration parameter is stopped, and the optimized value of the preset calibration parameter is obtained;

[0028] The step of outputting the calibration result of the specified sensor based on the optimized value of the preset calibration parameter includes:

[0029] The optimized value of the preset calibration parameter is used as the calibration result of the specified sensor.

[0030] In one embodiment, before optimizing the value of the preset calibration parameter corresponding to the specified sensor in the self-moving device, the method further includes:

[0031] obtaining a calibration parameter sample dataset associated with the specified sensor; wherein the calibration parameter sample dataset contains the calibration results of the specified sensor in a sample device; the sample device refers to a device with the same batch identifier as the self-moving device;

[0032] Using the calibration parameter sample dataset associated with the specified sensor, the values ​​of the preset calibration parameters corresponding to the specified sensor in the self-moving device are initialized to obtain the initial parameter values ​​of the preset calibration parameters;

[0033] Optimizing the values ​​of the preset calibration parameters corresponding to the specified sensor in the self-moving device using the running data includes: optimizing the initial parameter values ​​of the preset calibration parameters using the running data.

[0034] In one embodiment, optimizing the values ​​of the preset calibration parameters corresponding to the specified sensor in the self-moving device using the running data includes:

[0035] Optimizing the values ​​of the preset calibration parameters corresponding to the specified sensor in the self-moving device according to the running data and the objective function; wherein, the objective function is used to characterize the correlation between the values ​​of the preset calibration parameters and the running data of the self-moving device; the objective function is constructed under the least squares function constraint, and the least squares function is used to characterize the error between the predicted value of the preset calibration parameter and the parameter label value. Specification 2 / 16 Page 5 CN 121655594 A

[0036] In a second aspect, this application also provides a sensor calibration device for a self-moving device, the device comprising:

[0037] a data acquisition module, configured to acquire operating data of the self-moving device in the current time window;

[0038] an optimization module, configured to optimize the value of a preset calibration parameter corresponding to a specified sensor in the self-moving device using the operating data; the preset calibration parameter is an observable parameter selected from the calibration parameters of the specified sensor, the observable parameter being a parameter that can be estimated based on the operating data of the self-moving device;

[0039] a determination module, configured to output the calibration result of the specified sensor according to the optimized value of the preset calibration parameter.

[0040] In one embodiment, the optimization module is further configured to optimize the value of the preset calibration parameter corresponding to the specified sensor in the self-moving device when the feature points in the operating data in the current time window meet preset conditions.

[0041] In one embodiment, the device further includes:

[0042] the acquisition module, further configured to acquire image information of the self-moving device during operation;

[0043] the determination module, further configured to determine the pixel position of a feature point in the image information; and determine the projection position of the feature point based on the depth information between the self-moving device and the feature point;

[0044] a first calculation module, configured to calculate the reprojection error of the feature point based on the pixel position and the projection position; and when the reprojection error is greater than a preset error, the feature point satisfies the preset condition.

[0045] In one embodiment, the device further includes:

[0046] a selection module, configured to select at least two feature points in the target keyframe of the running data;

[0047] a search module, configured to search for matching feature points in other keyframes of the running data based on the at least two feature points; when the ratio between the number of matching feature points and the number of at least two feature points is less than a preset ratio, the feature points satisfy the preset condition.

[0048] In one embodiment, the device further includes:

[0049] the acquisition module, configured to acquire the yaw angle of the self-moving device in real time during the optimization process;

[0050] a second calculation module, configured to perform difference calculation on a preset number of yaw angles to obtain the difference in the yaw angle of the self-moving device during the optimization process; when the difference satisfies the difference condition, the value of the optimized preset calibration parameter is obtained.

[0051] In one embodiment, the difference includes variance or standard deviation;

[0052] The second calculation module is further configured to stop optimizing the value of the preset calibration parameter and obtain the optimized value of the preset calibration parameter when the variance of the yaw angle of the self-moving device during the optimization process is not greater than a preset variance, or the standard deviation of the yaw angle of the self-moving device during the optimization process is not greater than a preset standard deviation;

[0053] The determining module is further configured to use the optimized value of the preset calibration parameter as the calibration result of the specified sensor.

[0054] In one embodiment, the device further includes:

[0055] an acquisition module, configured to acquire a calibration parameter sample dataset associated with the specified sensor; wherein the calibration parameter sample dataset includes the calibration results of the specified sensor in a sample device; the sample device refers to a device with the same batch identifier as the self-moving device;

[0056] an initialization module, configured to initialize the value of the preset calibration parameter corresponding to the specified sensor in the self-moving device using the calibration parameter sample dataset associated with the specified sensor, to obtain the initialization parameter value of the preset calibration parameter; Specification 3 / 16 page 6 CN 121655594 A

[0057] The optimization module is further configured to optimize the initialization parameter value of the preset calibration parameter using the running data.

[0058] In one embodiment, the optimization module is further configured to optimize the values ​​of preset calibration parameters corresponding to a specified sensor in the self-moving device based on the operating data and the objective function; wherein the objective function is used to characterize the correlation between the values ​​of the preset calibration parameters and the operating data of the self-moving device; the objective function is constructed under the constraint of a least squares function, and the least squares function is used to characterize the predicted values ​​of the preset calibration parameters.Error between the parameter label value and the actual value.

[0059] In a third aspect, this application also provides a computer device. The computer device includes a memory and a processor, which, when executing the computer program, implements the steps in the sensor calibration method for the self-moving device provided in the above aspects.

[0060] In a fourth aspect, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps in the sensor calibration method for the self-moving device provided in the above aspects.

[0061] In a fifth aspect, this application also provides a computer program product. The computer program product includes a computer program, which, when executed by a processor, implements the steps in the sensor calibration method for the self-moving device provided in the above aspects.

[0062] The sensor calibration method for self-moving devices provided in this application embodiment selects calibration parameters that can be estimated based on operating data from the calibration parameters of the sensor as preset calibration parameters, and then uses the operating data of the self-moving device to perform online calibration of the preset calibration parameters. This can achieve rapid online calibration of at least some calibration parameters, improving the simplicity and efficiency of sensor calibration. At the same time, it can also realize timely correction of the preset calibration parameters based on the real-time operating data of the self-moving device, ensuring the real-time accurate representation of the relative position relationship of the sensors, thereby improving the accuracy of self-moving device control.

[0063] Figure 1 is an application environment diagram of the sensor calibration method for a self-moving device in one embodiment;

[0064] Figure 2 is a flowchart of the sensor calibration method for a self-moving device in one embodiment;

[0065] Figure 3 is a flowchart of initializing and optimizing the values ​​of preset calibration parameters corresponding to a specified sensor in a self-moving device in one embodiment;

[0066] Figure 4 is a structural block diagram of the sensor calibration device for a self-moving device in one embodiment;

[0067] Figure 5 is a structural block diagram of the sensor calibration device for a self-moving device in another embodiment;

[0068] Figure 6 is an internal structure diagram of a computer device in one embodiment. Detailed Description

[0069] To make the objectives, technical solutions, and advantages of this application clearer, the following describes this application in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0070] The sensor calibration method for a self-moving device provided in the embodiments of this application can be applied to the application environment shown in Figure 1. The self-moving device 102 communicates with the server 104 and the terminal 106 via a network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated into the server 104 or placed in the cloud. (See page 7 of the instruction manual, 4 / 16)CN 121655594 A or other network servers.

[0071] The self-moving device 102 can be a cleaning device or other mobile device that automatically cleans the indoor environment. The cleaning device can be a sweeping robot, a mopping robot, a vacuum cleaner, or a sweeping and mopping robot, etc.

[0072] When a system update is required, the server 104 can establish a network connection between the self-moving device 102 and the server 104 according to the user's operation, and then download the updated version of the system for update. The server 104 can be an independent physical server or a server cluster composed of multiple physical servers. It can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0073] The terminal 106 can be a smartphone, tablet, laptop, desktop computer, smartwatch, IoT device, and portable wearable device, etc. The user can use the terminal 106 to control the self-moving device to clean the entire indoor environment or a specific area within the indoor environment.

[0074] In one scenario example provided in this specification, the calibration of the camera and odometer of a robotic vacuum cleaner is illustrated below. During operation, the robotic vacuum cleaner typically needs to use both the camera and odometer to estimate its motion state. Therefore, accurate calibration between the camera and odometer is crucial for improving the stability of the robotic vacuum cleaner's operation. Traditional calibration methods usually require cumbersome manual operations to complete the calibration of the camera and odometer, resulting in low efficiency and complex, tedious operation.

[0075] In this scenario example, based on observability analysis, calibration parameters that can be estimated from the calibration parameters of the camera and odometer using the robotic vacuum cleaner's operating data can be selected as preset calibration parameters. These preset calibration parameters can be associated with and stored with the camera and odometer.

[0076] Subsequently, the preset calibration parameters can be calibrated online using the robotic vacuum cleaner's operating data. The online calibration can be performed in real time as the robotic vacuum cleaner operates. If the online calibration is performed in real time as the robot vacuum cleaner operates, the preset calibration parameters can be optimized online using the robot vacuum cleaner's real-time operating data, further ensuring the accuracy of the sensor calibration results on which the robot vacuum cleaner's operation depends, thereby improving the stability of the robot vacuum cleaner's operation. Of course, the online calibration can also be performed at a specified time point or according to the user's instructions, which is not limited in this specification.

[0077] During the online calibration of the camera and odometer, the robot vacuum cleaner's operating data can be used to determine...The stability of the robot vacuum's operation within the corresponding time window. If the self-moving device operates relatively stably, the preset calibration parameters do not need to be optimized; however, if the self-moving device operates poorly, the preset calibration parameters should be optimized. For example, the reprojection error of the feature points of the robot vacuum in a certain time window can be used to determine whether the preset calibration parameters need to be optimized. If optimization is required, the operating data in that time window can be used to optimize the preset calibration parameters. The data processing involved in sensor calibration is usually large. In this scenario example, by analyzing the actual operation of the self-moving device in a certain time window, it can be determined whether the preset calibration parameters need to be optimized in that time window. This can effectively reduce the data processing volume while ensuring the stability of the self-moving device's operation.

[0078] If it is determined that the preset calibration parameters need to be optimized, the operating data such as the robot vacuum's operating position, obstacle position, and distance value in the time window can be used to optimize the values ​​of the preset calibration parameters corresponding to the camera and odometer, and the optimized preset calibration parameter values ​​can be used as the calibration results of the odometer and camera. Afterwards, the robot vacuum cleaner can perform operation control based on the calibration results of the camera and odometer, thereby improving the stability of the robot vacuum cleaner's operation. (Instruction manual, page 5 / 16, CN 121655594 A)

[0079] In this scenario example, by pre-selecting calibration parameters that can be estimated based on operating data from the calibration parameters of the camera and odometer as preset calibration parameters, and then using the operating data of the self-moving device to perform online calibration of the pre-selected preset calibration parameters, at least some calibration parameters can be quickly calibrated online, improving the simplicity and efficiency of camera and odometer parameter calibration; at the same time, the data of the preset calibration parameters can be corrected in a timely manner based on the real-time operating data of the self-moving device, ensuring the real-time accurate representation of the relative position relationship of the sensors, thereby improving the accuracy of the self-moving device control.

[0080] Based on the above scenario example, this embodiment of the specification provides a sensor calibration method for a self-moving device. The method can be applied to the server that establishes a communication connection with the self-moving device, or to the terminal that establishes a communication connection with the self-moving device, or to the self-moving device itself. Taking the self-moving device in Figure 1 as an example, as shown in Figure 2, the method includes at least the following steps:

[0081] S202, collecting the running data of the self-moving device under the current time window.

[0082] The running data of the self-moving device over time can be extracted using a time window to obtain the running data under the corresponding time window. The time length covered by the time window can be set as needed. For ease of description, the time length covered by the time window can be described as the window length. When the running data is a video data stream, the window length is also...The number of keyframes can be used to measure the time window length. For example, if the number of keyframes is 5, then the window length is 5. A keyframe can refer to a video frame sampled at a fixed interval in a video frame sequence, such as sampling one video frame every two frames as a keyframe. Of course, when the running data is other types of data, the window length of the time window can be represented in other ways, which is not limited in this specification.

[0083] The current time window can be the time window on which the current sensor calibration is based. The running data extracted based on the current time window can be the real-time running data of the self-moving device or the historical running data.

[0084] The running data can include the data collected by the built-in sensors of the self-moving device during operation, and can also include the status data, relative position data, etc. of the self-moving device collected by other devices during operation. Such as the position, running time, attitude of the self-moving device, and video frames collected by the camera. The types of running data used for calibration of different calibration parameters usually have certain differences. The type of running data required for calibration of a certain calibration parameter can be analyzed in advance, and the running data type can be associated with the calibration parameter for storage. Correspondingly, during online calibration, only the data of the running data type corresponding to the calibration parameter is extracted to calibrate the calibration parameter, reducing the amount of running data collected and further improving the calibration processing efficiency.

[0085] The collection of running data can be carried out by sliding time windows, or randomly. Of course, other methods can also be used, which are not limited in this specification. For example, a time window of the target window length can be extracted from a certain time point as the current time window, and the running data of the mobile device under the current time window can be collected. For example, during the operation of the sweeping robot, the sweeping robot's position information, running time information, posture information, and video frame information under a time window with a window length of 5 are collected; among them, the video frame is the video key frame, since the window length is represented by the key frame. Then, the time window is slid, such as sliding the time window once every 3 seconds, and used as the current time window in turn, and the running data of the mobile device under the current time window is collected.

[0086] S204, using the running data, the values ​​of the preset calibration parameters corresponding to the specified sensors in the mobile device are optimized.

[0087] The designated sensor can refer to a sensor that needs to be calibrated, such as an odometer, camera, obstacle detector, distance detector, and speed and angle sensors of the self-moving device. The designated sensor can be a single sensor, that is, the calibration of the relative positional relationship of a single sensor with respect to the body of the self-moving device. For example, to calibrate the position of the odometer relative to the body, the designated sensor is the odometer. Alternatively, the designated sensor can also be two sensors. (See page 6 / 16 of the specification, 9 CN 121655594 A)The above sensor calibration refers to the calibration of the relative positional relationship between two or more sensors. For example, when calibrating the relative positional relationship between a camera and an odometer, the specified sensors refer to the camera and the odometer.

[0088] The preset calibration parameters are observable parameters selected from the calibration parameters of the specified sensors. Observable parameters refer to parameters that can be estimated based on the operating data of the self-moving device. For example, observability analysis can be used to analyze the calibration parameters of the specified sensors and select calibration parameters that can be estimated based on the operating data of the self-moving device as preset calibration parameters. The observability analysis can be performed using nonlinear system observability analysis, VI-SLAM system observability analysis, or INS system observability analysis based on geometric features. For example, for the odometer of a robotic vacuum cleaner, the preset calibration parameters may include at least one of extrinsic parameters (such as rotation parameters), map points, and attitude. The map points may refer to the position of the robotic vacuum cleaner on a map, which is a map of the indoor environment.

[0089] For example, during the operation of the robotic vacuum cleaner, if global coordinate system X-direction positioning is added to perform observability analysis on the calibration parameters of a specified sensor, it can be determined that the global positioning X-direction and yaw angle are observable parameters. Similarly, if global coordinate system Y-direction positioning is added to perform observability analysis on the calibration parameters of a specified sensor, it can be determined that the global positioning Y-direction and yaw angle are observable parameters. If global coordinate system Z-direction positioning is added to perform observability analysis on the calibration parameters of a specified sensor, it can be determined that the global positioning Z-direction is an observable parameter, while the yaw angle is an unobservable parameter. Furthermore, when the robotic vacuum cleaner performs pure rotational motion, observability analysis on the calibration parameters of a specified sensor can determine that the scale of the feature points is unobservable; when the robotic vacuum cleaner performs constant acceleration motion, observability analysis on the calibration parameters of a specified sensor can determine that the scale of the feature points is unobservable. Through the above observability analysis, observable parameters can be selected from the calibration parameters of the specified sensor as preset calibration parameters.

[0090] For example, regarding the calibration of the odometer and camera of a robotic vacuum cleaner, the intrinsic parameters of the odometer and camera generally do not change significantly and can be calibrated offline. The online calibration process can mainly focus on calibrating the extrinsic parameters of the odometer and camera. The extrinsic parameters of the odometer and camera include rotation parameters and translation parameters. The rotation parameters characterize the rotational transformation between the camera coordinate system and the odometer coordinate system, and the translation parameters characterize the translational transformation between the camera coordinate system and the odometer coordinate system.

[0091] The observability analysis of the rotation and translation parameters of the odometer and camera of the robotic vacuum cleaner can be performed based on the operating data. Since the robotic vacuum cleaner mostly moves in a straight line, the observability analysis of the rotation and translation parameters of the odometer and camera can be performed based on the operating data of the robotic vacuum cleaner during straight-line movement, ultimately determining the X, Y, and Z coordinates.Translational transformations of axes are unobservable, while rotational parameters are observable. Accordingly, rotational parameters can be used as preset calibration parameters for online calibration. Of course, by analyzing the robot vacuum cleaner in other operating modes, the operating mode for which translational parameters can be calibrated can be determined, and the translational parameters can be calibrated online based on the operating data in that operating mode.

[0092] Accordingly, observability analysis can be performed on the calibration parameters of a specified sensor in advance based on the operating data in different operating modes to determine the observable parameters of the calibration parameters in different operating modes, and the selected observable parameters can be used as preset calibration parameters and associated with the corresponding operating modes for storage. For example, for the above example, the rotational parameters of linear motion, odometer, and camera can be associated and stored. Subsequently, the preset calibration parameters of the specified sensor can be calibrated online based on the operating data of the robot vacuum cleaner in the corresponding operating mode. For the above example, the operating data corresponding to the linear motion of the robot vacuum cleaner can be obtained to calibrate the rotational parameters of the odometer and camera online. By differentiating operating modes to filter observable parameters, the selection of observable parameters can be made more accurate and comprehensive; at the same time, calibrating observable parameters based on operating data under the corresponding operating modes can also make the calibration of observable parameters more accurate.

[0093] Of course, if it is determined through analysis that the selection of observable parameters in the calibration parameters is unrelated to the operating mode, then the selected preset calibration parameters can be specially labeled. During online calibration, the preset calibration parameters can be calibrated without distinguishing the operating mode.

[0094] During the online calibration process, the pre-extracted preset calibration parameters can be obtained, and the values ​​of the preset calibration parameters corresponding to the specified sensors can be optimized using the collected operating data to ensure the real-time accurate representation of the relative position relationship of the sensors, thereby improving the accuracy of the control of the self-moving device.

[0095] For example, the values ​​of the preset calibration parameters can be optimized using a pre-constructed objective function and operating data. The objective function can be used to characterize the correlation between the values ​​of the preset calibration parameters and the operating data of the self-moving device.

[0096] First, the preset calibration parameters of the specified sensor can be calibrated offline, and the calibration results can be stored as parameter label values ​​of the preset calibration parameters. Then, the operating data of the sweeping robot under stable operation based on the aforementioned parameter label values ​​can be obtained, and the objective function can be constructed using this operating data and the parameter label values. During the construction of the objective function, the least squares function can also be used for constraints to accelerate the convergence of the objective function. The least squares function can be used to characterize the error between the value predicted by the preset calibration parameters based on the operating data and the parameter label value. For ease of description, the value predicted by the preset calibration parameters based on the operating data can be simplified as the predicted value of the preset calibration parameters in the following text.

[0097] Based on the predicted values ​​of the preset calibration parameters and the parameter label values, the objective function is updated until the value of the pre-constructed least squares function meets the preset value conditions, thus obtaining the objective function. Since the least squares function can characterize the error between the predicted values ​​of the preset calibration parameters and the parameter label values, this error continuously decreases during the updating process of the objective function, making the final constructed objective function more accurately characterize the correlation between the running data and the parameter label values.

[0098] For example, the least squares function can be the sum of squares of the differences between the predicted values ​​of the preset calibration parameters and the parameter label values ​​(true values):

[0099]

[0100] Where, represents the predicted value of the preset calibration parameters, and represents the parameter label value of the preset calibration parameters.

[0101] The expression of the objective function can be, or can be, where represents each running data, and , , , ..., are the function coefficients. In subsequent embodiments, let the expression of the objective function be.

[0102] Converting the above objective function into matrix form, we have X, therefore the expression of the least squares function is: .

[0103] Therefore, the least squares function can be regarded as a system of quadratic equations about , solving for the minimum value of the least squares function L, because the minimum value of L corresponds to the optimal value, so the value is continuously adjusted to make L the minimum.

[0104] Considering the least squares function as a function of L about the variable, calculating the first derivative of the least squares function, we can get: Specification 8 / 16 Page 11 CN 121655594 A

[0105]

[0106] Simplifying the above formula, we can get:

[0107]

[0108] Therefore, according to the above simplified formula, the value is calculated by inputting it into the objective function, and then the value of the least squares function L is calculated. When the value of the least squares function L satisfies the preset value condition, the objective function is obtained. Then, the constructed objective function and running data can be used to optimize the value of the preset calibration parameter.

[0109] Of course, the above-mentioned optimization method of the preset calibration parameter is a preferred example. The preset calibration parameter can also be optimized in other ways, such as by referring to the calibration parameter data optimization method used in the traditional offline calibration process. This is not limited here.

[0110] In one embodiment, the value of the preset calibration parameter corresponding to the specified sensor in the self-moving device can be optimized when the feature points in the running data under the current time window meet the preset conditions.

[0111] After obtaining the preset calibration parameter, it can be determined whether to optimize the preset calibration parameter based on whether the feature points in the running data under the current time window meet the preset conditions. When the feature points meet the preset conditions, then the preset calibration parameter is optimized.The preset calibration parameters need to be optimized; when the feature points do not meet the preset conditions, the preset calibration parameters do not need to be optimized.

[0112] The feature points can also be data points extracted from the running data that meet certain feature requirements. For example, when the running data includes image information, the feature points can refer to pixels in the image that meet certain feature requirements. For example, the feature points can be points where the grayscale value of the image changes drastically or points with large curvature on the edge of the image. Feature points in the image can be extracted using methods such as SIFT, SURF, and ORB. The preset conditions correspond to the analysis method used when analyzing the feature points of the running data. When the running data is laser point cloud data, the feature points can be data points extracted from the point cloud data that meet certain feature requirements.

[0113] The calibration of the sensor involves a large amount of data, especially for real-time calibration, which will occupy too much of the running resources of the self-moving device. Generally, if the self-moving device operates stably, the calibration parameters for the relative positional relationships between its sensors usually do not present significant problems. However, if the self-moving device's operational stability is poor within the current time window, the calibration parameters may not accurately reflect the relative positional relationships between its sensors. By analyzing the actual operation of the self-moving device within the current time window, determining whether sensor parameter calibration is necessary can effectively reduce data processing volume while ensuring the stability of the self-moving device's operation. Determining the self-moving device's operational status based on feature points can further reduce the data processing volume and resource consumption. Simultaneously, feature points can more accurately reflect the characteristics of the operational data, reducing noise interference and thus further improving the accuracy of determining the self-moving device's operational status.

[0114] In some embodiments, the determination of whether the preset calibration parameters need to be optimized can be made by using the reprojection error of the feature points, or by judging the observability of the feature points of a certain key frame in other key frames, as described below:

[0115] Method 1: Use whether the reprojection error of the feature points meets the preset conditions to determine whether the preset calibration parameters need to be optimized.

[0116] The reprojection error of the feature points can be calculated first, and then the reprojection error can be compared with the preset error. If the reprojection error is less than or equal to the preset error, the preset calibration parameters do not need to be optimized; if the reprojection error is greater than the preset error, the preset calibration parameters need to be optimized.

[0117] For example, the calculation steps of the reprojection error may include: acquiring image information collected by the built-in camera of the self-moving device during operation; determining the pixel position of the feature points in the image information; and based on the relationship between the self-moving device and the feature points...The depth information is used to determine the projection position of the feature point; the reprojection error of the feature point is calculated based on the pixel position and the projection position; when the reprojection error is greater than the preset error, the feature point meets the preset condition, indicating that the preset calibration parameter needs to be optimized. At this time, the preset calibration parameter is used as the preset calibration parameter for optimization.

[0118] Among them, the image information can be based on the key frame captured by the camera, that is, the video key frame. The depth information can refer to the distance between the mobile device and the feature point in the world coordinate system. It should be noted that the number of feature points can be one or more.

[0119] For example, suppose the relationship between the pixel position of a feature point and the spatial point position corresponding to the feature point is as follows:

[0120]

[0121] Where, is the depth information, is the pixel position (i.e., pixel coordinate) of the feature point, represents the position of the feature point in the world coordinate system, is the projection position (i.e., projection coordinate) of the feature point, represents the feature point projected from the world coordinate system to the camera coordinate system, and K is a coefficient.

[0122] The above relationship can be written in matrix form to obtain: .

[0123] Due to noise issues, the above relationship has an error. Therefore, by transforming the above relationship, the following error relationship can be obtained:

[0124]

[0125] Based on the above error relationship, the error can be calculated, which is the reprojection error of the feature point. Since the reprojection error considers not only the calculation error of the homography matrix but also the measurement error of the feature point, its accuracy is high, thus it can accurately determine whether the preset calibration parameters should be optimized, which is beneficial to improving the optimization effect. Specification 10 / 16 pages 13 CN 121655594 A

[0126] Method 2: Determine whether the preset calibration parameters should be optimized based on whether the observability of the feature points of a certain key frame in other key frames meets the preset conditions.

[0127] Select at least two feature points in the target key frame of the running data; based on the at least two feature points, search for matching feature points in other key frames of the running data; when the ratio between the number of matching feature points and the number of at least two feature points is greater than or equal to the preset ratio, the feature points meet the preset conditions.

[0128] Specifically, the running data may include video frames captured by the camera. Therefore, keyframes are sampled from the captured video frames, and feature points are selected in a target keyframe. Then, it is checked in other keyframes whether there are feature points that match the feature points (i.e., matching feature points). If matching feature points are found in other keyframes, it means that the feature points in the target keyframe are observable in other keyframes. When the ratio between the number of matching feature points and the number of extracted feature points is less than a preset ratio, such as when the feature point ratio is less than 80%, it means that the feature points meet the preset conditions, and it is determined that the preset calibration parameters need to be optimized. When the ratio between the number of matching feature points and the number of extracted feature points is greater than or equal to the preset ratio, it means that the feature points meet the preset conditions, and it is determined that the preset calibration parameters need to be optimized.When the preset ratio is equal to the preset ratio, such as when the feature point ratio is greater than 80%, it is determined that the preset calibration parameter does not need to be optimized.

[0129] For example, for keyframe a and keyframe b, assuming that feature points 1 to 10 are selected in keyframe a, and then in keyframe b, it is queried whether there are feature points that match feature points 1 to 10. If 9 matching feature points are found, it is determined that the preset calibration parameter does not need to be optimized; if only 7 matching feature points are found, it is determined that the preset calibration parameter needs to be optimized. When determining whether to optimize the preset calibration parameter, it is not necessary to use the entire running data for judgment. It is only necessary to judge the feature point matching of the keyframe in the running data. Therefore, even if there is a lot of data in the running data, it is possible to accurately determine whether to optimize the result, and it can also effectively reduce the amount of calculation.

[0130] S206, according to the value of the optimized preset calibration parameter, output the calibration result of the specified sensor.

[0131] The value of the optimized preset calibration parameter can be output as the calibration result of the specified sensor. Alternatively, during the optimization of the preset calibration parameters, it can be determined whether the optimized preset calibration parameters meet the optimization conditions. When the optimization conditions are met, the optimization process is stopped, and the optimized preset calibration parameters are obtained as the calibration result of the specified sensor.

[0132] For example, for the calibration of the rotation parameters of the camera and the odometer, the three-axis rotation is usually calibrated together. Therefore, the error of any axis will affect the calibration of the other axes. The accuracy of the three-axis rotation calibration can be determined by analyzing the running direction of the sweeping robot.

[0133] In some embodiments, the process of determining whether the optimized preset calibration parameters meet the optimization conditions can be determined by the difference in yaw angles, as follows:

[0134] During the optimization process, the yaw angle of the self-moving device is collected in real time; the difference of the preset number of yaw angles is calculated to obtain the difference in the yaw angle of the self-moving device during the optimization process; when the difference meets the difference condition, the optimized preset calibration parameters are obtained.

[0135] For example, during the optimization process, the yaw angle of the self-moving device can be collected in real time and stored in a buffer. When the number of stored yaw angles reaches a preset number, the collection of yaw angles stops, and the difference calculation begins, thereby avoiding excessive collection of yaw angles and prolonging the optimization time.

[0136] This difference can be variance or standard deviation. When the variance of the self-moving device's yaw angle during the optimization process is not greater than a preset variance, or the standard deviation of the self-moving device's yaw angle during the optimization process is not greater than a preset standard deviation, the optimization of the preset calibration parameter values ​​stops, and the optimized preset calibration parameter values ​​are obtained; then, the optimized preset calibration parameter values ​​are used as the calibration results of the specified sensor.

[0137] For example, after obtaining a preset number of yaw angles for the robot vacuum cleaner, the variance or standard deviation of the yaw angles of the robot vacuum cleaner during the optimization process is calculated. Then, the variance of the yaw angles is compared with the preset variance, or the standard deviation of the yaw angles is compared with the preset standard deviation, so as to determine whether to stop the optimization process based on the comparison results. During the optimization of the preset calibration parameters, the yaw angle is greatly affected by the preset calibration parameters. Using a preset number of yaw angles to evaluate whether to stop the optimization process can accurately determine whether the value of the preset calibration parameters has reached the optimal value, thereby avoiding excessive collection of yaw angles and prolonging the optimization time, and also helping to improve the accuracy of parameter calibration.

[0138] Alternatively, in some other embodiments, the attitude of the self-moving device can be collected in real time during the optimization process; the difference of the attitude of the self-moving device during the optimization process can be calculated by performing a preset number of attitude differences; when the difference meets the difference condition, the value of the optimized preset calibration parameters can be obtained. When optimizing the values ​​of preset calibration parameters, the posture of the self-moving device can be optimized. Therefore, the difference of a preset number of postures can be used to evaluate whether to stop the optimization process, which can avoid excessive posture collection and prolong the optimization time, and can also ensure the accuracy of posture optimization.

[0139] For example, during the optimization process, the posture of the sweeping robot can be collected in real time and stored in a buffer. When the number of stored postures reaches a preset number, posture collection is stopped and the difference calculation is started, thereby avoiding excessive posture collection and prolonging the optimization time.

[0140] In the above embodiments, by pre-selecting calibration parameters that can be estimated based on running data from the calibration parameters of the sensor as preset calibration parameters, and then using the running data of the self-moving device to perform online calibration of the pre-selected preset calibration parameters, at least some calibration parameters can be quickly calibrated online, improving the simplicity and efficiency of sensor calibration; at the same time, the preset calibration parameters can be corrected in a timely manner based on the real-time running data of the self-moving device, ensuring the real-time accurate representation of the relative position relationship of the sensor, thereby improving the accuracy of self-moving device control.

[0141] In some other embodiments, as shown in FIG3, before S204, the method further includes:

[0142] S302, acquiring a calibration parameter sample dataset associated with a specified sensor.

[0143] Wherein, the calibration parameter sample dataset contains the calibration results of the specified sensor in a sample device; the sample device refers to a device with the same batch identifier as the self-moving device. The batch identifier refers to information that identifies the production batch of the device. Typically, devices in the same production batch have similar device parameter information. For example, a robot vacuum cleaner with the same batch as the target...Using the same batch of robotic vacuum cleaners as samples, the preset calibration parameters corresponding to the cameras, odometers and other sensors in the samples are obtained, and the preset calibration parameters are associated with the sensors and stored; at the same time, the calibration results of the preset calibration parameters can also be stored. Since the parameters of the devices in the same batch are similar, when optimizing, the preset characterization parameters corresponding to the specified sensors in the same batch of devices can be reused, without the need to screen the preset calibration parameters again, which can greatly reduce the calibration complexity. The calibration results of the preset calibration parameters can also be used as the initialization data of the self-moving device to be calibrated, which can further accelerate the data convergence speed in the initial calibration process and improve the calibration efficiency.

[0144] S304, using the calibration parameter sample dataset associated with the specified sensor, the values ​​of the preset calibration parameters corresponding to the specified sensors in the self-moving device are initialized to obtain the initial parameter values ​​of the preset calibration parameters.

[0145] Using the values ​​of the preset calibration parameters in the calibration parameter sample dataset, the values ​​of the corresponding preset calibration parameters corresponding to the specified sensors in the self-moving device are initialized to obtain the initial parameter values ​​of the preset calibration parameters. For example, obtain the rotation parameters of the camera in a sample of a sweeping robot from the same batch as sweeping robot a, and then use the value of the rotation parameters of the camera to initialize the value of the rotation parameters of the camera in sweeping robot a, so as to obtain the initial parameter value of the rotation parameters of the camera in sweeping robot a.

[0146] S306, optimize the initial parameter value of the preset calibration parameters using running data.

[0147] As described in the above embodiment, the initial parameter value of the preset calibration parameters can be optimized using an objective function, and the optimization process can be constrained using a least squares function. The objective function can be constructed based on the sample device. By reusing the objective function, the complexity of calibration of the same batch of devices can be further reduced. Of course, the running data of the self-moving device to be calibrated during stable operation can also be used to update the reused objective function, so as to use the updated objective function to perform online calibration of the preset calibration parameters, thereby further improving the accuracy of online calibration. Of course, other methods can also be used to optimize the initialization parameter values, and this specification does not limit this.

[0148] In the above embodiments, the calibration parameter sample dataset associated with the specified sensor is used to initialize the values ​​of the preset calibration parameters corresponding to the specified sensor in the self-moving device. Since the calibration parameter sample dataset contains the calibration results of sample devices in the same batch as the self-moving device, a better initialization parameter value can be obtained. Therefore, when optimizing the initialization parameter values ​​of the preset calibration parameters using running data, the optimization speed can be accelerated.Shortening the optimization time effectively improves optimization efficiency, thereby effectively improving parameter calibration efficiency. Furthermore, before optimizing the initial parameter values, the objective function is first optimized using the least squares function, and then the optimized objective function is used to optimize the initial parameter values ​​of the preset calibration parameters. This reduces the complexity of online calibration while making the sensor calibration more accurate.

[0149] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the steps or stages in other steps.

[0150] Based on the same inventive concept, this application also provides a sensor calibration device for a self-moving device for implementing the sensor calibration method for the self-moving device described above. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more embodiments of the sensor calibration device for a self-moving device provided below can be found in the limitations of the sensor calibration method for the self-moving device described above, and will not be repeated here.

[0151] In one embodiment, as shown in FIG4, a sensor calibration device for a self-moving device is provided, comprising: a data acquisition module 402, an optimization module 404, and a determination module 406, wherein:

[0152] The data acquisition module 402 is used to acquire the operating data of the self-moving device in the current time window;

[0153] The optimization module 404 is used to optimize the values ​​of preset calibration parameters corresponding to a specified sensor in the self-moving device using the operating data; the preset calibration parameters are observable parameters selected from the calibration parameters of the specified sensor, and observable parameters refer to parameters that can be estimated based on the operating data of the self-moving device;

[0154] The determination module 406 is used to output the calibration result of the specified sensor according to the optimized values ​​of the preset calibration parameters.

[0155] In one embodiment, the optimization module 404 is further used to optimize the values ​​of the preset calibration parameters corresponding to the specified sensor in the self-moving device when the feature points in the operating data in the current time window meet preset conditions;

[0156] In one embodiment, as shown in FIG5, the device further comprises:

[0157] The acquisition module 402 is also used to acquire image information of the self-moving device when it is working;

[0158] The determination module 406 is also used to determine the pixel position of the feature point in the image information; and to determine the projection position of the feature point based on the depth information between the self-moving device and the feature specification page 13 / 16 16 CN 121655594 A;

[0159] The first calculation module 408 is used to calculate the reprojection error of the feature point based on the pixel position and the projection position; when the reprojection error is greater than the preset error, the feature point meets the preset condition.

[0160] In one embodiment, as shown in FIG5, the device further includes:

[0161] The selection module 410 is used to select at least two feature points in the target key frame of the running data;

[0162] The search module 412 is used to search for matching feature points in other key frames of the running data based on at least two feature points; when the ratio between the number of matching feature points and the number of at least two feature points is less than the preset ratio, the feature point meets the preset condition.

[0163] In one embodiment, as shown in FIG5, the device further includes:

[0164] a data acquisition module 402, which is further configured to acquire the yaw angle of the self-moving device in real time during the optimization process;

[0165] a second calculation module 414, which is configured to perform difference calculation on a preset number of yaw angles to obtain the difference of the yaw angle of the self-moving device during the optimization process; when the difference meets the difference condition, the value of the optimized preset calibration parameter is obtained.

[0166] In one embodiment, the difference includes variance or standard deviation;

[0167] the second calculation module 414 is further configured to stop optimizing the value of the preset calibration parameter when the variance of the yaw angle of the self-moving device during the optimization process is not greater than the preset variance, or the standard deviation of the yaw angle of the self-moving device during the optimization process is not greater than the preset standard deviation, and obtain the value of the optimized preset calibration parameter;

[0168] a determination module 406, which is further configured to use the value of the optimized preset calibration parameter as the calibration result of the specified sensor.

[0169] In the above embodiments, by pre-selecting calibration parameters that can be estimated based on operating data from the calibration parameters of the sensor as preset calibration parameters, and then using the real-time operating data of the self-moving device to perform online calibration of the pre-selected preset calibration parameters, it is possible to achieve rapid online calibration of at least some calibration parameters, thereby improving the simplicity and efficiency of camera and odometer parameter calibration; at the same time, it is also possible to promptly correct the preset calibration parameters based on the real-time operating data of the self-moving device, ensuring the real-time accurate representation of the relative position relationship of the sensors, thereby improving the accuracy of self-moving device control.

[0170] In one embodiment, as shown in FIG5, the device further includes:

[0171] an acquisition module 416, used to acquire the calibration parameter sample dataset associated with the specified sensor; wherein, theThe calibration parameter sample dataset contains the calibration results of the specified sensor in the sample device; the sample device refers to a device with the same batch identifier as the self-moving device;

[0172] The initialization module 418 is used to initialize the value of the preset calibration parameter corresponding to the specified sensor in the self-moving device using the calibration parameter sample dataset associated with the specified sensor, and obtain the initial parameter value of the preset calibration parameter;

[0173] The optimization module 404 is also used to optimize the initial parameter value of the preset calibration parameter using the running data.

[0174] In one embodiment, the optimization module 404 is also used to optimize the value of the preset calibration parameter corresponding to the specified sensor in the self-moving device according to the running data and the objective function; wherein, the objective function is used to characterize the correlation between the value of the preset calibration parameter and the running data of the self-moving device; the objective function is constructed under the least squares function constraint, and the least squares function is used to characterize the error between the predicted value of the preset calibration parameter and the parameter label value.

[0175] In the above embodiments, by using the same batch of sample devices to specify the calibration parameter sample dataset corresponding to the designated sensor, the values ​​of the preset calibration parameters corresponding to the designated sensor in the self-moving device are initialized, thereby obtaining a better initialization parameter value. Therefore, when using running data to optimize the initialization parameter value of the preset calibration parameter, the optimization speed can be accelerated, the optimization time shortened, and the optimization efficiency effectively improved, thereby effectively improving the parameter calibration efficiency.

[0176] Each module in the sensor calibration device of the above self-moving device can be implemented entirely or partially by software, hardware, or a combination thereof. Each module can be embedded in the processor of the computer device in hardware form or independent of the processor, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0177] In one embodiment, a computer device is provided, which can be a self-moving device (such as a sweeping robot), and its internal structure diagram can be as shown in Figure 6. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, while the communication interface, display unit, and input devices are also connected to the system bus via input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for...The processor exchanges information with external devices. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner. The wireless method can be implemented through WIFI, mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, it implements a sensor calibration method for a self-moving device. The display unit of the computer device is used to form a visually visible image. It can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covering the display screen, or buttons or touchpads set on the casing of the computer device, etc.

[0178] Those skilled in the art will understand that the structure shown in FIG6 is only a block diagram of a part of the structure related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine some components, or have different component arrangements.

[0179] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program. When the processor executes the computer program, it implements the steps of a sensor calibration method for a self-moving device.

[0180] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, wherein the computer program, when executed by a processor, implements the steps of a sensor calibration method for a self-moving device.

[0181] In one embodiment, a computer program product is provided, including a computer program, wherein the computer program, when executed by a processor, implements the steps of a sensor calibration method for a self-moving device.

[0182] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data need to comply with the relevant laws, regulations and standards of relevant countries and regions.

[0183] Those skilled in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when the computer program is executed, it can include the processes of the embodiments of the above methods. Wherein, any reference to memory, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (Read-Only Memory). (See specification page 15 / 16, 18 CN 121655594 A)Memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in this application may include at least one of relational databases and non-relational databases. Non-relational databases may include distributed databases based on blockchain, etc., and are not limited thereto. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited thereto.

[0184] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered as within the scope of this specification.

[0185] The above embodiments only illustrate several implementation methods of this application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the patent application. It should be noted that for those skilled in the art, several modifications and improvements can be made without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims. Instruction manual, page 16 / 16, 19 CN 121655594 A, Figure 1; Instruction manual, Figure 1 / 4, page 20 CN 121655594 A, Figure 2; Figure 3; Instruction manual, Figure 2 / 4, page 21 CN 121655594 A, Figure 4; Figure 5; Instruction manual, Figure 3 / 4, page 22 CN 121655594 A, Figure 6; Instruction manual, Figure 4 / 4, page 23 CN 121655594 A, Abstract: Abdominal ultrasound examinationmethod, system and device SENSOR CALIBRATION METHOD, APPARATUS, DEVICE AND STORAGE MEDIUM FOR SELF-MOVING DEVICE Abstract The present application relates to a sensor calibration method, apparatus, device, storage medium and computer program product for a self-moving device. The method includes: collecting operation data of the self-moving device within a current time window; using the operation data to optimize values of preset calibration parameters corresponding to a specified sensor in the self-moving device, wherein the preset calibration parameters are observable parameters selected from calibration parameters of the specified sensor, and the observable parameters refer to parameters that can be estimated based on the operation data of the self-moving device; and outputting a calibration result of the specified sensor according to the optimized values of the preset calibration parameters. Adoption of the method can greatly improve the calibration efficiency of the sensor in theself-moving device.

Claims

1. A sensor calibration method for a self-moving device, characterized in that, The method includes: Data collected from the mobile device within the current time window; Using the operational data, the values ​​of preset calibration parameters corresponding to the specified sensors in the self-moving device are optimized. During the optimization process, the real-time attitude of the self-moving device is calculated to obtain the attitude difference during the optimization process. When the difference meets the difference condition, the optimized values ​​of the preset calibration parameters are obtained. Based on the optimized values ​​of the preset calibration parameters, the calibration result of the specified sensor is output.

2. The method according to claim 1, characterized in that, The optimization of the preset calibration parameters corresponding to the specified sensors in the self-moving device includes: The self-moving device is used to collect image information during operation. Determine the pixel positions of feature points in the image information; Based on the depth information between the self-moving device and the feature point, the projection position of the feature point is determined; Calculate the reprojection error of the feature point based on the pixel position and the projection position; When the reprojection error is greater than the preset error, the value of the preset calibration parameter corresponding to the specified sensor in the self-moving device is optimized.

3. The method according to claim 1, characterized in that, The optimization of the preset calibration parameters corresponding to the specified sensors in the self-moving device includes: Select at least two feature points in the target keyframe of the running data; Based on the at least two of the aforementioned feature points, find matching feature points in other keyframes of the running data; When the ratio between the number of matching feature points and the number of at least two feature points is less than a preset ratio, the value of the preset calibration parameter corresponding to the specified sensor in the self-moving device is optimized.

4. The method according to claim 1, characterized in that, During the optimization process, the difference in attitude of the self-moving device in real time is calculated, and the difference in attitude during the optimization process includes: During the optimization process, the posture of the self-moving device is collected in real time; Store the attitude in a buffer; When the number of attitudes stored in the buffer reaches a preset number, the difference between the attitudes stored in the buffer is calculated to obtain the difference between the attitudes during the optimization process.

5. The method according to claim 1, characterized in that, The attitude includes the yaw angle; During the optimization process, the difference in attitude of the self-moving device in real time is calculated, and the difference in attitude during the optimization process includes: During the optimization process, the yaw angle of the self-moving device is collected in real time; The yaw angle is stored in the buffer; When the number of yaw angles stored in the buffer reaches a preset number, the difference between the yaw angles stored in the buffer is calculated to obtain the difference between the yaw angles during the optimization process.

6. The method according to claim 5, characterized in that, The difference includes variance or standard deviation; when the difference meets the difference condition, the optimized value of the preset calibration parameter is obtained, including: When the variance of the yaw angle of the self-moving device during the optimization process is not greater than the preset variance, or the standard deviation of the yaw angle of the self-moving device during the optimization process is not greater than the preset standard deviation, the optimization of the preset calibration parameter value is stopped, and the optimized preset calibration parameter value is obtained. The step of outputting the calibration result of the specified sensor based on the optimized preset calibration parameter values ​​includes: using the optimized preset calibration parameter values ​​as the calibration result of the specified sensor.

7. The method according to any one of claims 1 to 6, characterized in that, Before collecting the running data from the mobile device within the current time window, the method further includes: Based on the operating data of the self-moving device under different operating modes, the observability analysis is performed on the calibration parameters of the specified sensor to obtain observable parameters under different operating modes; wherein, the observable parameters are preset calibration parameters; The preset calibration parameters are associated with and stored in relation to the operating mode; The step of optimizing the values ​​of preset calibration parameters corresponding to a specified sensor in the self-moving device using the operational data includes: The values ​​of the preset calibration parameters stored in the associated storage are optimized using the running data belonging to the running mode under the current time window.

8. The method according to any one of claims 1 to 6, characterized in that, The self-moving device includes a robotic vacuum cleaner, and the designated sensors include a camera and an odometer; Before collecting the running data from the mobile device within the current time window, the method further includes: From the calibration parameters of the camera and the odometer, calibration parameters that can be estimated based on the operating data of the sweeping robot are selected; wherein, the selected calibration parameters are preset calibration parameters; After associating the preset calibration parameters with the camera and the odometer, the preset calibration parameters are stored. The step of optimizing the values ​​of preset calibration parameters corresponding to a specified sensor in the self-moving device using the operational data includes: The stored values ​​of the preset calibration parameters are optimized using the operational data.

9. A sensor calibration device for a self-moving device, characterized in that, The device includes: The data acquisition module is used to collect running data from mobile devices within the current time window. The optimization module is used to optimize the values ​​of preset calibration parameters corresponding to specified sensors in the self-moving device using the running data, and to calculate the difference in the real-time attitude of the self-moving device during the optimization process, so as to obtain the difference in attitude during the optimization process. When the difference meets the difference condition, the optimized values ​​of the preset calibration parameters are obtained. The determination module is used to output the calibration result of the specified sensor based on the optimized values ​​of the preset calibration parameters.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

12. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the method described in any one of claims 1 to 8.

Images