Sensor calibration method and device of self-moving equipment, equipment and storage medium

Abstract

The invention relates to a sensor calibration method, device and equipment of self-moving equipment, a storage medium and a computer program product. The method comprises the following steps: acquiring operation data of the self-moving equipment in a current time window; optimizing a numerical value of a preset calibration parameter corresponding to a specified sensor in the self-moving equipment by utilizing the operation data; the preset calibration parameter is an observable parameter screened from the calibration parameters of the specified sensor, and the observable parameter is a parameter which can be estimated based on the operation data of the self-moving device; and outputting a calibration result of the specified sensor according to the optimized numerical value of the preset calibration parameter. By adopting the method, the calibration efficiency of the sensor in the self-moving equipment can be greatly improved.

Description

[0001] This application is a divisional application filed with the Chinese Patent Office on August 9, 2022, with application number 2022109521704, entitled "Sensor Calibration Method, Apparatus, Device and Storage Medium for Self-Moving Equipment", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of data processing technology for self-moving devices, and in particular to a sensor calibration method, apparatus, device, and storage medium for self-moving devices. Background Technology

[0003] A robotic vacuum cleaner is a self-moving device used in homes to clean indoor environments. It can clean automatically without supervision and plans its cleaning path based on the indoor environment. It uses a camera on the self-moving device to collect image information of the room. The central processing unit (CPU) of the self-moving device can calculate the geometric information of objects in the three-dimensional space of the room and reconstruct or identify the objects, thereby recognizing the indoor environment and completing the cleaning work.

[0004] In order for self-moving equipment to perform cleaning work accurately, the sensors of the self-moving equipment need to be calibrated. In traditional calibration schemes, the calibration process usually requires complicated manual operations, which greatly reduces the efficiency of self-moving equipment calibration. Summary of the Invention

[0005] Therefore, it is necessary to provide a sensor calibration method, apparatus, device, and storage medium for self-moving devices to address the aforementioned technical problems, which can greatly improve the calibration efficiency of sensors in self-moving devices.

[0006] In a first aspect, this application provides a sensor calibration method for a self-moving device, the method comprising:

[0007] Data collected from the mobile device within the current time window;

[0008] Using the operational data, the values ​​of preset calibration parameters corresponding to a specified sensor in the self-moving device are optimized; the preset calibration parameters are observable parameters selected from the calibration parameters of the specified sensor, and the observable parameters refer to parameters that can be estimated based on the operational data of the self-moving device.

[0009] Based on the optimized values ​​of the preset calibration parameters, the calibration result of the specified sensor is output.

[0010] In one embodiment, optimizing the value of the preset calibration parameter corresponding to the specified sensor in the self-moving device includes:

[0011] If the feature points in the running data under the current time window meet the preset conditions, the values ​​of the preset calibration parameters corresponding to the specified sensors in the self-moving device are optimized.

[0012] In one embodiment, the feature points in the running data under the current time window are determined to satisfy a preset condition using at least the following method:

[0013] The self-moving device is used to collect image information during operation.

[0014] Determine the pixel positions of feature points in the image information;

[0015] Based on the depth information between the self-moving device and the feature point, the projection position of the feature point is determined;

[0016] Calculate the reprojection error of the feature point based on the pixel position and the projection position;

[0017] When the reprojection error is greater than the preset error, the feature point satisfies the preset condition.

[0018] In one embodiment, the feature points in the running data under the current time window are determined to satisfy a preset condition using at least the following method:

[0019] Select at least two of the feature points in the target keyframe of the running data;

[0020] Based on the at least two of the aforementioned feature points, find matching feature points in other keyframes of the running data;

[0021] When the ratio between the number of matching feature points and the number of at least two feature points is less than a preset ratio, the feature points satisfy the preset condition.

[0022] In one embodiment, the method further includes:

[0023] During the optimization process, the yaw angle of the self-moving device is collected in real time;

[0024] The difference between the yaw angles of the preset number of yaw angles is calculated to obtain the difference in the yaw angle of the self-moving device during the optimization process;

[0025] When the difference satisfies the difference condition, the optimized value of the preset calibration parameter is obtained.

[0026] In one embodiment, the difference includes variance or standard deviation; the step of outputting the calibration result of the specified sensor based on the optimized value of the preset calibration parameters includes:

[0027] When the variance of the yaw angle of the self-moving device during the optimization process is not greater than the preset variance, or the standard deviation of the yaw angle of the self-moving device during the optimization process is not greater than the preset standard deviation, the optimization of the preset calibration parameter value is stopped, and the optimized preset calibration parameter value is obtained.

[0028] The step of outputting the calibration result of the specified sensor based on the optimized values ​​of the preset calibration parameters includes:

[0029] The optimized values ​​of the preset calibration parameters are used as the calibration results of the specified sensor.

[0030] In one embodiment, before optimizing the value of the preset calibration parameter corresponding to the specified sensor in the self-moving device, the method further includes:

[0031] Obtain the calibration parameter sample dataset associated with the specified sensor; wherein, the calibration parameter sample dataset contains the calibration results of the specified sensor in a sample device; the sample device refers to a device with the same batch identifier as the self-moving device;

[0032] Using the calibration parameter sample dataset associated with the specified sensor, the values ​​of the preset calibration parameters corresponding to the specified sensor in the self-moving device are initialized to obtain the initial parameter values ​​of the preset calibration parameters.

[0033] The step of optimizing the value of the preset calibration parameter corresponding to the specified sensor in the self-moving device using the operating data includes: optimizing the initialization parameter value of the preset calibration parameter using the operating data.

[0034] In one embodiment, optimizing the values ​​of preset calibration parameters corresponding to a specified sensor in the self-moving device using the operational data includes:

[0035] Based on the operating data and the objective function, the values ​​of preset calibration parameters corresponding to a specified sensor in the self-moving device are optimized; wherein, the objective function is used to characterize the correlation between the values ​​of the preset calibration parameters and the operating data of the self-moving device; the objective function is constructed under the constraint of the least squares function, and the least squares function is used to characterize the error between the predicted value of the preset calibration parameter and the parameter label value.

[0036] Secondly, this application also provides a sensor calibration device for a self-moving device, the device comprising:

[0037] The data acquisition module is used to collect running data from mobile devices within the current time window.

[0038] An optimization module is used to optimize the values ​​of preset calibration parameters corresponding to a specified sensor in the self-moving device using the operating data; the preset calibration parameters are observable parameters selected from the calibration parameters of the specified sensor, and the observable parameters refer to parameters that can be estimated based on the operating data of the self-moving device.

[0039] The determination module is used to output the calibration result of the specified sensor based on the optimized values ​​of the preset calibration parameters.

[0040] In one embodiment, the optimization module is further configured to optimize the value of a preset calibration parameter corresponding to a specified sensor in the self-moving device when the feature points in the running data under the current time window meet preset conditions.

[0041] In one embodiment, the device further includes:

[0042] The acquisition module is also used to acquire image information of the self-moving device when it is working;

[0043] The determining module is further configured to determine the pixel position of the feature point in the image information; and determine the projection position of the feature point based on the depth information between the self-moving device and the feature point;

[0044] The first calculation module is used to calculate the reprojection error of the feature point based on the pixel position and the projection position; when the reprojection error is greater than a preset error, the feature point satisfies the preset condition.

[0045] In one embodiment, the device further includes:

[0046] A selection module is used to select at least two of the feature points in the target keyframe of the running data;

[0047] The search module is used to search for matching feature points in other key frames of the running data based on the at least two feature points; when the ratio between the number of matching feature points and the number of at least two feature points is less than a preset ratio, the feature point satisfies the preset condition.

[0048] In one embodiment, the device further includes:

[0049] The acquisition module is also used to acquire the yaw angle of the self-moving device in real time during the optimization process;

[0050] The second calculation module is used to calculate the difference of the yaw angle of a preset number of yaw angles to obtain the difference of the yaw angle of the self-moving device during the optimization process; when the difference meets the difference condition, the value of the preset calibration parameter after optimization is obtained.

[0051] In one embodiment, the difference includes variance or standard deviation;

[0052] The second calculation module is further configured to stop optimizing the value of the preset calibration parameter when the variance of the yaw angle of the self-moving device during the optimization process is not greater than the preset variance, or the standard deviation of the yaw angle of the self-moving device during the optimization process is not greater than the preset standard deviation, and obtain the optimized value of the preset calibration parameter.

[0053] The determination module is also used to take the optimized value of the preset calibration parameters as the calibration result of the specified sensor.

[0054] In one embodiment, the device further includes:

[0055] The acquisition module is used to acquire a calibration parameter sample dataset associated with the specified sensor; wherein, the calibration parameter sample dataset contains the calibration results of the specified sensor in a sample device; the sample device refers to a device with the same batch identifier as the self-moving device;

[0056] An initialization module is used to initialize the values ​​of preset calibration parameters corresponding to the specified sensor in the self-moving device using the calibration parameter sample dataset associated with the specified sensor, so as to obtain the initialization parameter values ​​of the preset calibration parameters.

[0057] The optimization module is also used to optimize the initialization parameter values ​​of the preset calibration parameters using the running data.

[0058] In one embodiment, the optimization module is further configured to optimize the values ​​of preset calibration parameters corresponding to a specified sensor in the self-moving device based on the operating data and the objective function; wherein the objective function is used to characterize the correlation between the values ​​of the preset calibration parameters and the operating data of the self-moving device; the objective function is constructed under the constraint of a least squares function, and the least squares function is used to characterize the error between the predicted values ​​of the preset calibration parameters and the parameter label values.

[0059] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, which, when executing the computer program, implements the steps in the sensor calibration method for the self-moving device provided in the above aspects.

[0060] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps in the sensor calibration method for the self-moving device provided in the above aspects.

[0061] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps in the sensor calibration method for the self-moving device provided in the above aspects.

[0062] The sensor calibration method for self-moving devices provided in this application pre-selects calibration parameters that can be estimated based on operating data from the sensor's calibration parameters as preset calibration parameters, and then uses the self-moving device's operating data to perform online calibration of the pre-selected preset calibration parameters. This enables rapid online calibration of at least some calibration parameters, improving the simplicity and efficiency of sensor calibration. Simultaneously, it also enables timely correction of the preset calibration parameters based on the self-moving device's real-time operating data, ensuring accurate real-time representation of the sensor's relative positional relationship, thereby improving the accuracy of self-moving device control. Attached Figure Description

[0063] Figure 1 This is an application environment diagram of a sensor calibration method for a self-moving device in one embodiment;

[0064] Figure 2 This is a flowchart illustrating a sensor calibration method for a self-moving device in one embodiment;

[0065] Figure 3 This is a flowchart illustrating the process of initializing and optimizing the values ​​of preset calibration parameters corresponding to a specified sensor in a self-moving device in one embodiment.

[0066] Figure 4 This is a structural block diagram of a sensor calibration device for a self-moving device in one embodiment;

[0067] Figure 5 This is a structural block diagram of a sensor calibration device for a self-moving device in one embodiment;

[0068] Figure 6 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0069] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0070] The sensor calibration method for self-moving devices provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, the self-mobile device 102 communicates with the server 104 and the terminal 106 via a network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated onto the server 104, or it can be located in the cloud or on other network servers.

[0071] The self-moving device 102 can be a cleaning device or other mobile device that automatically cleans the indoor environment. The cleaning device can be a robot vacuum cleaner, a mop, a vacuum cleaner, or a robot vacuum and mop combo.

[0072] When a system update is required, server 104 can establish a network connection between the mobile device 102 and server 104 based on user operations, and then download and update the system. Server 104 can be a standalone physical server or a server cluster consisting of multiple physical servers. It can be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0073] Terminal 106 can be a smartphone, tablet, laptop, desktop computer, smartwatch, IoT device, or portable wearable device, etc. Users can use terminal 106 to control the self-moving device to clean the entire indoor environment or a specific area within the indoor environment.

[0074] This manual provides a scenario example, using the calibration of a robot vacuum cleaner's camera and odometry as an example. During operation, robot vacuum cleaners typically require the combined use of a camera and odometry to estimate their motion state. Therefore, accurate calibration between the camera and odometry is crucial for improving the stability of the robot vacuum cleaner's operation. Traditional calibration methods usually require cumbersome manual operations to complete the calibration of the camera and odometry, resulting in low efficiency and complex, tedious procedures.

[0075] In this scenario example, based on observability analysis, calibration parameters that can be estimated from the robot vacuum's operating data can be selected from the calibration parameters of the camera and odometer, and these can be used as preset calibration parameters. These preset calibration parameters can then be associated with and stored using the camera and odometer.

[0076] Subsequently, the pre-selected preset calibration parameters can be calibrated online using the robot vacuum's operating data. This online calibration can be performed in real-time as the robot vacuum operates. If the online calibration is performed in real-time, the robot vacuum's real-time operating data can be used to optimize the preset calibration parameters online, further ensuring the accuracy of the sensor calibration results upon which the robot vacuum's operation depends, thereby improving the stability of the robot vacuum's operation. Of course, the online calibration can also be performed at a specified time or according to user instructions; this specification does not limit this.

[0077] During the online calibration of the camera and odometer, the operational stability of the robotic vacuum cleaner within a given time window can be determined based on its operational data. If the self-moving device operates stably, optimization of the preset calibration parameters is unnecessary; however, if the self-moving device's operational stability is poor, the preset calibration parameters should be optimized. For example, the reprojection error of feature points within a certain time window can be used to determine whether optimization of the preset calibration parameters is necessary. If optimization is required, the operational data within that time window can then be used to optimize the preset calibration parameters. Sensor calibration typically involves a large amount of data processing. In this scenario example, by analyzing the actual operation of the self-moving device within a certain time window to determine whether optimization of the preset calibration parameters is necessary, the operational stability of the self-moving device can be ensured while effectively reducing the amount of data processing.

[0078] If it's determined that the preset calibration parameters need optimization, the robot vacuum's operating data, such as its position, obstacle locations, and distances within a time window, can be used to optimize the values ​​of the preset calibration parameters corresponding to the camera and odometer. The optimized preset calibration parameter values ​​are then used as the calibration results for the odometer and camera. Afterward, the robot vacuum can use these calibration results for operational control, thereby improving the stability of its operation.

[0079] In this scenario example, by pre-selecting calibration parameters that can be estimated based on operational data from the calibration parameters of the camera and odometer as preset calibration parameters, and then using the operational data of the self-moving device to perform online calibration of the pre-selected preset calibration parameters, it is possible to achieve rapid online calibration of at least some calibration parameters, improving the simplicity and efficiency of camera and odometer parameter calibration. At the same time, the preset calibration parameters can be corrected in a timely manner based on the real-time operational data of the self-moving device, ensuring the real-time and accurate representation of the relative positional relationship of the sensors, thereby improving the accuracy of self-moving device control.

[0080] Based on the above scenario examples, this specification provides a sensor calibration method for a self-moving device. This method can be applied to the server that establishes a communication connection with the self-moving device, or to the terminal that establishes a communication connection with the self-moving device, or to the self-moving device itself. This method can be applied to... Figure 1 Let's take the self-moving device in the example as an illustration, such as... Figure 2 As shown, the method includes at least the following steps:

[0081] S202, collected from the mobile device's operating data within the current time window.

[0082] Time windows can be used to extract runtime data from a self-moving device over time, yielding runtime data within the corresponding time window. The duration covered by the time window can be set as needed. For ease of description, the duration covered by the time window can be described as the window length. When the runtime data is a video data stream, the window length can also be measured by the number of keyframes; for example, if the number of keyframes is 5, then the window length is 5. A keyframe can refer to a video frame sampled at fixed intervals in a video frame sequence, such as sampling one video frame every two frames as a keyframe. Of course, when the runtime data is other types of data, the window length of the time window can be represented in other ways, which are not limited in this specification.

[0083] The current time window can be the time window on which the sensor calibration is based. The operational data extracted based on the current time window can be real-time operational data of the mobile device or historical operational data.

[0084] Operational data can include data collected by the self-moving device's built-in sensors during operation, as well as status data and relative position data collected by other devices. Examples include the self-moving device's position, operation time, attitude, and video frames captured by the camera. The types of operational data used for calibration of different calibration parameters often differ. The required data types for calibrating a specific parameter can be pre-analyzed, and these data types can be associated and stored with the calibration parameter. Accordingly, during online calibration, only the data of the operational data type corresponding to the calibration parameter is extracted for that parameter, reducing the amount of operational data collected and further improving calibration processing efficiency.

[0085] Data collection can be performed using a sliding time window, randomly, or in other ways; this manual does not limit the methods. For example, a time window of a target length can be extracted from a specific point in time as the current time window, and data collection can be performed on the mobile device within that current time window. For instance, during the operation of a robotic vacuum cleaner, data such as its position, running time, posture, and video frames can be collected within a time window of length 5; the video frames are keyframes, as the window length is represented using keyframes. Then, the time window is slid, for example, at 3-second intervals, and each time a window is used as the current time window, and data collection can be performed on the mobile device within that current time window.

[0086] S204, using operational data, optimizes the values ​​of preset calibration parameters corresponding to a specified sensor in the self-moving device.

[0087] The designated sensor can refer to a sensor that needs to be calibrated, such as an odometer, camera, obstacle detector, distance detector, and speed and angle sensors of the self-moving device. The designated sensor can be a single sensor, meaning the calibration involves the relative position of a single sensor to the body of the self-moving device. For example, to calibrate the position of the odometer relative to the body, the designated sensor is the odometer. Alternatively, the designated sensor can also be the calibration of two or more sensors, meaning the calibration involves the relative position of two or more sensors. For example, to calibrate the relative position between a camera and an odometer, the designated sensors are both the camera and the odometer.

[0088] The preset calibration parameters are observable parameters selected from the calibration parameters of a specified sensor. Observable parameters refer to parameters that can be estimated based on the operating data of the self-moving device. For example, observability analysis can be used to analyze the calibration parameters of a specified sensor and select calibration parameters that can be estimated based on the operating data of the self-moving device, which are then used as preset calibration parameters. The observability analysis can be performed using methods such as nonlinear system observability analysis, VI-SLAM system observability analysis, or INS system observability analysis based on geometric features. For example, for the odometer of a robotic vacuum cleaner, the preset calibration parameters may include at least one of extrinsic parameters (such as rotational parameters), map points, and attitude. The map points may refer to the position of the robotic vacuum cleaner on a map, which is a map of the indoor environment.

[0089] For example, during the operation of a robotic vacuum cleaner, if observability analysis is performed on the calibration parameters of a specified sensor using global coordinate system X-direction positioning, the global positioning X-direction and yaw angle can be determined as observable parameters. Similarly, if observability analysis is performed using global coordinate system Y-direction positioning, the global positioning Y-direction and yaw angle can be determined as observable parameters. If observability analysis is performed using global coordinate system Z-direction positioning, the global positioning Z-direction can be determined as an observable parameter, while the yaw angle is an unobservable parameter. Furthermore, when the robotic vacuum cleaner performs pure rotational motion, observability analysis on the calibration parameters of the specified sensor can determine that the scale of feature points is unobservable; similarly, when the robotic vacuum cleaner performs constant acceleration motion, observability analysis on the calibration parameters of the specified sensor can also determine that the scale of feature points is unobservable. Through these observability analyses, observable parameters can be selected from the calibration parameters of the specified sensor as preset calibration parameters.

[0090] For example, when calibrating the odometer and camera of a robotic vacuum cleaner, the intrinsic parameters of the odometer and camera generally do not change significantly and can be calibrated offline. The online calibration process can mainly focus on calibrating the extrinsic parameters of the odometer and camera. The extrinsic parameters of the odometer and camera include rotation parameters and translation parameters. The rotation parameters characterize the rotational transformation between the camera coordinate system and the odometer coordinate system, while the translation parameters characterize the translational transformation between the camera coordinate system and the odometer coordinate system.

[0091] Observability analysis can be performed on the rotation and translation parameters of the odometer and camera of a robotic vacuum cleaner based on operational data. Since robotic vacuum cleaners mostly move in a straight line, observability analysis can be conducted on the rotation and translation parameters of the odometer and camera based on the operational data during this linear motion. This analysis ultimately determines that translational transformations along the X, Y, and Z axes are not observable, while rotational parameters are observable. Accordingly, rotational parameters can be used as preset calibration parameters for online calibration. Alternatively, by analyzing the robotic vacuum cleaner in other operating modes, operating modes suitable for calibrating translational parameters can be identified, and online calibration of translational parameters can be performed based on the operational data in those modes.

[0092] Correspondingly, observability analysis can be performed on the calibration parameters of specified sensors in advance based on operating data under different operating modes. This determines the observable parameters of the calibration parameters under different operating modes, and the selected observable parameters are used as preset calibration parameters, associated and stored with the corresponding operating modes. For example, in the above example, the linear motion, odometer, and camera rotation parameters can be associated and stored. Subsequently, the preset calibration parameters of the specified sensors can be calibrated online based on the operating data of the robot vacuum cleaner in the corresponding operating mode. In the above example, the operating data corresponding to the linear motion of the robot vacuum cleaner can be obtained to calibrate the odometer and camera rotation parameters online. By differentiating operating modes to select observable parameters, the selection of observable parameters can be made more accurate and comprehensive; at the same time, calibrating the observable parameters based on operating data under the corresponding operating modes can also make the observable parameter calibration more accurate.

[0093] Of course, if it is determined through analysis that the selection of observable parameters in the calibration parameters is independent of the operating mode, then the selected preset calibration parameters can be specially labeled. During online calibration, the preset calibration parameters can be calibrated without distinguishing the operating mode.

[0094] During online calibration, pre-extracted preset calibration parameters can be obtained, and the values ​​of the preset calibration parameters corresponding to the specified sensor can be optimized using the collected operating data to ensure the real-time and accurate representation of the relative position relationship of the sensors, thereby improving the accuracy of automatic mobile device control.

[0095] For example, the values ​​of preset calibration parameters can be optimized using a pre-built objective function and runtime data. The objective function can be used to characterize the correlation between the values ​​of the preset calibration parameters and the runtime data of the self-moving device.

[0096] First, the preset calibration parameters of a specified sensor can be calibrated offline, and the calibration results can be stored as parameter label values ​​for the preset calibration parameters. Then, the operating data of the robotic vacuum cleaner under stable operation based on the aforementioned parameter label values ​​can be acquired, and an objective function can be constructed using this operating data and the parameter label values. During the construction of the objective function, a least squares function can be used for constraints to accelerate its convergence. The least squares function can be used to characterize the error between the value predicted by the preset calibration parameters based on the operating data and the parameter label value. For ease of description, the value predicted by the preset calibration parameters based on the operating data will be simplified as the predicted value of the preset calibration parameters in the following text.

[0097] Based on the predicted values ​​and parameter label values ​​of the preset calibration parameters, the objective function is updated until the value of the pre-constructed least squares function meets the preset value conditions, thus obtaining the objective function. Since the least squares function can characterize the error between the predicted values ​​and parameter label values ​​of the preset calibration parameters, this error continuously decreases during the update process of the objective function, allowing the finally constructed objective function to more accurately represent the correlation between the running data and the parameter label values.

[0098] For example, the least squares function can be the sum of squares of the differences between the predicted values ​​of the preset calibration parameters and the parameter label values ​​(true values):

[0099]

[0100] in, This represents the predicted value of the preset calibration parameters. The parameter label value represents the preset calibration parameters.

[0101] The expression for the objective function can be Or it could be , This represents various operational data. , , … These are the function coefficients. In subsequent embodiments, let the expression of the objective function be: .

[0102] Converting the above objective function into matrix form, we have: X, therefore the expression for the least squares function is: .

[0103] Therefore, the least squares function can be viewed as a function of... Given a system of quadratic equations, find the minimum value of the least squares function L, because when L is minimized, the corresponding... The value is optimal, therefore continuously adjust... The value is adjusted to minimize L.

[0104] The least squares function is viewed as an L-squared function. The function of the variable can be obtained by calculating the first derivative of the least squares function:

[0105]

[0106] Simplifying the above formula, we get:

[0107]

[0108] Therefore, the simplified formula above can be used to calculate... ,Will Input into the objective function to calculate The value of the least squares function L is then calculated. When the value of the least squares function L meets the preset value conditions, the objective function is obtained. Then, the constructed objective function and the running data can be used to optimize the values ​​of the preset calibration parameters.

[0109] Of course, the above-mentioned optimization method for preset calibration parameters is a preferred example. The preset calibration parameters can also be optimized in other ways, such as by referring to the calibration parameter data optimization method used in the traditional offline calibration process. This is not a limitation.

[0110] In one embodiment, if the feature points in the running data under the current time window meet preset conditions, the values ​​of preset calibration parameters corresponding to a specified sensor in the self-moving device can be optimized.

[0111] After obtaining the preset calibration parameters, it can be determined whether to optimize these parameters based on whether the feature points in the running data within the current time window meet the preset conditions. If the feature points meet the preset conditions, the preset calibration parameters need to be optimized; if the feature points do not meet the preset conditions, the preset calibration parameters do not need to be optimized.

[0112] The feature points can also be data points extracted from the running data that meet certain feature requirements. For example, when the running data includes image information, the feature points can refer to pixels in the image that meet certain feature requirements. For instance, the feature points can be points where the image grayscale value changes drastically or points with large curvature at the image edge. Feature points in the image can be extracted using methods such as SIFT, SURF, and ORB. The preset conditions correspond to the analysis method used when analyzing the feature points of the running data. When the running data is laser point cloud data, the feature points can be data points extracted from the point cloud data that meet certain feature requirements.

[0113] Sensor calibration involves a large amount of data, especially real-time calibration, which consumes a significant amount of the automated mobile device's operating resources. Generally, if the automated mobile device operates stably, the calibration parameters regarding the relative positions of its sensors usually do not present major issues. However, if the automated mobile device's operational stability is poor within the current time window, the calibration parameters may not accurately reflect the relative positions of the sensors. By analyzing the actual operation of the automated mobile device within the current time window, determining whether sensor parameter calibration is necessary can effectively reduce the data processing volume while ensuring the stability of the automated mobile device's operation. Determining the operational status of the automated mobile device based on feature points can further reduce the data processing volume and resource consumption for this determination. Simultaneously, feature points can more accurately reflect the characteristics of the operational data, reducing noise interference and thus further improving the accuracy of the determination of the automated mobile device's operational status.

[0114] In some embodiments, the determination of whether the preset calibration parameters should be optimized can be made by using the reprojection error of feature points, or by judging the observability of feature points in a certain keyframe in other keyframes, as described below:

[0115] Method 1 uses whether the reprojection error of feature points meets preset conditions to determine whether the preset calibration parameters need to be optimized.

[0116] First, calculate the reprojection error of the feature points, and then compare the reprojection error with the preset error. If the reprojection error is less than or equal to the preset error, the preset calibration parameter does not need to be optimized; if the reprojection error is greater than the preset error, the preset calibration parameter needs to be optimized.

[0117] For example, the steps for calculating the reprojection error may include: acquiring image information captured by the built-in camera of the self-moving device during operation; determining the pixel position of the feature point in the image information; determining the projection position of the feature point based on the depth information between the self-moving device and the feature point; calculating the reprojection error of the feature point based on the pixel position and the projection position; when the reprojection error is greater than the preset error, the feature point meets the preset condition, indicating that the preset calibration parameter needs to be optimized, and at this time the preset calibration parameter is used as the preset calibration parameter for optimization.

[0118] The image information can be keyframes captured by a camera, i.e., video keyframes. Depth information can refer to the distance between the mobile device and the feature points in the world coordinate system. It should be noted that the number of feature points can be one or more.

[0119] For example, suppose the relationship between the pixel position of a feature point and the spatial position of its corresponding point is as follows:

[0120]

[0121] in, For depth information, This refers to the pixel position (i.e., pixel coordinates) of the feature point. This indicates the position of the feature point in the world coordinate system. K represents the projected position (i.e., projected coordinates) of the feature point, indicating the transformation of the feature point from the world coordinate system to the camera coordinate system. K is a coefficient.

[0122] Writing the above relationship in matrix form, we get: .

[0123] Due to noise issues, the above relationship contains an error. Therefore, by transforming the relationship, we can obtain the following error relationship:

[0124]

[0125] Based on the above error relationship, the error can be calculated. This error This refers to the reprojection error of the feature points. Since the reprojection error considers not only the calculation error of the homography matrix but also the measurement error of the feature points, it has high precision. Therefore, it can accurately determine whether the preset calibration parameters need to be optimized, which is beneficial to improving the optimization effect.

[0126] Method 2: Determine whether to optimize the preset calibration parameters based on whether the observability of a feature point in a certain key frame in other key frames meets the preset conditions.

[0127] Select at least two feature points in the target key frame of the running data; based on the at least two feature points, search for matching feature points in other key frames of the running data; when the ratio between the number of matching feature points and the number of at least two feature points is greater than or equal to a preset ratio, the feature points meet the preset conditions.

[0128] Specifically, the operational data can include video frames captured by the camera. Therefore, keyframes are sampled from the captured video frames. Feature points are then selected from a target keyframe, and other keyframes are searched for matching feature points. If a matching feature point is found in other keyframes, it indicates that the feature point in the target keyframe is observable in those other keyframes. When the ratio between the number of matching feature points and the number of extracted feature points is less than a preset ratio (e.g., the feature point ratio is less than 80%), it indicates that the feature points meet the preset conditions, and the preset calibration parameter needs optimization. When the ratio between the number of matching feature points and the number of extracted feature points is greater than or equal to the preset ratio (e.g., the feature point ratio is greater than 80%), the preset calibration parameter does not need optimization.

[0129] For example, for keyframes a and b, suppose feature points 1 to 10 are selected in keyframe a. Then, in keyframe b, it is checked whether there are feature points that match feature points 1 to 10. If 9 matching feature points are found, it is determined that the preset calibration parameter does not need to be optimized; if only 7 matching feature points are found, it is determined that the preset calibration parameter needs to be optimized. When determining whether to optimize the preset calibration parameter, it is not necessary to use the entire running data for judgment; only the feature point matching of the keyframes in the running data needs to be judged. Therefore, even if the running data contains a lot of data, the result of whether to optimize can be accurately determined, and the amount of computation can be effectively reduced.

[0130] S206, based on the optimized preset calibration parameters, outputs the calibration result of the specified sensor.

[0131] The optimized values ​​of the preset calibration parameters can be output as the calibration result of the specified sensor. Alternatively, during the optimization process, it can be determined whether the optimized values ​​of the preset calibration parameters meet the optimization conditions. If the conditions are met, the optimization process is stopped, and the optimized values ​​of the preset calibration parameters are obtained as the calibration result of the specified sensor.

[0132] For example, when calibrating the rotation parameters of the camera and odometer, the three-axis rotation is usually calibrated together. Therefore, an error in any one axis will affect the calibration of the other axes. The accuracy of the three-axis rotation calibration can be determined by analyzing the robot's running direction.

[0133] In some embodiments, the process of determining whether the value of the optimized preset calibration parameter meets the optimization conditions can be based on the difference in yaw angles, as follows:

[0134] During the optimization process, the yaw angle of the self-moving device is collected in real time; the difference between a preset number of yaw angles is calculated to obtain the difference in yaw angle of the self-moving device during the optimization process; when the difference meets the difference condition, the value of the preset calibration parameter after optimization is obtained.

[0135] For example, during the optimization process, the yaw angle of the mobile device can be collected in real time and stored in a buffer. When the number of stored yaw angles reaches a preset number, the collection of yaw angles stops and the difference calculation begins, thereby avoiding the collection of too many yaw angles and prolonging the optimization time.

[0136] This difference can be either variance or standard deviation. When the variance of the yaw angle of the self-moving device during the optimization process is not greater than the preset variance, or the standard deviation of the yaw angle of the self-moving device during the optimization process is not greater than the preset standard deviation, the optimization of the preset calibration parameter values ​​is stopped, and the optimized preset calibration parameter values ​​are obtained. Then, the optimized preset calibration parameter values ​​are used as the calibration results of the specified sensor.

[0137] For example, after obtaining a preset number of yaw angles for the robotic vacuum cleaner, the variance or standard deviation of the yaw angles during the optimization process is calculated. Then, the variance or standard deviation of the yaw angles is compared with the preset variance or standard deviation. Based on the comparison results, it is determined whether to stop the optimization process. During the optimization of preset calibration parameters, the yaw angle is significantly affected by these parameters. Using a preset number of yaw angles to evaluate whether to stop the optimization process allows for accurate determination of whether the preset calibration parameters have reached their optimal values. This avoids excessive yaw angle data collection, which prolongs the optimization time, and also improves the accuracy of parameter calibration.

[0138] Alternatively, in other embodiments, the posture of the self-moving device can be collected in real time during the optimization process; the difference between a preset number of postures can be calculated to obtain the difference in posture of the self-moving device during the optimization process; when the difference meets the difference condition, the value of the optimized preset calibration parameter is obtained. Since the posture of the self-moving device can be optimized while optimizing the value of the preset calibration parameter, directly using the difference between a preset number of postures to evaluate whether to stop the optimization process can avoid excessive posture collection and thus prolong the optimization time, and can also ensure the accuracy of posture optimization.

[0139] For example, during the optimization process, the robot's posture can be collected in real time and stored in a buffer. When the number of stored postures reaches a preset number, posture collection stops and difference calculation begins, thus avoiding excessive posture collection and extending the optimization time.

[0140] In the above embodiments, by pre-selecting calibration parameters that can be estimated based on operating data from the calibration parameters of the sensor as preset calibration parameters, and then using the operating data of the self-moving device to perform online calibration of the pre-selected preset calibration parameters, it is possible to achieve rapid online calibration of at least some calibration parameters, thereby improving the simplicity and efficiency of sensor calibration. At the same time, it is also possible to promptly correct the preset calibration parameters based on the real-time operating data of the self-moving device, ensuring the real-time and accurate representation of the relative positional relationship of the sensors, thereby improving the accuracy of self-moving device control.

[0141] In other embodiments, such as Figure 3 As shown, before S204, the method further includes:

[0142] S302, Obtain the calibration parameter sample dataset associated with the specified sensor.

[0143] The calibration parameter sample dataset contains the calibration results of specified sensors in sample devices; sample devices refer to devices with the same batch identifier as the self-moving device. The batch identifier refers to information identifying the production batch of the device. Typically, devices in the same production batch have similar device parameter information. For example, using a robotic vacuum cleaner from the same batch as the target one as a sample, the preset calibration parameters corresponding to the camera, odometer, and other sensors in the sample are obtained, and the preset calibration parameters are associated with the sensors and stored; simultaneously, the calibration results of the preset calibration parameters can also be stored. Since the parameters of devices in the same batch are similar, during optimization, the preset characterization parameters corresponding to specified sensors in devices in the same batch can be reused, eliminating the need to further filter preset calibration parameters, which can significantly reduce calibration complexity. Furthermore, the calibration results of the preset calibration parameters can be used as the initialization data of the self-moving device to be calibrated, which can further accelerate the data convergence speed in the initial calibration process and improve calibration efficiency.

[0144] S304. Using the calibration parameter sample dataset associated with the specified sensor, initialize the values ​​of the preset calibration parameters corresponding to the specified sensor in the self-moving device to obtain the initial parameter values ​​of the preset calibration parameters.

[0145] Using the values ​​of preset calibration parameters in the calibration parameter sample dataset, the values ​​of the corresponding preset calibration parameters for a specified sensor in the self-moving device are initialized to obtain the initialization parameter values ​​of the preset calibration parameters. For example, the rotation parameters of the camera in a sample of a robotic vacuum cleaner from the same batch as robotic vacuum cleaner A are obtained, and then the rotation parameters of the camera in robotic vacuum cleaner A are initialized using the values ​​of those camera rotation parameters to obtain the initialization parameter values ​​of the camera rotation parameters in robotic vacuum cleaner A.

[0146] S306 optimizes the initial parameter values ​​of the preset calibration parameters using running data.

[0147] As described in the above embodiments, the initial parameter values ​​of the preset calibration parameters can be optimized using an objective function, and the optimization process can be constrained using a least squares function. The objective function can be constructed based on the sample device; by reusing the objective function, the complexity of calibrating devices in the same batch can be further reduced. Alternatively, the reused objective function can be updated using operational data from the stable operation of the self-equipped device to be calibrated, allowing for online calibration of the preset calibration parameters using the updated objective function, thereby further improving the accuracy of online calibration. Of course, other methods can also be used to optimize the initial parameter values; this specification does not limit this approach.

[0148] In the above embodiments, a calibration parameter sample dataset associated with a specified sensor is used to initialize the values ​​of preset calibration parameters corresponding to the specified sensor in the self-moving device. Since the calibration parameter sample dataset contains calibration results from sample devices in the same batch as the self-moving device, a better initial parameter value can be obtained. Therefore, when optimizing the initial parameter values ​​of the preset calibration parameters using runtime data, the optimization speed can be accelerated, the optimization time shortened, and the optimization efficiency effectively improved, thereby effectively improving the parameter calibration efficiency. Furthermore, before optimizing the initial parameter values, the objective function is first optimized using a least squares function, and then the optimized objective function is used to optimize the initial parameter values ​​of the preset calibration parameters. This reduces the complexity of online calibration while making the sensor calibration more accurate.

[0149] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0150] Based on the same inventive concept, this application also provides a sensor calibration device for a self-moving device to implement the sensor calibration method for the self-moving device described above. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more embodiments of the sensor calibration device for self-moving devices provided below can be found in the limitations of the sensor calibration method for self-moving devices described above, and will not be repeated here.

[0151] In one embodiment, such as Figure 4 As shown, a sensor calibration device for a self-moving device is provided, comprising: a data acquisition module 402, an optimization module 404, and a determination module 406, wherein:

[0152] The acquisition module 402 is used to acquire running data from the mobile device within the current time window;

[0153] The optimization module 404 is used to optimize the values ​​of preset calibration parameters corresponding to a specified sensor in the self-moving device using operating data; the preset calibration parameters are observable parameters selected from the calibration parameters of the specified sensor, and observable parameters refer to parameters that can be estimated based on the operating data of the self-moving device.

[0154] The determination module 406 is used to output the calibration result of the specified sensor based on the optimized values ​​of the preset calibration parameters.

[0155] In one embodiment, the optimization module 404 is further configured to optimize the value of the preset calibration parameter corresponding to the specified sensor in the self-moving device when the feature points in the running data under the current time window meet the preset conditions.

[0156] In one embodiment, such as Figure 5 As shown, the device also includes:

[0157] The acquisition module 402 is also used to acquire image information from the mobile device during operation;

[0158] The determining module 406 is also used to determine the pixel position of the feature point in the image information; and to determine the projection position of the feature point based on the depth information between the self-moving device and the feature point.

[0159] The first calculation module 408 is used to calculate the reprojection error of feature points based on pixel position and projection position; when the reprojection error is greater than the preset error, the feature points meet the preset conditions.

[0160] In one embodiment, such as Figure 5 As shown, the device also includes:

[0161] The selection module 410 is used to select at least two feature points in the target keyframe of the running data;

[0162] The search module 412 is used to search for matching feature points in other key frames of the running data based on at least two feature points; when the ratio between the number of matching feature points and the number of at least two feature points is less than a preset ratio, the feature points meet the preset conditions.

[0163] In one embodiment, such as Figure 5 As shown, the device also includes:

[0164] The acquisition module 402 is also used to acquire the yaw angle of the mobile device in real time during the optimization process;

[0165] The second calculation module 414 is used to calculate the difference of a preset number of yaw angles to obtain the difference of the yaw angle of the self-moving device during the optimization process; when the difference meets the difference condition, the value of the optimized preset calibration parameter is obtained.

[0166] In one embodiment, the difference includes variance or standard deviation;

[0167] The second calculation module 414 is also used to stop optimizing the value of the preset calibration parameter when the variance of the yaw angle of the self-moving device during the optimization process is not greater than the preset variance, or the standard deviation of the yaw angle of the self-moving device during the optimization process is not greater than the preset standard deviation, and to obtain the optimized value of the preset calibration parameter.

[0168] The determination module 406 is also used to take the optimized preset calibration parameter values ​​as the calibration results of the specified sensor.

[0169] In the above embodiments, by pre-selecting calibration parameters that can be estimated based on operating data from the calibration parameters of the sensors as preset calibration parameters, and then using the real-time operating data of the self-moving device to perform online calibration of the pre-selected preset calibration parameters, it is possible to achieve rapid online calibration of at least some calibration parameters, thereby improving the simplicity and efficiency of camera and odometer parameter calibration. At the same time, it is also possible to promptly correct the preset calibration parameters based on the real-time operating data of the self-moving device, ensuring the real-time and accurate representation of the relative positional relationship of the sensors, and thus improving the accuracy of self-moving device control.

[0170] In one embodiment, such as Figure 5 As shown, the device also includes:

[0171] The acquisition module 416 is used to acquire the calibration parameter sample dataset associated with the specified sensor; wherein, the calibration parameter sample dataset contains the calibration results of the specified sensor in a sample device; the sample device refers to a device with the same batch identifier as the self-moving device;

[0172] The initialization module 418 is used to initialize the value of the preset calibration parameter corresponding to the specified sensor in the self-moving device using the calibration parameter sample dataset associated with the specified sensor, so as to obtain the initialization parameter value of the preset calibration parameter.

[0173] The optimization module 404 is also used to optimize the initial parameter values ​​of the preset calibration parameters using the running data.

[0174] In one embodiment, the optimization module 404 is further configured to optimize the values ​​of preset calibration parameters corresponding to a specified sensor in the self-moving device based on the operating data and the objective function; wherein the objective function is used to characterize the correlation between the values ​​of the preset calibration parameters and the operating data of the self-moving device; the objective function is constructed under the constraint of a least squares function, and the least squares function is used to characterize the error between the predicted value of the preset calibration parameter and the parameter label value.

[0175] In the above embodiments, by using a sample dataset of calibration parameters corresponding to a specified sensor from the same batch of sample devices, the values ​​of the preset calibration parameters corresponding to the specified sensor in the self-moving device are initialized, thereby obtaining better initial parameter values. Therefore, when optimizing the initial parameter values ​​of the preset calibration parameters using running data, the optimization speed can be accelerated, the optimization time can be shortened, and the optimization efficiency can be effectively improved, thereby effectively improving the parameter calibration efficiency.

[0176] Each module in the sensor calibration device of the aforementioned self-moving device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0177] In one embodiment, a computer device is provided, which may be a self-moving device (such as a robotic vacuum cleaner), and its internal structure diagram may be as follows: Figure 6 As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a sensor calibration method for a self-moving device. The display unit of the computer device is used to form a visually visible image. It can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons or a touchpad set on the casing of the computer device.

[0178] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0179] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of a sensor calibration method for an automatic mobile device.

[0180] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of a sensor calibration method for an independent mobile device.

[0181] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of a sensor calibration method for a self-moving device.

[0182] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0183] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0184] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0185] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A sensor calibration method for a self-moving device, characterized in that, The method includes: The data collected from the mobile device within the current time window includes feature points. If the reprojection error of the feature point is greater than a preset error, or if the ratio between the number of matching feature points and the number of feature points is less than a preset ratio, the values ​​of the preset calibration parameters corresponding to the specified sensor in the self-moving device are optimized using the running data; the feature point is the feature point in the target keyframe, and the matching feature point is the feature point in other keyframes that matches the feature point. Based on the optimized values ​​of the preset calibration parameters, the calibration result of the specified sensor is output.

2. The method according to claim 1, characterized in that, The method further includes: The self-moving device is used to collect image information during operation. Determine the pixel positions of feature points in the image information; Based on the depth information between the self-moving device and the feature point, the projection position of the feature point is determined; The reprojection error of the feature point is calculated based on the pixel position and the projection position.

3. The method according to claim 1, characterized in that, The method further includes: Select at least two of the feature points in the target keyframe of the running data; Based on the at least two of the aforementioned feature points, the matching feature points are searched in other keyframes of the running data.

4. The method according to any one of claims 1 to 3, characterized in that, Before collecting the running data from the mobile device within the current time window, the method further includes: Based on the operating data of the self-moving device under different operating modes, the observability analysis is performed on the calibration parameters of the specified sensor to obtain observable parameters under different operating modes; wherein, the observable parameters are preset calibration parameters; The preset calibration parameters are associated with and stored in relation to the operating mode; The step of optimizing the values ​​of preset calibration parameters corresponding to a specified sensor in the self-moving device using the operational data includes: The values ​​of the preset calibration parameters stored in the associated storage are optimized using the running data belonging to the running mode under the current time window.

5. The method according to any one of claims 1 to 3, characterized in that, The self-moving device includes a robotic vacuum cleaner, and the designated sensors include a camera and an odometer; Before collecting the running data from the mobile device within the current time window, the method further includes: From the calibration parameters of the camera and the odometer, calibration parameters that can be estimated based on the operating data of the sweeping robot are selected; wherein, the selected calibration parameters are preset calibration parameters; After associating the preset calibration parameters with the camera and the odometer, the preset calibration parameters are stored. The step of optimizing the values ​​of preset calibration parameters corresponding to a specified sensor in the self-moving device using the operational data includes: The stored values ​​of the preset calibration parameters are optimized using the operational data.

6. The method according to claim 5, characterized in that, The step of selecting calibration parameters that can be estimated based on the operating data of the sweeping robot from the calibration parameters of the camera and the odometer includes: Based on the operating data of the sweeping robot, an observability analysis was performed on the rotation parameters and translation of the camera and the odometer to obtain the observability analysis results. If the observability analysis results indicate that the rotation parameter is observable, then the rotation parameter is selected; wherein the rotation parameter is a calibration parameter that can be estimated based on the operating data of the sweeping robot.

7. A sensor calibration device for a self-moving device, characterized in that, The device includes: The acquisition module is used to acquire running data from the mobile device within the current time window, and the running data includes feature points; An optimization module is used to optimize the values ​​of preset calibration parameters corresponding to a specified sensor in the self-moving device using the running data when the reprojection error of the feature point is greater than a preset error, or the ratio between the number of matching feature points and the number of feature points is less than a preset ratio; the feature point is a feature point in the target keyframe, and the matching feature point is a feature point in other keyframes that matches the feature point. The determination module is used to output the calibration result of the specified sensor based on the optimized values ​​of the preset calibration parameters.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the method described in any one of claims 1 to 6.

Images