Camera dynamic calibration method, device, electronic device and storage medium

Abstract

The present invention provides a camera dynamic calibration method, device, electronic device and storage medium, which can dynamically calibrate and adjust the determined camera external parameters with the help of the operation data of the mobile robot. Based on this, it can be ensured that the camera data obtained by the mobile robot after dynamic calibration is consistent with the real scene, and alleviate or even avoid data anomalies caused by changes in the camera installation position due to bumpy operation of the mobile robot.

Description

Technical Field

[0001] The present invention relates to the field of software technology, and more specifically, to a camera dynamic calibration method, device, electronic equipment and storage medium. Background Art

[0002] For mobile robots, in addition to ensuring the accuracy of sensor installation, calibrating the external parameters of cameras (including depth cameras, point cloud cameras, etc.) is an important task in processing data sources.

[0003] At present, although there are many methods to try to improve the calibration accuracy of the camera, so that the installation position and the parameters configured in the algorithm can be consistent, and the world observed by the mobile robot from the camera is consistent with the real situation. However, in the process of productization and operation, there are still many situations where the installation position of the camera will change slightly, such as disassembling and reassembling the camera, aging during operation, and looseness caused by vibration.

[0004] Therefore, how to dynamically calibrate the camera extrinsic parameters during the operation of the mobile robot is an important guarantee for the smooth operation of the robot. Summary of the invention

[0005] In view of this, in order to solve the above problems, the present invention provides a camera dynamic calibration method, device, electronic device and storage medium, and the technical solution is as follows:

[0006] One aspect of the present invention provides a camera dynamic calibration method, the method comprising:

[0007] Capturing operation data of the mobile robot under the determined camera external parameters, wherein the operation data includes camera data and inertial navigation data;

[0008] Performing attitude adjustment on the camera data based on the inertial navigation data to determine an attitude angle corresponding to the camera data;

[0009] Extracting principal plane features from the camera data, and adjusting the camera extrinsic parameters according to the attitude angle and the principal plane features.

[0010] Preferably, the performing attitude adjustment on the camera data based on the inertial navigation data to determine the attitude angle corresponding to the camera data includes:

[0011] Aligning the inertial navigation data and the camera data according to timestamps;

[0012] Determine, based on the alignment result, first target inertial navigation data and second target inertial navigation data that are closest to the camera data in the inertial navigation data, wherein a timestamp of the first target inertial navigation data is earlier than that of the camera data, and a timestamp of the second target inertial navigation data is later than that of the camera data;

[0013] The attitude angle corresponding to the camera data is calculated by interpolation according to the first target inertial navigation data and the second target inertial navigation data.

[0014] Preferably, adjusting the camera extrinsic parameters according to the attitude angle and the main plane feature includes:

[0015] Calculating the tilt degree of the mobile robot using the attitude angle;

[0016] processing the main plane feature based on the degree of tilt;

[0017] A calibration adjustment amount is calculated according to the tilt degree, the main plane feature and the processed main plane feature, and the calibration adjustment amount is the basis for adjusting the camera extrinsic parameters.

[0018] Preferably, the adjusting the camera extrinsic parameters according to the attitude angle and the main plane feature further includes:

[0019] The calibration adjustment amounts are averaged to obtain actual calibration adjustment amounts, so as to adjust the camera extrinsic parameters according to the actual calibration adjustment amounts.

[0020] Preferably, before averaging the calibration adjustment amounts to obtain the actual calibration adjustment amounts, the method further comprises:

[0021] The calibration adjustment amount is screened according to a determined screening condition, wherein the screening condition is determined based on that the main plane feature belongs to a ground feature.

[0022] Another aspect of the present invention provides a camera dynamic calibration device, the device comprising:

[0023] A data capture module, used to capture the operation data of the mobile robot under the determined camera external parameters, wherein the operation data includes camera data and inertial navigation data;

[0024] An attitude adjustment module, used for performing attitude adjustment on the camera data based on the inertial navigation data to determine an attitude angle corresponding to the camera data;

[0025] The extrinsic parameter adjustment module is used to extract the main plane features in the camera data and adjust the camera extrinsic parameters according to the attitude angle and the main plane features.

[0026] Preferably, the posture adjustment module is specifically used for:

[0027] The inertial navigation data and the camera data are aligned according to the timestamp; based on the alignment result, first target inertial navigation data and second target inertial navigation data that are closest to the camera data are determined, wherein the timestamp of the first target inertial navigation data is earlier than that of the camera data, and the timestamp of the second target inertial navigation data is later than that of the camera data; and an attitude angle corresponding to the camera data is calculated in an interpolation manner according to the first target inertial navigation data and the second target inertial navigation data.

[0028] Preferably, the extrinsic parameter adjustment module for adjusting the camera extrinsic parameter according to the attitude angle and the main plane feature is specifically used for:

[0029] The inclination degree of the mobile robot is calculated using the attitude angle; the main plane features are processed based on the inclination degree; and a calibration adjustment amount is calculated according to the inclination degree, the main plane features and the processed main plane features, wherein the calibration adjustment amount is the basis for adjusting the camera extrinsic parameters.

[0030] Another aspect of the present invention provides an electronic device, comprising: at least one memory and at least one processor; the memory stores a program, the processor calls the program stored in the memory, and the program is used to implement any one of the camera dynamic calibration methods described.

[0031] Another aspect of the present invention provides a storage medium, wherein the storage medium stores computer executable instructions, and the computer executable instructions are used to execute any one of the camera dynamic calibration methods described above.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] The present invention provides a camera dynamic calibration method, device, electronic device and storage medium, which can dynamically calibrate and adjust the determined camera external parameters with the help of the operation data of the mobile robot. Based on this, it can be ensured that the camera data obtained by the mobile robot after dynamic calibration is consistent with the real scene, and alleviate or even avoid data anomalies caused by changes in the camera installation position due to bumpy operation of the mobile robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0035] Figure 1 A method flow chart of a camera dynamic calibration method provided by an embodiment of the present invention;

[0036] Figure 2 A partial method flow chart of a camera dynamic calibration method provided by an embodiment of the present invention;

[0037] Figure 3 An alignment schematic diagram provided for an embodiment of the present invention;

[0038] Figure 4 A schematic diagram of ground features provided by an embodiment of the present invention;

[0039] Figure 5 A flowchart of another part of the camera dynamic calibration method provided by an embodiment of the present invention;

[0040] Figure 6 A schematic diagram of ground features provided by an embodiment of the present invention;

[0041] Figure 7 Another ground feature schematic diagram provided by an embodiment of the present invention;

[0042] Figure 8 A schematic diagram of the structure of a camera dynamic calibration device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0045] The embodiment of the present invention provides a camera dynamic calibration method, which can be applied to a processor built into a mobile robot, and can also be applied to a backend server of the mobile robot. The method flow chart of the method is as follows: Figure 1 As shown, the following steps are included:

[0046] S10, capturing the operating data of the mobile robot under the determined camera external parameters, where the operating data includes camera data and inertial navigation data.

[0047] In the embodiment of the present invention, the determined camera extrinsic parameters may be existing camera extrinsic parameters that have not been dynamically calibrated, such as camera extrinsic parameters calibrated at the factory, or camera extrinsic parameters that were dynamically calibrated in the last round, which is not limited in the embodiment of the present invention. In addition, the operation data is the real data of the mobile robot performing the task under the determined camera parameters.

[0048] In addition, in an embodiment of the present invention, the camera and inertial navigation system of the mobile robot operate at respective frame rates, so the camera data and inertial navigation data in the captured operating data have respective frame rates. Accordingly, the camera data is composed of depth data at multiple consecutive times, and each depth data has its unique timestamp, and the inertial navigation data is composed of imu (Inertial measurement unit) data at multiple consecutive times, and each imu data has its unique timestamp.

[0049] S20, adjusting the attitude of the camera data based on the inertial navigation data to determine an attitude angle corresponding to the camera data.

[0050] In the embodiment of the present invention, since the imu can sense the posture of the mobile robot, the camera data and the inertial navigation data can be preprocessed based on the timestamp by analyzing the data characteristics of the camera data and the inertial navigation data to ensure that the frame rates of the two are the same, so that the depth data has an imu data with the same timestamp as its timestamp, and then the posture angle of the imu data is used as the posture angle of the depth data.

[0051] It should be noted that the above preprocessing includes the necessary operations of alignment and interpolation used in the subsequent embodiments of the present invention, and may also include other auxiliary operations such as invalid data filtering, which is not limited in the embodiments of the present invention.

[0052] In the specific implementation process, step S20 "adjusting the attitude of the camera data based on the inertial navigation data to determine the attitude angle corresponding to the camera data" can adopt the following steps, and the method flow chart is as follows Figure 2 As shown, the following steps are included:

[0053] S201, aligning the inertial navigation data and the camera data according to the timestamp.

[0054] In the embodiment of the present invention, as above, the camera data and the inertial navigation data have their own frame rates, so the depth data and the imu data cannot be matched one by one according to the timestamps they have. At this time, the depth data and the imu data can be aligned according to their respective timestamps, taking time as the dimension.

[0055] See also Figure 3As shown in the alignment diagram, the depth data a_1 and imu data b_1 have the same timestamp, and the depth data a_3 and imu data b_4 have the same timestamp. Therefore, the depth data a_1 and imu data b_1 are aligned, the depth data a_3 and imu data b_4 are aligned, and the depth data a_2 has its timestamp between imu data b_2 and imu data b_3, so the depth data a_2 is between imu data b_2 and imu data b_3.

[0056] S202, determining first target inertial navigation data and second target inertial navigation data that are closest to the camera data in the inertial navigation data based on the alignment result, wherein the timestamp of the first target inertial navigation data is earlier than that of the camera data, and the timestamp of the second target inertial navigation data is later than that of the camera data.

[0057] In the embodiment of the present invention, continue with Figure 3 For example, for the depth data a_1 and the depth data a_3, since there are aligned imu data, the attitude angle of the imu data b_1 can be directly used as the attitude angle corresponding to the depth data a_1, and the attitude angle of the imu data b_4 can be used as the attitude angle of the depth data a_3.

[0058] For the depth data a_2, the target inertial navigation data closest to it are imu data b_2 and imu data b_3. Therefore, the first target inertial navigation data of the depth data a_2 is imu data b_2, and the second target inertial navigation data is imu data b_3.

[0059] S203, calculating the attitude angle corresponding to the camera data in an interpolation manner according to the first target inertial navigation data and the second target inertial navigation data.

[0060] In the embodiment of the present invention, interpolation processing is performed in the following manner:

[0061] (imu1.t-depth.t) / (depth.t-imu2.t) is equivalent to

[0062] (imu1.quaternion-depth.quaternion) / (depth.quaternion-imu2.quaternion)

[0063] Continue with Figure 3For example, for depth data a_2, in order to obtain its corresponding attitude angle, imu data b_2 and imu data b_3 can be interpolated. Therefore, the above imu1.t is the timestamp of imu data b_3, depth.t is the timestamp of depth data a_2, imu2.t is the timestamp of imu data b_2, imu1.quaternion is the attitude angle of imu data b_3, depth.quaternion is the attitude angle corresponding to depth data a_2, and imu2.quaternion is the attitude angle of the timestamp of imu data b_2.

[0064] S30, extracting the main plane features in the camera data, and adjusting the camera extrinsic parameters according to the attitude angle and the main plane features.

[0065] In the embodiment of the present invention, when the mobile robot performs a task, the main plane observed by the mobile robot is generally the ground. Figure 4 The ground features shown in the figure are the main plane features extracted from the camera data by the present invention as the ground features. Specifically, for each depth data in the camera data, the maximum likelihood can be used to estimate the fitting plane therein, and then the features of the maximum fitting plane are used as the ground features. Thus, one main plane feature can be extracted from one depth data, and the main plane feature is used as the ground feature of the depth data.

[0066] When adjusting the camera extrinsic parameters, the goal is to minimize the deviation between the overall optimization of the ground features in the camera data and the actual ground features, and calculate a calibration adjustment amount for adjusting the camera extrinsic parameters. In the specific implementation process, the "adjusting the camera extrinsic parameters according to the attitude angle and the main plane features" in step S30 can be carried out in the following steps. The method flow chart is as follows Figure 5 As shown:

[0067] S301, calculating the tilt degree of the mobile robot using the attitude angle.

[0068] S302, processing the main plane feature based on the tilt degree.

[0069] S303, calculating a calibration adjustment amount according to the tilt degree, the main plane features and the processed main plane features, where the calibration adjustment amount is the basis for adjusting the camera extrinsic parameters.

[0070] In the embodiment of the present invention, the relevant coordinate system is first described:

[0071] The mobile robot has a central coordinate system, which takes the center of the two wheels of the mobile robot as the coordinate origin, and the central coordinate system is recorded as the R coordinate system; the camera has a camera coordinate system when initially installed, and the camera coordinate system is recorded as the C coordinate system; during the operation of the mobile robot, there are quite a few situations in which the position of the camera will change slightly, and accordingly, the camera coordinate system will also change accordingly, and the camera coordinate system is recorded as the C' coordinate system; for the camera data, it exists in the form of a point cloud, so the present invention also involves a point cloud coordinate system, and the point cloud coordinate system is recorded as the P coordinate system. In addition, relative to the ground, the mobile robot also has a ground projection coordinate system, and the coordinate origin of the ground projection coordinate system is the projection of the center of the two wheels of the mobile robot on the ground, and the ground projection coordinate system is recorded as the B coordinate system. Ideally, if the position of the camera does not change, the R coordinate system and the B coordinate system coincide.

[0072] The transformation matrices between the above coordinate systems satisfy the following conditions:

[0073] P2R=C2R*C`2C*P2C`

[0074] Among them, P2R represents the transformation matrix of the P coordinate system relative to the R coordinate system, C2R represents the transformation matrix of the C coordinate system relative to the R coordinate system, C`2C represents the transformation matrix of the C` coordinate system relative to the C coordinate system, and P2C` represents the transformation matrix of the P coordinate system relative to the C` coordinate system.

[0075] Obviously, when the camera actually observes the depth data, it is based on P2C`, so P2C` can be replaced by the main ground feature. In addition, C`2C is the conversion matrix that the present invention needs to fine-tune and calibrate for the camera, that is, the calibration adjustment amount used to adjust the camera extrinsic parameters.

[0076] For example, the corresponding attitude angle of a depth data can represent the tilt degree of the mobile robot at this time. Therefore, the tilt degree of the mobile robot at this attitude angle can be calculated by the trans matrix conversion. The transformation matrix of the tilt degree is recorded as R2B. Therefore, considering the imu attitude, the transformation matrix of the P coordinate system relative to the B coordinate system is P2B = R2B*P2R.

[0077] Continuing with the example of depth data, assuming that the corresponding main plane feature is recorded as Pplane2R, the point cloud relative to the B coordinate system after considering the imu posture is PplaneF=R2B*Pplane2R, that is, the main plane feature after processing is recorded as PplaneF. Since the plane point cloud recognized by the mobile robot with the highest probability during operation is the ground, PplaneF is ideally a rotation unit quaternion with a translation of (0, 0, 0) and a rotation of (0, 0, 0, 1). Combined with the above-mentioned "P2R=C2R*C`2C*P2C`", it can be deduced that:

[0078] R2B.inverse()*PplaneF=C2R*C`2C*P2C`

[0079] Therefore, C`2C = C2R.inverse() * R2B.inverse() * PplaneF * P2C`.inverse(), where inverse() represents the inverse matrix of the transformation matrix. For example, C2R.inverse() represents the inverse matrix of the transformation matrix of the C coordinate system relative to the R coordinate system. Since C2R is determined at the time of installation, the degree of inclination is substituted into R2B, the processed main plane features are substituted into PplaneF, and the main plane features are substituted into P2C`, and C`2C can be calculated, which is the calibration adjustment corresponding to a depth data.

[0080] In summary, when the mobile robot is performing a task, each depth data in the camera data involves ground recognition. If a perfect ground point cloud is not required, a calibration adjustment corresponding to each depth data can be calculated through the above calculation. Obviously, in the embodiment of the present invention, one calibration adjustment can be selected from multiple calibration adjustments corresponding to all depth data as the calibration adjustment for actually adjusting the camera extrinsic parameters. The selection rule can be set in combination with the scene, for example, a calibration adjustment belonging to the median can be selected.

[0081] In the embodiment of the present invention, the quaternion average algorithm is used to average the multiple calibration adjustment values ​​corresponding to all the depth data to obtain the actual calibration adjustment value, and the camera extrinsic parameters are adjusted with the actual calibration adjustment value. This can satisfy the C`2C with the minimum cost of all conversion calibration matrices, that is, the actual calibration result.

[0082] In addition, since the main plane features extracted from the depth data cannot be guaranteed to be ground features, in some other embodiments, after obtaining the calibration adjustment corresponding to each depth data, all calibration adjustment amounts can be screened according to the determined screening conditions to ensure that the extracted main ground features are ground features. Specifically, an allowable range can be set in the screening conditions, and only calibration adjustment amounts within the allowable range can be screened, and then one calibration adjustment amount can be selected from the screened calibration adjustment amounts as the actual calibration adjustment amount for adjusting the camera extrinsic parameters, or all the screened calibration adjustment amounts can be averaged to obtain the actual calibration adjustment amount.

[0083] It should be noted that by substituting the actual calibration adjustment amount back into the P2B calculation, the acquisition of the point cloud will not be affected by the robot's bumps or sensor installation errors.

[0084] It should also be noted that, in order to verify the effect of the present invention, the present invention compares the overall optimization deviation between the ground features before and after the application of the present invention and the actual ground features. Figure 6 Before the present invention is applied, the ground features extracted from the depth data have a large deviation from the overall optimization of the actual ground features. Figure 7 The ground feature map shown in the figure shows that the overall optimization deviation between the ground features extracted from the depth data and the actual ground features is substantially zero. Obviously, the present invention can alleviate or even avoid data anomalies caused by changes in the camera installation position of the mobile robot due to bumpy operation.

[0085] Based on the camera dynamic calibration method provided in the above embodiment, an embodiment of the present invention provides a device for executing the camera dynamic calibration method. The structural diagram of the device is shown in FIG. Figure 8 As shown, including:

[0086] A data capture module 10 is used to capture the operation data of the mobile robot under the determined camera external parameters, and the operation data includes camera data and inertial navigation data;

[0087] An attitude adjustment module 20, used to perform attitude adjustment on the camera data based on the inertial navigation data to determine an attitude angle corresponding to the camera data;

[0088] The extrinsic parameter adjustment module 30 is used to extract the main plane features in the camera data and adjust the camera extrinsic parameters according to the attitude angle and the main plane features.

[0089] Optionally, the posture adjustment module 20 is specifically used for:

[0090] The inertial navigation data and the camera data are aligned according to the timestamp; based on the alignment result, the first target inertial navigation data and the second target inertial navigation data closest to the camera data are determined, the timestamp of the first target inertial navigation data is earlier than that of the camera data, and the timestamp of the second target inertial navigation data is later than that of the camera data; according to the first target inertial navigation data and the second target inertial navigation data, the attitude angle corresponding to the camera data is calculated by interpolation.

[0091] Optionally, an extrinsic parameter adjustment module 30 for adjusting the camera extrinsic parameters according to the attitude angle and the main plane features is specifically used for:

[0092] The tilt degree of the mobile robot is calculated using the attitude angle; the main plane features are processed based on the tilt degree; the calibration adjustment amount is calculated based on the tilt degree, the main plane features and the processed main plane features, and the calibration adjustment amount is the basis for adjusting the camera extrinsic parameters.

[0093] Optionally, the external parameter adjustment module 30 is further used for:

[0094] The calibration adjustment amount is averaged to obtain the actual calibration adjustment amount, so as to adjust the camera extrinsic parameters according to the actual calibration adjustment amount.

[0095] Optionally, the external parameter adjustment module 30 is further used for:

[0096] The calibration adjustment amount is screened according to the determined screening condition, and the screening condition is determined based on that the main plane feature belongs to the ground feature.

[0097] It should be noted that the detailed functions of each functional module in the embodiment of the present invention can be found in the corresponding disclosed part of the above-mentioned camera dynamic calibration method embodiment, and will not be repeated here.

[0098] Based on the camera dynamic calibration method provided in the above embodiment, an embodiment of the present invention further provides an electronic device, which includes: at least one memory and at least one processor; the memory stores a program, the processor calls the program stored in the memory, and the program is used to implement the camera dynamic calibration method.

[0099] Based on the camera dynamic calibration method provided by the above embodiment, an embodiment of the present invention further provides a storage medium, in which computer executable instructions are stored, and the computer executable instructions are used to execute the camera dynamic calibration method.

[0100] The above is a detailed introduction to a camera dynamic calibration method, device, electronic device and storage medium provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

[0101] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0102] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device that includes a series of elements is inherent to the elements, or also includes elements inherent to these processes, methods, articles or devices. In the absence of further restrictions, the elements defined by the sentence "including a..." do not exclude the presence of other identical elements in the process, method, article or device that includes the elements.

[0103] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A camera dynamic calibration method, characterized in that: The method comprises: Capturing operation data of the mobile robot under the determined camera external parameters, wherein the operation data includes camera data and inertial navigation data; Performing attitude adjustment on the camera data based on the inertial navigation data to determine an attitude angle corresponding to the camera data; Extracting principal plane features from the camera data, and adjusting the camera extrinsic parameters according to the attitude angle and the principal plane features; The adjusting the camera extrinsic parameters according to the attitude angle and the main plane feature comprises: calculating the inclination degree of the mobile robot by using the attitude angle; processing the main plane feature based on the inclination degree; calculating a calibration adjustment amount according to the inclination degree, the main plane feature and the processed main plane feature, wherein the calibration adjustment amount is the basis for adjusting the camera extrinsic parameters; The calculating of the calibration adjustment amount according to the tilt degree, the main plane feature and the processed main plane feature comprises: calculating the calibration adjustment amount according to a central coordinate system, a camera coordinate system, a point cloud coordinate system and a ground projection coordinate system; wherein the central coordinate system is a coordinate system with the center of the two wheels of the mobile robot as the coordinate origin, and the ground projection coordinate system is a coordinate system with the projection of the center of the two wheels of the mobile robot on the ground as the coordinate origin; The calibration adjustment amount is calculated by the following formula: C`2C = C2R.inverse()*R2B.inverse()*PplaneF*P2C`.inverse(); wherein, C`2C is a calibration adjustment corresponding to a depth data, C2R.inverse() is the inverse matrix of the transformation matrix of the C coordinate system relative to the R coordinate system, R2B.inverse() is the inverse matrix of the transformation matrix of the R coordinate system relative to the B coordinate system, PplaneF is the processed main plane feature, P2C`.inverse() is the inverse matrix of the transformation matrix of the P coordinate system relative to the C` coordinate system, the R coordinate system is the central coordinate system, the C coordinate system is the camera coordinate system, the P coordinate system is the point cloud coordinate system, the B coordinate system is the ground projection coordinate system, and the C` coordinate system is the camera coordinate system during the operation of the mobile robot.

2. The method according to claim 1, characterized in that The step of performing attitude adjustment on the camera data based on the inertial navigation data to determine an attitude angle corresponding to the camera data includes: Aligning the inertial navigation data and the camera data according to timestamps; Determine, based on the alignment result, first target inertial navigation data and second target inertial navigation data that are closest to the camera data in the inertial navigation data, wherein a timestamp of the first target inertial navigation data is earlier than that of the camera data, and a timestamp of the second target inertial navigation data is later than that of the camera data; The attitude angle corresponding to the camera data is calculated by interpolation according to the first target inertial navigation data and the second target inertial navigation data.

3. The method according to claim 1, characterized in that The adjusting the camera extrinsic parameters according to the attitude angle and the main plane feature also includes: The calibration adjustment amounts are averaged to obtain actual calibration adjustment amounts, so as to adjust the camera extrinsic parameters according to the actual calibration adjustment amounts.

4. The method according to claim 3, characterized in that Before averaging the calibration adjustment amounts to obtain the actual calibration adjustment amounts, the method further includes: The calibration adjustment amount is screened according to a determined screening condition, wherein the screening condition is determined based on that the main plane feature belongs to a ground feature.

5. A camera dynamic calibration device, characterized in that: The device comprises: A data capture module, used to capture the operation data of the mobile robot under the determined camera external parameters, wherein the operation data includes camera data and inertial navigation data; An attitude adjustment module, used for performing attitude adjustment on the camera data based on the inertial navigation data to determine an attitude angle corresponding to the camera data; An extrinsic parameter adjustment module, used for extracting the main plane features in the camera data, and adjusting the camera extrinsic parameters according to the attitude angle and the main plane features; The extrinsic parameter adjustment module for adjusting the camera extrinsic parameters according to the attitude angle and the main plane feature is specifically used to: calculate the inclination degree of the mobile robot using the attitude angle; process the main plane feature based on the inclination degree; calculate the calibration adjustment amount according to the inclination degree, the main plane feature and the processed main plane feature, and the calibration adjustment amount is the basis for adjusting the camera extrinsic parameters; The calculating of the calibration adjustment amount according to the tilt degree, the main plane feature and the processed main plane feature comprises: calculating the calibration adjustment amount according to a central coordinate system, a camera coordinate system, a point cloud coordinate system and a ground projection coordinate system; wherein the central coordinate system is a coordinate system with the center of the two wheels of the mobile robot as the coordinate origin, and the ground projection coordinate system is a coordinate system with the projection of the center of the two wheels of the mobile robot on the ground as the coordinate origin; The calibration adjustment amount is calculated by the following formula: C`2C = C2R.inverse()*R2B.inverse()*PplaneF*P2C`.inverse(); wherein, C`2C is a calibration adjustment corresponding to a depth data, C2R.inverse() is the inverse matrix of the transformation matrix of the C coordinate system relative to the R coordinate system, R2B.inverse() is the inverse matrix of the transformation matrix of the R coordinate system relative to the B coordinate system, PplaneF is the processed main plane feature, P2C`.inverse() is the inverse matrix of the transformation matrix of the P coordinate system relative to the C` coordinate system, the R coordinate system is the central coordinate system, the C coordinate system is the camera coordinate system, the P coordinate system is the point cloud coordinate system, the B coordinate system is the ground projection coordinate system, and the C` coordinate system is the camera coordinate system during the operation of the mobile robot.

6. The device according to claim 5, characterized in that The posture adjustment module is specifically used for: Aligning the inertial navigation data and the camera data according to the timestamp; determining first target inertial navigation data and second target inertial navigation data that are closest to the camera data in the inertial navigation data based on the alignment result, wherein the timestamp of the first target inertial navigation data is earlier than that of the camera data, and the timestamp of the second target inertial navigation data is later than that of the camera data; The attitude angle corresponding to the camera data is calculated by interpolation according to the first target inertial navigation data and the second target inertial navigation data.

7. An electronic device, characterized in that: The electronic device includes: at least one memory and at least one processor; the memory stores a program, the processor calls the program stored in the memory, and the program is used to implement the camera dynamic calibration method described in any one of claims 1-4.

8. A storage medium, characterized in that: The storage medium stores computer executable instructions, and the computer executable instructions are used to execute the camera dynamic calibration method described in any one of claims 1 to 4.

Images