External parameter calibration failure degree determination method and device, electronic equipment and storage medium

Abstract

This application provides a method, apparatus, electronic device, and storage medium for determining the degree of failure of extrinsic parameter calibration, relating to the field of data processing technology. The method acquires the target position of a target device using a first sensor; acquires the target relative motion attitude of the target device using a second sensor; calculates the current error statistics based on the target position, the target relative motion attitude, and the extrinsic parameter calibration between the first and second sensors; and determines the degree of failure of the extrinsic parameter calibration based on the current error statistics. This allows for online monitoring of the effectiveness of the current extrinsic parameter calibration, thereby ensuring the reliability and safety of the equipment during operation.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular, to a method, apparatus, electronic device, and storage medium for determining the degree of failure of external parameter calibration. Background Technology

[0002] In mainstream attitude estimation fields both domestically and internationally, data fusion is typically employed to enhance the reliability and accuracy of data acquisition. This involves combining data from multiple sensors of the same or different categories. For example, in unmanned aerial vehicles (UAVs), inertial navigation sensors and multiple visual sensors are used for data fusion positioning. Similarly, in the field of autonomous driving, inertial navigation sensors, LiDAR sensors, and visual sensors are used for data fusion positioning.

[0003] In the process of using sensors to estimate the two-dimensional plane or three-dimensional space attitude of a device, it is necessary to obtain the relative positions of each sensor through calibration, so as to realize the data conversion and fusion between the sensors and finally obtain the overall spatial attitude of the device (e.g., vehicle or drone).

[0004] Currently, there are a large number of calibration algorithms in this field, such as Zhang Zhengyou's calibration method for pure vision systems, the nine-point calibration method for vision-robotic arm systems, and various online data calibration methods for visual-inertial odometry (VIO) systems.

[0005] However, in VIO systems or similar systems, even after the attitude conversion relationship (i.e., extrinsic parameter calibration) between multiple positioning devices is set, and a very high-precision extrinsic parameter calibration has been obtained, various software and hardware problems may still exist. For example, the mechanical mechanism may be deformed, loosened, aged, or even damaged, causing the calibration parameters to fail, thus making it impossible to ensure the reliability and safety of the equipment during operation. Summary of the Invention

[0006] This application provides a method, apparatus, electronic device, and storage medium for determining the degree of failure of external parameter calibration, in order to improve the above-mentioned problems.

[0007] In a first aspect, embodiments of this application provide a method for determining the degree of failure of extrinsic parameter calibration. The method includes: acquiring a target position of a target device using a first sensor, the target position including the current actual position of the target device and its position at a previous moment; acquiring a target relative motion posture of the target device using a second sensor, wherein the target relative motion posture is the relative motion posture of the target device at the current moment relative to the previous moment; calculating a current error statistical result based on the target position, the target relative motion posture, and the extrinsic parameter calibration between the first and second sensors; and determining the degree of failure of the extrinsic parameter calibration based on the current error statistical result.

[0008] Secondly, embodiments of this application provide an apparatus for determining the degree of failure of extrinsic parameter calibration. The apparatus includes: a position acquisition module, configured to acquire the target position of a target device via a first sensor, the target position including the current actual position of the target device and its position at the previous moment; an attitude acquisition module, configured to acquire the target relative motion attitude of the target device via a second sensor, wherein the target relative motion attitude is the relative motion attitude of the target device at the current moment relative to the previous moment; an error calculation module, configured to calculate a current error statistical result based on the target position, the target relative motion attitude, and the extrinsic parameter calibration between the first and second sensors; and a failure judgment module, configured to determine the degree of failure of the extrinsic parameter calibration based on the current error statistical result.

[0009] Thirdly, embodiments of this application provide an electronic device. The electronic device includes a memory, one or more processors, and one or more application programs. The one or more application programs are stored in the memory and configured to, when invoked by one or more processors, cause the one or more processors to execute the method provided in embodiments of this application.

[0010] Fourthly, embodiments of this application provide a computer-readable storage medium. This computer-readable storage medium stores program code that, when invoked by a processor, causes the processor to execute the method provided in embodiments of this application.

[0011] This application provides a method, apparatus, electronic device, and storage medium for determining the failure degree of extrinsic parameter calibration. The method calculates the current error statistics based on the target position of the target device obtained by the first sensor, the target relative motion posture of the target device obtained by the second sensor, and the extrinsic parameter calibration between the two sensors. Based on the error statistics, the failure degree of the extrinsic parameter calibration is determined. The method can adaptively judge the failure degree of the current extrinsic parameter calibration based on the reprojection error data (i.e., the error statistics), thereby enabling online monitoring of whether the current extrinsic parameter calibration is effective, and thus ensuring the reliability and safety of the equipment during operation. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a schematic diagram illustrating an application scenario of an external parameter calibration failure degree determination method provided in an exemplary embodiment of this application;

[0014] Figure 2 This is a flowchart illustrating a method for determining the degree of failure of external parameter calibration according to an embodiment of this application;

[0015] Figure 3 This is a flowchart illustrating a method for determining the degree of failure of external parameter calibration provided in another embodiment of this application;

[0016] Figure 4 This is a schematic diagram of the spatial attitude transformation relationship in the reprojection calculation process provided by an exemplary embodiment of this application;

[0017] Figure 5 This is a schematic diagram of the process for determining the degree of failure of external parameter calibration provided in an exemplary embodiment of this application;

[0018] Figure 6 This is a flowchart illustrating a method for determining the degree of failure of external parameter calibration provided in another embodiment of this application;

[0019] Figure 7 This is a flowchart illustrating an exemplary embodiment of the method for determining the degree of failure of external parameter calibration provided in this application;

[0020] Figure 8 This is a structural block diagram of an external parameter calibration failure degree determination device provided in an embodiment of this application;

[0021] Figure 9 This is a structural block diagram of an electronic device provided in an embodiment of this application;

[0022] Figure 10 This is a structural block diagram of a computer-readable storage medium provided in an embodiment of this application. Detailed Implementation

[0023] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0024] Please see Figure 1, Figure 1 This is a schematic diagram illustrating an application scenario of an extrinsic parameter calibration failure degree determination method provided in an exemplary embodiment of this application. The extrinsic parameter calibration failure degree determination system 100 includes a first sensor 110, a second sensor 120, and a target device 130. The first sensor 110, the second sensor 120, and the target device 130 can communicate via wired or wireless means. The first sensor 110 can acquire the position of the target device 130 in real time based on a first sensor coordinate system. The second sensor 120 can acquire the relative motion attitude of the target device 130 between two adjacent moments based on a second sensor coordinate system.

[0025] The first sensor 110 and the second sensor 120 can be devices with positioning functions. These devices may include, but are not limited to, vision sensors, inertial measurement units (IMUs), global positioning systems (GPS), lidar sensors, and millimeter-wave radar sensors. In some embodiments, the first sensor 110 can be a vision sensor, such as an image sensor. The second sensor 120 can be an IMU. It should be noted that in practical applications, the first sensor 110 and the second sensor 120 can be any pairwise combination of the aforementioned devices with positioning functions.

[0026] The target device 130 can be a device that needs to be located, such as a drone or a vehicle. The first sensor 110 and the second sensor 120 can be mounted on the target device 130 to obtain the position of the target device or the relative motion attitude between adjacent time points, respectively.

[0027] Please see Figure 2 , Figure 2 This is a flowchart illustrating a method for determining the degree of failure in external parameter calibration according to an embodiment of this application. This method can be applied to the above-mentioned... Figure 1 The external parameter calibration failure determination system 100 shown, or as mentioned below Figure 8 The external parameter calibration failure degree determination device 200 shown. The method may include the following steps S110-S140.

[0028] Step S110: Obtain the target position of the target device through the first sensor. The target position includes the current actual position of the target device and the position at the previous moment.

[0029] Here, "target device" refers to a device with positioning requirements, such as a drone or a vehicle. "Target position" refers to the spatial attitude of the target device in the first sensor coordinate system. For example, if the first sensor is a vision sensor, then the target position of the target device is its spatial attitude in the vision sensor coordinate system. The target position of the target device can also be the target position of its feature points.

[0030] In some implementations, the first sensor can acquire the target location of the target device in real time. In other implementations, the first sensor can acquire the target location of the target device at preset time intervals, wherein the preset time interval can be set according to actual needs, for example, 5 milliseconds. Specifically, the first sensor can acquire the target location of the target device by extracting feature points of the target device.

[0031] In some implementations, the first sensor may acquire the target location of the target device (feature points) by means of feature point extraction.

[0032] Step S120: Obtain the target relative motion attitude of the target device through the second sensor, wherein the target relative motion attitude is the relative motion attitude of the target device at the current moment relative to the previous moment.

[0033] In some implementations, the second sensor can obtain the target relative motion attitude of the target device through a triangulation algorithm.

[0034] Step S130: Calculate the current error statistics based on the target position, the target relative motion attitude, and the external parameter calibration between the first and second sensors.

[0035] In this context, extrinsic parameter calibration refers to parameters that are pre-set and stored in the system. Extrinsic parameter calibration can be set manually or automatically generated by the system. Extrinsic parameter calibration can be a specific matrix used to convert data in the first sensor coordinate system to data in the second sensor coordinate system, or vice versa.

[0036] It should be noted that the first sensor uses the first sensor coordinate system, while the second sensor uses the second sensor coordinate system. That is, the target position obtained by the first sensor and the target relative motion attitude obtained by the second sensor are in different coordinate systems. Therefore, extrinsic parameter calibration is needed to transform the target position and target relative motion attitude to the same coordinate system for calculation.

[0037] In some implementations, extrinsic parameter calibration can be used to transform the target position and the target relative motion attitude to the same coordinate system, and the reprojection error of the target device can be calculated based on the transformed target position and the target relative motion attitude, and the current error statistics can be calculated based on the reprojection error.

[0038] In some implementations, if a feature point of the target device is used for error determination, the reprojection error of a feature point can be used as the current error statistics result and stored in the system.

[0039] In other implementations, to avoid interference caused by individual mismatches or calculation errors, multiple feature points of the target device can be used, and the reprojection error of each feature point can be calculated separately, resulting in multiple reprojection errors. The average of these multiple reprojection errors is then used as the current error statistics. This improves the accuracy and reliability of the judgment results.

[0040] Step S140: Determine the degree of failure of the external parameter calibration based on the current error statistics.

[0041] Among them, the degree of failure of external parameter calibration can characterize the situation or cause of external parameter calibration failure, and different degrees of failure of external parameter calibration can characterize different situations or causes of external parameter calibration failure.

[0042] In some implementations, the current error statistics can be compared with a failure judgment threshold, and the degree of failure of the external parameter calibration can be determined based on the comparison result. The degree of failure of the external parameter calibration can be determined based on the failure judgment threshold. There is a mapping relationship between the failure judgment threshold and the degree of failure of the external parameter calibration. This mapping relationship can be one-to-one, one-to-many, many-to-one, or many-to-many. That is, different failure judgment thresholds in this embodiment correspond to different degrees of failure of the external parameter calibration, and this mapping relationship is positively correlated; that is, the larger the failure judgment threshold, the higher the degree of failure. The failure judgment threshold for the external parameter calibration can be a pre-set fixed parameter, or it can be determined based on the average of the error statistics corresponding to different operating stages of the target device.

[0043] In some implementations, multiple threshold comparison intervals can be set, with different threshold comparison intervals corresponding to different degrees of failure in external parameter calibration. A target threshold comparison interval can be determined based on the current error statistics, and the degree of failure corresponding to the target threshold comparison interval can be used as the degree of failure in external parameter calibration.

[0044] In some implementations, after determining the degree of failure of the external parameter calibration, the degree of failure can be sent to the technical personnel. This allows the technical personnel to understand the corresponding situation or cause of the external parameter calibration failure based on the degree of failure, and to take appropriate action, such as recalibration or repair.

[0045] In some implementations, a binocular vision algorithm can be used to determine the current error statistics to ascertain the degree of failure of the external parameter calibration. For details, please refer to the relevant technologies, which will not be described in detail here.

[0046] The method for determining the failure level of extrinsic parameter calibration provided in this application calculates the current error statistics based on the target position of the target device obtained by the first sensor, the target relative motion attitude of the target device obtained by the second sensor, and the extrinsic parameter calibration between the two sensors. It then determines the failure level of the extrinsic parameter calibration based on the error statistics. This method can adaptively judge the current failure level of the extrinsic parameter calibration based on reprojection error data (i.e., error statistics), thereby enabling online monitoring of the effectiveness of the current extrinsic parameter calibration and ensuring the reliability and safety of the equipment during operation. Furthermore, the method provided in this application has a low computational load; for systems with high computational requirements such as VIO systems, the computational consumption is negligible, thus saving computation time and resources.

[0047] Please see Figure 3 , Figure 3 This is a flowchart illustrating a method for determining the degree of failure in external parameter calibration according to another embodiment of this application. This method can be applied to the above... Figure 1 The external parameter calibration failure determination system 100 shown, or as mentioned below Figure 8 The external parameter calibration failure degree determination device 200 shown. The method may include the following steps S210-S260.

[0048] It should be noted that the embodiments of this application calculate the current error statistics based on a feature point of the target device.

[0049] Step S210: Obtain the target position of the target device through the first sensor. The target position includes the current actual position of the target device and the position at the previous moment.

[0050] The target location of the target device refers to the target location of the target feature points of the target device. The target feature points can be randomly selected, or they can be feature points corresponding to the centroid or center of gravity of the target device.

[0051] Step S220: Obtain the target relative motion attitude of the target device through the second sensor, wherein the target relative motion attitude is the relative motion attitude of the target device at the current moment relative to the previous moment.

[0052] Among them, the target relative motion attitude of the target device refers to the target relative running attitude of the target feature point, that is, the relative motion attitude of the target feature point at the current moment relative to the previous moment.

[0053] For a detailed description of steps S210-S220, please refer to the aforementioned steps S110-S120, which will not be repeated here.

[0054] Step S230: Estimate the current estimated position of the target feature point based on the position of the target feature point at the previous moment, the target relative motion posture of the target feature point, and the external parameter calibration.

[0055] As mentioned earlier, extrinsic parameter calibration can be a matrix, typically a spatial attitude transformation matrix.

[0056] In some implementations, an inverse matrix transformation can be performed on the extrinsic parameter calibration to obtain the inverse matrix of the extrinsic parameter calibration. The current estimated position of the target feature point can be estimated based on the previous position of the target feature point, the target relative motion attitude of the target feature point, the extrinsic parameter calibration, and the inverse matrix of the extrinsic parameter calibration.

[0057] Specifically, the current estimated position of the target feature point can be estimated using the following expression (1):

[0058] (1)

[0059] The parameters in expression (1) have the following meanings:

[0060] “ "Represents the current estimated position of the target feature point at the current moment;"

[0061] “ "Represents the position of the target feature point at the previous moment;"

[0062] “ "This characterizes the relative motion posture of the target feature points between the current moment and the previous moment;"

[0063] “ "Characteristic calibration of external parameters between the first and second sensors; and"

[0064] “ "The inverse matrix that characterizes the external parameter calibration."

[0065] For a better understanding of the above expression (1), please refer to [link / reference needed]. Figure 4 , Figure 4This is a schematic diagram illustrating the spatial attitude transformation relationship in the reprojection calculation process provided by an exemplary embodiment of this application. In the VIO system, the degree of failure of external parameter calibration can be determined by calculating the reprojection error of the feature point in the two sensor coordinate systems at time K (the previous time) and time K+1 (the current time). Assume the target feature point is... Figure 4 One of the feature points shown, then Figure 4 The data in the data satisfy the following expressions (2)-(6):

[0066] (2)

[0067] (3)

[0068] (4)

[0069] (5)

[0070] (6)

[0071] The parameters have the following meanings:

[0072] “ "This characterizes the spatial pose of the target feature points in the coordinate system of the visual sensor (first sensor) at time K (the previous time).

[0073] “ "Represents the spatial pose of the target feature points in the visual sensor coordinate system at time K+1 (the current time);

[0074] “ "Characterizes data from two consecutive frames in the visual sensor coordinate system (e.g., " "and" Spatial attitude transformation relationship between () and ().

[0075] “ "This characterizes the spatial attitude of the target feature points in the IMU (second sensor) coordinate system at time K (the previous time).

[0076] “ "Represents the spatial attitude of the target feature points in the IMU coordinate system at time K+1 (the current time);

[0077] “ "Represents the data of two consecutive frames in the IMU coordinate system (e.g., " "and" Spatial attitude transformation relationship between () and ().

[0078] “ "Characteristic calibration of the external parameters between the vision sensor and the IMU;"

[0079] “ "The inverse matrix that characterizes the external parameter calibration."

[0080] Based on expressions (2)-(6), we can obtain the above expression (1).

[0081] Step S240: Calculate the current error statistics based on the current actual position and the current estimated position of the target feature point.

[0082] In some implementations, the difference between the current actual position and the current estimated position of the target feature point can be calculated as the current error statistics result.

[0083] Specifically, the current error statistics can be calculated using the following expression:

[0084] (7)

[0085] The meanings of the parameters or expressions are as follows:

[0086] “ "Characterizes the current error statistics;

[0087] “ "Represents the current actual location; and"

[0088] “ "Represents the current estimated position."

[0089] It should be noted that after each calculation of the current error statistics, the system stores the current error statistics in the system so that other operations can be performed based on the current error statistics. For example, the failure judgment threshold can be calculated based on the current error statistics.

[0090] Step S250: Obtain the failure judgment threshold.

[0091] Among them, the failure judgment threshold is an indicator for determining the degree of failure, and there can be one or more failure judgment thresholds.

[0092] In some implementations, the failure judgment threshold includes a first failure judgment threshold, a second failure judgment threshold, and a third failure judgment threshold. The first failure judgment threshold is greater than the second failure judgment threshold, and the second failure judgment threshold is greater than the third failure judgment threshold. Obtaining the failure judgment threshold may include the following steps: calculating the average of all error statistics within a first preset time period after setting the external parameters as the first failure judgment threshold; calculating the average of all error statistics from the start of operation of the target device to the current time as the second failure judgment threshold; and calculating the average of all error statistics within a second preset time period before the current time as the third failure judgment threshold.

[0093] The system determines the corresponding failure judgment threshold based on the error statistics of the target equipment at different operating stages. It can adaptively set the failure judgment threshold according to the error statistics, which can improve the accuracy of the failure judgment threshold and reduce the dependence on manual intervention in the judgment process.

[0094] Step S260: Determine the degree of failure of the external parameter calibration based on the current error statistics and failure judgment threshold.

[0095] In some implementations, the failure judgment thresholds include the aforementioned first failure judgment threshold, second failure judgment threshold, and third failure judgment threshold. A specific implementation of step S260 may include the following steps: if the current error statistics result is greater than the first failure judgment threshold, determine that the external parameter calibration has reached a first failure level; if the current error statistics result is less than or equal to the first failure judgment threshold and greater than the second failure judgment threshold, determine that the external parameter calibration has reached a second failure level, wherein the second failure level is lower than the first failure level; if the current error statistics result is less than or equal to the second failure judgment threshold and greater than the third failure judgment threshold, determine that the external parameter calibration has reached a third failure level, wherein the third failure level is lower than the second failure level. If the current error statistics result is less than or equal to the third failure judgment threshold, then it can be determined that the external parameter calibration is currently in a valid state.

[0096]

[0097] In this embodiment, as shown in Table 1, the first failure judgment threshold, the second failure judgment threshold, and the third failure judgment threshold define multiple failure judgment threshold intervals. Each failure judgment threshold interval corresponds to a failure degree or an effective state. Therefore, the target failure judgment threshold interval can be determined based on the current error statistics, and the failure degree or effective state corresponding to the target failure judgment threshold interval can be directly obtained. This allows for the determination of the failure degree of the external parameter calibration or the assessment that the external parameter calibration is in an effective state. The entire calculation process is simple and computationally inexpensive, thus saving computational resources and time, and improving judgment efficiency.

[0098] As an example, please refer to Figure 5 , Figure 5 This is a schematic flowchart illustrating the determination of the degree of failure of external parameter calibration provided in an exemplary embodiment of this application. "Characterizing the current error statistics," "Characterizing the first failure judgment threshold," "Characterizing the second failure judgment threshold," "This characterizes the third failure judgment threshold." "Characterizing the first degree of failure," "Characterizing the second degree of failure," "This characterizes the third degree of failure."

[0099] like" > If so, the failure level is "". ";like" < ≤ If so, the failure level is "". ";like" < ≤ If so, the failure level is "". ";like" ≤ If the value is "", then the external parameter calibration is currently in a valid state, and the external parameter calibration failure determination process can be terminated.

[0100] It should be noted that the specific failure judgment threshold range and the mapping relationship and number of failure degrees can be set according to actual needs, and are not limited to the three mapping relationships limited here.

[0101] The method for determining the failure degree of external parameter calibration provided in this application embodiment, in addition to having, Figure 2 In addition to the technical effects of the embodiments shown, the following technical effects are also achieved: by introducing the average error statistics of different operating stages of the target device as different failure judgment thresholds, and by adaptively determining the failure judgment thresholds, the influence of human factors can be reduced, ensuring high data consistency. This approach is applicable to VIO systems of different scales and accuracies, thereby improving the accuracy, real-time performance, versatility, and universality of the judgment results.

[0102] Please see Figure 6 , Figure 6 This is a flowchart illustrating a method for determining the degree of failure in external parameter calibration according to another embodiment of this application. This method can be applied to the above... Figure 1 The external parameter calibration failure determination system 100 shown, or as mentioned below Figure 8 The external parameter calibration failure degree determination device 200 shown. The method may include the following steps S310-S360.

[0103] It should be noted that the embodiments of this application calculate the current error statistics based on multiple feature points of the target device. Compared to Figure 3 In the illustrated embodiment, the current estimated position is multiple. The previous position of the device includes the previous position of each feature point of the target device, and the current actual position of the target device includes the current actual position of each feature point. The target relative motion attitude includes the target relative motion attitude of each feature point.

[0104] Step S310: Obtain the target position of multiple feature points of the target device through the first sensor. The target position includes the current actual position of the feature points and the position at the previous moment.

[0105] Step S320: The target relative motion posture of multiple feature points of the target device is acquired by the second sensor, wherein the target relative motion posture is the relative motion posture of the feature points of the target device at the current moment relative to the previous moment.

[0106] For details on how to obtain the target position and relative motion posture of each feature point in steps S310 and S320, please refer to steps S110-S120, which will not be repeated here.

[0107] Step S330: Based on the position of each feature point at the previous moment, the target relative motion posture of each feature point, and the external parameter calibration, estimate the current estimated position of each feature point to obtain multiple current estimated positions.

[0108] The specific implementation of estimating the current estimated position of each feature point based on its previous position, the target relative motion posture of each feature point, and the external parameter calibration is described in step S230 above and will not be repeated here.

[0109] Step S340: Based on the current actual position and the current estimated position of each feature point, calculate the reprojection error for each feature point, resulting in multiple reprojection errors. Calculate the average of the multiple reprojection errors as the current error statistics.

[0110] In some implementations, the reprojection error of each feature point can be calculated using the following expression:

[0111] (8)

[0112] The meanings of the parameters or expressions are as follows:

[0113] “ "Characteristic of the first Reprojection error of each feature point;

[0114] “ "Characteristic of the first The current actual position of each feature point; and

[0115] “ "Characteristic of the first The current estimated position of each feature point.

[0116] The current error statistics can be calculated using the following expression:

[0117] (9)

[0118] in," "Characterizes the current error statistics;" ")" is for calculating the average value; " "Characterizes the sum of reprojection errors of multiple feature points." "Characteristic data" The 2-norm is used to solve for the magnitude of a vector. Since the reprojection error is the positional deviation of points in a two-dimensional plane, i.e., " "Since two-dimensional vectors are inconvenient to calculate, the 2-norm method is used to calculate the magnitude of the vector, in order to simplify the calculation of two-dimensional vectors." Convert to a scalar.

[0119] It should be noted that after each calculation of the current error statistics, the system stores the current error statistics in the system so that other operations can be performed based on the current error statistics. For example, the failure judgment threshold can be calculated based on the current error statistics.

[0120] Step S350: Obtain the failure judgment threshold.

[0121] Step S360: Determine the degree of failure of the external parameter calibration based on the current error statistics and failure judgment threshold.

[0122] For a detailed description of steps S350-S360, please refer to the aforementioned steps S250-S260, which will not be repeated here.

[0123] The method for determining the failure degree of external parameter calibration provided in this application embodiment, in addition to having, Figure 2 In addition to the technical effects of the illustrated embodiments, the following technical effects are also achieved: By introducing the average error statistics of the target device at different operating stages as different failure judgment thresholds, and adaptively determining the failure judgment thresholds, the influence of human factors can be reduced, ensuring high data consistency. This approach is applicable to VIO systems of different scales and precisions, thereby improving the accuracy, real-time performance, versatility, and universality of the judgment results. Furthermore, the embodiments of this application use multiple feature points of the target device for comprehensive judgment, which can avoid interference caused by individual mismatches or calculation errors, thereby improving the reliability and accuracy of the judgment results.

[0124] Please see Figure 7 , Figure 7 This is a flowchart illustrating a method for determining the degree of failure of external parameter calibration provided in an exemplary embodiment of this application. This method can be applied to the above... Figure 1 The external parameter calibration failure determination system 100 shown, or as mentioned below Figure 8 The external parameter calibration failure degree determination device 200 shown.

[0125] An image sensor is used as the first sensor, and an IMU is used as the second sensor. The image sensor can acquire feature points of the target device and estimate the current actual position of the feature points based on the relative position of the feature points of the target device with the first sensor. The reference frame feature point position refers to the position of the feature point at the previous moment. The IMU can acquire the relative motion attitude of the target device (feature points) between the current moment and the previous moment. The system can directly read known extrinsic parameter calibrations. The system can estimate the current estimated position (i.e., reprojection position) of the feature points based on the extrinsic parameter calibration, the position of the feature points at the previous moment, and the relative motion attitude. It can also calculate and store the current error statistics based on the current estimated position and the current actual position. The system can calculate a failure judgment threshold based on the current error statistics and the previously calculated error statistics. The failure degree of the extrinsic parameter calibration is determined based on the failure judgment threshold and the current error statistics. For parts not described in detail in this exemplary embodiment, please refer to the relevant parts in the foregoing embodiments, which will not be repeated here.

[0126] Please see Figure 8 , Figure 8 This is a structural block diagram of an external parameter calibration failure degree determination device according to an embodiment of this application. The external parameter calibration failure degree determination device 200 can be applied to the aforementioned external parameter calibration failure degree determination system 100. The external parameter calibration failure degree determination device 200 includes a position acquisition module 210, an attitude acquisition module 220, an error calculation module 230, and a failure judgment module 240.

[0127] The location acquisition module 210 is used to acquire the target location of the target device through the first sensor, wherein the target location includes the current actual location of the target device and the location at the previous moment;

[0128] The attitude acquisition module 220 is used to acquire the target relative motion attitude of the target device through the second sensor, wherein the target relative motion attitude is the relative motion attitude of the target device at the current moment relative to the previous moment;

[0129] Error calculation module 230 is used to calculate the current error statistics result based on the target position, the target relative motion posture, and the external parameter calibration between the first sensor and the second sensor;

[0130] The failure determination module 240 is used to determine the degree of failure of the external parameter calibration based on the current error statistics results.

[0131] In some implementations, the error calculation module 230 is further configured to estimate the current estimated position of the target device based on the previous position of the target device, the relative motion posture of the target, and the external parameter calibration; and to calculate the current error statistics based on the current actual position and the current estimated position of the target device.

[0132] In some implementations, the current estimated position is multiple, the previous position of the target device includes the previous position of each feature point of the target device, and the target relative motion attitude includes the target relative motion attitude of each feature point. The error calculation module 230 is further configured to estimate the current estimated position of each feature point based on the previous position of each feature point, the target relative motion attitude of each feature point, and the extrinsic parameter calibration.

[0133] In some embodiments, the current actual position of the target device includes the current actual position of each feature point. The error calculation module 230 is further configured to calculate the reprojection error of each feature point based on the current actual position and the current estimated position of each feature point, thereby obtaining multiple reprojection errors; and calculate the average of the multiple reprojection errors as the current error statistics result.

[0134] In some implementations, the failure determination module 240 is also used to obtain a failure determination threshold; and to determine the degree of failure of the external parameter calibration based on the current error statistics and the failure determination threshold.

[0135] In some embodiments, the failure judgment threshold includes a first failure judgment threshold, a second failure judgment threshold, and a third failure judgment threshold, wherein the first failure judgment threshold is greater than the second failure judgment threshold. The failure judgment module 240 is further configured to calculate the average of all error statistics within a first preset time period after the external parameter calibration is set as the first failure judgment threshold; calculate the average of all error statistics from the start of operation of the target device to the current time as the second failure judgment threshold; and calculate the average of all error statistics within a second preset time period before the current time as the third failure judgment threshold.

[0136] In some embodiments, the failure determination module 240 is further configured to: determine that the external parameter calibration has reached a first failure level if the current error statistics result is greater than the first failure determination threshold; determine that the external parameter calibration has reached a second failure level if the current error statistics result is less than or equal to the first failure determination threshold and greater than the second failure determination threshold, wherein the second failure level is lower than the first failure level; and determine that the external parameter calibration has reached a third failure level if the current error statistics result is less than or equal to the second failure determination threshold and greater than the third failure determination threshold, wherein the third failure level is lower than the second failure level.

[0137] Those skilled in the art will clearly understand that the external parameter calibration failure degree determination device 200 provided in the embodiments of this application can implement the external parameter calibration failure degree determination method provided in the embodiments of this application. The specific working process of the above-mentioned device and module can be found in the corresponding process of the external parameter calibration failure degree determination method in the embodiments of this application, and will not be repeated here.

[0138] In the embodiments provided in this application, the coupling, direct coupling, or communication connection between the modules shown or discussed may be indirect coupling or communication coupling through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms. The embodiments of this application do not limit this.

[0139] Furthermore, the functional modules in the embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules, and this application embodiment does not impose any restrictions on this.

[0140] Please see Figure 9 , Figure 9 This is a structural block diagram of an electronic device provided in an embodiment of this application. The electronic device 300 may include one or more of the following components: a memory 310, one or more processors 320, and one or more application programs, wherein the one or more application programs may be stored in the memory 310 and configured to, when invoked by one or more processors 320, cause one or more processors 320 to execute the above-described external parameter calibration failure degree determination method provided in the embodiment of this application.

[0141] The processor 320 may include one or more processing cores. The processor 320 uses various interfaces and lines to connect to various parts of the entire electronic device 300, and is used to run or execute instructions, programs, code sets or instruction sets stored in the memory 310, as well as to call and run or execute data stored in the memory 310, and perform various functions of the electronic device 300 and process data.

[0142] In some implementations, the processor 320 may be implemented in at least one of the following hardware forms: digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA).

[0143] In some implementations, processor 320 may integrate one or a combination of several of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into processor 320 and may be implemented using a separate communication chip.

[0144] The memory 310 may include random access memory (RAM) or read-only memory (ROM). The memory 310 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 310 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function, instructions for implementing the various method embodiments described above, etc. The data storage area may store data created by the electronic device 300 during use.

[0145] Please see Figure 10 , Figure 10 This is a structural block diagram of a computer-readable storage medium provided in an embodiment of this application. The computer-readable storage medium 400 stores program code 410, which is configured to, when called by a processor, cause the processor to execute the above-described method for determining the degree of failure of external parameter calibration provided in this embodiment of the application.

[0146] The computer-readable storage medium 400 may be an electronic storage device such as flash memory, electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), hard disk, or ROM.

[0147] In some embodiments, the computer-readable storage medium 400 includes a non-volatile computer-readable storage medium (Non-TCRSM). The computer-readable storage medium 400 has storage space for program code 410 that performs any of the method steps described above. This program code 410 can be read from or written to one or more computer program products. The program code 410 may be compressed in a suitable form.

[0148] In summary, this application provides a method, apparatus, electronic device, and storage medium for determining the degree of failure of extrinsic parameter calibration, relating to the field of data processing technology. The method acquires the target position of a target device using a first sensor; acquires the target relative motion attitude of the target device using a second sensor; calculates the current error statistics based on the target position, the target relative motion attitude, and the extrinsic parameter calibration between the first and second sensors; and determines the degree of failure of the extrinsic parameter calibration based on the current error statistics. This allows for online monitoring of the effectiveness of the current extrinsic parameter calibration, thereby ensuring the reliability and safety of the equipment during operation.

[0149] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and not to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for determining a failure degree of an extrinsic calibration, characterized in that, The method comprises: obtaining a target position of a target device through a first sensor, the target position comprising a current actual position and a previous position of the target device; obtaining a target relative motion posture of the target device through a second sensor, wherein the target relative motion posture is a relative motion posture of the target device at a current time relative to a previous time; calculating a current error statistical result according to the target position, the target relative motion posture, and an external parameter calibration between the first sensor and the second sensor; determining a failure degree of the external parameter calibration according to the current error statistical result; the step of determining the failure degree of the external parameter calibration according to the current error statistical result comprises: obtaining a failure judgment threshold, the failure judgment threshold comprising a first failure judgment threshold, a second failure judgment threshold, and a third failure judgment threshold, the first failure judgment threshold being greater than the second failure judgment threshold, the second failure judgment threshold being greater than the third failure judgment threshold, the first failure judgment threshold being a mean value of all error statistical results in a first preset time period after the external parameter calibration is set, the second failure judgment threshold being a mean value of all error statistical results between a time when the target device starts running and a current time, and the third failure judgment threshold being a mean value of all error statistical results in a second preset time period before the current time; if the current error statistical result is greater than the first failure judgment threshold, determining that the external parameter calibration reaches a first failure degree; if the current error statistical result is less than or equal to the first failure judgment threshold and greater than the second failure judgment threshold, determining that the external parameter calibration reaches a second failure degree, wherein the second failure degree is lower than the first failure degree; if the current error statistical result is less than or equal to the second failure judgment threshold and greater than the third failure judgment threshold, determining that the external parameter calibration reaches a third failure degree, wherein the third failure degree is lower than the second failure degree.

2. The method of claim 1, wherein, the step of calculating a current error statistical result according to the target position, the target relative motion posture, and an external parameter calibration between the first sensor and the second sensor comprises: estimating a current estimated position of the target device according to a previous position of the target device, the target relative motion posture, and the external parameter calibration; calculating a current error statistical result according to a current actual position and the current estimated position of the target device.

3. The method of claim 2, wherein, The current estimated position is multiple, the previous position of the target device comprises a previous position of each feature point of the target device, the target relative motion posture comprises a target relative motion posture of each feature point, and the step of estimating a current estimated position of the target device according to a previous position of the target device, the target relative motion posture, and the external parameter calibration comprises: estimating a current estimated position of each feature point according to a previous position of each feature point, a target relative motion posture of each feature point, and the external parameter calibration.

4. The method of claim 3, wherein, The current actual position of the target device includes current actual positions of the each feature point, and the step of calculating the current error statistical result according to the current actual position and the current estimated position of the target device includes: calculating a re-projection error of each feature point according to the current actual position and the current estimated position of the each feature point, to obtain a plurality of re-projection errors; calculating a mean value of the plurality of re-projection errors as the current error statistical result.

5. An external parameter calibration failure degree determination apparatus characterized by comprising: The method comprises: a position acquisition module configured to acquire a target position of a target device through a first sensor, the target position including a current actual position of the target device and a position at a previous time; a posture acquisition module configured to acquire a target relative motion posture of the target device through a second sensor, the target relative motion posture being a relative motion posture of the target device at a current time relative to a previous time; an error calculation module configured to calculate a current error statistical result according to the target position, the target relative motion posture, and an external parameter calibration between the first sensor and the second sensor; a failure determination module configured to determine a failure degree of the external parameter calibration according to the current error statistical result; the failure determination module is further configured to acquire a failure determination threshold, the failure determination threshold including a first failure determination threshold, a second failure determination threshold, and a third failure determination threshold, the first failure determination threshold being greater than the second failure determination threshold, the second failure determination threshold being greater than the third failure determination threshold, the first failure determination threshold being a mean value of all error statistical results within a first preset time period after the external parameter calibration is set, the second failure determination threshold being a mean value of all error statistical results between a time when the target device starts running and the current time, and the third failure determination threshold being a mean value of all error statistical results within a second preset time period before the current time; if the current error statistical result is greater than the first failure determination threshold, it is determined that the external parameter calibration reaches a first failure degree; if the current error statistical result is less than or equal to the first failure determination threshold and greater than the second failure determination threshold, it is determined that the external parameter calibration reaches a second failure degree, wherein the second failure degree is lower than the first failure degree; and if the current error statistical result is less than or equal to the second failure determination threshold and greater than the third failure determination threshold, it is determined that the external parameter calibration reaches a third failure degree, wherein the third failure degree is lower than the second failure degree.

6. An electronic device, comprising: The method comprises: a memory; one or more processors; one or more application programs, wherein the one or more application programs are stored in the memory and are configured to, when invoked by the one or more processors, cause the one or more processors to perform the method of any one of claims 1-4.

7. A computer readable storage medium, characterized in that, The computer-readable storage medium stores program code, which is configured to, when invoked by a processor, cause the processor to perform the method of any one of claims 1-4.

Images