External parameter evaluation methods, devices, electronic equipment and storage media

By obtaining the difference between the real and calculated coordinates of the target of the vision sensor, and combining the allowable deviation of the Euler angles of the extrinsic parameter, the accuracy of the extrinsic parameter of the vision sensor is evaluated using the three-dimensional sliding window method. This solves the problem of poor application effect of vision sensor caused by poor extrinsic parameter calibration and realizes accurate extrinsic parameter evaluation.

CN117422773BActive Publication Date: 2026-11-17GUANGZHOU XIAOPENG CONNECTIVITY TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202311381777.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-10-23
Publication Date
2026-11-17
Estimated Expiration
2043-10-23

AI Technical Summary

Technical Problem

Without a clear understanding of the quality of extrinsic parameter calibration, the application effect of vision sensors may be poor, necessitating an effective method for evaluating extrinsic parameters.

Method used

By obtaining the target's true coordinates in the specified world coordinate system, the calculated coordinates of the target in the pixel coordinate system are solved using the extrinsic parameters to be evaluated. The difference between the calculated coordinates and the true coordinates is used to determine the evaluation error. The allowable error range is determined based on the allowable deviation of the Euler angles of the extrinsic parameters. The accuracy of the extrinsic parameters is evaluated by combining the three-dimensional sliding window method.

Benefits of technology

This invention enables accurate evaluation of the extrinsic parameters of a vision sensor by combining the inverse projection of the target coordinates and Euler angles, even when the true values ​​of the extrinsic parameters are unknown, thus ensuring the effective application of the vision sensor.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117422773B_ABST
    Figure CN117422773B_ABST
Patent Text Reader

Abstract

This application relates to a method, apparatus, electronic device, and storage medium for evaluating extrinsic parameters. The method includes: acquiring the true coordinates of a target in a specified world coordinate system; after acquiring the target's coordinates in a pixel coordinate system, calculating the target's coordinates in the world coordinate system based on the extrinsic parameters to be evaluated; obtaining an evaluation error based on the calculated and true coordinates of the target; determining an allowable error range for the target's coordinates in the world coordinate system based on the allowable deviations of each Euler angle of the extrinsic parameters; and evaluating the extrinsic parameters to be evaluated corresponding to the evaluation error based on the determined allowable error range. The solution provided by this application can evaluate extrinsic parameters at the angular level.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of visual sensor technology, and in particular to an external parameter evaluation method, apparatus, electronic device, and storage medium. Background Technology

[0002] Visual sensors are increasingly being used in fields such as robotics, online inspection, industrial flaw detection, and autonomous driving. For example, visual sensors can capture images and analyze and process them to detect and locate target objects in the images.

[0003] To implement the aforementioned techniques, it is typically necessary to calibrate the parameters of the vision sensor. These parameters include intrinsic and extrinsic parameters. The intrinsic parameters describe the inherent properties of the vision sensor itself, while the extrinsic parameters describe the position and orientation of the vision sensor within the 3D scene. The extrinsic parameters can characterize the coordinate transformation relationship between the visual coordinate system and a specified world coordinate system (or global coordinate system).

[0004] Using extrinsic parameters without knowing their calibration quality can lead to poor performance of the vision sensor if the calibration is inadequate. Therefore, a technique for evaluating extrinsic parameters is needed. Summary of the Invention

[0005] To address or partially address the problems existing in related technologies, this application provides an external parameter evaluation method, apparatus, electronic device, and storage medium, which can evaluate external parameters from a perspective.

[0006] The first aspect of this application provides a method for evaluating external parameters, the method comprising:

[0007] Obtain the target's true coordinates in the specified world coordinate system;

[0008] After obtaining the target's coordinates in the pixel coordinate system, the target's calculated coordinates in the world coordinate system are obtained based on the external parameters to be evaluated.

[0009] The evaluation error is obtained based on the calculated and actual coordinates of the target.

[0010] The allowable error range of the target's coordinates in the world coordinate system is determined based on the allowable deviation of each Euler angle of the external parameters;

[0011] Based on the determined allowable error range, the external parameters to be evaluated corresponding to the evaluation error are evaluated.

[0012] In an optional implementation, obtaining the target's true coordinates in a specified world coordinate system includes:

[0013] The target, positioned at a set location, is measured using a measuring tool to obtain its true coordinates in the world coordinate system.

[0014] In an optional implementation, the step of determining the allowable error range of the target's coordinates in the world coordinate system based on the allowable deviations of the various Euler angles of the extrinsic parameters includes:

[0015] Using Euler angles as zero as a reference, the angle range of each Euler angle is determined according to the set threshold of each Euler angle, and a three-dimensional sliding window is set according to the allowable deviation of each Euler angle, wherein the allowable upper deviation is less than the upper limit of the threshold.

[0016] The three-dimensional sliding window is used to traverse each Euler angle according to a set step size, wherein each time the three-dimensional sliding window slides once, a target error E of the true coordinate is obtained;

[0017] Select the error vector E with the largest magnitude from all the obtained target errors E. max The error vector E with the smallest magnitude min This serves as the allowable error limit for the actual coordinates.

[0018] In an optional implementation, obtaining a target error E of the true coordinates includes:

[0019] Based on the current position of the three-dimensional sliding window, determine the first set of coordinates of the target in the world coordinate system, and the reference coordinates corresponding to the median value of the current three-dimensional sliding window;

[0020] Calculate the error vector of each element in the first coordinate set relative to the reference coordinates, and select the error vector that is in the same direction as the evaluation error from all the calculated error vectors;

[0021] The error vector with the largest magnitude is selected from the chosen error vectors as the target error E.

[0022] In an optional implementation, the step of evaluating the extrinsic parameter to be evaluated corresponding to the evaluation error based on a determined allowable error range includes:

[0023] Compare the evaluation error with the error vector E max The error vector E min The modulus length;

[0024] If the magnitude of the evaluation error is less than or equal to the error vector E min If the modulus is equal to the required value, then the external parameter to be evaluated meets the accuracy requirements.

[0025] If the magnitude of the evaluation error is greater than the error vector E min The magnitude is less than or equal to the error vector E.max If the modulus is such that the external parameter to be evaluated may meet the accuracy requirements;

[0026] If the magnitude of the evaluation error is greater than the error vector E max If the modulus is less than or equal to the required value, then the external parameter to be evaluated does not meet the accuracy requirements.

[0027] In an optional implementation, the step of determining the allowable error range of the target's coordinates in the world coordinate system based on the allowable deviations of the various Euler angles of the extrinsic parameters includes:

[0028] Using the Euler angles of the external parameter to be evaluated as a reference, the angle range of the corresponding Euler angles is obtained according to the allowable deviation of each Euler angle;

[0029] Determine the second set of coordinates of the target in the world coordinate system based on the obtained angle range;

[0030] Calculate the error vector of each element in the second coordinate set relative to the calculated coordinates, which serves as the allowable error range for the calculated coordinates.

[0031] In an optional implementation, the step of evaluating the extrinsic parameter to be evaluated corresponding to the evaluation error based on a determined allowable error range includes:

[0032] Determine whether the opposite vector of the evaluation error exists within the allowable error range of the calculated coordinates. If so, the external parameter to be evaluated meets the accuracy requirement; otherwise, it does not.

[0033] A second aspect of this application provides an external parameter evaluation device, the device comprising:

[0034] The real coordinate acquisition module is configured to acquire the real coordinates of the target in a specified world coordinate system;

[0035] The coordinate calculation module is configured to, after obtaining the target's coordinates in the pixel coordinate system, calculate the target's coordinates in the world coordinate system based on the external parameters to be evaluated.

[0036] An evaluation error acquisition module is configured to obtain the evaluation error based on the calculated and actual coordinates of the target.

[0037] The allowable error determination module is configured to determine the allowable error range of the target's coordinates in the world coordinate system based on the allowable deviations of each Euler angle of the extrinsic parameters;

[0038] The extrinsic parameter evaluation module is configured to evaluate the extrinsic parameters to be evaluated corresponding to the evaluation error based on a determined error allowable range.

[0039] A third aspect of this application provides an electronic device, comprising:

[0040] Processor; and

[0041] A memory that stores executable code, which, when executed by the processor, causes the processor to perform the method described above.

[0042] A fourth aspect of this application provides a computer-readable storage medium having executable code stored thereon, which, when executed by a processor of an electronic device, causes the processor to perform the method described above.

[0043] The technical solution provided in this application may include the following beneficial effects:

[0044] The technical solution of this application first obtains the evaluation error based on the calculated and true coordinates of the target in a specified world coordinate system. Then, it determines the allowable error range of the target's coordinates in the world coordinate system based on the allowable deviation mapping of the three Euler angles in the extrinsic rotation matrix. Finally, it evaluates the extrinsic parameters to be evaluated corresponding to the evaluation error by combining the determined allowable error range. This application inversely projects the two-dimensional coordinates of the target in the pixel coordinate system to three-dimensional space to obtain the calculated coordinates, compares them with the true coordinates to determine the evaluation error, and uses the Euler angles pitch, roll, and yaw in the extrinsic rotation matrix to evaluate the accuracy of the sensor's extrinsic parameters.

[0045] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0046] The above and other objects, features and advantages of this application will become more apparent from the more detailed description of exemplary embodiments thereof in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments thereof.

[0047] Figure 1 This is a schematic diagram illustrating an application scenario of the external parameter evaluation method shown in the embodiments of this application;

[0048] Figure 2 This is a schematic flowchart illustrating the external parameter evaluation method in an embodiment of this application;

[0049] Figure 3 This is a schematic diagram of the process for determining the target vector E in the extrinsic parameter evaluation method shown in the embodiments of this application;

[0050] Figure 4 This is a schematic diagram of the external parameter evaluation device shown in the embodiments of this application;

[0051] Figure 5 This is a schematic diagram of the structure of an electronic device shown in an embodiment of this application. Detailed Implementation

[0052] Embodiments of this application will now be described in more detail with reference to the accompanying drawings. While embodiments of this application are shown in the drawings, it should be understood that this application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to make this application more thorough and complete, and to fully convey the scope of this application to those skilled in the art.

[0053] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0054] It should be understood that although the terms "first," "second," "third," etc., may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0055] In related technologies, it is usually necessary to calibrate the parameters of vision sensors. These parameters include the intrinsic and extrinsic parameters of the vision sensor. Using extrinsic parameters without knowing their calibration quality can lead to poor performance of the vision sensor, especially if the calibration is inadequate. Therefore, a technique for evaluating extrinsic parameters is needed.

[0056] To address the aforementioned issues, this application provides an extrinsic parameter evaluation method that combines Euler angles pitch, roll, and yaw in the extrinsic parameter rotation matrix to evaluate the accuracy of sensor extrinsic parameters.

[0057] The technical solutions of the embodiments of this application are described in detail below with reference to the accompanying drawings.

[0058] Figure 1 This is a schematic diagram illustrating an application scenario of the external parameter evaluation method shown in the embodiments of this application.

[0059] This method can evaluate the extrinsic parameters of cameras installed on vehicles. In the extrinsic parameter evaluation method, the vehicle is first selected and parked in a designated evaluation scene to determine the site coordinate system, i.e., the designated world coordinate system. The selected vehicle is parked under certain conditions, ensuring that the vehicle coordinate system coincides with or undergoes a certain transformation relationship with the designated world coordinate system. Identifiable targets are set in the designated evaluation scene. These targets should ideally cover all areas within the camera's field of view (FOV), and each target surface should maintain a different angle with the camera's optical axis. During evaluation, the camera in the vehicle acquires images of the targets. A recognition algorithm is then used to identify the two-dimensional coordinates of the icons in the target images in the pixel coordinate system. These targets include, but are not limited to, Aruco codes, checkerboard patterns, concentric circles, etc.

[0060] Figure 2 This is a schematic flowchart illustrating the external parameter evaluation method in an embodiment of this application.

[0061] See Figure 2 This application provides an external parameter evaluation method, which may include steps S001 to S005.

[0062] S001: Obtain the true coordinates of the target in the specified world coordinate system.

[0063] Step S001 may include:

[0064] The target is placed at a set location using a measuring tool to obtain the target's true coordinates in the world coordinate system.

[0065] In this embodiment, the measuring tool can be a total station or a laser instrument, etc. The total station or laser instrument is used to measure the coordinates of each target in the site coordinate system as the true coordinates of the target in the specified world coordinate system.

[0066] S002: After obtaining the target's coordinates in the pixel coordinate system, calculate the target's coordinates in the world coordinate system based on the external parameters to be evaluated.

[0067] In this embodiment, the vehicle is parked at a specific location in the evaluation scenario. During the evaluation, the camera in the vehicle captures images of the target. The recognition algorithm identifies the two-dimensional coordinates of the icon in the target image in the pixel coordinate system. These two-dimensional coordinates can be obtained by first transforming the camera's intrinsic parameters to obtain the target's coordinates in the camera coordinate system, and then by transforming the camera's extrinsic parameters to be evaluated to obtain the calculated coordinates of the target in the world coordinate system. Different extrinsic parameters to be evaluated correspond one-to-one with the calculated coordinates.

[0068] It should be noted that the extrinsic parameters of the camera refer to the transformation relationship between the camera's coordinate system and the world coordinate system (which in this application can also be the vehicle coordinate system / site coordinate system). The transformation between the two coordinate systems is a rigid body transformation process, therefore the extrinsic parameters are represented by the rotation matrix R and the translation vector T, where the rotation matrix can be represented by the angles of rotation about the three coordinate axes, namely Euler angles roll, pitch, and yaw, as shown below. Figure 1 As shown, the Euler angles roll, pitch, and yaw represent the angles of rotation around the vehicle's Xb, Yb, and Zb axes, respectively. The translation vector is represented by the distance translated along the coordinate axes.

[0069] Taking extrinsic parameter calibration as an example, the characteristic corner points of the target are generally used. The two-dimensional point pixels and the prior measured coordinates (true coordinates) of the three-dimensional point are identified in the image of the camera and matched one-to-one. The extrinsic parameters (R, T) are solved by the algorithm, where R is the rotation matrix and T is the translation vector.

[0070] For the translation vector T, its relationship with the camera design value t can be established:

[0071]

[0072] In the formula, Q is the coordinate in the camera coordinate system, and P is the coordinate in the world coordinate system.

[0073] As can be seen, in some cases, extrinsic parameters can be represented solely by the rotation matrix R.

[0074] S003: The evaluation error is obtained based on the calculated and actual coordinates of the target.

[0075] In this embodiment, the evaluation error is the difference between the calculated coordinates and the true coordinates of the target, i.e., the evaluation error Error = P - P0, where P is the calculated coordinates and P0 is the true coordinates, and the magnitude of the evaluation error is |P - P0|.

[0076] S004: Determine the allowable error range of the target's coordinates in the world coordinate system based on the allowable deviation of each Euler angle of the external parameters.

[0077] In step S004, there are two ways to determine the allowable range of the solution error: one is to determine that the calculated coordinates are within the error range of the true coordinates, and the other is to determine that the true coordinates are within the error range of the calculated coordinates.

[0078] In this application, the true values ​​of each Euler angle in the external parameter rotation matrix are unknown, making it inconvenient to intuitively represent the Euler angles. The embodiment of this application uses the mathematical relationship between the external parameter and the target's coordinates in the world coordinate system to reflect the deviation of the external parameter Euler angle through the error of the three-dimensional coordinates.

[0079] S005: Based on the determined allowable error range, evaluate the external parameters to be evaluated corresponding to the evaluation error.

[0080] In this embodiment, after determining the corresponding allowable error range in step S004, it can be determined whether each Euler angle of the external parameter to be evaluated meets the accuracy requirements based on the relative relationship between the evaluation error corresponding to the external parameter to be evaluated and the determined allowable error range.

[0081] In an optional embodiment, step S004 may include:

[0082] Using Euler angles as zero as a reference, the angle range of each Euler angle is determined according to the set threshold of each Euler angle, and a three-dimensional sliding window is set according to the allowable deviation of each Euler angle, wherein the allowable upper deviation is less than the upper limit of the threshold.

[0083] A three-dimensional sliding window is used to traverse each Euler angle with a set step size. Each time the three-dimensional sliding window slides, a target error E of the true coordinate is obtained.

[0084] Select the error vector E with the largest magnitude from all the obtained target errors E. max The error vector E with the smallest magnitude min This serves as the allowable error limit for the true coordinates.

[0085] Among them, such as Figure 3 As shown, in step S004, the specific steps to obtain a target error E with true coordinates may include:

[0086] S401: Based on the current position of the 3D sliding window, determine the first set of coordinates of the target in the world coordinate system, and the reference coordinates corresponding to the current median of the 3D sliding window;

[0087] S402: Calculate the error vector of each element in the first coordinate set relative to the reference coordinate, and select the error vector with the same direction as the evaluation error from all the calculated error vectors;

[0088] S403: Select the error vector with the largest magnitude from the selected error vectors as the target error E.

[0089] Further, step S005 may include:

[0090] Comparison of evaluation error and error vector E max Error vector E min The modulus length;

[0091] If the magnitude of the evaluation error is less than or equal to the error vector E min If the modulus length is such that the external parameters to be evaluated meet the accuracy requirements;

[0092] If the magnitude of the evaluation error is greater than the error vector E min The magnitude is less than or equal to the error vector E. max If the modulus is large enough, then the external parameters to be evaluated may meet the accuracy requirements;

[0093] If the magnitude of the evaluation error is greater than the error vector E max If the modulus length is too short, then the external parameters to be evaluated do not meet the accuracy requirements.

[0094] In this embodiment, the true value of Euler angles is actually unknown, making it impossible to directly calculate the error corresponding to the Euler angles. Considering that the error range of the true value of Euler angles is within the range of the design value ± a set threshold, the error of the approximate true value can be obtained by calculating the error corresponding to different Euler angles within this range.

[0095] When evaluating the accuracy of external parameters, it is unreasonable to directly use the Euclidean distance between the true coordinates and the calculated coordinates, as different angles contribute differently to the distance. If the angle changes by the same amount, the pitch angle contributes less to the distance than the roll and yaw angles. Therefore, we first determine the vector difference between the calculated and true coordinates; this vector difference is the evaluation error. The error is a continuous function of the Euler angles. When the angle range is a connected set, the error is also a connected set. If the evaluation error is an element of this error set, then the evaluation error is acceptable. Although the three Euler angles are coupled together, making it difficult to directly determine the monotonicity of the evaluation error function, it is clear that when |pitch| < 3°, the evaluation error monotonically decreases and monotonically increases on roll (or yaw) in [-0.3°, 0] and [0, 0.3°], respectively. Therefore, the extreme value of the evaluation error function must be obtained when roll or yaw = ±0.3°.

[0096] If we define the allowable upper deviation of Euler angles as 0.3° and the allowable lower deviation as -0.3°, then the allowable deviation of Euler angles is ±0.3°, and the threshold value for Euler angles is set to 3°. Through coordinate transformation, the error set can be obtained based on the independent variable range of ±0.3° for Euler angles.

[0097] When determining the range of the error function, calculating every point within the range of the independent variable would be overly cumbersome. Considering the monotonicity of the function, it is sufficient to calculate the error corresponding to the boundary points of the independent variable. In this embodiment, Euler angles are taken as zero as a reference. The angle range of each Euler angle [-3°, 3°] is determined according to the set threshold of each Euler angle. A three-dimensional sliding window is set according to the allowable deviation of each Euler angle. This three-dimensional sliding window can be understood as a cube with a length of 0.6°. The three-dimensional sliding window is used to traverse each Euler angle with a set step size of 0.1°. Each time the three-dimensional sliding window slides, a target error E of a true coordinate is obtained. The Euler angle direction represented in the cube has the same name as the Euler angle currently being traversed. A total of [number] errors can be obtained. There are several target errors E, where Ni = (2 × 3° - 0.6°) / 0.1°, and i = θ. The target error E can be determined based on the current position of the 3D sliding window. This involves determining the first set of coordinates of the target in the world coordinate system and the reference coordinates corresponding to the current midpoint of the 3D sliding window. The error vectors of the elements in the first set of coordinates relative to the reference coordinates are calculated. Among all the calculated error vectors, the error vector with the same direction as the evaluation error is selected. Finally, the error vector with the largest magnitude is chosen as the target error E. The target error E is then selected from all the obtained target errors E. max The error vector E with the smallest magnitude min Compare |P-P0| and |E max |、|E min If |P-P0|≤|E min If |E|, then the external parameter to be evaluated meets the accuracy requirement, that is, the error of the external parameter to be evaluated is within the allowable deviation ±0.3°, which can be understood as each Euler angle of the external parameter to be evaluated being within its respective allowable deviation; if |E| min |<|P-P0|≤|E max If |P-P0|>|E, then the external parameter to be evaluated may meet the accuracy requirements, meaning the error of the external parameter to be evaluated may be within ±0.3° or may not be within ±0.3°, requiring specific analysis; if |P-P0|>|E max If |, then the error of the external parameter to be evaluated is not within ±0.3°.

[0098] It should be noted that when |P-P0|≤|E min When |E| is determined to be qualified, the evaluation of the external parameter to be evaluated can be achieved. min |<|P-P0|≤|E max The specific circumstances analysis is not the core of this application, so this part will not be elaborated further.

[0099] In this embodiment, the range of each Euler angle in the extrinsic rotation matrix and the sliding window are defined. The error corresponding to each sliding window in each sliding step is determined by traversing the sliding window, and then the most reliable error, i.e., |E|, is found within the threshold range. min |

[0100] In another alternative embodiment, step S004 may include:

[0101] Using the Euler angles of the external parameter to be evaluated as a reference, the angle range of the corresponding Euler angles is obtained according to the allowable deviation of each Euler angle;

[0102] Determine the second set of coordinates of the target in the world coordinate system based on the obtained angle range;

[0103] Calculate the error vector of each element in the second coordinate set relative to the calculated coordinates, which serves as the allowable error range for the calculated coordinates.

[0104] Further, step S005 may include:

[0105] Determine whether there is an opposite vector of the evaluation error within the allowable error range of the calculated coordinates. If so, the external parameter to be evaluated meets the accuracy requirements; otherwise, it does not.

[0106] In this embodiment, the allowable range of solution error is achieved by determining that the true coordinates are within the error range of the calculated coordinates. The calculated coordinates correspond to the extrinsic parameters to be evaluated, using the Euler angles of the extrinsic parameters to be evaluated. Based on the baseline, the angle range of the corresponding Euler angles is obtained according to the allowable deviation of ±0.3° for each Euler angle; the second set of coordinates of the target in the world coordinate system is determined according to the obtained angle range, where the second coordinate is set as P. i Calculate the error vector of each element in the second coordinate set relative to the calculated coordinates, which serves as the allowable error range for the calculated coordinates. The allowable error range for the calculated coordinates is P. i The set of -P.

[0107] Wherein, the evaluation error Error = P - P0, and the allowable range of error for calculating the coordinates is P. i -P is a set, if P0-P is P i If the elements in the set of -P are used to prove that the true value of the extrinsic parameter is within the error range of the extrinsic parameter to be evaluated, that is, if the opposite vector of the evaluation error exists within the allowable error range of the calculated coordinates, it can be confirmed that the error of the extrinsic parameter to be evaluated meets the accuracy requirements.

[0108] Similarly, in this embodiment, the accuracy requirement of the extrinsic parameter to be evaluated can also be proven by the fact that the true value of the extrinsic parameter is an element in the second coordinate set.

[0109] This application inversely projects the two-dimensional coordinates of the target in the pixel coordinate system to a three-dimensional space to obtain calculated coordinates, and compares them with the true coordinates to determine the evaluation error. It then uses Euler angles (pitch, roll, and yaw) in the extrinsic rotation matrix to evaluate the accuracy of the sensor's extrinsic parameters. The angular parameters in the rotation matrix are mapped to the inverse projection error for application in evaluating extrinsic parameter accuracy. This evaluation can be performed by determining the allowable error of the true value using a sliding window method when the true value of the extrinsic parameter is unknown, or by using the extrinsic parameter to be evaluated as a benchmark and determining whether the true value of the extrinsic parameter is within the allowable error range of the calculated coordinates through allowable deviation.

[0110] Corresponding to the aforementioned application function implementation method embodiments, this application also provides an external parameter evaluation device, an electronic device, and corresponding embodiments.

[0111] Figure 4 This is a schematic diagram of the external parameter evaluation device shown in the embodiments of this application.

[0112] like Figure 4 As shown in the figure, this application embodiment also provides an external parameter evaluation device, the device comprising:

[0113] The real coordinate acquisition module 001 is configured to acquire the real coordinates of the target in the specified world coordinate system;

[0114] The coordinate calculation module 002 is configured to calculate the target's coordinates in the world coordinate system based on the external parameters to be evaluated after obtaining the target's coordinates in the pixel coordinate system.

[0115] The evaluation error is obtained by module 003, which is configured to obtain the evaluation error based on the calculated coordinates and the actual coordinates of the target;

[0116] The allowable error determination module 004 is configured to determine the allowable error range of the target's coordinates in the world coordinate system based on the allowable deviation of each Euler angle of the extrinsic parameters.

[0117] The external parameter evaluation module 005 is configured to evaluate the external parameters to be evaluated corresponding to the evaluation error based on a determined error allowable range.

[0118] The real coordinate acquisition module 001 can be specifically configured to use a measuring tool to measure the target placed at a set position and obtain the real coordinates of the target in the world coordinate system.

[0119] In one embodiment, the allowable error determination module 004 is configured to use Euler angles as zero as a reference, determine the angle range of each Euler angle according to a set threshold for each Euler angle, and set a three-dimensional sliding window according to the allowable deviation of each Euler angle, wherein the allowable upper deviation is less than the upper limit of the threshold; use the three-dimensional sliding window to traverse each Euler angle at a set step size, wherein each time the three-dimensional sliding window slides once, a target error E of the true coordinate is obtained; and select the error vector E with the largest magnitude from all the obtained target errors E. max The error vector E with the smallest magnitude min The target error E, which serves as the allowable limit for the true coordinates, can be obtained by: determining the first set of coordinates of the target in the world coordinate system based on the current position of the 3D sliding window, and the reference coordinates corresponding to the current value in the 3D sliding window; calculating the error vectors of the elements in the first set of coordinates relative to the reference coordinates, and selecting the error vector with the same direction as the evaluation error from all the calculated error vectors; and selecting the error vector with the largest magnitude from the selected error vectors as the target error E.

[0120] Accordingly, the external parameter evaluation module 005 is specifically configured to compare the evaluation error with the error vector E. max Error vector E min The magnitude of the evaluation error; if the magnitude of the evaluation error is less than or equal to the error vector E. min If the magnitude of the evaluation error is greater than the error vector E, then the extrinsic parameter to be evaluated meets the accuracy requirements; if the magnitude of the evaluation error is greater than the error vector E, then the evaluation error meets the accuracy requirements. min The magnitude is less than or equal to the error vector E. max If the magnitude of the evaluation error is greater than the error vector E, then the extrinsic parameter to be evaluated may meet the accuracy requirements; if the magnitude of the evaluation error is greater than the error vector E, then the evaluation error may meet the accuracy requirements. max If the modulus length is too short, then the external parameters to be evaluated do not meet the accuracy requirements.

[0121] This embodiment maps the angular parameters in the rotation matrix to the inverse projection error for evaluation of extrinsic parameter accuracy. It also allows for evaluation by determining the allowable error of the true value using a sliding window method when the true value of the extrinsic parameter is unknown. This embodiment inversely projects the two-dimensional coordinates of the target in the pixel coordinate system to three-dimensional space to obtain calculated coordinates, compares these coordinates with the true coordinates to determine the evaluation error, and combines the Euler angles pitch, roll, and yaw in the extrinsic parameter rotation matrix to evaluate the accuracy of the sensor's extrinsic parameters.

[0122] In another optional embodiment, the allowable error determination module 004 can also be configured to use the Euler angles of the extrinsic parameter to be evaluated as a reference, obtain the angle range of the corresponding Euler angles according to the allowable deviation of each Euler angle; determine the second coordinate set of the target in the world coordinate system according to the obtained angle range; and calculate the error vector of the elements in the second coordinate set relative to the calculated coordinates as the allowable error range of the calculated coordinates.

[0123] Correspondingly, the external parameter evaluation module 005 can be configured to determine whether there is an opposite vector of the evaluation error within the allowable error range of the calculated coordinates. If so, the external parameter to be evaluated meets the accuracy requirements; otherwise, it does not.

[0124] This embodiment evaluates the extrinsic parameter by determining whether its true value falls within the allowable error range of the calculated coordinates, using the extrinsic parameter as a benchmark and allowing for permissible deviations. Specifically, it inversely projects the target's two-dimensional coordinates in the pixel coordinate system to three-dimensional space to obtain calculated coordinates, compares these coordinates with the true coordinates to determine the evaluation error, and uses Euler angles (pitch, roll, and yaw) in the extrinsic parameter rotation matrix to evaluate the accuracy of the sensor's extrinsic parameters.

[0125] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated further here.

[0126] Figure 5 This is a schematic diagram of the structure of an electronic device shown in an embodiment of this application.

[0127] See Figure 5 The electronic device 500 includes a memory 510 and a processor 520.

[0128] The processor 520 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0129] Memory 510 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage devices. ROM may store static data or instructions required by the processor 520 or other modules of the computer. Permanent storage devices may be read-write storage devices. Permanent storage devices may be non-volatile storage devices that retain stored instructions and data even when the computer is powered off. In some embodiments, permanent storage devices use mass storage devices (e.g., magnetic or optical disks, flash memory) as permanent storage devices. In other embodiments, permanent storage devices may be removable storage devices (e.g., floppy disks, optical drives). System memory may be a read-write storage device or a volatile read-write storage device, such as dynamic random access memory. System memory may store some or all of the instructions and data required by the processor during operation. Furthermore, memory 510 may include any combination of computer-readable storage media, including various types of semiconductor memory chips (e.g., DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and disks and / or optical disks may also be used. In some embodiments, memory 510 may include a removable storage device that is readable and / or writable, such as a laser disc (CD), a read-only digital multifunction optical disc (e.g., DVD-ROM, dual-layer DVD-ROM), a read-only Blu-ray disc, an ultra-high-density optical disc, a flash memory card (e.g., SD card, mini SD card, Micro-SD card, etc.), a magnetic floppy disk, etc. Computer-readable storage media do not contain carrier waves or transient electronic signals transmitted wirelessly or via wired connections.

[0130] The memory 510 stores executable code, which, when processed by the processor 520, can cause the processor 520 to execute part or all of the methods described above.

[0131] Furthermore, the method according to this application can also be implemented as a computer program or computer program product, which includes computer program code instructions for performing some or all of the steps in the method described above.

[0132] Alternatively, this application may be implemented as a computer-readable storage medium (or a non-transitory machine-readable storage medium or a machine-readable storage medium) storing executable code (or computer program or computer instruction code) thereon, which, when executed by a processor of an electronic device (or server, etc.), causes the processor to perform part or all of the steps of the methods described above according to this application.

[0133] The various embodiments of this application have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. An external parameter evaluation method, characterized by, The method includes: Obtain the target's true coordinates in the specified world coordinate system; After obtaining the target's coordinates in the pixel coordinate system, the target's calculated coordinates in the world coordinate system are obtained based on the external parameters to be evaluated. The evaluation error is obtained based on the calculated and actual coordinates of the target. The allowable error range of the target's coordinates in the world coordinate system is determined based on the allowable deviation of each Euler angle of the external parameters; Based on the determined allowable error range, the external parameters to be evaluated corresponding to the evaluation error are evaluated. The permissible error range for determining the target's coordinates in the world coordinate system based on the permissible deviations of each Euler angle of the extrinsic parameters includes: Using Euler angles as zero as a reference, the angle range of each Euler angle is determined according to a set threshold for each Euler angle, and a three-dimensional sliding window is set according to the allowable deviation of each Euler angle, wherein the allowable upper deviation is less than the upper limit of the threshold; the three-dimensional sliding window is used to traverse each Euler angle according to a set step size, wherein each time the three-dimensional sliding window slides once, a target error E of the true coordinate is obtained; from all the obtained target errors E, the error vector Emax with the largest magnitude and the error vector Emin with the smallest magnitude are selected as the allowable error limit of the true coordinate.

2. The extrinsic parameter evaluation method of claim 1, wherein, Obtaining the target's true coordinates in the specified world coordinate system includes: The target, positioned at a set location, is measured using a measuring tool to obtain its true coordinates in the world coordinate system.

3. The extrinsic parameter evaluation method of claim 1, wherein, The process of obtaining a target error E for the true coordinates includes: Based on the current position of the three-dimensional sliding window, determine the first set of coordinates of the target in the world coordinate system, and the reference coordinates corresponding to the median value of the current three-dimensional sliding window; Calculate the error vector of each element in the first coordinate set relative to the reference coordinates, and select the error vector that is in the same direction as the evaluation error from all the calculated error vectors; The error vector with the largest magnitude is selected from the chosen error vectors as the target error E.

4. The extrinsic parameter evaluation method of claim 1, wherein, The evaluation of the external parameters to be evaluated based on the determined allowable error range includes: Compare the magnitudes of the evaluation error with those of the error vectors Emax and Emin; If the magnitude of the evaluation error is less than or equal to the magnitude of the error vector Emin, then the external parameter to be evaluated meets the accuracy requirements. If the magnitude of the evaluation error is greater than the magnitude of the error vector Emin, and less than or equal to the magnitude of the error vector Emax, then the external parameter to be evaluated may meet the accuracy requirements. If the magnitude of the evaluation error is greater than the magnitude of the error vector Emax, then the external parameter to be evaluated does not meet the accuracy requirements.

5. The extrinsic parameter evaluation method of claim 1, wherein, The permissible error range for determining the target's coordinates in the world coordinate system based on the permissible deviations of each Euler angle of the extrinsic parameters includes: Using the Euler angles of the external parameter to be evaluated as a reference, the angle range of the corresponding Euler angles is obtained according to the allowable deviation of each Euler angle; Determine the second set of coordinates of the target in the world coordinate system based on the obtained angle range; Calculate the error vector of each element in the second coordinate set relative to the calculated coordinates, which serves as the allowable error range for the calculated coordinates.

6. The extrinsic parameter evaluation method of claim 5, wherein, The evaluation of the external parameters to be evaluated based on the determined allowable error range includes: Determine whether the opposite vector of the evaluation error exists within the allowable error range of the calculated coordinates. If so, the external parameter to be evaluated meets the accuracy requirement; otherwise, it does not.

7. An external parameter evaluation device, characterized by The device includes: The real coordinate acquisition module is configured to acquire the real coordinates of the target in a specified world coordinate system; The coordinate calculation module is configured to, after obtaining the target's coordinates in the pixel coordinate system, calculate the target's coordinates in the world coordinate system based on the external parameters to be evaluated. An evaluation error acquisition module is configured to obtain the evaluation error based on the calculated and actual coordinates of the target. The allowable error determination module is configured to determine the allowable error range of the target's coordinates in the world coordinate system based on the allowable deviations of each Euler angle of the extrinsic parameters; An external parameter evaluation module is configured to evaluate the external parameters to be evaluated corresponding to the evaluation error based on a determined error allowable range. The step of determining the allowable error range of the target's coordinates in the world coordinate system based on the allowable deviation of each Euler angle of the extrinsic parameters includes: using the Euler angle as zero as a reference, determining the angle range of each Euler angle according to a set threshold for each Euler angle, and setting a three-dimensional sliding window according to the allowable deviation of each Euler angle, wherein the allowable upper deviation is less than the upper limit of the threshold; using the three-dimensional sliding window to traverse each Euler angle according to a set step size, wherein each time the three-dimensional sliding window slides once, a target error E of the true coordinates is obtained; selecting the error vector Emax with the largest magnitude and the error vector Emin with the smallest magnitude from all the obtained target errors E as the allowable error limit of the true coordinates.

8. An electronic device, comprising: include: processor; as well as A memory having executable code stored thereon, which, when executed by the processor, causes the processor to perform the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, It stores executable code that, when executed by a processor of an electronic device, causes the processor to perform the method as described in any one of claims 1 to 6.

Citation Information

Patent Citations

  • External parameter calibration method, device and system of camera and storage medium

    CN113822943A

  • Dynamic calibration method and device, driving state detection method, medium and equipment

    CN114663458A