Adaptive matching vehicle-mounted all-round view camera calibration method and system and storage medium

Abstract

The invention discloses an adaptive matching vehicle-mounted around-view camera calibration method and system and a storage medium, and the method comprises the steps: obtaining coordinate values of corner points in each target in a calibration scene under a vehicle body coordinate system, and recording the coordinate values as vehicle body coordinate true value corner points; extracting image corner points in a sampling image collected by the vehicle-mounted all-around camera, and projecting the image corner points to a camera coordinate system based on parameters of the vehicle-mounted all-around camera to generate image detection corner points; based on the initial camera external parameters, performing coordinate transformation on the vehicle body coordinate true value angular point to a camera coordinate system, and generating a transformation angular point; calculating an association degree matrix based on the image detection angular points and the conversion angular points, and selecting an optimal matching point pair based on the association degree matrix; and calculating an external parameter matrix of the vehicle-mounted around-view camera under the condition of minimizing the re-projection error by using an iterative optimization mode. According to the technical scheme of the invention, high-precision calibration of the vehicle-mounted around-view camera can be realized under the condition that false detection or missing detection exists in the angular points in the image.

Description

Technical Field

[0001] The present application relates to the technical field of autonomous driving, and more specifically, to an adaptively matched on-board surround view camera calibration method, an adaptively matched on-board surround view camera calibration system, and a computer-readable storage medium. Background Art

[0002] The traditional vehicle-mounted surround view calibration method requires placing a regular chessboard around the vehicle body, measuring and calculating the coordinates of each corner point of the chessboard to the center of the vehicle body, and then sorting the coordinates of the chessboard corner points detected on the image to obtain the correspondence between the corner points on the image and the coordinates of the vehicle body corner points, thereby establishing the correspondence between the image and the vehicle body coordinate system in each direction, and then using PNP to solve the relevant external parameters.

[0003] For example, in the Chinese invention patent application "Calibration method for multiple on-board cameras and intelligent car" with publication number CN116128973A, the daily checkerboard calibration plate is replaced with a calibration ring, but it is usually assumed that all image points (corner points) can be detected.

[0004] However, in actual application scenarios, due to the damage or stains on the chessboard of the production line or uncontrollable factors of the offline calibration environment, the corner points on the images captured by the on-board surround-view camera to be calibrated may be missed or misdetected. Ultimately, it is impossible to sort the corner points and obtain the corresponding relationship of the corner points through the regular corner point matching method, which requires multiple calibrations, wastes calibration time, and there is even the possibility of calibration failure due to missed corner points. Summary of the invention

[0005] The purpose of this application is to calibrate a vehicle-mounted surround view camera and improve the accuracy of the calibration when there are false detections or missed detections of corner points in the image.

[0006] The technical solution of the first aspect of the present application is: to provide an adaptive matching vehicle-mounted surround view camera calibration method, the method comprising: obtaining the coordinate value of the corner point of each target in the calibration scene in the vehicle body coordinate system, recorded as the vehicle body coordinate true value corner point P w ; Extract the image corner points in the sampled image collected by the on-board surround view camera, and project the image corner points to the camera coordinate system based on the parameters of the on-board surround view camera to generate image detection corner points P c1 , where the sampled image includes the corner points of each target; based on the initial camera external parameters, the true value corner point P of the vehicle body coordinates w Perform coordinate transformation to the camera coordinate system and generate the conversion corner point P c0 ; Detect corner points P based on the image c1 , conversion corner point P c0, calculate the correlation matrix, and select the best matching point pair based on the correlation matrix; use iterative optimization to calculate the external parameter matrix of the vehicle-mounted surround view camera under the minimum reprojection error, where the minimized reprojection error is determined by the best matching point pair, and the best matching point pair is determined by the image detection corner point P c1 , conversion corner point P c0 Sure.

[0007] In any of the above technical solutions, further, the best matching point pair includes a first best matching point pair, and selecting the best matching point pair based on the association matrix specifically includes: calculating a first cost matrix based on the association matrix; calculating the conversion corner point P based on the first cost matrix using the KM algorithm c0 Detect corner points P from the image c1 The maximum weight perfect match between them is generated to generate the best matching relationship; based on the best matching relationship, the first best matching point pair is selected.

[0008] In any of the above technical solutions, further, an iterative optimization method is used to calculate the external parameter matrix of the vehicle-mounted surround view camera under the minimized reprojection error, specifically including: based on the first best matching point pair, calculating the minimized reprojection error, and performing iterative optimization; judging whether the number of iterative optimization times is greater than or equal to the first optimization times threshold, if so, using the first external parameter matrix calculated by the current iterative optimization as the external parameter matrix after iterative optimization, if not, calculating the error change amplitude between the two minimized reprojection errors in the two iterative optimization processes, when it is determined that the error change amplitude is less than or equal to the amplitude threshold, using the second external parameter matrix calculated by the current iterative optimization as the external parameter matrix after iterative optimization, and when it is determined that the error change amplitude is greater than the amplitude threshold, performing the next iterative optimization.

[0009] In any of the above technical solutions, further, the best matching point pair also includes a second best matching point pair, and the best matching point pair is selected based on the association matrix, which specifically includes: calculating the second cost matrix based on the association matrix; selecting the maximum element value max_a in any row i of the second cost matrix i , determine the maximum element value max_a i The number of columns is max_j; Based on the number of columns max_j, detect the corner point P in the image c1 In the conversion, select the conversion corner point P c0 The corresponding matching candidate points Based on the conversion corner point P c0 , match the candidate points and generate the second best matching point pair.

[0010] In any of the above technical solutions, further, an iterative optimization method is used to calculate the external parameter matrix of the vehicle-mounted surround view camera under the condition of minimizing the reprojection error, which also includes: calculating the minimization of the reprojection error based on the second best matching point pair.

[0011] In any of the above technical solutions, the method further includes: when any matching candidate point corresponds to multiple conversion corner points, obtaining multiple corresponding correlation elements in the correlation matrix; selecting the conversion corner point corresponding to the maximum value among the multiple correlation elements, and generating a second best matching point pair with the matching candidate point.

[0012] In any of the above technical solutions, the method further includes: based on the second best matching point pair, selecting a corresponding cost value in the second cost matrix; when it is determined that the cost value is less than or equal to the cost threshold, deleting the corresponding second best matching point pair; when it is determined that the cost value is greater than the cost threshold, retaining the corresponding second best matching point pair.

[0013] In any of the above technical solutions, further, the calculation formula for minimizing the reprojection error is:

[0014]

[0015] Where curr_err is the minimum reprojection error, They are the image detection corner point and conversion corner point in the i-th best matching point pair, max(N,M) is the total number of best matching point pairs, M is the number of image detection corner points, and N is the number of conversion corner points.

[0016] The technical solution of the second aspect of the present application is: to provide an adaptive matching vehicle-mounted surround view camera calibration system, the system comprising: a first corner point unit, the first corner point unit is configured to obtain the coordinate value of the corner point of each target in the calibration scene in the vehicle body coordinate system, recorded as the vehicle body coordinate true value corner point P w The second corner point unit is configured to extract the image corner points in the sampled image collected by the vehicle-mounted surround view camera, and project the image corner points to the camera coordinate system based on the parameters of the vehicle-mounted surround view camera to generate image detection corner points P c1 , wherein the sampled image includes corner points in each target; a corner point conversion unit, the corner point conversion unit is configured to convert the true value corner point P of the vehicle body coordinate based on the initial camera external parameters w Perform coordinate transformation to the camera coordinate system and generate the conversion corner point P c0 ; A corner point matching unit, the corner point matching unit is configured to detect a corner point P based on the image c1 , conversion corner point P c0, calculate the correlation matrix, and select the best matching point pair based on the correlation matrix; the external parameter iteration unit, the external parameter iteration unit is configured to calculate the external parameter matrix of the vehicle-mounted surround view camera under the minimum reprojection error by an iterative optimization method, wherein the minimized reprojection error is determined by the best matching point pair, and the best matching point pair is determined by the image detection corner point P c1 , conversion corner point P c0 Sure.

[0017] The technical solution of the third aspect of the present application is: a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, an adaptive matching vehicle-mounted surround view camera calibration method as described in any one of the technical solutions of the first aspect of the present application is implemented.

[0018] The beneficial effects of this application are:

[0019] The technical solution of this application first calculates and generates image detection corner points P based on image detection and parameters of the vehicle-mounted surround view camera. c1 , where the image detection corner point P c1 There is no requirement for the accuracy and quantity of the target points. Secondly, based on the measured corner points in the target, the conversion corner points P are calculated and generated using the external parameters of the on-board surround view camera to be calibrated. c0 ; Then, by calculating the correlation matrix, the image detection corner point P is calculated in the image c1 The transformed corner point P obtained by transforming the corner point in the real target c0 One-to-one matching is performed to obtain the best matching point pair; finally, the external parameter matrix of the vehicle-mounted surround view camera that minimizes the reprojection error is calculated through iterative optimization. In this way, the matching problem between the image and the real corner points is solved, and it is not necessary to detect all the corner points in the target in the image. This helps to calibrate the vehicle-mounted surround view camera when there are false detections or missed detections of corner points in the image, and improves the calibration accuracy, with good generalization.

[0020] In the process of corner point matching, in order to quickly obtain a more accurate extrinsic parameter matrix of the vehicle-mounted surround view camera and improve the accuracy of the extrinsic parameter calibration of the vehicle-mounted surround view camera, the present application performs global matching and local matching respectively, and performs iterative optimization respectively, so that the corner point matching process can perform adaptive matching when the image detection corner points are missing or there are false detections, and there is no need to sort all the corner points according to the rule method, thereby improving the calibration success rate of the vehicle-mounted surround view camera and improving the calibration efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The advantages of the above and / or additional aspects of the present application will become apparent and easily understood in the description of the embodiments in conjunction with the following drawings, in which:

[0022] Figure 1 is a schematic flow chart of a method for calibrating a vehicle-mounted surround view camera with adaptive matching according to an embodiment of the present application;

[0023] Figure 2 is a schematic diagram of a calibration scenario according to an embodiment of the present application;

[0024] Figure 3 It is a schematic block diagram of an adaptive matching vehicle-mounted surround view camera calibration system according to an embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to more clearly understand the above-mentioned purposes, features and advantages of the present application, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0026] In the following description, many specific details are elaborated to facilitate a full understanding of the present application. However, the present application may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present application is not limited to the specific embodiments disclosed below.

[0027] Embodiment 1:

[0028] like Figure 1 As shown, this embodiment provides an adaptive matching vehicle-mounted surround view camera calibration method, the method comprising:

[0029] Step 1: Obtain the coordinate values ​​of the corner points of each target in the calibration scene in the vehicle coordinate system, recorded as the true value corner point P of the vehicle coordinate system w ;

[0030] Specifically, Figure 2 As shown, a vehicle body coordinate system is established, with the projection of the center of the vehicle body or the center of the rear axle of the vehicle body on the ground as the origin of the vehicle body coordinate system, wherein the x-axis points to the forward direction, the y-axis points to the left side of the vehicle body, and the z-axis points to the top of the vehicle body.

[0031] Build a calibration scene in the vehicle coordinate system, set up several targets on the ground (such as a chessboard), and measure the coordinate value of each corner point in each target in the vehicle coordinate system, that is, the true value corner point P of the vehicle coordinate system. w .

[0032] It should be noted that the true value of the vehicle body coordinate corner point P can also be obtained by coordinate calculation. w .

[0033] For example, the i-th corner point, its body coordinate true value corner point P w is (X i ,Yi ,Z i ), that is, the coordinates of the i-th corner point in the chessboard relative to the center of the vehicle body, and the obtained true value corner point of the vehicle body coordinate is recorded as Where N is the total number of corner points in the target.

[0034] Step 2: Extract the image corner points from the sampled image collected by the vehicle-mounted surround view camera, and project the image corner points to the camera coordinate system based on the parameters of the vehicle-mounted surround view camera to generate image detection corner points P c1 , where the sampled image includes the corner points in each target;

[0035] Specifically, after the vehicle-mounted surround view camera acquires multiple sample images, a corner point extraction algorithm such as a growth method can be used to extract corner points from the acquired multiple sample images to obtain corner points in the sample images, namely, image corner points.

[0036] Afterwards, in order to avoid repeated calculations, based on the parameters of the onboard surround view camera, such as camera intrinsic parameters and distortion coefficients, the image corner points are projected into the camera coordinate system to generate image detection corner points, which are recorded as Where M is the number of corner points detected in the image. Here, Z cj =1, that is, the normalized coordinate value is obtained.

[0037] It should be noted that the number M of corner points detected in an image is the number of corner points extracted by a corner point extraction algorithm based on a sampled image, and is not necessarily the same as the total number N of corner points in the target.

[0038] Step 3: Based on the initial camera external parameters, calculate the true value of the corner point P of the vehicle body coordinates w Perform coordinate transformation to the camera coordinate system and generate the conversion corner point P c0 ;

[0039] Specifically, the external parameters of the vehicle-mounted surround view camera are initialized, and the external parameter matrices of each camera to the vehicle body coordinate system, such as the rotation matrix R and the translation matrix t, are estimated according to the angles between the camera and the vehicle body in each direction during design or the angle is directly estimated to be 90°.

[0040] Using the initialized camera external parameters, the vehicle coordinate true value corner point P w Perform coordinate transformation and projection transformation to the camera coordinate system. The resulting point set is denoted as P c ′ 0={(X c ′ i ,Y c ′ i ,Z c ′i )|0≤i≤N}, where

[0041]

[0042] Then normalize it to get the conversion corner point, recorded as:

[0043]

[0044] Step 4: Detect corner points P based on the image c1 (Candidate point), conversion corner point P c0 (True value point), calculate the correlation matrix, and select the best matching point pair based on the correlation matrix;

[0045] Specifically, this embodiment uses the cosine value to calculate the correlation between two elements and calculates the correlation matrix A. The corresponding calculation formula is:

[0046]

[0047] Among them, a ij represents the i-th conversion corner point P c0 Detect corner point P with the jth image c1 The strength of the correlation between ij The larger the value of , the stronger the correlation.

[0048] Therefore, based on this correlation, the best matching point pair can be determined by using the KM algorithm and taking the correlation matrix as the weight matrix (or cost matrix) required by the KM algorithm.

[0049] Step 5: Use iterative optimization to calculate the external parameter matrix of the on-board surround view camera under the condition of minimizing the reprojection error, where the minimization of the reprojection error is determined by the best matching point pair, which is determined by the image detection corner point P c1 , conversion corner point P c0 Sure.

[0050] Specifically, considering the conversion corner point P c0 It is based on the initial camera external parameters and the true value corner point P of the vehicle body coordinates. w Obtained through calculation, and the camera external parameters are the parameters to be calibrated; image detection corner point P c1 It is obtained from the image based on the parameters of the vehicle-mounted surround view camera (such as camera intrinsic parameters and distortion coefficients).

[0051] Therefore, the external parameter matrix of the vehicle-mounted surround view camera can be iteratively optimized by iterative optimization so that the conversion corner point P c0 Can detect corner points P with the image c1 Matched.

[0052] This embodiment uses the method of minimizing the reprojection error for iterative optimization. According to the best matching point pair (matching relationship) obtained previously, the PnP algorithm is used to calculate the camera extrinsic parameters (rotation matrix R, translation matrix t) corresponding to the minimized reprojection error under the current number of iterations. The calculation formula for minimizing the reprojection error can be:

[0053]

[0054] Where curr_err is the minimum reprojection error, are the image detection corner point and conversion corner point in the ath best matching point pair, respectively. max(N,M) is the total number of best matching point pairs, that is, the maximum value of N and M. M is the number of image detection corner points, and N is the number of conversion corner points.

[0055] It should be noted that since the camera external parameters have been updated, the corner points need to be converted Recalculate.

[0056] In any of the above embodiments, further, this embodiment also proposes a method for global matching of corner points, in which a relatively accurate external parameter matrix of the vehicle-mounted surround view camera can be quickly obtained. Therefore, the best matching point pair includes a first best matching point pair, and based on the correlation matrix, the best matching point pair is selected, specifically including:

[0057] Based on the association matrix, calculating a first cost matrix;

[0058] Based on the first cost matrix, the KM algorithm is used to calculate the conversion corner point P c0 Detect corner points P from the image c1 The maximum weight perfect match between them generates the best matching relationship;

[0059] Based on the best matching relationship, the first best matching point pair is selected.

[0060] Accordingly, the extrinsic parameter matrix of the vehicle-mounted surround view camera is calculated by iterative optimization to minimize the reprojection error, including:

[0061] Based on the first best matching point pair, the minimized reprojection error is calculated and iterative optimization is performed; wherein the calculation formula for minimizing the reprojection error is shown in the above formula.

[0062] Determine whether the number of iterative optimizations is greater than or equal to the first optimization number threshold. If so, it is considered that the iteration termination condition is met, and the first external parameter matrix calculated by the current iterative optimization is used as the external parameter matrix after iterative optimization. If not, calculate the error change amplitude between the two minimized reprojection errors in the two iterative optimization processes, and use this error change amplitude to determine whether the camera external parameters tend to be stable.

[0063] When the error change amplitude is determined to be less than or equal to the amplitude threshold, the iteration termination condition is considered to be met, and the second external parameter matrix calculated by the current iterative optimization is used as the external parameter matrix after iterative optimization. When the error change amplitude is determined to be greater than the amplitude threshold, the next iterative optimization is performed.

[0064] In any of the above embodiments, in order to improve the accuracy of the extrinsic calibration of the vehicle-mounted surround view camera, a local matching method is also proposed to calculate a more accurate local corner point matching relationship and gradually iterate and optimize the extrinsic parameters. Therefore, the best matching point pair also includes a second best matching point pair, and the best matching point pair is selected based on the correlation matrix, which specifically includes:

[0065] Based on the association matrix, calculating a second cost matrix;

[0066] Select the maximum element value max_a in any row i of the second cost matrix i , determine the maximum element value max_a i The number of columns max_j;

[0067] Based on the number of columns max_j, detect the corner point P in the image c1 In the conversion, select the conversion corner point P c0 The corresponding matching candidate points

[0068] Based on the conversion corner point P c0 , match the candidate points and generate the second best matching point pair.

[0069] Specifically, this embodiment uses the cosine value to calculate the correlation between two elements, calculates the correlation matrix, and selects each conversion corner point P based on the maximum correlation. c0 The corresponding image detection corner point P c1 , determine the matching relationship, that is, the i-th conversion corner point The corresponding matching candidate points (image detection corner points) The calculation formula is:

[0070]

[0071] Among them, 0≤i≤N,0≤j≤M.

[0072] In other words, select the maximum value max_a of each row i (representing the conversion corner point) i , find its corresponding column max_j (representing the image detection corner point), so as to determine the matching relationship from the i-th conversion corner point to the max_j-th matching candidate point, and form the second best matching point pair for local matching.

[0073] It should be noted that local matching can be performed after global matching or separately.

[0074] Accordingly, the extrinsic parameter matrix of the on-board surround view camera is calculated by iterative optimization to minimize the reprojection error, which also includes:

[0075] Based on the second best matching point pair, the minimized reprojection error is calculated, and then the calculated minimized reprojection error is used for iterative optimization. The iterative optimization method and termination condition can be the same as above, or can be set according to actual conditions.

[0076] In any of the above embodiments, further, it is necessary to screen the conversion corner points that are repeatedly matched, and the method further includes:

[0077] When any matching candidate point corresponds to multiple conversion corner points, multiple corresponding correlation elements in the correlation matrix are obtained to obtain the correlation of each best matching point pair;

[0078] The conversion corner point corresponding to the maximum value among multiple association degree elements is selected to generate the second best matching point pair with the matching candidate point, and the association (matching relationship) between the remaining conversion corner points and the matching candidate point is deleted to filter out the conversion corner points with repeated matching.

[0079] In any of the above embodiments, further, in the process of local matching, the best matching point pairs with weaker matching relationships may be deleted, and the method further includes:

[0080] Based on the second best matching point pair, selecting a corresponding cost value in the second cost matrix;

[0081] When it is determined that the cost value is less than or equal to the cost threshold, the corresponding second best matching point pair is deleted; wherein the cost threshold can be set manually or according to the maximum value of the cost value in the second best matching point pair;

[0082] When it is determined that the cost value is greater than the cost threshold, the corresponding second best matching point pair is retained.

[0083] Embodiment 2:

[0084] like Figure 3 As shown, this embodiment provides an adaptive matching vehicle-mounted surround view camera calibration system 100, and the system 100 includes:

[0085] The first corner point unit 101 is configured to obtain the coordinate value of the corner point of each target in the calibration scene in the vehicle body coordinate system, which is recorded as the true value corner point P of the vehicle body coordinate system. w ;

[0086] The second corner point unit 102 is configured to extract the image corner points in the sampled image collected by the vehicle-mounted surround view camera, and project the image corner points to the camera coordinate system based on the parameters of the vehicle-mounted surround view camera to generate image detection corner points P c1 , where the sampled image includes the corner points in each target;

[0087] The corner point conversion unit 103 is configured to convert the true value corner point P of the vehicle body coordinate based on the initial camera external parameters. w Perform coordinate transformation to the camera coordinate system and generate the conversion corner point P c0 ;

[0088] Corner point matching unit 104, the corner point matching unit 104 is configured to detect corner points P based on the image c1 , conversion corner point P c0 , calculate the association matrix, and select the best matching point pair based on the association matrix;

[0089] The external parameter iteration unit 105 is configured to calculate the external parameter matrix of the vehicle-mounted surround view camera under the condition of minimizing the reprojection error by using an iterative optimization method, wherein the minimizing the reprojection error is determined by the best matching point pair, and the best matching point pair is determined by the image detection corner point P c1 , conversion corner point P c0 Sure.

[0090] In any of the above embodiments, further, this embodiment also proposes a method for global matching of corner points, in which a relatively accurate external parameter matrix of the vehicle-mounted surround view camera can be quickly obtained. Therefore, the best matching point pair includes a first best matching point pair, and the corner point matching unit 104 is further configured as follows:

[0091] Based on the association matrix, calculating a first cost matrix;

[0092] Based on the first cost matrix, the KM algorithm is used to calculate the conversion corner point P c0 Detect corner points P from the image c1 The maximum weight perfect match between them generates the best matching relationship;

[0093] Based on the best matching relationship, the first best matching point pair is selected.

[0094] Accordingly, the external parameter iteration unit 105 is further configured as follows:

[0095] Based on the first best matching point pair, the minimized reprojection error is calculated and iterative optimization is performed; wherein the calculation formula for minimizing the reprojection error is shown in the above formula.

[0096] Determine whether the number of iterative optimizations is greater than or equal to the first optimization number threshold. If so, it is considered that the iteration termination condition is met, and the first external parameter matrix calculated by the current iterative optimization is used as the external parameter matrix after iterative optimization. If not, calculate the error change amplitude between the two minimized reprojection errors in the two iterative optimization processes, and use this error change amplitude to determine whether the camera external parameters tend to be stable.

[0097] When the error change amplitude is determined to be less than or equal to the amplitude threshold, the iteration termination condition is considered to be met, and the second external parameter matrix calculated by the current iterative optimization is used as the external parameter matrix after iterative optimization. When the error change amplitude is determined to be greater than the amplitude threshold, the next iterative optimization is performed.

[0098] In any of the above embodiments, in order to improve the accuracy of the extrinsic calibration of the vehicle-mounted surround view camera, a local matching method is also proposed to calculate a more accurate local corner point matching relationship and gradually iterate and optimize the extrinsic parameters. Therefore, the best matching point pair also includes a second best matching point pair, and the corner point matching unit 104 is further configured as follows:

[0099] Based on the association matrix, calculating a second cost matrix;

[0100] Select the maximum element value max_a in any row i of the second cost matrix i , determine the maximum element value max_a i The number of columns max_j;

[0101] Based on the number of columns max_j, detect the corner point P in the image c1 In the conversion, select the conversion corner point P c0 The corresponding matching candidate points

[0102] Based on the conversion corner point P c0 , match the candidate points and generate the second best matching point pair.

[0103] Correspondingly, the external parameter iteration unit 105 is further configured as follows:

[0104] Based on the second best matching point pair, the minimized reprojection error is calculated, and then the calculated minimized reprojection error is used for iterative optimization. The iterative optimization method and termination condition can be the same as above, or can be set according to actual conditions.

[0105] In any of the above embodiments, further, it is necessary to screen the conversion corner points that are repeatedly matched, and the corner point conversion unit 103 is further configured to:

[0106] When any matching candidate point corresponds to multiple conversion corner points, multiple corresponding correlation elements in the correlation matrix are obtained to obtain the correlation of each best matching point pair;

[0107] The conversion corner point corresponding to the maximum value among multiple association degree elements is selected to generate the second best matching point pair with the matching candidate point, and the association (matching relationship) between the remaining conversion corner points and the matching candidate point is deleted to filter out the conversion corner points with repeated matching.

[0108] In any of the above embodiments, further, in the process of local matching, the best matching point pairs with weaker matching relationships may be deleted, and the corner point matching unit 104 is further configured as follows:

[0109] Based on the second best matching point pair, selecting a corresponding cost value in the second cost matrix;

[0110] When it is determined that the cost value is less than or equal to the cost threshold, the corresponding second best matching point pair is deleted; wherein the cost threshold can be set manually or according to the maximum value of the cost value in the second best matching point pair;

[0111] When it is determined that the cost value is greater than the cost threshold, the corresponding second best matching point pair is retained.

[0112] Embodiment 3:

[0113] This embodiment further provides a computer-readable storage medium, in which a computer program is stored. When the stored computer program is executed by a processor, the adaptive matching vehicle-mounted surround view camera calibration method of any one of the above-mentioned embodiments 1 is implemented.

[0114] Specifically, the computer program includes computer instructions, which are stored in a computer-readable storage medium; the processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method steps in any one of the above-mentioned embodiments 1.

[0115] The above-mentioned computer-readable storage medium may include random access memory (RAM), read-only memory (ROM), compact disc read-only memory (CD-ROM) or other optical disc storage, magnetic disk storage or other magnetic storage devices, etc.

[0116] The processor may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the present embodiment.

[0117] So far, various embodiments of the present application have been described in detail. In order to avoid obscuring the concept of the present application, some details known in the art are not described. Based on the above description, those skilled in the art can fully understand how to implement the technical solution disclosed herein.

[0118] Although some specific embodiments of the present application have been described in detail through examples, those skilled in the art should understand that the above examples are only for illustration rather than for limiting the scope of the present application.

[0119] The steps in this application can be adjusted in order, combined, and deleted according to actual needs.

[0120] Although the present application is disclosed in detail with reference to the accompanying drawings, it should be understood that these descriptions are merely exemplary and are not intended to limit the application of the present application. The scope of protection of the present application is defined by the appended claims and may include various modifications, alterations and equivalents made to the invention without departing from the scope and spirit of the present application.

Claims

1. An adaptive matching vehicle-mounted surround view camera calibration method, characterized in that: The method comprises: Get the coordinates of the corner points of each target in the calibration scene in the vehicle coordinate system, and record them as the true value corner point P of the vehicle coordinate system. w ; Extract the image corner points in the sampled image collected by the vehicle-mounted surround view camera, and project the image corner points to the camera coordinate system based on the parameters of the vehicle-mounted surround view camera to generate image detection corner points P c1 , Wherein, the sampled image includes corner points in each of the targets; Based on the initial camera external parameters, the vehicle body coordinate true value corner point P w Perform coordinate transformation to the camera coordinate system and generate the conversion corner point P c0 ; Detect corner points P based on the image c1 , the conversion corner point P c0 , calculate the association matrix, and select the best matching point pair based on the association matrix; By using an iterative optimization method, the extrinsic parameter matrix of the vehicle-mounted surround view camera is calculated under the condition of minimizing the reprojection error. The minimized reprojection error is determined by the best matching point pair, which is determined by the image detection corner point P c1 , the conversion corner point P c0 Sure.

2. The method for calibrating a vehicle-mounted surround view camera with adaptive matching according to claim 1, characterized in that: The best matching point pair includes a first best matching point pair, and selecting the best matching point pair based on the association matrix specifically includes: Based on the association matrix, calculating a first cost matrix; Based on the first cost matrix, the KM algorithm is used to calculate the conversion corner point P c0 Detect corner points P from the image c1 The maximum weight perfect match between them generates the best matching relationship; Based on the best matching relationship, the first best matching point pair is selected.

3. The method for calibrating a vehicle-mounted surround view camera with adaptive matching according to claim 2, characterized in that: The method of calculating the external parameter matrix of the vehicle-mounted surround view camera under the condition of minimizing the reprojection error by using an iterative optimization method specifically includes: Based on the first best matching point pair, calculating the minimized reprojection error, and performing iterative optimization; Determine whether the number of iterative optimizations is greater than or equal to a first optimization number threshold, and if so, use the first external parameter matrix calculated by the current iterative optimization as the external parameter matrix after iterative optimization. If not, calculate the error change between the two minimized reprojection errors during the two iterative optimization processes. When it is determined that the error change amplitude is less than or equal to the amplitude threshold, the second external parameter matrix calculated by the current iterative optimization is used as the external parameter matrix after iterative optimization, When it is determined that the error change amplitude is greater than the amplitude threshold, the next iterative optimization is performed.

4. The adaptive matching vehicle-mounted surround view camera calibration method according to claim 1 or 2, characterized in that: The best matching point pair also includes a second best matching point pair, and the selecting the best matching point pair based on the association matrix specifically includes: Based on the association matrix, calculating a second cost matrix; Select the maximum element value max_a in any row i of the second cost matrix i , determine the maximum element value max_a i The number of columns max_j; Based on the number of columns max_j, a corner point P is detected in the image. c1 In the example, select the conversion corner point P c0 The corresponding matching candidate points Based on the conversion corner point P c0 , the matching candidate points, and generating the second best matching point pair.

5. The method for calibrating a vehicle-mounted surround view camera with adaptive matching according to claim 4, characterized in that: The method of calculating the external parameter matrix of the vehicle-mounted surround view camera under the condition of minimizing the reprojection error by iterative optimization also includes: Based on the second best matching point pair, the minimized reprojection error is calculated.

6. The method for calibrating a vehicle-mounted surround view camera with adaptive matching according to claim 4, characterized in that: The method further comprises: When any of the matching candidate points corresponds to multiple conversion corner points, obtaining multiple corresponding association degree elements in the association degree matrix; A conversion corner point corresponding to a maximum value among the plurality of the association degree elements is selected, and the second best matching point pair is generated together with the matching candidate point.

7. The method for calibrating a vehicle-mounted surround view camera with adaptive matching according to claim 4, characterized in that: The method further comprises: Based on the second best matching point pair, selecting a corresponding cost value in the second cost matrix; When it is determined that the cost value is less than or equal to the cost threshold, deleting the corresponding second best matching point pair; When it is determined that the cost value is greater than the cost threshold, the corresponding second best matching point pair is retained.

8. The method for calibrating a vehicle-mounted surround view camera with adaptive matching according to claim 1, characterized in that: The calculation formula for minimizing the reprojection error is: Where curr_err is the minimized reprojection error, They are the image detection corner point and conversion corner point in the i-th best matching point pair, max(N,M) is the total number of best matching point pairs, M is the number of image detection corner points, and N is the number of conversion corner points.

9. The adaptive matching vehicle-mounted surround view camera calibration system is characterized by: The system comprises: The first corner point unit is configured to obtain the coordinate value of the corner point of each target in the calibration scene in the vehicle body coordinate system, which is recorded as the true value corner point P of the vehicle body coordinate system. w ; The second corner point unit is configured to extract the image corner points in the sampled image collected by the vehicle-mounted surround view camera, and project the image corner points to the camera coordinate system based on the parameters of the vehicle-mounted surround view camera to generate image detection corner points P c1 , Wherein, the sampled image includes corner points in each of the targets; A corner point conversion unit is configured to convert the true value corner point P of the vehicle body coordinate based on the initial camera external parameters. w Perform coordinate transformation to the camera coordinate system and generate the conversion corner point P c0 ; A corner point matching unit, the corner point matching unit is configured to detect a corner point P based on the image c1 , the conversion corner point P c0 , calculate the association matrix, and select the best matching point pair based on the association matrix; An external parameter iteration unit is configured to calculate the external parameter matrix of the vehicle-mounted surround view camera under the condition of minimizing the reprojection error by using an iterative optimization method, The minimized reprojection error is determined by the best matching point pair, which is determined by the image detection corner point P c1 , the conversion corner point P c0 Sure.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images