An automatic calibration method and device for vehicle visual detection and a medium

Abstract

This invention belongs to the field of vehicle visual inspection technology, and provides an automatic calibration method, device, and medium for vehicle visual inspection. The method includes: statically calibrating a vehicle camera to obtain a projection matrix; automatically detecting long-range and short-range information of ground targets during vehicle operation; calculating short-range world coordinates based on the projection matrix and short-range information; collecting vehicle posture data between long-range and short-range information; calculating long-range world coordinates based on the short-range world coordinates and vehicle posture data; forming target data pairs based on long-range world coordinates and long-range information; collecting multiple sets of target data pairs, selecting target data pairs for least squares iteration, and calculating second intrinsic parameter coefficients and second extrinsic parameter coefficients. This application aims to solve the problems mentioned in the background art, such as the high manufacturing difficulty, operational difficulty, high cost, and maintenance difficulty of setting coding points and marker points within the field of view.

Description

Technical Field

[0001] This invention belongs to the field of vehicle visual inspection technology, and specifically relates to an automatic calibration method, device and medium for vehicle visual inspection. Background Technology

[0002] Vehicle vision recognition reconstructs the three-dimensional world by projecting information from the three-dimensional world onto a two-dimensional plane. This projection relationship is usually explained using the pinhole imaging model. However, in real-world scenarios, due to lens distortion and assembly issues, the simple pinhole imaging model cannot meet the requirements, and camera calibration is necessary.

[0003] In visual calibration, Zhang Zhengyou's planar calibration method is commonly used. This method uses a calibration board composed of two-dimensional squares for calibration, collects images of the calibration board in different poses, extracts the pixel coordinates of corner points in the images, calculates the initial values ​​of the camera's intrinsic and extrinsic parameters through the homography matrix, estimates the distortion coefficients using the nonlinear least squares method, and finally optimizes the parameters using the maximum likelihood estimation method.

[0004] However, when dealing with long-range targets with a wide field of view, in order to ensure the accuracy of the calibration, as in patent CN202210386190.X, it is necessary to construct a labeled scene and establish a mapping relationship between coded points and marker points within the field of view for calibration.

[0005] However, setting coding points and markers within the field of view is difficult to manufacture, operate, use, and maintain, making it hard to apply in practical engineering. Summary of the Invention

[0006] The purpose of this invention is to provide an automatic calibration method, device, and medium for vehicle visual inspection, in order to solve the problems mentioned in the background art, such as the difficulty in manufacturing, operation, use, and maintenance of setting coding points and marker points within the field of view.

[0007] To achieve the above-mentioned technical objectives, the technical solution adopted by the present invention is as follows:

[0008] In a first aspect, this application provides an automatic calibration method for vehicle visual detection, the method comprising:

[0009] Static calibration of the vehicle camera is performed, the first intrinsic parameter matrix and the first extrinsic parameter matrix are recorded, and the projection matrix is ​​obtained.

[0010] During vehicle operation, it automatically detects both long-range and short-range information of ground targets.

[0011] The near-range world coordinates are calculated based on the projection matrix and the near-range information.

[0012] Collect vehicle posture data between the long-distance information and the short-distance information;

[0013] Based on the near-range world coordinates and the vehicle posture data, the far-range world coordinates are calculated and obtained;

[0014] The target data pair is formed based on the long-distance world coordinates and the long-distance information;

[0015] Collect multiple sets of target data pairs, select the target data pairs for least squares iteration, and calculate the second intrinsic parameter coefficient and the second extrinsic parameter coefficient.

[0016] Select the remaining data pairs and use the reprojection method to verify the second intrinsic parameter coefficients and the second extrinsic parameter coefficients.

[0017] In conjunction with the first aspect, the static calibration of the vehicle camera, recording the first intrinsic parameter matrix and the first extrinsic parameter matrix, and obtaining the projection matrix includes:

[0018] Static calibration of the vehicle camera is performed using Zhang Zhengyou's calibration method. Based on a pinhole camera model, the projection matrix expression between the pixel coordinate system (u,v) and the world coordinate system (x,y,z) is obtained.

[0019]

[0020] Where Zc represents the magnification factor. This represents the first intrinsic parameter matrix. Let R and T represent the rotation and translation vectors of the camera coordinate system relative to the world coordinate system, respectively. In the first intrinsic parameter matrix: fx = F / dx and fy = F / dy, where F is the focal length, dx and dy represent the physical length of a pixel on the calibration board in the x and y directions, respectively, and u0 and v0 represent the difference in horizontal and vertical pixels between the center pixel coordinates and the origin pixel coordinates of the image.

[0021] In conjunction with the first aspect, the sequential automatic detection of long-range and short-range information of ground targets includes:

[0022] During vehicle operation, when the vehicle camera detects the ground target for the first time, it records the information of the ground target at this time and stores it as long-distance information. The long-distance information includes the pixel coordinate information (u1, v1) of the first image.

[0023] As the vehicle moves, when the distance between the vehicle and the ground target is less than a preset distance threshold, the information of the ground target at this time is recorded and stored as close-range information. The close-range information includes the second image pixel coordinate information (u2, v2).

[0024] In conjunction with the first aspect, the calculation of near-range world coordinates based on the projection matrix and near-range information includes:

[0025] The pixel coordinates of the second image are input into the projection matrix to calculate the near-range world coordinates (x2, y2, z2).

[0026] In conjunction with the first aspect, the step of calculating and obtaining the distant world coordinates based on the near-range world coordinates and the vehicle posture data includes:

[0027] The transformation equation T and the vehicle motion equation M are calculated from the vehicle posture data.

[0028] Model building By substituting the transformation equation T, the vehicle motion equation M, and the near-distance world coordinates (x2, y2, z2) into the model, the far-distance world coordinates (x1, y1, z1) are calculated.

[0029] In conjunction with the first aspect, the step of selecting the target data pair for least squares iteration includes:

[0030] Multiple sets of target data pairs are randomly selected for least squares iteration.

[0031] In conjunction with the first aspect, after selecting the remaining data pairs and verifying the second intrinsic parameter coefficients and the second extrinsic parameter coefficients using the reprojection method, the method further includes:

[0032] The verification error is obtained, and the magnitude of the verification error and the preset error threshold are determined. If the verification error is greater than the preset error threshold, the target data pair is determined to be erroneous, the target data pair is discarded, the target data pair is re-acquired, and the second internal parameter coefficient and the second external parameter coefficient are calculated.

[0033] Secondly, this application provides an automatic calibration device for vehicle visual inspection, comprising:

[0034] The static calibration module is used to perform static calibration on the vehicle camera, record the first intrinsic parameter matrix and the first extrinsic parameter matrix, and obtain the projection matrix.

[0035] The automatic detection module is used to automatically detect long-range and short-range information of ground targets while the vehicle is in motion;

[0036] The first calculation module is used to calculate the near-range world coordinates based on the projection matrix and near-range information;

[0037] The acquisition module is used to acquire vehicle posture data between the long-distance information and the short-distance information;

[0038] The second calculation module is used to calculate and obtain the far-distance world coordinates based on the near-distance world coordinates and the vehicle posture data;

[0039] The third calculation module is used to select the target data pair and perform least squares iteration to calculate the second intrinsic parameter coefficient and the second extrinsic parameter coefficient.

[0040] The verification module is used to select the remaining data pairs and verify the second intrinsic parameter coefficients and the second extrinsic parameter coefficients using the reprojection method.

[0041] Thirdly, this application provides a computer-readable storage medium including a program or instructions that, when the program or instructions are run on a computer, execute the method described in the first aspect.

[0042] The invention employing the above technical solution has the following advantages:

[0043] This invention collects vehicle posture data between long-range and short-range information, obtains the transformation equation T and vehicle motion equation M from the vehicle posture data, and establishes a model. The transformation equation T, the vehicle motion equation M, and the short-range world coordinates (x2, y2, z2) are then input into the model to calculate the long-range world coordinates (x1, y1, z1). Multiple target data pairs are collected, and multiple sets of target data pairs are randomly selected for least-squares iteration to calculate the second intrinsic parameter coefficients and the second extrinsic parameter coefficients. This achieves recalibration of the camera's intrinsic parameters, ensuring the camera's calibration accuracy for long-range, wide-field-of-view targets, and is simple and easy to maintain without environmental or scene dependence. Attached Figure Description

[0044] The present invention can be further illustrated by the non-limiting embodiments given in the accompanying drawings;

[0045] Figure 1 This is a flowchart illustrating an automatic calibration method for vehicle visual detection according to an embodiment of the present invention.

[0046] Figure 2 This is a scene example diagram of an automatic calibration method for vehicle visual detection according to an embodiment of the present invention;

[0047] Figure 3 This is a system block diagram of an automatic calibration device for vehicle visual detection according to an embodiment of the present invention;

[0048] The system includes a static calibration module 1, an automatic detection module 2, a first calculation module 3, an acquisition module 4, a second calculation module 5, a third calculation module 6, and a verification module 7. Detailed Implementation

[0049] The present application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that similar or identical parts are referred to by the same reference numerals in the drawings or description. Implementations not shown or described in the drawings are forms known to those skilled in the art. In the description of this application, terms such as "first" and "second" are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0050] This application provides an automatic calibration method for vehicle visual detection, such as... Figure 1-2 As shown, the method includes:

[0051] S1: Perform static calibration on the vehicle camera, record the first intrinsic parameter matrix and the first extrinsic parameter matrix, and obtain the projection matrix;

[0052] S2: Automatically detects long-range and short-range information of ground targets while the vehicle is in motion;

[0053] S3: Calculate the near-range world coordinates based on the projection matrix and the near-range information;

[0054] S4: Collect vehicle posture data between the long-distance information and the short-distance information;

[0055] S5: Calculate and obtain the far-distance world coordinates based on the near-distance world coordinates and the vehicle posture data;

[0056] S6: Form a target data pair based on the long-distance world coordinates and the long-distance information;

[0057] S7: Collect multiple sets of target data pairs, select the target data pairs for least squares iteration, and calculate the second intrinsic parameter coefficient and the second extrinsic parameter coefficient;

[0058] S8: Select the remaining data pairs and use the reprojection method to verify the second intrinsic parameter coefficients and the second extrinsic parameter coefficients.

[0059] In some optional embodiments, the step of statically calibrating the vehicle camera, recording the first intrinsic parameter matrix and the first extrinsic parameter matrix, and obtaining the projection matrix includes:

[0060] Static calibration of the vehicle camera is performed using Zhang Zhengyou's calibration method. Based on a pinhole camera model, the projection matrix expression between the pixel coordinate system (u,v) and the world coordinate system (x,y,z) is obtained.

[0061]

[0062] Where Zc represents the magnification factor. This represents the first intrinsic parameter matrix. Let R and T represent the rotation and translation vectors of the camera coordinate system relative to the world coordinate system, respectively. In the first intrinsic parameter matrix: fx = F / dx, fy = F / dy, where F is the focal length, dx and dy represent the physical length of a pixel on the calibration board in the x and y directions, respectively, and u0 and v0 represent the difference in horizontal and vertical pixels between the center pixel coordinates and the origin pixel coordinates. It should be noted that the camera calibration process involves solving a system of linear equations for the camera's intrinsic and extrinsic parameters. More than four corresponding coordinate data pairs are collected, and the system of equations contains five intrinsic variables and six extrinsic variables. Finally, maximum likelihood estimation is used to optimize and obtain the final result.

[0063] In some optional embodiments, the sequential automatic detection of long-range and short-range information of ground targets includes:

[0064] During vehicle operation, when the vehicle camera detects the ground target for the first time, it records the information of the ground target at this time and stores it as long-distance information. The long-distance information includes the pixel coordinate information (u1, v1) of the first image.

[0065] As the vehicle moves, when the distance between the vehicle and the ground target is less than a preset distance threshold, the information of the ground target at this time is recorded and stored as near-field information. The near-field information includes the pixel coordinate information (u2, v2) of the second image. It should be noted that, in this embodiment, the preset distance threshold is the distance at which the first intrinsic parameter matrix and the first extrinsic parameter matrix can guarantee calibration accuracy.

[0066] In some optional embodiments, calculating the near-range world coordinates based on the projection matrix and near-range information includes:

[0067] The pixel coordinates of the second image are substituted into the projection matrix to calculate the near-range world coordinates (x2, y2, z2). It should be noted that since the calibration accuracy of the first intrinsic parameter matrix and the first extrinsic parameter matrix can be guaranteed at close range, the near-range world coordinates (x2, y2, z2) can be calculated by substituted into the projection matrix.

[0068] In some optional embodiments, calculating and obtaining the distant world coordinates based on the near-range world coordinates and the vehicle posture data includes:

[0069] The transformation equation T and the vehicle motion equation M are calculated from the vehicle posture data.

[0070] Model building By substituting the transformation equation T, the vehicle motion equation M, and the near-distance world coordinates (x2, y2, z2) into the model, the far-distance world coordinates (x1, y1, z1) are calculated.

[0071] It should be noted that, since the calibration accuracy of the first intrinsic parameter matrix and the first extrinsic parameter matrix cannot be guaranteed at long distances, this embodiment calculates and obtains the long-distance world coordinates based on the near-distance world coordinates and the vehicle body posture data.

[0072] It should be noted that in this embodiment, vehicle attitude data is collected through a vehicle odometer.

[0073] In some optional embodiments, selecting the target data pair for least squares iteration includes:

[0074] Multiple sets of target data pairs are randomly selected for least squares iteration.

[0075] It should be noted that the least squares iteration method is a conventional technique in the existing technology, so it will not be elaborated on further.

[0076] In some optional embodiments, after selecting the remaining data pairs and verifying the second intrinsic parameter coefficients and the second extrinsic parameter coefficients using the reprojection method, the method further includes:

[0077] The verification error is obtained, and the magnitude of the verification error and the preset error threshold are determined. If the verification error is greater than the preset error threshold, the target data pair is determined to be erroneous, the target data pair is discarded, the target data pair is re-acquired, and the second internal parameter coefficient and the second external parameter coefficient are calculated.

[0078] This application also provides an automatic calibration device for vehicle visual detection, such as... Figure 3 As shown, it includes:

[0079] Static calibration module 1 is used to perform static calibration on the vehicle camera, record the first intrinsic parameter matrix and the first extrinsic parameter matrix, and obtain the projection matrix.

[0080] Automatic detection module 2 is used to automatically detect long-range and short-range information of ground targets during vehicle operation;

[0081] The first calculation module 3 is used to calculate the near-range world coordinates based on the projection matrix and near-range information;

[0082] Acquisition module 4 is used to acquire vehicle posture data between the long-distance information and the short-distance information; in this embodiment, the acquisition module is a vehicle odometer.

[0083] The second calculation module 5 is used to calculate and obtain the far-distance world coordinates based on the near-distance world coordinates and the vehicle posture data;

[0084] The third calculation module 6 is used to select the target data pair and perform least squares iteration to calculate the second internal parameter coefficient and the second external parameter coefficient.

[0085] Verification module 7 is used to select the remaining data pairs and verify the second intrinsic parameter coefficients and the second extrinsic parameter coefficients using the reprojection method.

[0086] This application also provides a computer-readable storage medium, including a program or instructions, which, when run on a computer, execute the above-described automatic calibration method for vehicle visual detection.

[0087] The above provides a detailed description of the automatic calibration method, apparatus, and medium for vehicle visual detection provided by the present invention. The specific embodiments are described only to aid in understanding the method and core ideas of the present invention. It should be noted that those skilled in the art can make various improvements and modifications to the present invention without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of the present invention.

Claims

1. An automatic calibration method for vehicle visual inspection, characterized in that, The method includes: Static calibration of the vehicle camera is performed, the first intrinsic parameter matrix and the first extrinsic parameter matrix are recorded, and the projection matrix is ​​obtained. During vehicle operation, it automatically detects both long-range and short-range information of ground targets. The near-range world coordinates are calculated based on the projection matrix and the near-range information. Collect vehicle posture data between the long-distance information and the short-distance information; Based on the near-range world coordinates and the vehicle posture data, the far-range world coordinates are calculated and obtained; The target data pair is formed based on the long-distance world coordinates and the long-distance information; Collect multiple sets of target data pairs, select the target data pairs for least squares iteration, and calculate the second intrinsic parameter coefficient and the second extrinsic parameter coefficient. Select the remaining data pairs and use the reprojection method to verify the second intrinsic parameter coefficients and the second extrinsic parameter coefficients.

2. The automatic calibration method for vehicle visual inspection according to claim 1, characterized in that, The static calibration of the vehicle camera, recording the first intrinsic parameter matrix and the first extrinsic parameter matrix, and obtaining the projection matrix includes: Static calibration of the vehicle camera is performed using Zhang Zhengyou's calibration method. Based on a pinhole camera model, the projection matrix expression between the pixel coordinate system (u,v) and the world coordinate system (x,y,z) is obtained. Where Zc represents the magnification factor. This represents the first intrinsic parameter matrix. Let R and T represent the rotation and translation vectors of the camera coordinate system relative to the world coordinate system, respectively. In the first intrinsic parameter matrix: fx = F / dx and fy = F / dy, where F is the focal length, dx and dy represent the physical length of a pixel on the calibration board in the x and y directions, respectively, and u0 and v0 represent the difference in horizontal and vertical pixels between the center pixel coordinates and the origin pixel coordinates of the image.

3. The automatic calibration method for vehicle visual inspection according to claim 2, characterized in that, The automatic detection of long-range and short-range information of ground targets in sequence includes: During vehicle operation, when the vehicle camera detects the ground target for the first time, it records the information of the ground target at this time and stores it as long-distance information. The long-distance information includes the pixel coordinate information (u1, v1) of the first image. As the vehicle moves, when the distance between the vehicle and the ground target is less than a preset distance threshold, the information of the ground target at this time is recorded and stored as close-range information. The close-range information includes the second image pixel coordinate information (u2, v2).

4. The automatic calibration method for vehicle visual inspection according to claim 3, characterized in that, The step of calculating near-range world coordinates based on the projection matrix and near-range information includes: The pixel coordinates of the second image are input into the projection matrix to calculate the near-range world coordinates (x2, y2, z2).

5. The automatic calibration method for vehicle visual inspection according to claim 4, characterized in that, The step of calculating and obtaining the distant world coordinates based on the near-range world coordinates and the vehicle posture data includes: The transformation equation T and the vehicle motion equation M are calculated from the vehicle posture data. Model building By substituting the transformation equation T, the vehicle motion equation M, and the near-distance world coordinates (x2, y2, z2) into the model, the far-distance world coordinates (x1, y1, z1) are calculated.

6. The automatic calibration method for vehicle visual inspection according to claim 1, characterized in that, The step of selecting the target data pair for least squares iteration includes: Multiple sets of target data pairs are randomly selected for least squares iteration.

7. The automatic calibration method for vehicle visual inspection according to claim 1, characterized in that, After selecting the remaining data pairs and verifying the second intrinsic parameter coefficients and the second extrinsic parameter coefficients using the reprojection method, the method further includes: The verification error is obtained, and the magnitude of the verification error and the preset error threshold are determined. If the verification error is greater than the preset error threshold, the target data pair is determined to be erroneous, the target data pair is discarded, the target data pair is re-acquired, and the second internal parameter coefficient and the second external parameter coefficient are calculated.

8. An automatic calibration device for vehicle visual inspection, characterized in that, include: The static calibration module is used to perform static calibration on the vehicle camera, record the first intrinsic parameter matrix and the first extrinsic parameter matrix, and obtain the projection matrix. The automatic detection module is used to automatically detect long-range and short-range information of ground targets while the vehicle is in motion; The first calculation module is used to calculate the near-range world coordinates based on the projection matrix and near-range information; The acquisition module is used to acquire vehicle posture data between the long-distance information and the short-distance information; The second calculation module is used to calculate and obtain the far-distance world coordinates based on the near-distance world coordinates and the vehicle posture data; The third calculation module is used to select the target data pair and perform least squares iteration to calculate the second intrinsic parameter coefficient and the second extrinsic parameter coefficient. The verification module is used to select the remaining data pairs and verify the second intrinsic parameter coefficients and the second extrinsic parameter coefficients using the reprojection method.

9. A computer-readable storage medium, characterized in that, Includes a program or instructions, wherein when the program or instructions are run on a computer, the method described in any one of claims 1 to 7 is executed.

Images