Camera production line calibration and verification integrated method and system, computer equipment and storage medium

Abstract

The invention belongs to the technical field of camera calibration, and particularly relates to a camera production line calibration and verification integrated method and system, computer equipment and a storage medium. Calibrating the camera by adopting a Zhang Zhengyou calibration method to obtain camera calibration parameters; performing distortion removal operation on the image to obtain pixel coordinates of the feature points after distortion removal; reconstructing the feature points in a world coordinate system Zw = 0 plane to obtain feature point reconstruction coordinates; and calculating the distance between the reconstructed feature points, and comparing the distance with the actual distance to obtain a distance measurement error, thereby realizing verification of camera parameters. The method is used for solving the problems that in the prior art, experimental environments need to be set up for camera calibration and verification respectively, much manual operation is needed in the process, the testing efficiency is low, and the requirement for calibration and verification of a camera production line cannot be met.

Description

Technical Field

[0001] The present invention belongs to the technical field of camera calibration, and specifically relates to a camera production line calibration and verification integrated method, system, computer equipment and storage medium. Background Art

[0002] With the development of machine vision and artificial intelligence technology, image reconstruction and perception of the surrounding environment (3D vision) has become a hot topic in academia and industry, and is widely used in video surveillance, visual navigation, industrial inspection, remote sensing and other fields. In order to determine the correspondence between the spatial position of an object and the position of an image, it is necessary to establish a camera geometry model and obtain the parameters of the camera geometry model through camera calibration. The accuracy of camera calibration directly affects the accuracy of 3D reconstruction and environmental perception, and is one of the key basic research contents in the field of machine vision.

[0003] At present, the reprojection error is generally used to evaluate the accuracy of camera calibration, that is, the difference between the actual imaging position of the object on the image plane and the projection position calculated by the geometric model. However, in applications such as machine vision, photogrammetry, and 3D reconstruction, it is necessary to obtain the spatial position relationship of the object, and the reprojection error in pixels cannot intuitively reflect the calibration accuracy of the camera. In the prior art, the accuracy of camera calibration is effectively verified through triangulation ranging experiments. Although the ranging error in millimeters can be output to reflect the accuracy of camera calibration, the camera calibration and verification experiments need to set up experimental environments separately, and a lot of manual operations are required in the process. The test efficiency is low and cannot meet the needs of camera production line calibration and verification. Summary of the invention

[0004] The present invention provides an integrated method for calibration and verification of a camera production line, which is used to solve the problems in the prior art that an experimental environment needs to be built separately, a large amount of manual operation is required in the process, the test efficiency is low, and the demand for calibration and verification of the camera production line cannot be met.

[0005] The present invention provides a camera production line calibration and verification integrated system, which is used to implement a camera production line calibration and verification integrated method.

[0006] The present invention provides a computer device for running a step algorithm flow of a camera production line calibration and verification integrated method.

[0007] The present invention provides a storage medium for storing a step algorithm flow of a camera production line calibration and verification integrated method.

[0008] The present invention is achieved through the following technical solutions:

[0009] A camera production line calibration and verification integrated method, the method comprising the following steps:

[0010] Step 1: Use Zhang Zhengyou calibration method to calibrate the camera and obtain the camera calibration parameters;

[0011] Step 2: performing a dedistortion operation on the image based on the camera calibration parameters to obtain pixel coordinates of the dedistorted feature points;

[0012] Step 3: Based on the pixel coordinates of the feature points after dedistortion, in the world coordinate system Z w =0 plane reconstruction feature points, obtain the reconstruction coordinates of the feature points;

[0013] Step 4: Reconstruct the coordinates according to the feature points, calculate the distance between the reconstructed feature points, and compare them with the actual distance to obtain the ranging error, thereby verifying the camera parameters.

[0014] Furthermore, the step 1 specifically includes the following steps:

[0015] Step 1.1: Use the camera to be calibrated to take multiple images of the calibration plate and obtain the pixel coordinates of the feature points;

[0016] Step 1.2: Establish a mapping relationship between the pixel coordinates of the feature point image coordinate system and the coordinates of the feature point world coordinate system, calibrate the camera according to the mapping relationship, and obtain the camera calibration parameters.

[0017] Furthermore, the step 1.1 also includes if there are problems such as uneven lighting, camera defocus, and excessive image noise during the calibration plate image capture process;

[0018] The reprojection error of each image is calculated based on the initial calibration, and abnormal images with too large reprojection errors are eliminated; the calibration plate image of the eliminated image pose is captured, and the camera is recalibrated.

[0019] According to the camera distortion model,

[0020]

[0021] In the formula, (x dist ,y dist ) is the coordinate of the distortion feature point in the imaging coordinate system; (x, y) is the coordinate of the ideal distortion-free feature point in the imaging coordinate system, where k1, k2, k3 are radial distortion coefficients; p1 and p2 are tangential distortion coefficients; the initial solution of the internal and external parameters is solved through the homography relationship between the feature point world coordinate system coordinate M and the feature point image coordinate system pixel coordinate m; then the solved internal and external parameters and distortion coefficients are globally optimized with the goal of minimizing the reprojection error.

[0022] The expression for minimizing the reprojection error is:

[0023]

[0024] In the formula, m ij is the pixel coordinate of the jth (j=1,…,m) feature point on the ith (i=1,…,n) captured image, m′ ij are the pixel coordinates of the reprojected feature points calculated according to the camera geometry model.

[0025] Furthermore, the step 2 is specifically as follows: assuming that the pixel coordinates m of the feature point image coordinate system detected on the distorted image are dist is dist ,v dist ), the imaging coordinate system coordinate of the feature point is (x dist ,y dist ), on the undistorted image, the pixel coordinate m of the feature point in the image coordinate system is (u, v), and the coordinate in the imaging coordinate system is (x, y).

[0026] The dedistortion operation is performed through the dedistortion model, and the pixel coordinate m of the dedistorted image coordinate system is obtained as (u, v).

[0027] Furthermore, the dedistortion model includes a radial distortion model and a tangential distortion model.

[0028] Furthermore, the step 3 is specifically as follows: assuming that the reconstructed feature points are in the world coordinate system Z w = 0 plane, then:

[0029]

[0030] Where u and v are pixel coordinates in the image coordinate system, X w is the value of the world coordinate system in the X direction, w is the value in the Y direction of the world coordinate system, R is the rotation matrix in the external parameters, K is the internal parameter matrix, T is the translation vector in the external parameters, Z c For depth information;

[0031] Solve for depth information Z c , and then use the camera inverse projection formula to obtain the world coordinates M′ of the feature point to be measured i , M′ j ,

[0032]

[0033] In the formula, Z ci , Z cj They are the feature points M′ i , M′ j The corresponding depth information.

[0034] Furthermore, the step 4 is specifically to obtain the distance L′ to be measured using the reconstructed world coordinate system coordinates of the feature pointij ,

[0035] L′ ij =||M′ i -M′ j ||2 (10)

[0036] Among them, M′ i , M′ j is the world coordinate of the feature point to be measured, ||||2 is the modulus of the vector;

[0037] L′ ij The actual distance L from the feature point ij Compare and get the relative error ε of ranging,

[0038]

[0039] After the ranging is completed, the camera calibration parameters, reprojection error and relative ranging error are output.

[0040] A camera production line calibration and verification integrated system, the system uses the camera production line calibration and verification integrated method as described above, the system includes:

[0041] The camera calibration module uses the Zhang Zhengyou calibration method to calibrate the camera to obtain the camera calibration parameters;

[0042] The image dedistortion module performs dedistortion operation on the image based on the camera calibration parameters to obtain the pixel coordinates of the feature points after dedistortion;

[0043] The feature point reconstruction module is based on the pixel coordinates of the feature points after dedistortion, in the world coordinate system Z w =0 plane reconstruction feature points, obtain the reconstruction coordinates of the feature points;

[0044] The parameter verification module calculates the distance between the reconstructed feature points based on the coordinates of the feature points, and compares it with the actual distance to obtain the ranging error, thereby verifying the camera parameters.

[0045] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the above method is implemented.

[0046] A computer-readable storage medium stores a computer program, and the computer program implements the above method when executed by a processor.

[0047] The beneficial effects of the present invention are:

[0048] The present invention is used to improve the efficiency of calibration and verification of a camera production line.

[0049] The present invention can simultaneously complete the ranging verification experiment on the existing camera production line calibration equipment without the need to separately configure the ranging verification environment, and can obtain the camera parameters, reprojection errors and ranging experiment results at one time.

[0050] The present invention has the advantages of high degree of automation, improved efficiency of production line calibration and verification, and cost savings.

[0051] The present invention is not limited to a specific camera geometric model and is widely applicable to calibration and verification of camera production lines with a variety of field of view ranges. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 It is a flow chart of the method of the present invention.

[0053] Figure 2 It is a schematic diagram of the camera ranging verification experiment of the present invention.

[0054] Figure 3 These are the experimental results of the reprojection error and ranging error of the present invention.

[0055] Figure 4 The characteristic point of the present invention is in the world coordinate system Z w = Schematic diagram of reconstruction of the 0 plane.

[0056] Figure 5 It is a system schematic diagram of the present invention.

[0057] Figure 6 It is a schematic diagram of a computer device of the present invention. DETAILED DESCRIPTION

[0058] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0059] It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0060] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, the singular forms "a", "an" and "the" are intended to include plural forms unless the context clearly indicates otherwise.

[0061] The following is attached to this application specification Figure 1-4 , the technical solutions in the embodiments of the present application are clearly and completely described. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application.

[0062] In the following description, many specific details are set forth to facilitate a full understanding of the present application, but the present application may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present application. Therefore, the present application is not limited to the specific embodiments disclosed below.

[0063] Embodiment 1

[0064] This embodiment provides a camera production line calibration and verification integrated method. Figure 1 As shown, the method comprises the following steps:

[0065] Step 1: Use Zhang Zhengyou calibration method to calibrate the camera and obtain the camera calibration parameters;

[0066] Step 1.1: Use the camera to be calibrated to shoot multiple calibration plate images to obtain the pixel coordinates of the feature points; the calibration plate pattern includes but is not limited to one or a combination of checkerboard, circular spots, field grid, BMW pattern and ChArUco; use a checkerboard calibration plate with known number of rows, columns and feature point spacing. The calibration plate should be as flat as possible and the calibration pattern should be as clear as possible. Take multiple calibration plate images in different postures, and the feature points should cover the entire field of view of the camera as much as possible. This embodiment uses a checkerboard calibration plate with 4 rows and 5 columns of feature points and a feature point spacing of 31 mm. Ten calibration images are taken with the calibration plate located in the center, top, bottom, left, right, top left, bottom left, top right, and bottom right of the image;

[0067] The Harris corner detection algorithm is used to extract the sub-pixel coordinates of the feature points. In this embodiment, the sub-pixel corner refinement window size is set to (11, 11), the number of iterations is 30, and the iteration accuracy is 0.001.

[0068] Furthermore, the step 1.1 also includes if there are problems such as uneven lighting, camera defocus, and excessive image noise during the calibration plate image capture process;

[0069] The reprojection error of each image is calculated based on the initial calibration, and abnormal images with too large reprojection errors are eliminated; the calibration plate image of the eliminated image pose is captured, and the camera is recalibrated.

[0070] Step 1.2: Establish a mapping relationship between the pixel coordinates of the feature point image coordinate system and the coordinates of the feature point world coordinate system, calibrate the camera according to the mapping relationship, and obtain the camera calibration parameters.

[0071] By obtaining the homography relationship between the world coordinates and pixel coordinates of the feature points on multiple images, the projection equation is established to solve the parameters of the pinhole camera model and construct the world coordinate system (O w -X w Y w Z w ), camera coordinate system (O c -X c Y c Z c ), imaging coordinate system (O-xy) and image coordinate system (O pix -uv) between the feature point. According to the pinhole model, the relationship between the feature point world coordinate system coordinate M and the feature point image coordinate system coordinate m can be expressed as:

[0072]

[0073] In the formula, Z c is the depth information, f x , f y are the focal lengths of the camera in the x and y directions of the imaging coordinate system, u0 and v0 are the principal point offsets of the camera in the x and y directions of the imaging coordinate system, K is the intrinsic parameter matrix; R and T are the rotation matrix and translation vector in the extrinsic parameters, respectively.

[0074] According to the camera distortion model,

[0075]

[0076] In the formula, (x dist ,y dist ) is the coordinate of the distortion feature point in the imaging coordinate system; (x, y) is the coordinate of the ideal distortion-free feature point in the imaging coordinate system, where k1, k2, k3 are radial distortion coefficients; p1, p2 are tangential distortion coefficients; the initial solution of the internal and external parameters is solved through the homography relationship between the feature point world coordinate system coordinate M and the feature point image coordinate system pixel coordinate m; then the solved internal and external parameters and distortion coefficients are globally optimized with the goal of minimizing the reprojection error.

[0077] The expression for minimizing the reprojection error is:

[0078]

[0079] In the formula, m ij is the pixel coordinate of the jth (j=1,…,m) feature point on the ith (i=1,…,n) captured image, m′ ij are the pixel coordinates of the reprojected feature points calculated according to the camera geometry model.

[0080] This embodiment uses the Levenberg-Marquardt optimization algorithm for iterative solution, the number of iterations is set to 500 times, and the iteration convergence accuracy is set to 1e-8.

[0081] Step 2: Dedistort the image based on the camera calibration parameters in step 1 to obtain the pixel coordinates of the dedistorted feature points;

[0082] Furthermore, let the pixel coordinates m of the feature point image coordinate system detected on the distorted image be dist is dist ,v dist ), the imaging coordinate system coordinate of the feature point is (x dist ,y dist ), on the undistorted image, the pixel coordinate m of the image coordinate system corresponding to the feature point is (u, v), and the coordinate of the imaging coordinate system is (x, y);

[0083] The dedistortion operation is performed through the dedistortion model, and the pixel coordinate m of the dedistorted image coordinate system is obtained as (u, v).

[0084] Furthermore, the dedistortion model includes a radial distortion model and a tangential distortion model.

[0085] This embodiment uses the following method to perform dedistortion based on the conversion relationship between the image coordinate system and the imaging coordinate system:

[0086]

[0087] In the formula, f x , f y are the focal lengths of the camera in the x and y directions of the imaging coordinate system, respectively; u0 and v0 are the principal point offsets of the camera in the x and y directions of the imaging coordinate system.

[0088] The obtained (x dist ,y dist ) as the initial value (x0,y0),

[0089]

[0090] The dedistorted imaging coordinate system coordinates (x, y) can be obtained by repeatedly iterating the distortion model.

[0091]

[0092] When the number of iterations or the iteration accuracy requirement is reached, the iteration is exited; the x obtained by the iteration n ,y n As the imaging coordinate system coordinates (x, y) on the undistorted image;

[0093] According to the conversion relationship between the image coordinate system and the imaging coordinate system,

[0094]

[0095] Through the above process, the pixel coordinate m of the dedistorted image coordinate system can be obtained as (u, v).

[0096] Step 3: Based on the pixel coordinates of the feature points after dedistortion in step 2, calculate the pixel coordinates in the world coordinate system Z w =0 plane reconstruction feature points, obtain the reconstruction coordinates of the feature points;

[0097] Assume that the reconstructed feature points are in the world coordinate system Z w =0 plane, then:

[0098]

[0099] Where u and v are pixel coordinates in the image coordinate system, X w is the value of the X-axis direction of the world coordinate system, Y w is the value of the Y axis direction of the world coordinate system, R is the rotation matrix in the external parameters, K is the internal parameter matrix, T is the translation vector in the external parameters, Z c For depth information;

[0100] The third dimension on both sides of the equation has only one unknown quantity Z c , solving this equation gives the depth information Z c .

[0101] Solve for depth information Z c , and then use the camera inverse projection formula to obtain the world coordinates M′ of the feature point to be measured i , M′ j ,

[0102]

[0103] In the formula, Z ci , Z cj They are the feature points M′ i , M′ j The corresponding depth information.

[0104] Step 4: Reconstruct the coordinates according to the feature points in step 3, calculate the distance between the reconstructed feature points, and compare them with the actual distance to obtain the ranging error, so as to verify the camera parameters;

[0105] The distance to be measured L′ is obtained by using the reconstructed world coordinate system coordinates of the feature points ij ,

[0106] L′ ij =||M′ i -M′ j ||2 (10)

[0107] Among them, M′ i , M′ j is the world coordinate of the feature point to be measured, ‖‖2 is the modulus of the vector;

[0108] L′ ij The actual distance L from the feature point ij The relative error ε of the distance measurement is obtained by comparison; the actual distance between the upper left corner feature point and the lower right corner feature point of the chessboard calibration plate measured in this embodiment is 155mm,

[0109]

[0110] After the ranging is completed, the camera calibration parameters, reprojection error and relative ranging error are output.

[0111] Embodiment 2

[0112] This embodiment provides a camera production line calibration and verification integrated system, such as Figure 5 As shown, the system uses a camera production line calibration and verification integrated method as described in Example 1, and the system includes:

[0113] The camera calibration module uses the Zhang Zhengyou calibration method to calibrate the camera to obtain the camera calibration parameters, and then performs global optimization based on the camera distortion model;

[0114] The specific working process of the camera calibration module is to use the camera to be calibrated to shoot multiple calibration plate images to obtain the pixel coordinates of the feature points; the calibration plate pattern includes but is not limited to a combination of one or more of a checkerboard, a circular spot, a field grid, a BMW pattern, and ChArUco; a checkerboard calibration plate with known rows, columns, and feature point spacing is used. The calibration plate is as flat as possible, and the calibration pattern is as clear as possible. Take multiple calibration plate images in different postures, and the feature points cover the entire field of view of the camera as much as possible. This embodiment uses a checkerboard calibration plate with 4 rows and 5 columns of feature points and a feature point spacing of 31 mm, and shoots 10 calibration images with the calibration plate located in the center, top, bottom, left, right, top left, bottom left, top right, and bottom right of the image.

[0115] The Harris corner detection algorithm is used to extract the sub-pixel coordinates of the feature points. In this embodiment, the sub-pixel corner refinement window size is set to (11, 11), the number of iterations is 30, and the iteration accuracy is 0.001.

[0116] Furthermore, if there are problems such as uneven lighting, camera defocus, or excessive image noise during the calibration plate image capture process,

[0117] The reprojection error of each image is calculated based on the initial calibration, and abnormal images with too large reprojection errors are eliminated; the calibration plate image of the eliminated image pose is captured, and the camera is recalibrated.

[0118] Furthermore, a mapping relationship between the pixel coordinates of the feature point image coordinate system and the coordinates of the feature point world coordinate system is established, and the camera is calibrated according to the mapping relationship to obtain the camera calibration parameters.

[0119] By obtaining the homography relationship between the world coordinates and pixel coordinates of the feature points on multiple images, the projection equation is established to solve the parameters of the pinhole camera model and construct the world coordinate system (O w -X w Y w Z w ), camera coordinate system (O c -X c Y c Z c ), imaging coordinate system (O-xy) and image coordinate system (O pix -uv) between the feature point. According to the pinhole model, the relationship between the feature point world coordinate system coordinate M and the feature point image coordinate system coordinate m can be expressed as:

[0120]

[0121] In the formula, Z c is the depth information, f x , f y are the focal lengths of the camera in the x and y directions of the imaging coordinate system, u0 and v0 are the principal point offsets of the camera in the x and y directions of the imaging coordinate system, K is the intrinsic parameter matrix; R and T are the rotation matrix and translation vector in the extrinsic parameters, respectively.

[0122] Furthermore, global optimization is performed based on the camera calibration parameters and the camera distortion model.

[0123] Going further, the camera distortion model,

[0124]

[0125] In the formula, (x dist ,y dist) is the coordinate of the distortion feature point in the imaging coordinate system; (x, y) is the coordinate of the ideal distortion-free feature point in the imaging coordinate system, where k1, k2, k3 are radial distortion coefficients; p1, p2 are tangential distortion coefficients; the initial solution of the internal and external parameters is solved through the homography relationship between the feature point world coordinate system coordinate M and the feature point image coordinate system pixel coordinate m; then the solved internal and external parameters and distortion coefficients are globally optimized with the goal of minimizing the reprojection error.

[0126] The expression for minimizing the reprojection error is:

[0127]

[0128] In the formula, m ij is the pixel coordinate of the jth (j=1,…,m) feature point on the ith (i=1,…,n) captured image, m′ ij are the pixel coordinates of the reprojected feature points calculated according to the camera geometry model.

[0129] This embodiment uses the Levenberg-Marquardt optimization algorithm for iterative solution, the number of iterations is set to 500 times, and the iteration convergence accuracy is set to 1e-8.

[0130] The image dedistortion module performs dedistortion operation on the image based on the camera calibration parameters to obtain the pixel coordinates of the feature points after dedistortion;

[0131] Furthermore, let the pixel coordinates m of the feature point image coordinate system detected on the distorted image be dist is dist ,v dist ), the imaging coordinate system coordinate of the feature point is (x dist ,y dist ), on the undistorted image, the pixel coordinate m of the feature point in the image coordinate system is (u, v), and the coordinate in the imaging coordinate system is (x, y).

[0132] The dedistortion operation is performed through the dedistortion model, and the pixel coordinate m of the dedistorted image coordinate system is obtained as (u, v).

[0133] Furthermore, the dedistortion model includes a radial distortion model and a tangential distortion model.

[0134] This embodiment uses the following method to perform dedistortion based on the conversion relationship between the image coordinate system and the imaging coordinate system:

[0135]

[0136] In the formula, f x , f yare the focal lengths of the camera in the x and y directions of the imaging coordinate system, respectively; u0 and v0 are the principal point offsets of the camera in the x and y directions of the imaging coordinate system;

[0137] The obtained (x dist ,y dist ) as the initial value (x0,y0),

[0138]

[0139] The dedistorted imaging coordinate system (x, y) can be obtained by repeatedly iterating the distortion model.

[0140]

[0141] When the number of iterations or the iteration accuracy requirement is reached, the iteration is exited and the x obtained by the iteration is n ,y n As the imaging coordinate system coordinates (x, y) on the undistorted image.

[0142] According to the conversion relationship between the image coordinate system and the imaging coordinate system,

[0143]

[0144] Through the above process, the pixel coordinate m of the dedistorted image coordinate system can be obtained as (u, v).

[0145] The feature point reconstruction module is based on the pixel coordinates of the feature points after dedistortion, in the world coordinate system Z w =0 plane reconstruction feature points, obtain the reconstruction coordinates of the feature points;

[0146] Assume that the reconstructed feature points are in the world coordinate system Z w = 0 plane, then:

[0147]

[0148] Where u and v are pixel coordinates in the image coordinate system, X w is the value of the X-axis direction of the world coordinate system, Y w is the value of the Y axis direction of the world coordinate system, R is the rotation matrix in the external parameters, K is the internal parameter matrix, T is the translation vector in the external parameters, Z c For depth information;

[0149] The third dimension on both sides of the equation has only one unknown quantity Z c , solving this equation gives the depth information Z c ;

[0150] Solve for depth information Z c , and then use the camera inverse projection formula to obtain the world coordinates M′ of the feature point to be measuredi , M′ j

[0151]

[0152] In the formula, Z ci , Z cj They are the feature points M′ i , M′ j The corresponding depth information.

[0153] The parameter verification module calculates the distance between the reconstructed feature points based on the coordinates of the feature points, and compares it with the actual distance to obtain the ranging error, thereby verifying the camera parameters.

[0154] The distance to be measured L′ is obtained by using the reconstructed world coordinate system coordinates of the feature points ij ,

[0155] L′ ij =||M′ i -M′ j ||2 (10)

[0156] Among them, M′ i , M′ j is the world coordinate of the feature point to be measured, ||||2 is the modulus of the vector;

[0157] L′ ij The actual distance L from the feature point ij The relative error ε of the distance measurement is obtained by comparison; the actual distance between the upper left corner feature point and the lower right corner feature point of the chessboard calibration plate measured in this embodiment is 155mm,

[0158]

[0159] After the ranging is completed, the camera calibration parameters, reprojection error and relative ranging error are output.

[0160] Embodiment 3

[0161] An embodiment of the present invention provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the memory is used to store software programs and modules, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory. The memory and the processor are connected via a bus. Specifically, the processor implements any step in the first embodiment above by running the computer program stored in the memory.

[0162] It should be understood that in the embodiments of the present invention, the processor referred to may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0163] The memory may include a read-only memory, a flash memory, and a random access memory, and provides instructions and data to the processor. A part or all of the memory may also include a nonvolatile random access memory.

[0164] As can be seen from the above, the electronic device provided by the embodiment of the present invention can implement the integrated camera production line calibration verification method as described in the first embodiment by running a computer program, and can obtain the camera parameters, reprojection error and ranging experiment results at one time, thereby reducing manual operations and improving test efficiency.

[0165] It should be understood that if the above-mentioned integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The above-mentioned computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the above-mentioned computer program includes computer program code, and the above-mentioned computer program code can be in source code form, object code form, executable file or some intermediate form. The above-mentioned computer-readable medium may include: any entity or device capable of carrying the above-mentioned computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the above-mentioned computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction.

[0166] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0167] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the above-mentioned device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0168] It should be noted that the methods and detailed examples provided in the above embodiments can be combined with the devices and equipment provided in the embodiments, and references can be made to each other, and no further details will be given.

[0169] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0170] In the embodiments provided by the present invention, it should be understood that the disclosed apparatus / terminal equipment and method can be implemented in other ways. For example, the apparatus / equipment embodiments described above are only illustrative, for example, the division of the above modules or units is only a logical function division, and in actual implementation, other division methods can be used, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0171] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions described in the above embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention.

Claims

1. A camera production line calibration and verification integrated method, characterized in that: The method comprises the following steps, Step 1: Use Zhang Zhengyou calibration method to calibrate the camera and obtain the camera calibration parameters; Step 2: performing a dedistortion operation on the image based on the camera calibration parameters to obtain pixel coordinates of the dedistorted feature points; Step 3: Based on the pixel coordinates of the feature points after dedistortion, in the world coordinate system Z w =0 plane reconstruction feature points, obtain the reconstruction coordinates of the feature points; Step 4: Reconstruct the coordinates according to the feature points, calculate the distance between the reconstructed feature points, and compare them with the actual distance to obtain the ranging error, thereby verifying the camera parameters.

2. The camera production line calibration and verification integrated method according to claim 1, characterized in that: The step 1 specifically includes the following steps: Step 1.1: Use the camera to be calibrated to take multiple images of the calibration plate and obtain the pixel coordinates of the feature points; Step 1.2: Establish a mapping relationship between the pixel coordinates of the feature point image coordinate system and the coordinates of the feature point world coordinate system, calibrate the camera according to the mapping relationship, and obtain the camera calibration parameters.

3. The camera production line calibration and verification integrated method according to claim 2, characterized in that: The step 1.1 also includes if there are problems such as uneven lighting, camera defocus, and excessive image noise during the calibration plate image capture process; The reprojection error of each image is calculated based on the initial calibration, and abnormal images with too large reprojection errors are eliminated; the calibration plate image of the eliminated image pose is captured, and the camera is recalibrated.

4. The camera production line calibration and verification integrated method according to claim 1, characterized in that: Specifically, step 2 is as follows: assuming that the pixel coordinates m of the feature point image coordinate system detected on the distorted image are dist is dist ,v dist ), the imaging coordinate system coordinate of the feature point is (x dist ,y dist ), on the undistorted image, the pixel coordinate m of the image coordinate system corresponding to the feature point is (u, v), and the coordinate of the imaging coordinate system is (x, y); Dedistortion operation is performed through the dedistortion model to obtain the pixel coordinate m of the dedistorted image coordinate system as (u, v); The dedistortion model includes a radial distortion model and a tangential distortion model.

5. The camera production line calibration and verification integrated method according to claim 1, characterized in that: Specifically, step 3 is as follows: assuming that the reconstructed feature points are in the world coordinate system Z w = 0 plane, then: Where u and v are pixel coordinates in the image coordinate system, X w is the value of the world coordinate system in the X direction, w is the value of the Y direction of the world coordinate system, R is the rotation matrix in the external parameters, K is the internal parameter matrix, T is the translation vector in the external parameters, Z c For depth information; Solve for depth information Z c , and then use the camera inverse projection formula to obtain the world coordinates M of the feature point to be measured i ′、M j ′, In the formula, Z ci , Z cj They are feature points M i ′、M j ’ corresponds to the depth information.

6. The camera production line calibration and verification integrated method according to claim 1, characterized in that: The step 4 specifically comprises obtaining the distance L to be measured by using the reconstructed world coordinate system coordinates of the feature point ij ′, L ij ′=||M i ′-M j ′||2 (10) Among them, M i ′、M j ′ is the world coordinate of the feature point to be measured, || ||2 is the modulus of the vector; L ij ′ and the actual distance L between the feature points ij Compare and get the relative error ε of ranging, After the ranging is completed, the camera calibration parameters, reprojection error and relative ranging error are output.

7. A camera production line calibration and verification integrated system, characterized in that: The system uses a camera production line calibration and verification integrated method as described in any one of claims 1 to 6, and the system includes: The camera calibration module uses the Zhang Zhengyou calibration method to calibrate the camera to obtain the camera calibration parameters; The image dedistortion module performs dedistortion operation on the image based on the camera calibration parameters to obtain the pixel coordinates of the feature points after dedistortion; The feature point reconstruction module is based on the pixel coordinates of the feature points after dedistortion, in the world coordinate system Z w =0 plane reconstruction feature points, obtain the reconstruction coordinates of the feature points; The parameter verification module calculates the distance between the reconstructed feature points based on the coordinates of the feature points, and compares it with the actual distance to obtain the ranging error, thereby verifying the camera parameters.

8. A computer device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images