A calibration method, system, chip, and device

Abstract

This invention discloses a calibration method, system, chip, and device. It simultaneously acquires checkerboard images from both left and right directions on a calibration board, and stitches the acquired images to obtain a panoramic checkerboard image. The Harris corner detection method is used to detect the corner positions in the panoramic checkerboard image. Based on the corner positions in the panoramic checkerboard image, linear positions on the checkerboard to be calibrated are selected, and the start and end points of these positions are obtained. The pixel equivalents of the checkerboard squares containing the start and end points are calculated. The pixel equivalents are used to calculate the mapping relationship between the pixel coordinates and world coordinates of the linear positions between the start and end points, completing the calibration. This invention simplifies the calibration process, eliminates the need to consider deformation issues caused by image stitching, and enhances the effectiveness of the calibration method after multi-camera stitching.

Description

Technical Field

[0001] This invention belongs to the field of visual measurement technology, specifically relating to a calibration method, system, chip, and device. Background Technology

[0002] Visual measurement, as an emerging technology in modern inspection, features non-contact operation, high precision, and high efficiency. A vision-based motorcycle external dimension inspection system can overcome the limitations of traditional measurement methods for measuring motorcycle appearance. A motorcycle is typically 2–3.5 meters long. To ensure measurement accuracy, limitations in the actual scene mean that a single camera cannot capture the entire object being measured; therefore, dual or even multiple cameras are required.

[0003] Multi-camera calibration is fundamental to visual measurement, and its accuracy significantly impacts the precision of subsequent measurements. To address this, Li Xing et al. proposed a "binocular camera calibration method based on partial corner points," which solves the problem of existing binocular calibration methods failing to cover certain areas, leading to larger errors in those areas. Zhang Qingchuan et al. invented a "high-precision calibration method for stereo vision 3D digital image correlation measurement," utilizing dual cameras to record images of the calibration board on a display and employing a dual-camera calibration method to complete the calibration, reducing costs while maintaining accuracy. Wang Can et al. proposed a "camera calibration method based on binocular calibration," obtaining a set of calibration physical features and a fitting plane that meet preset conditions through multiple single-camera and binocular calibrations, solving the problem of inaccurate camera calibration results due to manufacturing errors in the calibration board itself. Wu Li et al. invented a "multi-camera calibration method without a common field of view based on line structured light," using multiple cameras to capture laser lines to obtain multiple sets of fitting plane parameters, combining planar coplanar constraints to obtain the relative positional relationships of the cameras, and improving calibration accuracy through the LM optimization algorithm. In their research on "High-Precision Calibration Based on Multi-Camera Systems," Xiao Yifan et al. proposed a "Joint Calibration Method for Multiple Low-Pixel Cameras." This method selects one camera as the master camera and calculates the mapping matrix between the pixel coordinate systems of the other cameras and the master camera's pixel coordinate system, thus infinitely expanding the master camera's field of view. Simultaneously, a two-step calibration method is employed to improve calibration accuracy. Traditional multi-camera calibration methods require obtaining the intrinsic and extrinsic parameters of multiple cameras, unifying their coordinate systems, and deriving the mapping relationship from the image coordinate system to the world coordinate system. The entire calibration process is complex and computationally intensive.

[0004] When using dual or multiple cameras to capture images, the images need to be stitched together. The stitched image may have obvious "stitching lines." If the stitched image is a checkerboard calibration board image, the checkerboard at the "stitching lines" may be distorted. Figure 1As shown, using the corner coordinates obtained from the deformed checkerboard pattern as the calibration basis will lead to inaccurate camera calibration results, thus affecting the final measurement accuracy. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a calibration method, system, chip and device to address the shortcomings of the prior art, and to solve the technical problems of complex steps and low calibration accuracy caused by image stitching deformation of the calibration board in traditional multi-camera calibration methods.

[0006] The present invention adopts the following technical solution:

[0007] A calibration method includes the following steps:

[0008] S1. Simultaneously acquire chessboard images from both the left and right sides of the calibration board, and stitch the acquired images together to obtain a panoramic image of the chessboard.

[0009] S2. Use the Harris corner detection method to detect the corner positions in the checkerboard panoramic image obtained in step S1.

[0010] S3. Based on the corner positions in the panoramic image of the chessboard obtained in step S2, select the linear positions that need to be marked on the chessboard, obtain the start and end points of the positions, and calculate the pixel equivalent of the chessboard where the start and end points are located.

[0011] S4. Using the pixel equivalent obtained in step S3, calculate the mapping relationship between the pixel coordinates and world coordinates of the linear position between the starting point and the ending point to complete the calibration.

[0012] Specifically, step S1 is as follows:

[0013] S101. Use the SURF algorithm to extract feature points from the two images to be stitched together;

[0014] S102. Use the nearest neighbor and second nearest neighbor methods to save the confidence of the two optimal matching points;

[0015] S103. Sort the matching points in the two images, save the matching points of the images with high confidence into the same set, delete the matching points of the images with low confidence, and obtain the image sequence that can be correctly matched. Merge all matching points with confidence higher than the threshold into one set.

[0016] S104. Roughly estimate the parameters of the two images to be stitched together and calculate the rotation matrix. Use bundle adjustment to estimate the rotation matrix.

[0017] S105. Use a rotation matrix to correct the waveform of the two images to be stitched together, and then stitch the images together.

[0018] Specifically, step S2 is as follows:

[0019] S201. Calculate the gradient I of each image pixel in the x and y directions. x and I y The image is then filtered for noise using a Gaussian filter function, and a structure tensor is constructed.

[0020] S202. Calculate the corner response value R of all pixels in the original image;

[0021] S203. Compare the R value of each pixel in the image with the R values ​​of other pixels within the window range, and retain the pixel with the largest R value;

[0022] S204. Set a threshold and mark pixels with R values ​​greater than the threshold as corner points.

[0023] Specifically, step S3 is as follows:

[0024] Based on the required horizontal and vertical calibration mapping relationships, calculate the pixel distances of the fitted straight line Li to the corner point where the lines connecting the starting and ending points intersect, i = 1, 2, 3, 4 respectively;

[0025] Calculate the pixel equivalent of the starting and ending squares respectively.

[0026] Furthermore, the pixel distance to the fitted straight line at the corner point where the lines connecting the starting and ending points intersect is calculated as follows:

[0027] Use the departure point (x' i ,y' i The line fitted to the two corner points of the nearest four chessboard corner points that are the intersections of the lines connecting the starting and ending points is Li, and the line fitted to the point (x') is... i ,y' i The distance from the line to the line Li is d. i The line fitted to the two unfitted corner points is Li', and the line fitted to point (x') is... i ,y' i The distance d' from the line Li' i Point (x') i ,y' i The pixel distance d of the square where the ) is located ti for:

[0028] d ti =d' i +d i

[0029] The pixel equivalent d within this cell ni for:

[0030]

[0031] Where M is the actual size of each chessboard square.

[0032] Specifically, step S4 is as follows:

[0033] Calculate the actual distance between the starting point and the end point and the corner point where the lines connecting the two points intersect, and fit the straight line accordingly. Based on the actual distance, obtain the mapping relationship between the pixel coordinates of the linear position and the world coordinates.

[0034] Furthermore, the mapping relationship h between the pixel coordinates of the linear position and the world coordinates is as follows:

[0035]

[0036] Where, d mi For point (x') i ,y' i The actual size of the fitted straight line Li, i = 1, 2, 3, 4, d, is the angle point where the lines connecting the starting point and the ending point intersect. st d represents the actual distance between the fitted lines of the two corner points in the horizontal and vertical directions. i 'For point (x)' i ,y' i The distance d from the line Li' to the line Li' i For point (x) i ,y i The distance N is the fitted straight line Li, which is the angle between the lines connecting the starting point and the ending point to the intersection of the two points. t M represents the number of checkerboard squares between two corner points fitted by straight lines, where M is the actual size of each checkerboard square.

[0037] Secondly, embodiments of the present invention provide a calibration system, including:

[0038] The acquisition module simultaneously acquires chessboard images from both the left and right sides of the calibration board, and stitches the acquired images together to obtain a panoramic image of the chessboard.

[0039] The detection module uses the Harris corner detection method to detect the corner positions in the chessboard panoramic image obtained by the acquisition module.

[0040] The calculation module selects the linear positions to be marked on the chessboard grid based on the corner positions in the chessboard grid panoramic image obtained by the detection module, obtains the start and end points of the positions, and calculates the pixel equivalent of the chessboard grid where the start and end points are located.

[0041] The calibration module uses the pixel equivalent obtained from the calculation module to calculate the mapping relationship between the pixel coordinates and world coordinates of the linear position between the starting and ending points, and completes the calibration.

[0042] Thirdly, a chip includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described calibration method.

[0043] Fourthly, embodiments of the present invention provide an electronic device, including a computer program, which, when executed by a processor, implements the steps of the above-described calibration method.

[0044] Compared with the prior art, the present invention has at least the following beneficial effects:

[0045] A new calibration method is proposed. Traditional multi-camera calibration obtains the intrinsic and extrinsic parameters of the cameras and unifies the extrinsic parameters of the multiple cameras into the same coordinate system for global calibration. This method only needs to obtain the corner points of the chessboard and perform local calibration. While ensuring calibration accuracy, it simplifies the calibration process and eliminates the need to consider splicing deformation.

[0046] Furthermore, when stitching, its stitching and fusion methods are more adaptable to partial tilt and distortion compared to the SIFT and ORB algorithms, and the stitching effect is better, allowing multiple images to be stitched at once.

[0047] Furthermore, this method exhibits rotation invariance and scale invariance when extracting corner points. It typically provides high accuracy in corner detection by determining the corner location through calculations of pixel grayscale values ​​and eigenvalues ​​of the correlation matrix. This calculation method is robust to corner detection, effectively filtering out noise and flat areas in the image. Moreover, this method is simple to implement and requires minimal preprocessing steps or parameter adjustments.

[0048] Furthermore, when calculating pixel equivalents for local areas of an image, correlating the number of pixels within a local area with the actual size allows for a more accurate calculation of the measurement range, improving the accuracy of image processing and analysis. This is particularly beneficial for multi-camera calibration, which requires calculations on different images, enabling more accurate image comparison and analysis.

[0049] Furthermore, by obtaining the mapping relationship between pixel coordinates and world coordinates, accurate positioning and tracking of objects in images can be achieved in terms of location acquisition; in the field of 3D reconstruction, the shape and geometric information of 3D objects can be calculated; and in terms of scale measurement, the accurate size of objects can also be obtained. Consistent image processing can be performed on different devices and platforms, without being limited by resolution and size.

[0050] It is understood that the beneficial effects of the second to fourth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here.

[0051] In summary, the calibration process of this invention is simple, does not require consideration of the deformation problem caused by image stitching, simplifies the calibration process, and enhances the effectiveness of the calibration method after multi-camera stitching.

[0052] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0053] Figure 1 A schematic diagram showing the deformation of the checkerboard pattern at the suture line;

[0054] Figure 2 This is a schematic diagram of the method flow of the present invention;

[0055] Figure 3 This is a schematic diagram of the calibration board image stitching result in the method of the present invention, wherein (a) is an image taken by the left camera, (b) is an image taken by the right camera, and (c) is the stitching result after the images taken by the left and right cameras are stitched together;

[0056] Figure 4 This is a schematic diagram of the Harris algorithm detecting corner positions in the method of this invention;

[0057] Figure 5 This is a schematic diagram of the mapping relationship between the starting point and the ending point on the chessboard grid in the method of the present invention, wherein (a) is a schematic diagram of the starting point, the ending point and the four adjacent corner points, and (b) is a schematic diagram of the fitted straight line between the starting point, the ending point and the corner points;

[0058] Figure 6 A schematic diagram of a computer device provided in an embodiment of the present invention;

[0059] Figure 7 A block diagram of a chip provided for an embodiment of the present invention;

[0060] Figure 8 This is a schematic diagram of the background image provided in an embodiment of the present invention, wherein (a) is a background image captured by the left camera, (b) is a background image captured by the right camera, and (c) is a stitched background image;

[0061] Figure 9 This is a schematic diagram of calibration images provided in an embodiment of the present invention, wherein (a) is a calibration image taken by the left camera, and (b) is a calibration image taken by the right camera. Figure 9 (c) is the stitched calibration image;

[0062] Figure 10 This is a schematic diagram of the tested motor vehicle image provided in an embodiment of the present invention, wherein (a) is an image of the tested motor vehicle taken by the left camera, (b) is an image of the tested motor vehicle taken by the right camera, and (c) is a stitched image of the tested motor vehicle.

[0063] Figure 11 This is a schematic diagram of the edge portion of a motor vehicle under test provided in an embodiment of the present invention, wherein (a) is an image of the location of the edge point, and (b) is an enlarged image of the location of the edge point;

[0064] Figure 12 This is a schematic diagram of the fusion image of a chessboard pattern and a tested motor vehicle provided in an embodiment of the present invention;

[0065] Figure 13 The diagram shows the results of motor vehicle size measurement provided in an embodiment of the present invention. Detailed Implementation

[0066] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0067] In the description of this invention, it should be understood that the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0068] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0069] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Additionally, the character " / " in this invention generally indicates that the preceding and following objects have an "or" relationship.

[0070] It should be understood that although terms such as first, second, third, etc., may be used in the embodiments of the present invention to describe the preset range, these preset ranges should not be limited to these terms. These terms are only used to distinguish the preset ranges from one another. For example, without departing from the scope of the embodiments of the present invention, the first preset range may also be referred to as the second preset range, and similarly, the second preset range may also be referred to as the first preset range.

[0071] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0072] The accompanying drawings illustrate various structural schematic diagrams according to embodiments disclosed in this invention. These drawings are not to scale, and some details have been enlarged for clarity, and some details may have been omitted. The shapes of the various regions and layers shown in the drawings, as well as their relative sizes and positional relationships, are merely exemplary and may deviate from reality due to manufacturing tolerances or technical limitations. Furthermore, those skilled in the art can design regions / layers with different shapes, sizes, and relative positions as needed.

[0073] This invention provides a calibration method that determines the starting and ending points of a location by identifying the linear positions to be calibrated in an image. It then fits the distances of straight lines from the two points to the coordinates of the checkerboard corner points in the horizontal and vertical directions, calculates the number of complete checkerboard squares between the two points, and obtains the mapping relationship between the pixel coordinate system and the world coordinate system by combining the actual size of the checkerboard squares. This method overcomes the technical problem of inaccurate calibration results caused by stitching errors during the stitching process of dual cameras or even multiple cameras.

[0074] Please see Figure 2 The present invention provides a calibration method based on a motorcycle outline dimension visual inspection system, comprising the following steps:

[0075] S1. Control the left and right cameras to simultaneously capture images of the checkerboard on the calibration board, and then stitch the captured images together.

[0076] Please see Figure 3 The specific steps are as follows:

[0077] S101, Feature Point Detection

[0078] The SURF algorithm is used to extract feature points from the two images to be stitched together.

[0079] S102, Feature Point Matching

[0080] The confidence scores of the two best matching points are saved using the nearest neighbor and second nearest neighbor methods.

[0081] S103. Save feature points with high confidence.

[0082] The matching points in the two images are sorted, and matching points from the images with high confidence are saved into the same set. Matching points from the images with lower confidence are deleted, resulting in a sequence of images that can be correctly matched. Finally, all matching points with a confidence level higher than a threshold are merged into a single set.

[0083] S104. Roughly estimate the parameters of the two images to be stitched together and calculate the rotation matrix. Use bundle adjustment to further estimate the rotation matrix more accurately.

[0084] S105. Perform waveform correction on the image and stitch the images together.

[0085] S2. Obtain the stitched panoramic image of the chessboard pattern, and use the Harris corner detection algorithm to detect the corner positions in the image;

[0086] Please see Figure 4 The specific steps are as follows:

[0087] S201. Calculate the gradient I of each image pixel in the x and y directions. x and I y The image is then filtered for noise using a Gaussian filter function, and a structure tensor M is constructed.

[0088]

[0089] Among them, w u,v It is a discrete Gaussian function, and its expression is: I xy =I x ×I y .

[0090] S202. Calculate the corner response value R of all pixels in the original image;

[0091]

[0092] in, Let M be the determinant of matrix M. Let M be the trace of matrix M, and k be an empirical coefficient, ranging from 0.04 to 0.06.

[0093] S203, Local Nonmaximum Suppression

[0094] The R value of each pixel in the image is compared with the R values ​​of other pixels within a certain window range, and the pixel with the largest R value is retained.

[0095] S204. Set a threshold and mark pixels with R values ​​greater than the threshold as corner points.

[0096] S3. Select the linear positions to be marked on the chessboard, obtain the start and end points of the positions, and calculate the pixel equivalent of the chessboard where the two points are located.

[0097] S301. Based on the required horizontal and vertical calibration mapping relationship, calculate the pixel distance of the fitted straight line from the corner point where the lines connecting the starting point and the ending point intersect.

[0098] Please see Figure 5 Suppose we need to calibrate the linear positions between the starting point (x'1, y'1) and the ending point (x'2, y'2), and the linear positions between the starting point (x'3, y'3) and the ending point (x'4, y'4). ij ,y ij (i, j = 1, 2, 3, 4) is the point of departure (x') i ,y' i (i = 1, 2, 3, 4) The coordinates of the four nearest chessboard corner points are used to find the coordinates of the chessboard corner points (x, y, y). 13 ,y 13 ) and (x 14 ,y 14 ), (x 21 ,y 21 ) and (x 22 ,y 22 ), (x 31 ,y 31 ) and (x 33 ,y 33 ), (x 42 ,y 42 ) and (x 44 ,y 44 Fit the horizontal lines L1 and L2 and the vertical lines L3 and L4 respectively, and find the point (x'). i ,y' i The pixel distance d from the line i (i=1, 2, 3, 4), such as Figure 5 As shown in (a).

[0099] S302. Calculate the pixel equivalents in the cells containing the start and end points respectively.

[0100] Use the departure point (x' i ,y' i Fit the line Li' (i = 1, 2, 3, 4) to the two unfitted corner points among the four nearest chessboard corner points, as shown below. Figure 5 As shown in (b).

[0101] Find the distance d' from the point to the line. i (i = 1, 2, 3, 4), point (x') i ,y' i The pixel distance d of the square where the ) is located ti for:

[0102] d ti=d' i +d i (3)

[0103] The pixel equivalent d within this cell ni for:

[0104]

[0105] Where M is the actual size of each chessboard square.

[0106] S4. Calculate the mapping relationship between pixel coordinates and world coordinates of the linear position between the start and end points using pixel equivalents.

[0107] S401. Calculate the actual distance of the fitted straight line to the corner point where the lines connecting the starting point and the ending point intersect.

[0108] Given the pixel equivalents of the starting and ending points within their respective grid cells, then point (x') i ,y' i The actual size d of Li (i = 1, 2, 3, 4) mi for:

[0109] d mi =d i ×d ni (5)

[0110] S402. Based on the actual distance, obtain the mapping relationship between the pixel coordinates of the linear position and the world coordinates.

[0111] The number of checkerboard squares between the two fitted lines at their corners is N, therefore the actual distance d between the two lines is... st for:

[0112] d st =N t ×M(t=1、2) (6)

[0113] Based on the sum of the actual distance to the fitted line between the two corner points and the distance from the point to the line, the mapping relationship h between the pixel distance and the actual distance between the starting and ending points is obtained as follows:

[0114]

[0115] Where, d mi For point (x') i ,y' i The actual dimensions of Li (i = 1, 2, 3, 4) are given by d. st d represents the actual distance between the fitted lines of the two corner points in the horizontal and vertical directions. i 'For point (x)' i ,y' i The distance d from the line Li' to the line Li'i For point (x) i ,y i The distance N from line Li t M represents the number of checkerboard squares between two corner points fitted by straight lines, where M is the actual size of each checkerboard square.

[0116] In another embodiment of the present invention, a calibration system is provided, which can be used to implement the above calibration method. Specifically, the calibration system includes an acquisition module, a detection module, a calculation module, and a calibration module.

[0117] The acquisition module simultaneously acquires chessboard images from both the left and right sides of the calibration board, and stitches the acquired images together to obtain a panoramic image of the chessboard.

[0118] The detection module uses the Harris corner detection method to detect the corner positions in the chessboard panoramic image obtained by the acquisition module.

[0119] The calculation module selects the linear positions to be marked on the chessboard grid based on the corner positions in the chessboard grid panoramic image obtained by the detection module, obtains the start and end points of the positions, and calculates the pixel equivalent of the chessboard grid where the start and end points are located.

[0120] The calibration module uses the pixel equivalent obtained from the calculation module to calculate the mapping relationship between the pixel coordinates and world coordinates of the linear position between the starting and ending points, and completes the calibration.

[0121] In another embodiment of the present invention, a terminal device is provided, comprising a processor and a memory. The memory stores a computer program, which includes program instructions. The processor executes the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions to implement a corresponding method flow or corresponding function. The processor described in this embodiment of the present invention can be used for the operation of a calibration method, including:

[0122] Images of the checkerboard on the calibration board are simultaneously acquired from both the left and right directions and then stitched together to obtain a panoramic checkerboard image. The Harris corner detection method is used to detect the corner positions in the panoramic checkerboard image. Based on the corner positions in the panoramic checkerboard image, the linear positions to be calibrated on the checkerboard are selected, and the start and end points of the positions are obtained. The pixel equivalents of the checkerboard squares where the start and end points are located are calculated. The pixel equivalents are used to calculate the mapping relationship between the pixel coordinates of the linear positions between the start and end points and the world coordinates to complete the calibration.

[0123] Please see Figure 6 The terminal device is a computer device. In this embodiment, the computer device 60 includes a processor 61, a memory 62, and a computer program 63 stored in the memory 62 and executable on the processor 61. When executed by the processor 61, the computer program 63 implements the fluid composition calculation method in the reservoir stimulation wellbore of this embodiment. To avoid repetition, these details are not elaborated here. Alternatively, when executed by the processor 61, the computer program 63 implements the functions of each model / unit in the calibration system of this embodiment. To avoid repetition, these details are not elaborated here.

[0124] Computer device 60 can be a desktop computer, laptop, handheld computer, cloud server, or other computing device. Computer device 60 may include, but is not limited to, a processor 61 and a memory 62. Those skilled in the art will understand that... Figure 6 This is merely an example of computer device 60 and does not constitute a limitation on computer device 60. It may include more or fewer components than shown, or combine certain components, or different components. For example, computer device may also include input / output devices, network access devices, buses, etc.

[0125] The processor 61 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0126] The memory 62 can be an internal storage unit of the computer device 60, such as a hard disk or RAM of the computer device 60. The memory 62 can also be an external storage device of the computer device 60, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. equipped on the computer device 60.

[0127] Furthermore, the memory 62 may include both internal storage units of the computer device 60 and external storage devices. The memory 62 is used to store computer programs and other programs and data required by the computer device. The memory 62 can also be used to temporarily store data that has been output or will be output.

[0128] Please see Figure 7 The terminal device is a chip. In this embodiment, the chip 600 includes a processor 622, which may be one or more, and a memory 632 for storing computer programs executable by the processor 622. The computer program stored in the memory 632 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processor 622 may be configured to execute the computer program to perform the calibration method described above.

[0129] Additionally, chip 600 may also include a power supply component 626 and a communication component 650. The power supply component 626 can be configured to perform power management of chip 600, and the communication component 650 can be configured to enable communication of chip 600, such as wired or wireless communication. Furthermore, chip 600 may also include an input / output (I / O) interface 658. Chip 600 can operate on an operating system stored in memory 632.

[0130] In another embodiment of the present invention, a storage medium is also provided, specifically a computer-readable storage medium (memory). This computer-readable storage medium is a memory device in a terminal device used to store programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the terminal device and extended storage media supported by the terminal device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, this storage space also stores one or more instructions suitable for loading and execution by a processor. These instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device.

[0131] One or more instructions stored in a computer-readable storage medium can be loaded and executed by a processor to implement the corresponding steps of the calibration method in the above embodiments; one or more instructions in the computer-readable storage medium are loaded and executed by the processor to perform the following steps:

[0132] Images of the checkerboard on the calibration board are simultaneously acquired from both the left and right directions and then stitched together to obtain a panoramic checkerboard image. The Harris corner detection method is used to detect the corner positions in the panoramic checkerboard image. Based on the corner positions in the panoramic checkerboard image, the linear positions to be calibrated on the checkerboard are selected, and the start and end points of the positions are obtained. The pixel equivalents of the checkerboard squares where the start and end points are located are calculated. The pixel equivalents are used to calculate the mapping relationship between the pixel coordinates of the linear positions between the start and end points and the world coordinates to complete the calibration.

[0133] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0134] Example

[0135] Step 1: Build a vision measurement system for motor vehicles using a 20-megapixel ImagingSource camera and a 5-megapixel HGS industrial lens. Select objects with certain characteristics as the subject, use both cameras to photograph the background, stitch the captured images together, and obtain the stitching parameters. For example... Figure 8 As shown, where Figure 8 (a) The background image captured by the left camera. Figure 8 (b) The background image is captured by the right camera. Figure 8 (c) is the stitched background image.

[0136] Step 2: Based on the dimensions of the vehicle being tested, select a 1600×1060 (mm) checkerboard calibration board with a 50×50 (mm) grid as the calibration object, and use the Harris corner detection algorithm to detect the corner positions in the image. For example... Figure 9 As shown, where Figure 9(a) A calibration image taken by the left camera. Figure 9 (b) Take a calibration image for the right camera. Figure 9 (c) is the stitched calibration image.

[0137] Step 3: Capture and stitch together images of the vehicle under test using the left and right cameras, such as... Figure 10 As shown, where Figure 10 (a) Image of the vehicle being tested taken by the left camera. Figure 10 (b) The image of the vehicle under test is taken by the right camera. Figure 10 (c) is the stitched image of the tested motor vehicle.

[0138] Step 4: Obtain the vehicle's edge contour through image processing and extract the outermost edge points of the vehicle. For example... Figure 11 As shown in the figure, the longitudinal calibration start point 1 (3571, 818), the longitudinal calibration end point 2 (2933, 3235), the lateral calibration start point 3 (5811, 2625), and the lateral calibration start point 4 (2481, 1630) are marked.

[0139] Step 5: Using the outermost edge point as the linear position to be calibrated, obtain the start and end points of the position, and calculate the pixel equivalent of the square where the start and end points are located. The specific steps are as follows:

[0140] Based on the required horizontal and vertical calibration mapping relationships, the pixel distances to the corner points where the lines connecting the starting and ending points intersect are calculated, and the fitted lines are obtained. The coordinates of the four chessboard corner points closest to the vertical calibration starting point 1 are obtained. Using the corner point coordinates whose ordinates are greater than those of the vertical calibration starting point 1, a horizontal line L1 is fitted: y = -0.01319x + 974.487. The pixel distance from the vertical calibration starting point 1 to the line L1 is calculated as d1 = 109.362 pixels.

[0141] Similarly, by fitting the horizontal line L2 with the corner point whose ordinate is less than the vertical calibration endpoint 2: y = -0.02356x + 2927.46, the pixel distance from the vertical calibration endpoint 2 to the line L2 is calculated as d2 = 376.536 pixels.

[0142] Using the corner points whose x-coordinates are less than the horizontal calibration starting point 3, fit the vertical line L3: y = -326.981x + 189728, and calculate the pixel distance from the horizontal calibration starting point 3 to the line L3 as d3 = 16.6156 pixels;

[0143] Using the corner points whose x-coordinates are greater than the horizontal calibration starting point 4, a vertical line L4 is fitted: y = 41.9643x - 109885. The pixel distance from the horizontal calibration starting point 4 to the line L4 is calculated to be d4 = 176.328 pixels. Figure 12 As shown.

[0144] Calculate the pixel equivalents within the starting and ending grids respectively. Vertically define the pixel distance d of the grid containing starting point 1. t1 =118.644 pixels, the pixel equivalent d within this checkerboard is... n1 = 0.421mm / pixel; pixel distance d of the checkerboard grid where the vertical calibration endpoint 2 is located. t2 =119.02 pixels, the pixel equivalent d within this checkerboard square n2 = 0.42mm / pixel; pixel distance d of the checkerboard grid where the horizontal calibration starting point 3 is located. t3 =120.49 pixels, the pixel equivalent d within this checkerboard is... n3 = 0.415mm / pixel; pixel distance d of the checkerboard grid where the horizontal calibration endpoint 4 is located. t2 =119.733 pixels, the pixel equivalent d within this checkerboard is... n2 = 0.418mm / pixel.

[0145] Step 6: Calculate the mapping relationship between pixel coordinates and world coordinates of the linear position between the starting and ending points using pixel equivalents, thereby obtaining the dimensions of the measured motor vehicle. The specific steps are as follows:

[0146] Calculate the actual distance to the fitted straight line at the corner point where the lines connecting the starting and ending points intersect. Vertically calibrate the actual dimension d from starting point 1 to line L1. m1 = 46.0883mm, the actual dimension d from the longitudinal calibration endpoint 2 to the straight line L2 m2 =156.995mm, the actual dimension d from the horizontal calibration starting point 3 to the straight line L3. m3 = 6.895mm, the actual dimension d from the lateral calibration endpoint 4 to the straight line L4. m4 =73.6336mm.

[0147] Based on the actual distance, the mapping relationship between the pixel coordinates of the linear position and the world coordinates is obtained. The number of checkerboard squares between the starting and ending points of the vertical calibration is 16, therefore the actual distance d between the two lines is... s1 =800mm; the number of checkerboard squares between the horizontally marked starting and ending points is 26, so the actual distance d between the two straight lines is... s2 =1300mm.

[0148] The height of the motor vehicle can be calculated as h = d by summing the actual distance between the fitted lines at the two corner points and the distance from the points to the lines. s1 +d m1 +d m2 =1003.0833mm, vehicle length is w=d s2 +d m3 +d m4=1380.5286mm, such as Figure 13 As shown.

[0149] The measurement results of the present invention are compared with those of the single-arm three-dimensional measuring scribing machine, and the results are shown in Table 1.

[0150] Table 1. Comparison of measurement results between the single-arm three-dimensional measuring and marking machine and the present invention.

[0151]

[0152] In summary, this invention provides a calibration method, system, chip, and device. By measuring a motor vehicle, the linear position to be calibrated is determined, along with the starting and ending points. The distances from two points to the coordinates of the checkerboard corner points in the horizontal and vertical directions are fitted to straight lines. The number of complete checkerboard squares between the two points is calculated, and combined with the actual size of the checkerboard squares, the mapping relationship between the pixel coordinate system and the world coordinate system is obtained, thus calculating the size of the motor vehicle. This invention can perform visual measurements on large objects, overcoming the technical problem of inaccurate calibration results caused by stitching errors during the stitching process of dual or even multi-camera systems. The calibration process is simple, eliminating the need to consider deformation issues caused by image stitching, and further simplifying the calibration process, thus enhancing the effectiveness of the calibration method after multi-camera stitching.

[0153] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0154] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0155] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0156] In the embodiments provided by this invention, it should be understood that the disclosed devices / terminals and methods can be implemented in other ways. For example, the device / terminal embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0157] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0158] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0159] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random-access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0160] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0161] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0162] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0163] The above content is only for illustrating the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made to the technical solution based on the technical concept proposed in this invention shall fall within the scope of protection of the claims of this invention.

Claims

1. A calibration method characterized by, Includes the following steps: S1. Simultaneously acquire chessboard images from both the left and right sides of the calibration board, and stitch the acquired images together to obtain a panoramic image of the chessboard. S2. Use the Harris corner detection method to detect the corner positions in the checkerboard panoramic image obtained in step S1. S3. Based on the corner positions in the panoramic image of the chessboard obtained in step S2, select the linear positions to be marked on the chessboard, obtain the start and end points of the positions, and calculate the pixel equivalent of the chessboard where the start and end points are located, specifically: According to the calibration mapping relationship of the horizontal and vertical directions, the intersection angle point fitting straight line of the position start point and the end point to the two-point connecting line is calculated pixel distance, is calculated as follows: Using off-point The straight line fitted by the two corner points of the recent four chessboard corner points which are located at the intersection of the straight line fitted by the two corner points of the position start point and the end point is , the distance of the point to the straight line is , the straight line fitted by the two corner points which are not fitted is , the distance of the point to the straight line is , and the pixel distance of the chess grid where the point is located is : Pixel equivalent within this cell for: in, The actual dimensions of each chessboard square; Calculate the pixel equivalent of the starting and ending squares respectively; S4. Using the pixel equivalent obtained in step S3, calculate the mapping relationship between the pixel coordinates and world coordinates of the linear position between the starting point and the ending point to complete the calibration, specifically as follows: Calculate the actual distance to the fitted straight line at the corner point where the lines connecting the starting and ending points intersect; based on the actual distance, obtain the mapping relationship between the pixel coordinates of the linear position and the world coordinates. Specifically: in, For point Fit a straight line to the angle point where the lines connecting the starting point and the ending point intersect. The actual size, , The actual distance between the fitted lines at the two corner points in the horizontal and vertical directions. For point to the straight line distance, For point Fit a straight line to the angle point where the lines connecting the starting point and the ending point intersect. distance, The number of checkerboard squares between two corner points is determined by fitting a straight line to them. The actual size of each chessboard square.

2. The calibration method according to claim 1, characterized in that, Step S1 is as follows: S101. Use the SURF algorithm to extract feature points from the two images to be stitched together; S102. Use the nearest neighbor and second nearest neighbor methods to save the confidence of the two optimal matching points; S103. Sort the matching points in the two images, save the matching points of the images with high confidence into the same set, delete the matching points of the images with low confidence, and obtain the image sequence that can be correctly matched. Merge all matching points with confidence higher than the threshold into one set. S104. Roughly estimate the parameters of the two images to be stitched together and calculate the rotation matrix. Use bundle adjustment to estimate the rotation matrix. S105. Use a rotation matrix to correct the waveform of the two images to be stitched together, and then stitch the images together.

3. The calibration method according to claim 1, characterized in that, Step S2 is as follows: S201, Calculate the image pixel points in direction and gradient in direction and The image is then filtered for noise using a Gaussian filter function, and a structure tensor is constructed. ; S202. Calculate the corner response values ​​of all pixels in the original image. ; S203, For each pixel in the image The value is the same as other pixels within the window range. Values ​​are compared, and the maximum value is retained. The value of pixels; S204. Set a threshold value, Pixels with values ​​greater than the threshold are marked as corners.

4. A calibration system, characterized in that, include: The acquisition module simultaneously acquires chessboard images from both the left and right sides of the calibration board, and stitches the acquired images together to obtain a panoramic image of the chessboard. The detection module uses the Harris corner detection method to detect the corner positions in the chessboard panoramic image obtained by the acquisition module. The calculation module selects the linear positions to be marked on the chessboard based on the corner positions in the panoramic image of the chessboard obtained by the detection module, obtains the start and end points of the positions, and calculates the pixel equivalent of the chessboard where the start and end points are located. Specifically: Based on the required horizontal and vertical calibration mapping relationships, calculate the fitting straight lines at the corner points where the lines connecting the starting and ending points intersect. pixel distance, The calculation is as follows: Use the point of departure The line fitted to the two corner points of the nearest four chessboard squares that intersects the lines connecting the starting and ending points is: ,point to the straight line The distance is The lines fitted to the two unfitted corner points are: ,point to the straight line distance ,point pixel distance of the chess square for: Pixel equivalent within this cell for: in, The actual dimensions of each chessboard square; Calculate the pixel equivalent of the starting and ending squares respectively; The calibration module uses the pixel equivalent obtained from the calculation module to calculate the mapping relationship between the pixel coordinates and world coordinates of the linear position between the starting and ending points, and completes the calibration. Specifically: Calculate the actual distance to the fitted straight line at the corner point where the lines connecting the starting and ending points intersect; based on the actual distance, obtain the mapping relationship between the pixel coordinates of the linear position and the world coordinates. Specifically: in, For point Fit a straight line to the angle point where the lines connecting the starting point and the ending point intersect. The actual size, , The actual distance between the fitted lines at the two corner points in the horizontal and vertical directions. For point to the straight line distance, For point Fit a straight line to the angle point where the lines connecting the starting point and the ending point intersect. distance, The number of checkerboard squares between two corner points is determined by fitting a straight line to them. The actual size of each chessboard square.

5. A chip, characterized in that, A memory on which computer programs are stored; A processor for executing the computer program in the memory to implement the steps of the method of claim 1, 2, or 3.

6. An electronic device, characterized in that, Includes the chip as described in claim 5.

Images