Semi-automatic calibration method of intrinsic parameters of machine vision camera

Abstract

The present invention discloses a semi-automatic calibration method for the intrinsic parameters of a machine vision camera. The camera to be calibrated collects multiple initial image data of the calibration object located in different planes, and detects the target object in the image data frame by frame, and evaluates the pixel position calibration result of the current image frame containing the target object. If the calibration result meets the established expected standard, the current image frame without repeated data is saved; in this way, after all the initial image data complete the above evaluation and detection, a batch of calibration samples are obtained, and finally the batch samples are fully calibrated and tested, thereby obtaining the final camera intrinsic parameter calibration conclusion. The present invention effectively solves the problem that the camera intrinsic parameter data cannot be accurately calibrated. It can automatically filter and screen the calibration data that meets the expectations, greatly reducing the calibration workload; it can also automatically update and decide whether the calibration result meets the expectations by adopting the judgment conditions, so as to make the final calibration conclusion more accurate.

Description

Technical Field

[0001] The present invention relates to the field of machine vision technology, and in particular to a semi-automatic calibration method for an intrinsic parameter of a machine vision camera. Background Art

[0002] In the field of autonomous driving, cameras are often used as visual sensors. However, due to differences in manufacturing processes and actual installation methods, the image quality of cameras is often distorted and the pixel position is offset from the theoretical value. Therefore, in actual use, it is necessary to calibrate the internal parameters of the camera to achieve more accurate visual algorithm capabilities.

[0003] However, the current internal parameter calibration method often results in a large gap between the calibration result and the actual value, even though the internal parameter data is calibrated. This leads to large errors in actual use or the use of image data of certain planes for calibration results in poor results in other planes. For example, using a single or a small number of images for calibration will at least cause the following problems:

[0004] 1. The calibration results cannot fully reflect the actual intrinsic parameter level of the camera;

[0005] 2. The calibration results are good only in the calibration plane or some other planes, but the quality is poor in the whole space. Summary of the invention

[0006] In view of the above, the present invention aims to provide a semi-automatic calibration method for machine vision camera intrinsic parameters to solve the above-mentioned technical problems.

[0007] The technical solution adopted by the present invention is as follows:

[0008] The present invention provides a semi-automatic calibration method for an internal reference of a machine vision camera, which includes:

[0009] Using the preset calibration target, multiple calibration data images of different planes are collected by the camera;

[0010] Check frame by frame whether there is a preset detection target in the calibration data image;

[0011] Evaluate the calibration result of the current calibration data image with the detection target;

[0012] If the calibration result meets the evaluation expectation, detecting whether there are other repeated calibration data images based on the current calibration data image;

[0013] Save the current calibration data image without duplicate calibration data;

[0014] After all calibration data images have completed the above frame-by-frame calibration, a calibration sample set is obtained;

[0015] The calibration sample set is calibrated and tested, and the final camera intrinsic parameter calibration conclusion is output.

[0016] In at least one possible implementation manner, acquiring a plurality of calibration data images in different planes includes:

[0017] At the current distance between the calibration target and the camera, move the calibration target so that the calibration target is entirely within the image acquisition field of view of the camera;

[0018] During the movement, images of the calibration target relative to a plurality of preset positions in the image acquisition field of view are collected;

[0019] After changing the distance between the calibration target and the camera, the calibration target is moved again within the image acquisition field of view and a number of images are acquired, and so on, to obtain a plurality of calibration data images.

[0020] In at least one possible implementation, the frame-by-frame detection of whether a preset detection target exists in the calibration data image includes: if it is determined that the detection target exists, performing sub-pixel detection on the current calibration data image to obtain an actual pixel position of the detection target.

[0021] In at least one possible implementation manner, the semi-automatic calibration method further includes: if the calibration result does not meet the evaluation expectations, adjusting the accuracy of sub-pixel detection, and re-calibrating and evaluating the current calibration data image.

[0022] In at least one possible implementation manner, the evaluating the calibration result of the current calibration data image having the detection target includes:

[0023] Evaluate whether the calibration result of the current calibration data image meets expectations according to preset judgment conditions, wherein the judgment conditions include at least one of the following: reprojection error, error between actual imaging principal point and theoretical imaging principal point, error between actual focal length and theoretical focal length, error between actual distortion parameter and theoretical distortion.

[0024] In at least one possible implementation manner, the detecting whether there is another repeated calibration data image based on the current calibration data image includes:

[0025] If it is determined that there is a repeated calibration data image, the current calibration data image is discarded; or, the calibration data image with a relatively poor calibration result between the current calibration data image and the repeated calibration data image is discarded.

[0026] In at least one possible implementation manner, outputting the final camera intrinsic parameter calibration conclusion includes:

[0027] Using the same method as frame-by-frame calibration, all calibration data images in the calibration sample set are comprehensively calibrated;

[0028] After judging that the calibration result meets expectations, the final internal parameter calibration conclusion is directly output;

[0029] After determining that the calibration result does not meet expectations, adjust the calibration detection accuracy and repeat several rounds of calibration. If the calibration result or the number of calibrations or the accuracy adjustment exceeds the corresponding preset limit, it will prompt that the internal reference calibration has failed.

[0030] Compared with the prior art, the main design concept of the present invention is to use a preset calibration object, collect multiple initial image data of different planes through the camera to be calibrated, and detect whether there is a preset detection target in the image data frame by frame, and evaluate the pixel position calibration result of the current image frame with the detection target. If the pixel position calibration result meets the established expected standard, then based on the current image frame, it is judged whether there is image data repeated with it in the collected initial image data, and the current image frame without repeated data is saved; in this way, after all the initial image data complete the above frame-by-frame calibration and detection evaluation, a batch of calibration samples is constructed, and finally the batch samples are subjected to comprehensive calibration detection again, thereby obtaining the final camera intrinsic parameter calibration conclusion. The present invention effectively solves the problem that the camera intrinsic parameter data cannot be accurately calibrated, and can automatically filter and screen the calibration data that meets the expectations, greatly reducing the calibration workload; and can automatically update and decide whether the calibration results meet the expectations by adopting judgment conditions, so as to make the final calibration conclusion more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be further described below with reference to the accompanying drawings, in which:

[0032] Figure 1 A schematic diagram of a semi-automatic calibration method for an intrinsic parameter of a machine vision camera provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0033] Embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be interpreted as limiting the present invention.

[0034] The present invention proposes an embodiment of a semi-automatic calibration method for an internal reference of a machine vision camera. Specifically, Figure 1 shown, including:

[0035] Step S1, using a preset calibration target, collecting multiple calibration data images of different planes through a camera;

[0036] Taking the calibration plate as the calibration target as an example, calibration data is collected from an appropriate preset position away from the camera (in the process, ensure that the entire calibration plate is within the field of view of the camera): the calibration plate is gradually moved from the upper left corner of the camera field of view to the upper right corner of the field of view, and the lateral distance X and longitudinal distance Z between the center of the calibration plate and the center of the camera are estimated when the calibration plate is in the leftmost field of view and the rightmost field of view; then, the calibration plate is moved in the camera field of view to gradually move to the lower left corner and the lower right corner of the camera field of view; then, the calibration plate is moved a certain distance away from the camera, and the above steps are repeated to move the calibration plate at this distance to obtain a number of calibration data within the camera field of view; further, the calibration plate is moved away from the camera again, and calibration data is collected by analogy until the target object on the calibration plate cannot be clearly distinguished in the camera field of view (the distance exceeds the focusing range of the camera); in addition, it can also be pointed out that in actual operation, the calibration plate can be replaced with a calibration target in a natural scene.

[0037] For example, a checkerboard calibration plate is used for calibration data collection. During collection, the entire calibration plate is always within the camera's field of view (that is, when the calibration plate is subsequently moved to collect images, it is necessary to ensure that the camera's field of view is complete). First, move the calibration plate close to the camera and without changing the distance from the camera, so that its upper left corner is at the upper left corner of the image field of view, and record the horizontal distance X1 and vertical distance Z1 from the current center of the calibration plate to the center of the camera; then, slowly move the calibration plate so that its upper right corner is aligned with the upper right corner of the image acquisition field of view, and record the horizontal distance X2 and vertical distance Z2 from the current center of the calibration plate to the center of the camera; then move the calibration plate downward so that the lower edge of the calibration plate is aligned with the lower edge of the image, repeat the above steps, and obtain the calibration data image at this distance; in this way, change the certain distance between the calibration plate and the camera, and repeat the above process, so that after multiple calibrations, multiple sets of calibration images in different planes are obtained.

[0038] Step S2, searching whether there is a detection target in each of the calibration data images;

[0039] The detection target mentioned here may refer to a corner point or a circular image center or a camera calibration detection target in another format. Since it cannot be guaranteed that all calibration materials contain the detection target, before proceeding to subsequent steps, a mature image recognition algorithm may be used to determine whether each frame of calibration data image clearly contains the predetermined detection target. If it is determined that the detection target does not exist, the current frame image may be directly discarded and the next plane calibration data image may be detected. If it is determined that the detection target exists, sub-pixel detection may be performed: Generally speaking, the pixels of the detection target directly acquired may not be accurate enough. Therefore, in this embodiment, it is proposed that sub-pixel detection can be used to obtain a more accurate actual pixel position of the detection target.

[0040] Step S3, evaluating the calibration results for each frame image containing the detection target;

[0041] Each frame of calibration data image represents the imaging situation on a plane. In order to achieve the ideal internal parameter calibration result, it is necessary to ensure that the imaging on each plane has good performance. However, in the actual data acquisition project, many factors such as the movement speed and lighting of the calibration target will affect the performance of the calibration data, such as causing the current image to be blurred and other abnormal conditions. Accordingly, in some preferred embodiments of the present invention, it is proposed that the calibration result of the current calibration data image can be evaluated according to preset judgment conditions. The established evaluation conditions can refer to at least one of the following: reprojection error, the error between the actual imaging principal point and the theoretical imaging principal point, the error between the actual focal length and the theoretical focal length, the error between the actual distortion parameter and the theoretical distortion. That is, by calculating the error between the actual calibration result and the theoretical or standard calibration result, it can be judged whether each frame of the image meets the expectations.

[0042] Step S4: if the calibration result meets the evaluation expectation, then detecting whether there is a repeated calibration data image;

[0043] That is to say, for the images that have passed the evaluation, we can further detect whether there are other calibrated images within the predetermined adjacent pixel threshold. If so, it is determined that the current frame image is repeated. At this time, for the adjacent frame data, there are two options: first, discard the current frame; second, determine which calibration result of the current frame and the existing adjacent frame data is more in line with the expectations, and retain the one that is more in line with the expectations, and discard the one that is relatively unsatisfactory.

[0044] Combined with the preferred embodiment above, if the calibration result is evaluated as not meeting expectations, the sub-pixel detection accuracy is updated, and the current image frame is re-evaluated under the established conditions until the calibration result meets expectations, and then the next frame of calibration data image is calibrated and evaluated.

[0045] The above steps can be exemplified as follows: determine frame by frame whether there are corner points in the collected calibration data image, and the position and number of the corner points must meet the preset values; if the detection target meets the preset value, perform sub-pixel corner point detection (for example, when performing sub-pixel detection, a pixel threshold for sub-pixel corner point detection can be set. If the pixel threshold is greater than the pixel threshold and the sub-pixel corner points that meet the calibration conditions cannot be detected, then it is determined that the current frame does not meet the calibration requirements, and it can be directly discarded and the corner point detection of the next frame of image data is performed) and after the detection is completed, calibrate the current frame and determine whether conditions such as the reprojection error meet the preset conditions. If one or more evaluation conditions are not met, update the accuracy of the sub-pixel corner point detection, and then re-detect the sub-pixel corner points and perform the calibration evaluation steps; if the evaluation passes, determine whether there is calibration data within 30 pixels of the upper left corner and lower right corner of the current frame image. If so, it indicates that the data is repeated, and you can consider discarding the current frame or discarding the relatively worse calibration data of the two duplicates.

[0046] Continuing from the above, step S5, saving the calibration data image without duplicate calibration data;

[0047] Step S6: After all calibration data images are calibrated one by one, a calibration sample set is obtained;

[0048] To elaborate, if the current frame has no adjacent image data, that is, there is no repetition, the current frame will be added to the calibration queue and wait for all calibration data images. After the above calibration evaluation and repeated detection process for the detection target are completed frame by frame, a batch of calibration material samples will be accumulated in the queue, that is, all the calibration data in the queue can be used for comprehensive calibration of the target pixel position.

[0049] Step S7: perform calibration test on the calibration sample set and output the final calibration conclusion.

[0050] Similar to the calibration method of judging a single frame image mentioned above, at this time, the comprehensive results of all image data in the summarized queue can be calibrated and judged, and evaluated whether they meet expectations: if they meet, the final calibration result is directly output; if not, the calibration detection accuracy is adjusted and several rounds of calibration are performed again until the calibration result exceeds the preset calibration detection threshold. If there is still no calibration result that meets the conditions, it is judged as a calibration failure.

[0051] In summary, the main design concept of the present invention is to use a preset calibration object, collect multiple initial image data of different planes through the camera to be calibrated, and detect whether there is a preset detection target in the image data frame by frame, and evaluate the pixel position calibration result of the current image frame with the detection target. If the pixel position calibration result meets the established expected standard, then based on the current image frame, it is judged whether there is image data repeated with it in the collected initial image data, and the current image frame without repeated data is saved; in this way, after all the initial image data complete the above frame-by-frame calibration and detection evaluation, a batch of calibration samples is constructed, and finally the batch samples are subjected to comprehensive calibration detection again, thereby obtaining the final camera intrinsic parameter calibration conclusion. The present invention effectively solves the problem that the camera intrinsic parameter data cannot be accurately calibrated, and can automatically filter and screen the calibration data that meets the expectations, greatly reducing the calibration workload; and can automatically update and decide whether the calibration results meet the expectations by adopting judgment conditions, so as to make the final calibration conclusion more accurate.

[0052] If the expressions expressing orientation are mentioned in the embodiments of the present invention, they are relative concepts based on the embodiments. In addition, "at least one" means one or more, and "plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. Among them, A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can be represented by: a, b, c, a and b, a and c, b and c, or a and b and c, wherein a, b, c can be single or multiple.

[0053] The above describes in detail the structure, features and effects of the present invention based on the embodiments shown in the drawings, but the above is only a preferred embodiment of the present invention. It should be noted that the technical features involved in the above embodiments and their preferred methods can be reasonably combined and matched into a variety of equivalent schemes by those skilled in the art without departing from or changing the design ideas and technical effects of the present invention; therefore, the present invention is not limited to the scope of implementation shown in the drawings, and all changes made in accordance with the concept of the present invention, or modifications to equivalent embodiments with equivalent changes, which still do not exceed the spirit covered by the specification and drawings, should be within the protection scope of the present invention.

Claims

1. A semi-automatic calibration method for machine vision camera intrinsic parameters, characterized in that: include: Using the preset calibration target, multiple calibration data images of different planes are collected by the camera; Check frame by frame whether there is a preset detection target in the calibration data image; Evaluate the calibration result of the current calibration data image with the detection target; If the calibration result meets the evaluation expectation, detecting whether there are other repeated calibration data images based on the current calibration data image; Save the current calibration data image without duplicate calibration data; After all calibration data images have completed the above frame-by-frame calibration, a calibration sample set is obtained; The calibration sample set is calibrated and tested, and the final camera intrinsic parameter calibration conclusion is output.

2. The method for semi-automatic calibration of the intrinsic parameters of a machine vision camera according to claim 1, characterized in that: The collecting of multiple calibration data images in different planes comprises: At the current distance between the calibration target and the camera, move the calibration target so that the calibration target is entirely within the image acquisition field of view of the camera; During the movement, images of the calibration target relative to a plurality of preset positions in the image acquisition field of view are collected; After changing the distance between the calibration target and the camera, the calibration target is moved again within the image acquisition field of view and a number of images are acquired, and so on, to obtain a plurality of calibration data images.

3. The method for semi-automatic calibration of the intrinsic parameters of a machine vision camera according to claim 1, characterized in that: The frame-by-frame detection of whether there is a preset detection target in the calibration data image includes: if it is determined that the detection target exists, performing sub-pixel detection on the current calibration data image to obtain an actual pixel position of the detection target.

4. The method for semi-automatic calibration of the intrinsic parameters of a machine vision camera according to claim 3, characterized in that: The semi-automatic calibration method further includes: if the calibration result does not meet the evaluation expectation, adjusting the accuracy of sub-pixel detection, and re-calibrating and evaluating the current calibration data image.

5. The method for semi-automatic calibration of the intrinsic parameters of a machine vision camera according to claim 1, characterized in that: The evaluating of the calibration result of the current calibration data image having the detection target comprises: Evaluate whether the calibration result of the current calibration data image meets expectations according to preset judgment conditions, wherein the judgment conditions include at least one of the following: reprojection error, error between actual imaging principal point and theoretical imaging principal point, error between actual focal length and theoretical focal length, error between actual distortion parameter and theoretical distortion.

6. The method for semi-automatic calibration of the intrinsic parameters of a machine vision camera according to claim 1, characterized in that: The detecting whether there are other repeated calibration data images based on the current calibration data image includes: If it is determined that there is a repeated calibration data image, the current calibration data image is discarded; or, the calibration data image with a relatively poor calibration result between the current calibration data image and the repeated calibration data image is discarded.

7. The method for semi-automatic calibration of the intrinsic parameters of a machine vision camera according to any one of claims 1 to 6, characterized in that: The final camera intrinsic calibration conclusion output includes: Using the same method as frame-by-frame calibration, all calibration data images in the calibration sample set are comprehensively calibrated; After judging that the calibration result meets expectations, the final internal parameter calibration conclusion is directly output; After determining that the calibration result does not meet expectations, adjust the calibration detection accuracy and repeat several rounds of calibration. If the calibration result or the number of calibrations or the accuracy adjustment exceeds the corresponding preset limit, it will prompt that the internal reference calibration has failed.

Images