A high-precision camera calibration method, device and storage medium for micro field of view

Abstract

The present application discloses a high-precision camera calibration method, device and storage medium for a micro field of view, wherein the method includes: allowing a calibration plate to perform posture transformation movement in a focus plane; using a camera to shoot the calibration plate after each posture transformation movement, obtaining the pixel coordinates of the feature points on the calibration plate, and using the obtained pixel coordinates to fit the camera mathematical model; when the number of transformations of the calibration plate's posture transformation movement reaches a preset requirement, giving an initial value of an intrinsic parameter in the camera's intrinsic parameter matrix, and using the given initial value, obtaining the values of the remaining intrinsic parameters in the intrinsic parameter matrix and the values of all extrinsic parameters in the camera's extrinsic parameter matrix; calculating the current focal length value under the camera mathematical model through the estimated intrinsic and extrinsic parameter matrix of the camera; when the difference between the current focal length value and the preset focal length value is within a preset error range, substituting the given initial value into a nonlinear optimization equation, and after iterating to a preset accuracy range, obtaining all intrinsic and extrinsic parameters of the camera.

Description

Technical Field

[0001] The present application relates to the field of camera calibration technology, and in particular to a high-precision camera calibration method, device, and storage medium for a micro field of view. Background Art

[0002] The commonly used Zhang calibration method requires the calibration plate to perform at least three x-axis or y-axis rotations within the focus range to effectively initialize the camera's intrinsic parameters. However, in a microscopic field of view, where the depth of field is only a few millimeters, the calibration plate cannot perform sufficient pose movement within the limited space to facilitate the camera's intrinsic parameter initialization, causing the subsequent nonlinear optimization process to fall into a local optimum. Therefore, the Zhang calibration method cannot directly meet the calibration needs of microscopic fields of view.

[0003] To address the challenge of calibrating a limited depth of field in a microscopic field of view, Song Zhang's team has been using deblurring algorithms to increase the depth of field. This has led to a series of deblurring and depth-of-field enhancement methods to alleviate the problem of too small a depth of field in a microscopic field of view. However, deblurring algorithms can only mitigate the loss of calibration accuracy when the calibration plate is out of focus; they cannot achieve the same calibration accuracy as when it is not out of focus.

[0004] Xiang Peng's team used a high-precision motion platform to constrain the calibration plate to a specific motion within a limited depth of field. This involves two absolutely parallel planar motions and one relatively tilted, small-angle rotation. This approach leverages the principle of three points being collinear to increase the number of constraint equations, thus addressing the associated calibration challenges at a small depth of field. However, this method requires a high-precision motion platform, which is both costly and difficult to operate.

[0005] Currently, no effective solution has been proposed to the technical problems of low accuracy, low reliability, high cost and complex operation in the above-mentioned prior art camera lens calibration methods under micro field of view. Summary of the Invention

[0006] Embodiments of the present invention provide a high-precision camera calibration method, device, and storage medium for a micro field of view, to at least solve the technical problems of low accuracy, low reliability, high cost, and complex operation in the prior art camera lens calibration methods under a micro field of view.

[0007] According to one aspect of an embodiment of the present invention, a high-precision camera calibration method for a micro field of view is provided, comprising: allowing a calibration plate to perform posture transformation movement in a focus plane; using a camera to photograph the calibration plate after each posture transformation movement, obtaining pixel coordinates of feature points on the calibration plate, and using the obtained pixel coordinates to fit a camera mathematical model; when the number of transformations of the calibration plate's posture transformation movement reaches a preset requirement, giving an initial value of an intrinsic parameter in the camera's intrinsic parameter matrix, and using the given initial value, obtaining the values of the remaining intrinsic parameters in the intrinsic parameter matrix and the values of all extrinsic parameters in the camera's extrinsic parameter matrix; calculating the current focal length value under the camera mathematical model through the estimated intrinsic and extrinsic parameter matrices of the camera; when the difference between the current focal length value and the preset focal length value is within a preset error range, substituting the initial value into a nonlinear optimization equation, and after iterating to a preset accuracy range, obtaining all intrinsic and extrinsic parameters of the camera.

[0008] Optionally, before allowing the calibration plate to perform posture transformation movement in the focal plane, the method further includes: tilting and fixing the camera, and adjusting the focal plane of the camera to be on a horizontal plane.

[0009] Optionally, allowing the calibration plate to perform a posture transformation movement in the focus plane includes: allowing the calibration plate to perform a posture transformation movement in the focus xy plane, wherein the posture transformation movement includes a translation in the xy plane and a rotation along the z-axis.

[0010] Optionally, before giving an initial value of an intrinsic parameter in the intrinsic parameter matrix of the camera, the method also includes: determining whether the number of transformations of the calibration plate's posture transformation movement meets the preset requirements; when the number of transformations of the calibration plate's posture transformation movement does not meet the preset requirements, continuing to allow the calibration plate to perform posture transformation movement in the focus plane until the number of transformations of the calibration plate's posture transformation movement meets the preset requirements.

[0011] Optionally, an initial value of an intrinsic parameter in the intrinsic parameter matrix of the camera is given, and the given initial value is used to obtain the values of the remaining intrinsic parameters in the intrinsic parameter matrix and the values of all extrinsic parameters in the extrinsic parameter matrix of the camera, including: giving an initial value of the image distance or the x-axis coordinate of the optical axis in the intrinsic parameter matrix of the camera; using the Zhang calibration method, and using the given initial value, obtaining the values of the remaining intrinsic parameters in the intrinsic parameter matrix and the values of all extrinsic parameters in the extrinsic parameter matrix of the camera.

[0012] Optionally, the current focal length value under the camera mathematical model is calculated by estimating the intrinsic and extrinsic parameter matrix of the camera, including: calculating the object distance under the camera mathematical model by estimating the intrinsic and extrinsic parameter matrix of the camera; calculating the image distance under the camera mathematical model based on the object distance while considering the influence of the Sham angle; and calculating the current focal length value under the camera mathematical model according to the object distance and the image distance using the corresponding lens formula.

[0013] Optionally, before the initial value is substituted into the nonlinear optimization equation, the method further includes: determining whether the difference between the current focal length value and the preset focal length value is within a preset error range; when the difference between the current focal length value and the preset focal length value is not within the preset error range, updating the given initial value according to the difference between the current focal length value and the preset focal length value, and recalculating the current focal length value under the camera mathematical model using the updated initial value until the difference between the current focal length value and the preset focal length value is within the preset error range.

[0014] According to another aspect of an embodiment of the present invention, a storage medium is further provided. The storage medium includes a stored program, wherein when the program is running, a processor executes any one of the above methods.

[0015] According to another aspect of an embodiment of the present invention, a high-precision camera calibration device for a micro field of view is also provided, including: a posture transformation module for allowing a calibration plate to perform posture transformation movement within a focal plane; a pixel coordinate acquisition module for using a camera to shoot the calibration plate after each posture transformation movement, obtain the pixel coordinates of feature points on the calibration plate, and use the acquired pixel coordinates to fit the camera mathematical model; an internal and external parameter initialization module for giving an internal parameter initial value in the camera's internal parameter matrix when the number of transformations of the calibration plate's posture transformation movement reaches a preset requirement, and using the given initial value to obtain the values of the remaining internal parameters in the internal parameter matrix and the values of all external parameters in the camera's external parameter matrix; a current focal length value determination module for calculating the current focal length value under the camera mathematical model using the estimated internal and external parameter matrix of the camera; an internal and external parameter determination module for bringing the initial value into the nonlinear optimization equation when the difference between the current focal length value and the preset focal length value is within a preset error range, and after iterating to a preset accuracy range, obtaining all internal and external parameters of the camera.

[0016] According to another aspect of an embodiment of the present invention, a high-precision camera calibration device for a micro field of view is also provided, including: a processor; and a memory connected to the processor, for providing the processor with instructions for processing the following processing steps: allowing the calibration plate to perform posture transformation movement in the focus plane; using a camera to shoot the calibration plate after each posture transformation movement, obtaining the pixel coordinates of the feature points on the calibration plate, and using the obtained pixel coordinates to fit the camera mathematical model; when the number of transformations of the calibration plate performing posture transformation movement reaches a preset requirement, giving an initial value of an intrinsic parameter in the intrinsic parameter matrix of the camera, and using the given initial value, obtaining the values of the remaining intrinsic parameters in the intrinsic parameter matrix and the values of all extrinsic parameters in the extrinsic parameter matrix of the camera; calculating the current focal length value under the camera mathematical model through the estimated intrinsic and extrinsic parameter matrices of the camera; when the difference between the current focal length value and the preset focal length value is within a preset error range, substituting the initial value into the nonlinear optimization equation, and after iterating to the preset accuracy range, obtaining all internal and external parameters of the camera.

[0017] In an embodiment of the present invention, a calibration plate is first subjected to a posture transformation movement in a focal plane, and then a camera is used to photograph the calibration plate after each posture transformation movement, and the pixel coordinates of the characteristic points on the calibration plate are obtained. The obtained pixel coordinates are then used to fit the camera mathematical model. Secondly, when the number of transformations of the calibration plate in posture transformation movement reaches a preset requirement, an initial value of an intrinsic parameter in the intrinsic parameter matrix of the camera is given, and the given initial value is used to obtain the values of the remaining intrinsic parameters in the intrinsic parameter matrix and the values of all extrinsic parameters in the extrinsic parameter matrix of the camera. Then, the current focal length value under the camera mathematical model is calculated by estimating the intrinsic and extrinsic parameter matrices of the camera. Finally, when the difference between the current focal length value and the preset focal length value is within a preset error range, the initial value is substituted into the nonlinear optimization equation, and after iterating to a preset accuracy range, all the intrinsic and extrinsic parameters of the camera are obtained. The calibration method proposed by the present invention is extremely simple, and only requires moving the calibration plate several times in the focal plane, which is very easy to implement in engineering. Because the calibration plate operates only within the focal plane, all calibration operations in this invention are performed with the camera in full focus, eliminating the loss of feature point accuracy in out-of-focus conditions. This allows for reliable calibration in a micro-field of view, theoretically and practically providing highly reliable and accurate calibration results. This addresses the existing technical issues of low accuracy, low reliability, high cost, and complex operation in micro-field camera lens calibration methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0019] Figure 1 is a hardware structure block diagram of a computing device for implementing the method according to embodiment 1 of the present invention;

[0020] Figure 2 2 is a flow chart of a high-precision camera calibration method for a micro field of view according to the first aspect of Embodiment 1 of the present invention;

[0021] Figure 3 2 is a schematic diagram of the optical principle of the oblique projection pinhole camera according to Example 1 of the present invention;

[0022] Figure 4 2 is a schematic diagram of the overall process of the high-precision camera calibration method for a micro field of view according to Embodiment 1 of the present invention;

[0023] Figure 5 is a schematic diagram of a high-precision camera calibration device for a micro field of view according to embodiment 2 of the present invention; and

[0024] Figure 6 3 is a schematic diagram of a high-precision camera calibration device for a micro field of view according to Example 3 of the present invention. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the technical solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0026] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices. Example 1

[0027] According to this embodiment, an embodiment of a high-precision camera calibration method for a micro field of view is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0028] The method embodiment provided in this embodiment can be executed in a server or similar computing device. Figure 1 FIG1 shows a hardware structure block diagram of a computing device for implementing a high-precision camera calibration method for a micro field of view. Figure 1As shown, the computing device may include one or more processors (the processor may include but is not limited to a microprocessor MCU or a programmable logic device FPGA, etc.), a memory for storing data, and a transmission device for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a power supply and / or a camera. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0029] It should be noted that the one or more processors and / or other data processing circuits described above may generally be referred to herein as "data processing circuitry." The data processing circuitry may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuitry may be a single, independent processing module, or may be fully or partially integrated into any of the other components of the computing device. As described in the embodiments of the present invention, the data processing circuitry functions as a processor control (e.g., the selection of a variable resistor terminal path connected to an interface).

[0030] The memory can be used to store software programs and modules for application software, such as the program instructions / data storage device corresponding to the high-precision camera calibration method for micro-fields of view described in embodiments of the present invention. The processor executes the software programs and modules stored in the memory to perform various functional applications and data processing, thereby implementing the high-precision camera calibration method for micro-fields of view described in the aforementioned application. The memory can include high-speed random access memory (RAM) and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some embodiments, the memory can further include memory located remotely from the processor, which can be connected to the computing device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0031] The transmission device is used to receive or send data via a network. Specific examples of the aforementioned network may include a wireless network provided by a communications provider of the computing device. In one embodiment, the transmission device includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission device may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0032] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computing device.

[0033] It should be noted that, in some optional embodiments, the above Figure 1 The computing device shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of both hardware and software elements. Figure 1 This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the computing device described above.

[0034] In the above operating environment, according to the first aspect of this embodiment, a high-precision camera calibration method for a micro field of view is provided. Figure 2 A schematic diagram of the process is shown in FIG. Figure 2 As shown, the method includes:

[0035] S202: Allow the calibration plate to perform posture transformation movement in the focus plane.

[0036] In microscopic 3D reconstruction scenarios, the calibration of a multi-device 3D stereo camera system is crucial, directly impacting the accuracy of subsequent point cloud reconstruction. While stereo camera system calibration is based on the calibration of individual cameras, the present invention proposes a new, more accurate and reliable calibration method for oblique projection pinhole cameras (including but not limited to those employing a Sham angle). While the present invention will systematically illustrate the camera structure after the Sham angle has been introduced, the proposed calibration method is not limited to pinhole imaging cameras employing a Sham angle.

[0037] The scenario of the embodiment of the present invention is a method for calibrating the lens of an oblique projection camera under a micro field of view. Its field of view is generally less than 100mm*100mm, and the depth of field is also within a few mm. Among them, the field of view refers to the actual size of the image that can be captured by the camera when taking pictures. In the 100*100mm field of view mentioned here, if a 5-megapixel camera is used, the actual resolution of each pixel is about 20um. If the pixel resolution is less than 1um, optical imaging cannot be performed due to the diffraction problem of light. At this time, the field of view is about 2*2mm. So to be precise, it should be classified according to the resolution of a single pixel. Generally speaking, it is a field of view of 100*100mm to 2*2mm, under the premise of using a 5-megapixel camera.

[0038] In the embodiment of the present invention, Figure 3 As shown, the camera tilt angle images the object on the plane. Since the depth of field of the camera is only a few mm, the present invention adopts the calibration method of a plane calibration plate, which results in the calibration plate being able to move only within a limited depth of field. Without losing generality, we take the camera rotating around the y-axis to shoot a horizontal plane as an example to introduce the entire calibration principle and process. At this time, the intrinsic parameter matrix of the camera is Where, is the image distance in the x direction, is the offset between the image coordinate origin and the lens principal optical axis in the x direction, is the image distance in the y direction, is the offset between the image coordinate origin and the lens principal optical axis in the y direction.

[0039] It should be noted that the calibration principle of this patent is consistent with the camera tilting around any axis. The Y-axis rotation is used as an example here just for the sake of generality.

[0040] Optionally, before allowing the calibration plate to perform posture transformation movement in the focal plane, the method further includes: tilting and fixing the camera, and adjusting the focal plane of the camera to be on a horizontal plane.

[0041] Optionally, allowing the calibration plate to perform a posture transformation movement in the focus plane includes: allowing the calibration plate to perform a posture transformation movement in the focus xy plane, wherein the posture transformation movement includes a translation in the xy plane and a rotation along the z-axis.

[0042] In the embodiment of the present invention, first fix the camera in an inclined position, adjust the camera's focus plane to the horizontal plane, and then follow Figure 4Calibration is performed according to the flowchart shown. In addition, to further increase calibration accuracy, the aperture value can be adjusted to the minimum before camera calibration. After calibration, all internal and external camera parameters are accurately and reliably obtained. Finally, the aperture value is adjusted to the ideal range based on the specific working environment requirements. Typically, the aperture value of industrial lenses is marked with values ranging from 2.8 to 16. During calibration, the aperture is adjusted to the minimum value, that is, 2.8. During operation, it can be adjusted to any value according to the specific working environment requirements. At this time, adjusting the aperture value does not affect the calibration results. In general, the ideal aperture value range in industry is between 8-11, at which time the depth of field is relatively large and sufficient light intake can be guaranteed.

[0043] In the embodiment of the present invention, Figure 4 As shown, the calibration plate is subjected to posture transformation motion in the focused xy plane, including translation in the xy plane and rotation along the z axis.

[0044] S204: Use the camera to shoot the calibration plate after each posture transformation movement, obtain the pixel coordinates of the feature points on the calibration plate, and use the obtained pixel coordinates to fit the camera mathematical model.

[0045] In the embodiment of the present invention, Figure 4 As shown, a camera is used to capture the calibration plate after each pose transformation. Each time a photo is taken to calculate feature points, the pose of the calibration plate needs to change. Feature points include, but are not limited to, the center of a circle or the corners of a checkerboard. Taking corner points as an example, a commonly used corner point calculation method for calibration plates (not limited to the type of calibration plate pattern) is used to obtain the exact pixel coordinates of the corner points on the calibration plate after the pose transformation. These pixel coordinates are used to fit the camera mathematical model. The camera mathematical model is as follows:

[0046] in, The coordinates are the world coordinates of the corner points on the calibration plate, and are also given by the present invention based on the actual distance between the corner points. are the corner coordinates, i.e., the pixel coordinates obtained by the image corner search algorithm of the present invention. A is the camera intrinsic parameter matrix mentioned above. R and T are the rotation matrix and translation matrix of the extrinsic parameter matrix in the camera mathematical model, respectively. is an arbitrary scaling factor that varies for each input point. is the parameter model after the introduction of Sham angle, represents the camera intrinsic correction matrix of the Sham model, Represents the camera extrinsic rotation correction matrix of the Sham angle model.

[0047] S206: When the number of transformations of the calibration plate's posture transformation movement reaches a preset requirement, an initial value of an intrinsic parameter in the camera's intrinsic parameter matrix is given, and the given initial value is used to obtain the values of the remaining intrinsic parameters in the intrinsic parameter matrix and the values of all extrinsic parameters in the camera's extrinsic parameter matrix.

[0048] Optionally, before giving an initial value of an intrinsic parameter in the intrinsic parameter matrix of the camera, the method also includes: determining whether the number of transformations of the calibration plate's posture transformation movement meets the preset requirements; when the number of transformations of the calibration plate's posture transformation movement does not meet the preset requirements, continuing to allow the calibration plate to perform posture transformation movement in the focus plane until the number of transformations of the calibration plate's posture transformation movement meets the preset requirements.

[0049] Optionally, an initial value of an intrinsic parameter in the intrinsic parameter matrix of the camera is given, and the given initial value is used to obtain the values of the remaining intrinsic parameters in the intrinsic parameter matrix and the values of all extrinsic parameters in the extrinsic parameter matrix of the camera, including: giving an initial value of the image distance or the x-axis coordinate of the optical axis in the intrinsic parameter matrix of the camera; using the Zhang calibration method, and using the given initial value, obtaining the values of the remaining intrinsic parameters in the intrinsic parameter matrix and the values of all extrinsic parameters in the extrinsic parameter matrix of the camera.

[0050] In the embodiment of the present invention, Figure 4 As shown, it is determined whether the number of pose changes of the current calibration plate meets the preset requirements. If the number does not meet the preset requirements, the process returns to step S202. If the preset requirements are met, the process proceeds to the next step. The number of pose changes must be at least one (inclusive). The number of pose changes is determined subjectively by humans. Several poses (but at least one pose is required to provide input data for the model) can be calibrated. However, generally, the more poses used for calibration, the more fully the calibration plate covers the field of view, and the higher the accuracy.

[0051] Furthermore, an initial value of an internal parameter in the internal parameter matrix of a given camera is manually given, for example but not limited to, an image distance of a given camera is manually given Or the optical axis x-axis coordinate Then, Zhang’s calibration method is used to obtain the remaining camera intrinsic parameter matrices using the given camera initial values. And the external parameter matrix .in, is the rotation matrix, is the translation matrix.

[0052] S208: Calculate the current focal length value under the camera mathematical model using the estimated camera internal and external parameter matrix.

[0053] Optionally, the current focal length value under the camera mathematical model is calculated by estimating the intrinsic and extrinsic parameter matrix of the camera, including: calculating the object distance under the camera mathematical model by estimating the intrinsic and extrinsic parameter matrix of the camera; calculating the image distance under the camera mathematical model based on the object distance while considering the influence of the Sham angle; and calculating the current focal length value under the camera mathematical model according to the object distance and the image distance using the corresponding lens formula.

[0054] In the embodiment of the present invention, the object distance under the current camera mathematical model is first calculated by the external parameter matrix. Then, the image distance under the current camera mathematical model is calculated taking into account the influence of the Sham angle. Finally, the classic lens formula is used to calculate the object distance under the current camera mathematical model. Get the current focal length value under the camera mathematical model .

[0055] S210: When the difference between the current focal length value and the preset focal length value is within a preset error range, the initial value is substituted into the nonlinear optimization equation, and after iterating to within a preset accuracy range, all internal and external parameters of the camera are obtained.

[0056] Optionally, before the initial value is substituted into the nonlinear optimization equation, the method further includes: determining whether the difference between the current focal length value and the preset focal length value is within a preset error range; when the difference between the current focal length value and the preset focal length value is not within the preset error range, updating the given initial value according to the difference between the current focal length value and the preset focal length value, and recalculating the current focal length value under the camera mathematical model using the updated initial value until the difference between the current focal length value and the preset focal length value is within the preset error range.

[0057] In the embodiment of the present invention, Figure 4 As shown, first determine the current focal length value under the camera mathematical model and compare it with the default focal length value of the lens (wherein, lenses can be classified according to focal length. The currently commonly used classification focal lengths are 8mm, 16mm, 25mm, and 50mm, etc., which are equivalent to the available models). If the error range is within 1%, proceed to step 9. If the error range is greater than 1%, proceed to step 8.

[0058] In step 8, you need to calculate the difference between the default focal length value and the current focal length value, and use the difference to update the image distance of the camera given in step 4. Or the optical axis x-axis coordinate Then proceed to steps 5 to 7 until the difference between the current focal length value and the preset focal length value is within the preset error range.

[0059] Finally, all the obtained camera internal and external parameters are fed into an optimization algorithm (such as the maximum likelihood estimation method in Zhang's calibration method) that takes nonlinear distortion (such as radial distortion, tangential distortion, etc.) into account, and iterated to a preset accuracy range (usually set within 1% of the standard value) to obtain accurate and reliable camera internal and external parameters (that is, all parameters in the camera mathematical model).

[0060] In summary, the present invention has the following beneficial effects:

[0061] (1) The calibration method is extremely simple. It only requires moving the calibration plate several times in the focus plane, which is very easy to implement in engineering.

[0062] (2) Traditional calibration methods require the depth of field to be as large as possible, that is, the aperture value is adjusted to the largest possible value. However, this invention is different from the traditional method. It directly adjusts the aperture value to the minimum value. Only then can the calibration method be obtained.

[0063] (3) For the first time, the lens equation is introduced as a constraint equation and combined with the initialization process to obtain high-quality initialization parameters. The initialization parameters have a very important impact on the calibration nonlinear optimization results.

[0064] (4) Since the calibration plate only moves within the focal plane, all calibration actions are performed under full focus, which can provide highly reliable raw data. Unlike other calibration methods, which require large-scale movement in space and are prone to data collection after defocusing, thus introducing noise into the calibration data set.

[0065] Thus, the high-precision camera calibration method for micro-field of view proposed by the present invention first allows the calibration plate to perform posture transformation movement in the focal plane, then uses the camera to shoot the calibration plate after each posture transformation movement, obtains the pixel coordinates of the feature points on the calibration plate, and uses the obtained pixel coordinates to fit the camera mathematical model. Secondly, when the number of transformations of the calibration plate to perform posture transformation movement reaches the preset requirement, an initial value of an intrinsic parameter in the intrinsic parameter matrix of the camera is given, and the given initial value is used to obtain the values of the remaining intrinsic parameters in the intrinsic parameter matrix and the values of all extrinsic parameters in the extrinsic parameter matrix of the camera. Then, the current focal length value under the camera mathematical model is calculated by estimating the intrinsic and extrinsic parameter matrices of the camera. Finally, when the difference between the current focal length value and the preset focal length value is within the preset error range, the initial value is substituted into the nonlinear optimization equation. After iterating to the preset accuracy range, all the intrinsic and extrinsic parameters of the camera are obtained. The calibration method proposed by the present invention is extremely simple and only requires moving the calibration plate several times in the focal plane, which is very easy to implement in engineering. Because the calibration plate operates only within the focal plane, all calibration operations in this invention are performed with the camera in full focus, eliminating the loss of feature point accuracy in out-of-focus conditions. This allows for reliable calibration in a micro-field of view, theoretically and practically providing highly reliable and accurate calibration results. This addresses the existing technical issues of low accuracy, low reliability, high cost, and complex operation in micro-field camera lens calibration methods.

[0066] It should be noted that for the aforementioned method embodiments, for simplicity of description, they are all expressed as a series of action combinations. However, those skilled in the art should be aware that the present invention is not limited by the order of the actions described, because according to the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0067] Through the description of the above embodiments, those skilled in the art will clearly understand that the methods according to the above embodiments can be implemented using software plus the necessary general-purpose hardware platform. Of course, hardware can also be used, but in many cases the former is a more preferred embodiment. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, or optical disk) and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of the present invention. Example 2

[0068] Figure 5FIG. 5 shows a high-precision camera calibration device 500 for a micro field of view according to this embodiment, which corresponds to the method described in the first aspect of embodiment 1. Figure 5 As shown, the device 500 includes: a posture transformation module 510, which is used to allow the calibration plate to perform posture transformation movement in the focus plane; a pixel coordinate acquisition module 520, which is used to use a camera to shoot the calibration plate after each posture transformation movement, obtain the pixel coordinates of the feature points on the calibration plate, and use the obtained pixel coordinates to fit the camera mathematical model; an internal and external parameter initialization module 530, which is used to give an internal parameter initial value in the camera's internal parameter matrix when the number of transformations of the calibration plate's posture transformation movement reaches a preset requirement, and use the given initial value to obtain the values of the remaining internal parameters in the internal parameter matrix and the values of all external parameters in the camera's external parameter matrix; a current focal length value determination module 540, which is used to calculate the current focal length value under the camera mathematical model by using the estimated internal and external parameter matrix of the camera; an internal and external parameter determination module 550, which is used to bring the initial value into the nonlinear optimization equation when the difference between the current focal length value and the preset focal length value is within a preset error range, and after iterating to a preset accuracy range, obtain all internal and external parameters of the camera.

[0069] Optionally, the device 500 further includes: an aperture value adjustment module, configured to tilt and fix the camera, and adjust the focus plane of the camera to be on a horizontal plane.

[0070] Optionally, the posture transformation module 510 is specifically used to allow the calibration plate to perform posture transformation movement in the focused xy plane, wherein the posture transformation movement includes translation in the xy plane and rotation along the z axis.

[0071] Optionally, the device 500 also includes: a transformation quantity judgment module, used to judge whether the transformation quantity of the calibration plate's posture transformation movement meets the preset requirements; when the transformation quantity of the calibration plate's posture transformation movement does not meet the preset requirements, continue to let the calibration plate perform posture transformation movement in the focusing plane until the transformation quantity of the calibration plate's posture transformation movement meets the preset requirements.

[0072] Optionally, the internal and external parameter initialization module 530 is specifically used to: give an initial value of the image distance or the x-axis coordinate of the optical axis in the intrinsic parameter matrix of the camera; adopt Zhang's calibration method, using the given initial value, to obtain the values of the remaining intrinsic parameters in the intrinsic parameter matrix and the values of all extrinsic parameters in the extrinsic parameter matrix of the camera.

[0073] Optionally, the current focal length value determination module 540 is specifically used to: calculate the object distance under the camera mathematical model by estimating the intrinsic and extrinsic parameter matrices of the camera; calculate the image distance under the camera mathematical model based on the object distance while considering the influence of the Sham angle; and calculate the current focal length value under the camera mathematical model according to the object distance and the image distance using the corresponding lens formula.

[0074] Optionally, the device 500 also includes: a difference judgment module, used to judge whether the difference between the current focal length value and the preset focal length value is within a preset error range; when the difference between the current focal length value and the preset focal length value is not within the preset error range, updating the given initial value according to the difference between the current focal length value and the preset focal length value, and recalculating the current focal length value under the camera mathematical model using the updated initial value until the difference between the current focal length value and the preset focal length value is within the preset error range.

[0075] Thus, according to this embodiment, the calibration plate is first allowed to perform a posture transformation movement in the focal plane, and then the calibration plate after each posture transformation movement is photographed by the camera, the pixel coordinates of the feature points on the calibration plate are obtained, and the obtained pixel coordinates are used to fit the camera mathematical model. Secondly, when the number of transformations of the calibration plate to perform posture transformation movement reaches the preset requirement, an initial value of an intrinsic parameter in the intrinsic parameter matrix of the camera is given, and the given initial value is used to obtain the values of the remaining intrinsic parameters in the intrinsic parameter matrix and the values of all extrinsic parameters in the extrinsic parameter matrix of the camera. Then, the current focal length value under the camera mathematical model is calculated by the estimated intrinsic and extrinsic parameter matrix of the camera. Finally, when the difference between the current focal length value and the preset focal length value is within the preset error range, the initial value is substituted into the nonlinear optimization equation, and after iterating to the preset accuracy range, all the intrinsic and extrinsic parameters of the camera are obtained. The calibration method proposed by the present invention is extremely simple and only requires moving the calibration plate several times in the focal plane, which is very easy to implement in engineering. Because the calibration plate operates only within the focal plane, all calibration operations in this invention are performed with the camera in full focus, eliminating the loss of feature point accuracy in out-of-focus conditions. This allows for reliable calibration in a micro-field of view, theoretically and practically providing highly reliable and accurate calibration results. This addresses the existing technical issues of low accuracy, low reliability, high cost, and complex operation in micro-field camera lens calibration methods. Example 3

[0076] Figure 6 FIG. 6 shows a high-precision camera calibration device 600 for a micro field of view according to this embodiment, which corresponds to the method described in the first aspect of Example 1. Figure 6As shown, the device 600 includes: a processor 610; and a memory 620, which is connected to the processor 610 and is used to provide the processor 610 with instructions for processing the following processing steps: allowing the calibration plate to perform posture transformation movement in the focus plane; using a camera to shoot the calibration plate after each posture transformation movement, obtaining the pixel coordinates of the feature points on the calibration plate, and using the obtained pixel coordinates to fit the camera mathematical model; when the number of transformations of the calibration plate performing posture transformation movement reaches a preset requirement, giving an initial value of an intrinsic parameter in the intrinsic parameter matrix of the camera, and using the given initial value, obtaining the values of the remaining intrinsic parameters in the intrinsic parameter matrix and the values of all extrinsic parameters in the extrinsic parameter matrix of the camera; calculating the current focal length value under the camera mathematical model through the estimated intrinsic and extrinsic parameter matrix of the camera; when the difference between the current focal length value and the preset focal length value is within a preset error range, bringing the initial value into the nonlinear optimization equation, and after iterating to a preset accuracy range, obtaining all the intrinsic and extrinsic parameters of the camera.

[0077] Optionally, before allowing the calibration plate to perform posture transformation movement in the focus plane, the memory 620 is further used to provide the processor 610 with instructions for processing the following processing steps: tilting and fixing the camera, and adjusting the focus plane of the camera to be on the horizontal plane.

[0078] Optionally, allowing the calibration plate to perform a posture transformation movement in the focus plane includes: allowing the calibration plate to perform a posture transformation movement in the focus xy plane, wherein the posture transformation movement includes a translation in the xy plane and a rotation along the z-axis.

[0079] Optionally, before giving an initial value of an intrinsic parameter in the intrinsic parameter matrix of the camera, the memory 620 is also used to provide the processor 610 with instructions for processing the following processing steps: determining whether the number of transformations of the calibration plate's posture transformation movement meets the preset requirements; when the number of transformations of the calibration plate's posture transformation movement does not meet the preset requirements, continuing to allow the calibration plate to perform posture transformation movement in the focus plane until the number of transformations of the calibration plate's posture transformation movement meets the preset requirements.

[0080] Optionally, an initial value of an intrinsic parameter in the intrinsic parameter matrix of the camera is given, and the given initial value is used to obtain the values of the remaining intrinsic parameters in the intrinsic parameter matrix and the values of all extrinsic parameters in the extrinsic parameter matrix of the camera, including: giving an initial value of the image distance or the x-axis coordinate of the optical axis in the intrinsic parameter matrix of the camera; using the Zhang calibration method, and using the given initial value, obtaining the values of the remaining intrinsic parameters in the intrinsic parameter matrix and the values of all extrinsic parameters in the extrinsic parameter matrix of the camera.

[0081] Optionally, the current focal length value under the camera mathematical model is calculated by estimating the intrinsic and extrinsic parameter matrix of the camera, including: calculating the object distance under the camera mathematical model by estimating the intrinsic and extrinsic parameter matrix of the camera; calculating the image distance under the camera mathematical model based on the object distance while considering the influence of the Sham angle; and calculating the current focal length value under the camera mathematical model according to the object distance and the image distance using the corresponding lens formula.

[0082] Optionally, before the initial value is brought into the nonlinear optimization equation, the memory 620 is also used to provide the processor 610 with instructions for processing the following processing steps: determining whether the difference between the current focal length value and the preset focal length value is within a preset error range; when the difference between the current focal length value and the preset focal length value is not within the preset error range, updating the given initial value according to the difference between the current focal length value and the preset focal length value, and recalculating the current focal length value under the camera mathematical model using the updated initial value until the difference between the current focal length value and the preset focal length value is within the preset error range.

[0083] Thus, according to this embodiment, the calibration plate is first allowed to perform a posture transformation movement in the focal plane, and then the calibration plate after each posture transformation movement is photographed by the camera, the pixel coordinates of the feature points on the calibration plate are obtained, and the obtained pixel coordinates are used to fit the camera mathematical model. Secondly, when the number of transformations of the calibration plate to perform posture transformation movement reaches the preset requirement, an initial value of an intrinsic parameter in the intrinsic parameter matrix of the camera is given, and the given initial value is used to obtain the values of the remaining intrinsic parameters in the intrinsic parameter matrix and the values of all extrinsic parameters in the extrinsic parameter matrix of the camera. Then, the current focal length value under the camera mathematical model is calculated by the estimated intrinsic and extrinsic parameter matrix of the camera. Finally, when the difference between the current focal length value and the preset focal length value is within the preset error range, the initial value is substituted into the nonlinear optimization equation, and after iterating to the preset accuracy range, all the intrinsic and extrinsic parameters of the camera are obtained. The calibration method proposed by the present invention is extremely simple and only requires moving the calibration plate several times in the focal plane, which is very easy to implement in engineering. Because the calibration plate operates only within the focal plane, all calibration operations in this invention are performed with the camera in full focus, eliminating the loss of feature point accuracy in out-of-focus conditions. This allows for reliable calibration in a micro-field of view, theoretically and practically providing highly reliable and accurate calibration results. This addresses the existing technical issues of low accuracy, low reliability, high cost, and complex operation in micro-field camera lens calibration methods.

[0084] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0085] In the above embodiments of the present invention, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0086] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0087] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0088] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0089] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), a mobile hard drive, a magnetic disk, or an optical disk.

[0090] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A high-precision camera calibration method for micro-field of view, characterized in that: include: Let the calibration plate perform posture transformation movement in the focus plane; Use the camera to shoot the calibration plate after each pose transformation, obtain the pixel coordinates of the feature points on the calibration plate, and use the obtained pixel coordinates to fit the camera mathematical model; When the number of transformations of the calibration plate's pose transformation reaches the preset requirement, an initial value of an intrinsic parameter in the camera's intrinsic parameter matrix is given, and the values of the remaining intrinsic parameters in the intrinsic parameter matrix and all the extrinsic parameters in the camera's extrinsic parameter matrix are obtained using the given initial value; Calculate the current focal length value under the camera mathematical model through the estimated camera internal and external parameter matrix; Determine whether the difference between the current focal length value and the preset focal length value is within a preset error range; When the difference between the current focal length value and the preset focal length value is not within the preset error range, updating the given initial value according to the difference between the current focal length value and the preset focal length value, and recalculating the current focal length value under the camera mathematical model using the updated initial value until the difference between the current focal length value and the preset focal length value is within the preset error range; When the difference between the current focal length value and the preset focal length value is within the preset error range, the initial value is substituted into the nonlinear optimization equation, and after iterating to the preset accuracy range, all the internal and external parameters of the camera are obtained.

2. The method according to claim 1, characterized in that Before the calibration plate is transformed in position within the focal plane, the following steps are also required: Tilt and fix the camera, and adjust the camera's focus plane to be on the horizontal plane.

3. The method according to claim 1, characterized in that Let the calibration plate perform pose transformation in the focus plane, including: The calibration plate is allowed to perform posture transformation in the focused xy plane, where the posture transformation includes translation in the xy plane and rotation along the z axis.

4. The method according to claim 1, wherein Before giving an initial value of an intrinsic parameter in the intrinsic parameter matrix of the camera, it also includes: Determine whether the number of transformations of the calibration plate's posture transformation movement meets the preset requirements; When the number of transformations of the calibration plate's posture transformation movement does not reach the preset requirement, the calibration plate continues to be allowed to perform posture transformation movement in the focus plane until the number of transformations of the calibration plate's posture transformation movement reaches the preset requirement.

5. The method according to claim 1, wherein Given an initial value of an intrinsic parameter in the camera's intrinsic parameter matrix, and using the given initial value, obtain the values of the remaining intrinsic parameters in the intrinsic parameter matrix and the values of all extrinsic parameters in the camera's extrinsic parameter matrix, including: Given an initial value of the image distance or optical axis x-axis coordinate in the intrinsic parameter matrix of the camera; The Zhang calibration method is used to obtain the values of the remaining intrinsic parameters in the intrinsic parameter matrix and the values of all extrinsic parameters in the camera's extrinsic parameter matrix using the given initial values.

6. The method according to claim 1, characterized in that The current focal length value under the camera mathematical model is calculated by estimating the camera's internal and external parameter matrices, including: Calculate the object distance under the camera mathematical model through the estimated camera's internal and external parameter matrix; Considering the influence of Sham angle, the image distance under the camera mathematical model is calculated based on the object distance; Use the corresponding lens formula to calculate the current focal length value under the camera mathematical model based on the object distance and image distance.

7. A storage medium, characterized in that: The storage medium includes a stored program, wherein when the program is run, the processor executes the method according to any one of claims 1 to 6.

8. A high-precision camera calibration device for micro-field of view, characterized in that: include: The posture transformation module is used to allow the calibration plate to perform posture transformation movement within the focus plane; The pixel coordinate acquisition module is used to use the camera to shoot the calibration plate after each posture transformation, obtain the pixel coordinates of the feature points on the calibration plate, and use the obtained pixel coordinates to fit the camera mathematical model; The internal and external parameter initialization module is used to give an initial value of an internal parameter in the camera's internal parameter matrix when the number of transformations of the calibration plate reaches the preset requirement, and use the given initial value to obtain the values of the remaining internal parameters in the internal parameter matrix and the values of all external parameters in the camera's external parameter matrix; A current focal length determination module is used to calculate the current focal length value under the camera mathematical model by estimating the camera's internal and external parameter matrices; A judgment module, used to judge whether the difference between the current focal length value and the preset focal length value is within a preset error range; a current focal length value updating module for updating a given initial value according to the difference between the current focal length value and the preset focal length value when the difference between the current focal length value and the preset focal length value is not within a preset error range, and recalculating the current focal length value under the camera mathematical model using the updated initial value until the difference between the current focal length value and the preset focal length value is within a preset error range; The internal and external parameter determination module is used to bring the initial value into the nonlinear optimization equation when the difference between the current focal length value and the preset focal length value is within the preset error range, and after iterating to the preset accuracy range, obtain all the internal and external parameters of the camera.

9. A high-precision camera calibration device for micro-field of view, characterized in that: include: processor; as well as A memory, connected to the processor, configured to provide the processor with instructions for processing the following processing steps: Let the calibration plate perform posture transformation movement in the focus plane; Use the camera to shoot the calibration plate after each pose transformation, obtain the pixel coordinates of the feature points on the calibration plate, and use the obtained pixel coordinates to fit the camera mathematical model; When the number of transformations of the calibration plate's pose transformation reaches the preset requirement, an initial value of an intrinsic parameter in the camera's intrinsic parameter matrix is given, and the values of the remaining intrinsic parameters in the intrinsic parameter matrix and all the extrinsic parameters in the camera's extrinsic parameter matrix are obtained using the given initial value; Calculate the current focal length value under the camera mathematical model through the estimated camera internal and external parameter matrix; Determine whether the difference between the current focal length value and the preset focal length value is within a preset error range; When the difference between the current focal length value and the preset focal length value is not within the preset error range, updating the given initial value according to the difference between the current focal length value and the preset focal length value, and recalculating the current focal length value under the camera mathematical model using the updated initial value until the difference between the current focal length value and the preset focal length value is within the preset error range; When the difference between the current focal length value and the preset focal length value is within the preset error range, the initial value is substituted into the nonlinear optimization equation, and after iterating to the preset accuracy range, all the internal and external parameters of the camera are obtained.

Images