Camera automatic calibration method and system

Abstract

The invention relates to a camera automatic calibration method and system. The method comprises the following steps: respectively acquiring reference coordinate positions of centers of a plurality of calibration marks in a chuck table coordinate system; wherein the plurality of calibration marks are respectively fixed on the chuck table; controlling the chuck table to move to different positions along different directions, and acquiring a plurality of moving coordinate positions of each calibration mark in a chuck table coordinate system in each direction according to the reference coordinate position of each calibration mark; the chuck table is controlled to move, and calibration images of all the calibration marks on the corresponding reference coordinate positions and the corresponding moving coordinate positions are collected; obtaining a center pixel coordinate of each calibration mark in a camera coordinate system based on the calibration image; and matching the center pixel coordinate with the corresponding reference coordinate position and the moving coordinate position, and solving camera calibration parameters. The calibration precision of the camera is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of camera calibration, and in particular to a camera automatic calibration method and system. Background Art

[0002] Camera calibration plays a key role in the fields of computer vision, robotics, autonomous driving, and wafer defect detection. Its core technology is to establish a camera imaging model to determine the internal parameters (intrinsic parameters) and external parameters (extrinsic parameters) of the camera, and finally accurately map the pixel coordinates in the image to the actual three-dimensional world coordinates. The imaging model of camera calibration usually adopts the pinhole camera model and combines the lens distortion model to describe the imaging process of the camera. Its intrinsic parameters include focal length (f x , f y ), principal point coordinates (c x , c y ), and distortion coefficients (radial distortion, tangential distortion, etc.). Its extrinsic parameters include the rotation matrix (R) and the translation vector (T), which mainly describe the position and orientation of the camera in the world coordinate system. The accuracy of camera calibration directly affects the performance of subsequent vision tasks. Therefore, a good calibration method plays a crucial role in high-precision application scenarios such as wafer defect detection.

[0003] Currently, the commonly used camera calibration method is mainly the Zhang Zhengyou calibration method. This traditional calibration method mainly extracts corner points by taking multiple images of known geometric patterns (such as checkerboards) and optimizes and calculates the internal and external parameters of the camera. Although this calibration method is widely used and has a simple and easy-to-implement method, there are still some significant disadvantages. First, its calibration accuracy is affected by various factors, such as the quality of the calibration board, the accuracy of corner point detection, and image noise. This method is not applicable to high-precision application scenarios such as industrial inspection, medical imaging, and wafer defect detection. Second, this method assumes that the camera and the environment are static and is difficult to apply to dynamic environments. For example, for real-time and automatic camera calibration, this method is not very applicable and a technology capable of online calibration is needed. Summary of the Invention

[0004] Therefore, the present invention provides a camera automatic calibration method and system, aiming to solve the problem that the existing calibration technology is difficult to meet the requirements of specific scenarios such as industrial inspection (e.g., high-precision requirements in the field of wafer defect detection) in terms of accuracy and real-time performance.

[0005] To solve the above technical problems, a camera automatic calibration method of the present invention includes:

[0006] respectively obtaining the reference coordinate positions of the centers of multiple calibration marks in the chuck table coordinate system; wherein, the multiple calibration marks are respectively fixed on the chuck table;

[0007] Control the chuck table to move to different positions in different directions, and according to the reference coordinate positions of the calibration marks, obtain multiple moving coordinate positions of each calibration mark in the chuck table coordinate system in each direction;

[0008] Control the movement of the chuck table, and collect calibration images of each calibration mark at the corresponding reference coordinate positions and moving coordinate positions;

[0009] Based on the calibration images, obtain the central pixel coordinates of each calibration mark in the camera coordinate system;

[0010] Match the central pixel coordinates with the corresponding reference coordinate positions and moving coordinate positions, and solve the camera calibration parameters.

[0011] In an embodiment of the present invention, the calibration image of the calibration mark includes:

[0012] A calibration plate body, and the calibration plate body is square;

[0013] A central circular mark, as the center of the calibration mark, and the central circular mark is located at the center of the calibration plate body;

[0014] A plurality of square frame marks, which are arranged radially outward in sequence with the central circular mark as the center;

[0015] A plurality of triangular marks, which are arranged outward in sequence with the central circular mark as the starting point.

[0016] In an embodiment of the present invention, controlling the chuck table to move to different positions in different directions includes:

[0017] Control the chuck table to move in the four directions of up, down, left, and right, and move to different positions in each direction.

[0018] In an embodiment of the present invention, controlling the movement of the chuck table and collecting calibration images of each calibration mark at the corresponding reference coordinate positions and moving coordinate positions includes:

[0019] In response to the chuck table receiving the sent moving coordinate positions, control the chuck table to move to each moving coordinate position in sequence and send a photo-taking trigger signal, and send the calibration image to the host computer through a handshake signal;

[0020] In response to the host computer failing to receive all the image data, stop the current calibration and send a handshake failure instruction.

[0021] In an embodiment of the present invention, controlling the movement of the chuck table and collecting calibration images of each of the calibration marks at the corresponding reference coordinate positions and the moving coordinate positions includes:

[0022] In response to a delay between the time when the photographing signal is triggered during the image acquisition process and the actual imaging of the camera, delay compensation is performed.

[0023] In an embodiment of the present invention, the performing of the delay compensation includes:

[0024] Calibrating the time delay amount between the photographing trigger signal and the actual imaging of the camera;

[0025] Based on the time delay amount, controlling the chuck table to send out the photographing trigger signal in advance by the corresponding time, so as to eliminate the influence of the time delay on the imaging accuracy.

[0026] In an embodiment of the present invention, matching the central pixel coordinates with the corresponding reference coordinate positions and the moving coordinate positions, and solving the camera calibration parameters includes:

[0027] Converting the point p t :(x t , y t ) in the chuck coordinate system to the camera coordinate system as p c :(x c , y c ); there are rotations (θ) and scales (s) between the coordinate system of the chuck table and the image coordinate system, R(θ) is the rotation matrix, p i is the coordinate of the converted camera coordinate system, and there is a translation p offset :(x offset , y offset ) between the image coordinate system and the camera coordinate system;

[0028] p i = sR(θ)p t = p c + p offset

[0029]

[0030] Setting s1 = s×cos(θ), s2 = s×sin(θ), then:

[0031]

[0032] Solving the unknowns s1, s2, x offset , y offset .

[0033] The present invention also provides a camera automatic calibration system, including:

[0034] A reference coordinate position acquisition module, configured to respectively acquire the reference coordinate positions of the centers of a plurality of calibration markers in the chuck table coordinate system; wherein, the plurality of calibration markers are respectively fixed on the chuck table;

[0035] A moving coordinate position acquisition module, configured to control the chuck table to move to different positions in different directions, and according to the reference coordinate positions of the respective calibration markers, acquire a plurality of moving coordinate positions of the respective calibration markers in the chuck table coordinate system in each direction;

[0036] A calibration image acquisition module, configured to control the movement of the chuck table and acquire calibration images of the respective calibration markers at the corresponding reference coordinate positions and moving coordinate positions;

[0037] A central pixel coordinate acquisition module, configured to obtain the central pixel coordinates of the respective calibration markers in the camera coordinate system based on the calibration images;

[0038] A camera calibration parameter solving module, which matches the central pixel coordinates with the corresponding reference coordinate positions and moving coordinate positions, and solves the camera calibration parameters.

[0039] The above technical solution of the present invention has the following advantages compared with the prior art:

[0040] The camera automatic calibration method and system of the present invention solve the pain points of the existing calibration technology that it is difficult to meet the requirements of industrial inspection scenarios (especially wafer defect detection) in terms of accuracy and real-time performance. For the micron-level accuracy requirements, this method significantly improves the calibration accuracy by adopting a unique zigzag calibration pattern (utilizing precise geometric structures and fine pattern arrangements); and by fixing the calibration marker on the chuck table, the deep integration and online calibration function of the calibration system and the detection platform are realized. In addition, for the time delay problem between the trigger signal of the chuck table and the actual photographing of the camera, the present invention creatively proposes a delayed calibration method, effectively compensating for the accuracy error caused by the trigger time delay. The present invention also has the following advantages:

[0041] By optimizing the geometric structure and precise pattern arrangement, the accuracy of calibration feature recognition is greatly improved, which can meet the detection requirements at the micron level.

[0042] The calibration marker is fixed on the chuck table, and with the real-time data interaction mechanism, true online calibration is realized;

[0043] The calibration process is seamlessly connected with the detection operation, avoiding the problems of interrupting the process, consuming time and effort in traditional calibration methods, and greatly improving the detection efficiency.

[0044] A delay compensation method is proposed for the time delay between the trigger signal of the chuck table and the actual photographing of the camera, automatically correcting the imaging error caused by the time delay to ensure the calibration accuracy.

[0045] The system can perform real-time calibration according to environmental factors and changes in equipment status, ensuring the continuity and stability of the detection process;

[0046] The present invention has significantly improved both in calibration accuracy and calibration efficiency, is applicable to wafer defect detection and other industrial scenarios with strict requirements for high-precision calibration, and has broad application prospects and important practical value. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to make the content of the present invention easier to be clearly understood, the present invention will be further described in detail below according to specific embodiments of the present invention in conjunction with the drawings.

[0048] Figure 1 It is a flowchart of the automatic calibration method of the camera of the present invention.

[0049] Figure 2 It is a schematic diagram of the calibration mark of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0050] The present invention will be further described below in conjunction with the drawings and specific embodiments, so that those skilled in the art can better understand the present invention and be able to implement it, but the embodiments cited are not intended to limit the present invention.

[0051] In the present invention, if there is a description of directions (up, down, left, right, front and back), it is only for the convenience of describing the technical solution of the present invention, rather than indicating or implying that the technical features referred to must have a specific orientation, be constructed and operated in a specific orientation, so it cannot be understood as a limitation of the present invention.

[0052] In the present invention, the meaning of "several" is one or more, the meaning of "multiple" is two or more, and "greater than", "less than", "exceeding" etc. are understood as not including the present number; "above", "below", "within" etc. are understood as including the present number. In the description of the present invention, if there is a description of "first" and "second", it is only for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the technical features indicated or the sequence relationship of the technical features indicated.

[0053] In the present invention, unless otherwise clearly defined, terms such as "arranged", "installed", and "connected" should be understood in a broad sense. For example, they can be directly connected, or indirectly connected through an intermediate medium; they can be fixedly connected, or detachably connected, or integrally formed; they can be mechanically connected, or electrically connected or capable of communicating with each other; they can be the communication inside two components or the interaction relationship between two components. Those skilled in the art can reasonably determine the specific meanings of the above terms in the present invention in combination with the specific content of the technical solution.

[0054] Embodiment 1

[0055] Referring to Figure 1 as shown, an automatic calibration method for a camera of the present invention includes:

[0056] S1. Respectively obtain the reference coordinate positions of the centers of multiple calibration markers (markers) in the chuck table coordinate system; wherein, the multiple calibration markers are respectively fixed on the chuck table.

[0057] Specifically, collect the initial images of each calibration marker, based on the initial images, obtain the reference coordinate positions, and record and store them in a database as the reference data for subsequent calculations.

[0058] It should be noted that the chuck table is a precision mechanical platform used to fix and move wafers (or other objects to be measured), and is widely used in fields such as semiconductor manufacturing, lithography, wafer inspection, and micro-nano processing. The chuck table can perform high-precision movement in multiple degrees of freedom (X-axis, Y-axis, Z-axis, rotation θ, etc.). In addition, the chuck table is integrated with a control system and can move according to a preset trajectory, supporting a fully automatic detection process.

[0059] Specifically, referring to Figure 2 as shown, the calibration image of the calibration marker includes:

[0060] A calibration plate body, the calibration plate body is square;

[0061] A central circular marker, as the center of the calibration marker, the central circular marker is located at the center position of the calibration plate body;

[0062] Multiple square-ring markers, arranged radially outward in sequence with the central circular marker as the center;

[0063] Multiple triangular markers, arranged outward in sequence with the central circular marker as the starting point.

[0064] Through the multi-level and multi-scale arrangement of different geometric elements (circles, double squares, triangles), rich and stable visual features are provided, which helps advanced image processing algorithms to perform pattern recognition efficiently and with high accuracy, improves the resistance to noise and interference, and thus improves the accuracy and stability of camera parameter calibration.

[0065] Exemplarily, the side length of the square of the calibration plate body is 10 mm, and chamfers C with a length of 0.5 mm are provided at the four corners;

[0066] The radius R of the central circular mark is 0.25 mm; three double-square marks are provided, and the first double-square mark is 0.75 mm away from the central circular mark; the distances between adjacent two double-square marks are 0.6 mm and 0.7 mm respectively;

[0067] Three triangular marks are provided, the first triangular mark is 0.45 mm away from the central circular mark; the distances between adjacent two triangular marks are 1.05 mm and 1.15 mm respectively.

[0068] S2. Control the chuck table to move to different positions in different directions, and according to the reference coordinate positions of the calibration marks, obtain multiple moving coordinate positions of the calibration marks in the chuck table coordinate system in each direction.

[0069] Specifically, control the chuck table to move in the four directions of up, down, left, and right, and move to different positions in each direction.

[0070] S3. Control the movement of the chuck table, and collect calibration images of the calibration marks at the corresponding reference coordinate positions and moving coordinate positions.

[0071] Exemplarily, the chuck table moves precisely in the four directions of up, down, left, and right to a predetermined distance, five calibration images are taken in each direction, and combined with one calibration image at the initial position, a total of 21 images are collected for each calibration mark. Based on the reference coordinate position of the center of the central circle in the initial position image in the chuck table coordinate system, through the coordinate transformation algorithm, calculate the coordinate positions of the centers of the central circles of the calibration plate patterns in the other 20 images in the chuck table coordinate system.

[0072] A total of four calibration marks are used, and the coordinate positions of the centers of the central circles of 21 images need to be recorded for each calibration mark, totaling 84 groups of coordinate data. These coordinate data are sent to the chuck table, driving it to move to the specified positions in sequence and trigger photographing to ensure the comprehensiveness and systematicness of data collection.

[0073] S4. Based on the calibration images, obtain the central pixel coordinates of the calibration marks in the camera coordinate system.

[0074] Specifically, in response to the chuck table receiving the transmitted moving coordinate positions, the chuck table is controlled to move to each of the moving coordinate positions in sequence and send a photographing trigger signal, and a calibration image is sent to the host computer through a handshake signal.

[0075] When the host computer fails to receive all the image data, the current calibration is stopped and a handshake failure instruction is sent. At this time, it is necessary to trigger photographing again and then send the data.

[0076] In addition, after obtaining the calibration value, in view of the time delay problem between the trigger signal and the actual image collected by the camera during the flying shooting process, a delay calibration mechanism is introduced. Specifically, it includes:

[0077] In response to the time delay between the trigger signal for photographing and the actual imaging of the camera during the image acquisition process, delay compensation is performed. By accurately calibrating the time delay amount between the photographing trigger signal and the actual imaging of the camera;

[0078] Based on the time delay amount, the chuck table is controlled to send a photographing trigger signal in advance by the corresponding time, so as to eliminate the influence of the time delay on the imaging accuracy.

[0079] In addition, after successfully receiving all the images, existing image processing algorithms are used to perform high-precision detection on the images, and the central pixel coordinates of the central circles of all the calibration images are extracted to provide accurate input data for subsequent calibration calculations.

[0080] S5. Match the central pixel coordinates with the corresponding reference coordinate positions and the moving coordinate positions, and solve the camera calibration parameters. Specifically, it includes:

[0081] Convert the point p t : (x t , y t ) in the chuck coordinate system to the camera coordinate system as p c : (x c , y c ); there are rotation (θ) and scale (s) between the coordinate system of the chuck table and the image coordinate system, R(θ) is the rotation matrix, p i is the coordinate in the converted camera coordinate system, and there is a translation p offset : (x offset , y offset ) between the image coordinate system and the camera coordinate system;

[0082] p i = sR(θ)p t = p c + p offset

[0083]

[0084] Set \(s1 = s\times\cos(\theta)\), \(s2 = s\times\sin(\theta)\), then:

[0085]

[0086] For multiple markers, there are multiple offsets. By jointly solving for multiple markers, the equation can be further written as:

[0087]

[0088] Solve for the unknowns \(s1\), \(s2\), \(x\) offset , \(y\) offset .

[0089] In this embodiment, by adopting a unique zigzag calibration pattern (utilizing precise geometric structures and fine pattern arrangements), the calibration accuracy is significantly improved; and by fixing the calibration marker on the chuck table, the deep integration of the calibration system and the detection platform and the online calibration function are realized. In addition, for the time delay problem between the trigger signal of the chuck table and the actual photographing of the camera, the present invention creatively proposes a delayed calibration method, effectively compensating for the accuracy error caused by the trigger time delay.

[0090] Embodiment 2

[0091] Based on the same inventive concept, this embodiment provides a camera automatic calibration system. The principle of solving the problem is similar to that of the camera automatic calibration method, and the repeated parts will not be elaborated.

[0092] This embodiment provides a camera automatic calibration system, including:

[0093] A reference coordinate position acquisition module, configured to respectively acquire the reference coordinate positions of the centers of multiple calibration markers in the chuck table coordinate system; wherein, the multiple calibration markers are respectively fixed on the chuck table;

[0094] A moving coordinate position acquisition module, configured to control the chuck table to move to different positions in different directions, and according to the reference coordinate positions of the respective calibration markers, acquire multiple moving coordinate positions of the respective calibration markers in the chuck table coordinate system in each direction;

[0095] A calibration image acquisition module, configured to control the movement of the chuck table and acquire calibration images of the respective calibration markers at the corresponding reference coordinate positions and moving coordinate positions;

[0096] A central pixel coordinate acquisition module, configured to obtain the central pixel coordinates of each of the calibration marks in the camera coordinate system based on the calibration image;

[0097] A camera calibration parameter solving module, which matches the central pixel coordinates with the corresponding reference coordinate positions and moving coordinate positions to solve the camera calibration parameters.

[0098] Those skilled in the art should understand that the embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program code.

[0099] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0100] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including instruction means, and the instruction means implements the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0101] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0102] Finally, it should be noted that the above specific embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the examples, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered within the scope of the claims of the present invention.

Claims

1. A method for automatic calibration of a camera, characterized in that, Including: Respectively obtain the reference coordinate positions of the centers of multiple calibration marks in the chuck table coordinate system; wherein, the multiple calibration marks are respectively fixed on the chuck table; Control the chuck table to move to different positions in different directions, and according to the reference coordinate positions of the respective calibration marks, obtain multiple moving coordinate positions of the respective calibration marks in the chuck table coordinate system in each direction; Control the movement of the chuck table, and collect calibration images of the respective calibration marks at the corresponding reference coordinate positions and moving coordinate positions; Based on the calibration images, obtain the central pixel coordinates of the respective calibration marks in the camera coordinate system; Match the central pixel coordinates with the corresponding reference coordinate positions and moving coordinate positions, and solve the camera calibration parameters.

2. The method for automatic calibration of a camera according to claim 1, wherein The calibration image of the calibration mark includes: A calibration plate body, and the calibration plate body is square; A central circular mark, serving as the center of the calibration mark, and the central circular mark is located at the center position of the calibration plate body; Multiple square frame marks, radiating outwards in sequence with the central circular mark as the center; Multiple triangular marks, arranged outwards in sequence with the central circular mark as the starting point.

3. A camera automatic calibration method according to claim 1, characterized in that, Controlling the chuck table to move to different positions in different directions includes: Controlling the chuck table to move in four directions: up, down, left, and right, and moving to different positions in each direction.

4. A method for automatic calibration of a camera according to claim 1, wherein, Controlling the movement of the chuck table, and collecting calibration images of the respective calibration marks at the corresponding reference coordinate positions and moving coordinate positions includes: In response to the chuck table receiving the sent moving coordinate positions, control the chuck table to move to each of the moving coordinate positions in sequence and send a photo-taking trigger signal, and send the calibration image to the host computer through a handshake signal; In response to the situation that the host computer fails to receive all the image data, stop the current calibration and send a handshake failure instruction.

5. The automatic calibration method of a camera according to claim 1, wherein, Controlling the movement of the chuck table, and collecting calibration images of the respective calibration marks at the corresponding reference coordinate positions and moving coordinate positions includes: In response to a delay between the trigger signal for taking a photo and the actual imaging of the camera during the image acquisition process, perform delay compensation.

6. The automatic calibration method of a camera according to claim 5, wherein, The performing of the delay compensation includes: Calibrating the time delay amount between the photo-taking trigger signal and the actual imaging of the camera; Based on the time delay amount, control the chuck table to send the photo-taking trigger signal in advance by the corresponding time, so as to eliminate the influence of the time delay on the imaging accuracy.

7. A method for automatic calibration of a camera according to claim 1, wherein, Matching the central pixel coordinates with the corresponding reference coordinate positions and moving coordinate positions, and solving the camera calibration parameters includes: Convert the point p in the chuck coordinate system t :(x t , y t ) to the camera coordinate system as p c :(x c , y c ); There is a rotation (θ) and a scale (s) between the coordinate system of the chuck table and the image coordinate system. R(θ) is the rotation matrix, and p i is the coordinate in the converted camera coordinate system. There is a translation p offset :(x offset , y offset ) between the image coordinate system and the camera coordinate system; p i = sR(θ)p t = p c + p offset Set s1 = s×cos(θ), s2 = s×sin(θ), then: Solve for the unknowns s1, s2, x by the least squares method offset , y offset .

8. An automatic camera calibration system, characterized in that, Including: A reference coordinate position acquisition module, configured to respectively obtain the reference coordinate positions of the centers of multiple calibration marks in the chuck table coordinate system; wherein, the multiple calibration marks are respectively fixed on the chuck table; A moving coordinate position acquisition module, which is used to control the chuck table to move to different positions in different directions, and obtain multiple moving coordinate positions of each calibration mark in the chuck table coordinate system in each direction according to the reference coordinate positions of the calibration marks; A calibration image acquisition module, which is used to control the movement of the chuck table and acquire calibration images of each calibration mark at the corresponding reference coordinate positions and moving coordinate positions; A central pixel coordinate acquisition module, which is used to obtain the central pixel coordinates of each calibration mark in the camera coordinate system based on the calibration images; A camera calibration parameter solving module, which matches the central pixel coordinates with the corresponding reference coordinate positions and moving coordinate positions, and solves the camera calibration parameters.

Images