An automatic calibration method and system for industrial area array cameras

Abstract

The present invention provides an automatic calibration method and system for industrial area array cameras in the field of industrial area array camera calibration technology. The method includes: step S10, controlling the industrial area array camera to move to an area above a calibration feature; step S20, photographing the calibration feature to obtain a calibration photo, performing distortion correction on the calibration photo, using the calibration feature as a calibration mark point, and recording the machine coordinates of the photographing position at that time; step S30, dividing the pixel field of view into blocks to construct a point set of equally divided mark points; step S40, selecting the equally divided mark points and taking a difference between the calibration mark points, outputting a control parameter for the movement of the industrial area array camera based on the difference, completing an adaptive control process of a single-point spiral approximation, and recording the pixel coordinates and machine coordinates of the point; and step S50, executing an N-point calibration algorithm based on the pixel coordinates and machine coordinates to complete the calibration. The advantages of the present invention are: greatly improving the efficiency and safety of industrial area array camera calibration and greatly reducing the labor cost of industrial area array camera calibration.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial area array camera calibration, and in particular to an automatic calibration method and system for industrial area array cameras. Background Art

[0002] With the addition of industrial area scan cameras, industrial robots and servo equipment have replaced manual assembly and inspection processes in fields such as automotive manufacturing and new energy, greatly improving work efficiency. Industrial area scan cameras require calibration before use in vision guidance and inspection tasks to determine the mapping between pixel coordinates and machine coordinates. Because the relationship between the device coordinate system and the camera coordinate system cannot be determined before calibration, and the purpose of calibration is to determine this relationship, this is also the key reason why fully automatic calibration is currently unavailable. Therefore, manual and semi-automatic calibration methods have traditionally been used.

[0003] Manual calibration requires multiple manual collaborations between the device and the camera vision to complete the mapping between the device coordinate system and the camera coordinate system. Workers need to enter the device area to confirm the status of the calibration points, which consumes a lot of manpower. In addition, there are great safety risks when workers confirm the points inside the device.

[0004] Semi-automatic calibration is to automatically read pixel coordinates through software. It is automation at the software application level, but staff still need to enter the equipment area to perform multiple manual point alignments. Although it is semi-automated compared to manual calibration, it still requires human intervention and poses safety risks.

[0005] Therefore, how to provide an automatic calibration method and system for industrial area array cameras to improve the efficiency and safety of industrial area array camera calibration and reduce the cost of industrial area array camera calibration has become a technical problem that needs to be solved urgently. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to provide a method and system for automatic calibration of industrial area array cameras, so as to improve the efficiency and safety of industrial area array camera calibration and reduce the cost of industrial area array camera calibration.

[0007] In a first aspect, the present invention provides an automatic calibration method for an industrial area array camera, comprising the following steps:

[0008] Step S10: The camera driver establishes TCP / IP communication with the visual host computer. The device driver controls the industrial area array camera to move to the area above the preset calibration feature and adjusts it to a suitable imaging distance.

[0009] Step S20: The industrial area array camera takes a picture of the calibration feature to obtain a calibration picture, performs distortion correction on the calibration picture using a distortion correction algorithm, uses the calibration feature as a calibration mark point, and records the machine coordinates of the shooting position at this time;

[0010] Step S30: Divide the pixel field of view of the industrial area array camera into blocks, perform equal division according to the size of the pixel field of view, determine the pixel coordinates of the center point of the equal division block, and construct a point set of equal division mark points;

[0011] Step S40: sequentially select equally divided Mark points in the point set and perform subtraction with the calibration Mark points. The adaptive controller outputs control parameters for the movement of the industrial area array camera based on the subtraction, completing the adaptive control process of the single-point spiral approximation. The pixel coordinates and machine coordinates of the point are recorded. The process is repeated until the points in the point set are successfully fitted and recorded.

[0012] Step S50: Execute an N-point calibration algorithm according to the recorded pixel coordinates and machine coordinates to complete the automatic calibration process.

[0013] Furthermore, in step S20, during the process of taking the calibration photo, the light source is controlled to provide appropriate fill light to the imaging area of ​​the industrial area array camera.

[0014] Furthermore, the step S30 is specifically as follows:

[0015] The pixel field of view of the industrial area array camera is divided into blocks, and the pixel coordinates of the center point of each block are divided into nine equal parts according to the size of the pixel field of view. The center coordinates after equal division are used as equal division mark points, and a point set is constructed based on each equal division mark point.

[0016] Furthermore, the step S40 is specifically as follows:

[0017] Select one of the equally divided Mark points, calculate the difference g between the pixel coordinates of the equally divided Mark point and the calibration Mark point, input the difference g into the adaptive controller, and drive the device based on the X-axis control amount u output by the adaptive controller. x and Y-axis control value u y Control the industrial area array camera to move so that the calibration mark point spirally approaches the equally divided mark points in the camera field of view until the absolute value of the difference g is less than the set acceptance threshold, record the pixel coordinates and machine coordinates of the point, and repeat the process until the points in the point set are successfully fitted and recorded.

[0018] Furthermore, the output formula of the adaptive controller is:

[0019] u x =ω x*(B1*g+B2*(dg / dt)+B3*∫g(t)dt);

[0020] u y =ω y *(B1*g+B2*(dg / dt)+B3*∫g(t)dt);

[0021] Among them, ω x and ω y Both represent dynamic angle adjustment weights, and ω x =cosθ,ω y =sinθ, θ is the angle between the current frame and the initial direction; B1, B2, and B3 are constants obtained through Bayesian optimization; dg / dt is the rate of change of the difference.

[0022] In a second aspect, the present invention provides an automatic calibration system for industrial area array cameras, comprising the following modules:

[0023] The camera control module is used to establish TCP / IP communication between the camera driver and the visual host computer. The device driver controls the industrial area array camera to move to the area above the preset calibration feature and adjust it to the appropriate imaging distance.

[0024] The distortion correction module is used to take a picture of the calibration feature with an industrial area array camera to obtain a calibration photo, perform distortion correction on the calibration photo using a distortion correction algorithm, and use the calibration feature as a calibration mark point to record the machine coordinates of the shooting position at this time;

[0025] The point set construction module is used to divide the pixel field of view of the industrial area array camera into blocks, perform equal division operations according to the size of the pixel field of view, determine the pixel coordinates of the center point of the equal division block, and construct a point set of the equally divided mark points;

[0026] An adaptive control module is configured to sequentially select equally divided Mark points within the point set and perform subtraction with calibration Mark points. An adaptive controller outputs control parameters for the movement of the industrial area array camera based on the subtraction, completing the adaptive control process of a single-point spiral approximation, recording the pixel coordinates and machine coordinates of the point, and repeating the process until the point positions within the point set are successfully fitted and recorded.

[0027] The calibration module is used to execute an N-point calibration algorithm according to the recorded pixel coordinates and machine coordinates to complete an automatic calibration process.

[0028] Furthermore, in the distortion correction module, during the process of taking the calibration photo, the light source is controlled to provide appropriate fill light to the imaging area of ​​the industrial area array camera.

[0029] Furthermore, the point set construction module is specifically used to:

[0030] The pixel field of view of the industrial area array camera is divided into blocks, and the pixel coordinates of the center point of each block are divided into nine equal parts according to the size of the pixel field of view. The center coordinates after equal division are used as equal division mark points, and a point set is constructed based on each equal division mark point.

[0031] Furthermore, the adaptive control module is specifically configured to:

[0032] Select one of the equally divided Mark points, calculate the difference g between the pixel coordinates of the equally divided Mark point and the calibration Mark point, input the difference g into the adaptive controller, and drive the device based on the X-axis control amount u output by the adaptive controller. x and Y-axis control value u y Control the industrial area array camera to move so that the calibration mark point spirally approaches the equally divided mark points in the camera field of view until the absolute value of the difference g is less than the set acceptance threshold, record the pixel coordinates and machine coordinates of the point, and repeat the process until the points in the point set are successfully fitted and recorded.

[0033] Furthermore, the output formula of the adaptive controller is:

[0034] u x =ω x *(B1*g+B2*(dg / dt)+B3*∫g(t)dt);

[0035] u y =ω y *(B1*g+B2*(dg / dt)+B3*∫g(t)dt);

[0036] Among them, ω x and ω y Both represent dynamic angle adjustment weights, and ω x =cosθ,ω y =sinθ, θ is the angle between the current frame and the initial direction; B1, B2, and B3 are constants obtained through Bayesian optimization; dg / dt is the rate of change of the difference.

[0037] The advantages of the present invention are:

[0038] The industrial area array camera is controlled to move to the area above the calibration feature and take a photo to obtain a calibration photo. The calibration feature in the calibration photo is set as the calibration mark point. The pixel field of view of the industrial area array camera is divided into blocks. The pixel coordinates of the center point of each block are equally divided according to the size of the pixel field of view to construct a point set containing the equally divided mark points. The equally divided mark points in the point set are selected in turn to make a difference with the calibration mark point. The adaptive controller outputs the control parameters of the industrial area array camera movement based on the difference to complete the adaptive control process of single-point spiral approximation until the difference meets the preset conditions. The pixel coordinates and machine coordinates of the point are recorded. The N-point calibration algorithm is executed based on the recorded pixel coordinates and machine coordinates to complete the automatic calibration process. That is, the calibration point fitting is achieved through spiral approximation through the adaptive control algorithm of the adaptive controller, so as to realize automatic calibration in the unmapped state between the machine coordinate system and the camera coordinate system without manual collaboration. Ultimately, the efficiency and personnel safety of industrial area array camera calibration are greatly improved, and the labor cost and time cost of industrial area array camera calibration are greatly reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0040] Figure 1 The present invention is a flowchart of an automatic calibration method for an industrial area array camera.

[0041] Figure 2 The present invention is a structural diagram of an automatic calibration system for industrial area array cameras.

[0042] Figure 3 It is a schematic diagram of N-point calibration of the present invention.

[0043] Figure 4 It is a schematic diagram of the principle of the adaptive controller of the present invention.

[0044] Figure 5 It is a schematic diagram of the spiral approximation calibration of the present invention. DETAILED DESCRIPTION

[0045] The technical solution in the embodiments of the present application has the following overall idea: a calibration photo is obtained by controlling the industrial area array camera to move above the calibration object and taking a photo, the position of the calibration feature in the calibration photo is set as the calibration mark point, the pixel field of view of the industrial area array camera is divided into blocks to obtain equally divided mark points, and spiral approximation calibration is performed on each equally divided mark point in turn based on the calibration mark points to automatically calibrate the industrial area array camera without the need for manual collaboration, thereby improving the efficiency and safety of industrial area array camera calibration and reducing the cost of industrial area array camera calibration.

[0046] Please refer to Figures 1 to 5As shown, a preferred embodiment of the automatic calibration method of an industrial area array camera of the present invention includes the following steps:

[0047] Step S10: The camera driver establishes TCP / IP communication with the vision host computer. The device driver controls the industrial area array camera to move to the area above the preset calibration feature and adjusts it to a suitable imaging distance. The device driver can be a multi-axis robot, a PLC, or a single-chip microcomputer.

[0048] Step S20: The industrial area array camera photographs the calibration feature to obtain a calibration photo, performs distortion correction on the calibration photo using a distortion correction algorithm, and uses the calibration feature as a calibration mark point to record the machine coordinates of the shooting position at this time. In a specific implementation, the calibration mark point can be any feature with stable and unique imaging in the shooting plane;

[0049] Step S30: Divide the pixel field of view of the industrial area array camera into blocks, perform equal division according to the size of the pixel field of view, determine the pixel coordinates of the center point of the equal division block, and construct a point set of equal division mark points;

[0050] Step S40: sequentially select equally divided Mark points in the point set and perform subtraction with the calibration Mark points. The adaptive controller outputs control parameters for the movement of the industrial area array camera based on the subtraction, completing the adaptive control process of the single-point spiral approximation. The pixel coordinates and machine coordinates of the point are recorded. The process is repeated until the points in the point set are successfully fitted and recorded.

[0051] Step S50: Execute an N-point calibration algorithm according to the recorded pixel coordinates and machine coordinates to complete the automatic calibration process.

[0052] In step S20 , during the process of taking the calibration photo, the light source is controlled to provide appropriate fill light to the imaging area of ​​the industrial area array camera.

[0053] The step S30 is specifically as follows:

[0054] The pixel field of view of the industrial area array camera is divided into blocks, and the pixel coordinates of the center point of each block are divided into nine equal parts according to the size of the pixel field of view. The center coordinates after equal division are used as equal division mark points, and a point set is constructed based on each equal division mark point.

[0055] The step S40 is specifically as follows:

[0056] Select one of the equally divided Mark points, calculate the difference g between the pixel coordinates of the equally divided Mark point and the calibration Mark point, input the difference g into the adaptive controller, and drive the device based on the X-axis control amount u output by the adaptive controller. x and Y-axis control value u yControl the industrial area array camera to move so that the calibration mark point spirally approaches the equally divided mark points in the camera field of view until the absolute value of the difference g is less than the set acceptance threshold, record the pixel coordinates and machine coordinates of the point, and repeat the process until the points in the point set are successfully fitted and recorded.

[0057] That is, under the premise that there is no mapping relationship between the unknown device coordinate system and the camera coordinate system, the adaptive controller is used to achieve the guiding effect of the calibration mark point spirally approximating the equally divided mark point.

[0058] The output formula of the adaptive controller is:

[0059] u x =ω x *(B1*g+B2*(dg / dt)+B3*∫g(t)dt);

[0060] u y =ω y *(B1*g+B2*(dg / dt)+B3*∫g(t)dt);

[0061] Among them, ω x and ω y Both represent dynamic angle adjustment weights, which are key parameters for realizing spiral approximation, and ω x =cosθ,ω y = sinθ, where θ is the angle between the current frame and the initial direction; B1, B2, and B3 are constants obtained through Bayesian optimization; and dg / dt represents the rate of change of the difference. This adaptive controller uses a low-resolution proportional output in areas with large errors, allowing for rapid spiral approximation to the target point. It also uses a higher-resolution fuzzy output in areas with smaller errors. It can also adaptively adjust the approach direction based on the difference g, enabling point fitting even when the mapping between the device coordinate system and the camera coordinate system is uncertain.

[0062] A preferred embodiment of an automatic calibration system for industrial area array cameras according to the present invention includes the following modules:

[0063] The camera control module is used to establish TCP / IP communication between the camera driver and the vision host computer. The device driver controls the industrial area array camera to move to the area above the preset calibration feature and adjusts it to the appropriate imaging distance. The device driver can be a multi-axis robot, PLC, or single-chip microcomputer.

[0064] The distortion correction module is used to take a picture of the calibration feature with an industrial area array camera to obtain a calibration picture, perform distortion correction on the calibration picture using a distortion correction algorithm, and use the calibration feature as a calibration mark point to record the machine coordinates of the shooting position at that time. In a specific implementation, the calibration mark point can be any feature with stable and unique imaging in the shooting plane;

[0065] The point set construction module is used to divide the pixel field of view of the industrial area array camera into blocks, perform equal division operations according to the size of the pixel field of view, determine the pixel coordinates of the center point of the equal division block, and construct a point set of the equally divided mark points;

[0066] An adaptive control module is configured to sequentially select equally divided Mark points within the point set and perform subtraction with calibration Mark points. An adaptive controller outputs control parameters for the movement of the industrial area array camera based on the subtraction, completing the adaptive control process of a single-point spiral approximation, recording the pixel coordinates and machine coordinates of the point, and repeating the process until the point positions within the point set are successfully fitted and recorded.

[0067] The calibration module is used to execute an N-point calibration algorithm according to the recorded pixel coordinates and machine coordinates to complete an automatic calibration process.

[0068] In the distortion correction module, during the process of taking the calibration photo, the light source is controlled to provide appropriate fill light to the imaging area of ​​the industrial area array camera.

[0069] The point set construction module is specifically used for:

[0070] The pixel field of view of the industrial area array camera is divided into blocks, and the pixel coordinates of the center point of each block are divided into nine equal parts according to the size of the pixel field of view. The center coordinates after equal division are used as equal division mark points, and a point set is constructed based on each equal division mark point.

[0071] The adaptive control module is specifically used for:

[0072] Select one of the equally divided Mark points, calculate the difference g between the pixel coordinates of the equally divided Mark point and the calibration Mark point, input the difference g into the adaptive controller, and drive the device based on the X-axis control amount u output by the adaptive controller. x and Y-axis control value u y Control the industrial area array camera to move so that the calibration mark point spirally approaches the equally divided mark points in the camera field of view until the absolute value of the difference g is less than the set acceptance threshold, record the pixel coordinates and machine coordinates of the point, and repeat the process until the points in the point set are successfully fitted and recorded.

[0073] That is, under the premise that there is no mapping relationship between the unknown device coordinate system and the camera coordinate system, the adaptive controller is used to achieve the guiding effect of the calibration mark point spirally approximating the equally divided mark point.

[0074] The output formula of the adaptive controller is:

[0075] u x =ω x *(B1*g+B2*(dg / dt)+B3*∫g(t)dt);

[0076] u y =ω y *(B1*g+B2*(dg / dt)+B3*∫g(t)dt);

[0077] Among them, ω x and ω y Both represent dynamic angle adjustment weights, which are key parameters for realizing spiral approximation, and ω x =cosθ,ω y = sinθ, where θ is the angle between the current frame and the initial direction; B1, B2, and B3 are constants obtained through Bayesian optimization; and dg / dt represents the rate of change of the difference. This adaptive controller uses a low-resolution proportional output in areas with large errors, allowing for rapid spiral approximation to the target point. It also uses a higher-resolution fuzzy output in areas with smaller errors. It can also adaptively adjust the approach direction based on the difference g, enabling point fitting even when the mapping between the device coordinate system and the camera coordinate system is uncertain.

[0078] In summary, the advantages of the present invention are:

[0079] The industrial area array camera is controlled to move to the area above the calibration feature and take a photo to obtain a calibration photo. The calibration feature in the calibration photo is set as the calibration mark point. The pixel field of view of the industrial area array camera is divided into blocks. The pixel coordinates of the center point of each block are equally divided according to the size of the pixel field of view to construct a point set containing the equally divided mark points. The equally divided mark points in the point set are selected in turn to make a difference with the calibration mark point. The adaptive controller outputs the control parameters of the industrial area array camera movement based on the difference to complete the adaptive control process of single-point spiral approximation until the difference meets the preset conditions. The pixel coordinates and machine coordinates of the point are recorded. The N-point calibration algorithm is executed based on the recorded pixel coordinates and machine coordinates to complete the automatic calibration process. That is, the calibration point fitting is achieved through spiral approximation through the adaptive control algorithm of the adaptive controller, so as to realize automatic calibration in the unmapped state between the machine coordinate system and the camera coordinate system without manual collaboration. Ultimately, the efficiency and personnel safety of industrial area array camera calibration are greatly improved, and the labor cost and time cost of industrial area array camera calibration are greatly reduced.

[0080] Although the specific embodiments of the present invention are described above, those skilled in the art should understand that the specific embodiments described are merely illustrative and are not intended to limit the scope of the present invention. Equivalent modifications and changes made by those skilled in the art in accordance with the spirit of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. An automatic calibration method for industrial area array cameras, characterized by: The steps include: Step S10: The camera driver establishes TCP / IP communication with the visual host computer. The device driver controls the industrial area array camera to move to the area above the preset calibration feature and adjusts it to a suitable imaging distance. Step S20: The industrial area array camera takes a picture of the calibration feature to obtain a calibration picture, performs distortion correction on the calibration picture using a distortion correction algorithm, uses the calibration feature as a calibration mark point, and records the machine coordinates of the shooting position at this time; Step S30: Divide the pixel field of view of the industrial area array camera into blocks, perform equal division according to the size of the pixel field of view, determine the pixel coordinates of the center point of the equal division block, and construct a point set of equal division mark points; Step S40: sequentially select equally divided Mark points in the point set and perform subtraction with the calibration Mark points. The adaptive controller outputs control parameters for the movement of the industrial area array camera based on the subtraction, completing the adaptive control process of the single-point spiral approximation. The pixel coordinates and machine coordinates of the point are recorded. The process is repeated until the points in the point set are successfully fitted and recorded. Step S50: executing an N-point calibration algorithm according to the recorded pixel coordinates and machine coordinates to complete the automatic calibration process; The step S40 is specifically as follows: Select one of the equally divided Mark points, calculate the difference g between the pixel coordinates of the equally divided Mark point and the calibration Mark point, input the difference g into the adaptive controller, and drive the device based on the X-axis control value output by the adaptive controller. and Y-axis control amount Control the industrial area array camera to move so that the calibration mark point spirally approaches the equally divided mark points in the camera field of view until the absolute value of the difference g is less than the set acceptance threshold, record the pixel coordinates and machine coordinates of the point, and repeat the process until the points in the point set are successfully fitted and recorded; The output formula of the adaptive controller is: ； ； in, and Both represent dynamic angle adjustment weights, and , , is the angle between the current frame and the initial direction; 、 、 Represents the constant obtained through Bayesian optimization parameter adjustment; Indicates the rate of change of the difference.

2. The automatic calibration method for an industrial area array camera according to claim 1, wherein: In step S20 , during the process of taking the calibration photo, the light source is controlled to provide appropriate fill light to the imaging area of ​​the industrial area array camera.

3. The automatic calibration method for an industrial area array camera according to claim 1, wherein: The step S30 is specifically as follows: The pixel field of view of the industrial area array camera is divided into blocks, and the pixel coordinates of the center point of each block are divided into nine equal parts according to the size of the pixel field of view. The center coordinates after equal division are used as equal division mark points, and a point set is constructed based on each equal division mark point.

4. An automatic calibration system for industrial area array cameras, characterized by: Includes the following modules: The camera control module is used to establish TCP / IP communication between the camera driver and the visual host computer. The device driver controls the industrial area array camera to move to the area above the preset calibration feature and adjust it to the appropriate imaging distance. The distortion correction module is used to take a picture of the calibration feature with an industrial area array camera to obtain a calibration photo, perform distortion correction on the calibration photo using a distortion correction algorithm, and use the calibration feature as a calibration mark point to record the machine coordinates of the shooting position at this time; The point set construction module is used to divide the pixel field of view of the industrial area array camera into blocks, perform equal division operations according to the size of the pixel field of view, determine the pixel coordinates of the center point of the equal division block, and construct a point set of the equally divided mark points; An adaptive control module is configured to sequentially select equally divided Mark points within the point set and perform subtraction with calibration Mark points. An adaptive controller outputs control parameters for the movement of the industrial area array camera based on the subtraction, completing the adaptive control process of a single-point spiral approximation, recording the pixel coordinates and machine coordinates of the point, and repeating the process until the point positions within the point set are successfully fitted and recorded. A calibration module, configured to execute an N-point calibration algorithm based on the recorded pixel coordinates and machine coordinates to complete an automatic calibration process; The adaptive control module is specifically used for: Select one of the equally divided Mark points, calculate the difference g between the pixel coordinates of the equally divided Mark point and the calibration Mark point, input the difference g into the adaptive controller, and drive the device based on the X-axis control value output by the adaptive controller. and Y-axis control amount Control the industrial area array camera to move so that the calibration mark point spirally approaches the equally divided mark points in the camera field of view until the absolute value of the difference g is less than the set acceptance threshold, record the pixel coordinates and machine coordinates of the point, and repeat the process until the points in the point set are successfully fitted and recorded; The output formula of the adaptive controller is: ； ； in, and Both represent dynamic angle adjustment weights, and , , is the angle between the current frame and the initial direction; 、 、 Represents the constant obtained through Bayesian optimization parameter adjustment; Indicates the rate of change of the difference.

5. The automatic calibration system for industrial area array cameras according to claim 4, characterized in that: In the distortion correction module, during the process of taking the calibration photo, the light source is controlled to provide appropriate fill light to the imaging area of ​​the industrial area array camera.

6. The automatic calibration system for industrial area array cameras according to claim 4, wherein: The point set construction module is specifically used for: The pixel field of view of the industrial area array camera is divided into blocks, and the pixel coordinates of the center point of each block are divided into nine equal parts according to the size of the pixel field of view. The center coordinates after equal division are used as equal division mark points, and a point set is constructed based on each equal division mark point.

Images