A joint calibration device and method based on a line imaging combination system

Abstract

The present invention relates to a joint calibration device and method based on a line imaging combination system, which solves the problems of the need to use third-party tools or the cumbersome calibration process caused by the single calibration category during the calibration of line imaging units. The device sets multiple target light points on the light stand, and a number of line imaging units are set along the horizontal arc on the front side of the light stand, wherein a vertically arranged middle imaging unit is set in the middle, and a number of horizontally arranged line imaging units are respectively arranged on both sides of the middle imaging unit. The calibration method of the device includes 1. establishing a one-dimensional imaging relationship, 2. establishing a high-dimensional nonlinear equation, 3. solving the effect of the internal and external parameters of the line imaging combination system, 4. using reprojection distortion correction, 5. reconstructing the three-dimensional coordinates of the target light points to achieve the effect of measuring the position and posture of the target object. The multiple line imaging units of the present invention receive the one-dimensional coordinate information of each lit target light point, and realize the joint calibration of multiple line imaging units after calculation.

Description

Technical Field

[0001] The present invention belongs to the field of machine vision and relates to a camera calibration device and method, and in particular to a joint calibration device and method based on a line imaging combination system. Background Art

[0002] Visual positioning is an important technology in the field of machine vision. In visual positioning applications, camera calibration is necessary to determine the relationship between the world coordinates of a point on the surface of a spatial object and the corresponding point in the image. Camera calibration is one of the key technologies in machine vision, and the accuracy of camera calibration directly affects the accuracy of visual positioning results. The line imaging unit (i.e., the measurement unit of the linear array image sensor) has only one column of pixels and therefore only outputs one-dimensional image information, achieving characteristics such as wide field of view, high resolution, and high sampling frequency. However, due to the particularity of single-column imaging of the line imaging unit, there are few solutions for high-precision calibration of the line imaging unit, and it is time-consuming. Therefore, establishing a simple and effective constraint equation based on the imaging model of the line imaging unit has practical research value and practical application value.

[0003] The existing calibration system for multi-line imaging units requires another auxiliary measurement tool, such as a laser tracker. The calibration is not easy and can only be limited to a specific calibration system. The calibration scenario is inconsistent with the actual application scenario.

[0004] Regarding the related technologies, since the calibration of the line imaging unit requires additional auxiliary measurement tools, the measurement field is single and there is currently no effective solution. Summary of the Invention

[0005] The purpose of the present invention is to solve the problems of the need to use third-party tools or the cumbersome calibration process caused by the single calibration category during the calibration of line imaging units, and to provide a joint calibration device and method based on a line imaging combination system, including a calibration device and a calibration method of the line imaging combination system, to realize the joint calibration of multiple line imaging units.

[0006] The technical solution adopted by the present invention to solve its technical problems is: a joint calibration device based on a line imaging combination system, characterized in that it includes a vertically arranged support frame, a lamp holder mounting plate is arranged on the upper part of the support frame facing one side, a lamp holder is fixed on the lamp holder mounting plate, a plurality of supporting legs are provided on the lamp holder, and one or more target lights are respectively provided on each supporting leg, a line imaging combination system is arranged on the outside of the lamp holder, the line imaging combination system includes a plurality of line imaging units arranged along a horizontal arc, a vertically arranged middle imaging unit is arranged in the middle, and a plurality of horizontally arranged line imaging units are respectively arranged on both sides of the middle imaging unit, and each line imaging unit is sequentially line imaging unit 1, line imaging unit 1...line imaging unit N.

[0007] Preferably, the lamp stand is mounted on a lamp stand mounting plate using a rotatable and adjustable fixed angle indexing plate, the legs of the lamp stand are of different lengths, and some of the legs are provided with concave and convex positions for mounting target lights on the side facing the line imaging combination system.

[0008] Preferably, the support frame is provided with a vertical slide groove, and the lamp holder mounting plate is installed in the vertical slide groove by using a sliding connector.

[0009] Preferably, a base is provided at the bottom of the support frame, and the base includes a bottom plate and an electrical box fixed on the upper surface of the bottom plate. A power module, a single-chip microcomputer and a relay are arranged in the electrical box. The bottom plate is provided with multiple groups of fixing holes for fixing the electrical box. The electrical box can be fixed at different groups of fixing holes using locking blocks and locking quick-release pins.

[0010] A joint calibration method based on a line imaging combination system, using the above-mentioned joint calibration device, includes the following steps:

[0011] Step 1: Use the geometric characteristics of the line imaging unit to obtain the one-dimensional coordinate value of the target light point in the line imaging combination system, extract the one-dimensional coordinate at the center of the imaging spot, and establish a one-dimensional imaging relationship;

[0012] Step 2: Using the matrix association between the line imaging units, a high-dimensional nonlinear equation based on the line imaging combination system is obtained;

[0013] Step 3: Using the LM optimization algorithm based on the initial value of PSO to solve the high-dimensional nonlinear equation obtained in step 2, so as to simultaneously solve the internal and external parameter coefficients of the line imaging combination system;

[0014] Step 4: Using reprojection distortion correction, a multi-coefficient distortion term is introduced to further correct the repositioning error, thereby obtaining a more accurate internal and external distortion coefficient of the line imaging combination system.

[0015] Step 5: Use the calibrated internal and external parameter coefficients and distortion coefficients of the line imaging combination system to reconstruct the three-dimensional coordinates of the target light point to achieve the effect of measuring the position and posture of the target object.

[0016] Preferably, the one-dimensional imaging position relationship representation method in step 1 is:

[0017] Step 1.1: Use a measurement system composed of multiple line imaging units. The line imaging unit is a measurement unit of a linear array image sensor that outputs one-dimensional image information when the target light point is lit. For the intermediate imaging unit, during calibration, the target light points in space are polled and lit. A certain light point P1 forms a linear light spot L1 on the imaging plane of the intermediate imaging unit, which is perpendicular to the horizontal plane and parallel to the optical center line of the cylindrical lens. The one-dimensional coordinates of the point light source on the intermediate imaging unit are obtained by extracting the center coordinates of the light spot. The one-dimensional coordinates of each of the line imaging units 1, 2, ..., N are extracted in the same manner as for the intermediate imaging unit. The light point P1 forms a linear light spot on the imaging plane of the line imaging units 1, 2, ..., N, which is parallel to the horizontal plane and the optical center line of the cylindrical lens. The one-dimensional coordinates of the light point P1 on the line imaging units 1, 2, ..., N are extracted by extracting the center coordinates of the light spot. The one-dimensional coordinates of the remaining light points are extracted in the same manner as for the light point P1.

[0018] Step 1.2: Poll and light up the replaceable target lights, and each line imaging unit extracts the one-dimensional coordinates of its own light spot center. Taking light point P1 as the first target light point as an example, the one-dimensional coordinate calibration equation of the middle line imaging unit is:

[0019]

[0020] Where λ 1z is the one-dimensional coordinate value of the first target light point on the midline imaging unit, λ Z is the zero point error of the intermediate line imaging unit, f z is the focal length of the midline imaging unit, Y1 and Z1 are the coordinates of target light point 1 in the midline imaging unit coordinate system; for target light point j, the one-dimensional coordinate calibration equation of the midline imaging unit is:

[0021]

[0022] Preferably, the method for establishing the high-dimensional nonlinear equation in step 2 is:

[0023] Step 2.1: Taking the first target light point as an example, the focus of the middle line imaging unit is the world coordinate system O-XYZ, and the coordinate value of point P1 in this coordinate system is (X1, Y1, Z1); the remaining line imaging units on both sides are local coordinate systems; taking the left line imaging unit 1 as an example, the origin O' is located at the focus of the line imaging unit 1, the X' axis is parallel to the imaging direction of the line imaging unit 1, and the Z' axis is perpendicular to the imaging plane of the line imaging unit 1; the Y' axis is obtained from the X' and Z' axis directions using the right-hand rule; the coordinates of the object point in the coordinate system of the first target light point line imaging unit 1 are:

[0024]

[0025] Where X1, Y1, and Z1 are the coordinate values of the first target light point in the world coordinate system, X1′, Y1′, and Z1′ are the coordinate values of the first target light point in the coordinate system of the line imaging unit 1, and X s1 、Y s1 , Z s1 is the translation matrix between coordinate systems, R is the rotation matrix, expressed as:

[0026]

[0027] Where y, p, and r are the rotation angles of the coordinate system around the z-axis, y-axis, and x-axis, respectively;

[0028] Step 2.2: Use this coordinate transformation method to transform the coordinates of the line imaging units on both sides. Taking the first target light point as an example, the multiple coordinate constraint equations obtained by combining all the line imaging units are:

[0029]

[0030]

[0031] Where λ 1i is the one-dimensional coordinate value of the first target light point in the i-th line imaging unit, λ i is the zero point error in the i-th line imaging unit, a 1i 、b 1i 、c 1i 、a 3i 、b 3i 、c 3i is the rotation matrix parameter of the i-th line imaging unit, X si 、Y si , Z si is the translation matrix parameter of the i-th line imaging unit, for the j-th target light point,

[0032] Step 2.3: Using the known distances between the target lights on the light stand, we can obtain the distance constraint equations for multiple sets of calibration points:

[0033] D j =sqrt((X j -X j+1 ) 2 +(Y j -Y j+1 ) 2 +(Z j -Z j+1 ) 2 )

[0034] Where D jis the distance between the jth target light point and the j+1th target light point; the calibration device is moved to multiple positions in the measurement field and fixed. After the device is stable, the control system lights up the lights in sequence to obtain multiple sets of distance constraint equations and coordinate constraint equations. The above equations are combined to establish a high-dimensional nonlinear constraint equation.

[0035] Preferably, the method for solving the high-dimensional nonlinear equation in step 3 is:

[0036] Step 3.1: Use the PSO particle swarm algorithm to roughly obtain a set of global optimal solutions. Initial values are set in eight directions in the spatial coordinate system. The PSO iterative update is as follows: Initialize the PSO algorithm parameters and set N iterative particles; further, calculate the fitness value of each particle, that is, the error of the high-dimensional equation; set the best position of the current particle, set the best position among them, update the speed and position, and obtain a set of new particle positions. The calculation formula is as follows:

[0037]

[0038] Where, is the velocity of the nth particle at the k+1th iteration, x b is the current optimal position of the particle, x g is the global optimal position of all particles, is the position of the nth particle at the kth iteration, c1 and c2 are learning factors, r1 and r2 are random numbers, and w is the inertia weight;

[0039] Calculate the fitness value of the new particle. If the conditions are not met, jump back to the previous step to continue iteration, and use the best global position result in the eight directions as the initial value of the LM algorithm.

[0040] Step 3.2: Based on this iterative method, a global search is performed on the error function of the coordinates and the internal and external parameter coefficients. The parameter value with the minimum error function value is used as the initial value of the LM algorithm. The two algorithms are iteratively used to simultaneously solve the internal and external parameter coefficients of the camera system. The error function is as follows: F(f z ,λ z ,f i ,λ i ,X j ,Y j ,Z j ,a 1i ,b 1i ,c 1i ,a 3i ,b 3i ,c 3i ,X si ,Y si ,Z si)=0, all the internal and external parameter coefficients of the line imaging combination system can be solved by the LM algorithm based on the PSO initial value fitting.

[0041] Preferably, the error distortion compensation method in step 4 is:

[0042] Step 4.1: In the measurement system of this paper, the line imaging unit has only one-dimensional imaging information, that is, imaging only in one direction. Define the ideal image physical coordinate of the target light point in the line imaging unit as x. Taking the first target light point as an example, the actual image physical coordinate on the line imaging unit is λ 1i , and its distortion model is:

[0043]

[0044] Where δ is the distortion error, λ 1i is the one-dimensional coordinate value on the i-th line imaging unit, k i is the error distortion coefficient of the i-th line imaging unit, is the square of the distance between the imaging spot and the imaging center, which can be expressed as λ in the line imaging unit 1i Therefore, this formula can be expressed as

[0045] Step 4.2: Taking the calibration equation of the first target light point as an example, the calibration equation after adding the distortion coefficient is:

[0046]

[0047]

[0048] Where k z is the distortion error coefficient of the middle line imaging unit; the distortion term is introduced into the calibration equation of all line imaging units, and the compensation equation is solved using the LM optimization algorithm to obtain more accurate internal and external parameter coefficients of the multi-line imaging unit. The calibration equation for the target light point j is:

[0049]

[0050]

[0051] Preferably, the three-dimensional coordinate reconstruction method in step 5 is:

[0052] Step 5.1: After the system is calibrated, the position and posture changes of the equipment in real-world situations can be measured. In this process, the system captures the information of the target light point fixed on the equipment and jointly solves the data of the image formed by the target light point in multiple line imaging units. Based on the correspondence between the one-dimensional coordinates of the pixels of multiple line imaging units in the system and the world coordinate system, taking the first target light point as an example, the following equation can be obtained:

[0053]

[0054]

[0055] The solution of these multiple plane equations (X1, Y1, Z1) is the world coordinate of the target being measured. In this equation, X1, Y1, Z1 are the three-dimensional coordinates of the target beacon light point that need to be solved. The other coefficients are known and represent the internal and external parameter coefficients of the three line imaging units in the system after calibration. Solving this equation can obtain the three-dimensional coordinates of the reconstructed target light point, that is, the three-dimensional reconstruction of the target light point is completed. The reconstruction equation of target light point j is:

[0056]

[0057]

[0058] In the present invention, multiple line imaging units receive the one-dimensional coordinate information of each lit target light point, establish a nonlinear calibration equation, and then use the LM optimization algorithm formulated based on the PSO initial value to solve the internal and external parameter coefficients of the multi-line imaging units. The reliability of the internal and external parameter coefficients is increased by using reprojection distortion correction. The obtained internal and external parameter coefficients and distortion coefficients are used to measure the three-dimensional coordinates and target posture of the new target light point, realize three-dimensional reconstruction, and realize the joint calibration of multiple line imaging units. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The present invention will be further described below with reference to the accompanying drawings.

[0060] Figure 1 It is a structural schematic diagram of a calibration device of the present invention.

[0061] Figure 2 It is a schematic diagram of a lamp stand supporting structure of the present invention.

[0062] Figure 3 This is a schematic diagram of the structure of a lamp stand of the present invention.

[0063] Figure 4 It is a schematic diagram of a base structure of the present invention.

[0064] Figure 5 It is a structural schematic diagram of a calibration process of the present invention.

[0065] Figure 6 It is a schematic diagram of a calibration step of the present invention.

[0066] In the figure: 1. Support frame, 2. Lamp frame, 3. Base, 11. Right-angle connecting block, 12. Support rod, 13. Crossbeam bracket, 14. Lamp frame mounting plate, 15. Sliding connector, 16. Cable management trough, 21. Lower quick release, 22. Upper quick release, 23. Lock nut, 24. Flange bolt, 25. Spring, 26. Target light, 27. Star bracket, 28. Fixed angle indexing plate, 31. Bottom plate, 32. Electrical box, 33. Relay, 34. Single-chip microcomputer, 35. Locking quick release pin, 36. Locking block, 37. Universal wheel, 38. Handle, 39. Start / stop button, 310. Power module, 311. Card block, 312. Fixing hole. DETAILED DESCRIPTION

[0067] The present invention will be further described below through specific embodiments in conjunction with the accompanying drawings.

[0068] Example 1: A joint calibration device based on a line imaging combination system, such as Figure 1-4 As shown. Figure 1 As shown, the device includes a vertically arranged support frame 1, the bottom of the support frame is fixed on a movable base 3, a lamp holder mounting plate is provided on one side of the upper part of the support frame 1, and four lamp holders 2 are fixed on the lamp holder mounting plate.

[0069] The structure of the support frame 1 is as follows Figure 2 As shown, the lamp holder includes a vertically mounted support rod 12, a hollow profile with a cable management groove 16 on the back for easy cable routing. The bottom of the support rod 12 is secured to the base 3 via a right-angled connector 11. A vertical slide is provided on the support frame, and a lamp holder mounting plate 14 is mounted within the vertical slide using a sliding connector 15. The lamp holder mounting plate 14 is adjustable in height. Crossbeam brackets 13 are provided at the top and bottom of the front side of the plate for mounting lamp holders 2. A lamp holder 2 is mounted at each end of the crossbeam bracket.

[0070] The structure of lamp stand 2 is as follows Figure 3As shown, the main body of the light stand is a star-shaped bracket 27, which is equipped with several legs. Each right-angled end is connected to an upper quick-release member 22 and a lower quick-release member 21. The upper quick-release member 22 is ahead of and mounts one or more target lights 26. A mounting space for the target lights 26 is formed between the lower quick-release member 21 and the upper quick-release member 22. The upper and lower quick-release members are available in a variety of sizes. The upper quick-release member 22 can be flat or have a concave-convex stepped surface, with target lights 26 mounted on the stepped surfaces at different heights. A central hole is provided in the middle of the star-shaped bracket 27, and a fixed-angle indexing plate 28 is provided on the back side of the center of the star-shaped bracket 27. The fixed-angle indexing plate 28 is fixedly connected to the beam bracket 13 by a lock nut 23. The central hole of the star-shaped bracket 27 is connected to the fixed-angle indexing plate 28 by a flange bolt 24. A spring 25 is also provided between the end of the flange bolt 24 and the star-shaped bracket 27 to facilitate the rotation adjustment of the star bracket 27 and the fixed-angle indexing plate 28.

[0071] The structure of base 3 is as follows Figure 4 As shown, the base 3 includes a bottom plate 31 and an electrical box 32 fixed to the upper surface of the bottom plate. A power module 310, a single-chip microcomputer 34, and a relay 33 are arranged in the electrical box. The power module is clamped by a clamping block 311. The bottom plate 31 is provided with multiple groups of fixing holes 312 for fixing the electrical box 32. The electrical box 32 can be fixed at different groups of fixing holes using locking blocks 36 and locking quick-release pins 35. The top surface of the electrical box 32 is used to fix the support rod 12. Handles 38 are provided on both sides of the top surface of the electrical box 32, and the electrical box can be moved to adjust the installation position. A start-stop button is provided on the side wall of the electrical box. Universal wheels 37 are provided at the four corners of the bottom surface of the bottom plate 31.

[0072] This device is used for joint calibration of multi-line imaging units. Figure 5 As shown, a line imaging combination system is provided on the front side of the lamp stand 2, and the line imaging combination system includes a plurality of line imaging units arranged along a horizontal arc, wherein a vertically arranged middle imaging unit is provided in the middle, and a plurality of horizontally arranged line imaging units are respectively provided on both sides of the middle imaging unit, and the line imaging units are sequentially line imaging unit 1, line imaging unit 1...line imaging unit N.

[0073] Example 2: A joint calibration method based on a line imaging combination system, such as Figure 6 As shown, the combined calibration device in Example 1 includes the following steps:

[0074] Step 1: Use the geometric characteristics of the line imaging unit to obtain the one-dimensional coordinate value of the target light point in the line imaging combination system, extract the one-dimensional coordinate at the center of the imaging spot, and establish a one-dimensional imaging relationship;

[0075] Step 1.1: Use a measurement system composed of multiple line imaging units. The line imaging unit is a measurement unit of a linear array image sensor that outputs one-dimensional image information when the target light point is lit. For the intermediate imaging unit, during calibration, the target light points in space are polled and lit. A certain light point P1 forms a linear light spot L1 on the imaging plane of the intermediate imaging unit, which is perpendicular to the horizontal plane and parallel to the optical center line of the cylindrical lens. The one-dimensional coordinates of the point light source on the intermediate imaging unit are obtained by extracting the center coordinates of the light spot. The one-dimensional coordinates of each of the line imaging units 1, 2, ..., N are extracted in the same manner as for the intermediate imaging unit. The light point P1 forms a linear light spot on the imaging plane of the line imaging units 1, 2, ..., N, which is parallel to the horizontal plane and the optical center line of the cylindrical lens. The one-dimensional coordinates of the light point P1 on the line imaging units 1, 2, ..., N are extracted by extracting the center coordinates of the light spot. The one-dimensional coordinates of the remaining light points are extracted in the same manner as for the light point P1.

[0076] Step 1.2: Poll and light up the replaceable target lights, and each line imaging unit extracts the one-dimensional coordinates of its own light spot center. Taking light point P1 as the first target light point as an example, the one-dimensional coordinate calibration equation of the middle line imaging unit is:

[0077]

[0078] Where λ 1z is the one-dimensional coordinate value of the first target light point on the midline imaging unit, λ Z is the zero point error of the intermediate line imaging unit, f z is the focal length of the midline imaging unit, Y1 and Z1 are the coordinates of target light point 1 in the midline imaging unit coordinate system; for target light point j, the one-dimensional coordinate calibration equation of the midline imaging unit is:

[0079]

[0080] Step 2: Using the matrix association between the line imaging units, a high-dimensional nonlinear equation based on the line imaging combination system is obtained;

[0081] Step 2.1: Taking the first target light point as an example, the focus of the middle line imaging unit is the world coordinate system O-XYZ, and the coordinate value of point P1 in this coordinate system is (X1, Y1, Z1); the remaining line imaging units on both sides are local coordinate systems; taking the left line imaging unit 1 as an example, the origin O' is located at the focus of the line imaging unit 1, the X' axis is parallel to the imaging direction of the line imaging unit 1, and the Z' axis is perpendicular to the imaging plane of the line imaging unit 1; the Y' axis is obtained from the X' and Z' axis directions using the right-hand rule; the coordinates of the object point in the coordinate system of the first target light point line imaging unit 1 are:

[0082]

[0083] Where X1, Y1, and Z1 are the coordinate values of the first target light point in the world coordinate system, X1′, Y1′, and Z1′ are the coordinate values of the first target light point in the coordinate system of the line imaging unit 1, and X s1 、Y s1 , Z s1 is the translation matrix between coordinate systems, R is the rotation matrix, expressed as:

[0084]

[0085] Where y, p, and r are the rotation angles of the coordinate system around the z-axis, y-axis, and x-axis, respectively;

[0086] Step 2.2: Use this coordinate transformation method to transform the coordinates of the line imaging units on both sides. Taking the first target light point as an example, the multiple coordinate constraint equations obtained by combining all the line imaging units are:

[0087]

[0088]

[0089] Where λ 1i is the one-dimensional coordinate value of the first target light point in the i-th line imaging unit, λ i is the zero point error in the i-th line imaging unit, a 1i 、b 1i 、c 1i 、a 3i 、b 3i 、c 3i is the rotation matrix parameter of the i-th line imaging unit, X si 、Y si , Z si is the translation matrix parameter of the i-th line imaging unit, for the j-th target light point,

[0090] Step 2.3: Using the known distances between the target lights on the light stand, we can obtain the distance constraint equations for multiple sets of calibration points:

[0091] D j =sqrt((X j -X j+1 ) 2 +(Y j -Y j+1 ) 2 +(Z j -Z j+1 ) 2 )

[0092] Where D jis the distance between the jth target light point and the j+1th target light point; the calibration device is moved to multiple positions in the measurement field and fixed. After the device is stable, the control system lights up the lights in sequence to obtain multiple sets of distance constraint equations and coordinate constraint equations. The above equations are combined to establish a high-dimensional nonlinear constraint equation.

[0093] Step 3: Using the LM optimization algorithm based on the initial value of PSO to solve the high-dimensional nonlinear equation obtained in step 2, so as to simultaneously solve the internal and external parameter coefficients of the line imaging combination system;

[0094] Step 3.1: Use the PSO particle swarm algorithm to roughly obtain a set of global optimal solutions. Initial values are set in eight directions in the spatial coordinate system. The PSO iterative update is as follows: Initialize the PSO algorithm parameters and set N iterative particles; further, calculate the fitness value of each particle, that is, the error of the high-dimensional equation; set the best position of the current particle, set the best position among them, update the speed and position, and obtain a set of new particle positions. The calculation formula is as follows:

[0095]

[0096] Where, is the velocity of the nth particle at the k+1th iteration, x b is the current optimal position of the particle, x g is the global optimal position of all particles, is the position of the nth particle at the kth iteration, c1 and c2 are learning factors, r1 and r2 are random numbers, and w is the inertia weight;

[0097] Calculate the fitness value of the new particle. If the conditions are not met, jump back to the previous step to continue iteration, and use the best global position result in the eight directions as the initial value of the LM algorithm.

[0098] Step 3.2: Based on this iterative method, a global search is performed on the error function of the coordinates and the internal and external parameter coefficients. The parameter value with the minimum error function value is used as the initial value of the LM algorithm. The two algorithms are iteratively used to simultaneously solve the internal and external parameter coefficients of the camera system. The error function is as follows: F(f z ,λ z ,f i ,λ i ,X j ,Y j ,Z j ,a 1i ,b 1i ,c 1i ,a 3i ,b 3i ,c 3i ,X si ,Y si ,Zsi )=0, all the internal and external parameter coefficients of the line imaging combination system can be solved by the LM algorithm based on the PSO initial value fitting.

[0099] Step 4: Using reprojection distortion correction, a multi-coefficient distortion term is introduced to further correct the repositioning error, thereby obtaining a more accurate internal and external distortion coefficient of the line imaging combination system.

[0100] Step 4.1: In the measurement system of this paper, the line imaging unit has only one-dimensional imaging information, that is, imaging only in one direction. Define the ideal image physical coordinate of the target light point in the line imaging unit as x. Taking the first target light point as an example, the actual image physical coordinate on the line imaging unit is λ 1i , and its distortion model is:

[0101]

[0102] Where δ is the distortion error, λ 1i is the one-dimensional coordinate value on the i-th line imaging unit, k i is the error distortion coefficient of the i-th line imaging unit, is the square of the distance between the imaging spot and the imaging center, which can be expressed as λ in the line imaging unit 1i Therefore, this formula can be expressed as

[0103] Step 4.2: Taking the calibration equation of the first target light point as an example, the calibration equation after adding the distortion coefficient is:

[0104]

[0105]

[0106] Where k z is the distortion error coefficient of the middle line imaging unit; the distortion term is introduced into the calibration equation of all line imaging units, and the compensation equation is solved using the LM optimization algorithm to obtain more accurate internal and external parameter coefficients of the multi-line imaging unit. The calibration equation for the target light point j is:

[0107]

[0108]

[0109] Step 5: Use the calibrated internal and external parameter coefficients and distortion coefficients of the line imaging combination system to reconstruct the three-dimensional coordinates of the target light point to achieve the effect of measuring the position and posture of the target object;

[0110] Step 5.1: After the system is calibrated, the position and posture changes of the equipment in real-world situations can be measured. In this process, the system captures the information of the target light point fixed on the equipment and jointly solves the data of the image formed by the target light point in multiple line imaging units. Based on the correspondence between the one-dimensional coordinates of the pixels of multiple line imaging units in the system and the world coordinate system, taking the first target light point as an example, the following equation can be obtained:

[0111]

[0112]

[0113] The solution of these multiple plane equations (X1, Y1, Z1) is the world coordinate of the target being measured. In this equation, X1, Y1, Z1 are the three-dimensional coordinates of the target beacon light point that need to be solved. The other coefficients are known and represent the internal and external parameter coefficients of the three line imaging units in the system after calibration. Solving this equation can obtain the three-dimensional coordinates of the reconstructed target light point, that is, the three-dimensional reconstruction of the target light point is completed. The reconstruction equation of target light point j is:

[0114]

[0115]

Claims

1. A joint calibration method based on a line imaging combination system, characterized in that: A combined calibration device is used, comprising a vertically arranged support frame, a light frame mounting plate being provided on one side of the upper portion of the support frame, a light frame being fixed to the light frame mounting plate, a plurality of legs being provided on the light frame, each leg being provided with one or more target lights, a line imaging combination system being provided on the outer side of the light frame, the line imaging combination system comprising a plurality of line imaging units arranged along a horizontal arc, a vertically arranged middle line imaging unit being provided in the middle, and a plurality of horizontally arranged line imaging units being provided on either side of the middle line imaging unit, the line imaging units being sequentially designated as line imaging unit 1, line imaging unit 2, ..., line imaging unit N; The following steps are involved: Step 1: Use the geometric characteristics of the line imaging unit to obtain the one-dimensional coordinate value of the target light point in the line imaging combination system, extract the one-dimensional coordinate at the center of the imaging spot, and establish a one-dimensional imaging relationship; Step 2: Using the matrix association between the line imaging units, a high-dimensional nonlinear equation based on the line imaging combination system is obtained; Step 3: Using the LM optimization algorithm based on the initial value of PSO to solve the high-dimensional nonlinear equation obtained in step 2, so as to simultaneously solve the internal and external parameter coefficients of the line imaging combination system; Step 4: Using reprojection distortion correction, a multi-coefficient distortion term is introduced to further correct the repositioning error, thereby obtaining a more accurate internal and external distortion coefficient of the line imaging combination system. Step 5: Use the calibrated internal and external parameter coefficients and distortion coefficients of the line imaging combination system to reconstruct the three-dimensional coordinates of the target light point to achieve the effect of measuring the position and posture of the target object.

2. The joint calibration method based on the line imaging combination system according to claim 1, characterized in that: The one-dimensional imaging position relationship representation method in step 1 is: Step 1.1: Use a measurement system composed of multiple line imaging units. The line imaging unit is a measurement unit of a linear array image sensor that outputs one-dimensional image information when the target light point is lit. For the intermediate line imaging unit, during calibration, the target light points in space are polled and lit. A certain light point P1 forms a linear light spot L1 on the imaging plane of the intermediate line imaging unit, which is perpendicular to the horizontal plane and parallel to the optical center line of the cylindrical lens. The center coordinates of the light spot are extracted to obtain the one-dimensional coordinates of the point light source on the intermediate line imaging unit. The one-dimensional coordinates of line imaging units 1, 2, ..., N are extracted in the same manner as for the intermediate line imaging unit. Light point P1 forms a linear light spot parallel to the horizontal plane and the optical center line of the cylindrical lens on the imaging planes of line imaging units 1, 2, ..., N. The center coordinates of the light spot are extracted to obtain the one-dimensional coordinates of light point P1 on line imaging units 1, 2, ..., N. The one-dimensional coordinates of the remaining light points are extracted in the same manner as for light point P1. Step 1.2: Poll and light up the replaceable target lights, and each line imaging unit extracts the one-dimensional coordinates of its own light spot center. Taking light point P1 as the first target light point as an example, the one-dimensional coordinate calibration equation of the middle line imaging unit is: Where λ 1z is the one-dimensional coordinate value of the first target light point on the midline imaging unit, λ Z is the zero point error of the intermediate line imaging unit, f z is the focal length of the midline imaging unit, Y1 and Z1 are the coordinates of target light point 1 in the midline imaging unit coordinate system; for target light point j, the one-dimensional coordinate calibration equation of the midline imaging unit is:

3. The joint calibration method based on the line imaging combination system according to claim 2, characterized in that: The method for establishing the high-dimensional nonlinear equation in step 2 is: Step 2.1: Taking the first target light point as an example, the focus of the middle line imaging unit is the world coordinate system O-XYZ, and the coordinate value of point P1 in this coordinate system is (X1, Y1, Z1); the remaining line imaging units on both sides are local coordinate systems; taking the left line imaging unit 1 as an example, the origin O' is located at the focus of the line imaging unit 1, the X' axis is parallel to the imaging direction of the line imaging unit 1, and the Z' axis is perpendicular to the imaging plane of the line imaging unit 1; the Y' axis is obtained from the X' and Z' axis directions using the right-hand rule; the coordinates of the object point in the coordinate system of the first target light point line imaging unit 1 are: Where X1, Y1, and Z1 are the coordinate values of the first target light point in the world coordinate system, X1′, Y1′, and Z1′ are the coordinate values of the first target light point in the coordinate system of the line imaging unit 1, and X s1 、Y s1 , Z s1 is the translation matrix between coordinate systems, R is the rotation matrix, expressed as: Where y, p, and r are the rotation angles of the coordinate system around the z-axis, y-axis, and x-axis, respectively; Step 2.2: Use this coordinate transformation method to transform the coordinates of the line imaging units on both sides. Taking the first target light point as an example, the multiple coordinate constraint equations obtained by combining all the line imaging units are: Where λ 1i is the one-dimensional coordinate value of the first target light point in the i-th line imaging unit, λ i is the zero point error in the i-th line imaging unit, a 1i 、b 1i 、c 1i 、a 3i 、b 3i 、c 3i is the rotation matrix parameter of the i-th line imaging unit, X si 、Y si , Z si is the translation matrix parameter of the i-th line imaging unit, for the j-th target light point, Step 2.3: Using the known distances between the target lights on the light stand, we can obtain the distance constraint equations for multiple sets of calibration points: D j =sqrt((X j -X j+1 ) 2 +(Y j -Y j+1 ) 2 +(Z j -Z j+1 ) 2 ) Where D j is the distance between the jth target light point and the j+1th target light point; the calibration device is moved to multiple positions in the measurement field and fixed. After the device is stable, the control system lights up the lights in sequence to obtain multiple sets of distance constraint equations and coordinate constraint equations. The above equations are combined to establish a high-dimensional nonlinear constraint equation.

4. The joint calibration method based on the line imaging combination system according to claim 3, characterized in that: The solution method for the high-dimensional nonlinear equation in step 3 is: Step 3.1: Use the PSO particle swarm algorithm to roughly obtain a set of global optimal solutions. The initial values are set in eight directions in the spatial coordinate system. The PSO iterative update is as follows: Initialize the parameters of the PSO algorithm and set N iterative particles. Further, calculate the fitness value of each particle, that is, the error of the high-dimensional equation; set the current particle's best position, update the speed and position, and obtain a set of new particle positions. The calculation formula is as follows: Where, is the velocity of the nth particle at the k+1th iteration, x b is the current optimal position of the particle, x g is the global optimal position of all particles, is the position of the nth particle at the kth iteration, c1 and c2 are learning factors, r1 and r2 are random numbers, and w is the inertia weight; Calculate the fitness value of the new particle. If the conditions are not met, jump back to the previous step to continue iteration, and use the best global position result in the eight directions as the initial value of the LM algorithm. Step 3.2: Based on this iterative method, a global search is performed on the error function of the coordinates and the internal and external parameter coefficients. The parameter value with the minimum error function value is used as the initial value of the LM algorithm. The two algorithms are iteratively used to simultaneously solve the internal and external parameter coefficients of the camera system. The error function is as follows: F(f z ,λ z ,f i ,λ i ,X j ,Y j ,Z j ,a 1i ,b 1i ,c 1i ,a 3i ,b 3i ,c 3i ,X si ,Y si ,Z si )=0, all the internal and external parameter coefficients of the line imaging combination system can be solved by the LM algorithm based on the PSO initial value fitting.

5. The joint calibration method based on the line imaging combination system according to claim 4, characterized in that: The error distortion compensation method in step 4 is: Step 4.1: In the measurement system of this paper, the line imaging unit has only one-dimensional imaging information, that is, imaging only in one direction. Define the ideal image physical coordinate of the target light point in the line imaging unit as x. Taking the first target light point as an example, the actual image physical coordinate on the line imaging unit is λ 1i , and its distortion model is: Where δ is the distortion error, λ 1i is the one-dimensional coordinate value on the i-th line imaging unit, k i is the error distortion coefficient of the i-th line imaging unit, is the square of the distance between the imaging spot and the imaging center, which can be expressed as λ in the line imaging unit 1i ; Therefore, this formula can be expressed as Step 4.2: Taking the calibration equation of the first target light point as an example, the calibration equation after adding the distortion coefficient is: Where k z is the distortion error coefficient of the middle line imaging unit; the distortion term is introduced into the calibration equation of all line imaging units, and the compensation equation is solved using the LM optimization algorithm to obtain more accurate internal and external parameter coefficients of the multi-line imaging unit. The calibration equation for the target light point j is:

6. The joint calibration method based on the line imaging combination system according to claim 5, characterized in that: The three-dimensional coordinate reconstruction method in step 5 is: Step 5.1: After the system is calibrated, the position and posture changes of the equipment in real-world situations can be measured. In this process, the system captures the information of the target light point fixed on the equipment and jointly solves the data of the image formed by the target light point in multiple line imaging units. Based on the correspondence between the one-dimensional coordinates of the pixels of multiple line imaging units in the system and the world coordinate system, taking the first target light point as an example, the following equation can be obtained: The solution of these multiple plane equations (X1, Y1, Z1) is the world coordinate of the target being measured. In this equation, X1, Y1, Z1 are the three-dimensional coordinates of the target beacon light point that need to be solved. The other coefficients are known and represent the internal and external parameter coefficients of the three line imaging units in the system after calibration. Solving this equation can obtain the three-dimensional coordinates of the reconstructed target light point, that is, the three-dimensional reconstruction of the target light point is completed. The reconstruction equation of target light point j is:

7. The joint calibration method based on the line imaging combination system according to claim 1, characterized in that: The lamp stand is mounted on a lamp stand mounting plate using a rotatably adjustable fixed angle indexing plate. The legs of the lamp stand have different lengths, and some legs are provided with concave and convex positions for mounting target lights on the sides facing the line imaging combination system.

8. The joint calibration method based on the line imaging combination system according to claim 1, characterized in that: The support frame is provided with a vertical slide groove, and the lamp holder mounting plate is installed in the vertical slide groove by using a sliding connector.

9. The joint calibration method based on the line imaging combination system according to claim 1, characterized in that: A base is provided at the bottom of the support frame, and the base includes a bottom plate and an electrical box fixed on the upper surface of the bottom plate. A power module, a single-chip microcomputer and a relay are arranged in the electrical box. The bottom plate is provided with multiple groups of fixing holes for fixing the electrical box. The electrical box can be fixed at different groups of fixing holes using locking blocks and locking quick-release pins.

Images