A two-stage calibration method and device for a large field of view camera of a tunnel boring machine tool changing robot with an intermediate parameter introduced

Abstract

The application discloses a two-stage calibration method and device for a large-view-field camera of a tunnel boring machine (TBM) tool-changing robot by introducing intermediate parameters, and relates to the field of TBM automatic tool changing. The method comprises a pre-calibration stage and a final calibration stage: in the pre-calibration stage, Zhang's calibration method is used to calibrate the camera, and a camera principal point and a pixel size ratio are solved; in the final calibration stage, an intermediate parameter solving model is established, camera parameter step-by-step solving is realized, a focal length conversion model is constructed, intermediate parameters are separated, and finally all camera parameters are output. The application completes non-in-situ calibration of the large-view-field camera of the tool-changing robot in a narrow and safe area behind the cutterhead, and ensures calibration accuracy and operation convenience.

Description

Technical Field

[0001] This invention relates to the field of automatic cutter changers for tunnel boring machines, and in particular to a two-stage calibration method and apparatus for a wide field-of-view camera of a cutter changer robot for a tunnel boring machine that introduces intermediate parameters. Background Technology

[0002] Tunnel boring machines (TBMs) are large-scale specialized equipment for tunnel engineering, integrating mechanical, electrical, hydraulic, and optical technologies. They possess functions covering the entire tunnel construction process, including excavation, muck removal, and lining. However, difficulties in cutter head inspection and replacement severely restrict the safety and efficiency of tunnel construction. Cutter head changing robots based on vision technology have emerged to address this need. Visual positioning and wear measurement of the cutter head are the primary tasks in the robot's automatic cutter head changing process, and accurate camera parameters are a prerequisite for these tasks.

[0003] Although camera calibration can be performed offline when the tunnel boring machine (TBM) is stopped, the continuous and severe vibrations during tunneling inevitably cause changes in camera parameters, necessitating multiple offline calibrations throughout the project cycle. In particular, deep and long tunnel projects often feature complex geological characteristics such as extremely hard rock, strong rock bursts, large faults, and fractured zones, exacerbating the problems of high vibration intensity and long duration for the TBM. Under these conditions, even with relatively comprehensive camera hardware vibration-resistant and damping measures or secure connection schemes, changes in camera parameters cannot be completely prevented. Transporting the camera to an open area behind the tunnel for calibration is a tedious task, and repeated disassembly and reassembly further exacerbate problems such as camera loosening and parameter instability. Therefore, in-situ calibration must be performed at the in-situ measurement location on the back of the cutterhead. However, for TBMs with an excavation diameter of 7m, the required camera field of view can reach 2m × 2m, necessitating the use of large integrated or modular targets that meet the large field of view requirements for calibration. Large targets are expensive and difficult to transport from the rear to the in-situ measurement position on the back of the cutterhead. Furthermore, the complex spatial structure of this location makes it difficult to install and secure large targets, leading to inconvenient operation and safety risks, thus hindering accurate and flexible calibration. Existing camera calibration methods are mostly applicable to conventional scenarios, with few researchers studying camera calibration technologies suitable for harsh construction scenarios such as space-constrained and hazardous environments in tunnel boring machines. This technological bottleneck urgently needs to be overcome to support the precise and engineering applications of cutterhead changers. Summary of the Invention

[0004] The technical problem to be solved by this invention is to propose a two-stage calibration method and device for a large field-of-view camera of a tunnel boring machine cutter changer robot by introducing intermediate parameters. This method can solve the calibration problem of the large field-of-view camera of the cutter changer robot in harsh construction scenarios such as limited space and dangerous environment. It uses a single small two-dimensional target in a narrow and safe area behind the cutterhead to achieve accurate and flexible calibration of the large field-of-view camera, laying the foundation for automatic cutter changer technology of tunnel boring machines.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] A two-stage calibration method for a large field-of-view camera on a tunnel boring machine cutter changer robot that introduces intermediate parameters includes the following steps:

[0007] S11: Complete the two-stage calibration image acquisition, and obtain the pre-calibration image and the final calibration image in sequence;

[0008] S12: During the pre-calibration stage, the camera is calibrated using Zhang's calibration method to solve for the coordinates of the camera principal point. Ratio to pixel size d x / d y And used as a fixed value in the final calibration stage;

[0009] S13: In the final calibration stage, a system of linear equations is constructed based on radial collimation constraints, and some external parameters are solved independently using the least squares method. and

[0010] S14: Establish an intermediate parameter solution model, construct a system of linear equations, and solve for the intermediate parameters using the least squares method. The distortion coefficient λ of the single-parameter division model F ;

[0011] S15: Establish a focal length conversion model, combined with the distortion coefficient λ of the single-parameter division model. F Solve for the focal length f F And obtain the equivalent focal length. Separate the intermediate parameters to obtain the remaining extrinsic parameters. and

[0012] S16: Solve for the camera rotation matrix R using the orthogonality constraint of the rotation matrix. F With translation vector t F And the camera rotation matrix R F Perform orthogonalization processing to output the final set of all camera parameters.

[0013] Preferably, the two-stage calibration image acquisition process in step S11 includes the following steps:

[0014] S21: Select the camera installation position P1 and the calibration position P2, focus the lens on the calibration position P2, and acquire clear checkerboard images under different target postures for use in the pre-calibration stage.

[0015] S22: Keep the last target's posture unchanged and fixed, focus the lens on the measurement position P3, and acquire a set of blurred images of a phase-shifted circular grating array for use in the final calibration stage.

[0016] Preferably, the intermediate parameter solution model in step S14 can be based on the ideal projection point coordinates [u] in the final calibration stage. F v F ] T Under unknown conditions, only the actual projection point coordinates from the final calibration stage are used. Solving for intermediate parameters The distortion coefficient λ of the single-parameter division model F The mathematical expression is:

[0017]

[0018] In the formula, [x w y w ] T Let the target point be the world coordinates. The distortion radius is the actual projection point coordinates during the final calibration stage.

[0019] Preferably, the focal length conversion model in step S15 can convert the focal length f from the pre-calibration stage. I Focal length f converted to the final calibration stage F The mathematical expression is:

[0020]

[0021] In the formula, [v I v I ] T The coordinates of the ideal projection point during the pre-calibration stage can be obtained from the camera pre-calibration results.

[0022] Preferably, the external parameters in steps S13-S15 and intermediate parameters All contain a proportional scaling factor

[0023] Preferably, the orthogonality constraint of the rotation matrix in step S16 is used to solve... This results in a product without a scaling factor. Camera rotation matrix R F Translation vector t F .

[0024] Preferably, the target in steps S21 and S22 is a remote-controlled, self-powered, small two-dimensional calibration board that can be programmed to define its pattern and precisely control the spacing between target points.

[0025] Furthermore, the coordinates of the feature points of the phase-shifting circular grating array in step S22 correspond one-to-one with the coordinates of the corner points of the checkerboard grid in step S21 in terms of physical dimensions.

[0026] This invention also provides a two-stage calibration device for a wide field-of-view camera of a tool-changing robot, including a calibration pattern module, an image acquisition module, an image processing module, a pre-calibration module, and a final calibration module. The calibration pattern module includes a liquid crystal display target and a target fixing bracket. The target posture can be arbitrarily adjusted and kept fixed, and is used to display calibration images of a checkerboard pattern and a phase-shift circular grating array. The image acquisition module includes a lens and a wide field-of-view camera, used to acquire pre-calibration images and final calibration images. The image processing module is used to solve for the corner pixel coordinates of the clear checkerboard pattern image and the feature point pixel coordinates of the blurred phase-shift circular grating array image. The pre-calibration module is used to execute a pre-calibration algorithm and output the camera principal point coordinates. Ratio to pixel size d x / d y The final calibration module 5 is used to execute the final calibration algorithm and output all the final camera parameters.

[0027] The beneficial effects of this invention are as follows: Based on the actual construction scenario of tunnel boring machines, this invention utilizes a two-stage camera calibration method. A single small two-dimensional target can be used to complete the non-in-situ calibration of the large field-of-view camera of the tool-changing robot within a narrow, safe area behind the cutterhead. This ensures calibration accuracy and ease of operation, avoiding the prominent problems of high target cost, high operational difficulty, and high safety risks associated with in-situ calibration at the measurement location on the back of the cutterhead. This invention provides the possibility for visual positioning and wear measurement of the hobbing cutter during the automatic tool changing process of the tool-changing robot, laying the foundation for the successful development and application of tool-changing robots. Attached Figure Description

[0028] Figure 1 The flowchart of the two-stage calibration method for the large field-of-view camera of the tunnel boring machine cutter changer robot with the introduction of intermediate parameters is shown in the present invention.

[0029] Figure 2 This is a schematic diagram of the location of the two-stage calibration image acquisition process of the present invention;

[0030] Figure 3 This is a checkerboard image of the dual-stage target image of the present invention;

[0031] Figure 4 This is a phase-shifted circular grating array image of the dual-stage target image of the present invention with a phase shift value of 0;

[0032] Figure 5The phase-shifted circular grating array image of the dual-stage target image of the present invention has a phase shift value of 2π / 3;

[0033] Figure 6 The phase-shifted circular grating array image of the dual-stage target image of the present invention has a phase shift value of 4π / 3;

[0034] Figure 7 This is a schematic diagram of the dual-stage calibration device for the large field-of-view camera of the tool-changing robot of the present invention; in the figure: 1 calibration pattern module, 11 LCD display target, 12 target fixing bracket, 2 image acquisition module, 21 lens, 22 large field-of-view camera, 3 image processing module, 4 pre-calibration module, 5 final calibration module. Detailed Implementation

[0035] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0036] Reference Figure 1 A two-stage calibration method for a large field-of-view camera of a tunnel boring machine cutter changer robot that introduces intermediate parameters is described below:

[0037] S11: Complete the two-stage calibration image acquisition. Focus the lens on the calibration position P2 and acquire the pre-calibration image; focus the lens on the measurement position P3 and acquire the final calibration image.

[0038] S12: During the pre-calibration stage, the camera is calibrated using Zhang's calibration method to obtain the coordinates of the camera principal point. Ratio to pixel size d x / d y The camera principal point maintains geometric invariance across different focusing distances, exhibiting relatively stable coordinate values; the pixel size adheres to the camera's manufacturing parameters, demonstrating good stability. The camera principal point coordinates from the pre-calibration stage... Ratio to pixel size d x / d y It is used as a constant value in the final calibration stage.

[0039] S13: In the final calibration stage, based on the radial collimation constraint, a system of linear equations is constructed using the pixel coordinates of at least 5 actual projection points, and the equations containing the proportional scaling factor are solved using the least squares method. Some external parameters and

[0040] S14: Establish an intermediate parameter solution model, construct a system of linear equations using the pixel coordinates of at least four actual projection points, and solve the distortion coefficient λ of the single-parameter division model based on the least squares method. F and containing a proportional scaling factor intermediate parameters

[0041] S15: Establish a focal length conversion model, combined with the distortion coefficient λ of the single-parameter division model. F Solve for the focal length f F And obtain the equivalent focal length. Separate the intermediate parameters to obtain independent parameters that still contain a proportional scaling factor. Some external parameters and

[0042] S16: Solve for the camera rotation matrix R using the orthogonality constraint of the rotation matrix. F With translation vector t F And the camera rotation matrix R F Perform orthogonalization to output the final set of camera parameters, including intrinsic parameters, distortion coefficients, and extrinsic parameters.

[0043] Reference Figure 2 The two-stage calibration image acquisition process in step S11 above includes the following steps:

[0044] S21: Select camera installation position P1 and calibration position P2, both located within a small, safe area behind the tunnel boring machine cutterhead. Calibration position P2 can be located anywhere within the lens's focusing range in front of installation position P1. Focus the lens on calibration position P2 and acquire 10 to 20 clear checkerboard images of different target postures for use in the pre-calibration stage.

[0045] S22: After completing the acquisition of a clear checkerboard image, keep the last target's posture unchanged and fixed. Focus the lens on the measurement position P3 on the back of the cutter head and acquire a set of blurred images of the phase-shifted circular grating array for the final calibration stage. Typically, a three-step phase-shifting method can be used to obtain the high-precision coordinates of the feature points of the phase-shifted circular grating array. Therefore, a total of three phase-shifted circular grating array images need to be acquired, with phase shift values ​​of 0, 2π / 3, and 4π / 3, respectively.

[0046] The intermediate parameter solution model in step S14 above can be used to determine the ideal projection point coordinates [u] in the final calibration stage. F v F ] T Under unknown conditions, only the actual projection point coordinates from the final calibration stage are used. Solving for intermediate parameters The distortion coefficient λ of the single-parameter division model FThe mathematical expression is:

[0047]

[0048] In the formula, [x w y w ] T Let the target point be the world coordinates. The distortion radius is the actual projection point coordinates during the final calibration stage.

[0049] The focal length conversion model in step S15 above can convert the focal length f in the pre-calibration stage. I Focal length f converted to the final calibration stage F The mathematical expression is:

[0050]

[0051] In the formula, [u I v I ] T The coordinates of the ideal projection point during the pre-calibration stage can be obtained from the camera pre-calibration results.

[0052] External parameters in steps S13-S15 above and intermediate parameters All contain a proportional scaling factor

[0053] The orthogonality constraint of the rotation matrix in step S16 above can be used to solve... This results in a product without a scaling factor. Camera rotation matrix R F Translation vector t F .

[0054] The target in steps S21 and S22 above is a remote-controlled, self-powered, small two-dimensional calibration board that can be programmed to define its pattern and precisely control the spacing between target points.

[0055] Furthermore, referring to Figure 3-6 The coordinates of the feature points of the phase-shifting circular grating array in step S22 correspond one-to-one with the coordinates of the corner points of the checkerboard grid in step S21 in terms of physical dimensions. The number of feature points and corner points is 30 each.

[0056] In this patent, the letter 'I' in the superscript of all formulas indicates that the parameter is a parameter from the pre-calibration stage.

[0057] The letter F in the superscript of all formulas indicates that the parameter is a parameter from the final calibration stage.

[0058] The letter 's' in the superscript of all formulas indicates that the parameter contains a proportional scaling factor. The parameters.

[0059] In the diagram, the combination of r and the subscript d in the lower right corner represents the distortion radius, where r is the radius and d is the distortion.

[0060] Reference Figure 7 A dual-stage calibration device for a large field-of-view camera of a tunnel boring machine cutter changer robot that introduces intermediate parameters includes a calibration pattern module 1, an image acquisition module 2, an image processing module 3, a pre-calibration module 4, and a final calibration module 5.

[0061] The aforementioned calibration pattern module 1 includes a liquid crystal display target 11 and a target fixing bracket 12. The liquid crystal display target 11 can be adjusted in any orientation and remains unchanged, used to display calibration images of the checkerboard pattern and phase-shifting circular grating array. The liquid crystal display target 11 is a small two-dimensional calibration board, and a standard 10-inch size is sufficient to meet the requirements of two-stage calibration image acquisition, and it is easy to carry and use in the harsh construction environment of tunnel boring machines.

[0062] The image acquisition module 2 mentioned above includes a lens 21 and a wide field-of-view camera 22, used to acquire pre-calibration images and final calibration images. The lens 21 is a fixed-focus or zoom lens, and the wide field-of-view camera 22 is an industrial camera.

[0063] The image processing module 3 described above is used to solve for the corner pixel coordinates of the clear checkerboard image and the feature point pixel coordinates of the blurred image of the phase-shifted circular grating array.

[0064] The aforementioned pre-calibration module 4 is used to execute the pre-calibration algorithm and output the coordinates of the camera principal point. Ratio to pixel size d x / d y The pre-calibration algorithm is based on Zhang's calibration method in step S12 to solve for the camera parameters in the pre-calibration stage.

[0065] The aforementioned final calibration module 5 is used to execute the final calibration algorithm and output all final camera parameters. The final calibration algorithm uses the camera principal point coordinates from the pre-calibration stage. Ratio to pixel size d x / d y As input, the camera parameters are solved in the final calibration stage based on the mathematical theory in steps S13-S16, and the internal parameters, external parameters, and distortion coefficients are finally output.

[0066] This invention enables non-in-situ calibration of the large field-of-view camera for a tool-changing robot within a confined, safe area behind the cutterhead of a tunnel boring machine (TBM). This avoids the significant problems associated with in-situ calibration at the measurement location on the back of the cutterhead, such as high target cost, operational difficulty, and high safety risks. It provides a possibility for visual positioning and wear measurement of the cutter head during automatic tool changing in the tool-changing robot, laying the foundation for the successful development and application of tool-changing robots. This invention has good feasibility and economy and can be widely applied to various types of TBMs.

[0067] The foregoing has set forth many specific details to provide a thorough understanding of the invention. However, without departing from the spirit and scope of the invention, those skilled in the art can still directly determine or derive many other variations or modifications consistent with the principles of the invention based on the disclosure. Therefore, the invention is not limited to the specific embodiments disclosed above, and the scope of the invention should be understood and recognized as covering all such variations or modifications.

Claims

1. A two-stage calibration method for a large field-of-view camera of a tunnel boring machine cutter changer robot that introduces intermediate parameters, characterized in that: Includes the following steps: S11: Complete the two-stage calibration image acquisition, and obtain the pre-calibration image and the final calibration image in sequence; S12: During the pre-calibration stage, the camera is calibrated using Zhang's calibration method to solve for the coordinates of the camera principal point. Ratio to pixel size And used as a fixed value in the final calibration stage; S13: In the final calibration stage, a system of linear equations is constructed based on radial collimation constraints, and some external parameters are solved independently using the least squares method. , , , and ; S14: Establish an intermediate parameter solution model, construct a system of linear equations, and solve for the intermediate parameters using the least squares method. distortion coefficients of the single-parameter division model ; S15: Establish a focal length conversion model, combined with the distortion coefficient of the single-parameter division model. Solve for focal length And obtain the equivalent focal length. , Separate the intermediate parameters to obtain the remaining extrinsic parameters. , and ; S16: Obtain the camera rotation matrix by using the orthogonality constraint of the rotation matrix. With translation vector and the camera rotation matrix Perform orthogonalization processing to output the final set of all camera parameters.

2. The two-stage calibration method for a large field-of-view camera of a tunnel boring machine tool changer robot, as described in claim 1, characterized in that: The two-stage calibration image acquisition process described in step S11 includes the following steps: S21: Select the camera installation position P1 and the calibration position P2, focus the lens on the calibration position P2, and acquire clear checkerboard images under different target postures for use in the pre-calibration stage. S22: Keep the last target's posture unchanged and fixed, focus the lens on the measurement position P3, and acquire a set of blurred images of a phase-shifted circular grating array for use in the final calibration stage.

3. The two-stage calibration method for the large field-of-view camera of a tunnel boring machine cutter changer robot, as described in claim 1, is characterized in that: The intermediate parameter solution model described in step S14 can determine the ideal projection point coordinates in the final calibration stage. Under unknown conditions, only the actual projection point coordinates from the final calibration stage are used. Solve for the intermediate parameters , , distortion coefficients of the single-parameter division model The mathematical expression is: In the formula, Let the target point be the world coordinates. This is the distortion radius of the actual projected point coordinates during the final calibration stage.

4. The two-stage calibration method for a large field-of-view camera of a tunnel boring machine cutter changer robot, as described in claim 1, is characterized in that: The focal length conversion model described in step S15 can convert the focal length in the pre-calibration stage. Focal length converted to final calibration stage The mathematical expression is: In the formula, The coordinates of the ideal projection point during the pre-calibration stage can be obtained from the camera pre-calibration results.

5. The two-stage calibration method for a large field-of-view camera of a tunnel boring machine cutter changer robot, as described in claim 1, characterized in that: The external parameters mentioned in steps S13-S15 , , , , , , , and the aforementioned intermediate parameters All contain a proportional scaling factor .

6. The two-stage calibration method for a large field-of-view camera of a tunnel boring machine cutter changer robot, as described in claim 5, characterized in that: The orthogonality constraint of the rotation matrix mentioned in step S16 is used to solve... Thus, a result is obtained without the aforementioned proportional scaling factor. The camera rotation matrix Translation vector .

7. The two-stage calibration method for the large field-of-view camera of a tunnel boring machine tool changer robot with intermediate parameters as described in claim 2, characterized in that: The target mentioned in steps S21 and S22 is a remote-controlled, self-powered, small two-dimensional calibration board that can be programmed to define its pattern and precisely control the spacing between target points.

8. The two-stage calibration method for a large field-of-view camera of a tunnel boring machine cutter changer robot, as described in claim 2, characterized in that: The feature point coordinates of the phase-shifting circular grating array described in step S22 correspond one-to-one with the corner point coordinates of the checkerboard grid described in step S21 in terms of physical dimensions.

9. An apparatus for implementing the two-stage calibration method for a large field-of-view camera of a tunnel boring machine cutter changer robot, as described in any one of claims 1 to 6, characterized in that: The system includes a calibration pattern module (1), an image acquisition module (2), an image processing module (3), a pre-calibration module (4), and a final calibration module (5). The calibration pattern module (1) includes a liquid crystal display target (11) and a target fixing bracket (12). The target posture can be arbitrarily adjusted and kept fixed, and is used to display the calibration images of the checkerboard and the phase-shift circular grating array. The image acquisition module (2) includes the lens (21) and a large field-of-view camera (22), and is used to acquire the pre-calibration image and the final calibration image. The image processing module (3) is used to solve the corner pixel coordinates of the clear checkerboard image and the feature point pixel coordinates of the blurred phase-shift circular grating array image. The pre-calibration module (4) is used to execute the pre-calibration algorithm and output the camera principal point coordinates. Ratio to pixel size The final calibration module (5) is used to execute the final calibration algorithm and output the final set of all camera parameters.

Images