A non-line-of-sight structure light sensor calibration device and method for a track gauge inspection device based on double-mirror reflection

Abstract

This invention discloses a calibration device and method for a non-common-line structured light sensor of track gauge inspection equipment based on dual-mirror reflection. The calibration device includes a first camera and a second camera to be calibrated, as well as a first target, a first plane mirror, a second plane mirror, a second target, and a third target. The calibration method includes calibrating the extrinsic parameters between the first and second virtual cameras; calibrating the extrinsic parameters between the first and second virtual cameras, and / or the extrinsic parameters between the second virtual camera and the first camera; establishing a conversion relationship between the extrinsic parameters of the first and second cameras, and converting to obtain the extrinsic parameters between the first and second cameras. This invention addresses the problem that existing technologies cannot meet the requirements of rapid and accurate calibration and long-term stable operation in the engineering application of track gauge inspection equipment, achieving the goal of rapid and accurate calibration of track gauge inspection equipment.

Description

Technical Field

[0001] This invention relates to the field of track gauge measurement, and specifically to a calibration device and method for a non-common line structured light sensor in track gauge inspection equipment based on dual-mirror reflection. Background Technology

[0002] In track inspection equipment applications, to achieve synchronous, high-precision geometric measurements of the left and right rails, it is typically necessary to deploy line structured light sensors (high-precision industrial cameras + line lasers) on both sides of the inspection vehicle. However, due to limitations in the structural dimensions of the track inspection equipment, installation space, and on-site conditions, the two sets of line structured light sensors often lack a common field of view, forming a non-common line structured light sensor system. Under these conditions, traditional dual-sensor calibration methods relying on a common target or shared field of view are difficult to apply directly, making it difficult to accurately obtain the spatial relationship between the two sets of sensors, thus affecting the accuracy of track gauge measurement.

[0003] To address the calibration problem in track gauge detection when there is no common field-of-view camera, existing technologies have made numerous efforts, such as:

[0004] (1) Some existing technologies introduce a third-party camera that has a common field of view with both cameras to be calibrated as an intermediary, and use the coordinate system of the third-party camera as a bridge to calculate the three-dimensional coordinates of the marker point in the common field of view; on this basis, an objective function is constructed, and the intrinsic parameters, distortion and extrinsic parameters of the two target cameras are solved by the optimization algorithm.

[0005] (2) Some existing technologies use high-precision translation tracks and combine translation, rotation and other functions to adjust the pose of the target. First, one camera is used to acquire the target image, and then the target is moved into the field of view of another camera. By precisely controlling and recording the pose transformation data of the target in space, such as movement and rotation, spatial calculation is performed to indirectly establish the spatial relationship between the two non-co-view cameras.

[0006] (3) Some existing technologies require a large number of coded markers to be arranged in the measurement field to form a global control field, and combined with UAV technology, the three-dimensional coordinates of all coded markers in the global coordinate system are reconstructed using multi-view images. Then, the pose relationship between the camera to be calibrated and the global coordinate system is solved by the PnP algorithm, thereby indirectly unifying the coordinate systems of all cameras.

[0007] (4) Some existing technologies use high-precision laser trackers as the benchmark for spatial measurement to establish a global coordinate system; the laser tracker needs to be moved to transfer coordinates.

[0008] (5) The introduction of large public targets has led to many drawbacks in transportation, installation, use and maintenance due to their excessive size.

[0009] It can be seen that existing technologies either rely on sophisticated external equipment (such as high-precision translation tracks, laser trackers, coded markers, drones, etc.), resulting in large system size, high cost, complex operation, cumbersome on-site deployment, and stringent requirements for the environment of the site, making it difficult to deploy quickly on railway sites or maintenance workshops, and lacking portability and adaptability; or they rely on complex intermediate coordinate system transformation and global optimization algorithms, which are lengthy, complex in calculation, and have poor algorithm stability. They are also susceptible to the selection of initial values ​​and on-site interference, making it difficult to guarantee the repeatability and reliability of calibration results. Furthermore, they require high professional skills from operators and lack real-time performance, making it difficult to meet the engineering needs of rapid calibration, real-time detection, and long-term stable operation on track inspection sites.

[0010] Therefore, in the field of track gauge inspection, there is still a lack of a technical solution for calibrating the external structural parameters of a non-common-view binocular camera that is simple to use, efficient in process, and fast in calculation. Summary of the Invention

[0011] This invention provides a calibration device and method for a non-common line structured light sensor of track gauge inspection equipment based on dual-mirror reflection, in order to solve the problem that the existing technology is difficult to meet the requirements of rapid and accurate calibration and long-term stable operation in the engineering application of track inspection equipment, and to achieve the purpose of rapid and accurate calibration of track gauge inspection equipment.

[0012] This invention is achieved through the following technical solution:

[0013] A calibration device for a non-co-linear structured light sensor of a track gauge inspection equipment based on dual-mirror reflection includes a first camera and a second camera to be calibrated, a first target located behind the first camera and the second camera, a first plane mirror located in front of the first camera, a second plane mirror located in front of the second camera, a second target located between the first camera and the first plane mirror, and a third target located between the second camera and the second plane mirror.

[0014] To address the limitations of existing technologies in meeting the demands for rapid and accurate calibration and long-term stable operation in the engineering applications of track inspection equipment, this invention first proposes a calibration device for a non-common-line structured light sensor in track gauge inspection equipment based on dual-mirror reflection. The line structured light sensor to be calibrated consists of a first camera and a second camera that do not share a common field of view. This application places a first target behind the first and second cameras, and a first plane mirror and a second plane mirror in front of the first and second cameras, respectively. Furthermore, this application places a second target between the first camera and the first plane mirror, allowing the first camera to simultaneously observe the physical second target and its image within the first plane mirror. Similarly, a third target is placed between the second camera and the second plane mirror, allowing the second camera to simultaneously observe the physical third target and its image within the second plane mirror.

[0015] In practical use, this device defines the image of the first camera relative to the first plane mirror as the first virtual camera and the image of the second camera relative to the second plane mirror as the second virtual camera. First, the extrinsic parameters between the first camera and the first virtual camera, and between the second camera and the second virtual camera are calibrated. Then, the extrinsic parameters between the first virtual camera and the second virtual camera are calibrated. After that, the extrinsic parameters between the first virtual camera and the second camera, and / or between the second virtual camera and the first camera are calibrated. Finally, the extrinsic parameter conversion relationship between the first camera and the second camera can be established, and the extrinsic parameters between the first camera and the second camera can be obtained.

[0016] As can be seen, this application features a simple structure, low cost, and convenient operation, eliminating reliance on high-precision motion mechanisms or external precision equipment, and greatly improving the convenience and adaptability of deployment on railway sites. At the algorithm level, this application avoids complex intermediate coordinate system chain transfers and potentially unstable global nonlinear optimizations. Constraints are established through direct geometric-optical relationships, requiring only transformation via two virtual cameras, significantly enhancing the stability of the calibration process and the accuracy of the calibration results. Furthermore, this application can be integrated with existing mature processes such as camera intrinsic parameter calibration and line structured light plane calibration, thereby forming a complete and coherent sensor calibration workflow, directly outputting all parameters usable for 3D track gauge reconstruction.

[0017] Furthermore, the backward extensions of the principal optical axes of the first camera and the second camera intersect.

[0018] Furthermore, the second target is located within the field of view of the first camera and does not obstruct the image of the first target in the first plane mirror; the third target is located within the field of view of the second camera and does not obstruct the image of the first target in the second plane mirror.

[0019] That is, the second target will not prevent the first camera from observing the first target through the first plane mirror; similarly, the third target will not prevent the second camera from observing the first target through the second plane mirror.

[0020] Furthermore, the complete images of the first target and the second target in the first plane mirror are both located within the field of view of the first camera; the complete images of the first target and the third target in the second plane mirror are both located within the field of view of the second camera.

[0021] That is, the first camera can observe the complete images of the first target and the second target through the first plane mirror; the second camera can observe the complete images of the first target and the third target through the second plane mirror.

[0022] A calibration method for a non-common line structured light sensor in a track gauge inspection device based on dual-mirror reflection includes the following steps:

[0023] S1. Set a first target behind the first camera and the second camera to be calibrated, so that the first target is simultaneously located on the reverse extension line of the principal optical axis of the first camera and the second camera.

[0024] A first plane mirror is placed in front of the first camera, and a second target is placed between the first camera and the first plane mirror;

[0025] A second plane mirror is placed in front of the second camera, and a third target is placed between the second camera and the second plane mirror;

[0026] S2. Define the image of the first camera relative to the first plane mirror as the first virtual camera, and the image of the second camera relative to the second plane mirror as the second virtual camera;

[0027] S3. Calibrate the external parameters between the first camera and the first virtual camera, and the external parameters between the second camera and the second virtual camera;

[0028] S4. Calibrate the external parameters between the first virtual camera and the second virtual camera;

[0029] S5. Calibrate the extrinsic parameters between the first virtual camera and the second camera, and / or the extrinsic parameters between the second virtual camera and the first camera;

[0030] S6. Using the coordinate system where the first target is located as the reference coordinate system, establish the external parameter transformation relationship between the first camera and the second camera, and transform to obtain the external parameters between the first camera and the second camera.

[0031] As can be seen, this method utilizes two plane mirrors to construct virtual cameras for two physical cameras to be calibrated, transforming two physical cameras without a common field of view into two virtual cameras with a common field of view. By calibrating the extrinsic parameters between the virtual cameras and their corresponding physical cameras, as well as the extrinsic parameters between the two virtual cameras, and then performing spatial transformation, the extrinsic parameters between the first and second cameras can finally be obtained. This method is simple, convenient to operate, has relatively low computational load, and does not rely on external precision equipment, enabling rapid and accurate calibration of track gauge inspection equipment.

[0032] Furthermore, in step S3:

[0033] Methods for calibrating the extrinsic parameters between the first camera and the first virtual camera include:

[0034] S311. Simultaneously capture images of the actual second target and its mirror image in the first plane mirror using the first camera to obtain an image of the second target from the first camera.

[0035] S312. The image of the second target from the first camera is mirrored and flipped along the first plane mirror to obtain the image of the second target from the first virtual camera.

[0036] S313. Simultaneously take an image of the actual second target or the mirror image of the second target in the first plane mirror from the image of the first camera and the image of the second target from the image of the first virtual camera, perform calibration, and obtain the external parameters between the first camera and the first virtual camera.

[0037] This scheme obtains an image of the second target captured by the first virtual camera by mirroring the image taken by the first camera along the first plane mirror. This image also includes the image of the physical second target and the image of the second target reflected in the first plane mirror. That is, both the first camera and the first virtual camera obtain images of the second target located on opposite sides of the first plane mirror. The image of the second target located on the same side of the first plane mirror is then extracted, allowing both the first camera and the first virtual camera to simultaneously observe the second target on the same side of the first plane mirror. At this point, the first camera and the first virtual camera share a common field of view and observe the same target. Therefore, the extrinsic parameters of both cameras can be calibrated using existing dual-target calibration algorithms to obtain the extrinsic parameters between the first camera and the first virtual camera.

[0038] Similarly, methods for calibrating the extrinsic parameters between the second camera and the second virtual camera include:

[0039] S321. Simultaneously capture images of the actual third target and its mirror image in the second plane mirror using the second camera to obtain an image of the third target from the second camera.

[0040] S322. The image of the third target from the second camera is mirrored and flipped along the second plane mirror to obtain the image of the third target from the second virtual camera.

[0041] S323. Simultaneously take the image of the third target (either the image of the third target itself or the image of the third target reflected in the second plane mirror) from the image of the third target by the second camera and the image of the third target by the second virtual camera, perform calibration, and obtain the external parameters between the second camera and the second virtual camera.

[0042] Furthermore, step S4 specifically includes:

[0043] S401. Take an image of the first target in the first plane mirror using the first camera, and perform a mirror flip along the first plane mirror to obtain the image of the first target by the first virtual camera;

[0044] S402. Take an image of the first target in the second plane mirror using the second camera, and perform a mirror flip along the second plane mirror to obtain the image of the first target by the second virtual camera;

[0045] S403. Based on the imaging of the first target by the first virtual camera and the imaging of the first target by the second virtual camera, calibration is performed to obtain the external parameters between the first virtual camera and the second virtual camera.

[0046] Since the first camera in this application can observe the first target through the first plane mirror, and the second camera can observe the first target through the second plane mirror, it can be considered that both the first virtual camera and the second virtual camera can simultaneously observe the first target. This scheme obtains the image of the first target from the first virtual camera by mirroring the image observed by the first camera. Similarly, it obtains the image of the first target from the second virtual camera by mirroring the image observed by the second camera. At this point, the first virtual camera and the second virtual camera can be considered as two cameras with a common field of view that can observe the same target. Then, the extrinsic parameters of both cameras can be calibrated using existing dual-target calibration algorithms.

[0047] Furthermore, the extrinsic parameters between the first virtual camera and the second virtual camera are:

[0048] ;

[0049] In the formula: The rotation matrix represents the first virtual camera relative to the second virtual camera; represents the translation vector of the first virtual camera relative to the second virtual camera; P represents the coordinates of point P on the first target; The rotation matrix represents the first virtual camera relative to the reference coordinate system where the first target is located; The rotation matrix represents the second virtual camera relative to the reference coordinate system where the first target is located; This represents the translation vector of the second virtual camera relative to the reference coordinate system where the first target is located.

[0050] Those skilled in the art should understand that, having obtained , Then, the rotation matrix of the second virtual camera relative to the first virtual camera was obtained synchronously. Translation vector of the second virtual camera relative to the first virtual camera .

[0051] Furthermore, in step S5:

[0052] The external parameters between the first virtual camera and the second camera are:

[0053] ;

[0054] In the formula: The rotation matrix representing the second camera relative to the first virtual camera; This represents the translation vector of the second camera relative to the first virtual camera; The rotation matrix represents the second virtual camera relative to the first virtual camera; This represents the translation vector of the second virtual camera relative to the first virtual camera; The rotation matrix representing the second camera relative to the second virtual camera; This represents the translation vector of the second camera relative to the second virtual camera.

[0055] Similarly, the extrinsic parameters between the second virtual camera and the first camera are:

[0056] ;

[0057] In the formula: The rotation matrix represents the first camera relative to the second virtual camera; This represents the translation vector of the first camera relative to the second virtual camera; The rotation matrix represents the first virtual camera relative to the second virtual camera; This represents the translation vector of the first virtual camera relative to the second virtual camera; The rotation matrix representing the first camera relative to the first virtual camera; This represents the translation vector of the first camera relative to the first virtual camera.

[0058] Furthermore, in step S6, the extrinsic parameter conversion relationship between the first camera and the second camera is as follows:

[0059] ;

[0060] In the formula: The rotation matrix represents the first camera relative to the reference coordinate system; This represents the translation vector of the first camera relative to the reference coordinate system; The rotation matrix representing the second camera relative to the reference coordinate system; This represents the translation vector of the second camera relative to the reference coordinate system; The rotation matrix representing the second virtual camera relative to the first camera; The rotation matrix represents the second virtual camera relative to the first virtual camera; The rotation matrix representing the second camera relative to the second virtual camera; This represents the translation vector of the first virtual camera relative to the first camera; The rotation matrix represents the first virtual camera relative to the first camera; This represents the translation vector of the second virtual camera relative to the first virtual camera; This represents the translation vector of the second camera relative to the second virtual camera.

[0061] This solution, through the aforementioned extrinsic parameter conversion relationship, can obtain the extrinsic parameters between the first camera and the second camera as follows:

[0062] ;

[0063] In the formula: The rotation matrix represents the first camera relative to the second camera; This represents the translation vector of the first camera relative to the second camera.

[0064] Compared with the prior art, the present invention has at least the following advantages and beneficial effects:

[0065] 1. The present invention provides a calibration device and method for a non-common line structured light sensor of track gauge inspection equipment based on dual mirror reflection. It has the advantages of simple structure, low cost and convenient operation. It eliminates the dependence on high-precision motion mechanism or external precision equipment and greatly improves the convenience and adaptability of deployment on railway site.

[0066] 2. The present invention provides a calibration device and method for a non-coaxial structured light sensor of a track gauge inspection equipment based on dual-mirror reflection. This invention avoids the complex intermediate coordinate system chain transfer and potentially unstable global nonlinear optimization. It establishes constraints through direct geometric-optical relationships and only requires conversion through two virtual cameras, which significantly enhances the stability of the calibration process and the accuracy of the calibration results.

[0067] 3. The present invention provides a calibration device and method for a non-common line structured light sensor of a track gauge inspection equipment based on dual-mirror reflection. It can be integrated with mature processes such as camera intrinsic parameter calibration and line structured light plane calibration in the prior art, thereby forming a complete and coherent sensor calibration workflow and directly outputting all parameters that can be used for three-dimensional track gauge reconstruction. Attached Figure Description

[0068] The accompanying drawings, which are included to provide a further understanding of embodiments of the invention and form part of this application, do not constitute a limitation thereof. In the drawings:

[0069] Figure 1 This is a schematic diagram of the calibration device structure according to a specific embodiment of the present invention;

[0070] Figure 2 This is a schematic diagram of the calibration method according to a specific embodiment of the present invention;

[0071] Figure 3 This is a schematic diagram of the track gauge inspection method according to a specific embodiment of the present invention. Detailed Implementation

[0072] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments and accompanying drawings. The illustrative embodiments and descriptions of this invention are only for explaining this invention and are not intended to limit this invention.

[0073] Example 1:

[0074] A calibration device for a non-common line structured light sensor in track gauge inspection equipment based on dual-mirror reflection, such as... Figure 1 As shown, the system includes a first camera C1 and a second camera C2 to be calibrated, and the backward extensions of the principal optical axes of the first camera C1 and the second camera C2 intersect.

[0075] It also includes a first target T1 located behind the first camera C1 and the second camera C2, a first plane mirror M1 located in front of the first camera C1, a second plane mirror M2 located in front of the second camera C2, a second target T2 located between the first camera C1 and the first plane mirror M1, and a third target T3 located between the second camera C2 and the second plane mirror M2.

[0076] In this embodiment, the second target T2 is located within the field of view of the first camera C1 and does not obstruct the image of the first target T1 in the first plane mirror M1; the third target T3 is located within the field of view of the second camera C2 and does not obstruct the image of the first target T1 in the second plane mirror M2.

[0077] In this embodiment, the complete images of the first target T1 and the second target T2 in the first plane mirror M1 are both located within the field of view of the first camera C1; the complete images of the first target T1 and the third target T3 in the second plane mirror M2 are both located within the field of view of the second camera C2.

[0078] like Figure 1 As shown, by mirroring with the first plane mirror M1, we can obtain the first virtual camera V1, the mirror image T2' of the second target T2, and the mirror image T1' of the first target T1; by mirroring with the second plane mirror M2, we can obtain the second virtual camera V2, the mirror image T3' of the third target T3, and the mirror image T1'' of the first target T1.

[0079] In this embodiment, the extrinsic parameters between the first virtual camera V1 and the second virtual camera V2 are first calibrated; then the extrinsic parameters between the first virtual camera V1 and the second camera C2 are calibrated, and / or the extrinsic parameters between the second virtual camera V2 and the first camera C1 are calibrated; then, using the coordinate system where the first target T1 is located as the reference coordinate system, the extrinsic parameter transformation relationship between the first camera C1 and the second camera C2 is established, and the extrinsic parameters between the first camera C1 and the second camera C2 are obtained.

[0080] Example 2:

[0081] A calibration method for a non-common line structured light sensor in a track gauge inspection device based on dual-mirror reflection, such as... Figure 2 As shown, it includes the following steps:

[0082] Step S1: Set a first target T1 behind the first camera C1 and the second camera C2 to be calibrated, so that the first target T1 is simultaneously located on the reverse extension line of the principal optical axis of the first camera C1 and the second camera C2.

[0083] A first plane mirror M1 is placed in front of the first camera C1, and a second target T2 is placed between the first camera C1 and the first plane mirror M1.

[0084] A second plane mirror M2 is placed in front of the second camera C2, and a third target T3 is placed between the second camera C2 and the second plane mirror M2.

[0085] Step S2: Define the mirror image of the first camera C1 relative to the first plane mirror M1 as the first virtual camera V1, and the mirror image of the second camera C2 relative to the second plane mirror M2 as the second virtual camera V2. For example... Figure 1 As shown, the mirror image T2' of the second target T2, the mirror image T3' of the third target T3, and the mirror images T1' and T1'' of the first target T1 can also be obtained.

[0086] Furthermore, point P on the first target T1 is point P1 on T1' and point P2 on T1''.

[0087] Step S3: Calibrate the extrinsic parameters between the first camera C1 and the first virtual camera V1, and between the second camera C2 and the second virtual camera V2. Specifically:

[0088] The methods for calibrating the extrinsic parameters between the first camera C1 and the first virtual camera V1 include:

[0089] The first camera C1 simultaneously captures images of the actual second target T2 and its mirror image T2' within the first plane mirror M1, thus obtaining an image of the second target T2 captured by the first camera C1, i.e., images of T2 and T2' captured by the first camera C1.

[0090] The image of the second target T2 by the first camera C1 is mirrored and flipped along the first plane mirror M1 to obtain the image of the second target T2 by the first virtual camera V1, that is, to obtain the virtual captured images of T2 and T2' by the first virtual camera V1.

[0091] Simultaneously extract the images captured by the first camera C1 and the first virtual camera V1 at T2 / T2', perform calibration, and the extrinsic parameters between the first camera C1 and the first virtual camera V1 can be obtained, such as... , .

[0092] Similarly, methods for calibrating the extrinsic parameters between the second camera C2 and the second virtual camera V2 include:

[0093] The second camera C2 simultaneously captures images of the actual third target T3 and its mirror image within the second plane mirror M2, thus obtaining an image of the third target T3 captured by the second camera C2.

[0094] The image of the third target T3 by the second camera C2 is mirrored and flipped along the second plane mirror M2 to obtain the image of the third target T3 by the second virtual camera V2, that is, to obtain the virtual captured images of T3 and T3' by the second virtual camera V2.

[0095] Simultaneously extract the images captured by the second camera C2 and the second virtual camera V2 at T3 / T3', perform calibration, and the extrinsic parameters between the second camera C2 and the second virtual camera V2 can be obtained, such as... , .

[0096] Step S4: Calibrate the external parameters between the first virtual camera V1 and the second virtual camera V2.

[0097] Specifically, it includes:

[0098] The image of the first target T1 in the first plane mirror M1 is captured by the first camera C1 and then mirrored along the first plane mirror M1 to obtain the image of the first target T1 by the first virtual camera V1.

[0099] The image of the first target T1 in the second plane mirror M2 is captured by the second camera C2, and then mirrored along the second plane mirror M2 to obtain the image of the first target T1 by the second virtual camera V2.

[0100] Based on the imaging of the first target T1 by the first virtual camera V1 and the imaging of the first target T1 by the second virtual camera V2, calibration is performed to obtain the extrinsic parameters between the first virtual camera V1 and the second virtual camera V2, such as... , , , .

[0101] In this embodiment, the extrinsic parameters between the first virtual camera V1 and the second virtual camera V2 are obtained using the following formula:

[0102] ;

[0103] In the formula: The rotation matrix represents the first virtual camera V1 relative to the second virtual camera V2; represents the translation vector of the first virtual camera V1 relative to the second virtual camera V2; P represents the coordinates of point P on the first target T1; The rotation matrix represents the rotation of the first virtual camera V1 relative to the reference coordinate system of the first target T1; The rotation matrix represents the rotation of the second virtual camera V2 relative to the reference coordinate system of the first target T1; This represents the translation vector of the second virtual camera V2 relative to the reference coordinate system where the first target T1 is located.

[0104] In obtaining , After that, you can get , This is not difficult for those skilled in the art to implement.

[0105] Step S5: Calibrate the extrinsic parameters between the first virtual camera V1 and the second camera C2, and / or the extrinsic parameters between the second virtual camera V2 and the first camera C1.

[0106] This embodiment uses the extrinsic parameters between the first virtual camera V1 and the second camera C2 as an example for illustration: In the aforementioned steps, the conversion relationship between the real camera and the virtual camera, as well as the conversion relationship between the two virtual cameras, have been obtained. This application uses the second virtual camera V2 as the intermediate coordinate to obtain the extrinsic parameters between the second camera V2 and the first virtual camera V1:

[0107] ;

[0108] In the formula: The rotation matrix representing the second camera C2 relative to the first virtual camera V1; This represents the translation vector of the second camera C2 relative to the first virtual camera V1; The rotation matrix represents the second virtual camera V2 relative to the first virtual camera V1; This represents the translation vector of the second virtual camera V2 relative to the first virtual camera V1; The rotation matrix representing the second camera C2 relative to the second virtual camera V2; This represents the translation vector of the second camera C2 relative to the second virtual camera V2.

[0109] Similarly, the extrinsic parameters between the second virtual camera V2 and the first camera C1 are:

[0110] ;

[0111] In the formula: The rotation matrix represents the first camera C1 relative to the second virtual camera V2; This represents the translation vector of the first camera C1 relative to the second virtual camera V2; The rotation matrix represents the first virtual camera V1 relative to the second virtual camera V2; This represents the translation vector of the first virtual camera V1 relative to the second virtual camera V2; The rotation matrix represents the first camera C1 relative to the first virtual camera V1; This represents the translation vector of the first camera C1 relative to the first virtual camera V1.

[0112] Step S6: Using the coordinate system where the first target T1 is located as the reference coordinate system, establish the extrinsic parameter transformation relationship between the first camera C1 and the second camera C2, and transform to obtain the extrinsic parameters between the first camera C1 and the second camera C2.

[0113] In this embodiment, the extrinsic parameter conversion relationship between the first camera C1 and the second camera C2 is as follows:

[0114] ;

[0115] In the formula: The rotation matrix representing the first camera C1 relative to the reference coordinate system; This represents the translation vector of the first camera C1 relative to the reference coordinate system; The rotation matrix representing the second camera C2 relative to the reference coordinate system; This represents the translation vector of the second camera C2 relative to the reference coordinate system; The rotation matrix representing the second virtual camera V2 relative to the first camera C1; The rotation matrix represents the second virtual camera V2 relative to the first virtual camera V1; The rotation matrix representing the second camera C2 relative to the second virtual camera V2; This represents the translation vector of the first virtual camera V1 relative to the first camera C1; The rotation matrix represents the first virtual camera V1 relative to the first camera C1; This represents the translation vector of the second virtual camera V2 relative to the first virtual camera V1; The rotation matrix represents the first virtual camera V1 relative to the first camera C1; The rotation matrix represents the second virtual camera V2 relative to the first virtual camera V1; This represents the translation vector of the second camera C2 relative to the second virtual camera V2.

[0116] In this embodiment, the extrinsic parameters between the first camera C1 and the second camera C2 can be obtained by the following formula:

[0117] ;

[0118] In the formula: The rotation matrix represents the first camera C1 relative to the second camera C2; This represents the translation vector of the first camera C1 relative to the second camera C2.

[0119] The reference coordinate system in this embodiment is the world coordinate system.

[0120] In this embodiment, the extrinsic parameter calibration of two cameras with a common field of view can be performed using existing technical means, such as the calibration of the first camera C1 and the first virtual camera V1, the calibration of the second camera C2 and the second virtual camera V2, and the calibration of the first virtual camera V1 and the second virtual camera V2.

[0121] Example 3:

[0122] A track gauge inspection method, such as Figure 3 As shown, it includes the following steps:

[0123] Step 1: System setup of the non-common line structured light sensor.

[0124] The non-common-line structured light sensor mainly consists of two core components: a high-precision industrial camera and a line laser. During the structural layout, it is necessary to ensure that a single line structured light sensor can accurately acquire the cross-sectional contour information of one side of the track. The line laser emits laser light to illuminate the surface of the rail, which is then imaged by the industrial camera.

[0125] The two high-precision industrial cameras within the structured light sensors are designated as the first camera and the second camera, respectively.

[0126] Step 2: Perform single-target calibration on the first camera and the second camera respectively; including intrinsic parameter calibration and optical plane calibration.

[0127] The process of internal parameter calibration is as follows:

[0128] A high-precision planar target with a stable geometry and clear surface features is used. The target is fixed in front of the line structured light sensor, ensuring the line laser is clearly projected onto the calibration plate surface. The target is moved or rotated to present various poses within the sensor's field of view. In each pose, the following operations are performed: ① The camera is turned on, the line laser is turned off, and a clear image of the target (referred to as the "background image") is captured; ② The target remains absolutely still, the line laser is turned on, and an image of the target with laser stripes is captured (referred to as the "laser image").

[0129] For each acquired background image without laser, a monocular camera is calibrated using mature tools in OpenCV or MATLAB (e.g., the interface function calibrateCamera for Zhang's calibration method is provided in OpenCV). The calibration yields the camera's intrinsic parameter matrix K, distortion coefficient D, and the extrinsic parameters R (rotation matrix) and t (translation vector) for the transformation from the world coordinate system of the target to the camera coordinate system under the current pose.

[0130] The process of optical plane calibration is as follows:

[0131] Using the same set of "background images" and "laser images" in the same pose, the optical plane calibration of the line laser is performed, specifically including:

[0132] 1. For each pair of "background image" and "laser image", the background is removed through image processing to obtain the laser stripe region. Subpixel-level precision algorithms (such as the Steger algorithm), gray-scale centroid method, fitting method, or local extremum method are used to extract the pixel coordinates of the laser stripe center line.

[0133] 2. For each laser center point, using the camera intrinsic parameter matrix K and distortion coefficient D calibrated in the previous steps, as well as the transformation extrinsic parameters R and t from the world coordinate system of the target to the camera coordinate system under the current pose, calculate the corresponding three-dimensional coordinates of the point. Specifically, back-project the pixel point as a ray in the camera coordinate system and find its intersection with the target plane (Z=0) to obtain the three-dimensional point cloud.

[0134] 3. Merge the 3D point clouds generated under all poses, use the least squares method to perform plane fitting, solve the equation of the light plane in the camera coordinate system, and complete the light plane calibration.

[0135] Step 3: Using the apparatus described in Example 1 and / or the method described in Example 2, calibrate the non-co-linear structured light sensor to obtain the external parameters between the first camera and the second camera.

[0136] Step 4: Install the non-co-linear structured light sensors, calibrated in the above steps, onto the track inspection equipment to measure the track gauge. Two sets of line structured light sensors synchronously acquire images of the rails on both sides and laser point clouds (line lasers emit light stripes that illuminate the rail surface and are imaged by a camera). By calibrating the parameters, the light stripe points on the rail surface are reconstructed in three dimensions to obtain the three-dimensional point cloud data of the rail contour. Then, the two-dimensional image is converted into three-dimensional coordinates, and the track gauge value is output.

[0137] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0138] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

Claims

1. A calibration device for a non-co-linear structured light sensor in a track gauge inspection equipment based on dual-mirror reflection, comprising a first camera and a second camera to be calibrated, characterized in that, It also includes a first target located behind the first camera and the second camera, a first plane mirror located in front of the first camera, a second plane mirror located in front of the second camera, a second target located between the first camera and the first plane mirror, and a third target located between the second camera and the second plane mirror.

2. The calibration device for a non-common line structured light sensor of a track gauge inspection equipment based on dual-mirror reflection according to claim 1, characterized in that, The reverse extensions of the principal optical axes of the first camera and the second camera intersect.

3. The calibration device for a non-common line structured light sensor of a track gauge inspection equipment based on dual-mirror reflection according to claim 1, characterized in that, The second target is located within the field of view of the first camera and does not obstruct the image of the first target in the first plane mirror; the third target is located within the field of view of the second camera and does not obstruct the image of the first target in the second plane mirror.

4. The calibration device for a non-common line structured light sensor of a track gauge inspection equipment based on dual-mirror reflection according to claim 1, characterized in that, The complete images of the first target and the second target in the first plane mirror are both located within the field of view of the first camera; the complete images of the first target and the third target in the second plane mirror are both located within the field of view of the second camera.

5. A calibration method for a non-common line structured light sensor in a track gauge inspection device based on dual-mirror reflection, characterized in that, Includes the following steps: S1. Set a first target behind the first camera and the second camera to be calibrated, so that the first target is simultaneously located on the reverse extension line of the principal optical axis of the first camera and the second camera. A first plane mirror is placed in front of the first camera, and a second target is placed between the first camera and the first plane mirror; A second plane mirror is placed in front of the second camera, and a third target is placed between the second camera and the second plane mirror; S2. Define the image of the first camera relative to the first plane mirror as the first virtual camera, and the image of the second camera relative to the second plane mirror as the second virtual camera; S3. Calibrate the external parameters between the first camera and the first virtual camera, and the external parameters between the second camera and the second virtual camera; S4. Calibrate the external parameters between the first virtual camera and the second virtual camera; S5. Calibrate the extrinsic parameters between the first virtual camera and the second camera, and / or the extrinsic parameters between the second virtual camera and the first camera; S6. Using the coordinate system where the first target is located as the reference coordinate system, establish the external parameter transformation relationship between the first camera and the second camera, and transform to obtain the external parameters between the first camera and the second camera.

6. The calibration method for a non-common line structured light sensor in a track gauge inspection device based on dual-mirror reflection according to claim 5, characterized in that, In step S3: Methods for calibrating the extrinsic parameters between the first camera and the first virtual camera include: S311. Simultaneously capture images of the actual second target and its mirror image in the first plane mirror using the first camera to obtain an image of the second target from the first camera. S312. The image of the second target from the first camera is mirrored and flipped along the first plane mirror to obtain the image of the second target from the first virtual camera. S313. Simultaneously take an image of the actual second target or the mirror image of the second target in the first plane mirror from the image of the first camera and the image of the second target from the first virtual camera, perform calibration, and obtain the external parameters between the first camera and the first virtual camera. Methods for calibrating the extrinsic parameters between the second camera and the second virtual camera include: S321. Simultaneously capture images of the actual third target and its mirror image in the second plane mirror using the second camera to obtain an image of the third target from the second camera. S322. The image of the third target from the second camera is mirrored and flipped along the second plane mirror to obtain the image of the third target from the second virtual camera. S323. Simultaneously take the image of the third target (either the image of the third target itself or the image of the third target reflected in the second plane mirror) from the image of the third target by the second camera and the image of the third target by the second virtual camera, perform calibration, and obtain the external parameters between the second camera and the second virtual camera.

7. The calibration method for a non-common line structured light sensor in a track gauge inspection device based on dual-mirror reflection according to claim 5, characterized in that, Step S4 specifically includes: S401. Take an image of the first target in the first plane mirror using the first camera, and perform a mirror flip along the first plane mirror to obtain the image of the first target by the first virtual camera; S402. Take an image of the first target in the second plane mirror using the second camera, and perform a mirror flip along the second plane mirror to obtain the image of the first target by the second virtual camera; S403. Based on the imaging of the first target by the first virtual camera and the imaging of the first target by the second virtual camera, calibration is performed to obtain the external parameters between the first virtual camera and the second virtual camera.

8. The calibration method for a non-common line structured light sensor in a track gauge inspection device based on dual-mirror reflection according to claim 7, characterized in that, The extrinsic parameters between the first virtual camera and the second virtual camera are: ； In the formula: The rotation matrix represents the first virtual camera relative to the second virtual camera; This represents the translation vector of the first virtual camera relative to the second virtual camera; P represents the coordinates of point P on the first target; The rotation matrix represents the first virtual camera relative to the reference coordinate system where the first target is located; The rotation matrix represents the second virtual camera relative to the reference coordinate system where the first target is located; This represents the translation vector of the second virtual camera relative to the reference coordinate system where the first target is located.

9. A calibration method for a non-common line structured light sensor in a track gauge inspection device based on dual-mirror reflection, as described in claim 5, is characterized in that... In step S5: The external parameters between the first virtual camera and the second camera are: ； In the formula: The rotation matrix representing the second camera relative to the first virtual camera; This represents the translation vector of the second camera relative to the first virtual camera; The rotation matrix represents the second virtual camera relative to the first virtual camera; This represents the translation vector of the second virtual camera relative to the first virtual camera; The rotation matrix representing the second camera relative to the second virtual camera; This represents the translation vector of the second camera relative to the second virtual camera; The external parameters between the second virtual camera and the first camera are: ； In the formula: The rotation matrix represents the first camera relative to the second virtual camera; This represents the translation vector of the first camera relative to the second virtual camera; The rotation matrix represents the first virtual camera relative to the second virtual camera; This represents the translation vector of the first virtual camera relative to the second virtual camera; The rotation matrix representing the first camera relative to the first virtual camera; This represents the translation vector of the first camera relative to the first virtual camera.

10. A calibration method for a non-common line structured light sensor in a track gauge inspection device based on dual-mirror reflection, as described in claim 5, is characterized in that... In step S6, the extrinsic parameter conversion relationship between the first camera and the second camera is as follows: ； In the formula: The rotation matrix represents the first camera relative to the reference coordinate system; This represents the translation vector of the first camera relative to the reference coordinate system; The rotation matrix representing the second camera relative to the reference coordinate system; This represents the translation vector of the second camera relative to the reference coordinate system; The rotation matrix representing the second virtual camera relative to the first camera; The rotation matrix represents the second virtual camera relative to the first virtual camera; The rotation matrix representing the second camera relative to the second virtual camera; This represents the translation vector of the first virtual camera relative to the first camera; The rotation matrix represents the first virtual camera relative to the first camera; This represents the translation vector of the second virtual camera relative to the first virtual camera; This represents the translation vector of the second camera relative to the second virtual camera; The extrinsic parameters between the first and second cameras can be obtained by converting them using the following formula: ； In the formula: The rotation matrix represents the first camera relative to the second camera; This represents the translation vector of the first camera relative to the second camera.

Images