Method for visual perception of three-dimensional spatial line shape of large-area railway track

Abstract

The application discloses a kind of big area railway track three-dimensional space line visual perception method based on camera calibration, first, fixed camera is used to shoot plane calibration board in different positions and postures, calculate camera internal parameter and external parameter matrix, then camera radial distortion modeling and calculation, camera internal parameter and distortion are iteratively calculated using LM algorithm, angle and scale adaptive template matching obtain key point image coordinates, finally image coordinates are converted into three-dimensional real coordinates, and polynomial fitting track line is adopted, and the effectiveness of the algorithm is verified in laboratory.The application improves the automation, intelligence, accuracy and robustness of the intelligent identification of the railway track health monitoring real-time early warning subsystem, and provides a solution for the establishment of the large-area railway track three-dimensional space line online real-time early warning subsystem.

Description

Technical Field

[0001] This invention relates to a method for monitoring the three-dimensional spatial alignment health of railway tracks in a large area, specifically a method for visual perception of the three-dimensional spatial alignment of railway tracks in a large area based on camera calibration. Background Technology

[0002] To date, my country's total railway operating mileage exceeds 154,900 kilometers. After nearly a decade of large-scale construction, my country's railway tracks will soon enter a comprehensive maintenance period. The three-dimensional spatial alignment of railway tracks directly causes track irregularities, and accurate measurement of these irregularities is crucial for train operation safety and track damage assessment.

[0003] Track lines are the infrastructure that ensures high-density train operation and heavy transportation tasks. Due to the coupling effect between the train and the track, as well as factors such as foundation settlement, track irregularities occur, directly affecting the safety, stability, and comfort of train operation. Therefore, achieving high-precision inspection of track lines has become one of the necessary means to maintain safe train operation. The detection of track irregularities mainly manifests as the detection of track spatial alignment. Current detection methods are mainly divided into static inspection and dynamic inspection based on track inspection vehicles. Static inspection methods often rely on manual measurement or non-continuous measurement methods such as total stations and precision measurement networks. The main disadvantages of this type of method are low efficiency, high cost, and problems such as missed detections and long inspection cycles. Dynamic inspection technology using track inspection vehicles based on inertial unit measurement methods is currently the conventional method for high-speed railway line inspection, such as the T10 track inspection vehicle developed by Ensco in the United States, the La-serail track inspection vehicle developed by Image-Map, and the latest Gj-6 high-speed track inspection vehicle in China. However, this type of detection method is susceptible to speed, and at low speeds, the acceleration signal measured by the inertial measurement unit is relatively weak, which can easily lead to integral saturation during displacement integration, resulting in low measurement accuracy. In addition, sensors such as gyroscopes often accumulate static deviations during long-term operation, which further reduces the measurement accuracy.

[0004] Therefore, how to achieve high-precision three-dimensional dynamic detection of track alignment under various working conditions is an urgent problem to be solved. Research on high-precision visual perception methods that can identify the three-dimensional spatial alignment of railway tracks in large areas has important practical significance. Summary of the Invention

[0005] To address the problems of low efficiency, high cost, easy omissions, and long detection cycles in traditional static detection methods, and the weak acceleration signals measured by the inertial measurement unit of the dynamic track inspection vehicle at low speeds, which easily leads to integral saturation and low measurement accuracy during displacement integration calculations, as well as the accumulation of static deviations by sensors such as gyroscopes over long periods of operation, thus reducing measurement accuracy, this invention provides a three-dimensional spatial linearity visual perception method for large-area railway tracks.

[0006] The objective of this invention is achieved through the following technical solution:

[0007] A method for visually perceiving the three-dimensional spatial alignment of railway tracks over a large area includes the following steps:

[0008] Step 1: Take pictures of the planar calibration board at different positions and orientations using a fixed camera. Based on the basic matrix theory, determine the camera's internal and external parameters by measuring the geometric relationship between the camera and the calibration board.

[0009] Step 2: Considering the significant radial distortion, after establishing the radial distortion expression, the initial distortion value is obtained using the least squares method; the reprojection error is minimized by using the Levenberg-Marquardt (LM) algorithm to calculate the actual pixel coordinates and the calculated pixel coordinates, and the camera's intrinsic and extrinsic parameters and distortion coefficients are iterated repeatedly until convergence, thereby improving the instrument's measurement accuracy.

[0010] Step 3: Based on angle and scale adaptive template matching technology, after rotating the template at multiple angles to generate a template set, pyramid sampling is performed on the image to generate an image set. A small rectangular window slides within the image set, and the pixels in the window are compared with the template set each time, and a similarity score is calculated. If the similarity score reaches 0.95, it is considered that the matching part of the template has been found, thereby realizing the detection of relevant features in the image. After obtaining the pixel coordinates in the image corresponding to the template, the pixel coordinates are converted into three-dimensional real coordinates using the methods in Step 1 and Step 2. Polynomial fitting is used to fit the three-dimensional real coordinates of multiple key points of the railway track in a large area, and finally the representation and reconstruction of the three-dimensional spatial shape of the railway track is completed.

[0011] Step 4: Simulate various working conditions during the construction process using laboratory equipment, conduct long-term monitoring of the track using cameras set up in different directions, collect data using high-definition cameras equipped with zoom lenses and high-precision targets of multiple sizes, and reconstruct the three-dimensional spatial shape of the track according to the methods in Steps 1 to 3.

[0012] Compared with the prior art, the present invention has the following advantages:

[0013] 1. This invention identifies three-dimensional deformation of a track using a visual perception method. It uses cameras set up in various different orientations to monitor the track over a long period of time and explores the applicability of the track three-dimensional deformation recognition algorithm under bright light conditions.

[0014] 2. This invention is based on template matching technology that adapts to angle and scale. It identifies objects or features in an image by finding parts that match the template, thereby realizing the detection of relevant features in the image.

[0015] 3. This invention can serve as a component of a large-area railway track health monitoring and real-time early warning subsystem, directly and in real-time identifying and determining whether track irregularities caused by the three-dimensional spatial alignment of railway tracks in a large area are abnormal. This invention improves the automation, intelligence, accuracy, and robustness of the intelligent identification in the railway track health monitoring and real-time early warning subsystem, providing a solution for establishing a large-area railway track three-dimensional spatial alignment online real-time early warning subsystem. Attached Figure Description

[0016] Figure 1 This is a flowchart of the present invention;

[0017] Figure 2 The results of corner point identification for working conditions A1-A6;

[0018] Figure 3 The images show the original pictures and experimental results for working condition B1;

[0019] Figure 4 The images show the original pictures and experimental results for working condition B2.

[0020] Figure 5 The images shown are the original images and experimental results for working condition B3: (a) original image, (b) experimental results (daytime), and (c) experimental results (nighttime).

[0021] Figure 6 The images shown are the original images and experimental results for working condition B4: (a) original image, (b) experimental results (daytime), and (c) experimental results (nighttime).

[0022] Figure 7 The images shown are the original images and experimental results for working condition B5: (a) original image, (b) experimental results (daytime), and (c) experimental results (nighttime).

[0023] Figure 8 The images shown are the original images and experimental results for working condition B6: (a) original image, (b) experimental results (daytime), and (c) experimental results (nighttime).

[0024] Figure 9The images shown are the original images and experimental results for working condition B7: (a) original image, (b) experimental results (daytime), and (c) experimental results (nighttime).

[0025] Figure 10 The images shown are the original images and experimental results for working condition B8: (a) original image, (b) experimental results (daytime), and (c) experimental results (nighttime). Detailed Implementation

[0026] The technical solution of the present invention will be further described below with reference to the accompanying drawings, but it is not limited thereto. Any modifications or equivalent substitutions to the technical solution of the present invention that do not depart from the spirit and scope of the technical solution of the present invention should be covered within the protection scope of the present invention.

[0027] This invention provides a large-area railway track 3D spatial alignment visual perception method based on camera calibration. First, a fixed camera is used to capture images of a planar calibration board at different positions and orientations. The camera's intrinsic and extrinsic parameter matrices are calculated. Then, radial distortion of the camera is modeled and calculated. The LM algorithm is used to iteratively calculate the camera's intrinsic and extrinsic parameters and distortion. Angle and scale adaptive template matching is used to obtain the key point image coordinates. Finally, the image coordinates are converted into 3D real coordinates, and a polynomial fitting is used to fit the track alignment. The effectiveness of the algorithm is verified in the laboratory. Figure 1 As shown, the method specifically includes the following steps:

[0028] Step 1: Using a fixed camera, photograph the planar calibration board at different positions and orientations. Based on fundamental matrix theory, determine the camera's internal and external parameters by measuring the geometric relationship between the camera and the calibration board. This includes the following steps:

[0029] Step 1: The coordinates of each marked point i on the planar target are represented homogeneously in the world coordinate system, denoted as (U... i V i 1) T The pixel coordinate system is denoted as (u i v i 1) T Let A be the camera intrinsic parameter matrix, and let the extrinsic parameter matrix P have its first two columns denoted as R1 and R2, and its last column as T1. i Given the scaling factor, we can obtain formula (1):

[0030]

[0031] Let A(R1 R2 T1) be denoted as the homography matrix H, and let the nine elements of matrix H be H... 11 -H 33 Let the first three columns of H be denoted as (H1, H2, H3). Substituting them into equation (1) yields:

[0032]

[0033] Step 12: Eliminate z from equation (2) i We can obtain:

[0034]

[0035] In the formula, H is a non-full-rank homogeneous matrix with 8 independent unknown elements. Since each pair of markers can provide two constraint equations, when the number of markers on an image is greater than or equal to 4, the corresponding homogeneous matrix H can be obtained by least-squares linear solution. Since R1 and R2 satisfy the unity orthogonality relation, then:

[0036] R1 T R2=0 (4)

[0037] R1 T R1 = R2 T R2=1 (5)

[0038] Based on the relationship between H and R, we can know that:

[0039] R1 = A -1 H1,R2=A -1 H2 (6)

[0040] Step 13: Substituting equation (6) into (4) and (5) yields:

[0041]

[0042]

[0043] Let A -T A -1 =B, and the nine elements of B are denoted as B0. 11 -B 33 Let α, β, and γ be three equivalent parameters, and u0 and v0 be the coordinates of the center point of the imaging plane in the pixel coordinate system, then we can obtain:

[0044]

[0045]

[0046] From equation (9), we can see that B is a symmetric matrix, and its six-dimensional vector representation is:

[0047] b = [B 11 B 12 B 22 B 13 B 23 B 33 ] T (11)

[0048]

[0049] In the formula

[0050]

[0051] Step 14: The basic constraint equations for the camera's intrinsic parameters can be rewritten as follows:

[0052]

[0053] Since the homogeneous matrix H is known, and matrix v is composed of elements of H, matrix v can be obtained. Each calibration image provides two constraint equations, and b has six unknowns; therefore, b can be obtained from three calibration images. When there are more than three calibration images, the least squares method is used to solve for the optimal value. The camera's intrinsic parameters can be expressed as:

[0054]

[0055] After obtaining the camera's intrinsic parameters, the extrinsic parameters are then calculated. Based on A(R1 R2 T) = H, the (R1 R2 T) corresponding to each sample image is obtained. According to the spatial orientation relationship of R1, R2, and R3, we can obtain:

[0056] R3 = R1 × R2 (16)

[0057] This allows us to obtain the lens's intrinsic and extrinsic parameters.

[0058] Step 2: Considering the significant radial distortion, after establishing the radial distortion expression, the initial distortion value is obtained using the least squares method. The Levenberg-Marquardt (LM) algorithm is then used to minimize the reprojection error between the actual pixel coordinates and the calculated pixel coordinates. This process is iterated repeatedly with the camera's intrinsic and extrinsic parameters and distortion coefficients until convergence, thereby improving the instrument's measurement accuracy. Specifically, the steps include:

[0059] Step 21: Calibrate lens distortion. Based on Zhang Zhengyou's calibration method, consider the radial distortion, which has a significant impact on the distortion model:

[0060]

[0061] In the formula (x,y) and These represent the normalized image coordinates for the undistorted and distorted images, respectively, where r is the distance from the pixel to the image center point, i.e., r 2 =x 2 +y 2 k1 and k2 are distortion coefficients. The transformation relationship between coordinates (u,v) in the image coordinate system and coordinates in the pixel coordinate system is as follows:

[0062]

[0063] Step 22: In the pixel coordinate system of the distorted image expression:

[0064]

[0065] Substituting into the radial distortion formula, we get:

[0066]

[0067] Simplified to:

[0068]

[0069] Represented in matrix form:

[0070]

[0071] Let D be the left half of the above equation and k be the right half, for a simplified representation:

[0072] Dk=d (23)

[0073] The radial distortion coefficient is solved using least squares:

[0074]

[0075] Steps 2 and 3: At this point, the camera's intrinsic parameters, extrinsic parameters, and distortion coefficients have all been determined. The direct results can only be used as initial values. Next, the reprojection error is calculated based on the actual pixel coordinates and the calculated pixel coordinates. The reprojection error is optimized by minimizing the value, iterating through the camera's intrinsic and extrinsic parameters and distortion coefficients until convergence.

[0076] Zhang Zhengyou's calibration method establishes the relationship between three-dimensional world coordinates and two-dimensional pixel coordinates by using a fixed camera to photograph a planar calibration plate at different positions and orientations. The intrinsic, extrinsic, and distortion coefficients of the camera, obtained through linear solutions, are optimized using the Levenberg-Marquardt algorithm for nonlinear parameters, resulting in high theoretical accuracy and improving the measurement accuracy of instruments such as optical navigation systems.

[0077] Step 3: Based on angle and scale adaptive template matching technology, after rotating the template at multiple angles to generate a template set, pyramid sampling is performed on the image to generate an image set. A small rectangular window slides within the image set, and the pixels in the window are compared with the template set each time, and a similarity score is calculated. If the similarity score reaches 0.95, the matching part of the template is considered to have been found, thereby realizing the detection of relevant features in the image. After obtaining the pixel coordinates in the image corresponding to the template, the pixel coordinates can be converted into three-dimensional real coordinates using the technology in Step 2. Polynomial fitting is used to fit the three-dimensional real coordinates of multiple key points of a large area of ​​railway track, finally completing the representation and reconstruction of the three-dimensional spatial shape of the railway track. Specifically, the following steps are included:

[0078] Step 3: 1. Rotate the template image. The rotation angle can be determined according to actual needs, but 15 degrees is recommended. Generate matching template sets with different angles.

[0079] Step 32: Perform pyramid sampling on the images to generate an image set;

[0080] Step 3: The standard squared difference is used as the similarity evaluation index. The calculation method is shown in Equation (25). If the similarity score reaches 0.95, it is considered that the matching part of the template has been found, thereby realizing the detection of relevant features in the image.

[0081] Standard deviation matching

[0082]

[0083] Where R(x,y) represents the matching score, T(x′,y′) is the pixel value corresponding to the pixel point in the template image, and I(x+x′,y+y′) is the pixel value corresponding to the pixel point in the original image after coordinate transformation;

[0084] Steps 3 and 4: After obtaining the pixel coordinates in the image corresponding to the template, the pixel coordinates can be converted into three-dimensional real coordinates using the methods in Steps 1 and 2. Then, the three-dimensional real coordinates of multiple key points of the railway track in a large area are fitted using polynomials, and finally the representation and reconstruction of the three-dimensional spatial shape of the railway track is completed.

[0085] Step 4: Simulate various working conditions during the construction process using laboratory equipment, conduct long-term monitoring of the track using cameras set up in different orientations, collect data using high-definition cameras equipped with zoom lenses and high-precision targets of multiple sizes, and reconstruct the three-dimensional spatial shape of the track according to the algorithms proposed in Steps 1 to 3 of this invention.

[0086] Example:

[0087] This embodiment simulates various working conditions during construction using laboratory equipment, uses cameras set up in different directions for long-term monitoring of the track, and employs high-definition cameras equipped with zoom lenses and high-precision targets of multiple sizes to collect data. Based on the algorithm proposed in this invention, the three-dimensional spatial shape of the track is reconstructed.

[0088] This experiment was conducted at the Research and Development Institute of the Civil Engineering Department of China Railway Design Corporation and the Vibration and Noise Reduction Laboratory of the National Engineering Research Center for Digital Construction and Evaluation Technology of Urban Rail Transit. The laboratory has a track vibration reduction test field with a total track structure length of 29 meters, initially equipped with a floating slab track bed, an integral track bed, and trapezoidal sleepers. It possesses an EES-70W pure wheel-rail contact electromagnetic excitation system, capable of providing a rated sinusoidal excitation force of 70kN, a rated impact excitation force of 210kN, a continuous excitation force frequency of 1–1000Hz, synchronous vertical downward excitation of the rails by four wheels, and a guide transmission speed of 1cm / s in the non-excitation state, precisely locating the excitation point. Continuous forces include random, sinusoidal, and sinusoidal plus random; impact types include half-sine, sawtooth wave, triangular wave, rectangular, and trapezoidal wave. The experiment also employed a high-definition camera with a zoom lens and high-precision targets of multiple sizes to collect data, while contact displacement gauges and strain gauges were used to simultaneously collect data on the structural response. This experiment targets the key technical problem of track deformation recognition based on computer vision technology. Taking into account the specific conditions of the laboratory, two experimental schemes were set up: a three-dimensional deformation recognition experiment and a track alignment recognition experiment. Six experimental conditions with fixed-focus brightness and eight experimental conditions with short focal length, short duration, long focal length, and long duration were set up to verify the effectiveness and accuracy of the algorithm of this invention.

[0089] Step 1: Long-term monitoring of the track was conducted using six cameras positioned at different locations (conditions A1-A6) to explore the applicability of the track 3D deformation recognition algorithm under bright light conditions. A 10×7 square checkerboard (square side length 42mm) was used to calibrate the monitoring cameras set in fixed positions. Preliminary calculations were performed by identifying the checkerboard corner points in multiple images. The corner point finding results are as follows: Figure 2 As shown in Table 1, the intrinsic parameter matrix of the camera under various operating conditions is calculated using a calibration algorithm, and the extrinsic parameter matrix is ​​shown in Table 2.

[0090] Table 1. Camera intrinsic parameter matrix under operating conditions A1-A6 (unit: m)

[0091]

[0092] Table 2. Extrinsic parameter matrices of the camera under operating conditions A1-A6 (unit: m)

[0093]

[0094]

[0095] Step 2: The distortion coefficients of the camera are calculated using a calibration algorithm. The distortion coefficients of the camera under various working conditions are shown in Table 3. The maximum reprojection error calculated is 4.2%, and the calibration result meets the accuracy requirements. Based on the calibrated camera intrinsic and extrinsic parameter matrix and distortion coefficients, the three-dimensional coordinates of the verification feature points are calculated. The calculated grid side lengths and errors are shown in Table 4. Table 4 shows a maximum error of 2.20%, proving the accuracy of the algorithm of this invention.

[0096] Table 3. Camera distortion coefficients under operating conditions A1-A6

[0097] Operating Condition A1 [[2.47405691e+00],[-4.30626547e+00],[1.49802283e-01],[-3.75261198e-02],[-1.90180294e+03]] Operating Condition A2 [[-1.64414349],[15.13780174],[-0.06224365],[0.07152748],[9.15188976]] Operating Condition A3 [[-1.19017184e+00],[1.93563079e+02],[-1.34458799e-02],[-1.73458315e-02],[8.21513164e+01]] Operating Condition A4 [[-0.62847077],[1.84755018],[0.00758155],[0.03058512],[-2.16991317]] Operating Condition A5 [[-4.73889434e-01],[7.96197528e-01],[-1.86975641e-05],[-1.83599651e-02],[-7.21082027e-01]] Operating Condition A6 [[-4.80055453e+00],[2.43620772e+03],[1.17720827e-01],[3.39573228e-02],[4.37262797e+00]]

[0098] Table 4 shows the predicted grid side lengths and relative errors under operating conditions A1-A6.

[0099] project Grid side length (mm) relative error Operating Condition A1 41.987 0.03％ Operating Condition A2 42.681 1.62％ Operating Condition A3 42.063 0.15％ Operating Condition A4 42.927 2.20％ Operating Condition A5 42.334 0.79％ Operating Condition A6 41.568 1.03％

[0100] Step 3: Identify the track alignment based on template matching technology algorithm. The track alignment is identified by visual perception method. Conditions B1 and B2 simulate the changes in the distance of the camera installation position and the size of the field of view by changing the lens focal length, representing the short focal length and short-term conditions respectively. Conditions B3-B8 are long-term monitoring of the track using 6 cameras installed in different positions, representing the fixed focal length long-term conditions. A total of 8 experimental conditions were set up.

[0101] Step 4: For operating conditions B1 and B2, Figure 3 , Figure 4 This is the result of processing using a track alignment recognition algorithm. The dark boxes represent the identified track fasteners, while the light-colored thick solid lines represent the calculated track alignment. As can be seen from the image, the algorithm accurately identified all track fasteners and accurately drew a track alignment consistent with the actual track direction.

[0102] Figure 5 This is the result of processing using the track alignment recognition algorithm for working condition B3. From Figure 4 As can be seen, the algorithm accurately identified all track fasteners and accurately drew the track alignment consistent with the actual track direction. Furthermore, to demonstrate the algorithm's robustness to nighttime images, the algorithm was used to identify the track alignment in nighttime images. As shown in the figure, the algorithm also accurately identified all track fasteners and accurately drew the track alignment consistent with the actual track direction.

[0103] Figures 6-10This is the processing result of the track alignment recognition algorithm used for working conditions B4-B8. It can be seen that the algorithm accurately identified most of the track fasteners, while fasteners that were too small were not detected, and it accurately drew the track alignment consistent with the actual track direction. Furthermore, to demonstrate the algorithm's robustness to nighttime images, the algorithm was used to identify the track alignment in nighttime images. As can be seen from the image, the algorithm also accurately identified most of the track fasteners, while fasteners that were too small were not detected, and it accurately drew the track alignment consistent with the actual track direction.

[0104] This experiment, based on camera calibration and 3D coordinate calculation results, combined with track fastener detection and line shape recognition technologies, explores the influence of hardware selection, deployment location, algorithm type, and deformation recognition accuracy. Figure 8 Taking condition B6 as an example, the image acquisition device has a resolution of 1920×1080. After calibrating the camera, the obtained camera coordinates are [414.03538884, 432.69867928, 7999.08883207]. The origin of this coordinate system is located at the upper left corner of the handheld calibration checkerboard. Therefore, the distance from the camera's optical center to the origin of the calibration board can be calculated to be 8021.5 mm, meaning the straight-line distance from the camera to the object under test is approximately 8 meters. Simultaneously, the calculated size of the calibration board grid is 10.5 pixels, while the actual grid size is 42 mm. Therefore, the conversion factor is 4 mm / pixel. Considering different algorithm types (e.g., whether sub-pixel technology or optical flow is used), the lowest reliable accuracy for deformation recognition is 4 mm. In summary, with a camera resolution of 1920×1080 and a distance of approximately 8 meters, the deformation recognition accuracy can reach about 4mm, indicating that the reconstructed value matches the true value well. This proves the effectiveness and accuracy of the method, as well as its practicality in recognizing the three-dimensional spatial alignment of railway tracks in large areas.

Claims

1. A method for visually perceiving the three-dimensional spatial alignment of railway tracks over a large area, characterized in that... The method includes the following steps: Step 1: Take pictures of the planar calibration board at different positions and orientations using a fixed camera. Based on the basic matrix theory, determine the camera's internal and external parameters by measuring the geometric relationship between the camera and the calibration board. Step 2: Considering the significant radial distortion, after establishing the radial distortion expression, the least squares method is used to obtain the initial distortion value; the Levenberg-Marquardt algorithm is used to minimize the reprojection error between the actual pixel coordinates and the calculated pixel coordinates, and the camera's intrinsic and extrinsic parameters and distortion coefficients are iterated repeatedly until convergence, thereby improving the instrument's measurement accuracy. Step 3: Based on the angle and scale adaptive template matching technology, after rotating the template at multiple angles to generate a template set, the image is pyramid sampled to generate an image set. A small rectangular window slides within the image set, and each time the pixels in the window are compared with the template set, and the similarity score is calculated. If the similarity score reaches 0.95, it is considered that the matching part of the template has been found, thereby realizing the detection of relevant features in the image. After obtaining the pixel coordinates in the image corresponding to the template, the pixel coordinates are converted into three-dimensional real coordinates through the methods in steps one and two. The three-dimensional real coordinates of multiple key points of the railway track in a large area are then fitted using a polynomial to finally complete the representation and reconstruction of the three-dimensional spatial shape of the railway track. Step 4: Simulate various working conditions during the construction process using laboratory equipment, conduct long-term monitoring of the track using cameras set up in different directions, collect data using high-definition cameras equipped with zoom lenses and high-precision targets of multiple sizes, and reconstruct the three-dimensional spatial shape of the track according to the methods in Steps 1 to 3.

2. The method for visually perceiving the three-dimensional spatial alignment of railway tracks in a large area according to claim 1, characterized in that... Step one specifically includes the following steps: Step 1: The coordinates of each marked point i on the planar target are represented homogeneously in the world coordinate system, denoted as (U... i V i 1) T The pixel coordinate system is denoted as (u i v i 1) T Let A be the camera intrinsic parameter matrix, and let the extrinsic parameter matrix P have its first two columns denoted as R1 and R2, and its last column as T1. i Given the scaling factor, we can obtain formula (1): Let A(R1 R2 T1) be denoted as the homography matrix H, and let the nine elements of matrix H be H... 11 -H 33 Let the first three columns of H be denoted as (H1, H2, H3). Substituting them into equation (1) yields: Step 12: Eliminate z from equation (2) i We can obtain: In the formula, H is a non-full-rank homogeneous matrix, and R1 and R2 satisfy the identity orthogonality relation, then: R1 T R2=0 (4) R1 T R1=R2 T R2=1 (5) Based on the relationship between H and R, we can know that: R1=A -1 H1,R2=A -1 H2 (6) Step 13: Substituting equation (6) into (4) and (5) yields: H1 T A -T A -1 H1=H2 T A -T A -1 H2=1 (8) Let A -T A -1 =B, and the nine elements of B are denoted as B0. 11 -B 33 Let α, β, and γ be three equivalent parameters, and u0 and v0 be the coordinates of the center point of the imaging plane in the pixel coordinate system, respectively. We can obtain: From equation (9), we can see that B is a symmetric matrix, and its six-dimensional vector representation is: b=[B 11 B 12 B 22 B 13 B 23 B 33 ] T (11) In the formula Step 14: The basic constraint equations for the camera's intrinsic parameters can be rewritten as follows: The camera's internal parameters can be expressed as: Based on A(R1 R2 T) = H, we can obtain (R1 R2 T) for each sample. Based on the spatial orientation relationship of R1, R2, and R3, we can obtain: R3 = R1 × R2 (16) This allows us to obtain the lens's intrinsic and extrinsic parameters.

3. The method for visually perceiving the three-dimensional spatial alignment of railway tracks in a large area according to claim 1, characterized in that... Step two specifically includes the following steps: Step 21: Calibrate lens distortion. Based on Zhang Zhengyou's calibration method, consider the radial distortion, which has a significant impact on the distortion model: In the formula (x,y) and These represent the normalized image coordinates for the undistorted and distorted images, respectively, where r is the distance from the pixel to the image center point, i.e., r 2 =x 2 +y 2 k1 and k2 are distortion coefficients. The transformation relationship between coordinates (u,v) in the image coordinate system and coordinates in the pixel coordinate system is as follows: Step 22: In the pixel coordinate system of the distorted image expression: Substituting into the radial distortion formula, we get: Simplified to: Represented in matrix form: Let D be the left half of the above equation and k be the right half, for a simplified representation: Dk=d (23) The radial distortion coefficient is solved using least squares: Steps 2 and 3: Calculate the reprojection error based on the actual pixel coordinates and the calculated pixel coordinates. Optimize the reprojection error by minimizing the value, and iterate the camera's intrinsic and extrinsic parameters and distortion coefficients repeatedly until convergence.

4. The method for visually perceiving the three-dimensional spatial alignment of railway tracks in a large area according to claim 1, characterized in that... Step three specifically includes the following steps: Step 3:

1. Rotate the template image to generate matching template sets at different angles; Step 32: Perform pyramid sampling on the images to generate an image set; Step 3: Use the standard squared difference as the similarity evaluation index. If the similarity score reaches 0.95, it is considered that the matching part of the template has been found, thereby realizing the detection of relevant features in the image. Steps 3 and 4: After obtaining the pixel coordinates in the image corresponding to the template, the pixel coordinates can be converted into three-dimensional real coordinates using the methods in Steps 1 and 2. Then, the three-dimensional real coordinates of multiple key points of the railway track in a large area are fitted using polynomial fitting, and finally the representation and reconstruction of the three-dimensional spatial shape of the railway track is completed.

5. The method for visually perceiving the three-dimensional spatial alignment of railway tracks in a large area according to claim 4, characterized in that... In step three, the formula for calculating the standard square deviation is: Where R(x,y) represents the matching score, T(x′,y′) is the pixel value corresponding to the pixel point in the template image, and I(x+x′,y+y′) is the pixel value corresponding to the pixel point in the original image after coordinate transformation.

Images