A Machine Vision-Based Method for Estimating the Spatial Pose of Ballast on Ballasted Track Surface

Abstract

This invention belongs to the field of rail transit technology, specifically disclosing a machine vision-based method for estimating the spatial pose of ballast on the surface of ballasted track. The method includes: deploying ArUco codes on the ballast surface, constructing a point cloud model through 3D scanning, establishing a local coordinate system based on marker positions, calibrating camera parameters, solving the coordinate system correlation of the first frame image, and estimating the spatial pose of the ballast. The rationality of the logic and the robustness of the algorithm were tested in the laboratory, and the accuracy of the method was ensured through multi-marker error analysis. Furthermore, various combinations of markers were used to achieve ballast group motion tracking, providing a high-precision kinematic data foundation for intelligent monitoring of track bed conditions, facilitating subsequent research on the correlation between ballast motion characteristics and track bed conditions.

Description

Technical Field

[0001] This invention relates to the field of rail transit technology, specifically to a machine vision-based method for estimating the spatial pose of ballast on the surface of ballasted track. Background Technology

[0002] As a critical infrastructure, railway tracks are mainly divided into ballastless tracks and ballasted tracks. Although ballastless tracks have high stability and uniform stiffness and are widely used in high-speed railways, they have the disadvantage of being difficult to repair. On the other hand, ballasted tracks, with their advantages of excellent elasticity, good drainage, and convenient maintenance, remain irreplaceable in special sections and high-speed railways with speeds below 250 km / h.

[0003] The core issue in the performance degradation of ballasted tracks lies in the challenge of assessing the condition of the ballast bed: the dynamic behavior of the lower crushed stone ballast structure is significantly more complex than that of the upper structure. With the increasing speed and heavy load of trains, the microscopic mechanical behavior (fracture, deformation) and kinematic response (displacement, rotation, rearrangement) of ballast particles under cyclic loading can lead to the accumulation of macroscopic defects. Therefore, accurately quantifying the spatial orientation evolution of the ballast has significant engineering value for track condition prediction, research on ballast bed instability mechanisms, and force chain distribution and strength analysis.

[0004] Current research on the motion characteristics of ballast in ballasted railway tracks mainly relies on two methods: discrete element method (DEM) simulation and intelligent sensing experiments. DEM simulation has been widely used in studying the mechanical properties of granular materials such as railway ballast, and has formed a relatively mature methodological system. Zhang Zhihai et al. constructed a refined coupled model of a three-sleeper tamping device, track panel, and ballasted track bed based on the discrete element method and multibody dynamics co-simulation method, and analyzed the ballast motion characteristics and energy evolution law during tamping operations; Zhang Jie et al. used the discrete element method to study the mechanical influence mechanism of ballast embedding on subgrade soil deformation from multiple perspectives; Xu Peng et al. simulated different axle load conditions through discrete element simulation, focusing on analyzing the stress distribution law of track bed bearing capacity; Chen Cheng et al. used three-dimensional scanning technology to reconstruct the real geometric morphology of ballast particles, combined with the discrete element method to simulate single-sleeper ballast box test, and explored the real stress boundary conditions; Liu Ganzhong established a three-sleeper track bed model on the bridge based on discrete element simulation technology, and found that the ballast generated significant flow in the bottom and shoulder areas of the sleeper when the sleeper vibrated, and the flow velocity near the sleeper was relatively high; Peng Hui et al. used discrete element software to construct a simplified two-dimensional track bed model at the mid-span of the main span on the bridge, and revealed the ballast flow characteristics and track bed thickness evolution law under the coupling effect of temperature cycle and train load from macroscopic and microscopic dimensions.

[0005] Current research trends focus on a refined description of the actual motion mechanism of ballast particles. By integrating emerging technologies such as intelligent particle sensors and machine vision with traditional track bed experiments, the correlation mechanism between ballast motion and track bed state is revealed. Xiao Yuanjie et al. quantified the mapping relationship between ballast deformation behavior and dynamic stress amplitude based on large-scale triaxial experiments, and used SmartRock sensors in plate vibration compaction tests to monitor particle rotation differences at different spatial points, analyzing the evolution law of ballast acceleration energy spectrum throughout the compaction process. Wang Meng et al. used large-scale monotonic loading triaxial compression experiments, introduced the dirty condition variable, and used intelligent sensor motion data to construct a coupled model of dirt rate-macroscopic shear strength-microscopic particle rotation. Wang Meng et al. simulated the track bed compaction process through indoor rotational compaction experiments, and established a quantitative correlation mechanism between motion characteristics and compaction degree based on the relative rotation angle and rotation ratio of ballast. Fu et al. used intelligent sensors to identify the service status of full-scale track beds and proposed a track bed state identification method that integrates multi-dimensional particle motion characteristics.

[0006] In current research on track bed structural defects, electromagnetic feature-based methods for identifying the condition of crushed stone track beds have been widely applied: Khakiev et al. constructed a quantitative index of track bed humidity by integrating radar reflection signals from different layers of track bed, and verified its accuracy through on-site excavation; Wang Shilei et al. analyzed the time-frequency characteristics of radar electromagnetic signals, extracted multi-parameter curves evolving along the track, and determined that three indicators were significantly correlated with the service condition of the track bed; Silvast et al. proposed using spectral domain integration to construct a characterization index for track bed contamination rate. In addition to electromagnetic identification technology, emerging detection methods continue to expand their application dimensions: Liang et al. used infrared thermal imaging technology to compare the differences in internal thermodynamic properties between clean and contaminated track beds and screened the optimal detection indicators; Bian et al. quantitatively analyzed the movement behavior of ballast particles under different train speed / axle load conditions by color-coding ballast particles; Kumara et al. developed an image analysis-based ballast assessment technology to achieve accurate quantification of sand contamination and simultaneous generation of particle size distribution curves.

[0007] Current research on ballast motion characteristics mainly relies on intelligent particle sensors, but these sensors suffer from three major limitations: accumulated time-domain errors, distortion in contact mechanics simulation, and long-term power supply constraints. While traditional machine vision marking methods can partially compensate for these shortcomings, existing research is limited by issues such as the size deviation of circular markers, the lack of vertical information in two-dimensional projected coordinates (only able to capture Z-axis rotation), the ability to record only two-dimensional planar motion data, data interruption due to occlusion, and insufficient sensitivity for recognizing micro-rotations. These limitations make it difficult to meet the requirements for analyzing the six degrees of freedom motion of ballast. Summary of the Invention

[0008] To address the shortcomings of existing technologies, such as displacement error accumulation, missing rotation dimensions, and insufficient anti-occlusion capabilities, this invention introduces a 2D-3D correspondence method from the field of pose estimation. It proposes a machine vision-based method for estimating the spatial pose of ballast on the surface of ballasted track. The aim is to achieve operational efficiency in rapid multi-target detection within a single frame, breakthrough accuracy in simultaneous analysis of sub-millimeter displacement and 0.5° rotation, complete information output of six-degree-of-freedom pose parameters, and system scalability for simultaneous tracking of multi-particle motion. This provides a high-precision data foundation for the study of ballast motion characteristics and ultimately fills the technological gap in accurate detection of ballast motion across all dimensions.

[0009] To achieve the above objectives, the present invention provides the following technical solution: a method for estimating the spatial pose of ballast on the surface of ballasted track bed based on machine vision, comprising the following steps:

[0010] S1. Apply ArUco codes to the ballast surface: Select a flat surface of the ballast particles and apply three 7×7 ArUco codes with IDs of 0, 1, and 2 respectively.

[0011] S2. Ballast Profile 3D Scan: The marked ballast particles are scanned from multiple angles using a 3D scanning instrument to obtain a point cloud model of the ballast profile.

[0012] S3. Establish a local coordinate system for ballast particles based on the three ArUco code positions;

[0013] S4. Perform camera calibration;

[0014] S5. Establish the relationship between the three ArUco code coordinate systems and the local coordinate system using the first frame image;

[0015] S6. Implement ballast spatial pose estimation.

[0016] Preferably, the ArUco code mentioned in step S1:

[0017] ArUco codes are tags with unique information encoding. Their structure includes an outer black border and an inner binary information grid. Different IDs of ArUco codes contain unique information, providing position and orientation references for machine vision inspection. Recognition and detection are implemented using the ArucoDetection algorithm in the Python environment, and an independent coordinate system is established for each tag: an independent coordinate system is established with the top left corner of the tag as the origin, the tag edge as the X / Y axis, and the vertical plane of the tag as the Z axis.

[0018] Preferably, step S3 specifically includes:

[0019] a) Identify the positions of the three ArUco codes in the point cloud processing software;

[0020] b) Select the top left corner of each ArUco code (i.e., its coordinate system origin) as the reference point (ID 0→A, ID 1→B, ID2→C);

[0021] c) Construct a local coordinate system: with A as the origin, vector AB→ AB The X-axis is defined as the x-axis, and plane ABC (the plane containing points A, B, and C) is the XY plane. The axis perpendicular to this plane is the Z-axis.

[0022] d) Calculate the centroid of the point cloud Coordinates in the local coordinate system.

[0023] Preferably, the centroid calculation formula is as follows:

[0024] ;

[0025] In the formula, Let A, B, and C be the three-dimensional spatial coordinates.

[0026] ;

[0027] In the formula, Let A, B, and C be the unit vectors of the coordinate axes. We obtain each vector through the coordinates of the three points A, B, and C, and then use this formula to determine the direction of each coordinate axis.

[0028] ;

[0029] Because the ballast particles have a uniform mass distribution, their centroid and center of mass coincide in space. The coordinates of the center of mass can be calculated using the centroid calculation formula. In the formula, The centroid coordinates are in the software coordinate system. The number of mass points, The coordinates of the center of mass are obtained by weighted averaging of the positions of the mass points.

[0030] The transformation from the software coordinate system to the local coordinate system is achieved through a homogeneous transformation matrix. This matrix provides a unified description of spatial rotation and translation transformations, facilitating precise and reversible transformations between coordinate systems.

[0031] ;

[0032] Wherein, the rotation matrix R:

[0033] ;

[0034] In the rotation matrix, The angle of rotation around the Y-axis. The rotation angle is about the Z-axis. The angle is the rotation around the X-axis; The transformed The shaft is in the original Components on the axis, Transformed The shaft is in the original Components on the axis, The transformed The shaft is in the original Components on the axis, Transformed The shaft is in the original Components on the axis, The transformed The shaft is in the original Components on the axis, Transformed The shaft is in the original Components on the axis, Transformed The shaft is in the original Components on the axis, Transformed The shaft is in the original Components on the axis, Transformed The shaft is in the original Components on the axis; the specific form of the rotation matrix is ​​as follows:

[0035] ;

[0036] Translation vector :

[0037] ;

[0038] This indicates the amount of translation along each axis of the original coordinate system;

[0039] Centroid coordinates in local coordinate system Centroid coordinates in the software coordinate system After homogeneous transformation, we obtain:

[0040] .

[0041] Preferably, step S4 is implemented as follows:

[0042] By utilizing the correspondence between the feature points with known precise three-dimensional coordinates on the calibration plate and their two-dimensional projection points in the image, and based on the pinhole imaging model and lens distortion model, the intrinsic parameters (characterizing the geometric and optical properties of the imaging system) and extrinsic parameters (describing the pose transformation of the camera relative to the calibration plate) of the camera are solved. The obtained parameters will be used as known conditions for subsequent calculations of ballast pose changes based on the 2D-3D correspondence.

[0043] Preferably, step S5 includes:

[0044] The first frame image containing three ArUco codes is acquired. Using the first frame image containing all three ArUco codes, the position of each ArUco code in the camera coordinate system is identified and calculated. Based on this, the transformation relationship between the coordinate system of each ArUco code and the local coordinate system is established, and the pose relationship between the local coordinate system and the camera coordinate system is further derived.

[0045] Preferably, the principle behind the transformation relationship between the ArUco code's own coordinate system and the local coordinate system, as well as the transformation relationship between the local coordinate system and the camera coordinate system, is as follows: Assume the motion of any point on a rigid body consists of translation and rotation. This motion is considered as the synthesis of translation with respect to the center of mass and rotation about the center of mass. Therefore, expressing the motion of the center of mass is equivalent to expressing the motion of the entire rigid body, as detailed below:

[0046] 1) Calculate the rotation relationship between the ArUco code's own coordinate system and the camera coordinate system:

[0047] ;

[0048] In the formula, For the rotation matrix of the ArUco code, For calibration, a fixed transformation is used;

[0049] 2) Calculate the position of the rigid body point in the camera coordinate system. :

[0050] ;

[0051] In the formula, This is the location of the ArUco code. These are calibration parameters;

[0052] 3) Calculate the position of the centroid in the camera coordinate system. ;

[0053] ;

[0054] In the formula, The center of mass is fixed at a fixed position;

[0055] 4) Through and To fully represent the transformation relationship, all the rotation matrices above are third-order matrices, in the following form:

[0056] ;

[0057] In the formula, The angle of rotation around the Y-axis. To bypass Axis rotation angle, The angle is the rotation around the X-axis.

[0058] Preferably, in step S6:

[0059] Based on the coordinate system transformation relationship established in the first frame, subsequent frames estimate the ballast spatial pose by identifying at least one ArUco code and performing pose calculation and error analysis. When at least two ArUco codes are identified in a subsequent frame, the centroid positions corresponding to each code are independently calculated; the Euclidean distance between the two centroid positions is then calculated. ;like Less than the preset threshold If the judgment result is reliable, the average value of the centroid position is taken as the output; if This is considered dirty data, thus constructing a ballast time-space displacement dataset.

[0060] The beneficial effects of this invention are significantly reflected in three aspects:

[0061] Technological Breakthrough: Overcoming the dual limitations of contact mechanics distortion caused by the non-realistic profile of intelligent sensors and the lack of rotational dimension in the field of machine vision, a six-degree-of-freedom pose estimation method based on multi-label space solution (utilizing the rigid body constraint relationship of non-collinear three labels) and sub-pixel level vector analysis (improving feature localization accuracy to 0.1 pixel level through sub-pixel corner detection algorithm) is proposed to achieve accurate capture of the full-dimensional motion of ballast.

[0062] Superior performance: Laboratory verification shows that this method achieves sub-millimeter displacement accuracy (±0.5mm) and 0.5° rotation resolution, and has a high pose estimation continuity rate under partially occluded conditions, far exceeding traditional marking methods.

[0063] Forward-looking applications: It provides a full-dimensional and high-precision kinematic dataset for the study of the correlation mechanism between ballast movement and track bed condition, achieves a breakthrough in the precise detection technology of six-degree-of-freedom pose of ballast track granular space, provides a core algorithm module for the intelligent health monitoring system of ballast railways, and supports preventive maintenance decisions based on track bed condition. Attached Figure Description

[0064] Figure 1 This is a schematic diagram of the machine vision-based method for estimating the spatial pose of ballast on the surface of a ballasted track bed in the embodiment.

[0065] Figure 2 This is a schematic diagram of the 7×7 ArUco code with ID=0 in the embodiment;

[0066] Figure 3 This is a schematic diagram of ArUco codes pasted on the ballast surface in the embodiment;

[0067] Figure 4 This is a schematic diagram of the 3D scanned ballast profile in the embodiment;

[0068] Figure 5This is a schematic diagram of the ballast profile point cloud model in the embodiment;

[0069] Figure 6 This is a schematic diagram illustrating the software processing for extracting ArUco code corner coordinates in the embodiment.

[0070] Figure 7 This is a schematic diagram of a local coordinate system in the embodiment;

[0071] Figure 8 This is a schematic diagram illustrating the principle of the conversion relationship in the embodiment;

[0072] Figure 9a This is a schematic diagram of the initialization of single-track ballast monitoring in the embodiment;

[0073] Figure 9b This is a schematic diagram of subsequent frames for single-track ballast monitoring in the embodiment;

[0074] Figure 10a This is a schematic diagram illustrating the initialization of ballast particle monitoring in the embodiment.

[0075] Figure 10b This is a schematic diagram of subsequent frames for monitoring ballast particles in the embodiment;

[0076] Figure 11a This is a schematic diagram of the recognition results under dark conditions in the example;

[0077] Figure 11b This is a schematic diagram of the recognition results under natural light conditions in the example. Detailed Implementation

[0078] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0079] To address the technical bottlenecks in the background technology, this invention proposes a machine vision-based method for estimating the spatial pose of surface ballast, achieving a triple breakthrough in its core functional objectives:

[0080] First, by using the ArUco code three-dimensional spatial coordinate reconstruction and vector analysis algorithm, the ballast displacement and the complete rotation angle around the X, Y, and Z axes are calculated simultaneously, breaking through the limitation of two-dimensional projection dimension.

[0081] Secondly, a multi-label topology association solution mechanism is designed so that pose estimation continuity can be maintained based on spatial geometric constraints when the labels are occluded.

[0082] Finally, by combining sub-pixel-level corner detection technology, the synchronous and precise capture of ballast particles' micro-rotation at the 0.5° level and spatial displacement at the millimeter level was achieved, thus completing the quantitative analysis of their spatial pose.

[0083] This method will drive innovation in ballast track maintenance: based on millimeter-level displacement accuracy and sub-angle-level rotation resolution, a non-contact long-term monitoring system will be constructed to fill the gaps in existing technologies in terms of motion dimension integrity, environmental robustness, and mechanism correlation, and directly support panoramic perception and preventive decision-making of track bed status in intelligent operation and maintenance of rail transit.

[0084] This invention provides a technical solution: a machine vision-based method for estimating the spatial pose of ballast on the surface of ballasted track beds, the process of which is as follows: Figure 1 As shown, it includes the following steps:

[0085] S1. Applying ArUco codes to the ballast surface: In this example, three 7×7 ArUco codes with IDs of 0, 1, and 2 are applied to the smooth surface of the ballast particles. The ArUco code format is as follows: Figure 2 As shown, the ArUco code is affixed to the ballast surface. Figure 3 As shown.

[0086] ArUco codes are visual markers with unique IDs. Their structure consists of an outer black border and an inner binary information grid, achieving high-precision recognition through the ArucoDetection algorithm in the Python environment. This marker establishes an independent coordinate system for each ballast particle, with the top-left corner as the origin, the marker edges as the X and Y axes, and the vertical plane as the Z axis, providing a spatial position and orientation reference for machine vision.

[0087] S2. Ballast Profile 3D Scanning: A 3D scanning instrument is used to scan the ballast particles with affixed ArUco codes from multiple angles, such as... Figure 4 As shown, a precise point cloud model of the ballast profile is obtained, such as... Figure 5 As shown.

[0088] S3. Establish a local coordinate system for the ballast particles based on the three ArUco code positions.

[0089] Based on the ballast profile point cloud model obtained in step S2, the coordinates of the upper left corner points (i.e., the origins of their coordinate systems) of the three ArUco codes are identified in the point cloud processing software. IDs 0, 1, and 2 are assigned to reference points A, B, and C, respectively. A local coordinate system is constructed with point A as the origin, vector AB as the positive X-axis, and the plane uniquely determined by points A, B, and C as the XY plane. The axis passing through the origin A and perpendicular to the XY plane is the Z-axis. Since the ballast has a uniform mass, the coordinates of the centroid O in the local coordinate system can be calculated using the centroid calculation formula. Finally, a local coordinate system with A as the origin is established, and the position of the centroid O is determined within this coordinate system (see corner coordinate extraction). Figure 6 The coordinate system configuration is shown in Figure 7 ).

[0090] The formula for calculating the centroid is as follows:

[0091] ;

[0092] In the formula, Let A, B, and C be the three-dimensional spatial coordinates.

[0093] ;

[0094] In the formula, Let A, B, and C be the unit vectors of the coordinate axes. We obtain each vector through the coordinates of the three points A, B, and C, and then use this formula to determine the direction of each coordinate axis.

[0095] ;

[0096] Because the ballast particles have a uniform mass distribution, their centroid and center of mass coincide in space. The coordinates of the center of mass can be calculated using the centroid calculation formula. In the formula, The centroid coordinates are in the software coordinate system. The number of mass points, The coordinates of the center of mass are obtained by weighted averaging of the positions of the mass points.

[0097] The transformation from the software coordinate system to the local coordinate system is achieved through a homogeneous transformation matrix. This matrix provides a unified description of spatial rotation and translation transformations, facilitating precise and reversible transformations between coordinate systems.

[0098] ;

[0099] Wherein, the rotation matrix R:

[0100] ;

[0101] In the rotation matrix, The angle of rotation around the Y-axis. The rotation angle is about the Z-axis. The angle is the rotation around the X-axis; The transformed The shaft is in the original Components on the axis, The transformed The shaft is in the original Components on the axis, The transformed The shaft is in the original Components on the axis, The transformed The shaft is in the original Components on the axis, The transformed The shaft is in the original Components on the axis, The transformed The shaft is in the original Components on the axis, The transformed The shaft is in the original Components on the axis, The transformed The shaft is in the original Components on the axis, The transformed The shaft is in the original Components on the axis; the specific form of the rotation matrix is ​​as follows:

[0102] ;

[0103] Translation vector :

[0104] ;

[0105] This indicates the amount of translation along each axis of the original coordinate system;

[0106] Centroid coordinates in local coordinate system Centroid coordinates in the software coordinate system After homogeneous transformation, we obtain:

[0107] .

[0108] S4. Camera calibration.

[0109] By utilizing the correspondence between the feature points with known precise three-dimensional coordinates on the calibration plate and their two-dimensional projection points in the image, and based on the pinhole imaging model and lens distortion model, the intrinsic parameters (characterizing the geometric and optical properties of the imaging system) and extrinsic parameters (describing the pose transformation of the camera relative to the calibration plate) of the camera are solved. The obtained parameters will be used as known conditions for subsequent calculations of ballast pose changes based on the 2D-3D correspondence.

[0110] S5. Establish the connection between the three ArUco code coordinate systems and the local coordinate system using the first frame image.

[0111] Using the first frame image containing all three ArUco codes, the 3D coordinates of points A, B, and C in the camera coordinate system are identified and obtained. Based on this, the rotation-translation transformation from each ArUco code coordinate system to the local coordinate system and the pose transformation from the local coordinate system to the camera coordinate system are calculated. The transformation principle (e.g.) Figure 8 (As shown) Based on the property that rigid body motion can be decomposed into translation of the center of mass and rotation about the center of mass, that is, the pose of the center of mass can characterize the motion of a rigid body with all degrees of freedom, as follows:

[0112] 1) Calculate the rotation relationship between the ArUco code's own coordinate system and the camera coordinate system:

[0113] ;

[0114] In the formula, For the rotation matrix of the ArUco code, For calibration, a fixed transformation is used;

[0115] 2) Calculate the position of the rigid body point in the camera coordinate system. :

[0116] ;

[0117] In the formula, This is the location of the ArUco code. For calibration parameters;

[0118] 3) Calculate the position of the centroid in the camera coordinate system. ;

[0119] ;

[0120] In the formula, The center of mass is fixed at a fixed position;

[0121] 4) Through and To fully represent the transformation relationship, all the rotation matrices above are third-order matrices, in the following form:

[0122] ;

[0123] In the formula, The angle of rotation around the Y-axis. To bypass Axis rotation angle, The angle is the rotation around the X-axis.

[0124] S6. Ballast pose space estimation.

[0125] Based on the coordinate system transformation relationship established in the first frame, subsequent frames use ArUco code recognition, pose calculation, and error analysis to estimate the ballast spatial pose. When two or more markers are detected, error analysis is triggered: the centroid position corresponding to each marker is calculated independently, and the Euclidean distance is obtained. ;like ( (For a preset threshold), output the average centroid position; if This is considered dirty data, thus enabling the construction of a ballast time-space displacement dataset. For example... Figure 9a The image shows the initialization of the first frame. Figure 9b As shown, this is for subsequent frame identification.

[0126] By configuring unique ArUco code combinations for different ballast particles, the single-particle pose calculation process can be reused. Figure 10a The image shows the initialization of the first frame for multi-channel ballast particles. Figure 10b The image shows the subsequent frame recognition of multiple ballast particles, enabling parallel tracking of the six-degree-of-freedom motion of the ballast particle group.

[0127] Environmental adaptability verification: Algorithm testing was conducted under complex lighting conditions such as strong light, shadow, and low light. Figure 11a The image shows the identification code in a darkened state. Figure 11b As shown in the natural light condition, the system still maintains high recognition accuracy, proving its robustness under complex lighting conditions.

[0128] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0129] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” as used in the embodiments of this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0130] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0131] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0132] The terms "first" and "second" used in the embodiments are merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first" and "second" can be interchanged in a specific order or sequence where permitted. It should be understood that the objects distinguished by "first" and "second" can be interchanged where appropriate so that the embodiments described herein can be implemented in an order other than those illustrated or described herein.

Claims

1. A method for estimating the spatial pose of ballast on the surface of a ballasted track bed based on machine vision, characterized in that, Includes the following steps: S1. Apply ArUco codes to the ballast surface: Select a flat surface of the ballast particles and apply three 7×7 ArUco codes with IDs of 0, 1, and 2 respectively. S2. Ballast Profile 3D Scan: The marked ballast particles are scanned from multiple angles using a 3D scanning instrument to obtain a point cloud model of the ballast profile. S3. Establish a local coordinate system for ballast particles based on the three ArUco code positions; S4. Perform camera calibration; S5. Establish the relationship between the coordinate systems of the three ArUco codes and the local coordinate system using the first frame image: Using the first frame image containing all three ArUco codes, identify and calculate the position of each ArUco code in the camera coordinate system; based on this, establish the transformation relationship from the coordinate system of each ArUco code to the local coordinate system, and further derive the pose relationship between the local coordinate system and the camera coordinate system; specifically including the following: 1) Calculate the rotation relationship between the ArUco code's own coordinate system and the camera coordinate system: In the formula, For the rotation matrix of the ArUco code, For calibration, a fixed transformation is used; 2) Calculate the position of the rigid body point in the camera coordinate system In the formula, t P This is the location of the ArUco code. For calibration parameters; 3) Calculate the position of the centroid in the camera coordinate system. In the formula, The center of mass is fixed at a fixed position; 4) Through and To fully represent the transformation relationship, all the rotation matrices above are third-order matrices, in the following form: In the formula, θ is the rotation angle about the Y-axis, and φ is the rotation angle about the Z-axis. The angle is the rotation around the X-axis; S6. Implement ballast spatial pose estimation.

2. The method for estimating the spatial pose of ballast on the surface of ballasted track bed based on machine vision according to claim 1, characterized in that: In step S1, the ArUco code is a tag with unique information encoding, whose structure includes an outer black border and an inner binary information grid; different IDs of ArUco codes contain unique information, providing position and orientation references for machine vision detection; recognition and detection are implemented through the ArucoDetection algorithm in the Python environment, and an independent coordinate system is established for each tag: the coordinate system takes the upper left corner of the tag as the origin, the tag edge direction as the X-axis and Y-axis, and the axis perpendicular to the tag plane as the Z-axis.

3. The method for estimating the spatial pose of ballast on the surface of ballasted track bed based on machine vision according to claim 1, characterized in that: Step S3 specifically includes the following: Based on the ballast profile point cloud model, the positions of the three ArUco codes are identified in the point cloud processing software; the upper left corner of each ArUco code is selected as a reference point, where ID 0 corresponds to point A, ID 1 corresponds to point B, and ID 2 corresponds to point C; a local coordinate system is established: with point A as the origin, vector AB as the X-axis, the plane containing points A, B, and C as the XY plane, and the axis perpendicular to this plane as the Z-axis; according to the centroid calculation formula, the coordinates of the point cloud centroid in this local coordinate system are solved.

4. The method for estimating the spatial pose of ballast on the surface of ballasted track bed based on machine vision according to claim 3, characterized in that: The formula for calculating the centroid is as follows: A = p0 B = p1 C = p2 In the formula, p0, p1, and p2 are the three-dimensional spatial coordinates of A, B, and C; In the formula, Let A, B, and C be the unit vectors of the coordinate axes. We obtain each vector through the coordinates of the three points A, B, and C, and then use this formula to determine the direction of each coordinate axis. Because the ballast particles have a uniform mass distribution, their centroid and center of mass coincide in spatial position. The coordinates of the center of mass are calculated using the centroid calculation formula; where O software Here are the coordinates of the centroid in the software coordinate system, and n is the number of mass points. (x i ,y i ,z i The coordinates of the mass point are given by the coordinates of the mass point. The coordinates of the centroid are calculated by weighted averaging of the positions of the mass points. The transformation from the software coordinate system to the local coordinate system is achieved through the homogeneous transformation matrix T, which uniformly describes spatial rotation and translation transformations, facilitating precise and reversible transformations between coordinate systems. Wherein, the rotation matrix R: In the rotation matrix, θ is the rotation angle around the Y-axis. The rotation angle is about the Z-axis. r is the rotation angle about the X-axis; XX Let r be the component of the transformed X' axis on the original X-axis. XY r represents the component of the transformed Y' axis on the original X-axis. XZ r represents the component of the transformed Z' axis on the original X-axis. YX r represents the component of the transformed X' axis on the original Y axis. YY r represents the component of the transformed Y' axis on the original Y-axis. YZ r represents the component of the transformed Z' axis on the original Y axis. ZX r represents the component of the transformed X' axis on the original Z axis. ZY r represents the component of the transformed Y' axis on the original Z axis. ZZ The Z' axis represents the component of the original Z axis after the transformation; the specific form of the rotation matrix is ​​as follows: Translation vector t: t=[t X t Y t Z ] This indicates the amount of translation along each axis of the original coordinate system; Centroid coordinates O in the local coordinate system local Centroid coordinates O in the software coordinate system software After homogeneous transformation, we obtain: O local =T·O software 。 5. The method for estimating the spatial pose of ballast on the surface of ballasted track bed based on machine vision according to claim 1, characterized in that: In step S4, the intrinsic and extrinsic parameters of the camera are solved by utilizing the correspondence between the feature points with known precise three-dimensional coordinates on the calibration plate and their two-dimensional projection points in the image, based on the pinhole imaging model and the lens distortion model. The obtained parameters will be used as known conditions for subsequent calculation of ballast pose changes based on the 2D-3D correspondence.

6. The method for estimating the spatial pose of ballast on the surface of ballasted track bed based on machine vision according to claim 1, characterized in that: In step S6, based on the transformation relationship between the ArUco code coordinate system and the local coordinate system established in the first frame, subsequent frames achieve ballast spatial pose estimation by identifying at least one ArUco code and performing pose calculation and error analysis.

7. The method for estimating the spatial pose of ballast on the surface of ballasted track bed based on machine vision according to claim 6, characterized in that: The error analysis and judgment include: a) When at least two ArUco codes are identified in subsequent frames, the centroid positions corresponding to each marker are calculated independently. b) Calculate the Euclidean distance δ between the two centroids; c) If δ is less than the preset threshold δ max The judgment result is reliable, and the average value of the centroid position is taken as the output; d) If δ≥δ max The judgment result is unreliable and is considered dirty data.

Images