Railway state detection method and device

Abstract

The application provides a track state detection method and device, and belongs to the field of railway track detection. The method comprises the following steps: acquiring a two-dimensional image and a three-dimensional point cloud, detecting track diseases according to the two-dimensional image and the three-dimensional point cloud; acquiring IMU data and GNSS data; detecting track spatial linear according to the IMU data and the GNSS data; and determining the track state according to the track disease detection result and the track spatial linear detection result. The method can detect track diseases and track spatial linear, and then determine the track state.

Description

Technical Field

[0001] This application belongs to the field of railway track inspection, and in particular relates to a track condition inspection method and device. Background Technology

[0002] Railways are the nation's main transportation artery, playing a vital role in the national economy, public travel, and national defense. Railway tracks directly support and guide the safe and stable operation of trains, making their safety paramount. During operation, the track structure inevitably suffers various defects under the long-term high-intensity, high-frequency impact loads of trains and the influence of the external environment, such as rail fractures, fastener failures, and ballast cracks. Significant irregularities in the track alignment also occur. Failure to detect these defects in advance can lead to serious traffic accidents. Therefore, track inspection and maintenance are crucial for the safe operation and management of railways.

[0003] Existing methods for detecting track defects are mainly divided into two categories: manual inspection and instrument-based inspection.

[0004] Manual inspection (also known as "track patrol") involves conducting safety checks on the tracks between 24:00 and 4:00 every night. This manual inspection method limits each person to a maximum of 3km per night, requiring a large number of workers and increasing maintenance labor costs. With the rapid development of railways and the continuous increase in track mileage, the need for track maintenance is also increasing, while maintenance windows are gradually decreasing. Traditional manual inspection methods suffer from drawbacks such as high work frequency, low efficiency, high labor costs, susceptibility to subjective human factors in inspection results, incomplete data, and a high risk of missed inspections, making them unsuitable for new requirements. Furthermore, the management of measurement data from manual inspections is fragmented, with diverse data formats and types, creating "information silos" among different measurement units, equipment management units, and measurement teams.

[0005] Currently, track defect detection devices are mainly divided into three types: mounted, traction, and self-propelled. Mounted devices directly mount the detection equipment onto existing operating or maintenance vehicles to perform safety inspections on the target track. Traction devices integrate the detection equipment onto a non-powered track frame, using an existing track vehicle to pull the non-powered track frame for safety monitoring. Self-propelled devices are similar to mounted devices, but the platform is no longer an existing vehicle but a separately designed track trolley with its own drive system. The inspection device is mounted on the trolley to perform track inspections. Compared to traditional manual inspection, instrument detection has advantages such as high efficiency, low labor costs, and high accuracy. However, existing detection devices are limited by their structure and weight. Furthermore, mounted and traction devices are constrained by the operating time and routes of existing operating vehicles, making them inconvenient and inflexible in actual inspection tasks.

[0006] In track alignment detection, there are three main methods: track gauge, relative measuring trolley (gyroscope), and static track inspection trolley (total station). Track gauge provides limited and insufficient information, static track inspection trolley has high accuracy but low efficiency, and relative measuring trolley has slightly higher efficiency but cannot obtain the absolute position of the track.

[0007] Several patents exist in the field of intelligent inspection of track instruments. Patent CN 216372227U discloses a detachable, lightweight, manned intelligent track inspection robot, and patent CN 109976354A discloses a detachable, automatically moving track inspection trolley. However, both patents have relatively poor scalability and cannot achieve intelligent detection of track irregularities. Furthermore, they lack detailed descriptions of specific detection methods. Patent CN 117774575A discloses a multi-functional integrated electric carrier platform. This patent has a relatively large overall structure, insufficient space utilization and integration, and also lacks descriptions of specific detection methods, making it impossible to achieve systematic detection of visualized track structural defects. Patent CN110936978 B discloses a method and device for measuring interlayer gaps in ballastless track based on a measuring trolley. This patent's track inspection mainly targets specific CRTSII slab ballastless track interlayer gap defects, limiting the types of defects detected and making it difficult to meet the needs of daily track inspection. Summary of the Invention

[0008] In view of this, this application provides a track condition detection method and apparatus, which aims to detect track defects and spatial alignment, thereby determining the track condition.

[0009] Firstly, this application provides a method for detecting orbital state, including: Acquire 2D images and 3D point clouds; Track defect detection is performed based on two-dimensional images and three-dimensional point clouds; Acquire IMU and GNSS data; Orbital spatial alignment detection based on IMU and GNSS data; The track condition is determined based on the results of track defect detection and track spatial alignment detection.

[0010] Optionally, the steps for detecting track defects based on two-dimensional images and three-dimensional point clouds are as follows: A fused image is obtained by fusing the two-dimensional features of a two-dimensional image and the depth features of a three-dimensional point cloud. A first neural network model is constructed, and training samples are built based on the fused images under historical conditions to train the first neural network model and obtain the track defect detection model. The fused image is input into the track defect detection model to obtain the track defect detection results.

[0011] Optionally, the step of fusing two-dimensional features of a two-dimensional image and depth features of a three-dimensional point cloud includes: Preprocessing of 2D images and 3D point clouds; A second neural network model is constructed, comprising a 2D encoder, a 3D encoder, a cross-modal fusion module, and a decoder. The 2D encoder is used to acquire two-dimensional features of a two-dimensional image, and the 3D encoder is used to acquire depth features of a three-dimensional point cloud. The cross-modal fusion module is used to fuse the two-dimensional features and the depth features. The decoder is used to form a fused image based on the fused two-dimensional features and depth features. Training samples were constructed using two-dimensional images and three-dimensional point clouds from historical states to train the second neural network model, resulting in an image fusion model. The preprocessed 2D image and 3D point cloud are input into the image fusion model to obtain the fused image.

[0012] Optionally, after obtaining the track defect detection results, the method further includes: The three-dimensional depth image is preprocessed, and edge features of the detected objects are extracted by Canny edge detection, edge regions of straight objects are detected by Hough circle transform, and edge regions of curved objects are detected by probabilistic Hough line transform. Finally, distance information is obtained by combining the extracted edges with the pixel size, thereby determining the size information of the disease.

[0013] Optionally, the steps for orbital spatial alignment detection based on IMU data and GNSS data include: A state vector is constructed based on IMU data, mileage data, and sensor error parameters; Establish state equations to describe the relationship between inertial measurement data and state variables; Establish measurement equations and introduce GNSS data to constrain the state; The optimal estimation of the orbital state is achieved through an iterative Kalman filter process. Based on the determined orbital state, the orbital alignment is determined; The track is segmented based on the results of the alignment assessment.

[0014] Optionally, based on the determined track state, the trajectory alignment determination includes: The curvature of the track at each point is calculated using numerical differentiation, and the track shape is determined based on the curvature changes. Alternatively, based on different orbital shapes in historical states, several labeled training samples can be constructed, and a orbital shape detection model can be trained using these training samples. The orbital shape can then be determined using the orbital shape detection model.

[0015] Optionally, the steps of segmenting the track based on the alignment determination result include: The track data is used as data points, and a point set is formed based on several data points; the track data includes curvature, slope, and coordinates; Clustering algorithms are used to cluster data points in the point set, grouping points with similar geometric features into one class, and then segmenting the track.

[0016] Optionally, the IMU data includes acceleration data and attitude angle data, and the sensor error parameters include acceleration error and mounting angle error.

[0017] Alternatively, the method for determining the installation angle error is as follows: Based on the principle of coordinate system transformation, an error model including installation angle error parameters is established; An optimization algorithm is used to solve for the installation angle error parameters in the error model, and the combination of the installation angle error parameters of the measured data and the theoretical data is obtained as the installation angle error.

[0018] Secondly, this application provides a track condition detection device, comprising: The first acquisition module is used to acquire two-dimensional images and three-dimensional point clouds. The track defect detection module is used to detect track defects based on two-dimensional images and three-dimensional point clouds. The second acquisition module is used to acquire IMU data and GNSS data; The orbital spatial alignment detection module is used to perform orbital spatial alignment detection based on IMU data and GNSS data; The determination module is used to determine the track status based on the track defect detection results and the track spatial alignment detection results.

[0019] The beneficial effects of the technical solution provided in this application include: This application provides a track condition detection method. First, it acquires a two-dimensional image and a three-dimensional point cloud of the track. Then, it performs pixel-level matching and fusion of the acquired two-dimensional planar image and three-dimensional solid data, complementing the respective advantages of the two-dimensional image and the three-dimensional point cloud data in object surface texture detection and shape structure detection to obtain a real-scene image of the track. Subsequently, track defects are detected based on the real-scene image, which helps improve detection accuracy. Simultaneously, the spatial linearity of the track is detected based on IMU data and GNSS data used during the inspection process. This achieves integrated detection of track spatial linearity and track defects, thereby determining the track condition. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0021] Figure 1 This is a flowchart of a method for detecting track defects according to an embodiment of this application; Figure 2 A schematic diagram of a track defect detection process provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of a second neural network model provided in an embodiment of this application; Figure 4 A schematic diagram illustrating the detailed structure of a second neural network model provided in an embodiment of this application; Figure 5 A two-dimensional image provided in one embodiment of this application; Figure 6 A depth map provided for one embodiment of this application; Figure 7 A fused image provided in one embodiment of this application; Figure 8 This is a schematic diagram of the orbital spatial alignment detection process provided in an embodiment of this application.

[0022] Figure 9 This is a structural block diagram of a track spatial alignment detection device provided in an embodiment of this application; Figure 10 This is a structural block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0024] In the rail transit industry, several existing track inspection devices employ relatively traditional image recognition technology. Because one dimension of information (height or depth) is lost during the imaging process, measuring the dimensions of a three-dimensional object using a two-dimensional image becomes extremely difficult (mathematically known as the "inverse process"). Consequently, such devices often only perform qualitative identification, such as determining the presence of defects, but cannot perform quantitative detection, such as measuring specific dimensions, and cannot detect defects in the height direction, such as loose bolts. Furthermore, because image quality is highly susceptible to changes in lighting, vibration, and other shooting conditions, the robustness of the equipment faces significant challenges, resulting in high complexity of defect detection algorithms, time-consuming computation, low detection accuracy, and a lack of real-time performance.

[0025] To address the aforementioned problems, this application provides a method for detecting orbital state. See [link to relevant documentation]. Figure 1 This application provides a flowchart of a method for detecting track defects, including: S101. Obtain two-dimensional images and three-dimensional point clouds.

[0026] S102. Based on the two-dimensional image and three-dimensional point cloud, track defects are detected.

[0027] See Figure 2 In some examples, step S102 includes: Step 1: Fuse the two-dimensional features of the two-dimensional image and the depth features of the three-dimensional point cloud to obtain a fused image; In some examples, step 1 includes: The first step is to preprocess the 2D image and the 3D point cloud.

[0028] In some examples, the preprocessing process is as follows: Input data preprocessing involves projecting the 3D point cloud onto a 2D image plane using calibration parameters (camera intrinsic and extrinsic parameters) to generate pixel-level corresponding depth maps and normal maps.

[0029]

[0030] In the formula: This represents the depth value corresponding to the pixel position after a 3D point is projected onto a 2D image plane, which is the vertical distance from the 3D point to the camera's imaging plane. This represents the unit normal vector of the surface containing a 3D point, used to describe the orientation of the surface at that point. Each pixel in the normal map stores a component of this vector. The x-coordinate (column coordinate) of a pixel in a 2D image plane is the index of the pixel in the horizontal direction of the image. Represents the vertical coordinate (row coordinate) of a pixel in a 2D image plane, which is the index of the pixel in the vertical direction of the image.

[0031] Normalize the 2D image ([0,255]→[) [1,1]), linear scaling of the depth map (matching sensor range). Random rotation (±15°), translation (±0.1m), and noise addition (σ=0.005m) of the point cloud. Affine transformation, color jitter, and random occlusion applied to the 2D image.

[0032] The second step is to construct a second neural network model, which includes a 2D encoder, a 3D encoder, a cross-modal fusion module, and a decoder. The 2D encoder is used to acquire the two-dimensional features of the two-dimensional image, and the 3D encoder is used to acquire the depth features of the three-dimensional point cloud. The cross-modal fusion module is used to fuse the two-dimensional features and the depth features. The decoder is used to form a fused image based on the fused two-dimensional features and the depth features.

[0033] See Figure 3 The specific structure of the second neural network model is shown in the illustration.

[0034] The third step involves using two-dimensional images and three-dimensional point clouds from historical states to construct training samples, which are then used to train the second neural network model, resulting in an image fusion model.

[0035] The fourth step is to input the preprocessed 2D image and 3D point cloud into the image fusion model to obtain the fused image.

[0036] In this application, two-dimensional images and three-dimensional point clouds are complementary in the disease detection process, as detailed below: (1) Track bed inspection: detect foreign objects, sleeper fragments, cracks, damaged signal equipment, water seepage, water accumulation in ditches, and excessive sleeper settlement (three-dimensional image information, elevation information).

[0037] (2) Inspection of fasteners: Inspection of spring clip detachment, spring clip loosening, bolt floating (three-dimensional image information, elevation information), nut detachment / damage, spring clip exiting / displacement, spring clip breakage, gauge plate detachment, gauge plate damage.

[0038] (3) Rail inspection: Inspect the rail profile (three-dimensional image information, elevation information), spalling, and breakage.

[0039] (4) Turnout inspection: tight contact of switch rail, loose fixed connection bolts, floating fastener bolts (three-dimensional image information, elevation information), and detachment of elastic clips.

[0040] (5) Joint inspection: misaligned joints, excessive seams, detached clamps, etc.

[0041] In other words, some track defects cannot be detected by relying solely on two-dimensional images. By fusing two-dimensional images and three-dimensional point clouds, the depth information (or height information) in the three-dimensional point cloud is fused into the two-dimensional image to obtain a fused image containing depth information, which can then better detect track defects.

[0042] See Figure 4 This is a schematic diagram illustrating the detailed structure of the second neural network model provided in this application. The input to the 2D encoder is a single 2D RGB image, in the form of an H×W×3 tensor (H and W are the height and width of the image, and 3 corresponds to the three RGB channels). The output is a multi-scale 2D image feature map, typically containing three feature maps F1, F2, and F3 at different resolutions. Taking ResNet-50 as an example, the output format is: F1 (deep features): 16H×16W×C1.

[0043] F2 (Middle Layer Characteristics): 8H×8W×C2.

[0044] F3 (shallow features): 4H×4W×C3. Where C1, C2, and C3 are the number of channels in each feature map (e.g., in ResNet-50, C1=2048, C2=1024, C3=512).

[0045] The input to the 3D encoder is an unordered 3D point cloud, in the form of an N×3 tensor (N is the number of points in the point cloud, and 3 corresponds to the X, Y, and Z 3D coordinates). The output of the 3D encoder is 3D global features and multi-scale local features processed by PointNet++ and 3D CNN.

[0046] Global features: 1×D vectors (D is the feature dimension, such as 1024), representing the overall semantics of the point cloud.

[0047] Multi-scale local features: Feature maps with multiple receptive fields, in the form of M×C (M is the number of sampling points, C is the number of channels), will eventually be processed into tensors aligned with the resolution of the 2D feature maps, which will facilitate subsequent fusion.

[0048] The cross-modal fusion module receives the following inputs: multi-scale feature maps F1, F2, and F3 from the 2D encoder and 3D features (global features + local features) from the 3D encoder. The output is the fused multi-scale feature map. It takes the form of a tensor with the same resolution as the input 2D feature map, such as , Etc. Fusion methods typically employ feature concat or channel attention weighting (CWA) to effectively combine the texture information of 2D images with the geometric information of 3D point clouds.

[0049] The second neural network employs joint supervision using multiple losses, as shown in the following formula:

[0050] In the formula, , , These are the weighting coefficients for each loss term, used to balance the contributions of different monitoring objectives. Indicates L1 loss, Indicates SSIM loss, This indicates Chamfer loss.

[0051] See Figures 5 to 7 , Figure 5 It is a two-dimensional image. Figure 6 For depth map, Figure 7 To merge images.

[0052] Step 2: Construct the first neural network model, and based on the fused images under historical conditions, construct training samples to train the first neural network model to obtain the track defect detection model.

[0053] In some examples, the first neural network model includes: the YOLOv8 object detection model.

[0054] By conducting in-depth analysis of the visual characteristics of different track structural components such as rails, fasteners, sleepers, and turnouts—and supplementing this by "establishing a deep learning model for target detection of track structural components, the location of track structural components in images can be automatically identified."

[0055] An automatic defect detection system for track structure components was constructed based on deep learning technology. Using the YOLOv8 target detection model, transfer learning was employed to learn features from key components such as track fasteners, sleepers, and rail connectors, establishing a multi-scale defect identification framework.

[0056] By analyzing high-resolution track images, the system can accurately locate various typical defects such as cracks, corrosion, and loose bolts, and achieve pixel-level coordinate positioning and confidence level assessment. This detection system supports real-time output of defect type, location coordinates, and severity classification, and can be integrated into the intelligent analysis platform of track inspection vehicles, effectively improving detection efficiency by 3-5 times and providing an intelligent solution for track structure condition assessment.

[0057] Step 3: Input the fused image into the track defect detection model to obtain the track defect detection results.

[0058] This application, based on "2D + 3D vision joint measurement technology," performs pixel-level matching and fusion of 2D planar images and 3D point cloud data acquired by the equipment to construct a real-scene data model. Then, through in-depth analysis of the visual features of different track structural components such as rails, fasteners, sleepers, and turnouts, the advantages of 2D images and 3D data in surface texture detection and shape structure detection are complemented. Specific detection algorithms for different defects are specifically researched, enabling not only qualitative identification of track defects but also quantitative detection of defect values.

[0059] It should be noted that this application uses RandLA-Net to segment multiple components of ballastless track in 3D point cloud: taking RandLA-Net's ability to efficiently process large-scale point clouds as the core, it targets the point cloud data of multiple components such as rails, elastic clips, and base plates in the ballastless track scene.

[0060] First, a random sampling strategy is used to reduce the size of the point cloud while preserving key geometric features, thus solving the problem of low processing efficiency in traditional point cloud networks. Subsequently, a local feature aggregation module (such as local spatial coding and attention mechanism) is used to capture the subtle geometric differences of different components (such as the long strip structure of the rail, the bending shape of the elastic bar, the linear features of the plate seam, etc.), and a feature representation from local to global is gradually constructed through a hierarchical feature extraction network. In the implementation process, the original track point cloud is first preprocessed (including coordinate normalization, noise filtering and multi-component labeling), and then the network parameters are optimized through end-to-end training so that the model learns the unique geometric and topological features of each component. Finally, a softmax classifier is used to achieve pixel-level segmentation of eight key components, including rails, elastic rails, base plates, and gauge blocks, providing accurate structural information support for the automated detection and maintenance of ballastless tracks.

[0061] In some examples, after obtaining the results of track defect detection, the method further includes: The three-dimensional depth image is preprocessed, and edge features of the detected objects are extracted by Canny edge detection, edge regions of straight objects are detected by Hough circle transform, and edge regions of curved objects are detected by probabilistic Hough line transform. Finally, distance information is obtained by combining the extracted edges with the pixel size, thereby determining the size information of the disease.

[0062] S103, Acquire IMU data and GNSS data.

[0063] S104. Perform orbital spatial alignment detection based on IMU and GNSS data.

[0064] See Figure 8 The calculation process for the orbital spatial shape is shown in detail: exist Figure 8 In the process, initial alignment is a crucial step when the inertial measurement unit (IMU) is started. Its core is to determine the initial attitude relationship between the inertial measurement unit (IMU) coordinate system and the navigation coordinate system (such as the Northeast Sky ENU).

[0065] The purpose of initial alignment is to provide the IMU with initial attitude angles (roll, pitch, and heading); otherwise, inertial navigation cannot calculate the position and attitude of the vehicle from scratch.

[0066] Common methods: divided into self-alignment (using only IMU data) and assisted alignment (combining external data such as GNSS / total station).

[0067] Mechanical coding: refers to the raw data output and preprocessing process of sensors (gyroscopes, accelerometers) in an inertial measurement unit (IMU).

[0068] Encoding content: Includes hardware output decoding of the sensor, temperature compensation, zero bias calibration and other operations, converting the sensor's raw electrical signals into physical quantities (angular velocity, specific force) to provide clean input data for subsequent inertial navigation calculations.

[0069] Repeatability is one of the core metrics for evaluating the performance of an IMU sensor. It refers to the degree of consistency in the output of the sensor when measuring the same physical quantity multiple times under the same environmental and input conditions.

[0070] The Allan variance method is a classic tool for analyzing IMU random errors. By performing variance analysis on long-term output data of the sensor, it can accurately separate different types of random errors (such as angle random walk, zero-bias instability, rate random walk, etc.).

[0071] Process noise optimization refers to the process of dynamically adjusting the "process noise covariance matrix Q" in Kalman filtering based on the random error characteristics of the IMU obtained from Allan variance analysis.

[0072] The error model is a collection of multiple types of errors: primarily including IMU inherent errors (gyroscope drift, accelerator bias), navigation calculation errors (position / velocity / attitude errors), and mounting angle errors (installation deviation between the IMU and the carrier coordinate system). The mounting angle error model is a part of the error model, used to describe the impact of the installation angle deviation between the IMU and the carrier on navigation calculations.

[0073] Observation model: This is a model that describes the mathematical relationship between external observation data such as GNSS and inertial navigation state variables. In GNSS / IMU integrated navigation, the observation model typically establishes the residual relationship between GNSS position / velocity measurements and IMU-derived position / velocity, which is used in the measurement update stage of Kalman filtering.

[0074] Carrier motion constraints are a priori constraints imposed on navigation state estimation based on the physical motion characteristics of the carrier.

[0075] Combined filtering and smoothing refers to fusing high-frequency prediction data from IMUs with low-frequency observation data from GNSSs, using Kalman filtering (such as Extended Kalman Filter (EKF) and Unscented Kalman Filter (UKF)) as the core.

[0076] The specific method consists of two steps: "prediction" and "update". The prediction process uses the IMU's error model to recursively derive the current state prediction value and error covariance. The update process uses the GNSS observation model to calculate the measurement residuals, corrects the predicted values ​​using Kalman gain, and obtains the optimal state estimate.

[0077] The purpose of combined filtering and smoothing is to combine the high-frequency continuity of the IMU with the high precision of GNSS to suppress the accumulated error of the IMU and obtain a stable and accurate navigation state.

[0078] Smoothing process: Based on filtering, the historical state is re-estimated using past and future observation data (typical methods include Rauch-Tung-Striebel smoothing).

[0079] The smoothing process employs the following method: first, a forward state estimate is obtained through Kalman filtering, and then the historical state is corrected by recursively calculating from the last time step and using subsequent observation information.

[0080] The purpose of the smoothing process is to improve the accuracy of historical state estimation, especially suitable for post-analysis scenarios such as track detection, and can obtain more accurate track state results than real-time filtering.

[0081] In some examples, the orbital spatial alignment detection module 5 performs orbital spatial alignment detection based on IMU data and GNSS data, including the following steps: Step 1: Construct a state vector based on IMU data, mileage data, and sensor error parameters.

[0082] In some examples, the IMU data includes acceleration data and attitude angle data, and the sensor error parameters include acceleration error and mounting angle error.

[0083] In some examples, the installation angle error is determined as follows: Based on the principle of coordinate system transformation, an error model including installation angle error parameters is established; An optimization algorithm is used to solve for the installation angle error parameters in the error model, and the combination of the installation angle error parameters of the measured data and the theoretical data is obtained as the installation angle error.

[0084] More specifically, the process for determining the installation angle error and acceleration error is as follows: In the installation angle correction of inertial navigation measurement, when the design alignment of the track is known, this method can provide a reliable reference for the calculation of installation angle error based on the accurate theoretical trajectory.

[0085] In railway track inspection scenarios, a section of straight track with a clear design alignment and no obvious defects, or a curved track with a specific radius of curvature, is selected as the standard motion trajectory. Simultaneously, key geometric parameters of this track section are accurately measured and recorded, such as the coordinates of the track centerline and the superelevation value.

[0086] Step 1: Establish a theoretical model: Based on the track design and the kinematic principles of the carrier, establish a theoretical motion model of the carrier on a standard trajectory. For linear motion, theoretical acceleration and angular velocity can be calculated based on Newton's laws of motion; for curvilinear motion, factors such as centripetal acceleration must also be considered. For example, on a curved track, the centripetal acceleration can be calculated based on the curve radius R and the carrier's speed v. .

[0087] The second step is to compare the measured data with the theoretical data: The collected inertial sensor measurement data is compared point-by-point with the theoretical data calculated from the theoretical motion model. Assume that at a certain moment, the theoretical acceleration along the direction of orbital movement should be... The actual measured value is The difference between the two This refers to the acceleration deviation that includes the influence of installation angle error.

[0088] Step 3: Constructing the Error Model: Based on the principle of coordinate system transformation, an error model including the installation angle error parameter is established. A rotation matrix is ​​typically used to describe the relationship between the carrier coordinate system and the sensor coordinate system, and the installation angle error is embedded as a parameter into the rotation matrix. Let the installation angle error parameter be... , , , , , , represent the rotation angles of the sensor coordinate system relative to the carrier coordinate system in different planes. This is achieved through the rotation matrix. The measured values ​​in the sensor coordinate system are transformed into the carrier coordinate system and compared with the theoretical values ​​to construct the error equation.

[0089] Step 4: Optimization Algorithm Solution: Optimization algorithms such as least squares and genetic algorithms are used to solve for the installation angle error parameters in the error model. Taking least squares as an example, its objective is to minimize the sum of squares of the errors between the measured data and the theoretical data, i.e. By iteratively calculating and continuously adjusting the installation angle error parameter until it reaches its minimum value, the parameter obtained at this point is the estimated installation angle error.

[0090] Step 2: Establish state equations to describe the relationship between inertial measurement data and state variables.

[0091] In some examples, the core of the state equation is to use gyroscope / accelerometer measurement data, combined with navigation dynamics, to describe the changes in position, velocity, attitude (and error).

[0092] The state equations include the position update equation, velocity update equation, and attitude update equation, and their specific forms are as follows: Position update equation:

[0093] In the formula, Indicates the updated position. , , For RNU velocity components, , The radii of curvature of the Earth's meridian and circumpolar orbit. It is high for the earth.

[0094] Velocity update equation:

[0095] In the formula, Indicates the updated speed. It is the coordinate transformation of the accelerometer measurement data from the carrier system to the navigation system (IMU core input). The attitude direction cosine matrix is ​​updated from gyroscope measurement data. It represents the projection of the Earth's rotational angular velocity vector onto the navigation coordinate system (n-system, usually the local NED / ENU geographic coordinate system). The projection of the rotational angular velocity of the navigation coordinate system (n-frame) relative to the geocentric coordinate system (e-frame) onto the n-frame is also often referred to as the "entrainment angular velocity" or "position rate".

[0096] Attitude update equation:

[0097] In the formula, This represents gyroscope measurement data (IMU core input). , This represents the attitude angle transformation matrix, which is determined by the current attitude.

[0098] Step 3: Establish measurement equations and introduce GNSS data to constrain the state.

[0099] The core of the measurement equations for GNSS-IMU integrated navigation is establishing the constraint relationship between GNSS observations and the inertial state. The measurement update stage used for Kalman filtering constrains the cumulative error of the IMU through high-precision GNSS position and velocity observations.

[0100] The measurement equation is as follows:

[0101] In the formula, Indicates GNSS measurement value, (6-dimensional); This represents the measurement function (non-linear), describing the mapping from navigation state to GNSS measurements. Since GNSS measurements directly correspond to the position and velocity of the navigation state, therefore... (Only position and velocity terms are extracted; attitude is ignored); Indicates 9-dimensional navigation status ; This indicates GNSS measurement noise.

[0102] Step 4: Achieve optimal estimation of the orbital state through the Kalman filter iterative process (prediction and update).

[0103] Input for Step 4: Gyroscope angular velocity (3D) + accelerometer specific force (3D), and the measurement value of IMU sampling time k. The input for Step 4 includes the angular velocity / accelerometer data to be given to the IMU in each step, the GNSS position / velocity data, and the results calculated in the previous step.

[0104] The output of step 4 is the final filtered orbital position (latitude and longitude), velocity (ENU), and attitude (roll / pitch / heading). The output of step 4 is the most accurate orbital position, velocity, and attitude at the current moment.

[0105] The specific process of step 4 is as follows: Prediction: Using IMU data to "guess" the current trajectory (like predicting the next position by feeling while driving, calculated at high frequency, such as 1000 times per second); Update: If GNSS data is available, use the quasi-GNSS data to "correct" the previous guess (like when checking navigation to correct deviations, update at a low frequency, such as once per second). Optimal estimation of orbital state: Using a combination of IMU (high frequency but prone to drift) and GNSS (low frequency but accurate) data, calculate the result that is closest to the actual orbit under the existing conditions (not without error, but with minimal error).

[0106] Step 5: Based on the determined track status, determine the track alignment.

[0107] In some examples, step 5 includes: The curvature of the track at each point is calculated using numerical differentiation, and the track shape is determined based on the curvature changes. Alternatively, based on different orbital shapes in historical states, several labeled training samples can be constructed, and a orbital shape detection model can be trained using these training samples. The orbital shape can then be determined using the orbital shape detection model.

[0108] More specifically, the specific process of calculating the curvature at each point on the track using numerical differentiation methods and determining the track's shape based on the curvature changes is as follows: A curvature threshold is set: when the curvature value of a certain track segment is close to 0, it is judged as a straight line; if the curvature value remains constant within a certain range, it is judged as a curve; if the curvature value shows a linear trend, it is judged as a transition curve. The collected track data undergoes preprocessing such as filtering and noise reduction to eliminate measurement errors and interference factors. The curvature of each point on the track is calculated using numerical differentiation methods. For discrete track point sequences... The curvature can be approximated using the second-order difference method. The curvature calculation formula is as follows:

[0109] More specifically, based on the different orbital shapes in historical states, several labeled training samples are constructed, and a orbital shape detection model is trained using these training samples. The specific process of determining the orbital shape using the orbital shape detection model is as follows: Using machine learning algorithms, a classification model is built by training on a large amount of labeled track alignment data to learn the characteristic patterns of different alignments and thus determine the alignment of new track data. Track data containing various alignments, including straight lines, curves, and transition curves, is collected and manually labeled. The data is divided into training, validation, and test sets. Multiple features of the track data are extracted, such as curvature, slope, coordinate sequences of track points, and rate of change of curvature. The trained model is evaluated using the validation and test sets, and model parameters are adjusted to improve accuracy. After passing the evaluation, the model is applied to determine the alignment of actual track data.

[0110] Step 6: Divide the track into segments based on the alignment results.

[0111] In some examples, step 6 includes: The orbital data is used as points in a spatial point set, where the orbital data includes curvature, slope, and coordinates; Clustering algorithms are used to cluster points in the point set, grouping points with similar structural features into one class, and then segmenting the track.

[0112] More specifically, the clustering process is as follows: Treating track data as a set of points in space, clustering algorithms are used to group points with similar geometric features into one class, with each class corresponding to a track segment. Data representation: The track data is converted into a feature vector format suitable for clustering, such as using information like the curvature, slope, and coordinates of the track points to form a multi-dimensional feature vector. First, the number of clusters K is determined, and K cluster centers are randomly initialized. The distance (e.g., Euclidean distance) from each data point to each cluster center is calculated, and the data point is assigned to the class belonging to the nearest cluster center. The center of each class is recalculated, and this process is repeated until the cluster centers no longer change significantly or the maximum number of iterations is reached. Each clustering result corresponds to a track segment, and the point where the cluster center is located or the transition point between adjacent clusters can be used as a segmentation point.

[0113] In this application, multi-source information such as acceleration / angular velocity data from an inertial measurement unit (IMU), relative displacement data from an odometer, and absolute position information from satellite positioning (such as GPS) is fused, and algorithms (such as Kalman filtering) are used to dynamically correct errors. While the IMU can sense motion in real time, it suffers from accumulated errors. Therefore, an odometer and external positioning are used to provide auxiliary calibration, ultimately outputting high-precision position, attitude, and velocity information to ensure accurate measurement of the track inspection vehicle's own trajectory.

[0114] Based on the precise pose data provided by integrated navigation, and combined with the inertial reference method, the track inspection vehicle is treated as a "mobile reference platform." IMUs measure vehicle vibration (vertical and lateral acceleration) and attitude changes (pitch and roll angles), and combined with odometer displacement data, geometric deviations such as track elevation, level, and gauge are calculated through integration. For example, the integral of the vehicle's vertical vibration reflects the longitudinal undulations of the track, while changes in lateral acceleration correspond to track irregularities. Millimeter-level accuracy detection is achieved through multi-sensor data fusion.

[0115] Integrated navigation ensures the stability of the measurement benchmark, while the inertial benchmark method directly correlates with the orbital geometric deformation. The combination of the two enables dynamic, real-time, and high-precision detection.

[0116] S105. Determine the track condition based on the track defect detection results and track spatial alignment detection results.

[0117] Figure 9 This is a structural block diagram of a track condition detection device according to an embodiment of this application. See also... Figure 9 ,include: The first acquisition module 11 is used to acquire two-dimensional images and three-dimensional point clouds. The track defect detection module 12 is used to detect track defects based on two-dimensional images and three-dimensional point clouds; The second acquisition module 13 is used to acquire IMU data and GNSS data; The track space alignment detection module 14 is used to perform track space alignment detection based on IMU data and GNSS data; The determination module 15 is used to determine the track status based on the track defect detection results and the track spatial alignment detection results.

[0118] Figure 10 This is a structural block diagram of an electronic device provided according to an embodiment of this application. See also... Figure 10 Electronic devices may include Figure 9 The aforementioned track status detection device. Typically, the electronic equipment includes: a processor 21 and a memory 22.

[0119] Processor 21 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 21 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 21 may also include a main processor and a coprocessor. The main processor is used to process data in the wake-up state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor used to process data in the standby state. Memory 22 may include one or more computer-readable storage media, which may be non-transitory. Memory 22 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in memory 22 is used to store at least one instruction, which is executed by processor 21 to implement the track state detection method performed by an electronic device provided in the method embodiments of this application.

[0120] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting track status, characterized in that, include: Acquire 2D images and 3D point clouds; Track defect detection is performed based on two-dimensional images and three-dimensional point clouds; Acquire IMU and GNSS data; Orbital spatial alignment detection based on IMU and GNSS data; The track condition is determined based on the results of track defect detection and track spatial alignment detection.

2. The track state detection method according to claim 1, characterized in that, The steps for detecting track defects based on two-dimensional images and three-dimensional point clouds are as follows: A fused image is obtained by fusing the two-dimensional features of a two-dimensional image and the depth features of a three-dimensional point cloud. A first neural network model is constructed, and training samples are built based on the fused images under historical conditions to train the first neural network model and obtain the track defect detection model. The fused image is input into the track defect detection model to obtain the track defect detection results.

3. The track state detection method according to claim 2, characterized in that, The steps for fusing two-dimensional features from a two-dimensional image and depth features from a three-dimensional point cloud include: Preprocessing of 2D images and 3D point clouds; A second neural network model is constructed, comprising a 2D encoder, a 3D encoder, a cross-modal fusion module, and a decoder. The 2D encoder is used to acquire two-dimensional features of a two-dimensional image, and the 3D encoder is used to acquire depth features of a three-dimensional point cloud. The cross-modal fusion module is used to fuse the two-dimensional features and the depth features. The decoder is used to form a fused image based on the fused two-dimensional features and depth features. Training samples were constructed using two-dimensional images and three-dimensional point clouds from historical states to train the second neural network model, resulting in an image fusion model. The preprocessed 2D image and 3D point cloud are input into the image fusion model to obtain the fused image.

4. The track state detection method according to claim 2, characterized in that, After obtaining the defect detection results of the track, the method further includes: The three-dimensional depth image is preprocessed, and edge features of the detected objects are extracted by Canny edge detection, edge regions of straight objects are detected by Hough circle transform, and edge regions of curved objects are detected by probabilistic Hough line transform. Finally, distance information is obtained by combining the extracted edges with the pixel size, thereby determining the size information of the disease.

5. The orbital state detection method according to any one of claims 1 to 4, characterized in that, The steps for orbital spatial alignment detection based on IMU and GNSS data include: A state vector is constructed based on IMU data, mileage data, and sensor error parameters; Establish state equations to describe the relationship between inertial measurement data and state variables; Establish measurement equations and introduce GNSS data to constrain the state; The optimal estimation of the orbital state is achieved through an iterative Kalman filter process. Based on the determined orbital state, the orbital alignment is determined; The track is segmented based on the results of the alignment assessment.

6. The track state detection method according to claim 5, characterized in that, Based on the determined orbital state, the orbital alignment determination includes: The curvature of the track at each point is calculated using numerical differentiation, and the track shape is determined based on the curvature changes. Alternatively, based on different orbital shapes in historical states, several labeled training samples can be constructed, and a orbital shape detection model can be trained using these training samples. The orbital shape can then be determined using the orbital shape detection model.

7. The track condition detection method according to claim 5, characterized in that, The steps for segmenting the track based on the alignment assessment results include: The track data is used as data points, and a point set is formed based on several data points; the track data includes curvature, slope, and coordinates; Clustering algorithms are used to cluster data points in the point set, grouping points with similar geometric features into one class, and then segmenting the track.

8. The track condition detection method according to claim 5, characterized in that, The IMU data includes acceleration data and attitude angle data, and the sensor error parameters include acceleration error and mounting angle error.

9. The track condition detection method according to claim 5, characterized in that, The method for determining the installation angle error is as follows: Based on the principle of coordinate system transformation, an error model including installation angle error parameters is established; An optimization algorithm is used to solve for the installation angle error parameters in the error model, and the combination of the installation angle error parameters of the measured data and the theoretical data is obtained as the installation angle error.

10. A track condition detection device, characterized in that, include: The first acquisition module is used to acquire two-dimensional images and three-dimensional point clouds. The track defect detection module is used to detect track defects based on two-dimensional images and three-dimensional point clouds. The second acquisition module is used to acquire IMU data and GNSS data; The orbital spatial alignment detection module is used to perform orbital spatial alignment detection based on IMU data and GNSS data; The determination module is used to determine the track status based on the track defect detection results and the track spatial alignment detection results.

Images