Train advancing track autonomous verification system based on spatio-temporal feature fusion

Abstract

The invention relates to the technical field of train tracks, and discloses a train traveling track autonomous verification system based on spatio-temporal feature fusion, which comprises a multi-source sensing acquisition module used for acquiring spatio-temporal information of track environments in front of and behind a train, a sensing data preprocessing module used for filtering a three-dimensional point cloud of the spatio-temporal information, and a verification module used for verifying the three-dimensional point cloud of the spatio-temporal information. The target identification and feature extraction module is used for identifying a signal lamp state based on an image and a depth map of spatio-temporal information; the traffic decision and track verification module is used for issuing a communication and instruction according to a target identification result; the method is used for detecting the running state of each module of the system. By fusing point cloud data, image information and a GNSS positioning result, a unified spatial-temporal characteristic model is constructed, high-precision fitting and dynamic offset judgment of a train running track are achieved, and the effect that the track detection precision is remarkably superior to that of a single sensor scheme is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of train tracks, and in particular to an autonomous train track verification system based on spatiotemporal feature fusion. Background Art

[0002] With the rapid development of automated train control and intelligent transportation systems, train positioning accuracy and trajectory stability are becoming increasingly critical elements for ensuring operational safety and improving system efficiency. In particular, in high-speed rail, urban rail transit, and complex marshaling operations, real-time acquisition and deviation detection of train trajectories directly impact path planning, collision warning, and the stable operation of autonomous driving systems.

[0003] Existing technologies often rely on GNSS (Global Navigation Satellite System) or ground-based odometers and inertial navigation systems. However, relying solely on GNSS presents challenges such as signal obstruction, large drift errors, and unstable positioning in tunnels and complex urban environments. Ground-based landmarks are also highly dependent on these systems, resulting in high deployment costs and difficulty adapting to the dynamic operating environment. Furthermore, a lack of awareness of the track itself and surrounding obstacles makes trajectory determination difficult to achieve in terms of environmental adaptability and real-time performance. Summary of the Invention

[0004] In response to the shortcomings of the existing technology, the present invention provides an autonomous train trajectory verification system based on the fusion of spatiotemporal features. It solves the problem that in the existing technology, the detection and verification of train running trajectories often rely on a single type of sensing means, using only GNSS for positioning and tracking, or relying on image recognition to estimate the track centerline. Such methods can meet basic positioning requirements in ideal or open environments, but in actual railway operation scenarios, they face the problem of insufficient detection accuracy due to multiple interference factors.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solutions: a train trajectory autonomous verification system based on spatiotemporal feature fusion, comprising: Multi-source sensing and acquisition module, used to obtain spatiotemporal information of the track environment in front of and behind the train; The perception data preprocessing module is used to filter, crop regions, and extract boundaries of 3D point clouds of spatiotemporal information, and perform histogram enhancement and depth map construction on image data; The target recognition and feature extraction module is used to identify the status of traffic lights, switch connections, and obstacle locations based on spatiotemporal images and depth maps, and extract their 3D semantic features based on point cloud geometry information. The trajectory modeling and deviation detection module is used to fit the current running trajectory based on the track edge points and GNSS trajectory, and determine whether the train deviates from the designed track; The passage decision and trajectory verification module is used to determine whether the train meets the passage conditions based on the target recognition results and trajectory modeling results, and output the instructions of allowing passage, restricting passage or stopping; The communication and command issuing module is used to send the trajectory judgment results and system status to the train control system through the CAN communication protocol; The system self-check and fault monitoring module is used to detect the operating status of each module in the system and output the system fault status when the data is abnormal or the recognition fails.

[0006] Preferably, the spatiotemporal information includes three-dimensional laser radar point cloud images in front of and behind the vehicle, image data collected by a binocular camera, and GNSS positioning data; After acquisition, the 3D lidar point cloud is downsampled through voxel filtering, and the point cloud within 60 meters in front of the track is retained using a region cropping method. The left and right track edge lines and center lines are extracted based on the RANSAC algorithm, and curvature analysis is achieved in combination with frame sliding trajectory estimation.

[0007] Preferably, the perception data preprocessing module performs histogram equalization and bilateral filtering enhancement on the image data, and generates a depth map by combining the binocular image pair. Figure 1 And input into the target detection network improved by YOLO architecture to identify signal lights, switch structures and obstacles.

[0008] Preferably, the trajectory modeling and deviation detection module constructs the track centerline and compares the GNSS real-time trajectory with the centerline, determines whether there is a trajectory deviation by calculating the trajectory offset and the heading angle, and outputs a deviation alarm when it detects that the vehicle has entered the switch area by mistake or the trajectory has suddenly changed.

[0009] Preferably, the passage decision and trajectory verification module makes decisions based on the signal light status, switch status, obstacle spatial position and trajectory matching degree, wherein when the shortest distance between the obstacle and the track centerline is lower than the set threshold, it is deemed to be impassable, and when the signal light and switch status are inconsistent, confirmation is made through a multi-frame voting mechanism.

[0010] Preferably, the communication and instruction issuing module sends the judgment result in the form of a structured message via CAN communication; The message includes obstacleStatus, lightStatus, turnoutStatus, obstacleDistance and systemStatus fields, and supports real-time refresh frequency.

[0011] Preferably, the system self-check and fault monitoring module includes three parts: image perception anomaly detection, laser occlusion recognition and recognition confidence assessment. When any module fails to recognize, data is missing or occluded severely for more than five consecutive frames, a system fault flag will be output and synchronized to the train main control system.

[0012] Preferably, the multi-source perception acquisition module includes a combination of a laser radar and a binocular camera located at the front end of the train; The laser radar has a horizontal viewing angle of 120 degrees and a vertical angle range of -10 to +10 degrees. The binocular camera's viewing angle covers a range of 2.5 meters to the left and right of the track center, which is used to realize environmental modeling and status recognition of the train's forward direction.

[0013] Preferably, the target recognition and feature extraction module performs confidence fusion on the data from the image and lidar, and only passes the result to the traffic decision module when the traffic light recognition result meets the template conditions in both color recognition and spatial positioning and the recognition confidence exceeds the threshold, so as to avoid misrecognition and misjudgment.

[0014] The train trajectory autonomous verification method based on spatiotemporal feature fusion includes the following steps: Step 1: Collect point cloud, image, and GNSS positioning data of the environment in front of the train; Step 2: Enhance the image, filter and crop the point cloud to generate a depth map; Step 3: Identify signal lights, switch connection status, and obstacles, and extract their 3D semantic features; Step 4: Fit the orbit centerline and combine it with the GNSS trajectory to determine whether there is trajectory deviation; Step 5: Make a traffic safety assessment based on the target recognition results and trajectory judgment; Step 6: Send the track verification status to the train control system through the CAN communication module; Step 7: Monitor the system operation status in real time and output a system fault signal when identification fails or data is abnormal.

[0015] The present invention provides an autonomous train trajectory verification system based on spatiotemporal feature fusion. It has the following beneficial effects: 1. The present invention integrates point cloud data, image information and GNSS positioning results to construct a unified spatiotemporal feature model, achieving high-precision fitting and dynamic offset determination of train running trajectories, and achieving trajectory detection accuracy significantly better than that of a single sensor solution.

[0016] 2. The present invention realizes multimodal fusion judgment of traffic safety status by identifying traffic light status, turnout direction and obstacle information and extracting their three-dimensional semantic features, thereby obtaining a highly robust autonomous traffic judgment effect that can adapt to complex line scenarios.

[0017] 3. By constructing an adaptive abnormality recognition and fault output mechanism, the present invention can actively identify system abnormal states when the image is blurred, the point cloud is occluded, or the recognition confidence level decreases, thereby achieving system-level security protection and improving operational reliability and system controllability.

[0018] 4. The present invention transmits the trajectory status, target recognition results and system operation status to the train main control system in real time through the CAN communication module, thereby achieving efficient coordination between the trajectory verification results and the train control system, and enhancing the closed-loop control capability of the train's fully automatic operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a stereoscopic diagram of the process flow of the binocular camera data reading module of the train track autonomous verification system based on spatiotemporal feature fusion of the present invention; Figure 2 This is a flow chart of the binocular camera target detection module of the train trajectory autonomous verification system based on spatiotemporal feature fusion of the present invention; Figure 3 This is a schematic diagram of the message file processing flow of the train trajectory autonomous verification system based on spatiotemporal feature fusion of the present invention; Figure 4 This is a diagram showing the relationship between system modules of the autonomous train trajectory verification system based on spatiotemporal feature fusion according to the present invention; Figure 5 The figure is a flow chart of the method for autonomous verification of train trajectory based on spatiotemporal feature fusion of the present invention. DETAILED DESCRIPTION

[0020] The following will clearly and completely describe the technical solution of the present invention in conjunction with the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0021] Please see the attached Figure 1 -Attached Figure 4 The embodiment of the present invention provides a train trajectory autonomous verification system based on spatiotemporal feature fusion, including: Multi-source sensing and acquisition module, used to obtain spatiotemporal information of the track environment in front of and behind the train; The perception data preprocessing module is used to filter, crop regions, and extract boundaries of 3D point clouds of spatiotemporal information, and perform histogram enhancement and depth map construction on image data; The target recognition and feature extraction module is used to identify the status of traffic lights, switch connections, and obstacle locations based on spatiotemporal images and depth maps, and extract their 3D semantic features based on point cloud geometry information. The trajectory modeling and deviation detection module is used to fit the current running trajectory based on the track edge points and GNSS trajectory, and determine whether the train deviates from the designed track; The passage decision and trajectory verification module is used to determine whether the train meets the passage conditions based on the target recognition results and trajectory modeling results, and output the instructions of allowing passage, restricting passage or stopping; The communication and command issuing module is used to send the trajectory judgment results and system status to the train control system through the CAN communication protocol; The system self-check and fault monitoring module is used to detect the operating status of each module in the system and output the system fault status when the data is abnormal or the recognition fails.

[0022] Spatiotemporal information includes 3D LiDAR point cloud images in front of and behind the vehicle, image data collected by binocular cameras, and GNSS positioning data; After acquisition, the 3D lidar point cloud is downsampled through voxel filtering. The region cropping method is used to retain the point cloud within 60 meters in front of the track. The left and right track edge lines and center lines are extracted based on the RANSAC algorithm, and curvature analysis is achieved by combining frame sliding trajectory estimation.

[0023] Specifically, as one of the main visual components, the three-dimensional laser radar is responsible for modeling the spatial structure of the track ahead. In order to improve the processing efficiency and modeling accuracy of point cloud data, a series of pre-processing operations are required after point cloud collection. This section details the specific implementation method and processing flow of this part. Due to the huge number of original point clouds collected by the laser radar during operation, direct modeling without processing will not only affect the calculation efficiency, but may also introduce redundant interference points, especially in rainy fog and sudden changes in light. Therefore, it is necessary to reduce the dimension of the initial data and perform regional screening; The 3D LiDAR point cloud is first downsampled using a voxel grid filter. The core idea of voxel filtering is to divide the 3D space into a number of regular voxel units, retaining a representative point near the center of each voxel. As an option, the voxel size is set to 0.1 meters to balance accuracy and computational speed. In actual operation, this size can meet the requirements of track resolution under high-speed operation without causing significant geometric distortion. After completing the voxel filtering, the region clipping step will be entered. The cropping operation limits the range of the laser point cloud in the Z-axis (height) and XY plane (forward and left / right). The forward direction is set to 0 to 60 meters, the left / right direction is set to 5 meters each, based on the track centerline, and the vertical height range is -1.5 meters to +2 meters. This range essentially covers key detection elements such as the track area, lateral structures, and low-lying obstacles. The cropping operation significantly compresses the point cloud, enhancing the focus of subsequent boundary fitting. The cropped point cloud is then processed for track edge extraction. Pass-through filtering is first used to filter near-ground points. A Z-axis threshold range of ±0.2 meters above ground level is set to identify feature points that may lie on the track surface. Linear fitting based on the RANSAC (Random Sample Consensus) algorithm is then used to model the left and right track edges. RANSAC iterations are set to 1000, and the inlier threshold is set to 0.03 meters to improve the robustness of edge line detection. The edge points are projected onto the XY plane and then fitted to reduce the error accumulation caused by height fluctuations. After the left and right track boundaries are fitted, the midpoint is calculated to further generate the track centerline. The track centerline is not only used as a reference for the current attitude, but also forms the basis for trajectory modeling and offset detection. Combined with the train positioning trajectory provided by GNSS, this system uses a frame sliding window strategy for trajectory sequence fitting and curvature modeling. In each time frame, the centerline fitting results in the past few seconds are selected, and combined with the real-time GNSS positioning points to form a time series trajectory segment. By calculating the derivative change of the trajectory model in the forward direction at the current moment, the curvature change trend is obtained. When entering a curve or switch area, the system can output a curvature increase signal in advance to assist in judging whether the train is in a reasonable passage area. Noise filtering and abnormal point elimination strategies are also considered in the trajectory modeling process. For example, the historical trajectory points are mean filtered before fitting to eliminate errors caused by short-term GPS jumps or point cloud mutations. The lidar point cloud preprocessing and track edge extraction process can achieve stable modeling of the track structure, and high-precision curvature estimation can be performed in conjunction with the GNSS sliding trajectory, providing strong support for offset detection. This technical solution effectively solves the problems of large errors and instability in traditional single-frame trajectory judgment. The perception data preprocessing module performs histogram equalization and bilateral filtering enhancement on the image data, and generates a depth map by combining the binocular image pair. Figure 1 And input into the target detection network improved by YOLO architecture to identify signal lights, switch structures and obstacles.

[0024] Specifically, the visual perception function is mainly realized by the binocular camera installed at the front of the train, aiming to accurately identify key structures and dynamic targets in the track environment. This module not only provides image information, but also constructs a depth map for spatial positioning; The captured raw image is first preprocessed. Histogram equalization is used to improve brightness distribution and enhance overall contrast, making color objects such as traffic lights easier to distinguish. To reduce noise interference while maintaining edge clarity, a bilateral filtering operation is further introduced. This step removes random noise from the image while preserving details such as obstacle boundaries and turnout segments. Next, the system constructs a depth map based on the left and right camera images, uses a semi-global matching method to extract the disparity map, and combines the camera calibration parameters to obtain the depth value. These depth information can be directly paired with the image frame to locate the detected object in space, and the image and depth map are jointly input into a YOLO series improved network. Structurally, the network introduces an attention mechanism in the backbone feature extraction part to improve the response effect to small targets, and adds channel attention or spatial enhancement modules to improve the recognition accuracy of long-distance traffic lights and switch connection heads. The detection model not only outputs a two-dimensional detection frame, but also returns the spatial coordinates and distance estimation of the target in combination with the depth map. In some scenarios, such as signal recognition before entering the station, its output will be accompanied by a confidence parameter for linkage judgment with the traffic decision module; When the target confidence exceeds the set threshold and the spatial location coincides with the track center area, it is marked as a valid detection result. The model adopts a lightweight inference deployment strategy, which can run stably at a high frame rate on the onboard edge device, balancing real-time performance and accuracy. In addition, to adapt to diverse track scenarios, a rich variety of typical image data is introduced during the training phase, including complex backgrounds such as platform occlusion, foreign object placement, and rain and snow reflections. The three-dimensional semantic output formed by the image recognition module combined with depth information can not only support target alignment in the trajectory modeling process, but also provide reliable spatial input for obstacle judgment, greatly enhancing the system's adaptability in complex environments.

[0025] The trajectory modeling and deviation detection module constructs the track centerline and compares the real-time GNSS trajectory with the centerline. It determines whether there is trajectory deviation by calculating the trajectory offset and heading angle, and outputs a deviation alarm when it detects that the vehicle has entered the switch area or the trajectory has suddenly changed.

[0026] Specifically, trajectory modeling and deviation detection adopt a construction method based on multi-source fusion, aiming to evaluate the consistency between the train's running trajectory and the expected path in real time. The core is to use the track structure information extracted by three-dimensional lidar, combined with GNSS positioning results, to build a trajectory reference model and perform deviation analysis. After the track edge line is obtained by fitting the lidar point cloud data, its center line is further calculated. The center line serves as the spatial reference benchmark for the ideal running path of the train. In some implementations, the centerline interpolation method will be used to smooth the edge line fitting results to ensure the continuity and curvature consistency of the center line to avoid misjudgment; To enhance positioning stability, a short-time sliding window is used to perform mean filtering on GNSS data to reduce the impact of signal jitter. The trajectory modeling module compares the current GNSS coordinate point with the orbit centerline, and calculates the angle between the lateral offset distance and the heading angle. When the offset distance exceeds the set threshold, or the heading angle changes suddenly, such as a sharp deviation from the centerline direction, it can be considered that there is a risk of trajectory deviation. To avoid false alarms caused by short-time errors, the system is designed with a continuity judgment mechanism. Only when the deviation lasts for more than a certain period of time or number of frames will a formal deviation signal be output; Spatial topological structure model of the turnout area. When the train position is detected to fall within the non-target turnout branch trajectory range, and the actual trajectory is obviously deviated from the main line trajectory, it will be judged as "mistakenly entering the turnout." This judgment method combines the three elements of spatial position, directional trend and turnout opening and closing status to improve the accuracy and robustness of the judgment. In actual deployment, this module updates the trajectory model and deviation analysis results every 100 milliseconds, which can meet the dynamic response requirements of medium-speed train operation environment; Trajectory modeling output not only provides data support for deviation detection but also serves as a fundamental input for traffic flow determination. By continuously monitoring trajectory status, path deviations, abnormal movements, or track switching during operation can be quickly captured, providing the system with a stable basis for spatial decision-making.

[0027] The passage decision and trajectory verification module makes decisions based on the signal light status, switch status, obstacle spatial position and trajectory matching. When the shortest distance between the obstacle and the track centerline is lower than the set threshold, it is deemed impassable. When the signal light and switch status are inconsistent, confirmation is made through a multi-frame voting mechanism.

[0028] Specifically, the passage decision and trajectory verification module is responsible for comprehensively judging the safety of trains before they start running. Based on multiple types of information from the perception and modeling units, it comprehensively determines whether the train is qualified to enter the target track section. The judgment logic revolves around the core dimension of "passability." Input data mainly includes three categories: the current status of the signal light, the connection status of the switch structure, and the spatial location information of the obstacles ahead. In addition, the degree of match between the track centerline provided by the trajectory modeling module and the current GNSS trajectory of the train is also used as a key reference. The module first determines whether the trajectory status is normal. If there is a significant deviation, the system will automatically enter the pending or pause state and will not perform further traffic judgment. Based on the consistency of the trajectory, it will then determine whether there is a structural blockage in the passage. For obstacle detection, the system uses the shortest distance between the center of the obstacle and the centerline of the track as the primary basis, setting a minimum safe distance threshold of 1.2 meters. If any obstacle is detected to be less than this distance from the centerline of the track, the location is deemed unsafe to pass and is directly marked as "blocked." The module will also make a weighted judgment based on the longitudinal position of the obstacle, its motion state (if determinable), and the degree of occlusion in the image. If the obstacle is located at the edge of the track and is spatially circumventable, it will enter the "pass pending confirmation" state for access and confirmation by the upper control system. For the logical consistency between the signal light and the switch, this module does not rely on the results of a single frame, but introduces a multi-frame voting mechanism. Within a period of time (such as 5 consecutive frames), if the signal light state is red and the switch state points to the target track, or other inconsistencies with the standard interlocking occur, it is considered that there is a state conflict and a blocking judgment is output. This voting strategy is mainly to solve occasional errors caused by recognition jitter, short-term occlusion or misrecognition, and to ensure the stability of decision-making. During the voting window, if the system detects that the state is stable, the pass judgment result will take effect; Ultimately, the module encapsulates the fusion judgment results into a traffic status output, which is divided into three categories: "Passage Allowed," "Passage Restricted," and "Traffic Stopped." Optionally, the traffic status can also be accompanied by a reason identification field, such as "Obstacle Too Close," "Track Deviation," or "Signal Abnormal," for subsequent module or manual review.

[0029] The communication and instruction issuing module sends the judgment results in the form of structured messages through CAN communication; The message includes the obstacleStatus, lightStatus, turnoutStatus, obstacleDistance, and systemStatus fields, and supports real-time refresh rate.

[0030] Specifically, the communication and command issuing module is used to transmit the traffic decisions, trajectory status and recognition results within the system to the train controller or upper decision-making system in real time. This module uses the CAN (Controller Area Network) bus protocol as a communication carrier to ensure the reliability and anti-interference ability of data transmission. Each frame message has a fixed structure and adopts a standard frame format. The system status data is encapsulated into structured fields and packaged into a unified communication message for delivery. The data frame content is designed with versatility and compactness in mind, making it easy for the on-board control unit to directly parse it. The communication message mainly includes the following fields: obstacleStatus: indicates the status of the obstacle ahead, including its existence, type, and whether it is blocked; lightStatus: reflects the current recognition status of the traffic light, classified according to standard traffic light status, such as red, green, and unknown; turnoutStatus: records the current physical connection direction of the turnout, indicating whether it is pointing to the target track segment; obstacleDistance: The distance between the nearest obstacle currently identified and the front of the train, in meters; SystemStatus: used to reflect whether the current system is operating normally, some functions are invalid, or there is an identification anomaly; The data field is encapsulated using Boolean values or simplified encoding, and the overall message size is controlled between 8 and 16 bytes to ensure efficient transmission in standard CAN frames. To adapt to the rapidly changing environment ahead during train operation, the module supports a high-frequency refresh mechanism. In actual deployment, the system continuously updates the latest status at a refresh cycle of 100 milliseconds, that is, at a rate of more than 10 frames per second. If no system response is received within several consecutive frames, the upper-level control system can determine that the communication is interrupted and automatically trigger the protection logic; The module supports intra-frame parity bits and timestamp fields to prevent miscoding caused by message disorder or excessive delay. As an extension mechanism, an independent "heartbeat signal" channel can also be set to periodically identify the system as active. In addition, to facilitate debugging and maintenance, the module also provides a debug port to output the complete communication data stream. In an experimental or test environment, a CAN analyzer can be used to read the state changes of each frame in real time to verify whether the data content is consistent with the logical judgment. The communication module establishes a clear, stable, and timely bridge between the perception and control layers, playing a key role in the overall operation of the train system. It not only transmits information about anomalies such as track deviations and blocked passages, but also provides continuous confirmation signals when the system is stable, supporting the safe and autonomous operation of the train.

[0031] The system self-check and fault monitoring module includes three parts: image perception anomaly detection, laser occlusion recognition and recognition confidence assessment. When any module fails to recognize, has missing data or is severely occluded for more than five consecutive frames, a system fault flag will be output and synchronized to the train main control system.

[0032] Specifically, the system self-check and fault monitoring module is used to evaluate the operating status of key perception and recognition functions in real time. This module belongs to the monitoring and fault tolerance layer in the overall system architecture, responsible for proactively detecting potential problems such as perception degradation, data anomalies, or calculation failures. This module consists of three parts: image perception anomaly detection, laser occlusion recognition, and recognition confidence assessment. These three parts operate independently in a modular manner and uniformly output system-level status signals. Image perception anomaly detection focuses on the data stream input from the binocular camera. It uses indicators such as image clarity assessment, frame rate monitoring, and brightness histogram distribution to determine whether the image is blurry, frame dropped, too dark, or has abnormal exposure. When the image continuously exhibits low contrast, blurred edges, and inability to focus, it can be determined that the image perception module is in an abnormal state. The laser occlusion recognition part focuses on changes in point cloud density, coverage, and scanning intensity. The system will dynamically count the number of point clouds within a specified range in front (such as within 5 meters in front). When the number of valid points in the area is lower than the set threshold (such as less than 30% of the original point cloud volume), or when large areas of holes appear in the scan layer, it can be judged as a forward occlusion anomaly, which may be caused by rain, snow, accumulated water, transparent obstacles, etc. The recognition confidence assessment part is used to determine whether the model continues to output reliable results in multi-target detection tasks. In the present invention, key categories of targets such as traffic lights, obstacles, and switches are required to exceed the set confidence threshold (such as 0.6), otherwise it is considered a recognition failure. If the confidence of multiple consecutive frames of output is lower than the standard, the system will trigger a recognition stability alarm.

[0033] This module uses a frame counting method to perform fault judgment. When any submodule encounters recognition failure, image damage, point cloud occlusion, or low confidence in more than five consecutive frames, it is considered that the current system has entered an abnormal working state. At this time, a system fault flag will be immediately generated and synchronously transmitted to the train main control system through the communication module. This flag can trigger emergency braking, manual intervention, or fault reset processes. A fault self-recovery detection mechanism can also be set. That is, when the fault duration is less than a certain set window (such as 2 seconds) and returns to normal, the system automatically clears the fault flag and returns to the operating state. This maintains sensitivity to anomalies while avoiding false triggering; The module provides a status code output interface for logging and remote diagnosis. Each fault type is assigned a unique code, which the main control system uses to quickly trace the source and isolate the function. This self-test module provides the system with basic health awareness capabilities, effectively enhancing overall operational stability and safety protection capabilities. It is particularly suitable for railway scenarios that require long-term, high-continuity operation.

[0034] The multi-source perception acquisition module includes a LiDAR and binocular camera combination located at the front of the train; The laser radar has a horizontal viewing angle of 120 degrees and a vertical angle coverage of -10 to +10 degrees. The binocular camera's viewing angle covers a range of 2.5 meters to the left and right of the track center, which is used to realize environmental modeling and status recognition of the train's forward direction.

[0035] Specifically, the forward environment modeling and state recognition functions are jointly realized by a multi-source sensor combination installed on the head of the train. This combination mainly includes a laser radar and a binocular camera, which work together to provide high-precision spatial structure modeling and image semantic information extraction capabilities respectively. The laser radar adopts a fixed forward installation method, located above the center axis of the train. The installation height can be set between 1.5 and 2.2 meters depending on the train model to avoid obstruction by the front structure and optimize the downward angle. The laser radar has a 120-degree field of view coverage in the horizontal direction and a vertical coverage range of -10° to +10°. This angle setting ensures that the laser beam can achieve wide detection in the direction of the train's advance, covering the main track area and the buffer zones on both sides. At the same time, it provides a certain angle of up and down detection capability for identifying abnormal terrain such as protrusions, track interruptions or depressions. The vertical angle coverage angle can also effectively capture the height difference information between the ground and obstacles, which is of great significance for building accurate track edge line and centerline models. In conjunction with the LiDAR, a binocular camera is also fixedly mounted in front of the train, with the horizontal angle of the lens aligned with the track centerline. This provides strong imaging stability and vibration resistance. The binocular camera's field of view is designed to cover a range of 2.5 meters to the left and right of the track center. The vertical viewing angle is configured according to the focal length and installation height to ensure complete imaging within a distance of 6 to 30 meters. This setting ensures that the system can simultaneously observe the sleeper area on both sides, the switch opening, and near-track obstacles based on the normal track width (standard track 1.435 meters), achieving complete trackside visual perception. The data collected by the two types of sensors are synchronized in time and aligned in spatial coordinates. The system uses a unified coordinate system modeling mechanism to map point clouds, images, depth maps, and GNSS information to the same reference frame, making it easier for subsequent processing modules to efficiently perform target fusion and decision-making. To enhance the system's perception stability under occlusion or single-point failure conditions, the lidar and binocular camera can be set to redundant complementary operation mode. When image occlusion or exposure anomalies are detected, the distance and position of the obstacle can be determined solely by the point cloud; conversely, when the laser data is weakened, the image channel maintains semantic-level perception of signal lights, turnout directions, etc. This combined perception mechanism enables the system to have full structural coverage within a range of at least 30 meters in front of the train, ensuring multi-angle and cross-modal understanding of the operating environment, and providing basic data support for trajectory fitting, obstacle detection and passage judgment.

[0036] The target recognition and feature extraction module performs confidence fusion on the data from the image and lidar. Only when the traffic light recognition results meet the template conditions in both color recognition and spatial positioning and the recognition confidence exceeds the threshold, the results are passed to the traffic decision module to avoid misidentification and misjudgment.

[0037] Specifically, the target recognition and feature extraction module is used to extract key information related to train traffic decisions from images and lidar data, including signal light status, switch structure, and obstacles ahead. In order to reduce the probability of misidentification of a single data source in complex environments, the system has designed a confidence-based multimodal fusion mechanism to ensure the stability and credibility of the output results. The image recognition part is mainly responsible for color recognition and target positioning tasks. Taking the signal light as an example, the system first extracts significant color areas such as red and green from the binocular image, and combines the spatial position to determine whether it is in the valid recognition area. To prevent false triggering, the system introduces template comparison logic for the recognition results; Template conditions include the height range of traffic lights, the spatial position offset range, and target shape comparison. If a target in the image is correct in color but abnormal in position (such as deviating from the track centerline or the height is below the set threshold), the system will judge the result as low confidence or invalid recognition and will not use it. LiDAR data mainly provides geometric structure support. For traffic lights, the system will try to find vertical poles or light box structures in the point cloud corresponding to their identified positions as a basis for geometric verification. By calculating the point cloud elevation mutation and structural contours, the confidence level of traffic light judgment can be further enhanced; In addition, in multi-target recognition tasks, the system supports a confidence weighting strategy, which assigns weights to the confidence of the image source (such as the output value of the neural network) and the similarity of the geometric features of the point cloud, and calculates the fusion confidence score through weighted calculation. If the fusion confidence is between the interval [0.4, 0.6], it can be marked as "pending confirmation" status, and the system retains multiple frames of historical judgment to assist in confirmation. By adding an anti-false touch mechanism, when the confidence is lower than the minimum threshold (such as <0.4), the system assumes that the target does not exist and enters the empty recognition state. This setting can effectively suppress false positive problems caused by strong light reflection, shadow ghosts or image edge interference. The recognition output structure uniformly includes: target type, spatial position, color information, confidence, recognition status mark and other fields, providing a clear and explainable input basis for the subsequent passage decision module; Through the image and point cloud fusion strategy, the system achieves stable detection of key elements such as traffic lights, reduces traffic misjudgments caused by false recognition triggers, and improves overall operational safety and fault tolerance.

[0038] Please see the attached Figure 1 -Attached Figure 3 and attached Figure 5 A train trajectory autonomous verification method based on spatiotemporal feature fusion includes the following steps: Step 1: Collect point cloud, image, and GNSS positioning data of the environment in front of the train; Step 2: Enhance the image, filter and crop the point cloud to generate a depth map; Step 3: Identify signal lights, switch connection status, and obstacles, and extract their 3D semantic features; Step 4: Fit the orbit centerline and combine it with the GNSS trajectory to determine whether there is trajectory deviation; Step 5: Make a traffic safety assessment based on the target recognition results and trajectory judgment; Step 6: Send the track verification status to the train control system through the CAN communication module; Step 7: Monitor the system operation status in real time and output a system fault signal when identification fails or data is abnormal.

[0039] Specifically, the system first performs sensory data collection. LiDAR is used to construct the three-dimensional spatial structure ahead, while the binocular camera provides image information and depth information. Meanwhile, the GNSS module outputs real-time positioning data. This data is synchronized and processed with a unified timestamp to ensure spatial and temporal consistency. The system then enters the data preprocessing process. Image data is enhanced through histogram equalization and bilateral filtering to reduce image blur and noise interference in complex environments. Simultaneously, depth information is calculated based on the binocular disparity map, and a forward spatial dot matrix is established. The laser point cloud is downsampled through voxel filtering, and the forward region of interest is spatially cropped, retaining valid points in the track area. After entering the recognition phase, the system combines image and point cloud data to perform multi-target recognition. Signal light recognition relies on color judgment and spatial position templates, combined with laser data to verify its geometric characteristics. The switch structure is confirmed by the track shape in the image and the connection direction of the point cloud. Obstacle recognition focuses on whether there are irregular reflective areas above the track surface within 3 meters around the centerline, and outputs their 3D coordinates and occupied range. After target extraction, the system constructs the track centerline and fits the running trajectory. This process forms the centerline based on the left and right trackside data extracted by laser and compares it with the GNSS real-time positioning point. If the actual train track shows characteristics such as lateral deviation, sudden change in heading angle, or abnormal continuous displacement, it is determined to be a track deviation. Once the trajectory status is confirmed, the system will combine the recognition results to determine the safety of passage. If the recognized signal light is in the prohibited state, the direction of the turnout is inconsistent, or the distance between the obstacle and the center line is lower than the set threshold (such as 1.2 meters), the system will mark it as "access restricted" or "stop", otherwise it will enter the "allowed passage" state. The judgment rules support confidence weighted judgment and multi-frame voting mechanism to enhance stability; The judgment results are sent to the train's main control system in real time via the CAN communication module. The message content includes track status, recognition results, obstacle distance, and system operation flags, and is continuously updated at a frequency of 10Hz to ensure the continuity and real-time response of the command. At the same time, the system continuously monitors its own operating status. Module-level alarms are triggered when images are blurred, point clouds are obscured, or recognition result confidence is too low. If any subsystem experiences five consecutive abnormal frames, or if multiple recognition results fail simultaneously, a system-level fault signal is generated and output synchronously for the train control system to implement safety response measures. Finally, through the integrated closed-loop processing chain of perception-recognition-modeling-judgment-communication-monitoring, the train's trajectory verification and autonomous operation capabilities in complex environments are effectively enhanced.

[0040] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. The autonomous train track verification system based on spatiotemporal feature fusion is characterized by: include: Multi-source sensing and acquisition module, used to obtain spatiotemporal information of the track environment in front of and behind the train; The perception data preprocessing module is used to filter, crop regions, and extract boundaries of 3D point clouds of spatiotemporal information, and perform histogram enhancement and depth map construction on image data; The target recognition and feature extraction module is used to identify the status of traffic lights, switch connections, and obstacle locations based on spatiotemporal images and depth maps, and extract their 3D semantic features based on point cloud geometry information. The trajectory modeling and deviation detection module is used to fit the current running trajectory based on the track edge points and GNSS trajectory, and determine whether the train deviates from the designed track; The passage decision and trajectory verification module is used to determine whether the train meets the passage conditions based on the target recognition results and trajectory modeling results, and output the instructions of allowing passage, restricting passage or stopping; The communication and command issuing module is used to send the trajectory judgment results and system status to the train control system through the CAN communication protocol; The system self-check and fault monitoring module is used to detect the operating status of each module in the system and output the system fault status when the data is abnormal or the recognition fails.

2. The train trajectory autonomous verification system based on spatiotemporal feature fusion according to claim 1 is characterized by: The spatiotemporal information includes three-dimensional lidar point cloud images in front of and behind the vehicle, image data collected by binocular cameras, and GNSS positioning data; After acquisition, the 3D lidar point cloud is downsampled through voxel filtering, and the point cloud within 60 meters in front of the track is retained using a region cropping method. The left and right track edge lines and center lines are extracted based on the RANSAC algorithm, and curvature analysis is achieved in combination with frame sliding trajectory estimation.

3. The train trajectory autonomous verification system based on spatiotemporal feature fusion according to claim 1 is characterized by: The perception data preprocessing module performs histogram equalization and bilateral filtering enhancement on the image data, and generates a depth map by combining the binocular image pair. The image and depth map are input into the target detection network improved by the YOLO architecture to identify traffic lights, switch structures and obstacles.

4. The train trajectory autonomous verification system based on spatiotemporal feature fusion according to claim 1 is characterized by: The trajectory modeling and deviation detection module constructs the track centerline and compares the real-time GNSS trajectory with the centerline. It determines whether there is a trajectory deviation by calculating the trajectory offset and the heading angle, and outputs a deviation alarm when it detects that the track has been entered into the switch area or the trajectory has changed suddenly.

5. The train trajectory autonomous verification system based on spatiotemporal feature fusion according to claim 1 is characterized by: The traffic decision and trajectory verification module makes decisions based on the status of the signal light, the status of the turnout, the spatial position of the obstacle and the matching degree of the trajectory, wherein the obstacle When the shortest distance to the track centerline is lower than the set threshold, it is deemed impassable. When the signal light and switch status are inconsistent, confirmation is made through a multi-frame voting mechanism.

6. The train trajectory autonomous verification system based on spatiotemporal feature fusion according to claim 1 is characterized by: The communication and instruction issuing module sends the judgment result in the form of a structured message via CAN communication; The message includes obstacleStatus, lightStatus, turnoutStatus, obstacleDistance and systemStatus fields, and supports real-time refresh frequency.

7. The train trajectory autonomous verification system based on spatiotemporal feature fusion according to claim 1 is characterized by: The system self-check and fault monitoring module includes three parts: image perception anomaly detection, laser occlusion recognition and recognition confidence assessment. When any module fails to recognize, has missing data or is severely occluded for more than five consecutive frames, a system fault flag will be output and synchronized to the train main control system.

8. The train trajectory autonomous verification system based on spatiotemporal feature fusion according to claim 1 is characterized by: The multi-source perception acquisition module includes a laser radar and binocular camera combination located at the front end of the train; The laser radar has a horizontal viewing angle of 120 degrees and a vertical angle range of -10 to +10 degrees. The binocular camera's viewing angle covers a range of 2.5 meters to the left and right of the track center, which is used to realize environmental modeling and status recognition of the train's forward direction.

9. The train trajectory autonomous verification system based on spatiotemporal feature fusion according to claim 1 is characterized by: The target recognition and feature extraction module performs confidence fusion on the data from the image and lidar. Only when the traffic light recognition result meets the template conditions in both color recognition and spatial positioning and the recognition confidence exceeds the threshold, the result is passed to the passage decision module to avoid misrecognition and misjudgment.

10. A train track autonomous verification method based on spatiotemporal feature fusion, applied to a train track autonomous verification system based on spatiotemporal feature fusion according to any one of claims 1 to 9, characterized in that: The method comprises the following steps: Step 1: Collect point cloud, image, and GNSS positioning data of the environment in front of the train; Step 2: Enhance the image, filter and crop the point cloud to generate a depth map; Step 3: Identify signal lights, switch connection status, and obstacles, and extract their 3D semantic features; Step 4: Fit the orbit centerline and combine it with the GNSS trajectory to determine whether there is trajectory deviation; Step 5: Make a traffic safety assessment based on the target recognition results and trajectory judgment; Step 6: Send the track verification status to the train control system through the CAN communication module; Step 7: Monitor the system operation status in real time and output a system fault signal when identification fails or data is abnormal.

Images