Track obstacle real-time detecting and braking system based on multi-sensor redundancy

Abstract

The invention relates to the field of rail traffic intelligent control, and discloses a rail obstacle real-time detection and braking system based on multi-sensor redundancy, which comprises at least two groups of sensing modules arranged at the front end and the rear end of a train, and each group of sensing module comprises a laser radar used for collecting three-dimensional point cloud data of a forward or backward rail area of the train; the binocular camera is used for acquiring color image data and depth image data in corresponding directions; and the robot controller is in communication connection with the sensing module and is used for receiving sensing data acquired by the laser radar and the binocular camera and executing obstacle recognition, redundant information fusion and brake control logic. By constructing a multi-sensor redundancy sensing structure based on the laser radar and the binocular camera, high-robustness recognition of a dynamic or static obstacle target in a complex orbit environment is realized, and the effect of remarkably improving the obstacle detection accuracy and the false alarm suppression capability is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent control of rail transit, and specifically to a real-time detection and braking system for track obstacles based on multi-sensor redundancy. Background Art

[0002] In the context of the rapid development of railway train automation and intelligence, how to achieve high-precision perception of the forward track area, accurately identify potential obstacles, and respond in a timely manner has become a key issue that urgently needs to be solved in the train safety control system. Traditional trains usually rely on manual visual inspection or fixed monitoring systems to complete the monitoring of obstacles ahead. However, limited by the line of sight, reaction time, and subjective judgment differences of people, it is easy to miss detections or make misjudgments in complex environmental conditions, and it cannot meet the strict requirements for safety and real-time performance of high-speed train operation.

[0003] In recent years, lidar and image vision technologies have gradually been introduced into the field of rail transit to improve the forward perception ability. Lidar has the advantages of being resistant to light interference and providing three-dimensional spatial information, while image recognition can assist in identifying structural details and semantic information. However, the existing obstacle detection systems based on a single sensor still have significant deficiencies in practicality. The point cloud quality of lidar systems is easily affected in high humidity, rain, and snow weather. At the same time, the point cloud processing flow of some systems is rough, and the ROI area and outliers are not effectively filtered, resulting in insufficient recognition accuracy or waste of computing resources; image recognition systems have poor stability under backlight, shadow, or occlusion conditions and are difficult to continuously output accurate recognition results. In addition, most of the two perception methods are in an independent operating state, lacking an information fusion mechanism, unable to form redundancy and complementarity at critical moments, and the system fault tolerance ability is insufficient.

[0004] In terms of track geometric structure recognition, existing solutions often only focus on trajectory point tracking or two-dimensional image line detection, and it is difficult to model the three-dimensional structural characteristics of the actual track. Especially in turnout areas or curved track scenarios, it is difficult to accurately judge the track extension direction and branch situation, thus affecting the judgment of the relationship between obstacles and the track. For dynamic obstacles, such as pedestrians breaking in or animals crossing, existing systems generally adopt a fixed-distance judgment method and do not introduce dynamic modeling mechanisms such as velocity vectors and predicted paths, making it difficult to give early warnings. In addition, environmental factors such as rain, snow, and dust occlusion will cause problems such as abnormal laser echoes and blurred images. Most existing systems do not design corresponding anomaly detection and self-check mechanisms, and the system stability and robustness are insufficient. Summary of the Invention

[0005] Aiming at the deficiencies of the existing technology, the present invention provides a real-time detection and braking system for track obstacles based on multi-sensor redundancy, which solves the problems of low recognition accuracy and single perception method in the existing technology in complex environments.

[0006] To achieve the above object, the present invention is realized through the following technical solutions: A real-time detection and braking system for track obstacles based on multi-sensor redundancy, comprising: at least two sets of sensing modules arranged at the front and rear ends of the train, each set of the sensing modules comprising: A lidar, configured to collect three-dimensional point cloud data of the track area in front of or behind the train; A binocular camera, configured to acquire color image data and depth image data in the corresponding direction; A robot controller, communicatively connected to the sensing modules, configured to receive the sensing data collected by the lidar and the binocular camera, and execute obstacle recognition, redundant information fusion and braking control logic; A locomotive message processing module, communicatively connected to the robot controller, configured to generate a control signal in a standard communication frame format according to the braking control information; A CAN communication interface module, configured to send the control signal to the train control system to achieve deceleration or braking control of the train.

[0007] Preferably, the robot controller comprises: A point cloud processing module, configured to perform filtering, downsampling, obstacle clustering and track area fitting based on the point cloud data of the lidar; An image processing module, configured to perform obstacle recognition, turnout detection and signal lamp recognition on the images collected by the binocular camera; A deep neural network object detection module, configured to perform feature extraction and inference recognition on the images; A decision-making module, configured to fuse the recognition results from different sensing modules, determine whether there are obstacles, and generate braking control information according to the detection results of the obstacles; A self-checking module, configured to determine whether there are image blurring, sparse point clouds or device abnormalities based on the states of the sensing modules, and generate fault status information when detecting abnormalities.

[0008] Preferably, the point cloud processing module is configured to perform: Region filtering to extract the point cloud of the forward track area; Using the Euclidean clustering algorithm to identify obstacle targets in the point clusters; For each obstacle point cluster, calculate its external geometry and determine whether it enters the predicted trajectory of the train; Output the obstacle presence status and the nearest distance information when the obstacle meets the collision condition.

[0009] Preferably, the image processing module is configured to: Extract the RGB image and the depth image from the image data collected by the binocular camera; Based on a preset detection area, only extract the obstacle image area within the track range; Output structured image recognition results containing the position and type of obstacles.

[0010] Preferably, the decision-making module is configured as follows: If the lidar and binocular camera both output obstacle presence information for multiple consecutive detection frames, generate a braking signal; If only one side of the perception module outputs obstacle information, perform frame number accumulation verification and output a deceleration signal when the results of consecutive frames are consistent; If there is point cloud loss or image blur, call the self-check module to generate system status exception information.

[0011] Preferably, the robot controller is communicatively connected to each perception module via Ethernet, and the locomotive message processing module is communicatively connected to the train control system via CAN to Ethernet.

[0012] Preferably, the control signal in the CAN communication interface module is in the CAN frame format with the following fields: Obstacle status field, used to indicate whether there is an obstacle; Turnout status field, used to indicate whether the turnout is in the target route position; Signal light status field, used to indicate whether the signal light allows passage; Obstacle distance field, used to indicate the minimum distance between the obstacle and the front end of the train; System status field, used to indicate whether there is a perception device failure or recognition anomaly.

[0013] Preferably, the depth ranging range of the binocular camera is from 1.5 meters to 50 meters, the point cloud acquisition frequency of the lidar is 10 frames per second, the horizontal field of view angle is 120 degrees, and the vertical field of view angle is 25 degrees.

[0014] Preferably, the self-check module is configured as follows: Based on the data integrity, density, and recognition confidence of consecutive frames of sensors, determine whether the perception module is in an abnormal state; Output a system operation failure flag in the abnormal state and prohibit the automatic issuance of braking instructions.

[0015] A real-time detection and braking method for track obstacles based on multi-sensor redundancy includes the following steps: The lidar collects three-dimensional point cloud data of the forward track area, and the point cloud processing module performs region filtering, downsampling, and obstacle clustering recognition; The binocular camera collects images and extracts depth maps, and the image processing module and the deep neural network target detection module perform obstacle detection; The decision-making module fuses the detection results of the lidar and the binocular camera, determines whether there is an obstacle, and generates corresponding braking control information according to its type and position; The self-check module analyzes the state of the sensing module and outputs system abnormal state information when detecting blurred images or missing point clouds; The locomotive message processing module generates a standard CAN communication frame according to the control information; The CAN communication frame is sent to the train control system through the communication interface to achieve deceleration or braking control of the train.

[0016] The present invention provides a real-time detection and braking system for track obstacles based on multi-sensor redundancy. It has the following beneficial effects: 1. By constructing a multi-sensor redundancy perception structure based on lidar and binocular cameras and introducing a multi-modal data fusion algorithm for obstacle recognition and track information discrimination, the present invention realizes highly robust recognition of dynamic or static obstacle targets in complex track environments, and achieves the effects of significantly improving the obstacle detection accuracy and false alarm suppression ability.

[0017] 2. By introducing a self-check module to monitor and analyze the operating state of the sensing devices in real time and realizing closed-loop linkage with the braking control logic, the present invention realizes intelligent degradation and fault tolerance control in the case of blurred images, missing point clouds or decreased module recognition confidence, and achieves the effects of improving the safety reliability and fault tolerance response ability of the train braking system under abnormal working conditions, and effectively ensuring the continuous availability of the system in multiple scenarios.

[0018] 3. By setting up a locomotive message processing module and realizing state linkage with the train control system based on the CAN communication interface, the present invention realizes standardized coding and real-time transmission of obstacle recognition information, system state signals and braking instructions, and achieves the effects of significantly enhancing the system engineering integrability, cross-model compatibility and on-site deployment efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is a schematic diagram of the system architecture of the present invention; Figure 2 is a schematic diagram of the method steps of the present invention; Figure 3 is a schematic diagram of the binocular camera data reading process of the present invention; Figure 4 is a schematic diagram of the decision-making module judgment process of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0020] Next, in combination with the accompanying drawings of the present invention, the technical solutions of the present invention will be clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without making creative efforts shall fall within the scope of protection of the present invention.

[0021] Please refer to the attached Figure 1 and the attached Figure 3 and the attached Figure 4 , the embodiment of the present invention provides a real-time detection and braking system for track obstacles based on multi-sensor redundancy, including: at least two groups of sensing modules arranged at the front and rear ends of the train, and each group of sensing modules includes: A lidar for collecting three-dimensional point cloud data of the track area in front of or behind the train; a binocular camera for obtaining color image data and depth image data in the corresponding direction. The depth ranging range of the binocular camera is from 1.5 meters to 50 meters, the point cloud acquisition frequency of the lidar is 10 frames per second, the horizontal field of view angle is 120 degrees, and the vertical field of view angle is 25 degrees.

[0022] Specifically, in the present invention, the lidar is used to collect three-dimensional point cloud data of the track area in front of or behind the train, as the core spatial sensing component of the system. The lidar emits laser beams in a rotating scan mode, measures the precise distance from the target point to the lidar by receiving the reflected signals, and simultaneously records the reflection intensity and angular position. The data acquisition frequency of 10 frames per second ensures that even when the train is running at a medium speed, the point cloud update of the track environment is still timely enough. This frequency is a relatively balanced choice in engineering, taking into account both the real-time perception and the processing burden at the backend.

[0023] The horizontal field of view angle of the lidar is set to 120 degrees, which can basically cover the edges of both sides of the track and meet the detection requirements of single-line segments, curves, and turnout crossings. The vertical field of view angle is 25 degrees to detect low obstacles on the track surface within a large longitudinal range, such as falling rocks, ballast accumulations, and even debris bags. If the vertical angle is set too small, some obstacles may be missed, and if it is too large, a large amount of invalid data will be generated. Therefore, this 25-degree angle is actually determined after repeated simulations of the scenario.

[0024] In terms of installation, the lidar is generally installed at the top of the front end of the train, slightly off the central axis, with a certain downward inclination, so that it can just "scan" the front track area with a certain depth. In subsequent versions, if the train runs in both directions, a set will also be installed at the tail to achieve reverse detection.

[0025] The binocular camera is specifically used to obtain color image data and depth image data in the same direction as the lidar. There is a fixed distance between the two cameras, called the baseline. By calculating the disparity between the left and right images, the depth position of each pixel point in the scene can be deduced.

[0026] The effective range of the binocular camera used in the system is 1.5 to 50 meters. Distances below 1.5 meters will be inaccurate due to large parallax, and distances above 50 meters will not be sufficient to support effective parallax calculations. This range just covers the critical braking area in front of the train, which is also the core judgment window for obstacle identification.

[0027] In addition to obtaining depth, binocular images can also provide features such as object color, shape, and texture, which are critical for distinguishing semantic targets such as traffic lights, people, and animals. Each frame of the camera's image data is marked synchronously with the laser radar acquisition moment to ensure that there will be no spatial dislocation during subsequent multi-source fusion. Ultimately, through this combination, the system not only achieves complete modeling of the spatial contour of the area in front of the track, but also has the ability to recognize object type, color, and semantics, improving recognition accuracy and environmental adaptability.

[0028] The robot controller is connected to the perception module to receive the perception data collected by the laser radar and the binocular camera, and perform obstacle recognition, redundant information fusion and braking control logic, including: Point cloud processing module, used to perform filtering, downsampling, obstacle clustering and track area fitting based on LiDAR point cloud data; configuration execution: Regional filtering is used to extract the point cloud of the forward track area; the Euclidean clustering algorithm is used to identify obstacle targets in the point cluster; for each obstacle point cluster, its surface geometry is calculated and whether it enters the expected train trajectory is determined; when the obstacle meets the collision condition, the obstacle existence status and the closest distance information are output.

[0029] The image processing module is used to perform obstacle recognition, turnout detection and signal light recognition on the images collected by the binocular camera; the image processing module is configured to: extract RGB images and depth images from the image data collected by the binocular camera; based on the preset detection area, only extract the obstacle image area within the track range; output the structured image recognition result including the obstacle position and type.

[0030] Deep neural network target detection module, used for feature extraction and inference recognition of images; The decision module is used to integrate the recognition results from different perception modules, determine whether there is an obstacle, and generate braking control information based on the obstacle detection results; the configuration is: If the laser radar and the binocular camera output obstacle existence information for multiple consecutive detection frames, a braking signal is generated; If only one side of the perception module outputs obstacle information, the frame number accumulation verification is performed, and a deceleration signal is output when the results of consecutive frames are consistent; If there is point cloud loss or image blur, the self-check module is called to generate system status abnormality information.

[0031] The self-check module is used to determine whether there is image blur, sparse point cloud or device abnormality based on the status of each perception module, and generate fault status information when an abnormality is detected. The configuration is: Based on the data integrity, density and recognition confidence of the sensor's continuous frames, determine whether the perception module is in an abnormal state; In an abnormal state, the system operation fault flag is output and the automatic issuance of the braking command is prohibited.

[0032] Specifically, the robot controller communicates with the LivoxHAP laser radar installed at the front of the train to receive 10 frames of 3D point cloud data per second in real time. After receiving each frame of data, the controller first performs non-empty detection to remove abnormal frames. Subsequently, the point cloud data enters the preprocessing process, which includes three processing steps: first, the region of interest (ROI) is set using a pass-through filter to only retain point clouds within 15 meters in front (corresponding to the safe braking distance of the train) and 4.5 meters in the vertical direction (the maximum height range of the train); second, a statistical filter is used to remove outliers. This method is based on the average distance distribution calculation from the point to the neighboring point, assuming a Gaussian distribution, and points that fall outside the standard deviation range are considered noise and removed; third, downsampling is performed through voxel filtering. The system records the original number of points before filtering, and sets a 0.1m voxel grid. The number of points after compression is about 1 / 5 of the original points, which significantly reduces the subsequent calculation pressure.

[0033] The preprocessed point cloud data is sent to the RANSAC plane segmentation module to extract points in the track ground plane and generate candidate track areas. Track geometric feature recognition is divided into two categories: straight track and curved track. For the close-distance track area, it is assumed to be equidistant parallel lines. After fitting multiple straight line segments through RANSAC, the system calculates the angle θ and the average spacing t between each pair of straight lines, and compares them with the standard track gauge T of the train. When the angle θ is less than the set threshold T1 and |t–T|≤T2, it is judged as a valid track; for long-distance segments, B-spline or quadratic curve models are used to approximate the track direction. After the track detection is completed, the turnout is further identified in combination with the intersection characteristics of the track line and the change in the turning angle. The system outputs the recognition results in a structured manner: the status of the main track, whether the secondary track is aligned, the coordinates of the turnout center point, and the deviation angle.

[0034] Based on the completion of track recognition, the system sets up an obstacle detection ROI area above the track area. At the same time, considering the problem of increased point cloud density caused by rain and snow weather, the rain and snow filtering logic (such as density statistics and point-plane roughness judgment) is enabled to prevent false clustering. Euclidean clustering algorithm is used for obstacle recognition, with the clustering radius and minimum point number threshold set, and the three-dimensional bounding box, center coordinates, size and ID number of each obstacle are output. The controller further constructs a collision prediction area according to the track curvature information, extends the track center line by a certain distance to generate a safety buffer zone, and judges whether the obstacle is located within the collision area. For dynamic obstacles, the system estimates their moving speed through multi-frame position changes, sets a speed threshold v and calculates the time required for them to enter the track, and judges whether deceleration or emergency braking is required according to the corresponding train running distance.

[0035] In the obstacle clustering results, the system synchronously performs signal light point cloud detection, combines geometric contour features and preset models for rough classification, and assists the image recognition results to complete the confirmation of the signal light state (such as the position of the red / green light pole, whether it is blocked, etc.). All the above recognition results - including turnout status, obstacle status, signal light status and their position information - are packed and transmitted by the controller to the decision-making module. The decision-making module fuses multi-modal information, comprehensively judges the obstacle type, track position, signal state and dynamic risk, and generates braking level control information: deceleration, braking, ignoring or prompting.

[0036] To ensure the stable operation of the system, the self-check module continuously receives the status feedback data of the lidar and binocular camera. In the self-check algorithm, the lidar anomaly criteria include that the number of point clouds between frames drops by more than 50%, the average echo intensity is lower than the set lower limit, and the point cloud density in the ROI area is lower than the threshold. For image anomalies, it is judged that the Laplacian sharpness of the image is lower than the set value, or the detection confidence is continuously low in multiple cycles. If any abnormal state lasts for more than 3 frames, the self-check module will generate a fault flag bit, write it into the system operation status field, and notify the decision-making module to block the braking signal, and the system will enter the "safe fault-tolerant mode" to avoid false braking triggers caused by misrecognition.

[0037] The robot controller converts all recognition, judgment and decision results into structured communication data, and sends them to the train control system via the Ethernet port or CAN forwarding interface. The CAN message contains fields such as obstacle type code, response level, remaining distance, recognition source (point cloud / image / fusion), etc., ensuring that each braking signal received by the control system can be traced and analyzed, and is applicable to various scenarios such as train protection, crossing control and remote inspection.

[0038] The locomotive message processing module, which is communicatively connected to the robot controller, is used to generate control signals in the standard communication frame format according to the braking control information; Specifically, in the present invention, the locomotive message processing module serves as an interface bridge between the robot controller and the train control system. Its main task is to convert the braking control information, system status, obstacle recognition results, etc. from the controller into control signals in the standard communication frame format and transmit them to the on-vehicle execution system via the CAN bus. This module communicates and docks with the robot controller through Gigabit Ethernet, using a data frame protocol based on TCP or UDP, and receives the output structure data of the decision-making module every 50 ms. The structure contains the obstacle presence status, obstacle type number, distance between the obstacle and the locomotive head, signal light recognition status, turnout direction judgment result, and system self-check status code.

[0039] The message processing module disassembles fields, performs legality verification, and packs protocols on the received data. The system uses a unified CAN frame format, and the message IDs are grouped by function. The typical settings are as follows: 0x180 represents obstacle status information, 0x181 represents the details of the recognition target, and 0x1F0 is used for system status reports. Each CAN frame is limited to 8 bytes, and the field allocation is strictly set according to the bit width and function mapping. For example, in the obstacle status frame, the first byte represents the obstacle presence flag (0x00 no obstacle, 0x01 confirmed presence), the second byte represents the obstacle type code (such as 0x01 for personnel, 0x02 for equipment, 0x03 for animals, etc.), the third to fourth bytes are the obstacle distance (16-bit integer, unit: 0.1 meters), and the fifth byte is the recognition path source flag (0x01 for lidar, 0x02 for binocular images, 0x03 for fusion decision). The remaining bytes are reserved for expanding custom fields or timestamps.

[0040] In addition to the basic recognition results, this module also synchronously outputs system operation status information, such as "image blur detection", "point cloud sparsity warning", "control instruction masking" status, through the bit identification coding in the status field. For example, the first bit represents an abnormality at the image end, and setting it indicates that the current image recognition link fails; the second bit represents an abnormality at the radar end, and the third bit represents system fault locking. The status frame is broadcast periodically for the train control system to monitor the health of the intelligent recognition link in real time.

[0041] The entire message processing process adopts an interrupt-triggered frame assembly logic to ensure that once the robot controller updates the output data, this module can respond to frame assembly and distribution within the shortest delay. At the same time, a set of transmission buffer queues and CRC check mechanisms are provided inside the module to handle possible frame delays, redundancy cleaning, or link instability problems during communication. In engineering deployment, this module is usually integrated into the EN50155 industrial communication node, supports the standard CAN2.0B protocol, and the baud rate is default set to 500 kbps, which can be adjusted according to the vehicle type.

[0042] Through the settings of this module, the system realizes the data closed-loop docking from intelligent recognition to the vehicle-mounted system and the transmission of safety control signals.

[0043] The CAN communication interface module is used to send control signals to the train control system to achieve the deceleration or braking control of the train. The control signal is in the CAN frame format with the following fields: Obstacle status field, used to indicate whether there is an obstacle; Switch status field, used to indicate whether the switch is in the target route position; Signal light status field, used to indicate whether the signal light allows passage; Obstacle distance field, used to indicate the minimum distance between the obstacle and the front end of the train; System status field, used to indicate whether there is a failure of the sensing device or an abnormal recognition.

[0044] Specifically, in the present invention, the CAN communication interface module is arranged between the robot controller and the train control system, serving as a data transmission bridge between the two. Its main function is to pack the braking control signal output by the robot controller into a CAN communication frame according to the standard format and send it to the automatic driving or train control unit of the train via the physical bus, so as to achieve the automatic deceleration or braking control of the train. This module is built with a standard CAN2.0B protocol stack, supports a transmission rate of 500 kbps, and has communication functions such as frame buffer scheduling, CRC check, fault fallback, and link redundancy, ensuring the high-reliability transmission of control information in a complex rail transit environment.

[0045] The control signal is transmitted using a unified format of CAN data frame. Each control frame contains an 8-byte data segment, and the main field structure is as follows: Obstacle status field (1 byte): used to indicate whether there is a detectable obstacle currently. 0x00 indicates no obstacle, 0x01 indicates that an obstacle is confirmed to exist, and 0x02 indicates that the fusion path judgment is inconsistent, and early warning is required but braking is not immediate.

[0046] Switch status field (1 byte): output by the image recognition module, indicating whether the current switch is in the target running direction. 0x00 indicates that the switch is not aligned, 0x01 indicates that it is aligned with the target track, and 0xFF indicates that it is not recognized.

[0047] Signal light status field (1 byte): used to indicate the status of the signal light in front of the train. 0x00 is a red light prohibiting passage, 0x01 is a green light allowing passage, 0x02 is a yellow light warning, and 0xFF indicates that the signal recognition fails or is invisible.

[0048] Obstacle Distance Field (2 bytes, 16 bits): Represents the minimum forward distance of an obstacle in units of 0.1 meters using an unsigned integer. For example, 0x01F4 indicates that the forward obstacle is 20.4 meters away from the train head. If there is no obstacle, this field is filled with 0xFFFF.

[0049] System Status Field (1 byte): Used to identify the current perception link status. The definitions of each bit are as follows: bit0 for image recognition failure, bit1 for lidar anomaly, bit2 for fusion failure, bit3 for frame synchronization interruption, and bit7 for system serious fault lock.

[0050] Reserved Field (2 bytes): Reserved for future expansion such as the integration of external data like GPS position, speed information, ambient light, etc. Currently, it is fixed to 0x0000.

[0051] The CAN communication interface module uses a dual mechanism of polling + interruption to schedule frame sending tasks. Each time the robot controller generates a new braking judgment result, an interruption will be triggered to write to the sending FIFO buffer queue of this module. The highest sending frequency is set to one frame per 50 ms inside the module. If the same type of message appears repeatedly within the scheduling period, the latest data will be automatically merged and redundant data will be eliminated to prevent bus congestion. Each frame automatically attaches a CRC check code and records a timestamp before sending, which is used for communication link log recording and later traceability.

[0052] At the same time, to ensure high-reliability transmission, the module supports a redundant structure of dual CAN channels. The main channel is connected to the CAN main line of the train control system, and the backup channel is connected to the auxiliary communication link; when the main channel communication is abnormal or the number of bus arbitration failures exceeds the set threshold, the module will switch to the backup channel for sending to ensure that the braking information is delivered in real time. The module automatically records the fault flag in case of abnormal frame reception, abnormal bus voltage, etc., and feeds back the status to the train control system through the system status field.

[0053] This CAN communication interface module can be integrated into an embedded control platform. Its physical form is an industrial controller unit installed on a DIN rail, with the capabilities of shock resistance, dust prevention, and wide temperature range (-40°C to +70°C), meeting the communication safety requirements of rail transit vehicles. In system deployment, only by connecting to the train CAN network through a standard DB9 or M12 interface, the intelligent obstacle recognition and active braking control functions can be realized for plug-and-play.

[0054] Please refer to Appendix Figure 2 , A real-time detection and braking method for track obstacles based on multi-sensor redundancy, including the following steps: The lidar collects three-dimensional point cloud data of the forward track area, and the point cloud processing module performs area filtering, downsampling, and obstacle clustering recognition; The binocular camera captures images and extracts depth maps, and the image processing module and the deep neural network object detection module perform obstacle detection; The decision-making module fuses the detection results of the lidar and the binocular camera, judges whether there are obstacles, and generates corresponding braking control information according to their types and positions; The self-checking module analyzes the state of the perception module and outputs system abnormal state information when detecting blurred images or missing point clouds; The locomotive message processing module generates a standard CAN communication frame according to the control information; The CAN communication frame is sent to the train control system through the communication interface to realize the deceleration or braking control of the train.

[0055] Specifically, in the intelligent detection and control system for track obstacles of the present invention, the robot controller serves as the core intelligent unit, collaborating with the lidar, the binocular camera, the self-checking unit, and the train communication interface to complete the full-process closed-loop control from obstacle recognition to braking control. When the system is running, the lidar is fixedly installed at the front end of the train to collect three-dimensional point cloud data within a range of 30 meters forward in real time. This data is transmitted to the point cloud processing module at a frequency of 10 frames per second. First, it undergoes ROI region filtering, restricting the processing range to the area within ±1 meter of the track center and 5 to 25 meters in depth. Subsequently, the VoxelGrid voxel downsampling operation is performed to compress the original million-level point cloud to less than 50,000 points while maintaining the dense structure, significantly reducing the computational load. The downsampled point cloud data is input into the Euclidean clustering module, where the adjacent radius and the minimum point number threshold are set to identify independent obstacle clusters, and it is judged whether they pose a potential threat based on their positions, shapes, and the coincidence degree with the front of the vehicle trajectory. Finally, parameters such as obstacle flags, bounding box coordinates, obstacle IDs, and the minimum forward distance are output.

[0056] At the same time, the binocular camera synchronously captures color image pairs and generates a high-precision depth map through calibration parameter calculation. The image processing module extracts the track region (ROI) in the image and sends it to the deep neural network object detection module for forward inference. The lightweight YOLOv5-tiny structure is used to quickly identify typical obstacle types (such as personnel, equipment, vehicle wrecks, etc.), and spatial mapping is performed in combination with the depth map to obtain the three-dimensional coordinates, classification numbers, and confidence levels of the obstacle targets. If there is a blurred or insufficient illumination phenomenon at the image end, the image processing module will trigger the clarity analysis algorithm to judge the recognition reliability through the Laplacian variance and edge information.

[0057] The above two-way perception information finally flows into the decision-making module inside the controller. The decision-making module polls and fuses the recognition results of the current frame of lidar and binocular images at a cycle of 50 ms. If there is consistency in position and category between the two (the obstacle position error is less than 0.5 meters and the category codes match), it is determined as a valid obstacle, and braking control information is immediately generated; if only one side of the perception module recognizes an obstacle, the time integration accumulation method is used to calculate the confidence value integration, and a warning or deceleration command is generated after exceeding the set threshold. For special category obstacles (such as misaligned turnouts, red signal lights, etc.), the decision-making module sets response strategies of different levels according to predefined rules to ensure that the control response is targeted and hierarchical when the obstacle type and position change.

[0058] To ensure the reliability of the system, a self-check module is built into the controller to continuously monitor the operating status of the perception link. The self-check module determines whether there are device failures or abnormal recognitions by analyzing indicators such as image clarity, lidar point cloud density, and timestamp synchronization delay. If phenomena such as blurred images, sparse point clouds, and continuous loss of recognition results occur, the self-check module will output a system status abnormal flag within 3 frames, block the direct issuance of control signals, and package and upload the status bits to the train control system for entering the manual takeover or safety degradation mode.

[0059] The braking control information generated by the robot controller is further encapsulated into a standard CAN communication frame through the locomotive message processing module. The frame structure includes an obstacle status field, an obstacle distance field, a turnout status field, a signal light status field, and a system status field. Each field is defined in byte order to ensure clear semantics within the frame and compatibility with the train CAN receiving end. The message processing module internally has a frame buffer, a scheduling queue, and a CRC check mechanism to ensure the stability and integrity of the frame under high-frequency transmission.

[0060] Finally, the CAN communication frame is sent to the train control system through the integrated CAN communication interface module to achieve automatic deceleration or emergency braking control of the train. When an obstacle is confirmed to exist and the distance is less than the set safety threshold (such as 10 meters), the system immediately outputs a forced braking command; if there is only an abnormal recognition or warning status, a deceleration or manual confirmation suggestion is output.

[0061] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A real-time detection and braking system for track obstacles based on multi-sensor redundancy, characterized in that, Comprising: At least two sets of sensing modules arranged at the front and rear ends of the train, each set of the sensing modules comprising: A lidar for collecting three-dimensional point cloud data of the track area in front of or behind the train; A binocular camera for obtaining color image data and depth image data in the corresponding direction; A robot controller communicatively connected to the sensing modules, configured to receive the sensing data collected by the lidar and the binocular camera, and execute obstacle recognition, redundant information fusion and braking control logic; A locomotive message processing module communicatively connected to the robot controller, configured to generate a control signal in a standard communication frame format according to the braking control information; A CAN communication interface module for sending the control signal to the train control system to achieve deceleration or braking control of the train.

2. The real-time detection and braking system for track obstacles based on multi-sensor redundancy according to claim 1, characterized in that: The robot controller comprises: A point cloud processing module configured to perform filtering, downsampling, obstacle clustering and track area fitting based on the point cloud data of the lidar; An image processing module configured to perform obstacle recognition, turnout detection and signal lamp recognition on the images collected by the binocular camera; A deep neural network object detection module configured to perform feature extraction and inference recognition on the images; A decision module configured to fuse the recognition results from different sensing modules, determine whether there are obstacles, and generate braking control information according to the detection results of the obstacles; A self-check module configured to determine whether there are image blurring, sparse point clouds or device anomalies based on the states of the sensing modules, and generate a fault status information when an anomaly is detected.

3. The real-time detection and braking system for track obstacles based on multi-sensor redundancy according to claim 2, characterized in that: The point cloud processing module is configured to perform: Region filtering to extract the point cloud of the forward track area; Using the Euclidean clustering algorithm to identify obstacle targets in the point clusters; For each obstacle point cluster, calculate its external geometry and determine whether it enters the predicted trajectory of the train; Output the obstacle presence status and the nearest distance information when the obstacle meets the collision condition.

4. The real-time detection and braking system for track obstacles based on multi-sensor redundancy according to claim 2, characterized in that: The image processing module is configured to: Extract the RGB image and the depth image from the image data collected by the binocular camera; Based on a preset detection area, only extract the obstacle image area within the track range; Output a structured image recognition result including the position and type of the obstacle.

5. The real-time detection and braking system for track obstacles based on multi-sensor redundancy according to claim 2, characterized in that: The decision module is configured to: Generate a braking signal if the lidar and the binocular camera both output obstacle presence information in a continuous plurality of detection frames; If only one side of the sensing module outputs obstacle information, perform frame number accumulation verification, and output a deceleration signal when the results of consecutive frames are consistent; If there is point cloud loss or image blurring, call the self-check module to generate a system status anomaly information.

6. The real-time detection and braking system for track obstacles based on multi-sensor redundancy according to claim 1, characterized in that: The robot controller is communicatively connected to each sensing module through Ethernet, and the locomotive message processing module is communicatively connected to the train control system through CAN to Ethernet.

7. The real-time detection and braking system for track obstacles based on multi-sensor redundancy according to claim 1, characterized in that: The control signal in the CAN communication interface module is in a CAN frame format with the following fields: An obstacle status field for indicating whether there is an obstacle; A turnout status field for indicating whether the turnout is in the target route position; A signal lamp status field for indicating whether the signal lamp allows passage; An obstacle distance field for indicating the minimum distance between the obstacle and the front end of the train; A system status field for indicating whether there is a sensor failure or an identification anomaly.

8. The real-time detection and braking system for track obstacles based on multi-sensor redundancy according to claim 1, characterized in that: The depth measurement range of the binocular camera is from 1.5 meters to 50 meters. The point cloud acquisition frequency of the lidar is 10 frames per second, the horizontal field of view is 120 degrees, and the vertical field of view is 25 degrees.

9. The real-time detection and braking system for track obstacles based on multi-sensor redundancy according to claim 2, characterized in that: The self-check module is configured to: Based on the data integrity, density, and recognition confidence of consecutive frames of sensors, determine whether the perception module is in an abnormal state; Output a system operation failure flag in the abnormal state and prohibit the automatic issuance of braking instructions.

10. A real-time detection and braking method for track obstacles based on multi-sensor redundancy, according to the real-time detection and braking system for track obstacles based on multi-sensor redundancy described in any one of claims 1-9, characterized in that, Including the following steps: The lidar acquires three-dimensional point cloud data of the forward track area, and the point cloud processing module performs regional filtering, downsampling, and obstacle clustering recognition; The binocular camera acquires images and extracts depth maps, and the image processing module and the deep neural network object detection module perform obstacle detection; The decision-making module fuses the detection results of the lidar and the binocular camera, determines whether there are obstacles, and generates corresponding braking control information according to their types and positions; The self-check module analyzes the state of the perception module and outputs system abnormal state information when detecting blurred images or missing point clouds; The locomotive message processing module generates a standard CAN communication frame according to the control information; The CAN communication frame is sent to the train control system through the communication interface to achieve deceleration or braking control of the train.

Images