A rail transit train high-speed running environment detection method, device and system
By combining high-definition cameras and monocular depth estimation models with optical flow data, the problem of insufficient detection accuracy in the high-speed operation environment detection of rail transit trains has been solved, achieving accurate identification of obstacles and infrastructure anomalies, and improving detection efficiency and safety.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- ZHUZHOU CSR TIMES ELECTRIC CO LTD
- Filing Date
- 2024-11-28
- Publication Date
- 2026-05-29
AI Technical Summary
Existing technologies for detecting high-speed trains in rail transit environments suffer from problems such as motion blur and large inter-frame parallax, leading to insufficient detection accuracy. In particular, it is difficult to accurately identify obstacles and infrastructure anomalies when trains are moving at high speeds.
High-definition cameras are used to acquire visible light images and optical flow data. Combined with monocular depth estimation models and feature point matching, high-precision distance information and point clouds are generated. Anomalies in target status are identified through 3D map reconstruction and trend analysis.
It enables precise detection in high-speed environments, improves detection efficiency and accuracy, reduces false alarm and false alarm rates, provides intelligent early warning and maintenance plans, and enhances the safety and operational efficiency of rail transit.
Smart Images
Figure CN122116296A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of rail transit technology, specifically to a method, device, and system for detecting the high-speed operating environment of rail transit trains. Background Technology
[0002] With the continuous development of high-speed rail technology, high-speed rail has become an important means of transportation for people. The railway network is large in scale, covers different climate and environmental conditions, and has complex operating scenarios. As the train speed continues to increase, the braking distance is getting longer and longer. When foreign objects intrude into the railway clearance or the line infrastructure malfunctions, it will seriously threaten the operational safety of high-speed trains. Therefore, it is urgent to adopt intelligent technical means to conduct status detection and abnormal trend analysis of the train line in a non-contact manner, and guide the on-site investigation and handling in a timely manner to ensure train operation safety.
[0003] With the development of sensor technology, the installation of non-contact sensors (LiDAR, millimeter-wave radar, cameras, etc.) to collect real-time data ahead of the train and the use of computer vision technology for real-time target detection to ensure driving safety have been widely researched and maturely applied in the automotive field. In the rail transit sector, projects such as intelligent driving, intelligent maintenance, and safety assurance extensively utilize various types of image sensors. The rapidly developing computer vision technologies, such as target detection and 3D reconstruction, have also provided more solutions for non-contact detection-assisted driving safety. Existing intelligent driving perception systems use cameras and LiDAR to detect obstacles ahead of the train and determine whether an obstacle encroaches on the boundary through a series of logical judgments. This method achieves good results when the train is running at low speeds. However, when there is a relatively high-speed movement between the data acquisition device and the target to be inspected, problems such as motion blur, large inter-frame parallax, and information sparsity can lead to the inability to accurately identify and locate the target based on image and point cloud data. Similarly, in train overhead contact line inspection, this can result in the omission of abnormal information under high-speed movement. In the inspection of infrastructure such as tracks and tunnels, in order to avoid large inter-frame parallax that makes the inspection points sparse, the actual operating speed of the inspection vehicle usually needs to be kept below 60km / h, which also limits the improvement of inspection efficiency. Summary of the Invention
[0004] To address the technical problems existing in the prior art, the present invention provides a method, apparatus, and system for accurately detecting the high-speed operating environment of rail transit trains.
[0005] To solve the above-mentioned technical problems, the technical solution proposed by this invention is as follows:
[0006] A method for detecting the high-speed operating environment of rail transit trains, comprising the following steps:
[0007] Acquire visible light images, high-speed pulse vision and event information of the train's operating environment and surrounding infrastructure, and generate high-definition image frame data, optical flow data and corresponding timestamps;
[0008] The image frame data and optical flow data are aligned and fused according to timestamps, and then input into a pre-trained monocular depth estimation model to obtain depth estimation information. At the same time, feature points in image frames with adjacent time intervals are calculated based on the aligned and fused image frames and optical flow data, and the angular velocity of the target motion is estimated by feature point matching. In addition, standard reference objects and target information in the train operating environment are identified based on the image frames.
[0009] By combining monocular depth estimation, angular velocity, and the size information of the standard reference object in the physical world, distance information corresponding to pixels in the high-definition image at the corresponding moment or point cloud information can be generated.
[0010] Based on distance and target information, abnormal target status and abnormal target encroachment on the safe zone for train operation are identified.
[0011] Preferably, it also includes 3D map reconstruction and updating: based on depth maps or point clouds, combined with positioning and inertial navigation data, a 3D map of the train operation line is constructed; when the 3D maps reconstructed based on depth maps or point clouds at different times have significant differences in any area, it is determined whether the map information needs to be updated.
[0012] Preferably, it also includes trend analysis and early warning: analyzing and predicting the contact wire height, facility or target location data in the reconstructed 3D map, and issuing an early warning when the predicted deviation of a specific target will cause encroachment or abnormality within a preset time.
[0013] Preferably, the specific steps for generating high-definition image frame data are as follows: using the pulse-image reconstruction algorithm, high-definition high-quality image frame data is obtained through image filtering and weighted summation, and a timestamp is output.
[0014] Preferably, traditional optical flow algorithms and deep learning-based optical flow algorithms are used to estimate optical flow information and timestamps using image frame sequences, pulse streams, and event streams.
[0015] Preferably, the training steps of the monocular depth estimation model are as follows: In the experimental environment, a lidar or binocular camera is added, and data is collected synchronously. The aligned and fused image frames and optical flow data are used as model inputs and matched with the point clouds or depth maps generated by the lidar or binocular camera as model outputs to train and optimize the monocular depth estimation model.
[0016] Preferably, the step of estimating the angular velocity of the target motion is as follows: using the aligned and fused image frames and optical flow data, feature points in different image frames at adjacent times are calculated, and the angular velocity of the target motion relative to the camera system is estimated by feature point matching.
[0017] The present invention also discloses a high-speed operation environment detection device for rail transit trains, comprising an onboard data acquisition and real-time analysis unit, a ground data processing and analysis unit, and a decision support and alarm unit; the onboard data acquisition and real-time analysis unit, the ground data processing and analysis unit, and the decision support and alarm unit are interconnected.
[0018] The vehicle-mounted data acquisition and real-time analysis unit includes a high-speed camera, a data processing and recording module, and a power supply and communication module; the high-speed camera and the data processing and recording module are both connected to the power supply and communication module.
[0019] The ground data processing and analysis unit includes a data center, an image processing server, and a data analysis platform; the data center, image processing server, and data analysis platform are connected in sequence.
[0020] The decision support and alarm unit includes an alarm module and a decision support module; the alarm module is connected to the data processing and recording module and the data analysis platform respectively; the decision support module is connected to the data analysis module.
[0021] The present invention further discloses a computer-readable storage medium having a computer program stored thereon, the computer program executing the steps of the method described above when run by a processor.
[0022] The present invention also discloses a high-speed operation environment detection system for rail transit trains, including a memory and a processor connected to each other. The memory stores a computer program, which executes the steps of the method described above when run by the processor.
[0023] Compared with the prior art, the advantages of the present invention are as follows:
[0024] This invention is mainly used in operating vehicles or inspection vehicles. It offers numerous advantages over manual and automatic analysis based on existing train 6C, 6A, and other equipment, including high-speed detection, rich information with less omissions, and a higher degree of intelligence. Specifically, it includes:
[0025] Real-time performance and efficiency: By capturing and transmitting data in real time through high-speed cameras, maintenance personnel can promptly understand the operating status of equipment and identify potential problems, improving the timeliness and efficiency of maintenance. Compared to traditional manual inspections and existing mobile inspection vehicles, this technology can significantly shorten inspection cycles, improve inspection efficiency, and reduce labor costs.
[0026] Comprehensiveness and Accuracy: Cameras can cover areas that are difficult to reach with traditional inspections, such as the top of tunnels and high-altitude sections of overhead contact lines, providing a more comprehensive field of view. With the help of image processing technology and algorithms, equipment defects and anomalies can be identified more accurately, reducing false alarm and missed alarm rates.
[0027] Intelligentization and Automation: Combining artificial intelligence and big data technologies, this technology enables intelligent analysis and prediction, allowing for the early detection of potential problems and the development of corresponding maintenance plans. Automated inspection reduces interference from human factors, improving the objectivity and accuracy of inspection results.
[0028] Data accumulation and analysis: The accumulation of data from long-term operation provides maintenance personnel with rich historical data resources, which helps to analyze equipment aging patterns, failure modes, etc., and provides a scientific basis for equipment updates and maintenance. Attached Figure Description
[0029] Figure 1 This is a flowchart of an embodiment of the operating environment detection method of the present invention.
[0030] Figure 2 This is a block diagram of the operating environment detection device of the present invention in an embodiment. Detailed Implementation
[0031] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.
[0032] like Figure 1 As shown, the method for detecting the high-speed operating environment of rail transit trains provided in this embodiment of the invention includes the following steps:
[0033] S1: High-definition high-speed camera image acquisition. Using one or more high-speed, high-resolution cameras, it acquires visible light images, high-speed pulse vision, and event information of the train's operating environment and surrounding infrastructure, ensuring camera time synchronization and outputting signal data and time simultaneously.
[0034] S2: High-resolution image generation. High-resolution visible light images are generated using a pulse-image reconstruction algorithm. Methods for this step include, but are not limited to, the cumulative method, the pulse time interval method, methods based on convolutional neural networks (CNN) or spiking neural networks (SNN), generative models based on diffusion models, and combinations of one or more of the above methods. High-resolution, high-quality image frame data is obtained through image filtering and weighted summation, and a timestamp is output.
[0035] S3: High-density optical flow data generation. This method employs both traditional optical flow algorithms and deep learning-based optical flow algorithms, using image frame sequences, pulse streams, and event streams to estimate optical flow data and timestamps. It leverages the high resolution of image frame sequences and the high temporal density of pulse streams and event streams to ensure the high spatiotemporal density of the optical flow data.
[0036] S4: Data fusion and alignment. Image frames and optical flow data are aligned and fused according to timestamps using methods such as nearest neighbor matching, and then used as input for subsequent algorithms such as depth estimation, object recognition, and segmentation.
[0037] S5: Monocular Depth Estimation Based on Deep Learning. In the experimental environment, data acquisition devices such as LiDAR or binocular cameras are added to collect data synchronously with the current system. The aligned and fused image frames and optical flow data are used as model inputs and matched with the point clouds or depth maps generated by the LiDAR or binocular cameras as model outputs to train and optimize the monocular depth estimation model. In practical applications, the aligned and fused image frames and optical flow data are input into the monocular depth estimation model to obtain depth information.
[0038] S6: Feature point extraction, matching, and distance estimation. Feature points (x and x') in different image frames at adjacent times (t and t') are calculated using aligned and fused image frames and optical flow data. The angular velocity of the target relative to the camera system is estimated by feature point matching (calculated based on the coordinate displacement of x and x' in the image, the time difference, and camera intrinsic parameters).
[0039] S7: Standard Reference Object Recognition. By annotating image data and training instance segmentation or panoramic segmentation models based on deep learning models, it identifies standard reference objects such as tracks, overhead contact lines and supports, and signs in the train operating environment, and outputs information such as the target category and its position in the image, or the pixel position of the target.
[0040] S8: Depth (Distance) Fusion and Calibration. Combining monocular depth estimation, feature point angular velocity, and the size information of standard reference objects in the physical world (such as track spacing, support distance, sign size, etc.), it provides distance information for pixels in the high-resolution image at the corresponding time or generates point cloud information.
[0041] S9: Anomaly Detection. Combining distance and target information, it identifies and judges anomalies such as target status anomalies (deformation, missing, or loosening of key components of infrastructure such as the overhead contact system) and target encroachment on the safe operating area of trains.
[0042] S10: 3D Map Reconstruction and Update. Based on depth maps or point clouds, and combined with positioning and inertial navigation data, a 3D map of the train route is constructed. When the 3D maps reconstructed from depth maps or point clouds at different times show significant differences in any area (e.g., a target deviates by more than 0.1m), a manual check is prompted to confirm whether the map information needs to be updated.
[0043] S11: Trend Analysis and Early Warning: Analyze and predict data such as catenary height, facility or target location in the reconstructed 3D map. When the predicted deviation of a specific target is expected to cause encroachment or abnormality in the next week or month, issue an early warning to prompt manual judgment.
[0044] This invention can fully perceive the environment within the field of view in relatively high-speed motion scenarios, realize the detection of various target states and obstacle intrusions in the train operation environment, and combine image-based 3D reconstruction technology to perform 3D modeling of the train operation perimeter environment. By comparing and analyzing the reconstructed environment model data at different times, it is possible to reason about the perimeter intrusion and abnormal development trends, which is of great significance for improving the safety of high-speed rail operation.
[0045] This invention utilizes the high frame rate, high resolution, and real-time performance of high-speed cameras to accurately detect various target states and obstacles in the operating environment of rail transit trains. It also identifies abnormal trends through 3D reconstruction combined with statistical analysis of historical data, thereby improving the safety and operational efficiency of rail transit.
[0046] like Figure 2 As shown, the high-speed operation environment detection device for rail transit trains according to an embodiment of the present invention includes an onboard data acquisition and real-time analysis unit, a ground data processing and analysis unit, and a decision support and alarm unit; the onboard data acquisition and real-time analysis unit, the ground data processing and analysis unit, and the decision support and alarm unit are interconnected.
[0047] The onboard data acquisition and real-time analysis unit includes:
[0048] High-speed camera (array): One or more high-speed, high-resolution cameras are installed on the top and sides of the operating vehicle to ensure that the cameras can cover the overhead contact line, track, tunnel walls and the surrounding environment.
[0049] Data processing and recording module: Integrated inside the vehicle, responsible for receiving, processing and storing real-time data streams from cameras and possible auxiliary sensor data (such as GPS location, vehicle speed, etc.).
[0050] Power supply and communication module: Ensures stable power supply to the camera and data recording unit, and wirelessly transmits alarm and abnormal data information to the ground data center. After the train operation ends, the onboard system will transmit all collected data back to the ground data center wirelessly or via wired connection.
[0051] The ground data processing and analysis unit includes:
[0052] Data center: Receives and stores data from the vehicle system, including video, images, and auxiliary information.
[0053] Image processing server: preprocesses video and image data (e.g., denoising and enhancement); enables target detection (e.g., catenary status, track defects, tunnel structural damage, foreign object intrusion, etc.), tracking and classification in the train operation environment; and performs 3D modeling of the train operation perimeter environment.
[0054] Data analytics platform: Utilizes big data analytics, machine learning, and other technologies to perform statistical analysis on historical data, identify abnormal trends, and predict potential risks.
[0055] The decision support and alarm unit includes:
[0056] Alarm module: Based on image processing and analysis results, it automatically triggers the alarm mechanism and generates corresponding alarm information for different types of abnormal situations.
[0057] Decision Support Module: Provides maintenance personnel with an intuitive interface that displays real-time monitoring results, historical data analysis reports, and anomaly trend predictions, assisting in the development of predictive maintenance plans and emergency response strategies to avoid equipment damage and safety risks.
[0058] This invention installs cameras on operating vehicles to achieve comprehensive detection of the overhead contact line, operating environment, track, and tunnels. During train operation, the data collected by the cameras is collected in real time, and the data is analyzed in real time using deep learning algorithms. When an anomaly is detected, the onboard system can immediately send an alarm to the driver or control center, enabling real-time detection and handling of problems.
[0059] This invention is mainly used in operating vehicles or inspection vehicles. It offers numerous advantages over manual and automatic analysis based on existing train 6C, 6A, and other equipment, including high-speed detection, rich information with less omissions, and a higher degree of intelligence. Specifically, it includes:
[0060] Real-time performance and efficiency: By capturing and transmitting data in real time through high-speed cameras, maintenance personnel can promptly understand the operating status of equipment and identify potential problems, improving the timeliness and efficiency of maintenance. Compared to traditional manual inspections and existing mobile inspection vehicles, this technology can significantly shorten inspection cycles, improve inspection efficiency, and reduce labor costs.
[0061] Comprehensiveness and Accuracy: Cameras can cover areas that are difficult to reach with traditional inspections, such as the top of tunnels and high-altitude sections of overhead contact lines, providing a more comprehensive field of view. With the help of image processing technology and algorithms, equipment defects and anomalies can be identified more accurately, reducing false alarm and missed alarm rates.
[0062] Intelligentization and Automation: Combining artificial intelligence and big data technologies, this technology enables intelligent analysis and prediction, allowing for the early detection of potential problems and the development of corresponding maintenance plans. Automated inspection reduces interference from human factors, improving the objectivity and accuracy of inspection results.
[0063] Data accumulation and analysis: The accumulation of data from long-term operation provides maintenance personnel with rich historical data resources, which helps to analyze equipment aging patterns, failure modes, etc., and provides a scientific basis for equipment updates and maintenance.
[0064] The present invention further discloses a computer-readable storage medium having a computer program stored thereon, the computer program executing the steps of the method described above when run by a processor.
[0065] The present invention also discloses a high-speed operation environment detection system for rail transit trains, including a memory and a processor connected to each other. The memory stores a computer program, which executes the steps of the method described above when run by the processor.
[0066] The medium and system of the present invention, corresponding to the methods described above, also have the advantages described above.
[0067] The present invention can implement all or part of the processes in the methods of the above embodiments, or it can be implemented by hardware related to computer program instructions. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, it can implement the steps of the above method embodiments. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable storage medium includes: any entity or device capable of carrying computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. The memory is used to store computer programs and / or modules. The processor implements various functions by running or executing the computer programs and / or modules stored in the memory, and by calling data stored in the memory. The memory may include high-speed random access memory, as well as non-volatile memory, such as hard disks, RAM, plug-in hard disks, smart media cards (SMC), secure digital (SD) cards, flash cards, at least one disk storage device, flash memory device, or other volatile solid-state storage devices.
[0068] Definitions:
[0069] 3D reconstruction: By processing and analyzing two-dimensional images or data containing depth information, a three-dimensional model of an object or scene is restored and constructed. The process involves feature extraction, camera calibration, and three-dimensional information calculation, and is one of the key technologies in the field of computer vision.
[0070] GNSS: Global Navigation Satellite System, is a high-precision radio navigation and positioning system based on artificial Earth satellites.
[0071] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should be considered within the scope of protection of the present invention.
Claims
1. A method for detecting the high-speed operating environment of rail transit trains, characterized in that, Including the following steps: Acquire visible light images, high-speed pulse vision and event information of the train's operating environment and surrounding infrastructure, and generate high-definition image frame data, optical flow data and corresponding timestamps; The image frame data and optical flow data are aligned and fused according to timestamps, and then input into a pre-trained monocular depth estimation model to obtain depth estimation information. At the same time, feature points in different image frames at adjacent times are calculated based on the aligned and fused image frames and optical flow data, and the angular velocity of the target motion is estimated by matching the feature points. And to identify standard reference objects and target information in the train operating environment based on image frames; By combining monocular depth estimation, angular velocity, and the size information of the standard reference object in the physical world, distance information corresponding to pixels in the high-definition image at the corresponding moment or point cloud information can be generated. Based on distance and target information, abnormal target status and abnormal target encroachment on the safe zone for train operation are identified.
2. The method for detecting the high-speed operating environment of rail transit trains according to claim 1, characterized in that, It also includes 3D map reconstruction and updating: based on depth maps or point clouds, combined with positioning and inertial navigation data, a 3D map of the train operation line is constructed; when the 3D maps reconstructed based on depth maps or point clouds at different times have significant differences in any area, it is determined whether the map information needs to be updated.
3. The method for detecting the high-speed operating environment of rail transit trains according to claim 2, characterized in that, It also includes trend analysis and early warning: analyzing and predicting the height of the overhead contact system, the location of facilities or targets in the reconstructed 3D map, and issuing an early warning when the predicted deviation of a specific target will cause encroachment or abnormality within a preset time.
4. The method for detecting the high-speed operating environment of rail transit trains according to claim 1, 2, or 3, characterized in that, The specific steps for generating high-definition image frame data are as follows: using the pulse-image reconstruction algorithm, high-definition, high-quality image frame data is obtained through image filtering and weighted summation, and a timestamp is output.
5. The method for detecting the high-speed operating environment of rail transit trains according to claim 1, 2, or 3, characterized in that, We employ both traditional optical flow algorithms and deep learning-based optical flow algorithms to estimate optical flow information and timestamps using image frame sequences, pulse streams, and event streams.
6. The method for detecting the high-speed operating environment of rail transit trains according to claim 1, 2, or 3, characterized in that, The training steps for a monocular depth estimation model are as follows: In the experimental environment, a lidar or binocular camera is added, and data is collected synchronously. The aligned and fused image frames and optical flow data are used as model inputs and matched with the point clouds or depth maps generated by the lidar or binocular camera as model outputs to train and optimize the monocular depth estimation model.
7. The method for detecting the high-speed operating environment of rail transit trains according to claim 1, 2, or 3, characterized in that, The steps for estimating the angular velocity of the target motion are as follows: using the aligned and fused image frames and optical flow data, feature points in different image frames at adjacent times are calculated, and the angular velocity of the target motion relative to the camera system is estimated by matching the feature points.
8. A high-speed operating environment detection device for rail transit trains, characterized in that, It includes an onboard data acquisition and real-time analysis unit, a ground data processing and analysis unit, and a decision support and alarm unit; the onboard data acquisition and real-time analysis unit, the ground data processing and analysis unit, and the decision support and alarm unit are interconnected; The vehicle-mounted data acquisition and real-time analysis unit includes a high-speed camera, a data processing and recording module, and a power supply and communication module; the high-speed camera and the data processing and recording module are both connected to the power supply and communication module. The ground data processing and analysis unit includes a data center, an image processing server, and a data analysis platform; the data center, image processing server, and data analysis platform are connected in sequence. The decision support and alarm unit includes an alarm module and a decision support module; the alarm module is connected to the data processing and recording module and the data analysis platform respectively; the decision support module is connected to the data analysis module.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer program, when run by a processor, performs the steps of the method as described in any one of claims 1-7.
10. A high-speed operating environment detection system for rail transit trains, comprising a memory and a processor interconnected, wherein the memory stores a computer program, characterized in that, The computer program, when run by a processor, performs the steps of the method as described in any one of claims 1-7.