Train operation environment monitoring system and method based on multi-source sensor fusion

Abstract

The invention provides a train operation environment monitoring system and method based on multi-source sensor fusion, and the system comprises a multi-source data module which is used for collecting the multi-dimensional data of a train operation environment in real time, and transmitting the multi-dimensional data to a main control module; the main control module is used for performing fusion processing on the multi-dimensional data, analyzing the health degree of the train operation environment through an artificial intelligence algorithm based on the processed data, and sending an analysis result to the decision output module; the decision output module is used for outputting a corresponding decision result according to the analysis result; and the network communication module is used for guaranteeing real-time data transmission between the multi-source data module and the main control module. According to the method, various types of sensors are deployed in the train, the rail line and the surrounding environment, an unmanned aerial vehicle detection technology is added, and data are processed and analyzed in combination with big data analysis and an artificial intelligence algorithm, so that the health degree of the train operation environment is accurately evaluated.

Description

Technical Field

[0001] This invention relates to the field of railway transportation technology, and in particular to a train operation environment monitoring system and method based on multi-source sensor fusion. Background Technology

[0002] In modern transportation systems, trains, as high-capacity and high-efficiency modes of transport, directly impact the safety of passengers' lives and property, as well as the stable operation of the social economy. As railway transportation develops towards higher speeds, heavier loads, and greater intelligence, the train operating environment is becoming increasingly complex. Traditional single-sensor detection methods are no longer sufficient to meet the demands for comprehensive, high-precision, and real-time monitoring of the operating environment. Therefore, there is an urgent need for an intelligent detection solution that can integrate data from multiple sensor sources, improve detection accuracy and real-time performance, to ensure train operation safety and enhance transportation efficiency. Summary of the Invention

[0003] The purpose of this invention is to provide a train operation environment monitoring system and method based on multi-source sensor fusion, aiming to solve the above-mentioned problems in the prior art.

[0004] This invention provides a train operation environment monitoring system based on multi-source sensor fusion, comprising: A multi-source data module, connected to the main control module and the network communication module, is used to collect multi-dimensional data of the train's operating environment in real time and transmit the multi-dimensional data to the main control module. The main control module, connected to the multi-source data module, decision output module and network communication module, is used to perform fusion processing on the multi-dimensional data, analyze the health status of the train operating environment based on the processed data through artificial intelligence algorithms, and send the analysis results to the decision output module. A decision output module, connected to the main control module, is used to output corresponding decision results based on the analysis results; The network communication module is connected to the multi-source data module and the main control module to ensure real-time data transmission between the multi-source data module and the main control module.

[0005] This invention provides a method for monitoring train operating environment based on multi-source sensor fusion, comprising: The multi-source data module collects multi-dimensional data of the train's operating environment in real time and transmits the multi-dimensional data to the main control module. The main control module integrates and processes the multi-dimensional data, and based on the processed data, it uses artificial intelligence algorithms to analyze the health of the train's operating environment and sends the analysis results to the decision output module. The decision output module outputs the corresponding decision results based on the analysis results. The network communication module ensures real-time data transmission between the multi-source data module and the main control module.

[0006] This invention also provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the steps of the above-described train operation environment monitoring method based on multi-source sensor fusion.

[0007] This invention also provides a computer-readable storage medium storing an information transmission implementation program, which, when executed by a processor, implements the steps of the above-described train operation environment monitoring method based on multi-source sensor fusion.

[0008] The embodiments of the present invention can include the following beneficial effects: Based on traditional locomotive identification technology, the embodiments of the present invention deploy various types of sensors along the train, track and surrounding environment and add UAV detection technology to collect multi-source data such as track status, vehicle equipment parameters, and meteorological conditions in real time. Data transmission is achieved using Internet of Things technology, and the data is processed and analyzed by combining big data analysis and artificial intelligence algorithms, thereby accurately assessing the health of the train operating environment and providing intelligent decision support to achieve preventive maintenance and safety assurance. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in one or more embodiments of this specification or in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 This is a schematic diagram of a train operation environment monitoring system based on multi-source sensor fusion according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the traffic operation optimization process according to an embodiment of the present invention; Figure 3 This is a system deployment diagram according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the installation position of the vehicle-mounted sensor according to an embodiment of the present invention; Figure 5 This is a schematic diagram illustrating the installation and operation of the 3D camera according to an embodiment of the present invention; Figure 6 This is a schematic diagram of the communication of the trackside sensor module according to an embodiment of the present invention; Figure 7This is a schematic diagram of unmanned aerial vehicle (UAV) nest communication according to an embodiment of the present invention; Figure 8 This is a flowchart illustrating the implementation and deployment of an embodiment of the present invention; Figure 9 This is a flowchart of a train operation environment monitoring method based on multi-source sensor fusion according to an embodiment of the present invention. Detailed Implementation

[0011] To enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the technical solutions in one or more embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of this document.

[0012] System Implementation Examples According to embodiments of the present invention, a train operation environment monitoring system based on multi-source sensor fusion is provided. Figure 1 This is a schematic diagram of a train operation environment monitoring system based on multi-source sensor fusion according to an embodiment of the present invention, as shown below. Figure 1 As shown, the train operation environment monitoring system based on multi-source sensor fusion according to an embodiment of the present invention specifically includes: The multi-source data module 10 is connected to the main control module and the network communication module, and is used to collect multi-dimensional data of the train operating environment in real time and transmit the multi-dimensional data to the main control module. Specifically, the multi-source data module includes: The onboard sensor unit includes long and short focal length cameras, lidar, and a 3D camera. It is used to detect track obstacles, track gauge, turnout status, track status, and track surface status in real time through several multi-source sensors deployed on the train, and to acquire corresponding three-dimensional image data and point cloud data. The turnout status includes whether the turnout transition point and the transition status of various turnout transition points are in place. The trackside sensor unit is used to monitor climate conditions, geological hazards, and trackside environmental conditions in real time through several multi-source sensors deployed along the railway line. The drone inspection unit, deployed at the trackside drone nest, is used to direct drones to perform routine survey and communication tasks as well as on-site surveys and data transmission for emergencies.

[0013] The main control module 12, connected to the multi-source data module, decision output module, and network communication module, is used to fuse and process the multi-dimensional data, analyze the health status of the train operating environment based on the processed data using artificial intelligence algorithms, and send the analysis results to the decision output module. Specifically, it is used for: Local caching and preliminary verification of multi-dimensional data collected by several multi-source sensors; By calibrating the extrinsic parameter matrix and using Formula 1, the lidar point cloud data is mapped from the radar coordinate system to the camera coordinate system, thus completing the spatial alignment of the 3D image data and the point cloud data. Formula 1; in, Indicates the spatial position of the target point in the camera's view. This indicates the original measured position of the target point from the radar's perspective. Represents the rotation matrix. Represents the translation vector; The system determines whether defects exist in the three-dimensional image data of the track status based on a preset threshold; and it extracts features from the three-dimensional image data of the track status, and classifies the defect type and its severity based on the obtained feature data using a deep learning model.

[0014] The decision output module 14 is connected to the main control module and is used to output corresponding decision results based on the analysis results; wherein, the decision results include control commands, alarm signals, braking commands or maintenance suggestions; The network communication module 16 is connected to the multi-source data module and the main control module to ensure real-time data transmission between the multi-source data module and the main control module.

[0015] The technical solutions of the present invention will be described in detail below with reference to the specific circumstances of the train operation environment monitoring system based on multi-source sensor fusion in the embodiments of the present invention.

[0016] This invention presents a scheme for detecting the health status of the train operating environment based on multi-sensor fusion, aiming to overcome the limitations of traditional detection methods and meet the growing safety and management needs of railway transportation. Figure 2 As shown, its core objectives encompass multiple aspects, including security assurance, operational optimization, and technology upgrades, specifically including: 1. Strengthen train operation safety assurance; 2. Improve the efficiency of railway operation and maintenance management; 3. Promote the intelligent development of railway transportation; 4. Promote the improvement of railway transportation service quality.

[0017] The relevant performance design requirements of this invention are shown in Table 1: Table 1 Performance Design Requirements like Figure 3 As shown, the components of this embodiment of the invention include: 1. Main Unit: As the brain, it is responsible for integrating, processing and analyzing the data from various sensors, and outputting braking signals according to the actual situation.

[0019] 2. Vehicle-mounted sensor modules, such as Figure 4 As shown: This includes long- and short-focal-length cameras and a LiDAR system. An image + point cloud fusion method is used to detect obstacles, track gauge, and switch transitions. The image data acquired by the cameras is located in the image pixel coordinate system (u, v), and is converted into the camera's three-dimensional coordinate system using intrinsic parameters (focal length, principal point, etc.). The point cloud acquired by the lidar is located in the lidar's three-dimensional coordinate system. The calibration objective is to solve the extrinsic parameter matrix. This allows points in the radar coordinate system to be transformed using the formula... By mapping to the camera coordinate system, spatial alignment of the "image + point cloud" is ultimately achieved.

[0020] 3. 3D cameras, such as Figure 5 As shown: Installed at the bottom of the locomotive, it is used to scan the track's outline dimensions, rail surface wear, etc. It monitors the current state of the rails in real time, acquiring 3D image data of the rails through a 3D camera scan, and then processes and extracts features from the collected data.

[0021] Ultimately, algorithmic analysis and deep learning-based intelligent recognition are used to determine whether the rail surface meets standards or what type of fault it is. Thresholds (such as wear depth and deformation) are set to determine the presence of defects. For example: A. When the vertical wear of the rail head is >12mm (according to railway standard TB / T 3562-2020), it is marked as "severe wear"; B. If there is a peeling depth > 5mm on the rail surface, it is marked as a "peeling defect".

[0022] For complex defects (such as cracks and welding defects), point cloud features are trained using models such as PointNet and PointTransformer to achieve automatic classification of defect types (such as transverse cracks and fish scale wear) and severity.

[0023] 4. Trackside sensor module, such as Figure 6 As shown: The trackside sensor module mainly realizes climate detection and geological disaster monitoring, and sends real-time data to the local rail transit bureau and the nearest train via 5G gimbal.

[0024] 5. Trackside drone nests, such as Figure 7 As shown, the main function of the drone pod is to respond proactively to geological disasters, proceed to the accident site for further verification, and transmit real-time video footage back to the traffic management bureau to provide support to maintenance personnel and facilitate further safety instructions. Simultaneously, drones can also serve as an important tool for inspecting railway lines. Drones primarily monitor relevant real-time conditions through onboard cameras.

[0025] Its main performance characteristics include: a. The unmanned shelter is connected to the railway power supply and can charge itself; b. The drone nest establishes communication with the nearest maintenance station to prevent intentional theft or damage to the drone; c. Drones can carry out relevant reconnaissance and communication tasks according to the instructions of the traffic management bureau; d. Nearby maintenance personnel can operate drones to complete railway inspection tasks.

[0026] In summary, the embodiments of the present invention need to complete the safety analysis of whether the switching status at various turnout switching points is in place; to scan the rail surface and detect the track condition when the train is traveling at high speed; and due to the fusion and cooperation of multiple sensors, the real-time performance of data transmission, the accuracy of command transmission, and the stability of network communication all need to be given special attention.

[0027] The specific implementation and deployment of the embodiments of the present invention are as follows: Figure 8 As shown, it includes: 1. Data acquisition for the operating environment 1.1 Deploy various sensors in the early stage, and have each sensor start the data acquisition task according to the preset frequency. Set the corresponding data acquisition frequency according to the characteristics of the sensor and the actual operating conditions. 1.2 The data collected by the sensor is locally cached and initially verified to ensure the integrity and accuracy of the data before being used in software development.

[0028] 2. Installation of related equipment and sensors If this solution is used on new trains, the train equipment sensors will be installed simultaneously during the new train manufacturing phase; if used on existing trains, they will be retrofitted during vehicle maintenance. Trackside sensors and drone pods will be deployed at key locations along the railway line, ensuring that the installation locations meet technical requirements, have good signal strength, and that the surrounding environment can support the normal operation of the relevant sensors. 3. Improve network infrastructure 5G base stations and fiber optic networks will be built along the railway line to ensure seamless signal coverage. Satellite communication equipment will be used as a supplement for remote mountainous areas and other network blind spots. 4. Overall linkage debugging After completing the hardware installation and network deployment, system integration testing is conducted to verify the accuracy of sensor data acquisition, the stability of data transmission, and the effectiveness of the algorithm model, in order to ensure the reliable operation of the system.

[0029] Method Implementation Examples According to embodiments of the present invention, a method for monitoring train operating environment based on multi-source sensor fusion is provided. Figure 9 This is a flowchart of a train operation environment monitoring method based on multi-source sensor fusion according to an embodiment of the present invention, as follows: Figure 9 As shown, the train operation environment monitoring method based on multi-source sensor fusion according to an embodiment of the present invention specifically includes: Step S901 involves real-time acquisition of multi-dimensional data about the train's operating environment via a multi-source data module, and transmission of this multi-dimensional data to the main control module. Specifically, this includes: The onboard sensor unit of the multi-source data module uses several multi-source sensors deployed on the train to detect track obstacles, track gauge, turnout status, track status, and track surface status in real time, and acquire corresponding three-dimensional image data and point cloud data; wherein, the turnout status includes whether the turnout transition point and the transition status of various turnout transition points are in place. The trackside sensor unit of the multi-source data module uses several multi-source sensors deployed along the railway line to monitor climate conditions, geological disasters and trackside environmental conditions in real time. The drone inspection unit, which uses multi-source data modules, directs drones to perform routine survey and communication tasks as well as on-site surveys and data transmission for emergencies.

[0030] Step S902 involves the main control module fusing the multi-dimensional data, analyzing the health of the train's operating environment based on the processed data using artificial intelligence algorithms, and sending the analysis results to the decision output module. Specifically, this includes: Local caching and preliminary verification of multi-dimensional data collected by several multi-source sensors; By calibrating the extrinsic parameter matrix and using Formula 1, the lidar point cloud data is mapped from the radar coordinate system to the camera coordinate system, thus completing the spatial alignment of the 3D image data and the point cloud data. Formula 1; in, Indicates the spatial position of the target point in the camera's view. This indicates the original measured position of the target point from the radar's perspective. Represents the rotation matrix. Represents the translation vector; The system determines whether defects exist in the three-dimensional image data of the track status based on a preset threshold; and it extracts features from the three-dimensional image data of the track status, and classifies the defect type and its severity based on the obtained feature data using a deep learning model.

[0031] Step S903: The decision output module outputs the corresponding decision result based on the analysis result; wherein, the decision result includes control command, alarm signal, braking command or maintenance suggestion; Step S904: Ensure real-time data transmission between the multi-source data module and the main control module through the network communication module.

[0032] The embodiments of the present invention are method embodiments corresponding to the system embodiments described above. The specific operations of each step can be understood by referring to the description of the system embodiments, and will not be repeated here.

[0033] In summary, this invention integrates multi-sensor technology, the Internet of Things, big data, and artificial intelligence algorithms to construct a complete train operation environment monitoring system, breaking through the limitations of traditional single-sensor monitoring. Through collaborative sensing at the sensor layer, intelligent fusion at the data transmission layer, and in-depth mining at the data processing and analysis layer, it achieves a fully intelligent upgrade from data acquisition to decision-making applications, providing new ideas and practical examples for technological innovation in the rail transit field and driving the industry towards intelligence and digitalization. The specific beneficial effects of this invention include: 1. Breaking away from the traditional task of identifying track obstacles, it also adds the identification of track gauge, turnout transitions, and track surface; 2. Integrating identification data from multiple sensors greatly improves the safety of train operation; 3. By integrating multiple sensors, traditional manual railway inspection tasks can be replaced, enabling more comprehensive and complete railway inspections. 4. Integrating drones into the entire system can improve the efficiency of understanding emergencies and the accuracy of sending relevant instructions, making the system more complete, powerful, and comprehensive.

[0034] Device Example 1 This invention provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, performs the steps described in the method embodiment.

[0035] Device Example 2 This invention provides a computer-readable storage medium storing an information transmission implementation program, which, when executed by a processor, performs the steps described in the method embodiment.

[0036] The computer-readable storage media described in this embodiment include, but are not limited to, ROM, RAM, disk, or optical disk.

[0037] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A train operation environment monitoring system based on multi-source sensor fusion, characterized in that... include: A multi-source data module, connected to the main control module and the network communication module, is used to collect multi-dimensional data of the train's operating environment in real time and transmit the multi-dimensional data to the main control module. The main control module, connected to the multi-source data module, decision output module and network communication module, is used to perform fusion processing on the multi-dimensional data, analyze the health status of the train operating environment based on the processed data through artificial intelligence algorithms, and send the analysis results to the decision output module. A decision output module, connected to the main control module, is used to output corresponding decision results based on the analysis results; The network communication module is connected to the multi-source data module and the main control module to ensure real-time data transmission between the multi-source data module and the main control module.

2. The system according to claim 1, characterized in that, The multi-source data module specifically includes: The onboard sensor unit includes long and short focal length cameras, lidar, and a 3D camera. It is used to detect track obstacles, track gauge, turnout status, track status, and track surface status in real time through several multi-source sensors deployed on the train, and to acquire corresponding three-dimensional image data and point cloud data. The turnout status includes whether the turnout transition point and the transition status of various turnout transition points are in place. The trackside sensor unit is used to monitor climate conditions, geological hazards, and trackside environmental conditions in real time through several multi-source sensors deployed along the railway line. The drone inspection unit, deployed at the trackside drone nest, is used to direct drones to perform routine survey and communication tasks as well as on-site surveys and data transmission for emergencies.

3. The system according to claim 2, characterized in that, The main control module is specifically used for: Local caching and preliminary verification of multi-dimensional data collected by several multi-source sensors; By calibrating the extrinsic parameter matrix and using Formula 1, the lidar point cloud data is mapped from the radar coordinate system to the camera coordinate system, thus completing the spatial alignment of the 3D image data and the point cloud data. Formula 1: in, Indicates the spatial position of the target point in the camera's view. This indicates the original measured position of the target point from the radar's perspective. Represents the rotation matrix. Represents the translation vector; The system determines whether defects exist in the three-dimensional image data of the track status based on a preset threshold; and it extracts features from the three-dimensional image data of the track status, and classifies the defect type and its severity based on the obtained feature data using a deep learning model.

4. The system according to claim 1, characterized in that, The decision results include control commands, alarm signals, braking commands, or maintenance recommendations.

5. A method for monitoring train operating environment based on multi-source sensor fusion, characterized in that... include: The multi-source data module collects multi-dimensional data of the train's operating environment in real time and transmits the multi-dimensional data to the main control module. The main control module integrates and processes the multi-dimensional data, and based on the processed data, it uses artificial intelligence algorithms to analyze the health of the train's operating environment and sends the analysis results to the decision output module. The decision output module outputs the corresponding decision results based on the analysis results. The network communication module ensures real-time data transmission between the multi-source data module and the main control module.

6. The method according to claim 5, characterized in that, The real-time collection of multi-dimensional data on the train operating environment through multi-source data modules specifically includes: The onboard sensor unit of the multi-source data module uses several multi-source sensors deployed on the train to detect track obstacles, track gauge, turnout status, track status, and track surface status in real time, and acquire corresponding three-dimensional image data and point cloud data; wherein, the turnout status includes whether the turnout transition point and the transition status of various turnout transition points are in place. The trackside sensor unit of the multi-source data module uses several multi-source sensors deployed along the railway line to monitor climate conditions, geological disasters and trackside environmental conditions in real time. The drone inspection unit, which uses multi-source data modules, directs drones to perform routine survey and communication tasks as well as on-site surveys and data transmission for emergencies.

7. The method according to claim 6, characterized in that, The main control module fuses and processes the multi-dimensional data, and based on the processed data, artificial intelligence algorithms are used to analyze the health status of the train's operating environment. Specifically, this includes: Local caching and preliminary verification of multi-dimensional data collected by several multi-source sensors; By calibrating the extrinsic parameter matrix and using Formula 1, the lidar point cloud data is mapped from the radar coordinate system to the camera coordinate system, thus completing the spatial alignment of the 3D image data and the point cloud data. Formula 1: in, Indicates the spatial position of the target point in the camera's view. This indicates the original measured position of the target point from the radar's perspective. Represents the rotation matrix. Represents the translation vector; The system determines whether defects exist in the three-dimensional image data of the track status based on a preset threshold; and it extracts features from the three-dimensional image data of the track status, and classifies the defect type and its severity based on the obtained feature data using a deep learning model.

8. The method according to claim 5, characterized in that, The decision results include control commands, alarm signals, braking commands, or maintenance recommendations.

9. An electronic device, characterized in that, include: The system includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the train operation environment monitoring method based on multi-source sensor fusion as described in any one of claims 5-8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores an information transmission implementation program, which, when executed by a processor, implements the steps of the train operation environment monitoring method based on multi-source sensor fusion as described in any one of claims 5-8.

Images