Running environment sensing system for heavy haul railway train

Abstract

The embodiment of the invention relates to the field of heavy load operation monitoring, and discloses a heavy load railway train operation environment sensing system. The system comprises a line environment monitoring device which is installed in a line extension direction; the foreign matter invasion monitoring device is arranged in the target monitoring protection area; the tunnel deformation monitoring device is mounted on the heavy haul train; the vehicle-mounted positioning device is mounted on the heavy-load train; and the data management platform is used for analyzing and processing according to the monitoring data fed back by each device so as to monitor the running safety state of the train. The system provided by the invention solves the technical problem of poor safety caused by incomplete monitoring of the running environment of the heavy haul railway train, and at least can achieve real-time monitoring and tracking of line dangerous situations such as perimeter invasion, natural disasters and facility deformation of the transportation line of the heavy haul railway and the position of the running vehicle; and data support and technical means are provided for scientific decision making of an operation dispatching center.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of heavy-load operation monitoring, and in particular to a heavy-load railway train operation environment perception system. Background Art

[0002] Existing monitoring of heavy-duty railway lines mostly relies on video data and fiber optic data to monitor the strain and vibration information of certain areas such as bridges and rails, as well as image information on the travel route; however, the heavy-duty railway line environment is complex and harsh, with poor geological stability and many disasters, which directly threaten the safety of train travel. At the same time, abnormal events such as rainstorms and earthquakes make line environmental disasters such as landslides, falling rocks, and floods sudden, which brings great challenges to the perception and early warning of the train travel environment. The existing technology is not comprehensive for the monitoring of the operating environment of heavy-duty railways, which leads to poor safety and easily causes safety accidents. Summary of the invention

[0003] The purpose of the embodiments of the present invention is to at least provide a heavy-load railway train operating environment perception system, which can at least solve the technical problem of poor safety caused by incomplete monitoring of the heavy-load railway train operating environment, and at least achieve real-time monitoring and tracking of line hazards such as perimeter intrusion, natural disasters, facility deformation, and the location of operating vehicles on heavy-load railway transport lines, thereby providing data support and technical means for scientific decision-making of the operation dispatching center.

[0004] In order to solve the above technical problems, at least one embodiment of the present application provides a heavy-haul railway train line safety monitoring system, which is characterized by comprising:

[0005] A line environment monitoring device, installed in the extension direction of the line, for monitoring first image data of the train line environment;

[0006] A foreign body intrusion monitoring device is arranged in a target monitoring and protection area, and is used to monitor at least one of a vibration signal, a second image data, and three-dimensional point cloud data of intruding foreign bodies in the target monitoring and protection area in the heavy-load railway train running line; the target monitoring and protection area includes a high and steep slope area and a tunnel entrance area;

[0007] A tunnel deformation monitoring device, installed on a heavy-load train, for monitoring tunnel profile data about a tunnel in a train running line;

[0008] On-board positioning device, installed on heavy-load trains, used for real-time monitoring of operating vehicle positions and tracking;

[0009] And a data management platform, the data management platform is connected to the line environment monitoring device, the foreign object intrusion monitoring device, the tunnel deformation monitoring device and the vehicle-mounted positioning device, and the data management platform is used to analyze and process the monitoring data fed back by each device to realize the train operation safety status monitoring.

[0010] Compared with the prior art, the heavy-load railway train line safety monitoring system provided in the embodiments of the present application can monitor and track line hazards such as perimeter intrusion, natural disasters, facility deformation, and the position of operating vehicles of heavy-load railway transportation lines in real time through the cooperation of line environment monitoring devices, foreign object intrusion monitoring devices, tunnel deformation monitoring devices, vehicle-mounted positioning devices, and data management platforms, thereby providing data support and technical means for scientific decision-making of the operation and dispatching center, improving the safety of heavy-load railway transportation, and the data of each device can be interoperable and combined to form early warning information, and the early warning information can be pushed to the dispatching center and station of heavy-load railway transportation in real time, so as to take measures in advance to deal with it and improve operational safety.

[0011] In addition, the line environment monitoring device includes a plurality of video surveillance cameras arranged along the extension direction of the line and an image storage server connected to the video surveillance cameras; the data management platform includes a video analysis processor, which is communicatively connected to the image storage server and is used to monitor whether there are foreign objects in the line environment according to the first image data. When foreign objects are detected and identified in the line environment, an alarm message is generated, and images and / or video information containing foreign objects are simultaneously captured from the first image data for display.

[0012] In addition, the foreign object intrusion monitoring device includes a vibration optical fiber for monitoring vibration signals and a laser camera for monitoring second image data and three-dimensional point cloud data of invading foreign objects; the vibration optical fiber is laid in the target monitoring and protection area; the data management platform includes a foreign object signal processor; the foreign object signal processor is connected to the vibration optical fiber and the laser camera, and a fusion recognition algorithm is deployed in the foreign object signal processor to identify whether there are foreign objects falling or invading in the target monitoring and protection area through the fusion recognition algorithm, and generate an alarm message when an abnormality is identified, and simultaneously capture the image and / or video information containing the foreign object from the second image data for display.

[0013] In addition, the foreign object intrusion device includes a high and steep slope monitoring sub-device and a tunnel entrance monitoring sub-device; the high and steep slope monitoring sub-device is used to monitor the clearance safety events along the heavy-load railway operation line; the tunnel entrance monitoring sub-device is used to monitor the clearance safety time at the top slope of the tunnel and the fence at the top of the cutting through which the heavy-load railway operation line passes.

[0014] In addition, the data management platform includes several video databases; each of the video databases is used to classify and store image data of the foreign object intrusion device under different conditions, and the data management platform also includes an evaluation algorithm for predicting foreign object intrusion disasters based on each of the video databases, and training and optimizing the evaluation algorithm. The evaluation algorithm is used to combine the vibration signal, the second image data and the three-dimensional point cloud data of the invading foreign object to predict whether it is dangerous when the train arrives at the area where the foreign object intrusion monitoring device is located.

[0015] In addition, the tunnel deformation monitoring device includes a laser ranging sensor and a laser scanning sensor installed on the train; the laser ranging sensor is used to measure tunnel profile data; the laser scanning sensor is used to measure rail profile data; the data management platform includes a tunnel section data processor, and the tunnel section data processor is used to compensate the tunnel profile data according to the rail profile data to generate complete tunnel section data, and use the tunnel section data to perform deformation and intrusion analysis on the tunnel section information to monitor whether the tunnel is deformed.

[0016] In addition, the vehicle-mounted positioning device includes a Beidou locator, a speed encoder and a laser displacement sensor; the Beidou locator is used to locate heavy-load trains through Beidou high-precision positioning, and the speed sensor and laser displacement sensor are used to measure the speed and locate the train operation; the tunnel deformation monitoring device is connected to the speed encoder and the laser displacement sensor, and is used to determine the position information of the tunnel deformation in combination with the toilet positioning.

[0017] In addition, the laser ranging sensor and the laser scanning sensor are located in the same sensor coordinate system, so that the tunnel profile data is compensated by the rail profile data monitored by the laser scanning sensor; an automatic calibration unit is set in the tunnel section data processor, which is used to convert the monitoring data of the laser ranging sensor and the laser scanning sensor into the track coordinate system through calibration, so as to combine the tunnel profile data with the rail profile data to form complete tunnel section data.

[0018] In addition, the automatic calibration unit stores a track coordinate system coordinate point set and an iterative closest point algorithm; the automatic calibration unit is used to iteratively calculate the track coordinate system coordinate point set and the coordinate point set of measurement data fed back by the laser ranging sensor and the laser scanning sensor through the iterative closest point algorithm to obtain a change matrix of the track coordinate system and the sensor coordinate system, so as to calibrate the laser ranging sensor and the laser scanning sensor through the change matrix.

[0019] In addition, the environmental perception system also includes a comprehensive display interface; the comprehensive display interface is used to display the monitoring results of the monitoring data fed back by the data management platform from each device. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0021] Figure 1 is a structural schematic diagram of a heavy-load railway train operating environment perception system provided according to an embodiment of the present invention;

[0022] Figure 2 is a schematic structural diagram of a line environment monitoring device provided according to an embodiment of the present invention;

[0023] Figure 3 is a schematic structural diagram of a foreign body intrusion monitoring device provided according to an embodiment of the present invention;

[0024] Figure 4 It is a data roadmap of an application example of a foreign body intrusion monitoring device provided according to an embodiment of the present invention;

[0025] Figure 5 This is an example diagram of a tunnel entrance monitoring sub-device monitoring a double-layer three-dimensional protection system of a tunnel entrance provided by an embodiment of the present invention;

[0026] Figure 6 is an exemplary block diagram of a random deformation monitoring device and a data management platform for managing its monitoring data according to an embodiment of the present invention;

[0027] Figure 7 is an exemplary composition diagram of a heavy-haul railway train operating environment perception system provided according to an embodiment of the present invention;

[0028] Figure 8 is an overall exemplary block diagram of a fusion monitoring system provided according to an embodiment of the present invention.

[0029] In the figure, 1. Line environment monitoring device; 11. Video surveillance camera; 12. Image storage server; 2. Foreign object intrusion monitoring device; 21. Vibrating optical fiber; 22. Laser camera; 3. Tunnel deformation monitoring device; 4. Vehicle-mounted positioning device; 5. Data management platform; 51. Video analysis processor; 52. Foreign object signal processor. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical scheme and advantages of the embodiments of the present invention clearer, the embodiments of the present invention will be described in detail below with reference to the accompanying drawings. However, it can be understood by those skilled in the art that in the embodiments of the present invention, many technical details are provided to enable readers to better understand the present invention. However, even without these technical details and various changes and modifications based on the following embodiments, the technical scheme claimed in the present invention can be implemented.

[0031] In order to facilitate understanding of the embodiments of the present application, relevant content about train monitoring is first introduced here.

[0032] In recent years, the number of railway transport vehicle safety accidents has increased. On the one hand, due to the changeable climate conditions, natural disasters such as landslides and mudslides occur from time to time. Under the influence of factors such as human construction, rain erosion, and wind impact, the tunnel entrance is prone to collapse and rockfall above the tunnel entrance, posing a serious threat to the safe operation of heavy-duty railway transport vehicles. Is it necessary to conduct comprehensive monitoring of the clearance safety threats of heavy-duty railway operation? By building an intelligent perception and early warning technology for the clearance safety of heavy-duty railway trains, formulating a photoelectric detection strategy for heavy-duty transport lines is an effective way to ensure the safe and efficient operation of trains. For example, Japan and Germany have their own advanced railway safety protection technologies, and have studied and comprehensively considered various factors of "man-machine-control-environment", enriching the theory of railway safety early warning systems. The Japanese Railway Shinkansen is equipped with a complete disaster prevention alarm system, which monitors all information on various natural disasters along the railway and inputs them all to the dispatching headquarters, closely monitors the dynamic operation of the line, and can ensure the driving safety of trains, setting a record of no train accidents for more than 40 years. Germany's high-speed rail transportation is equipped with a MAS90 safety warning system, which can dynamically monitor the operation of trains. Once there is a situation that affects the operation of trains or abnormalities in mobile equipment, the system will sound an alarm, providing protection for the safety of train operation. However, the means of limiting natural disasters and foreign object intrusions under the transportation conditions of heavy-load transport lines are not yet mature enough; the safe stopping distance of heavy-load trains on general lines is about 800 meters, and the braking distance on downhill sections is even longer. Taking into account the ramps, tunnels, curves, mountains, vegetation and other reasons on high slopes, the braking distance is much greater than the reliable observation distance of the train driver. Incomplete monitoring information can easily lead to an inability to respond in time when an emergency occurs, causing safety accidents.

[0033] Embodiment 1:

[0034] An embodiment of the present invention relates to a heavy-load railway train operating environment perception system.

[0035] Compared with the prior art, the embodiments of the present invention are as follows: Figure 1As shown, through the cooperation of the line environment monitoring device 1, the foreign body intrusion monitoring device 2, the tunnel deformation monitoring device 3, the vehicle-mounted positioning device 4 and the data management platform 5, the line dangers such as the perimeter intrusion, natural disasters, facility deformation and the position of the operating vehicles of the heavy-load railway transportation line can be monitored and tracked in real time, providing data support and technical means for the scientific decision-making of the operation dispatching center, improving the safety of heavy-load railway transportation, and the data of each device can be interconnected and combined to form early warning information, and the early warning information is pushed to the dispatching center and station of the heavy-load railway transportation in real time, so as to take measures in advance to deal with it and improve the operation safety.

[0036] The implementation details of the heavy-duty railway train operating environment perception system of this embodiment are described in detail below. The following content is only the implementation details provided for easy understanding and is not necessary for the implementation of this solution.

[0037] The embodiment of the present invention provides a heavy-duty railway train operating environment perception system, such as Figure 1 and Figure 7 As shown, it includes: a line environment monitoring device 1, a foreign object intrusion monitoring device 2, a tunnel deformation monitoring device 3, a vehicle-mounted positioning device 4 and a data management platform 5; wherein the data management platform 5 is connected to the line environment monitoring device 1, the foreign object intrusion monitoring device 2, the tunnel deformation monitoring device 3 and the vehicle-mounted positioning device 4 through a communication network, and is used to obtain the monitoring data of each device, and analyze and process the monitoring data to realize the train operation safety status monitoring.

[0038] Specifically, the line environment monitoring device 1 is installed in the extension direction of the line, and is used to monitor the first image data of the train line environment; the line environment monitoring device 1 takes image data of the environment of the heavy-load train running line, so that the data management platform 5 uses the first image data to analyze whether there is an abnormality in the line environment, that is, to determine whether the line clearance is safe by analyzing the shooting information;

[0039] The foreign body intrusion monitoring device 2 is arranged in the target monitoring and protection area, and is used to monitor at least one of the vibration signal, the second image data and the three-dimensional point cloud data of the intruding foreign body in the target monitoring and protection area in the heavy-load railway train running line; the target monitoring and protection area includes the high and steep slope area and the tunnel entrance area; the foreign body intrusion monitoring device 2 can detect whether there are vibration signals caused by human intrusion, rock collapse, landslide and mud-rock flow in the high and steep slope area and the tunnel entrance area, so as to realize long-distance and complex-boundary high and steep slope monitoring and all-round monitoring of the track area, slope protection, guardrail, mountain, etc. near the tunnel entrance;

[0040] The tunnel deformation monitoring device 3 is installed on a heavy-load train and is used to monitor the tunnel profile data about the tunnel in the train running line. By monitoring the tunnel profile data while the train is traveling in the tunnel, it is possible to analyze whether the tunnel is deformed and provide timely feedback to the staff. In addition, the train position monitored by the on-board positioning device 4 is used to facilitate the acquisition of the location information of the tunnel section with problems, thereby facilitating maintenance personnel to perform safety maintenance.

[0041] The vehicle-mounted positioning device 4 is installed on the heavy-load train and is used for real-time monitoring of the position and tracking of the operating vehicle. In this embodiment, the vehicle-mounted positioning device 4 is connected to the tunnel deformation monitoring device 3, and can monitor the position of the heavy-load train while locating the tunnel deformation position monitored by the tunnel deformation monitoring device 3.

[0042] Through the cooperation of the line environment monitoring device 1, the foreign body intrusion monitoring device 2, the tunnel deformation monitoring device 3, the vehicle positioning device 4 and the data management platform 5, it is possible to monitor and track the line dangers such as perimeter intrusion, natural disasters, facility deformation and the position of operating vehicles of the heavy-load railway transportation line in real time, provide data support and technical means for the scientific decision-making of the operation dispatching center, improve the safety of heavy-load railway transportation, the data of each device can be interconnected and combined to form early warning information, and the early warning information is pushed to the dispatching center and station of heavy-load railway transportation in real time, so as to take measures in advance to deal with it and improve the operation safety.

[0043] In some embodiments, the heavy-load railway train operating environment perception system also includes a comprehensive display interface; the comprehensive display interface is used to display the monitoring results of the monitoring data fed back by the data management platform 5 to each device; in this embodiment, the comprehensive display interface is displayed through a display device, and the display device is communicated with the data management platform 5. The specific display device can be a display device that can display images in a dispatching center, a station, or a heavy-load train, thereby providing data monitoring results and related video information to various staff members.

[0044] In some embodiments, Figure 2As shown, the line environment monitoring device 1 includes a plurality of video surveillance cameras 11 arranged along the extension direction of the line and an image storage server 12 connected to the video surveillance cameras 11; the data management platform 5 includes a video analysis processor 51, which is in communication with the image storage server 12 and is used to monitor whether there are foreign objects in the line environment according to the first image data, and when detecting and identifying the presence of foreign objects in the line environment, an alarm message is generated, and the image and / or video information containing the foreign objects is captured from the first image data for display. In this embodiment, the video analysis processor 51 is configured in a one-to-one correspondence with the image storage server 12, and the video analysis processor 51 is usually set near the image storage server 12, such as the machine room of each station, so that the video of the existing video surveillance system can be obtained, and the video of the newly added camera can also be obtained through the optical fiber network.

[0045] The video analysis processor 51 can automatically detect and identify foreign objects in the video based on machine vision and deep learning algorithms, generate alarm information, and push it to the dispatch center and the station in real time. The video output of the surveillance camera is processed by the video analysis processing server. Depending on the camera resolution, each video analysis processing server can realize real-time processing of 10-20 cameras.

[0046] In some embodiments, Figure 3 As shown, the foreign body intrusion monitoring device 2 includes a vibrating optical fiber 21 for monitoring vibration signals and a laser camera 22 for monitoring second image data and three-dimensional point cloud data of intruding foreign bodies; the vibrating optical fiber 21 is laid in the target monitoring protection area; the data management platform 5 includes a foreign body signal processor 52; the foreign body signal processor 52 is connected to the vibrating optical fiber 21 and the laser camera 22, and a fusion recognition algorithm is deployed in the foreign body signal processor 52 to identify whether there are foreign bodies falling or intruding in the target monitoring protection area through the fusion recognition algorithm, and generate alarm information when an abnormality is identified, and simultaneously intercept the image and / or video information containing foreign bodies from the second image data for display. The laser camera 22 is connected to the vibrating optical fiber 21, and when the vibrating optical fiber 21 generates a vibration signal, the laser camera 22 monitors the intruding foreign bodies and forms three-dimensional point cloud data.

[0047] Specifically, in order to adapt to the monitoring of high and steep slope areas and tunnel entrance areas, the foreign object intrusion device includes a high and steep slope monitoring sub-device and a tunnel entrance monitoring sub-device; wherein the high and steep slope monitoring sub-device and the tunnel entrance monitoring sub-device both include a vibration optical fiber 21 and a laser camera 22, so that the high and steep slope monitoring sub-device is used to monitor the clearance safety events along the heavy-load railway operation line; the tunnel entrance monitoring sub-device is used to monitor the clearance safety time at the fence on the top slope of the tunnel and the top of the cutting through which the heavy-load railway operation line passes.

[0048] Furthermore, the data management platform 5 includes several video databases; each of the video databases is used to classify and store the image data of the foreign object intrusion device under different conditions. The data management platform 5 also includes an evaluation algorithm for predicting foreign object intrusion disasters based on each of the video databases, and training and optimizing the evaluation algorithm. The evaluation algorithm is used to combine the vibration signal, the second image data, and the three-dimensional point cloud data of the intruding foreign object to predict whether the train is dangerous when it arrives at the area where the foreign object intrusion monitoring device 2 is located. By establishing video databases under different conditions, it is possible to accumulate samples for more detailed evaluation of perimeter intrusion and disaster alarms in the later stage, and improve the accuracy of monitoring and early warning.

[0049] In this embodiment, the high and steep slope monitoring sub-device adopts the multi-sensor fusion technology based on the vibrating optical fiber 21 and the laser camera 22 to realize the monitoring and alarm of clearance safety events such as perimeter intrusion, landslide, mud-rock flow, collapse and falling rocks along the railway. The tunnel entrance monitoring sub-device is constructed in a complex tunnel entrance scene. Figure 3 The double-layer three-dimensional protection system shown, the outer layer ( Figure 3 (At the middle blue line) a zone-type vibration fiber optic 21+ video intelligent analysis system is installed at the fence on the top slope of the tunnel and the top of the road cutting, focusing on protecting against the impact of large-scale falling rocks and large-scale mud and rock flows, and realizing boundary protection.

[0050] In some embodiments, the tunnel deformation monitoring device 3 includes a laser ranging sensor and a laser scanning sensor installed on the train; the laser ranging sensor is used to measure tunnel profile data; the laser scanning sensor is used to measure rail profile data; the data management platform 5 includes a tunnel section data processor, the tunnel section data processor is used to compensate the tunnel profile data according to the rail profile data to generate complete tunnel section data, and use the tunnel section data to perform deformation and intrusion analysis on the tunnel section information to monitor whether the tunnel is deformed. Considering that the tunnel deformation monitoring device 3 is installed on the train, and the data measured by the laser scanning sensor will produce a large error due to the vibration of the heavy-load train during operation, the rail profile data of the heavy-load train railway is monitored by the laser scanning sensor, thereby realizing data compensation.

[0051] In this embodiment, the tunnel section data collected by the tunnel deformation monitoring device 3 is based on the sensor coordinate system, so it is necessary to convert it to the track coordinate system to facilitate the formation of complete tunnel section data through vibration compensation. Among them, the laser ranging sensor and the laser scanning sensor are located in the same sensor coordinate system, so that the tunnel profile data is compensated by the rail profile data monitored by the laser scanning sensor; an automatic calibration unit is set in the tunnel section data processor, which is used to convert the monitoring data of the laser ranging sensor and the laser scanning sensor into the track coordinate system through calibration, so as to combine the tunnel profile data with the rail profile data to form complete tunnel section data.

[0052] Specifically, the automatic calibration unit stores a track coordinate system coordinate point set and an iterative closest point algorithm; the automatic calibration unit is used to iteratively calculate the track coordinate system coordinate point set and the coordinate point set of measurement data fed back by the laser ranging sensor and the laser scanning sensor through the iterative closest point algorithm to obtain a change matrix of the track coordinate system and the sensor coordinate system, so as to calibrate the laser ranging sensor and the laser scanning sensor through the change matrix.

[0053] Among them, the tunnel deformation monitoring device 3 adopts the laser radar automatic calibration method to improve the detection accuracy. The automatic calibration system uses the track environment where the on-board laser sensor is located as the calibration template, arbitrarily determines a vertical plane perpendicular to the rail as the calibration plane, and uses the total station and laser ink line meter to accurately measure the three-dimensional coordinates of some discrete points in the calibration plane in the measurement reference coordinate system. The measurement results are stored as standard template data as a template file. As long as the calibration environment remains unchanged, the template file can be directly used without repeated measurement, thereby greatly simplifying the workload of manual measurement.

[0054] In some embodiments, the vehicle-mounted positioning device 4 includes a Beidou locator, a speed encoder, and a laser displacement sensor; the Beidou locator is used to locate the heavy-load train through Beidou high-precision positioning, and the speed sensor and the laser displacement sensor are used to measure the speed and locate the train operation; the tunnel deformation monitoring device 3 is connected to the speed encoder and the laser displacement sensor, and is used to determine the position information of the tunnel deformation in combination with the speed measurement and positioning. Considering that the Beidou positioning signal in the tunnel is poor, after entering the tunnel, the position information of the heavy-load train in the tunnel is analyzed by the speed encoder and the laser displacement sensor, and then when the tunnel deformation information is monitored, the position information of the tunnel section with problems is determined in combination with the vehicle-mounted positioning device 4, so as to facilitate the maintenance personnel to perform safety maintenance.

[0055] Embodiment 2:

[0056] This embodiment is a further detailed description of the above-mentioned line environment monitoring device, foreign body intrusion monitoring device and tunnel deformation monitoring device.

[0057] like Figure 7 As shown, further, the video intelligent recognition algorithm in the line environment monitoring device mainly includes three parts: a semantic segmentation model, a zero-sample target detection model, and a reasoner based on a scene model. The input image is first input into the semantic segmentation model for semantic segmentation, and the semantic segmentation result is obtained to divide the scene into various parts according to the target type. At the same time, the image is input into the unknown target detection model, and the unknown target is detected to obtain the detection result, and the location where the unknown target appears is marked. Then the semantic segmentation, unknown target detection and prior knowledge about the scene are input into the reasoner. The reasoner performs reasoning based on the input information to understand the scene.

[0058] In terms of model-based reasoning, unlike the traditional target position reasoning method that only models the two-dimensional relative position, this system establishes a three-dimensional relative position relationship model between the device and the target. In this way, after obtaining the semantic segmentation results of the input image and the unknown category target detection results, the reasoning algorithm can combine the scene model to reason about the scene state based on these two results. First, the perspective of the scene is estimated based on the three-dimensional scene model, and a two-dimensional projection model of the scene model is established; secondly, the semantic segmentation results are matched and reasoned with the two-dimensional projection model to determine the known devices; then the unknown target detection results are matched and reasoned with the semantic decomposition results and the two-dimensional projection model after the known devices are eliminated to determine the position of the unknown target; finally, through the reasoning results of the first two steps, the state of the scene and whether there are threatening abnormal situations are judged.

[0059] Furthermore, for the foreign body intrusion monitoring device, the multi-sensor fusion technology roadmap based on vibrating optical fiber and laser camera is as follows: Figure 4 As shown in the figure, a positioning vibration optical fiber is laid in a long and steep slope area. The laser emitted by the laser passes through the optical path and enters the on-site optical cable. When natural disasters such as mudslides or falling rocks occur, the optical cable will feel the vibration of the external environment. Some lasers with on-site vibration information return along the optical cable, enter the photoelectric detection part through the optical path, and are converted into electrical signals. The electrical signals are collected and analyzed to generate alarms. According to the alarm position, the laser camera configured nearby can be linked to collect video information of the alarm position in real time. After the collection is completed, the video data is transmitted to the data management platform through 4G transmission for analysis and processing. Finally, the processed alarm video information is transmitted to the general monitoring room connected to the data management platform to observe the alarm information.

[0060] In this embodiment, the tunnel entrance monitoring subsystem is constructed in a complex tunnel entrance scenario as follows Figure 5The double-layer three-dimensional protection system shown, the outer layer ( Figure 5 (The middle blue line) is set up at the fence on the top slope of the tunnel and the top of the cutting to focus on protecting against the impact of large-scale rockfalls and large-scale debris flows, and to achieve boundary protection; within the railway limit ( Figure 5 In the red track area, a laser radar + video intelligent analysis fusion monitoring system is constructed as the second-level three-dimensional protection monitoring network to conduct real-time monitoring and alarm on foreign objects that invade the railway limit and are higher than the track surface, realize spatial protection and ensure safety.

[0061] For tunnel entrances and deep cuttings, on the one hand, we must guard against mudslides and falling rocks from the mountain, and on the other hand, we must guard against people and large animals invading the railway limits. Unlike people and large animals, which are generally in a state of continuous movement, mudslides and falling rocks are characterized by randomness and suddenness. They are moving at the moment of invading the railway limits, but are generally stationary after invading. When they are above the track surface, they will threaten the safety of high-speed train operation. At present, high-speed railway tunnel entrances are mainly protected by physical means. Rock dams and rock nets are generally used above the tunnel entrance to prevent mudslides and falling rocks. Fences are generally used above the tunnel and the perimeter of the cutting section to protect against the intrusion of people and large animals, but they do not have active monitoring and alarm functions. Therefore, for tunnel entrances with relatively complex surrounding environments, the fusion technology of zone-type vibration optical fiber and multiple video surveillance cameras is used to achieve all-round monitoring of the track area, slope protection, guardrails, mountains, etc. near the tunnel entrance. Among them, the vibration optical fiber is laid along the upper mountain, the slopes on both sides, the guardrails and other locations around the tunnel entrance, and virtual defense zones are set at certain intervals. When the interferometer detects vibration signals caused by human intrusion, rock collapse, landslide and mud-rock flow near the optical fiber, the signal processing algorithm outputs the alarm information and the defense zone number where the event occurred, and links the video surveillance camera for secondary identification.

[0062] By using LiDAR and camera to fuse information, more reliable targets can be obtained compared to monitoring with a single camera, so as to make accurate decisions for the system and improve the effectiveness of the system. The designed fusion detection system architecture is as follows: Figure 8 As shown, the on-site 3D lidar and camera are connected to the edge computing processor through a switch, and the fusion recognition algorithm is deployed on the edge processing device. When foreign object intrusion is detected, alarm information is transmitted to the monitoring platform through the network.

[0063] Further, for the tunnel deformation monitoring device, this embodiment specifically provides a tunnel deformation monitoring device, and the overall design scheme block diagram is as follows: Figure 6As shown in the figure, the hardware of the system consists of 2 two-dimensional laser ranging sensors LMS511 and 4 two-dimensional laser scanning sensors ZLDS200, 1 speed sensor, 1 laser displacement sensor and a data acquisition and processing system, which mainly realizes the functions of tunnel profile acquisition, rail inner profile acquisition, speed measurement and positioning and data transmission. Among them, the vehicle posture is used to compensate for the tunnel size data. Since the vehicle vibration causes a large error in the measured data, the vehicle posture must be monitored to achieve data compensation. The tunnel section data collected by the system is based on the sensor coordinate system. Through calibration, the data is converted to the track coordinate system, so that the data obtained by the two laser ranging sensors together constitute the complete tunnel data. The deformation and intrusion analysis can obtain the ellipticity value of the tunnel section and the coordinates of the intrusion point, and obtain the location information of the problematic tunnel section through positioning, which is convenient for maintenance personnel to perform safety maintenance.

[0064] Furthermore, the calibration process includes:

[0065] Align the scanning plane of the vehicle-mounted laser scanning sensor with the calibration plane. The automatic calibration software system directly reads the measurement data of the laser scanning sensor from the acquisition board through the data interface. The two-dimensional coordinates of the points on the calibration plane in the sensor coordinate system can be obtained through angle calculation, and the third-dimensional coordinate is set to zero.

[0066] The iterative closest point algorithm integrated in the calibration software uses the coordinate point set stored in the template file and the coordinate point set calculated based on the laser sensor measurement data for iterative calculation, and finally obtains the transformation matrix between the two to realize the calibration of the laser scanning sensor.

[0067] System calibration requires obtaining the coordinates of discrete points in the calibration plane in the measurement reference coordinate system, as well as the coordinate points of the laser scanning sensor scanning the calibration plane. The former is measured using a total station and a laser ink line meter. The measurement reference coordinate system is defined as the direction perpendicular to the track centerline in the track top plane as the X-axis, the track centerline in the track top plane as the Y-axis, and the vertical direction as the Z-axis. In order to measure points in the calibration plane, the three-dimensional coordinate system of the total station needs to be overlapped with the measurement reference coordinate system. The laser ink line meter can assist the total station in setting the coordinate system.

[0068] The laser ink line meter can emit four vertical laser lines and form a cross at the zenith. Adjust the position and angle of the laser ink line meter so that the laser line in one direction coincides with the calibration plane, and the laser line in the other direction coincides with the inner edge of the top of a rail. Place the tripod with the total station across the rail with the longitudinal laser line, hang the plumb bob at the bottom center of the total station, adjust the position and angle of the tripod so that the total station remains horizontal and the plumb line determined by the plumb bob intersects the longitudinal laser line, that is, the center of the total station falls on the longitudinal laser line. Turn on the total station and select its three-coordinate measurement function. Set the total station's east coordinate (i.e., Y coordinate) to coincide with the longitudinal laser line, set the direction of the total station's rotation of 90 degrees to the north coordinate (i.e., X coordinate), and the Z coordinate to the direction perpendicular to the two coordinate axes. Since the center line of the track is not easy to establish, the inner edge of the top of the rail is used as the Y axis. After measuring the calibration point using the total station, the X coordinate value of the calibration point needs to be compensated for the corresponding translation.

[0069] After the coordinates of the total station are set, point measurement can be performed. In the calibration plane marked by the horizontal laser line, a point is selected at a certain interval to measure and record its three-dimensional coordinates. After the measurement is completed, the X coordinates of all points are translated as the final measurement result.

[0070] The measurement of the calibration plane by the vehicle-mounted laser sensor requires the alignment of the laser sensor's scanning plane with the calibration plane. The laser emitted by the laser sensor is invisible light, so a laser detector is used to detect and determine the location of the laser scanning plane. The laser scanning plane is aligned with the calibration plane by adjusting the position of the vehicle equipped with the laser. The data of the laser sensor is directly read from the acquisition board through the data interface.

[0071] It can be seen from the above data collection process that in the process of calibration using the automatic calibration system, there is no need to specifically measure the coordinates of the same points in the measurement reference coordinate system and the sensor coordinate system and make a one-to-one correspondence. Subsequent data processing will automatically match the measured points. The iterative closest point algorithm is proposed on the basis of the method of calculating the transformation matrix between point sets. It can accurately calculate the transformation matrix between two point sets through multiple iterations. Given a model point set and a measurement point set, the purpose of the iterative closest point calculation is to obtain the optimal transformation matrix that transforms the measurement point set to the model point set. The iterative closest point algorithm searches for the point closest to a single measurement point from the model point set as its corresponding point, and searches for its corresponding point for each measurement point in turn to form the closest point set. The closest point set corresponds to the points in the measurement point set one-to-one, so the transformation matrix from the closest point set to the measurement point set can be directly calculated by the three-dimensional rectangular coordinate transformation method, and the measurement point set is transformed according to the matrix to obtain a new point set and the above process is repeated until the change in the mean square error of the distance between the measurement point set and the model point set is less than a certain value.

[0072] Among them, given the model point set and the measurement point set in the above data collection atmosphere, the purpose of the iterative closest point algorithm is to transform the measurement point set to the optimal transformation matrix of the model point set. Specifically, the iterative closest point algorithm will search the point closest to the single measurement point from the model point set as its corresponding point, and search the corresponding point of each measurement point in turn to form the closest point set. The closest point set corresponds to the points in the measurement point set one by one, so the transformation matrix from the closest point set to the measurement point set can be directly calculated through the three-dimensional rectangular coordinate transformation method, and the measurement point set is transformed according to the matrix to obtain a new point set and the above process is repeated until the change in the mean square error of the distance between the measurement point set and the model point set is less than a certain value.

[0073] In the process of iterative nearest point calculation, it is necessary to search for the point closest to the measured point. This process can be implemented by linear search, space segmentation, and position-sensitive hash query, which will not be introduced in detail here.

[0074] It should be understood that the expressions "mechanism", "device", "component" and the like used in this application are only a method for distinguishing different components, elements, parts, portions or assemblies at different levels. However, if other words can achieve the same purpose, the word can be replaced by other expressions.

[0075] Those skilled in the art can understand that the above-mentioned embodiments are specific examples for realizing the present invention, and in practical applications, the technical features of the above-mentioned embodiments can be arbitrarily combined. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification, and various changes can be made in form and details without departing from the spirit and scope of the present invention.

Claims

1. A heavy-load railway train operating environment perception system, characterized in that: include: A line environment monitoring device, installed in the extension direction of the line, for monitoring first image data of the train line environment; A foreign body intrusion monitoring device is arranged in a target monitoring and protection area, and is used to monitor at least one of a vibration signal, a second image data, and three-dimensional point cloud data of intruding foreign bodies in the target monitoring and protection area in the heavy-load railway train running line; the target monitoring and protection area includes a high and steep slope area and a tunnel entrance area; A tunnel deformation monitoring device, installed on a heavy-load train, for monitoring tunnel profile data about a tunnel in a train running line; On-board positioning device, installed on heavy-load trains, used for real-time monitoring of operating vehicle positions and tracking; And a data management platform, the data management platform is connected to the line environment monitoring device, the foreign object intrusion monitoring device, the tunnel deformation monitoring device and the vehicle-mounted positioning device, and the data management platform is used to analyze and process the monitoring data fed back by each device to realize the train operation safety status monitoring.

2. A heavy-load railway train operating environment perception system according to claim 1, characterized in that: The line environment monitoring device includes a plurality of video surveillance cameras arranged along the extension direction of the line and an image storage server connected to the video surveillance cameras; the data management platform includes a video analysis processor, which is communicatively connected to the image storage server and is used to monitor whether there are foreign objects in the line environment according to the first image data. When foreign objects are detected and identified in the line environment, an alarm message is generated, and images and / or video information containing foreign objects are simultaneously captured from the first image data for display.

3. A heavy-load railway train operating environment perception system according to claim 1, characterized in that: The foreign object intrusion monitoring device includes a vibration optical fiber for monitoring vibration signals and a laser camera for monitoring second image data and three-dimensional point cloud data of invading foreign objects; the vibration optical fiber is laid in the target monitoring and protection area; the data management platform includes a foreign object signal processor; the foreign object signal processor is connected to the vibration optical fiber and the laser camera, and a fusion recognition algorithm is deployed in the foreign object signal processor to identify whether there is a foreign object falling or intruding in the target monitoring and protection area through the fusion recognition algorithm, and generate an alarm message when an abnormality is identified, and simultaneously capture the image and / or video information containing the foreign object from the second image data for display.

4. A heavy-load railway train operating environment perception system according to claim 3, characterized in that: The foreign object intrusion device includes a high and steep slope monitoring sub-device and a tunnel entrance monitoring sub-device; the high and steep slope monitoring sub-device is used to monitor the clearance safety events along the heavy-load railway operation line; the tunnel entrance monitoring sub-device is used to monitor the clearance safety time at the top slope of the tunnel and the fence at the top of the road cutting through which the heavy-load railway operation line passes.

5. A heavy-load railway train operating environment perception system according to claim 3, characterized in that: The data management platform includes several video databases; each of the video databases is used to classify and store image data of the foreign object intrusion device under different conditions. The data management platform also includes an evaluation algorithm for predicting foreign object intrusion disasters based on each of the video databases, and training and optimizing the evaluation algorithm. The evaluation algorithm is used to combine the vibration signal, the second image data and the three-dimensional point cloud data of the invading foreign object to predict whether it is dangerous when the train arrives at the area where the foreign object intrusion monitoring device is located.

6. A heavy-load railway train operating environment perception system according to claim 1, characterized in that: The tunnel deformation monitoring device includes a laser ranging sensor and a laser scanning sensor installed on a train; the laser ranging sensor is used to measure tunnel profile data; the laser scanning sensor is used to measure rail profile data; the data management platform includes a tunnel section data processor, and the tunnel section data processor is used to compensate the tunnel profile data according to the rail profile data to generate complete tunnel section data, and use the tunnel section data to perform deformation and intrusion analysis on the tunnel section information to monitor whether the tunnel is deformed.

7. A heavy-load railway train operating environment perception system according to claim 6, characterized in that: The vehicle-mounted positioning device includes a Beidou locator, a speed encoder and a laser displacement sensor; the Beidou locator is used to locate heavy-load trains through Beidou high-precision positioning, and the speed sensor and the laser displacement sensor are used to measure the speed and locate the train operation; the tunnel deformation monitoring device is connected to the speed encoder and the laser displacement sensor, and is used to determine the position information of the tunnel deformation in combination with the toilet positioning.

8. A heavy-load railway train operating environment perception system according to claim 6, characterized in that: The laser ranging sensor and the laser scanning sensor are located in the same sensor coordinate system, so that the tunnel profile data is compensated by the rail profile data monitored by the laser scanning sensor; an automatic calibration unit is set in the tunnel section data processor, which is used to convert the monitoring data of the laser ranging sensor and the laser scanning sensor into the track coordinate system through calibration, so as to combine the tunnel profile data with the rail profile data to form complete tunnel section data.

9. A heavy-load railway train operating environment perception system according to claim 8, characterized in that: The automatic calibration unit stores a track coordinate system coordinate point set and an iterative closest point algorithm; the automatic calibration unit is used to iteratively calculate the track coordinate system coordinate point set and the coordinate point set of measurement data fed back by the laser ranging sensor and the laser scanning sensor through the iterative closest point algorithm to obtain a change matrix of the track coordinate system and the sensor coordinate system, so as to calibrate the laser ranging sensor and the laser scanning sensor through the change matrix.

10. A heavy-load railway train operating environment perception system according to any one of claims 1 to 9, characterized in that: It also includes a comprehensive display interface; the comprehensive display interface is used to display the monitoring results of the monitoring data fed back by the data management platform from each device.

Images