Safety state monitoring system for heavy haul railway tunnel portal

Abstract

The embodiment of the invention relates to the technical field of railway safety, and discloses a heavy haul railway tunnel portal safety state monitoring system, which comprises an optical fiber monitoring module, a video monitoring module, a radar monitoring module and a processor, and is characterized in that the optical fiber monitoring module, the video monitoring module and the radar monitoring module are respectively connected with the processor; the optical fiber monitoring module adopts a defense area type vibration optical fiber and is used for acquiring an optical signal of a first monitoring protection area in real time; the video monitoring module comprises a plurality of video monitoring cameras, and the plurality of video monitoring cameras are arranged along the heavy haul railway, are associated with each defense area and are used for acquiring image data; the radar monitoring module comprises at least one laser radar and is used for acquiring three-dimensional point cloud data of a second monitoring protection area; and the processor is used for realizing safety state monitoring according to the optical signal, the image data and the three-dimensional point cloud data. Optical signals, image data and three-dimensional point cloud data are fused, real-time monitoring and alarming of the heavy haul railway tunnel portal are achieved, and railway operation safety is improved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of railway safety technology, and in particular to a heavy-load railway tunnel entrance safety status monitoring system. Background Art

[0002] Heavy-haul railway refers to railways with heavy trains, heavy axle-load trucks, or extremely high traffic density and volume. When heavy-haul trains run on heavy-haul railways, the impact on the track will cause the vibration of the vehicle track structure, which will be transmitted to the dangerous rocks above the tunnel entrance through the track and the surrounding rock of the tunnel. There are often super-high and steep slopes above the tunnel entrance, which are often at risk of falling dangerous rocks. In this environment, the vibration generated by the train operation will cause vibration and instability of the super-high and steep dangerous rock mass above the tunnel entrance, which will threaten the smooth operation of the train and endanger transportation safety. The instability of the super-high and steep dangerous rock mass above the tunnel entrance is one of the main types of geological disasters, and the relative height of the super-high and steep dangerous rock mass is more than 300m. The dangerous rock has a large height difference, steep slope, large kinetic energy, and strong destructive power. Under the long-term vibration excitation of the train, it will accelerate, catalyze and induce the instability of the dangerous rock. If the dangerous rock falls and collides with the high-speed train, it will have serious consequences. Therefore, it is very necessary to monitor the safety status of the tunnel entrance of heavy-haul railways and issue early warnings.

[0003] As an advanced monitoring technology, fiber optic sensing technology has been applied to the monitoring and early warning of geological disasters. At present, fiber optic technology is most commonly used in the monitoring of foreign body intrusion of dangerous rockfalls. The optical fiber is fixed on the mountain within the monitored range with a flexible protective net as a carrier. The fiber optic Bragg grating sensor is fixed in the middle of the monitored range through the optical fiber. The monitoring and control system determines whether there is rockfall intrusion based on the received vibration signal. However, the fiber optic Bragg grating technology supported by the flexible protective net is more of a post-disaster early warning. It cannot be used to predict before dangerous rockfalls occur, and it only uses vibration indicators for monitoring. It can only determine the presence of foreign objects, but cannot accurately determine whether it is a rockfall, and there is a high possibility of false alarms. Summary of the invention

[0004] The purpose of the present invention is to at least provide a heavy-load railway tunnel entrance safety status monitoring system to solve the problem of single existing monitoring mode and high false alarm rate.

[0005] To solve the above technical problems, at least one embodiment of the present application provides a heavy-duty railway tunnel entrance safety status monitoring system, comprising: an optical fiber monitoring module, a video monitoring module, a radar monitoring module and a processor, wherein the optical fiber monitoring module, the video monitoring module and the radar monitoring module are respectively connected to the processor;

[0006] The optical fiber monitoring module adopts a zone-type vibration optical fiber, which is laid in the first monitoring and protection area, and the first monitoring and protection area where the zone-type vibration optical fiber is laid is divided into multiple zones according to preset distance intervals, so as to obtain the optical signal of the first monitoring and protection area in real time;

[0007] The video monitoring module includes a plurality of video surveillance cameras, which are installed along the heavy-load railway and associated with each defense zone to obtain image data;

[0008] The radar monitoring module includes at least one laser radar, which is used to obtain three-dimensional point cloud data of the second monitoring and protection area;

[0009] The processor is used to realize safety status monitoring based on light signals, image data and three-dimensional point cloud data.

[0010] In some embodiments, the processor is used to implement safety status monitoring based on the light signal, the image data, and the three-dimensional point cloud data, including:

[0011] Determine the category and rough location information of the intrusion target based on the optical signal and image data;

[0012] Determine the size and precise location information of the intrusion target based on image data and 3D point cloud data.

[0013] In some embodiments, determining the category and rough location information of the intrusion target based on the light signal and the image data includes:

[0014] Determine an abnormal event and the defense zone number where the abnormal event is located according to the change of the optical signal of the first monitoring protection area obtained in real time, wherein the abnormal event includes human intrusion, animal intrusion, rock collapse, landslide and mud-rock flow;

[0015] According to the defense zone number where the abnormal event is located, the video surveillance camera associated with the defense zone is controlled to obtain image data of the defense zone where the abnormal event is located;

[0016] Analyze the image data of the defense zone where the abnormal event occurs to obtain the category and rough location information of the intrusion target.

[0017] In some embodiments, the processor is further configured to generate alarm information based on the abnormal event, the zone number where the abnormal event is located, and the category and rough location information of the intrusion target.

[0018] In some embodiments, determining the size and precise location information of the intrusion target based on the image data and the three-dimensional point cloud data includes:

[0019] Perform two-dimensional target detection on the image data to obtain the position of the intrusion target in the two-dimensional image and the two-dimensional target detection result;

[0020] The three-dimensional point cloud data is intercepted according to the position of the intrusion target in the two-dimensional image to obtain the three-dimensional point cloud data of the area of ​​interest;

[0021] Perform 3D target detection on the 3D point cloud data of the area of ​​interest to obtain the 3D target detection result;

[0022] The two-dimensional target detection results and the three-dimensional target detection results are weighted and fused to obtain the size and precise location information of the intrusion target.

[0023] In some embodiments, before intercepting the three-dimensional point cloud data according to the position of the intrusion target in the two-dimensional image, the processor is further configured to:

[0024] The three-dimensional point cloud data is downsampled using a voxel filter to obtain the denoised three-dimensional point cloud data;

[0025] The denoised 3D point cloud data is converted to the world coordinate system and the track surface is segmented.

[0026] In some embodiments, performing 3D object detection on 3D point cloud data of a region of interest includes:

[0027] A pre-trained 3D object detection network is used to perform 3D object detection on the 3D point cloud data of the area of ​​interest.

[0028] In some embodiments, the video surveillance camera is a laser night vision camera.

[0029] In some embodiments, the first monitoring and protection area includes the area above the tunnel entrance, the top area of ​​the high and steep slope, the slope areas on both sides of the track, and the guardrail area.

[0030] In some embodiments, the second monitoring and protection area includes an area within the railway boundary.

[0031] The embodiment of the present application provides a heavy-duty railway tunnel entrance safety status monitoring system, including: an optical fiber monitoring module, a video monitoring module, a radar monitoring module and a processor, wherein the optical fiber monitoring module, the video monitoring module and the radar monitoring module are connected to the processor respectively; the optical fiber monitoring module adopts a zone-type vibration optical fiber, which is laid in a first monitoring and protection area, and the first monitoring and protection area where the zone-type vibration optical fiber is laid is divided into a plurality of zones according to a preset distance interval, so as to obtain the optical signal of the first monitoring and protection area in real time; the video monitoring module includes a plurality of video monitoring cameras, which are installed along the heavy-duty railway and associated with each zone, so as to obtain image data; the radar monitoring module includes at least one laser radar, which is used to obtain the three-dimensional point cloud data of the second monitoring and protection area; the processor is used to realize safety status monitoring according to the optical signal, the image data and the three-dimensional point cloud data. The optical signal, the image data and the three-dimensional point cloud data are integrated to realize the real-time monitoring and alarm of the heavy-duty railway tunnel entrance, reduce the false alarm rate and improve the railway operation safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] One or more embodiments are exemplarily described by the pictures in the corresponding drawings, and these exemplary descriptions do not constitute limitations on the embodiments.

[0033] Figure 1 It is a structural schematic diagram of a heavy-load railway tunnel entrance safety status monitoring system provided by an embodiment of the present application;

[0034] Figure 2 is a schematic diagram of a monitoring and protection area provided by an embodiment of the present application;

[0035] Figure 3 It is a schematic diagram of light-visual linkage provided by an embodiment of the present application. DETAILED DESCRIPTION

[0036] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the embodiments of the present application will be described in detail below in conjunction with the accompanying drawings. However, it will be appreciated by those skilled in the art that in the present application, many technical details are proposed in order to enable the reader to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical scheme claimed in the present application can also be implemented. The division of the following embodiments is for the convenience of description, and the specific implementation of the present application should not be construed as any limitation, and the various embodiments can be combined and referenced with each other under the premise of no contradiction.

[0037] The environment around the entrance of a heavy-duty railway tunnel is complex, and a single protective measure is difficult to deal with a wide variety of intruders. Therefore, it is necessary to integrate and process data from multiple different types of sensors to obtain more accurate, reliable and comprehensive target information. Different types of sensors (such as image, sound, radar, lidar, vibration sensor, etc.) can obtain different characteristics of the target or provide information in different aspects. By integrating and analyzing the data from these different sensors, more comprehensive and accurate target information can be obtained, thereby achieving efficient identification, tracking, positioning and prediction of the target, and thus completing the monitoring of the safety status of the entrance of a heavy-duty railway tunnel.

[0038] In order to solve the problem of single existing monitoring methods and high false alarm rate, this application adopts a multi-level fusion technology of zone-type vibration optical fiber, multiple video surveillance cameras and a single laser radar for the tunnel entrance and its associated areas, realizing all-round monitoring of the track area, slope protection, guardrail and mountain near the tunnel entrance. The vibration optical fiber is laid along the upper mountain, slopes on both sides, guardrails and other locations around the tunnel entrance, and virtual defense zones are set at certain intervals. When the interferometer detects vibration signals caused by human intrusion, rock collapse, landslide and mudslide near the vibration optical fiber, the signal processing algorithm outputs the alarm information and the specific defense zone number where the event occurs, and the video surveillance camera is linked for secondary identification. When the intrusion target invades the track area, the laser radar can further output the three-dimensional size and position of the target. Through the optical fiber and laser vision fusion system, a two-layer stereo real-time monitoring and protection system for the tunnel entrance status is constructed to ensure the safety of line operation.

[0039] The first-layer monitoring and protection area in the double-layer three-dimensional real-time monitoring and protection system is monitored by laying zone-type vibration optical fibers in the area above the tunnel entrance and the top area of ​​the steep slope, linking the monitoring cameras installed along the railway, and judging the intrusion behavior through a detection method based on decision-making fusion, focusing on protecting against the impact of large-scale rockfalls and large-scale mudslides to achieve boundary protection; the second-layer monitoring and protection area in the double-layer three-dimensional real-time monitoring and protection system is mainly for the area within the railway limit. The on-site data is collected through the combination of laser radar and monitoring camera, and the detection method based on feature-level fusion is used to carry out real-time monitoring and alarm for foreign objects that invade the railway limit and are above the track surface, so as to achieve spatial protection and ensure safety.

[0040] The implementation details of the heavy-duty railway tunnel entrance safety status monitoring system provided by this application will be specifically described below through specific embodiments. The following content is only provided to facilitate understanding of the implementation details and is not necessary for the implementation of this solution.

[0041] Embodiment 1:

[0042] Figure 1FIG. 1 is a schematic diagram of a heavy-duty railway tunnel entrance safety status monitoring system provided by an embodiment of the present application. Figure 1 As shown, the heavy-duty railway tunnel entrance safety status monitoring system provided in this embodiment may include: an optical fiber monitoring module 101, a video monitoring module 102, a radar monitoring module 103 and a processor 104, and the optical fiber monitoring module 101, the video monitoring module 102 and the radar monitoring module 103 are respectively connected to the processor 104.

[0043] The optical fiber monitoring module 101 adopts a zone-type vibration optical fiber, which is laid in the first monitoring and protection area, and the first monitoring and protection area laid with the zone-type vibration optical fiber is divided into multiple zones according to preset distance intervals, so as to obtain the optical signal of the first monitoring and protection area in real time; the video monitoring module 102 includes multiple video surveillance cameras, which are installed along the heavy-load railway and associated with each defense zone, so as to obtain image data; the radar monitoring module 103 includes at least one laser radar, which is used to obtain three-dimensional point cloud data of the second monitoring and protection area; the processor 104 is used to realize safety status monitoring according to optical signals, image data and three-dimensional point cloud data.

[0044] In this embodiment, a zone-type vibration optical fiber is used, which is much cheaper than a positioning-type vibration optical fiber. In addition, in this embodiment, the entire monitoring area is divided into more than a dozen defense zones, which can meet the positioning requirements of the safety status monitoring of the heavy-duty railway tunnel entrance. In this embodiment, the first monitoring and protection area includes the area above the tunnel entrance, the top area of ​​the high and steep slope, the slope area on both sides of the track, and the guardrail area, which realizes the extension of the boundary of the monitoring area and realizes the monitoring of dangerous situations such as intrusion around the tunnel entrance, collapse and falling rocks, landslides and mudslides.

[0045] The video surveillance camera in this embodiment uses a laser night vision camera, and matches the effective field of view of the vibration optical fiber in each defense zone. It adopts a binocular design with near and far focus. The near focus can see wide, and the far focus can see far and clearly, truly realizing real-time monitoring of complex tunnel entrance areas.

[0046] In this embodiment, the second monitoring and protection area includes the area within the railway limit. Compared with the pure visible light video analysis, the laser radar can obtain the specific location, shape and size information of the intruding foreign object through the three-dimensional point cloud, and then provide three-dimensional data information for the detection system.

[0047] For a schematic diagram of the monitoring and protection area, please refer to Figure 2 shown. Figure 2 The middle red line marks the first monitoring and protection area. The monitoring method is to lay zone-type vibration optical fibers in the area above the tunnel entrance and the top area of ​​the steep slope, and link them with the surveillance cameras installed along the railway. The detection method based on decision-making fusion is used to judge the intrusion behavior, focusing on protecting against the impact of large-scale rockfalls and large-scale debris flows, and realizing boundary protection. Figure 2 The middle yellow line marks the second monitoring and protection area, which is mainly aimed at the area within the railway limit. The laser radar and surveillance camera combination machine collects data on site, and adopts a detection method based on feature-level fusion to monitor and alarm foreign objects that invade the railway limit and are higher than the track surface in real time, so as to achieve spatial protection and ensure safety. The specific protection process includes: when the interferometer detects vibration signals caused by human intrusion, rock collapse, landslide, mudslide, etc. near the optical fiber, the signal processing algorithm outputs the alarm information and the defense zone number where the event occurs, and the video surveillance camera is linked for secondary identification. When the intrusion target invades the track area, the laser radar can further output the three-dimensional size and position of the target.

[0048] The heavy-duty railway tunnel entrance safety status monitoring system provided in this embodiment realizes real-time monitoring of the safety status of the tunnel entrance by integrating optical fiber, video and radar, thereby ensuring the safety of line operation. Specifically, the heavy-duty railway tunnel entrance safety status monitoring system provided in this embodiment may include: an optical fiber monitoring module, a video monitoring module, a radar monitoring module and a processor, wherein the optical fiber monitoring module, the video monitoring module and the radar monitoring module are respectively connected to the processor; the optical fiber monitoring module adopts a zone-type vibration optical fiber, which is laid in the first monitoring and protection area, and the first monitoring and protection area where the zone-type vibration optical fiber is laid is divided into multiple zones according to a preset distance interval, so as to obtain the optical signal of the first monitoring and protection area in real time; the video monitoring module includes multiple video monitoring cameras, which are installed along the heavy-duty railway and associated with each zone, so as to obtain image data; the radar monitoring module includes at least one laser radar, which is used to obtain the three-dimensional point cloud data of the second monitoring and protection area; the processor is used to realize safety status monitoring according to the optical signal, image data and three-dimensional point cloud data. By integrating optical signals, image data and 3D point cloud data, it realizes real-time monitoring and alarm of heavy-load railway tunnel entrances, reduces false alarm rates and improves railway operation safety.

[0049] Embodiment 2:

[0050] The heavy-duty railway tunnel entrance safety status monitoring system provided in this embodiment adopts a two-layer monitoring and protection strategy to ensure the safety of line operation. The first-layer monitoring and protection area monitoring method is to lay zone-type vibration optical fibers in the area above the tunnel entrance and the top area of ​​the steep slope, and link the monitoring cameras installed along the railway. The intrusion behavior is judged by the detection method based on decision-making fusion, focusing on protecting the impact of large-scale falling rocks and large-scale mudslides to achieve boundary protection; the second-layer detection and protection area is mainly aimed at the area within the railway limit. The laser radar and monitoring camera combination machine are used to collect data on the site, and the detection method based on feature-level fusion is used to monitor and alarm foreign objects that invade the railway limit and are higher than the rail surface in real time, so as to achieve spatial protection and ensure safety. In other words, the processor is used to realize safety status monitoring based on optical signals, image data and three-dimensional point cloud data, including: determining the category and rough location information of the intrusion target based on optical signals and image data; determining the size and precise location information of the intrusion target based on image data and three-dimensional point cloud data.

[0051] Specifically, the first-layer monitoring and protection area adopts a multi-zone optical sensing perimeter security alarm system, which receives information from the zone processing unit through the communication optical cable, monitors the changes in the optical signals of each zone in real time, and transmits the collected data to the alarm host. This process is carried out continuously to ensure real-time perception of the surrounding environment. The alarm host uses classification algorithms to process and analyze the data collected by the sensor. These algorithms can identify different types of events, such as human intrusion, animal crossing, vehicle passing, etc., and exclude some conventional false alarms. The host system identifies abnormal events by setting thresholds and analyzing the data. At the same time, the laser night vision camera that matches the effective field of view of the vibration fiber optic security alarm system adopts a binocular design with near and far focus. The near focus can see wide, and the far focus can see far and clear, truly realizing real-time monitoring of complex tunnel entrance areas. After the vibration fiber output is abnormal, the image of the abnormal zone is captured by the laser night vision camera, and the category and location information of the intrusion target are obtained from the image by processing the pictures collected by the monitoring camera.

[0052] The second-level detection and protection area uses laser radar and image sensor fusion monitoring, using the advantages of image and point cloud data to perform feature-level and decision-level fusion, thereby improving the final detection performance. Compared with pure visible light video analysis, laser radar can obtain the specific location, shape and size information of intruding foreign objects through three-dimensional point clouds, and then provide three-dimensional data information for the detection system. This solution can achieve cross-modal data fusion, reduce the impact of harsh conditions such as dark light, strong light, and haze at night on the detection results, and accurately analyze the size and location of intruding foreign objects.

[0053] Embodiment three:

[0054] The optical-visual linkage perimeter security technology combines the two technologies of fiber optic sensing and video intelligent sensing to perform perimeter security detection. This technical solution uses fiber optic sensing to perform all-weather, high-sensitivity perimeter intrusion detection, and combines the advantages of high resolution and visualization of cameras to perform video perimeter detection and capture evidence. The detection results of the two technologies can be further analyzed for reference, reducing missed reports and false alarms in perimeter detection and achieving a high level of perimeter protection.

[0055] In the heavy-duty railway tunnel entrance safety status monitoring system provided in this embodiment, the category and rough location information of the intrusion target are determined based on the optical signal and image data, which may specifically include: determining the abnormal event and the defense zone number where the abnormal event is located based on the change of the optical signal of the first monitoring and protection area obtained in real time, and the abnormal event includes human intrusion, animal intrusion, collapse and rockfall, landslide and mud-rock flow; according to the defense zone number where the abnormal event is located, controlling the video surveillance camera associated with the defense zone to obtain the image data of the defense zone where the abnormal event is located; analyzing the image data of the defense zone where the abnormal event is located to obtain the category and rough location information of the intrusion target. In some optional implementations, the processor is also used to generate alarm information based on the abnormal event, the defense zone number where the abnormal event is located, and the category and rough location information of the intrusion target.

[0056] The fusion technology of zone-type vibration optical fiber and video surveillance camera has significant advantages in the security protection of railway tunnel entrances. By timely and accurately monitoring and identifying potential threats, it provides strong support for the safe operation of trains. The multi-zone optical sensing perimeter security alarm system receives information from the zone processing unit through the communication optical cable, monitors the changes in the optical signals of each zone in real time, and transmits the collected data to the alarm host. This process is carried out continuously to ensure real-time perception of the surrounding environment. The alarm host uses classification algorithms to process and analyze the data collected by the sensor. These algorithms can identify different types of events, such as human intrusion, animal crossing, passing vehicle activities, etc., and exclude some conventional false alarms. The host system identifies abnormal events by setting thresholds and analyzing the data.

[0057] Optical-visual linkage is a technology that combines fiber optic sensing and video intelligent sensing to detect perimeter security. Fiber optic sensing is used for all-weather, high-sensitivity perimeter intrusion detection. At the same time, the advantages of high resolution and visualization of cameras are combined to perform video perimeter detection and capture evidence. The detection results of the two technologies can be further analyzed for reference, reducing missed reports and false alarms in perimeter detection and achieving a high level of perimeter protection. The overall solution is as follows: Figure 3As shown. The optical-visual linkage technology has four obvious characteristics: First, both fiber optic sensing and video sensing are used for perimeter protection; second, there is a temporal and spatial correlation between fiber optic sensing and video sensing, and detection can be performed on the same intrusion target or detection area; third, the optical fiber can drive the video for linkage capture, tracking and identification; fourth, the optical-visual linkage detection results can make full use of multi-dimensional sensing technology to make the best judgment. The specific execution process is as follows:

[0058] ① The optical fiber sensing perimeter security alarm system performs real-time detection. When an intrusion event occurs, the intrusion is detected through optical fiber, the pipeline sensing performs event identification, and the alarm information is uploaded;

[0059] ②Control the event-related surveillance camera to rotate to the corresponding preset position, and perform video detection, capture, and tracking of the alarm zone;

[0060] ③ The industrial computer uses the video data uploaded by the surveillance camera to detect and identify intrusion targets through deep learning algorithms. The recognition results include pictures, target categories, etc.

[0061] ④ The industrial control host uses the built-in event engine to conduct a comprehensive analysis of the alarm information received from the optical fiber vibration system and the video intelligent recognition results, and generates the final fused alarm information according to the preset conditions.

[0062] Embodiment 4:

[0063] The laser radar and image sensor fusion monitoring solution uses the advantages of image and point cloud data to perform feature-level and decision-level fusion, thereby improving the final detection performance. Compared with pure visible light video analysis, laser radar can obtain the specific location, shape and size information of intruding foreign objects through three-dimensional point clouds, and then provide three-dimensional data information for the detection system. Through this solution, cross-modal data fusion can be achieved, reducing the impact of harsh conditions such as dark light, strong light, and haze at night on the detection results, and at the same time, the size and location of intruding foreign objects can be accurately analyzed. Generally speaking, images contain more information such as the color and texture of objects, while point cloud recognition relies more on external contour features such as lines and corners, and reflection intensity can also be applied to them. The two sensors obtain different information, and each has its own advantages when completing specific tasks. The fusion of the two sensors can complement their advantages, and can obtain both the color and texture information of the object and its three-dimensional spatial contour information. In view of the complex situation of the tunnel entrance, the radar vision fusion perception perimeter security technology is adopted in this embodiment. In the heavy-duty railway tunnel entrance safety status monitoring system provided by this embodiment, the size and precise location information of the intrusion target are determined according to the image data and the three-dimensional point cloud data, including: performing two-dimensional target detection on the image data to obtain the position of the intrusion target in the two-dimensional image and the two-dimensional target detection result; intercepting the three-dimensional point cloud data according to the position of the intrusion target in the two-dimensional image to obtain the three-dimensional point cloud data of the area of ​​interest; performing three-dimensional target detection on the three-dimensional point cloud data of the area of ​​interest to obtain the three-dimensional target detection result; weighted fusion of the two-dimensional target detection result and the three-dimensional target detection result to obtain the size and precise location information of the intrusion target. In some optional implementations, before intercepting the three-dimensional point cloud data according to the position of the intrusion target in the two-dimensional image, the processor is also used to: downsample the three-dimensional point cloud data using a voxel filter to obtain the three-dimensional point cloud data after noise reduction; convert the three-dimensional point cloud data after noise reduction to the world coordinate system and segment the rail surface. In some optional implementations, performing three-dimensional target detection on the three-dimensional point cloud data of the area of ​​interest includes: performing three-dimensional target detection on the three-dimensional point cloud data of the area of ​​interest using a pre-trained three-dimensional target detection network.

[0064] First, a 2D image detector is used to propose a target box in the image. Then, the target point cloud is filtered by the position corresponding to the 2D box. The filtered point cloud is rotated around the spatial point corresponding to the center point of the object in the 2D image until the center line is aligned with the coordinate axis Y, so as to facilitate the subsequent point cloud aggregation. Among them, the center line is generated by back-projecting the center of the 2D object to the 3D space. The detection network takes the 3D original point cloud and the 2D image as input. The point cloud is filtered by the position corresponding to the 2D image detection box. The proposed point cloud is preliminarily clustered, and the features are extracted by the voting model network layer. Then, a fully convolutional network is used for multi-scale feature fusion. Finally, the fused features are input to the detection head, including the classification head and the regression head, to predict the detection results. After 3D target detection, the 3D detection box of the intrusion target can be obtained. According to the external parameter matrix of the monitoring camera and the lidar and the internal parameter matrix of the camera itself, it is converted to the image plane coordinate system to obtain a 2D box. At the same time, because the camera uses deep learning methods for detection, it is difficult to identify fewer foreign objects in the training samples, and due to changes in light and shadow, some false alarm information will always be generated. Therefore, the final decision requires comprehensive analysis and judgment to obtain the final alarm result.

[0065] In order to detect the intrusion target from the collected point cloud data, the original point cloud data needs to be processed. Inevitably, there will be some noise points in the laser radar measurement data. When there is too much noise, it will affect the subsequent point cloud processing process. Therefore, the voxelized filter VoxelGrid (stereo pixel grid) filter is used to downsample the point cloud, filter out the noise points in the point cloud data, reduce the point cloud data, and retain the shape characteristics of the point cloud at the same time. Because the coordinates of the point cloud data generated by the laser radar are based on the sensor itself, it may be tilted relative to the world coordinate system. In order to facilitate the subsequent point cloud target detection, the point cloud data needs to be converted to the world coordinate system. Then it is necessary to determine the spatial boundary area of ​​the railway for track surface segmentation, and then perform subsequent processing operations on the point cloud in the area. The rotation matrix R that converts the laser radar coordinate system to the world coordinate system is to convert the normal vector n of the plane to the rotation matrix of the world coordinate system Z = (0,0,1). By defining Z·n = |Z||n|cosθ by the dot product, the rotation angle can be obtained The plane where the rotation angle θ lies is the plane formed by n and Z, so the rotation axis of the coordinate system is also perpendicular to the rotation axis of the plane ω = n × Z = (a, b, c). Then the rotation matrix is ​​obtained according to the Rodriguez rotation formula:

[0066]

[0067] Since the amount of laser radar data is relatively large, and only foreign objects on the track surface need to be detected, while the point cloud on the ground and the point cloud outside the train running line do not need to be processed and detected, the track surface needs to be segmented after the point cloud coordinate system is converted to the world coordinate system, and the railway limits are delineated in the Y and Z dimensions. The point cloud direct filtering method is used to filter out the redundant points outside the railway limits and on the ground.

[0068] After extracting the 3D point cloud corresponding to the 2D image target, the 3D target detection network is used to identify the 3D target. The recognition algorithm can be roughly divided into the following main parts: data preprocessing; voting model network; RPN network. The RPN network includes the FCN network and the detection head. Among them, the laser radar and the camera have been jointly calibrated and can perform coordinate transformation. First, the target box is proposed in the image using a 2D detector. Then, the target point cloud is filtered by the frustum corresponding to the 2D box. The filtered point cloud is rotated around the spatial point corresponding to the center point of the object in the 2D image until the center line is aligned with the coordinate axis Y, so as to facilitate the subsequent point cloud aggregation. Among them, the center line is generated by back-projecting the center of the 2D object to the system 3D space. The detection network takes the 3D original point cloud and the 2D image as input. The point cloud is filtered by the frustum corresponding to the 2D image detection box. The proposed point cloud is preliminarily clustered and the features are extracted by the voting model network layer. Then, the FCN network is used for multi-scale feature fusion. Finally, the fused features are input to the detection head, including the classification head (CLS) and the regression head (REG), to predict the detection results.

[0069] The voting network model actually includes two parts: point cloud clustering and multi-scale feature extraction. For clustering, a voting module based on generalized Hough transform and a grouping module based on farthest point sampling are used here, while feature extraction uses Pointnet. Unlike all objects in the image that follow the principle of large near and small far, the size of objects in the point cloud still maintains the original scale, while the scale of people and cars is quite different, and the number of point clouds is also quite different. For people, features need to be aggregated in a smaller range, while for cars, it is the opposite. Therefore, the feature extraction module is multi-scale. We aggregate the point cloud at multiple scales based on Euclidean distance, and select four different scales in total. Multi-scale features are generated through Pointnet. The RPN (Region Proposal Networks) network, whose function is to propose targets. The algorithm designed in this chapter, the main function of its RPN network module is the fusion of multi-scale features and the prediction of target boxes, which can be divided into two modules, including the FCN module and the classification and regression head module. Among them, the FCN network design is mainly used for the fusion of multi-scale features, which is mainly composed of convolutional layers and deconvolutional layers. The feature map size is adjusted and spliced ​​through the convolutional layers and deconvolutional layers, and then further fusion and adjustment are performed using the convolutional layers. Finally, multi-scale splicing similar to the feature pyramid is performed to obtain the fused feature map. The input of the classification and regression head module is the fused feature map, which is mainly used to predict the type of the target and perform accurate positioning, and is also composed of convolutional layers.

[0070] It is understandable that in actual railway scenarios, only normal trains are allowed to pass within the railway operating limits. Other intruding foreign objects such as people, animals, fallen rocks and debris should be identified. Therefore, when the three-dimensional target within the limit is determined by the radar-vision fusion detection method, the category, three-dimensional information and confidence information of the obstacle can be obtained, and the normal trains passing through can be filtered out. The category and confidence information of the obstacle will also be output for the detection of a single image. Finally, after comprehensive analysis, different weight thresholds are used according to the different categories of obstacles detected, and the final detection results are obtained through calculation. When the lighting conditions are poor, such as at night, the threshold of the lidar detection alarm can be increased to compensate for the disadvantage of the decreased detection accuracy of the surveillance camera.

[0071] Those skilled in the art will appreciate that the above embodiments are specific embodiments for implementing the present application, and in actual applications, various changes may be made thereto in form and detail without departing from the spirit and scope of the present application.

Claims

1. A heavy-duty railway tunnel entrance safety status monitoring system, characterized in that: include: An optical fiber monitoring module, a video monitoring module, a radar monitoring module and a processor, wherein the optical fiber monitoring module, the video monitoring module and the radar monitoring module are respectively connected to the processor; The optical fiber monitoring module adopts a zone-type vibration optical fiber, which is laid in a first monitoring and protection area, and the first monitoring and protection area where the zone-type vibration optical fiber is laid is divided into a plurality of zones according to a preset distance interval, so as to obtain the optical signal of the first monitoring and protection area in real time; The video monitoring module includes a plurality of video monitoring cameras, which are installed along the heavy-load railway and associated with each defense zone for acquiring image data; The radar monitoring module includes at least one laser radar, which is used to obtain three-dimensional point cloud data of the second monitoring and protection area; The processor is used to implement safety status monitoring according to the light signal, the image data and the three-dimensional point cloud data.

2. The system according to claim 1, characterized in that The processor is used to implement safety status monitoring according to the light signal, the image data and the three-dimensional point cloud data, including: Determine the category and rough location information of the intrusion target according to the optical signal and the image data; The size and precise location information of the intrusion target are determined based on the image data and the three-dimensional point cloud data.

3. The system according to claim 2, characterized in that Determining the category and rough location information of the intrusion target according to the light signal and the image data includes: Determine an abnormal event and the defense zone number where the abnormal event is located according to the change of the optical signal of the first monitoring and protection area obtained in real time, wherein the abnormal event includes human intrusion, animal intrusion, rock collapse, landslide and mud-rock flow; According to the defense zone number where the abnormal event is located, the video surveillance camera associated with the defense zone is controlled to obtain image data of the defense zone where the abnormal event is located; Analyze the image data of the defense zone where the abnormal event occurs to obtain the category and rough location information of the intrusion target.

4. The system according to claim 3, characterized in that The processor is also used to generate alarm information according to the abnormal event, the defense zone number where the abnormal event is located, and the category and rough location information of the intrusion target.

5. The system according to claim 2, characterized in that Determining the size and precise location information of the intrusion target according to the image data and the three-dimensional point cloud data includes: Performing two-dimensional target detection on the image data to obtain the position of the intrusion target in the two-dimensional image and the two-dimensional target detection result; The three-dimensional point cloud data is intercepted according to the position of the intrusion target in the two-dimensional image to obtain the three-dimensional point cloud data of the area of ​​interest; Performing three-dimensional object detection on the three-dimensional point cloud data of the region of interest to obtain a three-dimensional object detection result; The two-dimensional target detection result and the three-dimensional target detection result are weightedly fused to obtain the size and precise position information of the intrusion target.

6. The system according to claim 5, characterized in that Before intercepting the three-dimensional point cloud data according to the position of the intrusion target in the two-dimensional image, the processor is further used to: Downsampling the three-dimensional point cloud data using a voxel filter to obtain noise-reduced three-dimensional point cloud data; The denoised three-dimensional point cloud data is converted to a world coordinate system and the rail surface is segmented.

7. The system according to claim 5, characterized in that The performing three-dimensional target detection on the three-dimensional point cloud data of the region of interest includes: A pre-trained three-dimensional object detection network is used to perform three-dimensional object detection on the three-dimensional point cloud data of the region of interest.

8. The system according to any one of claims 1 to 7, characterized in that: The video surveillance camera is a laser night vision camera.

9. The system according to any one of claims 1 to 7, characterized in that: The first monitoring and protection area includes the area above the tunnel entrance, the top area of ​​the high and steep slope, the slope areas on both sides of the track and the guardrail area.

10. The system according to any one of claims 1 to 7, characterized in that: The second monitoring and protection area includes the area within the railway limit.

Images