A method, device, system and storage medium for detecting intrusion of foreign objects in urban rail transit

Abstract

The present invention discloses an urban rail foreign object intrusion detection method, device, system and storage medium. The method includes: acquiring an image of the track to be detected in real time; inputting the image of the track to be detected into a pre-trained foreign object / track detection model to obtain a track contour and a foreign object position box; generating a limit section based on the foreign object position box and the track contour; determining whether there is a vertex of the foreign object position box within the limit section. If at least one vertex is within the limit section, a foreign object intrusion has occurred. By using a deep learning-based foreign object / track detection model to identify the track and foreign objects, it is possible to ensure excellent feature extraction ability and good feature expression ability, with strong recognition performance and high efficiency. Judging intrusion based on the track contour and the foreign object position box can better avoid false intrusion judgments. Therefore, it can ensure the detection efficiency and detection accuracy and achieve real-time detection of the entire road section.

Description

Technical Field

[0001] The present invention relates to the field of urban rail safety, and particularly to a method, device, system and storage medium for detecting foreign object intrusion in urban rails. Background Technique

[0002] With the advancement of urbanization, traffic problems have become increasingly prominent. In order to further improve the urban traffic carrying capacity, many cities have continuously increased the intensity of subway construction, and the resulting subway operation safety problems have also begun to attract much attention. The subway clearance is the spatial range to ensure the safe operation of the subway. When a foreign object intrudes into the clearance, it will pose a great threat to the operation safety of the subway vehicle. As an important guarantee for the safe operation of the subway, the subway foreign object intrusion detection system needs to detect foreign objects such as falling stones, dropped objects, spills, and pedestrian crossings that appear in different subway scenarios, and determine whether there is an event of a foreign object intruding into the subway safety clearance.

[0003] At present, the monitoring of foreign objects within the subway clearance in China is still manual inspection. Although this method is simple and direct, due to the characteristics of suddenness, irregularity, and unpredictability of foreign object intrusion events, which may occur at any time and anywhere on the track, this method can only detect intruding foreign objects during the time when the inspection personnel pass by, and cannot ensure safety without foreign objects at all times. The actual manual inspection has poor reliability, heavy workload, and high missed detection rate. Traditional foreign object detection using HOG or color histograms is easily affected by similar backgrounds, pose changes, sudden illumination changes, occlusion, target cross-movement, etc., resulting in a sharp decline in the robustness of features, and then leading to errors in tracking and other situations. The domestic railway disaster detection system has been applied in many built high-speed railways, such as the Beijing-Tianjin Railway, the Beijing-Shanghai Railway, and the Wuhan-Guangzhou Railway. However, most of them are based on sensor networks and belong to contact monitoring, which requires large-density laying along the railway line and has a high maintenance cost. The damage of some sensors is also likely to cause false monitoring results. With the development of optical sensors and computing technology, the railway foreign object intrusion detection technology based on image and video acquisition and processing has gradually developed.

[0004] Domestic experimental studies on foreign object intrusion discrimination based on various sensors have been carried out.

[0005] Chinese Patent "CN107172388B" uses the method of base station + sound sensor acquisition device for foreign object intrusion detection. When half of the acquisition devices in a certain base station area detect a foreign object, it is confirmed that there is a foreign object intrusion, and the detection effect for small targets is not good;

[0006] Chinese Patent "CN110133669A" uses a three-dimensional laser sensor for detection, which has high stability, fast response speed, and low false alarm and missed detection rates, but the cost is relatively high and the detection area is limited;

[0007] The Chinese patent "CN107097810B" uses an unmanned aerial vehicle (UAV) for intelligent identification and detection of foreign objects. However, it requires the UAV to fly at the same speed in the same direction as the train, which places too high a demand on the performance of the UAV control algorithm.

[0008] The paper "Level Crossings Obstacle Detection System Using Stereo Cameras" proposes a foreign object detection method based on binocular stereo vision technology. The system uses binocular cameras to capture images of the track area and detects whether there are intrusion foreign objects in the detection area through stereo matching and three-dimensional reconstruction. Since the current dense three-dimensional reconstruction technology based on stereo vision is not yet mature, misdetection occurs from time to time.

[0009] The paper "Railway Foreign Object Recognition Based on Dual Background Modeling and Difference Images" uses the dual background + image difference technology for foreign object intrusion detection. It is simple to implement and has high real-time performance, but lacks reliability and generality.

[0010] Classification of existing technologies:

[0011] The first type of method has a fixed detection area, including "A Three-Dimensional Laser Foreign Object Intrusion Monitoring Method and System" disclosed in the Chinese patent "CN110133669A", "A Parallel Foreign Object Intrusion Monitoring System for Railway Disaster Prevention" disclosed in the Chinese patent "CN103112479A", the paper "Vision Based Platform Monitoring System for Railway Station Safety", "An Orbital Foreign Object Intrusion Detection Device System and Method Based on Multi-Line Three-Dimensional Radar" disclosed in the Chinese patent "CN111398990A", etc. This type of method places sensors on accident-prone sections to scan the railway tracks, automatically detecting the presence of foreign objects without manual protection. This technology is expensive, has low intelligence, poor practicability, and can only detect a small area.

[0012] The second type of method detects the area where the train has passed, including "An Intelligent Identification and Early Warning Method and System for UAVs of Foreign Object Intrusion along Railway Lines" disclosed in the Chinese patent "CN107097810B", the papers "Railway Track Foreign Object Recognition Based on Dual Background Modeling and Differential Images", "Vision Based Platform Monitoring System for Railway Station Safety", and "FAST DETECTION STUDY OF FOREIGN OBJECT INTRUSION ON RAILWAY TRACK". This type of method mainly relies on traditional image information processing technology or radar technology, such as methods like Caddy edge detection and HOG histogram transformation for foreign object detection and intrusion determination, and is more susceptible to the influence of light, weather, and complex scenes. Summary of the Invention

[0013] The present invention provides an urban rail foreign object intrusion detection method, device, system, and storage medium to solve the problem that existing foreign object intrusion detection is difficult to achieve real-time detection of the entire section while ensuring the detection effect.

[0014] In a first aspect, an urban rail foreign object intrusion detection method is provided, including:

[0015] Obtaining in real-time an image of the track to be detected;

[0016] Inputting the image of the track to be detected into a pre-trained foreign object / track detection model to obtain a track contour and a foreign object position box; wherein, the foreign object / track detection model is trained based on a number of historical track images.

[0017] Generating a limit cross-section based on the foreign object position box and the track contour;

[0018] Determining whether there is a vertex of the foreign object position box within the limit cross-section. If at least one vertex is within the limit cross-section, then a foreign object intrusion has occurred.

[0019] Further, the image of the track to be detected and a number of historical track images are both collected by a camera disposed at the front end of the train.

[0020] Further, the foreign object / track detection model is trained based on a number of historical track images, including:

[0021] Obtaining a number of historical track images;

[0022] Label the tracks and foreign objects in several historical track images. Take each historical track image and its label as a training sample to construct a training sample dataset. Among them, when labeling the tracks, the area enclosed by the two steel rails is uniformly divided into the track.

[0023] Train a model based on the training sample dataset to obtain a foreign object / track detection model.

[0024] Furthermore, the training of the model based on the training sample dataset to obtain a foreign object / track detection model includes:

[0025] Divide the training sample dataset into a training set and a test set.

[0026] Define the visual task of the track foreign object target.

[0027] Use the training set and the test set to train and test the detection model constructed based on the Mask R-CNN network structure to obtain a foreign object / track detection model.

[0028] Furthermore, the labeling of the tracks and foreign objects in several historical track images includes:

[0029] Label the contours and their respective types of the tracks and foreign objects in each historical track image.

[0030] Furthermore, the generation of the limit section based on the foreign object position box and the track contour includes:

[0031] Draw a horizontal line at the lowest end of the obtained foreign object position box, and regard the two intersection points of this line and the two boundaries of the track contour as the two lower vertices of the limit section. Generate the limit section according to these two lower vertices and the vehicle limit in proportion.

[0032] Furthermore, it also includes:

[0033] After determining that a foreign object intrusion has occurred, send a warning signal.

[0034] Store the information of the foreign object and the intrusion time.

[0035] In a second aspect, a device for detecting foreign object intrusion in an urban rail is provided, including:

[0036] An image acquisition module for real-time acquisition of the track image to be detected.

[0037] A track and foreign object detection module for inputting the track image to be detected into a pre-trained foreign object / track detection model to obtain a track contour and a foreign object position box. Among them, the foreign object / track detection model is obtained by training the model based on several historical track images.

[0038] A limit cross-section generation module, configured to generate a limit cross-section based on a foreign object position box and an orbit profile;

[0039] An intrusion determination module, configured to determine whether there is a vertex of the foreign object position box within the limit cross-section. If at least one vertex is within the limit cross-section, a foreign object intrusion has occurred.

[0040] In a third aspect, a detection system for foreign object intrusion in urban rail transit is provided, including a camera, a host computer, and an alarm device;

[0041] The camera is configured to be arranged at the front end of the train, collect the track image in front of the train and send it to the host computer;

[0042] The host computer is configured to execute the above-mentioned method for detecting foreign object intrusion in urban rail transit after receiving the track image sent by the camera. After determining that a foreign object intrusion has occurred, a warning signal is sent to the alarm device;

[0043] The alarm device is configured to give an alarm when receiving the warning signal.

[0044] In a fourth aspect, a computer-readable storage medium is provided, which stores a computer program. The computer program is suitable for being loaded and executed by a processor to execute the above-mentioned method for detecting foreign object intrusion in urban rail transit.

[0045] Advantageous effects

[0046] The present invention provides a method, device, system, and storage medium for detecting foreign object intrusion in urban rail transit. First, a foreign object / orbit detection model is used to identify the orbit and foreign objects, obtaining the orbit profile and the foreign object position box; then a limit cross-section is generated based on the orbit profile and the foreign object position box, and it is determined whether a foreign object intrusion has occurred according to whether there is a vertex of the foreign object position box within the limit cross-section. Using a deep learning-based foreign object / orbit detection model for orbit and foreign object identification can ensure excellent feature extraction ability and good feature expression ability, with strong recognition performance and high efficiency; making the intrusion determination based on the orbit profile and the foreign object position box can better avoid false intrusion judgments. Therefore, it can ensure the detection efficiency and detection accuracy, and achieve real-time detection of the entire section. Moreover, based on images, the perceivable distance is farther, and more response time can be given to the train crew when a crisis situation is detected, with higher safety. Description of the drawings

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0048] Figure 1 It is a flowchart of a method for detecting foreign object intrusion in urban rail provided by an embodiment of the present invention;

[0049] Figure 2 It is a flowchart of training a foreign object / rail detection model provided by an embodiment of the present invention;

[0050] Figure 3 It is a schematic diagram of the vehicle gauge structure provided by an embodiment of the present invention;

[0051] Figure 4 It is the original image of an example provided by an embodiment of the present invention;

[0052] Figure 5 It is Figure 4 The labeled effect diagram of an example provided;

[0053] Figure 6 It is Figure 4 The intrusion determination status diagram of an example provided. Detailed implementation manners

[0054] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be described in detail below. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other implementation manners obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention.

[0055] Embodiment 1

[0056] As Figure 1 shown, this embodiment provides a method for detecting foreign object intrusion in urban rail, including:

[0057] S1: Obtain the track image to be detected in real time; the track image to be detected can be collected by a camera arranged at the front end of the train. Specifically, according to the train gauge specification, typical foreign objects are placed within the train gauge, and it is required that the foreign objects be as real as possible and the intrusion mode into the gauge be real and natural. The types, sizes, placement positions, etc. of the foreign objects can be determined by communicating with the subway safety supervision department and the operation department, and data collection of the foreign objects is carried out respectively when the train is stationary and running.

[0058] S2: Input the track image to be detected into a pre-trained foreign object / rail detection model to obtain the track contour and the foreign object position box. Among them, the foreign object / rail detection model is obtained by training the model based on a number of historical track images, and the specific process includes:

[0059] S2.1: Obtain a number of historical track images; the number of historical track images can be collected by a camera arranged at the front end of the train.

[0060] S2.2: Label the tracks and foreign objects in several historical track images, including labeling the outlines and their respective types of the tracks and foreign objects in each historical track image. Take each historical track image and its label as a training sample to construct a training sample dataset. When labeling the tracks, the area enclosed by the two steel rails is uniformly divided into tracks, which can avoid the trouble corresponding to the left and right steel rails in the multi-track scenario.

[0061] S2.3: Perform model training based on the training sample dataset to obtain a foreign object / track detection model. Specifically, it includes:

[0062] Divide the training sample dataset into a training set and a test set;

[0063] Define the visual task of track foreign object targets;

[0064] Use the training set and the test set to train and test the detection model constructed based on the Mask R-CNN network structure to obtain a foreign object / track detection model.

[0065] More specifically, the present invention regards the detection of tracks and foreign objects as an instance segmentation task, and uses a detection model constructed based on the Mask R-CNN network structure for detection. As Figure 2 shown, the model training process includes:

[0066] Define the visual task of track foreign object targets;

[0067] Use the RPN (Region Proposal Network) network to perform network extraction on the images in the samples to obtain RPs (region proposals);

[0068] Select a CNN network structure, perform pre-training using the ImageNet dataset to obtain pre-training parameters, and then obtain a pre-trained model;

[0069] Take the training set, its corresponding label, and the RPs as inputs, and perform secondary training on the pre-trained model through Mask R-CNN to obtain a foreign object / track detection model;

[0070] Use the test set to test and optimize the foreign object / track detection model to obtain the final foreign object / track detection model.

[0071] Specifically, first, the visual task is defined using the track image to be detected. The selective search algorithm is used to obtain the candidate regions of the sample image, and the coordinates of the candidate regions are input into the network for learning together with the example track image of the visual task. The example track image passes through the convolutional layer and pooling layer in the deep convolutional neural network, and finally the deep convolutional features are obtained. Then, based on the Mask R-CNN network structure, the features are normalized through the region of interest pooling layer, and finally the features are input into different fully connected branches to perform parallel regression calculation for feature classification and detection of the bounding box coordinate values. After multiple iterative trainings, finally, a foreign object / track detection model strongly related to the specified visual task is obtained, with trained weight parameters.

[0072] S3: Generate a limit section based on the foreign object position box and the track profile. Specifically, a horizontal line is drawn at the lowest end of the obtained foreign object position box, and the two intersection points of this line with the two boundaries of the track profile are regarded as the two lower vertices of the limit section. The limit section is generated proportionally based on these two lower vertices and the vehicle clearance. Among them, the vehicle clearance is formed by enveloping the train cross-section, and the structure is as Figure 3 shown.

[0073] S4: Determine whether there is a vertex of the foreign object position box within the limit section. If at least one vertex is within the limit section, then a foreign object intrusion has occurred.

[0074] S5: After determining that a foreign object intrusion has occurred, send a warning signal; and store the information of the foreign object and the intrusion time for accountability afterwards. The information of the foreign object includes position, type, and size.

[0075] Embodiment 2

[0076] This embodiment provides an urban rail foreign object intrusion detection device, including:

[0077] An image acquisition module for real-time acquisition of the track image to be detected;

[0078] A track and foreign object detection module for inputting the track image to be detected into a pre-trained foreign object / track detection model to obtain the track profile and the foreign object position box; among them, the foreign object / track detection model is obtained by training the model based on a number of historical track images;

[0079] A limit section generation module for generating a limit section based on the foreign object position box and the track profile;

[0080] An intrusion determination module for determining whether there is a vertex of the foreign object position box within the limit section. If at least one vertex is within the limit section, then a foreign object intrusion has occurred.

[0081] For other specific implementations, refer to the urban rail foreign object intrusion detection method provided in Embodiment 1, which will not be elaborated here.

[0082] Example 3

[0083] This embodiment provides an urban rail foreign object intrusion detection system, including a camera, a host computer, and an alarm device;

[0084] The camera is used to be set at the front end of the train, collect the track image in front of the train and send it to the host computer; a binocular camera can be used for the camera, and the camera is fixed on the train, which can better resist the influence of bad weather;

[0085] The host computer is used to execute the above-mentioned urban rail foreign object intrusion detection method after receiving the track image sent by the camera. After determining that a foreign object intrusion has occurred, it sends a warning signal to the alarm device; the host computer is also used to store the information of the foreign object and the intrusion time after a foreign object intrusion has occurred;

[0086] The alarm device is used to give an alarm when receiving the warning signal; in specific implementation, the alarm device can select an audible and visual alarm, or can also directly use a display screen to give an alarm.

[0087] For other specific implementations, refer to the urban rail foreign object intrusion detection method provided in Example 1, which will not be elaborated here.

[0088] Example 4

[0089] This embodiment provides a computer-readable storage medium, which stores a computer program, and the computer program is suitable for being loaded and executed by a processor to execute the above-mentioned urban rail foreign object intrusion detection method.

[0090] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0091] This application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of processes and / or blocks in the flowchart and / or block diagram can also be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate for implementing in the processFigure 1 one process or multiple processes and / or boxes Figure 1 a device for the functions specified in one box or multiple boxes.

[0092] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured article including an instruction device, and the instruction device implements the process Figure 1 one process or multiple processes and / or boxes Figure 1 the functions specified in one box or multiple boxes.

[0093] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide for implementing the process Figure 1 one process or multiple processes and / or boxes Figure 1 the steps of the functions specified in one box or multiple boxes.

[0094] It can be understood that the same or similar parts in the above embodiments can be referred to each other, and the content not described in detail in some embodiments can be referred to the same or similar content in other embodiments.

[0095] Any process or method description in the flowchart or described in other ways herein can be understood as representing a module, segment, or part of code including one or more executable instructions for implementing a specific logical function or process, and the scope of the preferred embodiments of the present invention includes additional implementations, where the functions can be executed not in the order shown or discussed, including in a substantially simultaneous manner or in the reverse order according to the involved functions, which should be understood by those skilled in the art to which the embodiments of the present invention belong.

[0096] The present invention also provides a specific example, as Figures 4 to 6 shown Figure 4 shown is an orbital image Figure 5 shown is the orbital image after annotation Figure 6 shown is the foreign object intrusion determination state diagram of the orbital image. As shown in the figure, the vertex of the foreign object position box is within the limit section, so foreign object intrusion has occurred.

[0097] The present invention provides a method, device, system and storage medium for detecting foreign object intrusion in urban rail transit. First, a foreign object / rail detection model is used to identify the rail and foreign objects, obtaining the rail contour and the foreign object position box; then, a limit section is generated based on the rail contour and the foreign object position box, and it is determined whether foreign object intrusion has occurred according to whether there are vertices of the foreign object position box within the limit section. The present invention regards the detection of rails and foreign objects as an instance segmentation task and adopts an instance segmentation model based on deep learning, which can ensure excellent feature extraction ability and good feature expression ability, and can obtain better detection effects and performance; the intrusion determination based on the rail contour and the foreign object position box can better avoid false intrusion judgments. Therefore, it can ensure the detection efficiency and detection accuracy and achieve real-time detection of the entire section of the road. Moreover, based on images, the perceivable distance is farther, and more response time can be given to the train crew when a crisis situation is found, and the safety is higher.

[0098] Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those of ordinary skill in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for detecting intrusion of foreign objects in urban rail transit, characterized in that, Including: Obtain the track image to be detected in real time; The track image to be detected is collected by a camera set at the front end of the train; Input the track image to be detected into a pre-trained foreign object / track detection model to obtain the track contour and the foreign object position box; among them, the foreign object / track detection model is obtained by training based on a number of historical track images; Generate a clearance cross-section based on the foreign object position box and the track contour; Determine whether there is a vertex of the foreign object position box within the clearance cross-section. If at least one vertex is within the clearance cross-section, foreign object intrusion has occurred; The foreign object / track detection model is obtained by training based on a number of historical track images, including: Obtain a number of historical track images; Label the tracks and foreign objects in a number of historical track images. Take each historical track image and its label as a training sample to construct a training sample data set; among them, when labeling the track, the area encompassed between the two rails is uniformly divided into the track; Perform model training based on the training sample data set to obtain a foreign object / track detection model; The performing model training based on the training sample data set to obtain a foreign object / track detection model includes: Divide the training sample data set into a training set and a test set; Define the visual task of the track foreign object target; Use the training set and the test set to train and test a detection model constructed based on the Mask R-CNN network structure to obtain a foreign object / track detection model; The generating a clearance cross-section based on the foreign object position box and the track contour includes: Make a horizontal line at the lowest end of the obtained foreign object position box, and regard the two intersection points of this line and the two boundaries of the track contour as the two lower vertices of the clearance cross-section, and generate the clearance cross-section according to these two lower vertices and the vehicle clearance in proportion.

2. The urban rail foreign object intrusion detection method according to claim 1, characterized in that Both the track image to be detected and a number of historical track images are collected by a camera set at the front end of the train.

3. The urban rail foreign object intrusion detection method according to claim 1, wherein, The labeling of the tracks and foreign objects in a number of historical track images includes: Label the contours and their respective types of the tracks and foreign objects in each historical track image.

4. The urban rail foreign object intrusion detection method according to claim 1, characterized in that Also including: After determining that foreign object intrusion has occurred, send out a warning signal; Store the information of the foreign object and the intrusion time.

5. An urban rail foreign object intrusion detection device for implementing the urban rail foreign object intrusion detection method as described in claim 1, characterized in that, Including: An image acquisition module for obtaining the track image to be detected in real time; A track and foreign object detection module for inputting the track image to be detected into a pre-trained foreign object / track detection model to obtain the track contour and the foreign object position box; among them, the foreign object / track detection model is obtained by training based on a number of historical track images; A clearance cross-section generation module for generating a clearance cross-section based on the foreign object position box and the track contour; An intrusion determination module for determining whether there is a vertex of the foreign object position box within the clearance cross-section. If at least one vertex is within the clearance cross-section, foreign object intrusion has occurred.

6. An urban rail foreign object intrusion detection system, characterized in that, Including a camera, a host computer, and an alarm device; The camera is used to be set at the front end of the train, collect the track image in front of the train and send it to the host computer; The host computer is configured to execute the urban rail foreign object intrusion detection method according to any one of claims 1 to 4 after receiving the track images sent by the camera, and send a warning signal to the alarm device after determining that a foreign object intrusion has occurred; The alarm device is configured to give an alarm when receiving the warning signal.

7. A computer-readable storage medium storing a computer program, characterized in that, The computer program is adapted to be loaded and executed by a processor to execute the urban rail foreign object intrusion detection method according to any one of claims 1 to 4.

Images