Rail transit safe operation and early warning system

Abstract

The invention discloses a safe operation and early warning system for rail transit. The safe operation and early warning system comprises an intelligent operation host, a radar sensor and an environment acquisition module, the radar sensor comprises a laser radar and a millimeter wave radar and is used for collecting 3D point cloud of the surrounding environment of the locomotive, the distance between surrounding environment objects and the locomotive, the movement speed and the locomotive angle; the environment acquisition module comprises a visible light image acquisition device and a thermal imaging image acquisition device and is used for acquiring visible light, a thermal imaging image and the ambient temperature around the locomotive; the intelligent operation host judges whether an obstacle exists or not according to the image and the environment temperature, judges the distance and category of the obstacle in combination with the vital sign condition of the obstacle, the track where the obstacle is located, 3D modeling, locomotive positioning information and the image, constructs a safety early warning strategy and sends out safety early warning; according to the system, whether an invader has vital signs or not can be quickly identified by adding thermal imaging, and the accuracy is improved; locomotive positioning, obstacle vital sign recognition and foreign matter invasion detection are achieved, the type, distance, size and movement trend of an obstacle are recognized when the obstacle is detected, alarm information is given through host computer operation, brake control is started, and the operation safety of a locomotive is guaranteed.

Description

Technical Field

[0001] The present invention relates to the field of locomotive operation safety, and in particular to a rail transit safety operation and early warning system. Background Art

[0002] In recent years, train collisions and derailment accidents caused by locomotives hitting foreign objects or accidents with the vehicles ahead have occurred frequently. The objects of such accidents are mostly divided into locomotives and objects; locomotives and the front and rear vehicles; locomotives and people or other animals. In order to improve the reliability of line operations and reduce operation and maintenance costs, it is necessary to use radar and other technologies to realize active obstacle detection to ensure the safety of locomotive operation. At the same time, the monitoring content of locomotive operation safety includes not only obstacle detection, but also the identification of other dangerous sources that may endanger the safety of locomotive operation, such as contact network status and surrounding environment hazard detection.

[0003] Traditional obstacle detection technologies mostly use laser radar, millimeter-wave radar combined with cameras and other detection technologies to detect the surrounding environment. This traditional detection method uses radar sensors to determine the distance of obstacles and identifies them through camera image recognition and other means. Due to environmental interference, especially the camera imaging part, it has relatively high requirements for light environment, etc., and it cannot meet the imaging effect under harsh environmental conditions.

[0004] In addition, the locomotive environment is quite special. Since it has a fixed running track, the surrounding trees and other objects have little impact on the locomotive operation. Obstacle detection is mainly used to identify static objects within the track range, vehicles running on the same line, and most importantly, temporary entry of foreign objects with vital signs such as people or animals, and vehicles temporarily entering non-enclosed areas. For foreign objects with vital signs, traditional radar detection cannot quickly distinguish between intruders with vital signs and intruders without vital signs.

[0005] Traditional cameras mainly use visible light, and visible light image videos are greatly affected by the external environment. For example, insufficient light at night, rainy and snowy weather, foggy and hazy weather, etc. will increase the difficulty of visible light image recognition. Moreover, as the detection distance increases, the imaging effect becomes worse, and it cannot better identify distant objects. Therefore, it is impossible to make more advanced judgments and warnings about obstacles.

[0006] In summary, the existing rail transit safety operation and early warning technologies cannot meet the actual needs of rail transit safety operation. Summary of the invention

[0007] The present invention discloses a rail transit safe operation and early warning system to overcome the technical problems in the existing obstacle detection technology that the traditional camera uses a single visible light method to obtain images and videos, which are easily interfered by light, greatly affected by the external environment, and cannot distinguish whether the obstacle has vital signs, and cannot provide faster and more accurate early warning information.

[0008] In order to achieve the above object, the technical solution of the present invention is:

[0009] A rail transit safety operation and early warning system, comprising: an intelligent computing host, a radar sensor and an environment acquisition module;

[0010] The radar sensor includes a laser radar sensor and a millimeter wave radar sensor. The laser radar sensor is used to collect 3D point cloud information of the locomotive's surrounding environment, and the millimeter wave radar sensor is used to collect the distance between the locomotive and the objects in the locomotive's surrounding environment, the movement speed and the angle with the locomotive;

[0011] The environment acquisition module includes a visible light image acquisition device and a thermal imaging image acquisition device, which are used to acquire a visible light image, a thermal imaging image and the temperature of the environment around the locomotive;

[0012] The positioning acquisition module includes a data acquisition unit and an inertial navigation system; the locomotive data is collected by using the data acquisition unit, and the locomotive positioning information is obtained in combination with the intelligent computing host or the locomotive positioning information is obtained through the inertial navigation system;

[0013] The intelligent computing host determines whether there is an obstacle based on the visible light image, thermal imaging image and ambient temperature. If it is confirmed, it combines the vital signs of the obstacle and the track position of the obstacle, combines the 3D modeling data, locomotive positioning information and the visible light image and thermal imaging image to determine the distance between the obstacle and the locomotive, the size of the obstacle and the type of the obstacle, builds a safety warning strategy and issues a corresponding level of safety warning; the 3D modeling data is data obtained by 3D modeling of the train track and trackside environment based on the data obtained by the radar sensor.

[0014] Furthermore, the intelligent computing host determines whether there is an obstacle based on the visible light image, thermal imaging image and ambient temperature. If it is confirmed, the intelligent computing host determines the distance between the obstacle and the locomotive, the size of the obstacle and the type of the obstacle based on the vital signs of the obstacle and the track position of the obstacle, the 3D modeling data, the locomotive positioning information and the visible light image and thermal imaging image, constructs a safety warning strategy and issues a safety warning of the corresponding level, including:

[0015] S11. Determine whether there is an obstacle, the vital signs of the obstacle and the track position of the obstacle based on the visible light image and the thermal imaging image. The track position of the obstacle is to confirm whether the obstacle is within the track range;

[0016] S12. Setting an obstacle safety warning strategy, and outputting a safety warning of a corresponding level in combination with the obstacle safety warning strategy, wherein the obstacle safety warning strategy is used to characterize the feature mapping relationship between the 3D modeling data, the locomotive positioning information, and the visible light image, the thermal imaging image, and the distance between the obstacle and the locomotive, the size of the obstacle, and the category of the obstacle.

[0017] Further, the obstacle safety warning strategy includes a first obstacle safety warning strategy and a second obstacle safety warning strategy;

[0018] The first obstacle safety warning strategy is that if the obstacle has no vital signs and is located within the locomotive track range, the intelligent computing host performs image processing on the visible light image, and combines the data obtained by the radar sensor, the 3D modeling data and the locomotive positioning information to determine the type and size of the obstacle and the distance between the obstacle and the locomotive, identifies the obstacle, and performs obstacle warning or obstacle alarm according to the results of distance judgment and identification, emits an audible and visual alarm and feeds back to the locomotive TCMS system for operation processing;

[0019] The second obstacle safety warning strategy is that if the obstacle has vital signs or the obstacle is a leading vehicle or other vehicle without vital signs, the intelligent computing host obtains obstacle data based on the visible light image, thermal imaging image, temperature of the obstacle and radar sensor obtained by the environment acquisition module, identifies the obstacle or vehicle with vital signs, and dynamically tracks the obstacle with vital signs in combination with the 3D modeling data and the locomotive positioning information, determines the distance between the obstacle and the locomotive, and performs obstacle warning, obstacle alarm or vehicle alarm based on the results of obstacle tracking and obstacle identification, emits an audible and visual alarm and feeds back to the locomotive TCMS system for operation and processing.

[0020] Furthermore, the first obstacle safety warning strategy and the second obstacle safety warning strategy specifically include:

[0021] S121, setting a first temperature threshold interval and a second temperature threshold interval and determining whether there is an obstacle ahead of the track, wherein the first temperature threshold interval is used to characterize a temperature threshold interval of an intruding object with vital signs, and the second temperature threshold interval is used to characterize a temperature threshold interval of an intruding object with a motor vehicle; the first temperature threshold interval is smaller than the second temperature threshold interval;

[0022] S122, obtaining the ambient temperature of the locomotive; if there is an obstacle and the temperature of the obstacle does not belong to the first temperature threshold interval and the second temperature threshold interval, the obstacle has no vital signs, and the process proceeds to S123 to identify and warn the obstacle without vital signs; if there is an obstacle and the temperature of the obstacle belongs to the first temperature threshold interval, the process proceeds to S125 to identify and warn the obstacle with vital signs; if the temperature of the obstacle belongs to the second temperature threshold interval, the process proceeds to S126 to identify the preceding vehicle;

[0023] S123, judging whether the obstacle is within the locomotive running track range according to the visible light image, if the obstacle is detected within the track range, determining the size, position and speed of the obstacle to facilitate subsequent determination of the obstacle category, and proceeding to S124 to determine the category of the obstacle;

[0024] S124, performing image processing on the visible light image and determining the obstacle category, if the obstacle category determined is the vehicle in front or another vehicle, dynamically tracking and determining the distance between the vehicle and the locomotive, and performing a vehicle alarm according to the analysis result, issuing an audible and visual alarm and feeding back to the locomotive TCMS system for operation processing;

[0025] S125. Determine whether the obstacle is within the locomotive running track range through the visible light image and the thermal imaging image. If the obstacle is detected within the track range, determine the size, position and speed of the obstacle, use the image recognition algorithm to identify the obstacle in the visible light image and the thermal imaging image, and distinguish the type of the obstacle in combination with the acquired temperature of the obstacle, determine whether it is a pedestrian and perform dynamic tracking, judge and analyze the distance between the obstacle and the locomotive, issue an obstacle warning or obstacle alarm according to the analysis result, and send an audible and visual alarm to the locomotive TCMS system for operation and processing;

[0026] S126. Determine whether the preceding vehicle or other vehicles are within the locomotive running track range through visible light images, thermal imaging images and ambient temperature. If so, dynamically track and analyze the distance between the preceding vehicle or other vehicles and the locomotive. Issue an obstacle warning or obstacle alarm based on the analysis results, issue an audible and visual alarm and provide feedback to the locomotive TCMS system for operational processing.

[0027] Furthermore, performing image processing on the visible light image and determining the obstacle category includes:

[0028] S1241: preprocessing the visible light image to obtain a preprocessed image;

[0029] S1242: extracting features of obstacles in the preprocessed image using a feature extraction method;

[0030] S1243: Using a neural network to classify obstacles according to the extracted features to obtain obstacle categories;

[0031] S1244: Determine the category of the obstacle without vital signs by combining the obstacle data obtained by the radar sensor and the locomotive positioning information.

[0032] Furthermore, an image recognition algorithm is used to identify obstacles in the visible light image and the thermal imaging image, and the category of the obstacle is distinguished in combination with the acquired temperature of the obstacle, and whether it is a pedestrian is determined and dynamically tracked, including:

[0033] S1251: Preprocessing the visible light image and the thermal imaging image using an image processing algorithm;

[0034] S1252: extracting features of obstacles from the preprocessed image using a feature extraction algorithm;

[0035] S1253: using a neural network to classify obstacles based on the extracted features;

[0036] S1254: Using regression algorithm to locate obstacles based on the extracted features;

[0037] S1255: judging the distance between the obstacle and the locomotive based on the distance, movement speed and angle between the obstacle and the locomotive obtained by the radar sensor, and identifying the obstacle with vital signs;

[0038] S1256: Dynamically track the obstacle by combining the obstacle data acquired by the radar sensor, the 3D modeling data and the locomotive positioning information.

[0039] Furthermore, the safety warning strategy includes but is not limited to an obstacle safety warning strategy, a fire safety warning strategy and a locomotive power grid safety warning strategy.

[0040] Furthermore, the fire safety early warning strategy includes:

[0041] S21, setting a third temperature threshold interval, the third temperature threshold interval is used to characterize the threshold interval of the combustion temperature, obtaining the ambient temperature of the locomotive and judging whether there is an obstacle in front of the track, the third temperature threshold interval is greater than the second temperature threshold interval;

[0042] S22: If there is an obstacle and the temperature of the obstacle does not belong to the first temperature threshold interval, the second temperature threshold interval and the third temperature threshold interval, the obstacle has no vital signs and the first obstacle safety warning strategy is executed; if there is an obstacle and the temperature of the obstacle is in the first temperature threshold interval or the second temperature threshold interval, the second obstacle safety warning strategy is executed; if there is no obstacle and the detected temperature is in the third temperature threshold interval, enter S23 for fire identification;

[0043] S23. Fire identification and confirmation is carried out through visible light images, thermal imaging images and ambient temperature. The location and distance information of the fire is obtained through the environmental data, 3D modeling data and locomotive positioning information obtained by the radar sensor. The fire spread trend is tracked, a fire alarm is issued, and feedback is given to the locomotive TCMS system for deceleration or parking.

[0044] Furthermore, the distance between the obstacle and the locomotive is judged and analyzed, and an obstacle warning or obstacle alarm is issued according to the analysis result, including:

[0045] If the distance between the obstacle and the locomotive is getting closer and closer, an obstacle alarm is issued, and the obstacle alarm means that the locomotive needs to brake urgently and sound the horn for warning;

[0046] If the distance between the obstacle and the locomotive gradually increases, an obstacle warning is issued, and the obstacle warning is that the locomotive needs to slow down and sound the horn to warn.

[0047] Beneficial effects: The present invention provides a rail transit safety operation and early warning system. On the basis of traditional radar detection, a thermal imaging image acquisition device is added to the environment acquisition module to improve the problem that the traditional camera uses a single visible light method to acquire images and videos that are easily interfered by light. By adding thermal imaging, the temperature of the detected object and image and video data can be acquired at the same time. By sensing the temperature of different objects, intruders with vital signs and intruders without vital signs can be quickly identified, and the accuracy rate can be improved.

[0048] During the operation of the train, the system can realize locomotive positioning, obstacle vital signs recognition and foreign object intrusion detection. When an obstacle is detected ahead, it identifies the obstacle type, distance, size, and movement trend, and stores the identified obstacle-related information, vehicle-related data, alarm records and related videos, and radar point cloud data in the host hard disk. The host calculates and gives alarm information, and activates braking control when necessary to ensure the safety of locomotive operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0050] Figure 1 A system block diagram of a rail transit safety operation and early warning system provided by the present invention;

[0051] Figure 2 This is an example diagram of the operation of locomotive LKJ;

[0052] Figure 3 It is the overall operation flow chart of the system of the present invention;

[0053] Figure 4 It is a flow chart of thermal imaging temperature processing of the present invention;

[0054] Figure 5 Operation flow chart of the safety warning strategy designed for the present invention. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0056] This embodiment provides a rail transit safety operation and early warning system, such as Figure 1 As shown, it includes: an intelligent computing host, a radar sensor, an environment acquisition module and a positioning acquisition module;

[0057] The positioning acquisition module includes a data acquisition unit and an inertial navigation system; the locomotive data is collected by using the data acquisition unit, and the locomotive positioning information is obtained in combination with the intelligent computing host or the locomotive positioning information is obtained through the inertial navigation system;

[0058] The data acquisition unit is used to collect locomotive operation status data, which includes but is not limited to locomotive TCMS system data and locomotive LKJ data;

[0059] The radar sensor includes a laser radar sensor and a millimeter wave radar sensor. The laser radar sensor is used to collect 3D point cloud information of the surrounding environment of the locomotive, and the millimeter wave radar sensor is used to collect the distance, speed and angle of objects in the surrounding environment of the locomotive;

[0060] The environment acquisition module includes a visible light image acquisition device and a thermal imaging image acquisition device, which are used to acquire a visible light image, a thermal imaging image and the temperature of the environment around the locomotive;

[0061] The intelligent computing host determines whether there is an obstacle based on the visible light image, thermal imaging image and ambient temperature. If it is confirmed, it combines the vital signs of the obstacle and the track position of the obstacle, combines the 3D modeling data, locomotive positioning information and the visible light image and thermal imaging image to determine the distance between the obstacle and the locomotive, the size of the obstacle and the type of the obstacle, builds a safety warning strategy and issues a corresponding level of safety warning; the 3D modeling data is data obtained by 3D modeling of the train track and trackside environment based on the data obtained by the radar sensor.

[0062] Specifically, the data acquisition unit is connected to the locomotive bus to obtain locomotive data, including but not limited to locomotive TCMS data and locomotive LKJ data; the locomotive TCMS system records the locomotive network data, including all data on the locomotive operating status, including locomotive operating speed, equipment failure information, external input voltage stability, etc. The locomotive LKJ equipment can provide locomotive kilometer mark, locomotive positioning, locomotive operation route planning and actual operation curve, etc.; the locomotive braking system provides corresponding braking force according to the locomotive operating speed and load condition to ensure the safety of locomotive braking. For abnormal weather such as rain and snow, the friction is increased by spreading sand and other operations to ensure braking safety; the data collected is used to estimate the locomotive position based on the locomotive speed, LKJ operation curve, etc. The locomotive LKJ operation curve is as follows: Figure 2 As shown;

[0063] The inertial navigation system can complete the locomotive positioning without locomotive speed, LKJ data, etc., and use inertial navigation, GPS, etc. to fine-tune the locomotive positioning data obtained by the intelligent computing host;

[0064] The visible light image acquisition device in this embodiment uses a binocular camera, which is installed in the middle of the locomotive windshield, about 2m above the track;

[0065] In this embodiment, a binocular camera is used. Through the focus design of the binocular camera, the problem of image position deviation of a single camera can be avoided;

[0066] The thermal imaging image acquisition device described in this embodiment uses a thermal imaging detector to obtain images and videos of the surrounding environment of the locomotive together with a binocular camera, and collects the ambient temperature and the temperature of obstacles through thermal imaging technology;

[0067] The radar sensor includes a laser radar sensor and a millimeter wave radar sensor. The laser radar sensor is not affected by light and can obtain 3D point cloud information of obstacles, but is affected by rain, snow and fog. The laser radar can accurately locate the distance of obstacles and can project the radar point cloud data onto the image through coordinate system conversion; the millimeter wave radar sensor is not affected by light and weather, and is more sensitive to the recognition of moving targets, and can realize the recognition of the movement speed and direction of obstacles;

[0068] The millimeter wave radar is installed on the train head shell or the coupler, located on the track centerline, and the installation restrictions are:

[0069] The maximum horizontal deviation from the vehicle center is 600mm, and the centerline installation is optimal;

[0070] The vertical distance from the rail surface ranges from 295mm to 800mm, and the test adjustment is carried out using a corner reflector;

[0071] The intelligent computing host covers functions such as data analysis and calculation, image processing, data processing, storage and exchange transmission, and realizes inertial navigation and precise positioning, obstacle recognition and detection, 3D modeling of train tracks and trackside environment, turnout recognition, curve area recognition, fire detection, locomotive power grid anomaly detection and other environmental safety operation guarantee functions. The overall operation of the system is as follows: Figure 3 As shown,

[0072] First, the intelligent computing host integrates and analyzes the data obtained by the visible light camera, radar sensor and thermal imaging to determine whether there are obstacles. The intelligent computing host uses the point cloud data of the radar and the assistance of video data to complete the 3D modeling of the entire track and trackside environment, and combines the data of the inertial navigation system and the data acquisition unit to complete the train positioning;

[0073] Secondly, based on 3D modeling and train positioning data, a comprehensive analysis of the track and trackside environment is conducted to determine whether there are obstacles or fires, as well as information such as the temperature and distance of the obstacles. The safety warning strategy is implemented, an obstacle warning is issued, the speed is reduced and the horn is sounded to warn, or an obstacle alarm is issued, emergency braking is performed and the horn is sounded to warn.

[0074] In a specific practical example, the intelligent computing host performs 3D modeling of the train track and trackside environment based on the data obtained by the radar sensor, and combines the visible light image and locomotive positioning information obtained by the environment acquisition module to determine the distance of the obstacle:

[0075] In this embodiment, the subway driving environment and scenes are relatively fixed and single. The radar point cloud data is used, and with the assistance of the video data and inertial navigation data obtained by the camera, the 3D modeling of the entire track and trackside environment is realized based on the SLAM (Simultaneous Localization and Mapping) algorithm to accurately confirm the limit area.

[0076] The steps to accurately locate the locomotive in combination with the inertial navigation system include:

[0077] First, the initial motion state of the locomotive, such as speed, position and other information, is obtained through the data acquisition unit, and the station information, arrival signal and kilometer mark (station spacing) information are obtained through the TCMS system and LKJ system, or the locomotive speed and position information is obtained through the speed sensor and laser radar;

[0078] Secondly, the locomotive position is calculated through inertial navigation (accelerometer, gyroscope) or kilometer mark information (or station spacing), and the train arrival is judged by the train travel distance (calculated by speed and time) and the speed. Every time the train arrives at the station, the calculated train travel distance is corrected by the kilometer mark (or station spacing) to solve the problem of cumulative error; the inertial navigation system is an integration system composed of inertial sensors and integrators; the system uses inertial sensors to measure the specific force and angular velocity information of the carrier, measures the acceleration of the vehicle in the inertial reference system through accelerometers, and measures the rotational motion of the carrier through gyroscopes. It can convert the inertial coordinate system to the navigation coordinate system, integrate the angular velocity relative to time, and combine the initial motion state (speed, position) of the locomotive to deduce the position of the locomotive;

[0079] Combine the visible light image and locomotive positioning information obtained by the environment acquisition module to determine the distance of the obstacle:

[0080] The image of the obstacle collected by the visible light image is located by the lidar sensor and the millimeter-wave radar sensor, and the 3D point cloud data and orientation are fed back to the intelligent computing host. The intelligent computing host accurately determines the distance between the obstacle and the locomotive through 3D modeling and locomotive positioning, and determines whether the obstacle is gradually moving away from the locomotive or approaching the locomotive based on the movement speed of the obstacle obtained by the millimeter-wave radar sensor, and issues a safety warning.

[0081] In this embodiment, 3D modeling of the train and the surrounding environment can convert the locomotive position into visual information, facilitate the determination of the obstacle position and the distance between the obstacle and the locomotive, and provide warning information.

[0082] In a specific embodiment, the intelligent computing host determines whether there is an obstacle based on the visible light image, thermal imaging image and ambient temperature. If it is confirmed, the distance between the obstacle and the locomotive, the size of the obstacle and the type of the obstacle are determined by combining the vital signs of the obstacle and the track position of the obstacle, the 3D modeling data, the locomotive positioning information and the visible light image and thermal imaging image, and the scheme for constructing a safety warning strategy and issuing a safety warning of a corresponding level is:

[0083] S11. Determine whether there is an obstacle, the vital signs of the obstacle and the track position of the obstacle based on the visible light image and the thermal imaging image. The track position of the obstacle is to confirm whether the obstacle is within the track range;

[0084] S12. Setting an obstacle safety warning strategy, and outputting a safety warning of a corresponding level in combination with the obstacle safety warning strategy, wherein the obstacle safety warning strategy is used to characterize the feature mapping relationship between the 3D modeling data, the locomotive positioning information, and the visible light image, the thermal imaging image, and the distance between the obstacle and the locomotive, the size of the obstacle, and the category of the obstacle.

[0085] Obstacles refer to objects that may affect the safety of vehicles during driving, such as people and poultry, vehicles, road obstacles, etc. In this embodiment, the thermal imaging detector is first used to determine whether the obstacle has vital signs. If there are no vital signs, the visible light image obtained by the binocular camera or the thermal imaging detector is used to detect obstacles, and accurately distinguish the target types, such as static obstacles, static vehicles stopped on the same line, and other obstacles without vital signs. If they are dynamic intruders with vital signs (people or poultry, etc.), dynamically intruding external vehicles, dynamically running vehicles on the same line, etc., the specific objects are identified through modeling and other methods, and the movement speed and direction of the dynamic intruders obtained by the radar sensor are combined to track the changing trend of the dynamic changing objects. Therefore, based on the thermal imaging detector, dynamic obstacle detection other than static obstacles can be performed and different warnings can be issued, thereby improving the safety of the locomotive.

[0086] In a specific embodiment, the obstacle safety warning strategy includes a first obstacle safety warning strategy and a second obstacle safety warning strategy;

[0087] The first obstacle safety warning strategy is that if the obstacle has no vital signs and is located within the locomotive track range, the intelligent computing host performs image processing on the visible light image, and combines the data obtained by the radar sensor, the 3D modeling data and the locomotive positioning information to determine the type and size of the obstacle and the distance between the obstacle and the locomotive, identifies the obstacle, and performs obstacle warning or obstacle alarm according to the results of distance judgment and identification, emits an audible and visual alarm and feeds back to the locomotive TCMS system for operation processing;

[0088] The second obstacle safety warning strategy is that if the obstacle has vital signs or the obstacle is a leading vehicle or other vehicle without vital signs, the intelligent computing host obtains obstacle data based on the visible light image, thermal imaging image, temperature of the obstacle and radar sensor obtained by the environment acquisition module, identifies the obstacle or vehicle with vital signs, and dynamically tracks the obstacle with vital signs in combination with the 3D modeling data and the locomotive positioning information, determines the distance between the obstacle and the locomotive, and performs obstacle warning, obstacle alarm or vehicle alarm based on the results of obstacle tracking and obstacle identification, emits an audible and visual alarm and feeds back to the locomotive TCMS system for operation and processing.

[0089] like Figure 5 As shown, the first obstacle safety warning strategy and the second obstacle safety warning strategy specifically include:

[0090] S121, setting a first temperature threshold interval and a second temperature threshold interval and determining whether there is an obstacle ahead of the track, wherein the first temperature threshold interval is used to characterize a temperature threshold interval of an intruding object with vital signs, and the second temperature threshold interval is used to characterize a temperature threshold interval of an intruding object with a motor vehicle; the first temperature threshold interval is smaller than the second temperature threshold interval;

[0091] S122, obtaining the ambient temperature of the locomotive; if there is an obstacle and the temperature of the obstacle does not belong to the first temperature threshold interval and the second temperature threshold interval, the obstacle has no vital signs, and the process proceeds to S123 to identify and warn the obstacle without vital signs; if there is an obstacle and the temperature of the obstacle belongs to the first temperature threshold interval, the process proceeds to S125 to identify and warn the obstacle with vital signs; if the temperature of the obstacle belongs to the second temperature threshold interval, the process proceeds to S126 to identify the preceding vehicle;

[0092] S123, judging whether the obstacle is within the locomotive running track range according to the visible light image, if the obstacle is detected within the track range, determining the size, position and speed of the obstacle to facilitate subsequent determination of the obstacle category, and proceeding to S124 to determine the category of the obstacle;

[0093] S124, performing image processing on the visible light image and determining the obstacle category, if the obstacle category determined is the vehicle in front or another vehicle, dynamically tracking and determining the distance between the vehicle and the locomotive, and performing a vehicle alarm according to the analysis result, issuing an audible and visual alarm and feeding back to the locomotive TCMS system for operation processing;

[0094] S125. Determine whether the obstacle is within the locomotive running track range through the visible light image and the thermal imaging image. If the obstacle is detected within the track range, determine the size, position and speed of the obstacle, use the image recognition algorithm to identify the obstacle in the visible light image and the thermal imaging image, and distinguish the type of the obstacle in combination with the acquired temperature of the obstacle, determine whether it is a pedestrian and perform dynamic tracking, judge and analyze the distance between the obstacle and the locomotive, issue an obstacle warning or obstacle alarm according to the analysis result, and send an audible and visual alarm to the locomotive TCMS system for operation and processing;

[0095] S126. Determine whether the preceding vehicle or other vehicles are within the locomotive running track range through visible light images, thermal imaging images and ambient temperature. If so, dynamically track and analyze the distance between the preceding vehicle or other vehicles and the locomotive. Issue an obstacle warning or obstacle alarm based on the analysis results, issue an audible and visual alarm and provide feedback to the locomotive TCMS system for operational processing.

[0096] Setting an obstacle safety warning strategy can quickly provide feedback on whether speed reduction or emergency braking is required when an obstacle is identified, thereby improving the safety of locomotive operation.

[0097] In a specific embodiment, the scheme for performing image processing on the visible light image and determining the obstacle category is:

[0098] S1241: Acquire a visible light image with obstacles obtained by the thermal imaging detector;

[0099] S1242: Preprocessing the visible light image to obtain a preprocessed image; the preprocessing includes but is not limited to grayscale processing, binarization and edge detection;

[0100] S1243: extracting features of obstacles from the preprocessed image using a feature extraction method; the feature extraction algorithm includes but is not limited to HOG, SIFT and SLAM algorithms;

[0101] S1244: using a neural network to classify obstacles according to the extracted obstacle features to obtain obstacle categories; the neural network includes but is not limited to SVM, Random Forest and convolutional neural network algorithms;

[0102] S125. Combining the obstacle data obtained by the radar sensor and the locomotive positioning information, determine the category of the obstacle without vital signs.

[0103] The locomotive or urban rail vehicle runs on rails. In this embodiment, based on a common image recognition algorithm, obstacles without vital signs detected by a visible light probe of a binocular camera or a thermal imaging detector can be detected;

[0104] Obstacles are identified through feature extraction, combined with LiDAR 3D point cloud data for feature analysis, and track information is found. This is used as a benchmark to limit obstacles within the range of the rails, thereby removing the impact of surrounding trees, etc., and output information such as obstacle type, size, and distance. Various interfaces can output alarm signals or send out sound and light alarm prompts to achieve early warning of obstacles without vital signs.

[0105] In a specific embodiment, an image recognition algorithm is used to identify obstacles in the visible light image and the thermal imaging image, and the type of the obstacle is distinguished in combination with the acquired temperature of the obstacle, and a scheme for determining whether it is a pedestrian and performing dynamic tracking is:

[0106] S1251: Preprocessing the visible light image and the thermal imaging image using an image processing algorithm; the preprocessing includes but is not limited to grayscale processing, binarization, and edge detection processing;

[0107] S1252: extracting features of obstacles from the preprocessed image using a feature extraction algorithm; the feature extraction algorithm includes but is not limited to HOG, SIFT, and SLAM algorithms;

[0108] S1253: using a neural network to classify obstacles according to the extracted features; the neural network includes but is not limited to SVM, Random Forest and convolutional neural network algorithms;

[0109] S1254: Using a regression algorithm to locate obstacles based on the extracted features; the regression algorithm includes but is not limited to a linear regression algorithm and a support vector regression algorithm;

[0110] S1255, identifying obstacles with vital signs based on the distance and speed of the obstacles obtained by the radar sensor;

[0111] S1256. Dynamically track the obstacle by combining the distance, speed and direction of the obstacle acquired by the radar sensor, the 3D modeling data and the locomotive positioning information.

[0112] In this embodiment, the Yolo algorithm is used as an example for image processing, and a separate CNN model is used to implement end-to-end target detection. The processing steps are as follows:

[0113] 1. Split the input visible light image into multiple S×S cells, such as 448x448;

[0114] 2. Each cell is responsible for detecting the target whose center point falls within the grid. The specific implementation process is to predict BB bounding boxes and the confidence score of the bounding box for each cell. The so-called confidence score includes two aspects: one is the possibility that the bounding box contains the target, and the other is the accuracy of the bounding box;

[0115] The probability that the bounding box contains the object is recorded as Pr(object). When the bounding box is the background (i.e., does not contain the object), Pr(object) = 0. When the bounding box contains the object, Pr(object) = 1. The accuracy of the bounding box can be characterized by the IOU (intersection over union) of the predicted box and the actual box, so the confidence can be defined as Pr(object)*IOU.

[0116] The size and position of the bounding box can be represented by 4 values: (x, y, w, h), where (x, y) is the center coordinate of the bounding box, and w and h are the width and height of the bounding box; therefore, the predicted value of each bounding box actually contains 5 elements: (x, y, w, h, c), where the first 4 represent the size and position of the bounding box, and the last value is the confidence;

[0117] 3. Each cell implements object recognition and classification and predicts the category probability value, which represents the probability that the target of the bounding box predicted by the cell belongs to each category. These probability values ​​are actually conditional probabilities under the confidence of each bounding box, that is, Pr(classi|object);

[0118] After Yolo uses a convolutional network to extract features, it uses a fully connected layer to obtain the predicted value, that is, the determined obstacle category. The network structure of Yolo is the same as the GooLeNet model, which contains 24 convolutional layers and 2 fully connected layers. For the convolutional layer and the fully connected layer, the Leaky ReLU activation function is used: max(x,0.1x), and the last layer uses a linear activation function.

[0119] By establishing multiple threads, interaction with external devices is achieved. When the detected temperature value is greater than the set first temperature threshold with vital signs, the intruder is identified and distinguished between humans and poultry.

[0120] For intrusive objects such as pedestrians with vital signs or poultry that temporarily break in, thermal imaging detectors can be set up to identify them through visible light heating imaging. Thermal imaging can effectively remove the influence of noise from the surrounding environment and can truly feedback the shape of the intruder, especially when there are overlapping objects. Due to the difference in temperature between different objects, the image recognition algorithm can well identify features such as object boundaries and object volumes, with higher accuracy and more comprehensive obstacle detection than the visible light image algorithm.

[0121] In a specific embodiment, the safety warning strategy includes but is not limited to an obstacle safety warning strategy, a fire safety warning strategy and a locomotive power grid safety warning strategy. The process of using an intelligent computing host to process thermal imaging data is as follows: Figure 4 As shown:

[0122] Perform thermal imaging image and video acquisition, and use the intelligent computing host to determine whether the location of the fault is a locomotive power equipment fault. If it is determined to be a fault, perform fault identification;

[0123] If the intelligent computing host detects an intrusion, it sets a high-frequency acquisition flag, performs high-frequency acquisition flags on image acquisition and video acquisition, saves visible light image / video high-frequency storage and thermal imaging image / video high-frequency storage, starts image recognition, and clears the flag after identifying the obstacle or intrusion;

[0124] At the same time, if you need to increase the brightness of the ambient light, you can also control the fill light.

[0125] Thermal imaging can be used to distinguish objects from lifeless objects by detecting their temperature values, allowing for more accurate identification.

[0126] In a specific embodiment, Figure 5 As shown, fire safety early warning strategies include:

[0127] S21, setting a third temperature threshold interval, the third temperature threshold interval is used to characterize the threshold interval of the combustion temperature, obtaining the ambient temperature of the locomotive and judging whether there is an obstacle in front of the track, the third temperature threshold interval is greater than the second temperature threshold interval;

[0128] S22: If there is an obstacle and the temperature of the obstacle does not belong to the first temperature threshold interval, the second temperature threshold interval and the third temperature threshold interval, the obstacle has no vital signs and the first obstacle safety warning strategy is executed; if there is an obstacle and the temperature of the obstacle is in the first temperature threshold interval or the second temperature threshold interval, the second obstacle safety warning strategy is executed; if there is no obstacle and the detected temperature is in the third temperature threshold interval, enter S23 for fire identification;

[0129] S23. Fire identification and confirmation is carried out through visible light images, thermal imaging images and ambient temperature. The location and distance information of the fire is obtained through the environmental data, 3D modeling data and locomotive positioning information obtained by the radar sensor. The fire spread trend is tracked, a fire alarm is issued, and feedback is given to the locomotive TCMS system for deceleration or parking.

[0130] In this embodiment, the first temperature threshold interval is the normal body temperature of a human or poultry, which is between 34-42 degrees. The temperature can be appropriately raised in high temperature seasons and appropriately lowered in low temperature seasons. The third temperature threshold is above 100 degrees, and whether it is a fire is determined based on this temperature.

[0131] In a specific embodiment, Figure 4 As shown in the figure, the distance between the obstacle and the locomotive is judged and analyzed, and the scheme for issuing obstacle warning or obstacle alarm according to the analysis result is:

[0132] If the distance between the obstacle and the locomotive is getting closer and closer, an obstacle alarm is issued, and the obstacle alarm means that the locomotive needs to brake urgently and sound the horn for warning;

[0133] If the distance between the obstacle and the locomotive gradually increases, an obstacle warning is issued, and the obstacle warning is that the locomotive needs to slow down and sound the horn to warn.

[0134] In this embodiment, the vehicle alarm is similar to the fire alarm. The obstacle position and distance information are obtained through radar and other data. An alarm is given according to the distance value. The distance between the obstacle / intrusion and the locomotive, the distance between the vehicle in front of the locomotive and the locomotive, and the fire situation in front of the locomotive are judged and analyzed. The operator judges whether it is necessary to stop the vehicle for processing based on experience according to the distance value or the current fire intensity, and sounds the horn to warn. If the distance is getting closer or the fire is getting bigger, it is necessary to stop the vehicle for processing, then an alarm message is given, an audible and visual alarm is issued, and the alarm is fed back to the TCMS system; if the distance trend shows that the distance between the obstacle / intrusion and the locomotive is increasing, the trend of the fire is gradually decreasing or does not affect the safe passage of the locomotive, that is, the safety is greater, and an early warning message can be given, an audible and visual alarm is issued, and the alarm is fed back to the TCMS system; the design of the early warning and alarm warning modes can greatly reduce the uncertainty in the operation of the locomotive, warn the intrusion to stay away from the locomotive before danger occurs, and avoid unnecessary danger. At the same time, if the distance is gradually moving away, the safety is greater, and an early warning is issued in the form of speed reduction to ensure the stability of the locomotive.

[0135] In a specific embodiment, thermal imaging can also be used to detect the surrounding environment that affects the safety of locomotive operation and to detect power grids. The solution is:

[0136] Use thermal imaging to detect defects in power equipment and categorize them according to their thermal state, such as general defects, serious defects, critical defects, etc.

[0137] The interpretation methods used for thermal imaging temperature include but are not limited to surface temperature judgment method, similar comparison method and relative temperature difference method;

[0138] The surface temperature determination method comprises:

[0139] Use the regional temperature measurement function of the thermal imaging camera to quickly help determine the surface temperature. Select any area to obtain the average, maximum, and minimum temperature values ​​of the area.

[0140] Use like-for-like comparisons including:

[0141] Use the point temperature measurement function to analyze and compare the temperature differences of corresponding parts of the same group of three-phase equipment, between the same-phase equipment, and between the same type of equipment;

[0142] The relative temperature difference method includes:

[0143] Use the isotherm function to intuitively display different temperature ranges on the screen and quickly discover abnormal temperature differences.

[0144] The structure and materials of power equipment are usually greatly affected by temperature. During normal operation, power equipment will generate a certain amount of heat. However, as power equipment ages or some contact points become rusty or poorly connected, causing increased resistance, excessive current, abnormal voltage, etc., the equipment will generate abnormal heating. Therefore, temperature detection can help understand the operating status of power equipment, help operation and maintenance personnel to promptly detect defects, and achieve more comprehensive locomotive safety warnings.

[0145] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A rail transit safety operation and early warning system, characterized in that: include: Intelligent computing host, radar sensor, environment acquisition module and positioning acquisition module; The radar sensor includes a laser radar sensor and a millimeter wave radar sensor. The laser radar sensor is used to collect 3D point cloud information of the locomotive's surrounding environment, and the millimeter wave radar sensor is used to collect the distance between the locomotive and the objects in the locomotive's surrounding environment, the movement speed and the angle with the locomotive; The environment acquisition module includes a visible light image acquisition device and a thermal imaging image acquisition device, which are used to acquire a visible light image, a thermal imaging image and the temperature of the environment around the locomotive; The positioning acquisition module includes a data acquisition unit and an inertial navigation system; the locomotive data is collected by using the data acquisition unit, and the locomotive positioning information is obtained in combination with the intelligent computing host or the locomotive positioning information is obtained through the inertial navigation system; The intelligent computing host determines whether there is an obstacle based on the visible light image, thermal imaging image and ambient temperature. If it is confirmed, it combines the vital signs of the obstacle and the track position of the obstacle, combines the 3D modeling data, locomotive positioning information and the visible light image and thermal imaging image to determine the distance between the obstacle and the locomotive, the size of the obstacle and the type of the obstacle, builds a safety warning strategy and issues a corresponding level of safety warning; the 3D modeling data is data obtained by 3D modeling of the train track and trackside environment based on the data obtained by the radar sensor.

2. The rail transit safe operation and early warning system according to claim 1, characterized in that: The intelligent computing host determines whether there is an obstacle based on the visible light image, thermal imaging image and ambient temperature. If it is confirmed, the intelligent computing host determines the distance between the obstacle and the locomotive, the size of the obstacle and the type of the obstacle based on the vital signs of the obstacle and the track position of the obstacle, the 3D modeling data, the locomotive positioning information and the visible light image and thermal imaging image, builds a safety warning strategy and issues a safety warning of the corresponding level, including: S11. Determine whether there is an obstacle, the vital signs of the obstacle and the track position of the obstacle based on the visible light image and the thermal imaging image. The track position of the obstacle is to confirm whether the obstacle is within the track range; S12. Setting an obstacle safety warning strategy, and outputting a safety warning of a corresponding level in combination with the obstacle safety warning strategy, wherein the obstacle safety warning strategy is used to characterize the feature mapping relationship between the 3D modeling data, the locomotive positioning information, and the visible light image, the thermal imaging image, and the distance between the obstacle and the locomotive, the size of the obstacle, and the category of the obstacle.

3. The rail transit safe operation and early warning system according to claim 1, characterized in that: The obstacle safety warning strategy includes a first obstacle safety warning strategy and a second obstacle safety warning strategy; The first obstacle safety warning strategy is that if the obstacle has no vital signs and is located within the locomotive track range, the intelligent computing host performs image processing on the visible light image, and combines the data obtained by the radar sensor, the 3D modeling data and the locomotive positioning information to determine the type and size of the obstacle and the distance between the obstacle and the locomotive, identifies the obstacle, and performs obstacle warning or obstacle alarm according to the results of distance judgment and identification, emits an audible and visual alarm and feeds back to the locomotive TCMS system for operation processing; The second obstacle safety warning strategy is that if the obstacle has vital signs or the obstacle is a leading vehicle or other vehicle without vital signs, the intelligent computing host obtains obstacle data based on the visible light image, thermal imaging image, temperature of the obstacle and radar sensor obtained by the environment acquisition module, identifies the obstacle or vehicle with vital signs, and dynamically tracks the obstacle with vital signs in combination with the 3D modeling data and the locomotive positioning information, determines the distance between the obstacle and the locomotive, and performs obstacle warning, obstacle alarm or vehicle alarm based on the results of obstacle tracking and obstacle identification, emits an audible and visual alarm and feeds back to the locomotive TCMS system for operation and processing.

4. The rail transit safe operation and early warning system according to claim 3, characterized in that: The first obstacle safety warning strategy and the second obstacle safety warning strategy include: S121, setting a first temperature threshold interval and a second temperature threshold interval and determining whether there is an obstacle ahead of the track, wherein the first temperature threshold interval is used to characterize a temperature threshold interval of an intruding object with vital signs, and the second temperature threshold interval is used to characterize a temperature threshold interval of an intruding object with a motor vehicle; the first temperature threshold interval is smaller than the second temperature threshold interval; S122, obtaining the ambient temperature of the locomotive; if there is an obstacle and the temperature of the obstacle does not belong to the first temperature threshold interval and the second temperature threshold interval, the obstacle has no vital signs, and the process proceeds to S123 to identify and warn the obstacle without vital signs; if there is an obstacle and the temperature of the obstacle belongs to the first temperature threshold interval, the process proceeds to S125 to identify and warn the obstacle with vital signs; if the temperature of the obstacle belongs to the second temperature threshold interval, the process proceeds to S126 to identify the preceding vehicle; S123, judging whether the obstacle is within the locomotive running track range according to the visible light image, if the obstacle is detected within the track range, determining the size, position and speed of the obstacle to facilitate subsequent determination of the obstacle category, and proceeding to S124 to determine the category of the obstacle; S124, performing image processing on the visible light image and determining the obstacle category, if the obstacle category determined is the vehicle in front or another vehicle, dynamically tracking and determining the distance between the vehicle and the locomotive, and performing a vehicle alarm according to the analysis result, issuing an audible and visual alarm and feeding back to the locomotive TCMS system for operation processing; S125. Determine whether the obstacle is within the locomotive running track range through the visible light image and the thermal imaging image. If the obstacle is detected within the track range, determine the size, position and speed of the obstacle, use the image recognition algorithm to identify the obstacle in the visible light image and the thermal imaging image, and distinguish the type of the obstacle in combination with the acquired temperature of the obstacle, determine whether it is a pedestrian and perform dynamic tracking, judge and analyze the distance between the obstacle and the locomotive, issue an obstacle warning or obstacle alarm according to the analysis result, and send an audible and visual alarm to the locomotive TCMS system for operation and processing; S126. Determine whether the preceding vehicle or other vehicles are within the locomotive running track range through visible light images, thermal imaging images and ambient temperature. If so, dynamically track and analyze the distance between the preceding vehicle or other vehicles and the locomotive. Issue an obstacle warning or obstacle alarm based on the analysis results, issue an audible and visual alarm and provide feedback to the locomotive TCMS system for operational processing.

5. The rail transit safe operation and early warning system according to claim 4, characterized in that: Performing image processing on the visible light image and determining the obstacle category includes: S1241: preprocessing the visible light image to obtain a preprocessed image; S1242: extracting features of obstacles in the preprocessed image using a feature extraction method; S1243: Using a neural network to classify obstacles according to the extracted features to obtain obstacle categories; S1244: Determine the category of the obstacle without vital signs by combining the obstacle data obtained by the radar sensor and the locomotive positioning information.

6. The rail transit safe operation and early warning system according to claim 4, characterized in that: The obstacles in the visible light image and the thermal imaging image are identified using an image recognition algorithm, and the types of obstacles are distinguished in combination with the acquired temperature of the obstacles, and whether they are pedestrians is determined and dynamically tracked, including: S1251: Preprocessing the visible light image and the thermal imaging image using an image processing algorithm; S1252: extracting features of obstacles from the preprocessed image using a feature extraction algorithm; S1253: using a neural network to classify obstacles based on the extracted features; S1254: Using regression algorithm to locate obstacles based on the extracted features; S1255: judging the distance between the obstacle and the locomotive based on the distance, movement speed and angle between the obstacle and the locomotive obtained by the radar sensor, and identifying the obstacle with vital signs; S1256: Dynamically track the obstacle by combining the obstacle data acquired by the radar sensor, the 3D modeling data and the locomotive positioning information.

7. The rail transit safe operation and early warning system according to claim 1, characterized in that: The safety warning strategies include but are not limited to obstacle safety warning strategies, fire safety warning strategies and locomotive power grid safety warning strategies.

8. The rail transit safe operation and early warning system according to claim 7, characterized in that: The fire safety early warning strategy includes: S21, setting a third temperature threshold interval, the third temperature threshold interval is used to characterize the threshold interval of the combustion temperature, obtaining the ambient temperature of the locomotive and judging whether there is an obstacle in front of the track, the third temperature threshold interval is greater than the second temperature threshold interval; S22: If there is an obstacle and the temperature of the obstacle does not belong to the first temperature threshold interval, the second temperature threshold interval and the third temperature threshold interval, the obstacle has no vital signs and the first obstacle safety warning strategy is executed; if there is an obstacle and the temperature of the obstacle is in the first temperature threshold interval or the second temperature threshold interval, the second obstacle safety warning strategy is executed; if there is no obstacle and the detected temperature is in the third temperature threshold interval, enter S23 for fire identification; S23. Fire identification and confirmation is carried out through visible light images, thermal imaging images and ambient temperature. The location and distance information of the fire is obtained through the environmental data, 3D modeling data and locomotive positioning information obtained by the radar sensor. The fire spread trend is tracked, a fire alarm is issued, and feedback is given to the locomotive TCMS system for deceleration or parking.

9. The rail transit safe operation and early warning system according to claim 3, characterized in that: The distance between the obstacle and the locomotive is judged and analyzed, and obstacle warning or obstacle alarm is issued according to the analysis results, including: If the distance between the obstacle and the locomotive is getting closer and closer, an obstacle alarm is issued, and the obstacle alarm means that the locomotive needs to brake urgently and sound the horn for warning; If the distance between the obstacle and the main engine gradually increases, an obstacle warning is issued, and the obstacle warning is that the locomotive needs to slow down and sound the horn to warn.

Images