Over-the-horizon obstacle sensing integrated system for high-speed intelligent motor train unit

Abstract

The invention belongs to the technical field of high-speed intelligent motor train unit sensing, and discloses a high-speed intelligent motor train unit beyond-visual-range obstacle sensing integrated system, which comprises a road end sensing unit used for acquiring road end video data along a railway; the train end sensing unit is used for acquiring train end sensing data of the motor train unit, and the train end sensing data comprises train end video data and laser point cloud data; the sensing unit is used for performing target detection on the road end video data and the vehicle end sensing data according to a target detection model to obtain a road end sensing result and a vehicle end sensing result, and performing risk judgment based on the road end sensing result and the vehicle end sensing result to obtain a risk state; and the early warning unit is used for displaying an early warning result and visual information on the motor train unit according to the risk state. According to the invention, the adaptability and stability of the high-speed intelligent motor train unit to complex traffic scenes are greatly improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of high-speed intelligent EMU perception, and particularly relates to an integrated system for perceiving ultra-long-range obstacles of high-speed intelligent EMUs. Background Art

[0002] With the remarkable progress of the high-speed railway network, breaking through the key technologies of high-speed intelligent EMU autonomous driving has become the key to promoting the intelligentization and autonomy process of China's high-speed rail. High-speed EMUs are one of the main forms of domestic railway transportation at present. However, due to the complexity of the environment and the instability of external information during the operation of high-speed trains, it is impossible to accurately and effectively perceive and predict the safety status ahead. Therefore, the research on the key technologies of the active safety system of high-speed intelligent EMUs is of great significance to the development of the railway transportation industry. Summary of the Invention

[0003] The purpose of the present invention is to provide an integrated system for perceiving ultra-long-range obstacles of high-speed intelligent EMUs to solve the problems existing in the above-mentioned prior art.

[0004] To achieve the above purpose, the present invention provides an integrated system for perceiving ultra-long-range obstacles of high-speed intelligent EMUs, including:

[0005] A roadside sensing unit for acquiring roadside video data along the railway;

[0006] A vehicle-end sensing unit for acquiring vehicle-end sensing data of the EMU, where the vehicle-end sensing data includes vehicle-end video data and laser point cloud data;

[0007] A perception unit for performing object detection on the roadside video data and the vehicle-end sensing data according to an object detection model to obtain a roadside perception result and a vehicle-end perception result, and performing risk discrimination based on the roadside perception result and the vehicle-end perception result to obtain a risk status;

[0008] An early warning unit for displaying an early warning result and visualization information on the EMU according to the risk status.

[0009] Optionally, the roadside sensing unit specifically includes:

[0010] A roadside camera arrangement module for arranging a plurality of roadside cameras, and each of the roadside cameras is evenly distributed along the railway at a preset interval;

[0011] A roadside video acquisition module for acquiring roadside video data by using each of the roadside cameras.

[0012] Optionally, the vehicle-end sensing unit specifically includes:

[0013] The equipment layout module is used to arrange the car-end camera and lidar at the front end of the multiple unit train;

[0014] The car-end sensing data acquisition module is used to obtain the car-end video data and lidar point cloud data in front of the multiple unit train by using the car-end camera and the lidar.

[0015] Optionally, the sensing unit specifically includes:

[0016] The target detection module is used to perform target detection on the road-end video data and the car-end sensing data according to the target detection model to obtain the road-end sensing result and the car-end sensing result;

[0017] The sensing result fusion and judgment module is used to fuse the road-end and car-end sensing results and determine whether there is a risk. When both the road-end sensing result and the car-end sensing result are that no obstacle information is detected, it outputs a risk-free state; when the road-end sensing result and the car-end sensing result are not both that no obstacle information is detected, it outputs a risk state.

[0018] Optionally, the target detection module specifically includes:

[0019] The training module is used to construct an initial target detection model and train the initial target detection model to obtain a trained target detection model;

[0020] The model application module is used to use the trained target detection module to perform the target detection task on each sensing data.

[0021] Optionally, the training module specifically includes:

[0022] The training data acquisition sub-module is used to acquire training data. The training data includes sensing training data and the corresponding target detection results. The sensing training data includes road-end video training data, car-end video training data, and point cloud training data. The target detection results include the target category and the position box coordinates;

[0023] The model training and optimization sub-module is used to input the training data into the initial target detection model for prediction and classification, and train with the goal of minimizing the loss between the initial training result after prediction and classification and the target detection result corresponding to the sensing training data to obtain a trained target detection module.

[0024] Optionally, it further includes a communication unit, and the communication unit is used to make each unit communicate and connect by using the ROS communication architecture.

[0025] The technical effect of the present invention is:

[0026] The present invention can effectively increase the driving perception distance of existing high-speed EMUs, and at the same time greatly improve the adaptability and stability of high-speed intelligent EMUs in the face of complex traffic scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0028] The drawings forming a part of this application are used to provide a further understanding of this application. The schematic embodiments of this application and their descriptions are used to explain this application and do not constitute an improper limitation of this application. In the drawings:

[0029] Figure 1 is a schematic flowchart in an embodiment of the present invention;

[0030] Figure 2 is a sensor layout diagram in an embodiment of the present invention;

[0031] Figure 3 is a schematic flowchart of the specific steps in an embodiment of the present invention;

[0032] Figure 4 is a visualization warning schematic diagram of an on-vehicle terminal in an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0033] Now, various exemplary embodiments of the present invention will be described in detail. This detailed description should not be considered as a limitation of the present invention, but rather as a more detailed description of certain aspects, characteristics, and implementation schemes of the present invention.

[0034] It should be understood that the terms described in the present invention are only for describing specific embodiments and are not used to limit the present invention. In addition, for the numerical ranges in the present invention, it should be understood that each intermediate value between the upper and lower limits of the range is also specifically disclosed. Each intermediate value within any stated value or stated range, as well as each smaller range between any other stated value or intermediate value within the stated range, is also included in the present invention. The upper and lower limits of these smaller ranges can be independently included or excluded from the range.

[0035] Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. Although only preferred methods are described in this invention, any method similar or equivalent to those described herein may also be used in the practice or testing of this invention. All documents mentioned in this specification are incorporated by reference to disclose and describe the methods related to the said documents. In case of conflict with any incorporated document, the content of this specification shall prevail.

[0036] Without departing from the scope or spirit of the present invention, various modifications and variations can be made to the specific embodiments of the description of the present invention, which are obvious to those skilled in the art. Other embodiments obtained from the description of the present invention are obvious to those skilled in the art. The description and examples of this application are merely exemplary.

[0037] Regarding the use of "comprising", "including", "having", "containing", etc. in this article, they are all open-ended terms, meaning including but not limited to.

[0038] It should be noted that, without conflict, the embodiments in this application and the features in the embodiments can be combined with each other. The following will refer to the drawings and combine with the embodiments to detail this application.

[0039] As Figure 1 - Figure 4 As shown, in this embodiment, a high-speed intelligent multiple unit over-the-horizon obstacle perception integration system is provided, including: a roadside sensing unit for acquiring roadside video data along the railway; a vehicle-end sensing unit for acquiring vehicle-end sensing data of the multiple unit, where the vehicle-end sensing data includes vehicle-end video data and laser point cloud data; a perception unit for performing target detection on the roadside video data and the vehicle-end sensing data according to a target detection model to obtain a roadside perception result and a vehicle-end perception result, and performing risk discrimination based on the roadside perception result and the vehicle-end perception result to obtain a risk state; and a warning unit for displaying a warning result and visualization information on the multiple unit according to the risk state.

[0040] This embodiment discloses a high-speed intelligent multiple unit over-the-horizon obstacle perception integration system. Aiming at the problems of complex driving environment and unpredictability of risk occurrence of high-speed intelligent multiple units, with the goal of promoting the intelligentization and safety development of high-speed intelligent multiple units, this embodiment performs over-the-horizon perception by integrating multi-sensors at the vehicle end and the roadside, obtains the perception information in front of the high-speed intelligent multiple unit in real time, and performs risk warning according to the detection result.

[0041] The high-speed intelligent multiple unit over-the-horizon obstacle perception integration system of this embodiment mainly consists of a roadside camera, a roadside computing unit, a vehicle-end camera, a vehicle-end lidar, a vehicle-end computing unit, and a vehicle-end terminal.

[0042] Train the algorithms and deploy: Train the image object detection algorithm and the lidar object detection algorithm. The datasets required for training the algorithms consist of publicly available network datasets and self-collected and labeled datasets. Among them, the self-collected image datasets are manually labeled using Labelme, and the self-collected lidar point cloud datasets are manually labeled using SUSTechPoints. Use the datasets to train the algorithms and deploy the devices separately.

[0043] The perception of roadside sensors is mainly completed by roadside cameras. A roadside camera is arranged at a fixed distance along the running track of the EMU, and a roadside computing unit is configured for each roadside camera. The roadside camera is responsible for perceiving the distance between its position and the position of the next roadside camera, and sending the perceived image information to the roadside computing unit to call the image object detection algorithm for processing.

[0044] The perception of vehicle-mounted sensors is mainly completed by vehicle-mounted cameras and vehicle-mounted lidars. The vehicle-mounted cameras and vehicle-mounted lidars are arranged at the front end of the EMU, responsible for perceiving a fixed distance in front of the EMU, and sending the perceived image information and point cloud information to the vehicle-mounted computing unit to call the image object detection algorithm and the lidar object detection algorithm for processing.

[0045] Vehicle-road collaborative perception mainly sends the processing results of the roadside computing unit and the vehicle-mounted computing unit to the vehicle-mounted terminal to achieve vehicle-road collaborative perception of the high-speed intelligent EMU.

[0046] The early warning of the vehicle-mounted terminal is to receive the perception results of the roadside and vehicle-mounted computing units at the vehicle-mounted terminal, obtain the final perception result, and display the warning information on the interface.

[0047] This embodiment can effectively improve the driving perception distance of the existing high-speed EMU, and at the same time greatly improve the adaptability and stability of the high-speed intelligent EMU in the face of complex traffic scenarios.

[0048] The specific implementation steps of the over-the-horizon obstacle perception integration system for the high-speed intelligent EMU provided in this embodiment include:

[0049] Step 1: Train the object detection algorithm and deploy the device: In this embodiment, data collection is carried out at key locations along the outside railway line of Yanshan University in Qinhuangdao City and near the railway station. Among them, the image datasets are data-labeled using labelme, and the point cloud datasets are labeled using SUSTechPoints, and combined with the publicly available network datasets to obtain the datasets required for algorithm training.

[0050] This embodiment is developed using the Python3.8+PyTorch framework, and the hardware configuration parameters are set as shown in Table 1:

[0051] Table 1 Hardware configuration parameters

[0052]

[0053] Train the YOLOv5 algorithm using an image dataset and deploy roadside computing units and in-vehicle computing units for processing raw camera image information. Train the PillarNet algorithm using a point cloud dataset and deploy in-vehicle computing units for processing raw lidar point cloud information.

[0054] Step 2: Deploy a roadside camera at fixed intervals along the running track of the multiple unit train. The roadside camera is responsible for sensing the distance between its position and the position of the next roadside camera and obtaining roadside video data. The in-vehicle camera and the in-vehicle lidar are deployed at the front end of the multiple unit train, responsible for sensing a fixed distance in front of the multiple unit train and obtaining in-vehicle video data and point cloud data.

[0055] Step 3: In this embodiment, ROS is used as the overall communication architecture. The roadside camera publishes the image data to a specified ROS topic, where each camera publishes to an independent topic, and converts the camera data into the ROS standard Image message format. The roadside computing unit subscribes to the image topic corresponding to the camera, uses the cv_bridge library to convert the ROS Image message into the OpenCV format, and calls the YOLOv5 object detection algorithm for processing to obtain detection results such as the target category and the coordinates of the position box.

[0056] The in-vehicle camera publishes the image data to a specified ROS topic, converts the camera data into the ROS standard Image message format. The in-vehicle computing unit subscribes to the image topic corresponding to the camera, uses the cv_bridge library to convert the ROS Image message into the OpenCV format, and calls the YOLOv5 object detection algorithm for processing to obtain detection results such as the target category and the coordinates of the position box.

[0057] The in-vehicle lidar publishes the point cloud data to a specified ROS topic, converts the lidar data into the ROS standard PointCloud2 message format. The in-vehicle computing unit subscribes to the corresponding topic, converts the ROS PointCloud2 message into a point cloud format that can be processed by the algorithm, and calls the PillarNet algorithm for processing to obtain information such as the position, shape, and category of the target.

[0058] Step 4: Obtain the perception results of the roadside and in-vehicle respectively. If obstacle information is detected, it is judged as a risky state. If there is no obstacle information, it is judged as a risk-free state.

[0059] Step 5: Each roadside computing unit and vehicle computing unit publish the roadside and vehicle perception results in the form of ROS topics respectively, which are subscribed by the in-vehicle terminal to obtain the final perception result. If both the roadside perception result and the vehicle perception result are risk-free, the final perception result is risk-free; otherwise, it is determined to be at risk.

[0060] Step 6: Use pyqt6 to draw a visualization interface to implement in-vehicle terminal visual warning. If the final perception result is at risk, the warning result and visualization information will be displayed on the terminal, such as Figure 4 The in-vehicle terminal visual warning schematic diagram of this embodiment is shown as follows.

[0061] The above is only a preferred specific embodiment of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed in the present application should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A long-range obstacle perception integration system for a high-speed intelligent multiple unit, characterized in that Including: A road-end sensing unit for acquiring road-end video data along the railway; A vehicle-end sensing unit for acquiring vehicle-end sensing data of the multiple unit train, where the vehicle-end sensing data includes vehicle-end video data and lidar point cloud data; A perception unit for performing object detection on the road-end video data and the vehicle-end sensing data according to an object detection model to obtain a road-end perception result and a vehicle-end perception result, and performing risk discrimination based on the road-end perception result and the vehicle-end perception result to obtain a risk status; An early warning unit for displaying an early warning result and visualization information on the multiple unit train according to the risk status.

2. The ultra-long-distance obstacle perception integration system for high-speed intelligent multiple units according to claim 1, characterized in that The road-end sensing unit specifically includes: A road-end camera arrangement module for arranging a plurality of road-end cameras, and each of the road-end cameras is evenly distributed along the railway at a preset interval; A road-end video acquisition module for acquiring road-end video data by using each of the road-end cameras.

3. The integrated system for ultra-long-range obstacle perception of a high-speed intelligent multiple unit according to claim 1, wherein The vehicle-end sensing unit specifically includes: A device arrangement module for arranging a vehicle-end camera and a lidar at the front end of the multiple unit train; A vehicle-end sensing data acquisition module for acquiring vehicle-end video data and lidar point cloud data in front of the multiple unit train by using the vehicle-end camera and the lidar.

4. The ultra-long-range obstacle perception integration system for high-speed intelligent multiple units according to claim 1, characterized in that The perception unit specifically includes: An object detection module for performing object detection on the road-end video data and the vehicle-end sensing data according to an object detection model to obtain a road-end perception result and a vehicle-end perception result; A perception result fusion and judgment module for fusing the road-end and vehicle-end perception results and discriminating whether there is a risk. When both the road-end perception result and the vehicle-end perception result are that no obstacle information is detected, a risk-free status is output; when the road-end perception result and the vehicle-end perception result are not both that no obstacle information is detected, a risk status is output.

5. The integrated system for ultra-long-range obstacle perception of a high-speed intelligent multiple unit according to claim 4, characterized in that, The object detection module specifically includes: A training module for constructing an initial object detection model and training the initial object detection model to obtain a trained object detection model; A model application module for using the trained object detection module to perform the object detection task of each sensing data.

6. The integrated system for ultra-long-range obstacle perception of a high-speed intelligent multiple unit according to claim 5, wherein The training module specifically includes: A training data acquisition sub-module for acquiring training data, where the training data includes sensing training data and corresponding object detection results. The sensing training data includes road-end video training data, vehicle-end video training data, and point cloud training data, and the object detection results include object categories and position box coordinates; A model training and optimization sub-module for inputting the training data into the initial object detection model for prediction and classification, and training with the goal of minimizing the loss between the initial training result after prediction and classification and the object detection result corresponding to the sensing training data to obtain a trained object detection module.

7. An over-the-horizon obstacle perception integration system for a high-speed intelligent multiple unit, as claimed in claim 1, wherein It also includes a communication unit, and the communication unit is used to communicatively connect each unit by using a ROS communication architecture.

Images