Real-time track obstacle detection method and system and medium

Abstract

The invention discloses a real-time track obstacle detection method and system and a medium. The method comprises the following steps: acquiring a visual image of a track area; segmenting the visual image by using a lightweight target segmentation model to obtain a single-frame track line, an obstacle and the category of the obstacle; according to the single-frame track line, a final track area is obtained based on dynamic multi-frame track area fitting; judging whether the obstacle is in the final track area or not, and if yes, obtaining a detection frame of the obstacle in the visual image; and according to the obtained detection frame of the obstacle in the visual image, projecting the obstacle detection frame in the visual image into the radar point cloud data to obtain an obstacle point cloud range frame, and according to the obstacle point cloud range frame, calculating the distance from the obstacle to the train head. According to the invention, real-time detection and distance measurement can be carried out on the track obstacle, and the track area can be accurately determined in the image, so that whether the obstacle is in the track area or not can be judged, and a more accurate obstacle distance measurement result can be obtained.

Description

Technical Field

[0001] The present invention relates to the technical field of track obstacle detection, and in particular to a real-time track obstacle detection method, system and medium. Background Art

[0002] Track obstacle detection is the foundation and important guarantee for the long-term safe and stable operation of urban rail transit. Track limit intrusion, tools accidentally left behind by staff, fallen rocks and objects falling into subway tracks, line hanging, etc., any small change in details may cause major accidents. As a kind of public rail transit tool with large passenger capacity and high frequency operation, trams and subways will have immeasurable consequences and cause great losses to people's lives and property safety once an accident occurs.

[0003] Therefore, achieving efficient, accurate, detailed and real-time track obstacle detection can effectively eliminate possible safety hazards, ensure the stable operation of rail transit vehicles and avoid accidents.

[0004] At present, most rail obstacle detections rely on manual work. According to a survey, there are more than 10,000 safety inspectors in Beijing's rail transit. Safety inspectors carry equipment and perform inspections in the huge rail network, which is time-consuming, labor-intensive, and inefficient. In addition, manual inspections cannot be performed during the operation of rail transit vehicles, have poor real-time performance, and can never eliminate safety hazards caused by subjective negligence.

[0005] Nowadays, some companies have cooperated with rail transit departments to develop intelligent rail obstacle detection robots, which are equipped with a variety of sensors and can effectively detect limit intrusions and foreign objects in the track. They cover the detection range of traditional manual detection and reduce the work pressure and intensity of rail inspection personnel. They are used in large subway networks such as Beijing Metro, Xi'an Metro, and Chengdu Metro.

[0006] However, the rail intelligent obstacle detection robot also has obvious defects. The most prominent point is that the rail intelligent obstacle detection robot needs to rely on existing tracks or independent tracks (such as top-mounted rails) to operate. The method of relying on existing tracks can only be carried out after the rail transit vehicle is shut down, and the method of using independent tracks requires high construction and operation and maintenance costs. In addition, the rail intelligent obstacle detection robot is generally large in size, has poor maneuverability and flexibility, and is inconvenient to install and deploy.

[0007] Another obstacle detection solution is to combine multiple sensors such as cameras and lidar into a detection module ( Figure 1 ) and connect to a dedicated host ( Figure 2), forming an active obstacle detection system for trains, which is installed at the front of the train to detect intrusions into the front limit of the train, foreign objects on the track, etc. The advantage of this solution is that the module is always fixed at the front of the train for on-board detection, with strong real-time performance and stable operation. The active obstacle detection system has become one of the essential functions of smart trains, effectively improving the reliability and safety of line operations and reducing operation and maintenance costs.

[0008] At present, the active obstacle detection system of trains mostly uses visual sensors + radars for obstacle detection and distance estimation. The visual sensors are responsible for obstacle detection, and the radars are responsible for obstacle distance estimation and size estimation. In terms of visual obstacle detection, solutions based on traditional image processing or deep learning are used. The solutions based on traditional image processing have high requirements for scenes, poor generalization ability and robustness, and are difficult to meet high real-time requirements, so they are rarely used. The solutions based on deep learning usually first obtain the track line and track ROI area through edge detection, Hoffman line detection or target detection, and use the target detection model to detect obstacles in the ROI area. This detection method cannot accurately determine the edge contour of the track area and is prone to misjudgment. Some studies have also proposed using instance segmentation for track identification and obstacle detection. This solution can obtain the contours of the track and obstacles more accurately, but it does not consider the problem of abnormal detection of track lines and track contours due to obstacle occlusion, and there is a certain risk of misjudgment.

[0009] In view of this, this application is hereby filed. Summary of the invention

[0010] The present invention aims to provide a real-time detection method, system and medium for track obstacles, aiming to effectively solve the problem of inaccurate detection of track lines and obstacle contours in the track obstacle detection method, the problem of abnormal detection of track areas due to obstacle occlusion, track misdetection, etc., and the problem of inaccurate obstacle ranging due to the time difference between camera and radar data acquisition.

[0011] The present invention is achieved through the following technical solutions:

[0012] In a first aspect, the present invention provides a method for real-time detection of track obstacles, the method comprising:

[0013] Acquire a visual image of the track area; the visual image is a visual image in front of the train during the train's movement;

[0014] A lightweight target segmentation model is used to segment the visual image to obtain single-frame track lines, obstacles, and obstacle categories;

[0015] According to the single-frame track line, the final track area is obtained based on the dynamic multi-frame track area fitting; the dynamic multi-frame track area fitting is to fit the track area based on the track line detection results of multiple frames of images;

[0016] Determine whether the obstacle is within the final track area. If not, return to continue acquiring the visual image and radar point cloud data of the track area; if so, obtain the detection box of the obstacle in the visual image;

[0017] According to the obtained obstacle detection frame in the visual image, the obstacle detection frame in the visual image is projected into the radar point cloud data to obtain the obstacle point cloud range frame, the obstacle point cloud range frame is dynamically adjusted, and the preliminary distance from the obstacle to the train front is calculated. Finally, the distance correction is performed to obtain the final distance from the obstacle to the train front.

[0018] Furthermore, the lightweight target segmentation model adopts the yolov5-tiny-seg instance segmentation model. The yolov5-tiny-seg instance segmentation model replaces the C3 structure of the feature extraction network Backbone in the traditional yolov5s-seg network with the Ghost structure, reduces the network depth, and then adds a CBAM attention module after the feature extraction network Backbone to fuse multi-scale features.

[0019] Furthermore, according to the single-frame track line, based on the dynamic multi-frame track area fitting, the final track area is obtained, including:

[0020] Calculate the track outer contour points according to the track line detected by the current image frame and the track lines detected by the previous N adjacent image frames of the current image frame, construct a track outer contour point set based on the track outer contour points, perform point set clustering to exclude discrete abnormal points, and obtain a clustered point set;

[0021] Based on the clustered point set, point set convex hull and contour detection are performed to obtain the final track area fitted by multi-frame track detection results.

[0022] Furthermore, the N value in the N adjacent image frames before the current image frame is dynamically adjusted according to the running state of the train, including:

[0023] When the train running speed is greater than 80 km / s, the track area changes rapidly, and N=1~2 is taken, that is, the current frame and the previous one or two frames of the current image frame are used to fit the track area;

[0024] When the train running speed is less than 80 km / s, the track area changes relatively slowly, and N=3-4 is taken, that is, the current frame and the first three to four frames of the current image frame are used to fit the track area;

[0025] When the train is in a turning state, the track area changes the fastest, and N=0-1 is taken, that is, the current frame is directly used, or the current frame and the previous frame of the current image frame are used to fit the track area.

[0026] Further, determining whether the obstacle is within the final track area includes:

[0027] According to the detected obstacle detection frame and the final track area, relative position judgment is performed; relative position judgment includes:

[0028] Take the left and right quarter points of the lower edge of the obstacle detection frame for relative position judgment. If one or both of the two quarter points fall within the track area, the obstacle is considered to exist in the final track area.

[0029] If the obstacle is within the final track area, the laser radar is used for distance measurement, and subsequent operations such as display and alarm are performed; if the obstacle is outside the final track area, no subsequent operations are performed.

[0030] Furthermore, according to the obtained obstacle detection frame in the visual image, the obstacle detection frame in the visual image is projected into the radar point cloud data to obtain the obstacle point cloud range frame, the obstacle point cloud range frame is dynamically adjusted, and the preliminary distance from the obstacle to the train head is calculated, and finally the distance correction is performed to obtain the final distance from the obstacle to the train head, including:

[0031] According to the obtained position of the obstacle in the visual image, the obstacle detection frame in the visual image is projected onto the radar point cloud data to obtain the initial point cloud range frame of the obstacle;

[0032] According to the running status of the train, the initial point cloud range frame of the obstacle is dynamically adjusted to obtain the final point cloud range frame of the obstacle;

[0033] Based on the final point cloud range frame, background point cloud is excluded and the preliminary distance of obstacles is calculated;

[0034] The preliminary distance to the obstacle is corrected according to the train running speed distance to obtain the final obstacle distance calculation result.

[0035] Furthermore, based on the final point cloud range frame, the background point cloud is excluded and the preliminary distance of the obstacle is calculated, including:

[0036] Calculate the mean of the point cloud depths in the final point cloud range box, retain points with depths less than the mean, and exclude points with depths greater than the mean;

[0037] The retained point cloud is clustered, and the largest point set is taken as the actual position of the obstacle. The points in a preset range with a smaller depth in this point set are taken as the surface position of the obstacle. The mean depth of the points in the preset range is calculated as the preliminary distance of the obstacle.

[0038] Furthermore, the corrected expression is:

[0039] D=d–s

[0040] s=vt

[0041] In the formula, D is the exact distance between the obstacle and the train at the current moment (in meters); d is the preliminary distance of the obstacle (in meters); s is the distance the train moves forward during the overall delay time (in meters); v is the current running speed of the train (in meters / second); t is the overall delay of the obstacle's preliminary distance moment relative to the current time of the train (in seconds), including the radar data acquisition delay and program running time.

[0042] In a second aspect, the present invention further provides a track obstacle real-time detection system, the system comprising:

[0043] An acquisition unit is used to acquire a visual image of a track area; the visual image is a visual image in front of the train during the train's movement;

[0044] A segmentation unit is used to segment the visual image using a lightweight target segmentation model to obtain single-frame track lines, obstacles, and obstacle categories;

[0045] The track area dynamic fitting unit is used to obtain the final track area based on the single-frame track line and the dynamic multi-frame track area fitting; the dynamic multi-frame track area fitting is to fit the track area based on the track line detection results of multiple frames of images;

[0046] The obstacle position determination unit is used to determine whether the obstacle is within the final track area. If not, it returns to continue to obtain the visual image and radar point cloud data in front of the train; if so, it obtains the detection frame of the obstacle in the visual image;

[0047] The obstacle distance calculation unit is used to project the obstacle detection frame in the visual image into the radar point cloud data to obtain the obstacle point cloud range frame according to the obtained obstacle detection frame in the visual image, dynamically adjust the obstacle point cloud range frame, calculate the preliminary distance from the obstacle to the front of the train, and finally perform distance correction to obtain the final distance from the obstacle to the front of the train.

[0048] In a third aspect, the present invention further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned real-time detection method for track obstacles is implemented.

[0049] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0050] 1. The present invention provides a real-time detection method, system and medium for track obstacles. The present invention can detect and measure the distance of track obstacles in real time, accurately determine the track area in the image, thereby judging whether the obstacle is in the track area, and obtain a more accurate obstacle distance measurement result by dynamically adjusting the point cloud range and distance correction method for calculating the obstacle distance.

[0051] 2. The present invention provides a real-time detection method, system and medium for track obstacles, proposes a solution for applying the instance segmentation model to the real-time obstacle detection work in subways, and proposes a lightweight target segmentation model, namely the yolov5-tiny-seg instance segmentation model, which is very suitable for deployment in edge devices such as train active obstacle detection systems to ensure the accuracy and real-time performance of obstacle detection.

[0052] 3. The present invention provides a real-time detection method, system and medium for track obstacles, which are based on dynamic multi-frame track area fitting of the train running status, and effectively avoid the problem of track area detection errors caused by track occlusion, track false detection, missed detection and the like.

[0053] 4. The present invention provides a real-time detection method, system and medium for track obstacles, which effectively improves the accuracy of track obstacle distance calculation based on dynamically adjusted point cloud range frame calculation and dynamic distance correction. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of this application, and do not constitute a limitation of the embodiments of the present invention. In the drawings:

[0055] Figure 1 This is a schematic diagram of the camera and lidar detection module;

[0056] Figure 2 Schematic diagram of a high computing power host dedicated to the detection module;

[0057] Figure 3 This is a flow chart of a real-time detection method for track obstacles of the present invention;

[0058] Figure 4 A detailed flow chart of a real-time detection method for track obstacles of the present invention;

[0059] Figure 5 Track detection is blocked by obstacles (tracks blocked by obstacles cannot be detected);

[0060] Figure 6 The track area is drawn (the track area does not contain obstacles, causing misjudgment);

[0061] Figure 7 The track area drawn by the present invention after multi-frame fitting (the track area effectively includes obstacles);

[0062] Figure 8 This is the single-frame track detection result (false detection occurred);

[0063] Fig. 9 The track area drawn based on the single-frame detection results (including the non-track area);

[0064] Fig.10 The track area drawn by the present invention after multi-frame fitting (accurately including the track area);

[0065] Fig.11 Schematic diagram of obstacle image detection frame and point cloud range frame of the present invention (the implementation frame is the image detection frame, and the dotted frame is the point cloud range frame);

[0066] Fig.12 This is a structural block diagram of a track obstacle real-time detection system of the present invention. DETAILED DESCRIPTION

[0067] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with embodiments and drawings. The exemplary embodiments of the present invention and their description are only used to explain the present invention and are not intended to limit the present invention.

[0068] Example 1

[0069] like Figure 3 and Figure 4 As shown, the present invention provides a real-time detection method for track obstacles, the method comprising:

[0070] Step 1, obtaining a visual image of the track area; the visual image is a visual image in front of the train during the train's movement;

[0071] Step 2: Use a lightweight target segmentation model to segment the visual image to obtain single-frame track lines, obstacles, and obstacle categories;

[0072] Specifically, the lightweight target segmentation model uses the yolov5-tiny-seg instance segmentation model to detect track lines and possible obstacles;

[0073] The yolov5-tiny-seg instance segmentation model replaces the C3 structure of the feature extraction network Backbone in the traditional yolov5s-seg network with the Ghost structure, and reduces the network depth, so that the final model obtained by training is only 1.5MB in size. A CBAM attention module is added after the feature extraction network Backbone to help the network better integrate multi-scale features and provide more accurate and targeted feature expressions, so that the model detection accuracy does not show a significant decrease compared to the yolov5s-seg network. It is very suitable for deployment in edge devices such as the train active obstacle detection system to ensure the accuracy and real-time performance of obstacle detection.

[0074] Step 3, according to the single-frame track line, based on the dynamic multi-frame track area fitting, the final track area is obtained; the dynamic multi-frame track area fitting is to fit the track area by the track line detection results of multiple frames of images;

[0075] Considering that obstacles in the track area will block the track to a certain extent, some non-obstructive track infrastructure may also affect the detection of track lines, resulting in missing or interrupted track line detection ( Figure 5 ), which seriously affects the judgment of whether the obstacle is within the track area ( Figure 6 ). And the track line detection based on independent frames occasionally has some abnormal detections ( Figure 8 ), resulting in abnormal track area detection ( Fig. 9 ), which will also affect the judgment of whether the obstacle is within the track area.

[0076] Since the relative position of the track line is relatively fixed in a short period of time during the operation of the train, it can be considered that the track area detected in several adjacent frames of continuous images is relatively fixed and supports each other. Therefore, the present invention is designed to fit the track area by the track line detection results of multiple frames of images, that is, dynamic multi-frame track area fitting.

[0077] Specifically, dynamic multi-frame track area fitting includes:

[0078] Calculate the track outer contour points according to the track line detected by the current image frame and the track lines detected by the previous N adjacent image frames of the current image frame, construct a track outer contour point set based on the track outer contour points, perform point set clustering to exclude discrete abnormal points, and obtain a clustered point set;

[0079] Based on the clustered point set, the point set convex hull and contour detection are performed to obtain the final track area fitted by the multi-frame track detection results ( Figure 7 )( Fig.10 (the area framed in red).

[0080] As a further implementation, the N value in the first N adjacent image frames of the current image frame is dynamically adjusted according to the train running status. In theory, the larger the N value, the better the track area fitting effect, but the dynamic changes of the track area during the train running process need to be considered. The value of N is as follows:

[0081] When the train speed is greater than 80 km / h, the track area changes rapidly, and N=1-2 is taken, that is, the current frame and one to two frames before the current image frame are used to fit the track area;

[0082] When the train running speed is less than 80 km / h, the track area changes relatively slowly, so N=3-4, that is, the current frame and the first three to four frames of the current image frame are used to fit the track area;

[0083] When the train is in a turning state, the track area changes the fastest, and N=0-1 is taken, that is, the current frame is directly used, or the current frame and the previous frame image are used to fit the track area.

[0084] Step 4: determine whether the obstacle is within the final track area. If not, return to continue acquiring the visual image and radar point cloud data of the track area; if so, obtain the detection box of the obstacle in the visual image;

[0085] Specifically, judging whether the obstacle is within the final track area includes:

[0086] According to the detected obstacle detection frame and the final track area, relative position judgment is performed; relative position judgment includes:

[0087] Take the left and right quarter points of the lower edge of the obstacle target frame for relative position judgment. If one or both of the two quarter points fall within the track area, the obstacle is considered to exist in the final track area.

[0088] If the obstacle is within the final track area, the laser radar is used for distance measurement, and subsequent operations such as display and alarm are performed; if the obstacle is outside the final track area, no subsequent operations are performed.

[0089] Step 5: According to the obtained position of the obstacle in the visual image, the obstacle detection frame in the visual image is projected into the radar point cloud data to obtain the obstacle point cloud range frame, the obstacle point cloud range frame is dynamically adjusted, and the preliminary distance from the obstacle to the front of the train is calculated. Finally, the distance correction is performed to obtain the final distance from the obstacle to the front of the train.

[0090] This is because in terms of obstacle distance measurement, laser radar point clouds are currently used for distance calculation. The conventional solution is to project the obstacle detection frame on the image into the point cloud to obtain the point cloud range frame of the obstacle, and calculate the distance of the obstacle by taking the depth mean and other methods. This solution does not take into account the time delay problem of radar point cloud acquisition. Due to the high speed of train operation, even a small delay may affect the obstacle distance calculation. In addition, if all point clouds within the obstacle point cloud range frame are used directly, the amount of calculation will be large, and there will be point clouds of background objects in the frame, which will affect the accuracy and real-time performance of the obstacle distance calculation.

[0091] Therefore, the design process of step 5 of the present invention is as follows:

[0092] 5.1, according to the obtained position of the obstacle in the visual image, the obstacle detection frame in the visual image is projected onto the radar point cloud data to obtain the initial point cloud range frame of the obstacle, and the distance from the obstacle to the front of the train can be further calculated.

[0093] 5.2. Generally, the speed at which the laser radar acquires point cloud data is slower than the speed at which the camera acquires images, and there is a time difference between the two. Since the train is always moving forward when running on the track, the range of obstacles in the image acquired by the detection device at the same time will be larger than the range of obstacles in the radar point cloud ( Fig.11 ), if the obstacle range frame in the radar point cloud is directly calculated based on the projection of the obstacle detection frame in the image, it will cause certain errors.

[0094] 5.3. Usually, in addition to the point cloud formed by the obstacle itself, there are also point clouds formed by objects on the side and behind the obstacle. If all the point clouds in the obstacle range box are directly used for distance calculation, it will not only take a long time, but also cause large errors.

[0095] 5.4, ​​for the problem in 5.3, the range of obstacles on the image is larger than the range of obstacles on the laser radar, and the faster the train speed, the greater the difference between the two at the same time. Therefore, the present invention proposes: according to the obtained position of the obstacle in the visual image, the obstacle detection frame in the visual image is projected into the radar point cloud to obtain the initial point cloud range frame of the obstacle, and then the initial point cloud range frame of the obstacle is dynamically adjusted according to the running state of the train to obtain the final point cloud range frame of the obstacle, that is, N times scaling is performed for subsequent calculations. Similar to the scheme adopted by the N value in the first N adjacent image frames of the current image frame, when the train running speed is greater than 80km / s, N=0.6~0.7 is taken; when the train running speed is less than 80km / s, N=0.8~0.9 is taken; when the train turns, the position of the obstacle relative to the train changes greatly, and the point range of the radar point cloud can be appropriately expanded, taking N=1.0~1.2.

[0096] 5.5. Regarding the problem in 5.2, since the train always moves forward when running on the track and the area is relatively fixed, it is only necessary to pay attention to the distance from the train to the front surface of the obstacle. Therefore, the present invention proposes to first calculate the mean of the point cloud depth in the final point cloud range box, and then only retain the points with a depth less than the mean for the next calculation, and directly exclude the points with a depth greater than the mean. In this way, the background object point cloud in the range box and the point cloud to the side and rear of the object itself can be effectively excluded, thereby reducing the subsequent calculation amount and possible errors.

[0097] 5.6. After obtaining the point cloud that has been scaled by the range box and filtered by the depth mean, cluster the retained point cloud. It can be considered that the largest type of point set represents the actual position of the obstacle. The 50% points with the smallest depth in the point set represent the surface position of the obstacle. The depth mean of these 50% points is calculated as the preliminary distance d (unit: m) of the obstacle.

[0098] 5.7 According to Section 5.2, the acquisition of LiDAR data has a delay. In actual calculation, the program will also take a certain amount of time to run. Assume that the overall delay of the obstacle preliminary distance result relative to the current time of the train is t (in seconds), and the current running speed of the train is v (in meters / seconds). During the delay time, the train moves forward a distance of s=vt (in meters). Therefore, it is necessary to make a distance correction based on the preliminary distance d of the obstacle, and finally obtain the precise distance D (in meters) between the obstacle and the train at the current time, that is:

[0099] D=d–s

[0100] The present invention can detect and measure the distance of track obstacles in real time, accurately determine the track area in the image, and thus judge whether the obstacle is within the track area, and obtain a more accurate obstacle distance measurement result by dynamically adjusting the point cloud range and distance correction method for calculating the obstacle distance.

[0101] Example 2

[0102] like Fig.12 As shown, the difference between this embodiment and embodiment 1 is that this embodiment provides a track obstacle real-time detection system, which has a one-to-one functional correspondence with a track obstacle real-time detection method in embodiment 1; the system includes:

[0103] An acquisition unit is used to acquire a visual image of a track area; the visual image is a visual image in front of the train during the train's movement;

[0104] A segmentation unit is used to segment the visual image using a lightweight target segmentation model to obtain single-frame track lines, obstacles, and obstacle categories;

[0105] The track area dynamic fitting unit is used to obtain the final track area based on the single-frame track line and the dynamic multi-frame track area fitting; the dynamic multi-frame track area fitting is to fit the track area based on the track line detection results of multiple frames of images;

[0106] The obstacle position determination unit is used to determine whether the obstacle is within the final track area. If not, it returns to continue to obtain the visual image and radar point cloud data of the track area; if so, it obtains the detection box of the obstacle in the visual image;

[0107] The obstacle distance calculation unit is used to project the obstacle detection frame in the visual image into the radar point cloud data to obtain the obstacle point cloud range frame according to the obtained obstacle detection frame in the visual image, dynamically adjust the obstacle point cloud range frame, calculate the preliminary distance from the obstacle to the front of the train, and finally perform distance correction to obtain the final distance from the obstacle to the front of the train.

[0108] Among them, the execution process of each unit can be executed according to the process steps of a real-time detection method for track obstacles in Example 1, and will not be repeated in this embodiment.

[0109] At the same time, the present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned real-time detection method for track obstacles is implemented.

[0110] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0111] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0112] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0113] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0114] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A real-time detection method for track obstacles, characterized in that: The method includes: Acquire a visual image of the track area; the visual image is a visual image in front of the train during the train's movement; The visual image is segmented using a lightweight target segmentation model to obtain single-frame track lines, obstacles, and obstacle categories; According to the single-frame track line, based on dynamic multi-frame track area fitting, a final track area is obtained; the dynamic multi-frame track area fitting is to fit the track area by the track line detection results of multiple frames of images; Determine whether the obstacle is within the final track area. If not, return to continue acquiring the visual image and radar point cloud data in front of the train. If so, obtain the detection frame of the obstacle in the visual image. According to the obtained obstacle detection frame in the visual image, the obstacle detection frame in the visual image is projected into the radar point cloud data to obtain the obstacle point cloud range frame, the obstacle point cloud range frame is dynamically adjusted, and the preliminary distance from the obstacle to the front of the train is calculated, and finally the distance correction is performed to obtain the final distance from the obstacle to the front of the train.

2. A real-time detection method for track obstacles according to claim 1, characterized in that: The lightweight target segmentation model adopts the yolov5-tiny-seg instance segmentation model. The yolov5-tiny-seg instance segmentation model replaces the C3 structure of the feature extraction network Backbone in the traditional yolov5s-seg network with the Ghost structure, reduces the network depth, and then adds a CBAM attention module after the feature extraction network Backbone to fuse multi-scale features.

3. A real-time detection method for track obstacles according to claim 1, characterized in that: According to the single-frame track line, based on the dynamic multi-frame track region fitting, the final track region is obtained, including: Calculate the track outer contour points according to the track line detected by the current image frame and the track lines detected by the previous N adjacent image frames of the current image frame, construct a track outer contour point set based on the track outer contour points, perform point set clustering to exclude discrete abnormal points, and obtain a clustered point set; Based on the clustered point set, point set convex hull and contour detection are performed to obtain the final track area fitted by multi-frame track detection results.

4. A real-time detection method for track obstacles according to claim 3, characterized in that: The N value in the first N adjacent image frames of the current image frame is dynamically adjusted according to the running status of the train, including: When the train speed is greater than 80 km / h, the track area changes rapidly, and N=1-2 is taken, that is, the current frame and one to two frames before the current image frame are used to fit the track area; When the train running speed is less than 80km / h, the track area is relatively slow, and N=3-4 is taken, that is, the current frame and the first three to four frames of the current image frame are used to fit the track area; When the train is in a turning state, the track area changes the fastest, and N=0-1 is taken, that is, the current frame is directly used, or the current frame and the previous frame of the current image frame are used to fit the track area.

5. A real-time detection method for track obstacles according to claim 1, characterized in that: Determining whether an obstacle is within the final track area includes: A relative position determination is performed based on the detected obstacle detection frame and the final track area; the relative position determination includes: Take the left and right quarter points of the lower edge of the obstacle detection frame for relative position judgment. If one or both of the two quarter points fall within the track area, it is considered that the obstacle exists in the final track area. If the obstacle is within the final track area, the laser radar is used for distance measurement, and subsequent operations such as display and alarm are performed; if the obstacle is outside the final track area, no subsequent operations are performed.

6. A method for real-time detection of track obstacles according to claim 1, characterized in that: According to the obtained obstacle detection frame in the visual image, the obstacle detection frame in the visual image is projected into the radar point cloud data to obtain the obstacle point cloud range frame, the obstacle point cloud range frame is dynamically adjusted, and the preliminary distance from the obstacle to the train head is calculated, and finally the distance correction is performed to obtain the final distance from the obstacle to the train head, including: According to the obtained obstacle detection frame in the visual image, the obstacle detection frame in the visual image is projected onto the radar point cloud data to obtain an initial point cloud range frame of the obstacle; Dynamically adjusting the initial point cloud range frame of the obstacle according to the running status of the train to obtain a final point cloud range frame of the obstacle; Based on the final point cloud range frame, background point cloud is excluded and a preliminary distance of obstacles is calculated; The preliminary distance to the obstacle is corrected according to the train running speed to obtain a final distance calculation result of the obstacle.

7. A real-time detection method for track obstacles according to claim 6, characterized in that: Based on the final point cloud range frame, background point cloud is excluded and the preliminary distance of obstacles is calculated, including: Calculate the mean value of the point cloud depth in the final point cloud range frame, retain points with depths less than the mean value, and exclude points with depths greater than the mean value; The retained point cloud is clustered, and the largest point set is taken as the actual position of the obstacle. The points in a preset range with a smaller depth in this point set are taken as the surface position of the obstacle. The mean depth of the points in the preset range is calculated as the preliminary distance of the obstacle.

8. A real-time detection method for track obstacles according to claim 6, characterized in that: The corrected expression is: D=d–s s=vt Where D is the exact distance between the obstacle and the train at the current moment, in meters; d is the preliminary distance of the obstacle, in meters; s is the distance the train moves forward during the overall delay time, in meters; v is the current running speed of the train, in meters per second; t is the overall delay of the initial distance of the obstacle relative to the current time of the train, in seconds, including the radar data acquisition delay and program running time.

9. A real-time detection system for track obstacles, characterized in that: The system includes: An acquisition unit, used to acquire a visual image of a track area; the visual image is a visual image in front of the train during the train's movement; A segmentation unit, used to segment the visual image using a lightweight target segmentation model to obtain single-frame track lines, obstacles and obstacle categories; A track region dynamic fitting unit, used for obtaining a final track region based on dynamic multi-frame track region fitting according to the single-frame track line and the track lines detected in N frames before the current frame; the dynamic multi-frame track region fitting is to fit the track region by the track line detection results of multiple frames of images; The obstacle position determination unit is used to determine whether the obstacle is within the final track area. If not, it returns to continue to obtain the visual image and radar point cloud data in front of the train; if so, it obtains the detection frame of the obstacle in the visual image; The obstacle distance calculation unit is used to project the obstacle detection frame in the visual image into the radar point cloud data to obtain the obstacle point cloud range frame according to the obtained obstacle detection frame in the visual image, dynamically adjust the obstacle point cloud range frame, calculate the preliminary distance from the obstacle to the front of the train, and finally perform distance correction to obtain the final distance from the obstacle to the front of the train.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, a real-time detection method for track obstacles as described in any one of claims 1 to 8 is implemented.

Images