A method and system for long distance rail identification and obstacle detection

Abstract

The application discloses a long-distance rail recognition and obstacle detection method and system, a long-focus optical system is combined with a super-high-speed industrial camera on a hardware level to build a high-frame-rate and long-distance optical detection system, a region of interest for obstacle detection is demarcated by recognizing and visually segmenting left and right rail lines on a software level, and a target detection-based track obstacle intrusion detection algorithm and other advanced visual tasks are executed, and finally high-frame-rate recognition and detection of long-distance rails and track obstacles are realized. The Schmidt-Cassegrain optical system with a focal length of 2000mm is combined with a super-high-speed industrial camera with a resolution of 1920*1080 and 3000 frames per second to build a long-distance high-frame-rate optical detection platform and erect on a driving position of a train, railway conditions within a range of 6km are detected in real time, and serious accidents caused by a too long braking distance of the train due to high-speed driving can be prevented. Compared with traditional train driver eye observation and laser radar scanning imaging modes, a long-distance optical observation platform and a rear-end real-time detection system can realize a longer detection distance, better detection and recognition robustness and a higher frame rate.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of optical design, computer vision and computer control, and particularly relates to long-focus optical system design, target detection and identification technology, real-time video acquisition and processing technology, and is a long-distance track obstacle intrusion real-time detection method. BACKGROUND

[0002] The patent application with the publication number CN116030430A discloses a rail identification method, device, equipment and storage medium. The method performs data fusion on rail data collected by an image identification device and a radio detection device, obtains rail fusion image data, performs data processing on the rail fusion image data, obtains a rail contour image, and extracts coordinate points of the rail contour image by using a sliding window to obtain a target rail position. By fusing the rail data collected by two sensors, the defects of low precision, easy failure and poor stability caused by using a single sensor are avoided, and all-weather identification and extraction of the rail are realized. However, the rail identification method based on multi-sensor information fusion has high requirements for the calibration accuracy of different sensors, and certain calibration errors in actual scenes lead to poor data fusion effect, and the sliding window method for obtaining track coordinate points based on the rail contour image has a pixel traversal process and is more suitable for scenes with low real-time requirements.

[0003] The patent application with the publication number CN117292351A discloses a subway obstacle detection system and method based on the background subtraction method. The method first pre-collects an image without obstacles as a background frame and collects a real-time image as a frame to be detected. The feature vectors of the two are extracted by a convolutional neural network and a pooling method and are vectorized and encoded. The background frame vectorization coding generates a search codebook, and the corresponding frame is obtained by referring to the search codebook after the frame to be detected is vectorized and encoded. The safety limit of rail operation is obtained based on a Sobel edge detection operator, a sliding window and a least squares method. The perspective transformation matrix is calculated through ORB feature matching and SVD, and the corresponding frame aligned with the frame to be detected is obtained through perspective transformation. The obstacle detection result image is obtained based on the background subtraction method and the regional texture similarity method. Although the method combines the track recognition and obstacle detection algorithms to realize an end-to-end track obstacle detection and recognition system, the image gradient detection algorithm based on the Sobel edge detection operator has poor robustness in the face of complex environments and different lighting conditions, and cannot perform the identification task in actual scenes. Meanwhile, the obstacle detection method based on the background subtraction method is similar to the template matching strategy, and the reliance on fixed prior information leads to false detection and missed detection of intrusion targets, and the overall method has poor practicability.

[0004] The patent application with the publication number CN108375771B discloses a detection method of a two-dimensional laser radar railway obstacle detection system, which comprises an adjusting platform, a control calculator and an alarm device are installed on the top left side of the adjusting platform, the control calculator is located on the left side of the alarm device, and a two-dimensional laser radar is connected to the top center of the adjusting platform through a screw.

[0005] The computer vision-based long-distance rail recognition and obstacle detection technology can effectively solve the problems of short visible distance, high false detection and missed detection rate caused by the driver's visual method, low scanning frequency and insufficient real-time caused by the laser radar scanning imaging, and reduce the influence of target detection invalidity caused by mutual shielding in a multi-obstacle scene. SUMMARY

[0006] In view of the defects of the prior art, the present application provides a long-distance rail recognition and obstacle detection method and system to solve the problems of limited application scene, low detection accuracy and poor real-time of the prior art.

[0007] To achieve the above-mentioned purpose, in a first aspect, the present application provides a long-distance rail recognition and obstacle detection method, comprising the following steps.

[0008] S1, optical system parameter adjustment and image acquisition; after the long-focus optical system is erected at the train driving position, the clear imaging of the rail 6km away is realized by controlling the focusing ring, and the exposure time of the camera is adjusted to ensure that the rails and nearby objects within the depth of field are within the visual range, and at the same time, it will not be overexposed. The high-speed motion camera acquisition function is turned on during the train running process, and the collected images are transmitted into the back end to perform upstream and downstream visual tasks in real time.

[0009] S2, image preprocessing: in the image, the semantic information of the rail is more special than other targets, which mainly has two characteristics, namely high gradient change rate and strong gradient continuity. Therefore, after the back end obtains the video frame captured by the camera, it is necessary to first preprocess the image to strengthen the gradient information of the rail and reduce the gradient influence of the background stray objects, which includes image center region cutting, image grayscale conversion, histogram equalization and setting Gaussian blur to reduce the gradient information of non-attention parts.

[0010] Since the observation system is fixed in the train, the train can keep relative static state with the vehicle when driving. In actual long-distance shooting, the track can always remain in the center of the frame without deviation due to the strong continuity of the track and the small rotation of the curved track. At the same time, considering that reducing the full-frame size can speed up the frame processing speed and reduce the interference of other objects in space to track identification, the quantitative cutting is carried out for the center area of the full-frame, which reduces the resolution while ensuring the integrity of the track.

[0011] In the actual track identification process, the main concern is whether the image has gradient information of the track, so the color information of the image is not needed when processing. In order to reduce information redundancy and improve real-time processing frame rate, the image needs to be converted to grayscale before performing the upstream identification task. The grayscale conversion of the image can reduce the channel number while preserving the texture and detail information.

[0012] Outdoor shooting is directly affected by the environment light, which affects the brightness of all objects in the field of view, so the actual visual experience is not good. The main feedback is that the brightness of all objects is similar, which weakens the gradient representation of the whole image. After histogram equalization operation, the contrast of the image can be significantly enhanced, and the gradient information of the alternating light and dark scene can be highlighted, realizing the visual enhancement of gradient detection. At the same time, in order to reduce the negative impact of other similar long strip objects on track identification, a 5*5 pixel Gaussian blur kernel is used to convolve the whole image to reduce its overall gradient threshold.

[0013] S3, track identification algorithm: after the image is preprocessed, the track information in the picture needs to be extracted and identified. The edge detection operator (Sobel, Canny, etc.) has insufficient edge information extraction ability in actual scene; on this basis, a multi-dimensional gradient weighted average operator group is used as a convolution kernel to extract the gradient information of the image. Gradient has directionality in two-dimensional space, so 8 different direction gradient operators are set to convolve the original image to obtain 8 different gradient feature outline maps. After weighted average processing of these outline maps, the outline map corresponding to the strong information of the track part of the original image is obtained.

[0014] The profile image is input into the rail recognition algorithm. The rail recognition algorithm is based on part of the prior information. For the first frame image, the left and right rail information at the lowermost edge (the nearest distance) of the image needs to be labeled. After the labeling is completed, no other operation is required. Two fixed-size left and right rail recognition boxes are set. The prior information is used as the center point of the recognition box. The algorithm performs slice processing on each frame image from bottom to top. The slice width is about 10-15 pixels. For each slice, the sum is calculated by row to obtain a one-dimensional matrix. The highest and second highest peaks in the one-dimensional matrix are searched. By comparing the index values, it can be known that the point with a smaller index value is on the left rail, and the point with a larger index value is on the right rail. The rail pixel point at the lowermost edge is given as the prior information. The rail pixel points are sequentially calculated by slice processing from bottom to top. After the pixel point coordinates of the current slice are obtained, the horizontal pixel difference between the coordinates of the current slice and the coordinates of the previous slice is calculated. The difference is used to move the horizontal position of the recognition box, so that the rail recognition requirement in the case of curved rails can be met. The width of the recognition box gradually decreases with the increase of the number of slices, but is always greater than the rail pixel width. After the same operation is performed on all slices, the point column of the left and right rails in the frame image can be obtained. The point column is subjected to Bezier function curve fitting to obtain the complete left and right rail curves of the current frame, and the rail recognition task is completed.

[0015] S4, obstacle detection algorithm: for the target detection and identification task of the rail obstacle, only whether there is an object in the rail and its vicinity is concerned. The objects far from the rail are not the objects of concern. Therefore, after the rail recognition is completed, the region of interest (ROI) for target detection is determined according to the pixel coordinates of the left and right rails. On the one hand, the object information in the non-rail region can be excluded. On the other hand, the ROI region is equivalent to cutting the original image, which can speed up the calculation efficiency of the detection model. The target detection and identification model adopts the classic YOLOv5 detection algorithm, and is trained according to the specific scene (rail track background) and specific obstacle targets (people, vehicles, animals, trees and stones, etc.) to obtain a detection model with higher adaptation to the current scene. Considering that in the long-distance application scene, some movable obstacles may move towards the rail, the area within 2m on both sides of the rail and the left and right rails is set as the actual warning area. When the pre-trained target is detected and identified in this area, the alarm device is triggered to remind the driver to take corresponding operation.

[0016] In a second aspect, the present application provides a long-distance rail recognition and obstacle detection system design, comprising:

[0017] The high-frame-rate video shooting system based on a long-focus optical system mainly comprises a front long-focus optical system and a super-high-speed industrial camera. Due to the large volume and weight of the long-focus optical system, a mirror frame module is needed to support the ground to stabilize the system.

[0018] The high-performance computer platform mainly has the characteristics of high-speed data throughput and real-time image processing module, and the platform needs to receive 500 frames of images returned by a shooting system per second and perform real-time processing and feedback to the front end.

[0019] Compared with the prior art, the present application has the following beneficial effects:

[0020] (1) The present application provides a long-distance rail recognition and obstacle detection method and system, which aims to solve the problems of large spatial sampling interval and short recognition distance caused by fixed position installation in the current distributed rail detection method, and the problems of complex system device and low sampling frequency in the rail obstacle detection method based on laser radar. The system is built by combining a long-focus optical system with a high-speed industrial camera, which ensures the detection distance and realizes a high-frame-rate detection method with strong real-time performance.

[0021] (2) The present application provides a long-distance rail recognition and obstacle detection method and system, which arranges a long-focus optical observation platform at the position of the train driver's cabin, and the angle of view needs to be directly opposite the front rail to ensure that the rail is always in the center area of the frame. Since the rail has the characteristics of continuity and small bending curvature, and the system field of view angle range can reach a 15m*15m space range in the two-dimensional plane, there is no need to adjust the alignment and focusing position of the optical system subsequently.

[0022] (3) The present application provides a long-distance rail recognition and obstacle detection method and system, which can effectively extract the gradient information of the rail based on the image feature extraction convolution operation of the multi-dimensional gradient weighted average operator group in complex scenes, and effectively weaken other gradient obvious objects in the background layer, so as to realize the acquisition of rail features. According to the feature map with rail gradient features extracted, the image is segmented and sliced along the y direction, and the slice at the lower edge is the initial position of the rail. Artificial prior information is used for rail line labeling, which is only performed in the initial stage of system operation, and does not need to be manually managed subsequently. The artificially labeled position is used as the center position of the left and right rail line recognition frame, and the pixel coordinates with the maximum gradient value in the left and right regions are queried for the next slice (the upper layer of the image lower edge slice), and the width of the recognition frame decreases in a negative exponential function trend with each upward slice. When the pixel coordinates with the maximum gradient value deviate from the center coordinates of the recognition frame, the recognition frame moves the overall coordinates to the deviated direction before performing the operation on the next slice, which can effectively handle special cases such as curved rails.

[0023] (4) This invention provides a method and system for long-distance railway track recognition and obstacle detection. After railway track recognition is completed, the region of interest is delineated based on the pixel coordinates obtained from the recognition, and obstacle detection is performed. The target detection and recognition task adopts the classic YOLOv5 detection model, which is retrained for actual railway track scenarios. Considering the category distribution of obstacles in actual scenarios, the training objects are newly added special categories such as people, vehicles, animals, trees, and stones, which have a high probability of occupying the track and running over the track. At the same time, considering that moving targets that have not entered the track at the current time but have the possibility of occupying the track in the future under long-distance detection, the system delineates the area within 2m on both sides of the railway track as the actual warning area, thus expanding the background area for obstacle detection. Attached Figure Description

[0024] Figure 1 This is a schematic diagram of the layered system architecture design provided in Embodiment 1 of the present invention;

[0025] Figure 2 This is a schematic diagram of a high-speed video acquisition system based on a telephoto optical module provided in Embodiment 1 of the present invention;

[0026] Figure 3 This is a schematic diagram of an actual data collection scenario provided in Embodiment 1 of the present invention;

[0027] Figure 4 This is a schematic diagram of the railway track recognition algorithm provided in Embodiment 1 of the present invention;

[0028] Figure 5 This is a schematic diagram of the track obstacle detection and recognition model framework provided in Embodiment 1 of the present invention; Detailed Implementation

[0029] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. It should be noted that the described embodiments are intended only to facilitate understanding of the invention and are not intended to limit it in any way. The drawings are all in a very simplified form and use non-precise proportions, only to facilitate and clarify the illustration of the embodiments of the invention; the structures shown in the drawings are part of the actual structures. Furthermore, the technical features involved in the various embodiments of the invention described below can be combined with each other as long as they do not conflict with each other.

[0030] Example 1

[0031] A method and system for long-distance railway track identification and obstacle detection, applicable to long-distance railway track obstacle detection and identification, is illustrated in the following diagram: Figure 1 As shown, it includes the following steps:

[0032] S1, Long-range telephoto optical module high-speed video acquisition system layout as follows: Figure 2As shown, considering the small space in the system on-board restricted cabin, the long-focus optical system is designed to adopt a Schmidt-Cassegrain return type optical system to reduce the system barrel length while ensuring the focal length remains unchanged. The long-focus optical observation system is connected to a high-speed industrial camera, and the main part of the optical observation platform is fixed in the central position of the cockpit through mechanical structure. The system is connected to the computing platform through CoaXPress protocol, thereby providing ultra-high bandwidth video stream transmission.

[0033] S2, the actual collection scene of the long-distance rail recognition and obstacle detection system is as shown in Figure 3 As shown, the long-focus optical observation system is directed at the front of the rail, and the pointing and focusing ring of the optical system needs to be adjusted during installation to ensure that it can focus on the rail several kilometers away in front. After the system is assembled and debugged, preprocessing and rail recognition operations are performed on each collected frame of image. The rail recognition algorithm flow is as shown in Figure 4 As shown, an image with a resolution of 1920*1080 pixels is input, and after image preprocessing, image gradient information extraction, rail prior information labeling, left and right rail line recognition, and Bezier function fitting rail line processes, a final image with the original image as the background and additional red rail lines is obtained.

[0034] S3, the track obstacle detection and recognition model framework of the long-distance rail recognition and obstacle detection system is as shown in Figure 5 As shown, the most popular YOLOv5 target detection and recognition network in the detection field is used, which is composed of an input end, a backbone network, a neck network, and a head network. The input end performs Mosaic data enhancement on the input image, and performs adaptive image scaling and adaptive anchor box calculation to facilitate network training. The two important modules in the backbone network are Focus structure and CSP structure. The Focus module in YOLOv5 performs slicing operation on the picture before it enters the Backbone, that is, it takes a value every other pixel in a picture, similar to adjacent downsampling. In this way, four pictures are obtained, which are complementary but have no information loss. In this way, the W and H information is concentrated in the channel space, and the input channel is expanded by 4 times, that is, the picture after splicing becomes 12 channels instead of the original RGB three-channel mode. Finally, the new picture obtained is subjected to convolution operation, and a two-fold down-sampled feature map without information loss is finally obtained. CSPNet mainly splits the feature map into two parts, one part performs convolution operation, and the other part and the result of the convolution operation of the previous part are concatenated. The neck network adopts the CSP2 structure designed by CSPNet to strengthen the network feature fusion capability. The head network adopts CIOU_LOSS as the loss function of the boundary box to realize the prediction of the target position.

[0035] To sum up, the above is only the preferred embodiment of the present application, and is not used to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method and system for long distance rail identification and obstacle detection, characterized in that: Comprise the following steps: 1) After the long-focus optical system is erected at the train driving position, the clear image of the track 6 km away is realized by controlling the focusing ring, and the exposure time of the camera is adjusted to ensure that the track and nearby objects within the depth of field are within the visual range and will not be overexposed. The high-speed motion camera acquisition function is turned on during the driving process, and the collected images are transmitted to the backend in real time to perform upstream and downstream visual tasks; 2) After the backend obtains the video frames captured by the camera, the image needs to be preprocessed to enhance the gradient information of the track and reduce the gradient influence of the background stray objects, which includes image center region cutting, image grayscale conversion, histogram equalization, and setting Gaussian blur to reduce the gradient information of non-concerned parts; 3) After the image preprocessing operation, the track information in the picture needs to be extracted and recognized. The edge detection operators such as Sobel and Canny have insufficient edge information extraction capability in actual scenes. On this basis, a multi-dimensional gradient weighted average operator group is used as a convolution kernel to extract the gradient information of the image. Gradient has directionality in two-dimensional space, so 8 different direction gradient operators are set to perform convolution operations on the original image to obtain 8 contour maps with different gradient features. After weighted average processing of these contour maps, the contour map corresponding to the strong information of the track part of the original image is obtained. The contour map is input into the track recognition algorithm, which is based on some prior information. For the first frame of image, the left and right track information at the nearest distance of the lowermost edge of the image needs to be labeled. After labeling, no other operation is needed. Two fixed-size left and right track recognition boxes are set, with the prior information as the center point of the recognition box. The algorithm performs slice processing from bottom to top for each frame of image, with a slice width of about 10-15 pixels; For each slice, the sum is calculated by row to obtain a one-dimensional matrix. In the one-dimensional matrix, the two points with the highest and second highest peaks are searched. By comparing the index values, it can be known that the point with the smaller index value is on the left track, and the point with the larger index value is on the right track. The pixel point at the lowermost edge of the track is given as prior information, and the pixel points of the track are processed slice by slice from bottom to top. After obtaining the pixel coordinates of the current slice, the horizontal pixel difference between the coordinates of the current slice and the coordinates of the previous slice is calculated. This difference is used to move the horizontal position of the recognition box, so that it can meet the requirements of track recognition in curved tracks. The width of the recognition box gradually decreases with the increase of the number of slices, but it is always greater than the pixel width of the track. After the same operation is performed on all slices, the point column of the left and right tracks in the image is obtained. The point column is fitted with a Bezier function curve to obtain the complete left and right track curves of the current frame, and the track recognition task is completed. 4) According to the pixel coordinates of the left and right rails, the region of interest (ROI) for target detection is determined, which can exclude non-track area object information on the one hand, and the ROI region is equivalent to cutting the original image to speed up the calculation efficiency of the detection model; the target detection and recognition model uses the classic YOLOv5 detection algorithm, and is trained for specific rail track scenes and specific obstacle targets including people, vehicles, animals, trees and stones to obtain a detection model with higher adaptation to the current scene; considering the possibility of moving obstacles moving towards the rail in the long-distance application scenario, the area within 2m on both sides of the rail and the left and right tracks is set as the actual warning area, and when the pre-trained target is detected and recognized in this area, the alarm device is triggered to remind the driver to take appropriate action; 2. A method and system for remote rail identification and obstacle detection as claimed in claim 1, wherein, In view of the problems of short detection distance, complex system device and low sampling frequency existing in the existing method, a long-focus optical system is combined with a high-speed industrial camera to build an optical observation system platform, which realizes a high frame rate detection method with strong real-time while ensuring the detection distance.

3. A method and system for remote rail identification and obstacle detection as claimed in claim 1, wherein, The long-focus optical observation platform is arranged at the train driver's cabin position, and the visual angle needs to be directly opposite the front rail to ensure that the rail is always in the center area of the frame. Since the rail has the characteristics of continuity and small bending curvature, and the system field of view angle range can reach a space range of 15m*15m in two-dimensional plane, subsequent adjustment of the alignment and focusing position of the optical system is not required.

4. A method and system for remote rail identification and obstacle detection as claimed in claim 1, wherein, The image feature extraction convolution operation based on the multi-dimensional gradient weighted average operator group can effectively extract the gradient information of the rail in complex scenes by utilizing the continuity and elongated features of the rail itself, and effectively weaken other gradient objects in the background layer, realizing the acquisition of rail features; according to the feature map with rail gradient features extracted, the image is segmented and sliced along the y direction, and the slice at the lower edge is the initial position of the rail. Artificial prior information is used for track line labeling, which is only performed in the initial stage of system operation and does not require manual management in the subsequent stage; taking the position labeled by the artificial as the center position of the left and right track recognition frame, the maximum gradient value of the left and right regions of the next slice, i.e. the lower edge slice of the image, is queried, and the width of the recognition frame decreases in a negative exponential function trend with each upward slice. When the gradient maximum position pixel coordinates deviate from the recognition frame center coordinates, the recognition frame moves the overall coordinates to the deviated direction before performing the operation on the next slice, effectively handling the special case of curved rails.

5. A method and system for remote rail identification and obstacle detection as claimed in claim 1, wherein, The target detection and recognition task uses the classic YOLOv5 detection model, which is retrained for actual rail scenes. Considering the class distribution of obstacles in the actual scene, the training object adds special classes of rail occupation and rail collision targets with high existence probability, including people, vehicles, animals, trees and stones. At the same time, considering the moving targets that have not entered the rail at the current time but have the possibility of future rail occupation in the long-distance detection situation, the system sets the area within 2m on both sides of the rail and the left and right tracks as the actual warning area, which widens the background area of obstacle detection.

Images