Track line detection method and device, equipment and storage medium
By using dedistortion and perspective transformation in track line detection, and sliding windows to detect track edges on convolutional images, the problem of high computing resources of neural network models is solved, efficient track line detection is achieved, and the improvement of Real-time and safety of autonomous driving systems.
Patent Information
- Application Number
- CN202410971667.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-19
- Publication Date
- 2024-11-08
Smart Images

Figure CN118918553A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of rail transit technology, and in particular to a track line detection method, device, equipment, and storage medium. Background Art
[0002] With the technological advancement in the field of transportation, the train automatic driving system has gradually become an innovative field in railway transportation. The train automatic driving system has gradually replaced the traditional manual driving method. The goal of this technology is to realize the automatic operation and navigation of railway trains, thereby improving the efficiency of the system, reducing the risk of accidents, providing a higher passenger experience, and reducing operating costs. In the train automatic driving system, the accurate identification of the track line is one of the core elements to achieve automatic navigation and safe operation, which directly affects the operation safety of the train and the reliability of the system.
[0003] Existing track line detection methods are based on neural network models, which learn the characteristics of track lines from a large amount of image data to achieve high-precision track line recognition. However, neural network models have high requirements for computing resources, which limits the real-time performance on embedded or edge devices. Summary of the invention
[0004] In order to solve one of the above-mentioned technical defects, the present application provides a track line detection method, device, equipment, and storage medium.
[0005] In a first aspect, the present application provides a track line detection method, the method comprising:
[0006] Get track line images;
[0007] After dedistorting the track line image, perspective transformation is performed on the dedistorted image to obtain a perspective transformed image;
[0008] Based on the perspective transformed image, determine the convolution image and the track area;
[0009] Based on the track area, the sliding window is slid on the convolution image, and the track edge within the sliding window after each sliding is determined, and the track line detection is performed based on the track edge.
[0010] Optionally, performing perspective transformation on the dedistorted image to obtain a perspective transformed image includes:
[0011] The track area directly in front of the vehicle in the dedistorted image is determined as the region of interest;
[0012] Determine the four vertex coordinates src_points of the target area; wherein the target area is the smallest area in the region of interest including the edges of the left and right track lines;
[0013] Determine the coordinates dst_points of the four vertices in the perspective transformed image;
[0014] The dedistorted image is transformed into a perspective transformed image by a projection matrix based on src_points and dst_points.
[0015] Optionally, determining a convolution image based on the perspective transformed image includes:
[0016] Performing histogram equalization processing on the image after perspective transformation to obtain a first processed image;
[0017] Performing Gaussian blur processing on the first processed image to obtain a second processed image;
[0018] fusing the dedistorted image with the second processed image to obtain a third processed image;
[0019] The third processed image is convolved based on the edge detection convolution kernel to obtain a convolved image.
[0020] Optionally, the edge detection convolution kernel is
[0021] Optionally, determining the track area based on the perspective transformed image includes:
[0022] Convert the color space of the perspective transformed image into the HLS color space to obtain the L channel image after space conversion;
[0023] Determine the total number of white pixels in each column of pixels in the L channel image;
[0024] The column with the largest total number of white pixels is determined as the third track line area;
[0025] Based on the third track line area, the left and right track line areas are determined.
[0026] Optionally, based on the third track line area, determining the left and right track line areas includes:
[0027] Clipping the L channel image in the target direction of the third track line area to obtain the driving track area; wherein the target direction is the direction in which the left and right tracks are located on the third track;
[0028] Detecting straight line segments in the driving track area;
[0029] Cluster the pixels of each straight line segment based on the column where each pixel point in the straight line segment is located;
[0030] The column where the clustered class center is located is determined as the left and right track line area.
[0031] Optionally, based on the track area, sliding the sliding window on the convolution image, and determining the track edge within the sliding window after each sliding, including:
[0032] Based on the initial pixel point, determine the initial position of the sliding window in the track area; if it is the first sliding, the initial pixel point is a preset pixel point, if it is not the first sliding, the initial pixel point is determined based on the last target position of the sliding window;
[0033] Based on the columns of the track area within the initial position, the initial position is adjusted to obtain the target position;
[0034] The sliding window on the convolution image slides to the target position;
[0035] The values of each pixel in the sliding window after sliding are summed by column, and the column with the maximum sum is determined as the track edge in the sliding window.
[0036] In a second aspect of the present application, a track line detection device is provided, the device comprising:
[0037] An acquisition module, used for acquiring a track line image;
[0038] A processing module, used for performing dedistortion processing on the track line image acquired by the acquisition module, and performing perspective transformation on the dedistorted image to obtain a perspective transformed image;
[0039] A determination module, used for determining a convolution image and a track area based on the perspective transformed image obtained by the processing module;
[0040] The detection module is used to slide the sliding window on the convolution image based on the track area, determine the track edge in the sliding window after each sliding, and perform track line detection based on the track edge.
[0041] In a third aspect of the present application, an electronic device is provided, including:
[0042] Memory;
[0043] Processor; and
[0044] Computer programs;
[0045] The computer program is stored in the memory and is configured to be executed by the processor to implement the method as described in the first aspect above.
[0046] In a fourth aspect of the present application, a computer-readable storage medium is provided, on which a computer program is stored; the computer program is executed by a processor to implement the method described in the first aspect above.
[0047] The present application provides a track line detection method, device, equipment, and storage medium, the method comprising: acquiring a track line image; performing dedistortion processing on the track line image, and then performing perspective transformation on the dedistorted image to obtain a perspective transformed image; based on the perspective transformed image, determining a convolution image and a track area; based on the track area, sliding a sliding window on the convolution image, and determining the track edge in the sliding window after each sliding, and performing track line detection based on the track edge.
[0048] The method of the present application determines the convolution image and the track area based on the image after perspective transformation; based on the track area, the sliding window is slid on the convolution image, and the track edge in the sliding window after each sliding is determined, and the track line detection is performed based on the track edge, thereby ensuring the real-time performance of the track line detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0050] Figure 1 A schematic diagram of a flow chart of a track line detection method provided in an embodiment of the present application;
[0051] Figure 2 A schematic diagram of a flow chart of another track line detection method provided in an embodiment of the present application;
[0052] Figure 3 A schematic diagram of taking multiple images including a calibration plate provided in an embodiment of the present application;
[0053] Figure 4 A schematic diagram of an image after perspective transformation provided in an embodiment of the present application;
[0054] Figure 5 A schematic diagram of a first processed image provided in an embodiment of the present application;
[0055] Figure 6 A schematic diagram of a second processed image provided in an embodiment of the present application;
[0056] Figure 7 A schematic diagram of a third processed image provided in an embodiment of the present application;
[0057] Figure 8 A schematic diagram of a pixel coordinate system provided in an embodiment of the present application;
[0058] Fig. 9 A schematic diagram of statistics of white pixels on the left and right tracks located on the right side of the third track provided in an embodiment of the present application;
[0059] Fig.10 A schematic diagram of statistics of white pixels on the left and right tracks of the third track provided in an embodiment of the present application;
[0060] Fig.11 A schematic diagram of determining left and right track line areas provided in an embodiment of the present application;
[0061] Fig.12 A schematic diagram of track line positioning provided in an embodiment of the present application;
[0062] Fig.13 A schematic diagram of the structure of a track line detection device provided in an embodiment of the present application;
[0063] Fig.14 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0064] In order to make the technical solutions and advantages in the embodiments of the present application more clearly understood, the exemplary embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than an exhaustive list of all the embodiments. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.
[0065] In the process of implementing this application, the inventors found that the existing track line detection method is based on a neural network model, which learns the characteristics of the track line from a large amount of image data to achieve high-precision track line recognition. However, the neural network model has a high demand for computing resources, which limits the real-time performance on embedded or edge devices.
[0066] In response to the above problems, a track line detection method, device, equipment, and storage medium are provided in an embodiment of the present application. The method includes: acquiring a track line image; after dedistorting the track line image, performing perspective transformation on the dedistorted image to obtain a perspective transformed image; based on the perspective transformed image, determining the convolution image and the track area; based on the track area, sliding the sliding window on the convolution image, and determining the track edge within the sliding window after each sliding, and performing track line detection based on the track edge. The method of the present application determines the convolution image and the track area based on the perspective transformed image; based on the track area, sliding the sliding window on the convolution image, and determining the track edge within the sliding window after each sliding, and performing track line detection based on the track edge, thereby ensuring the real-time performance of track line detection.
[0067] See also Figure 1 This embodiment provides a track line detection method, the method comprising:
[0068] 101, obtain a track line image.
[0069] The track line image of this step can be obtained by taking pictures with a vehicle-mounted camera.
[0070] In the specific implementation, Figure 2 As shown, a video including track lines can be first captured by a vehicle-mounted camera (eg, with a resolution of 1280×720), and then each frame image (eg, with a resolution of 1280×720) can be extracted from the video, each frame image being a track line image.
[0071] 102 , after dedistorting the track line image, perspective transformation is performed on the dedistorted image to obtain a perspective transformed image.
[0072] Step 102 actually includes two processing processes, namely, a dedistortion processing process and a perspective transformation processing process. Figure 2 , the implementation process of step 102 is as follows:
[0073] 102-1, the track line image is subjected to dedistortion processing to obtain a dedistorted image.
[0074] There is distortion in the image captured by the vehicle-mounted camera, but it is usually not easy to be detected by the eyes. Such distortion will have an adverse effect on tasks such as track line detection, so after obtaining the track line image in step 101, the distortion removal process must be performed first.
[0075] When performing de-distortion processing, it can be achieved through camera calibration. Camera calibration can be achieved by shooting an object of known shape, calculating the deviation between the position of the object in the real world and the position in the image (i.e., the distortion coefficient), and then using this deviation (i.e., the distortion coefficient) to correct the distortion in other images.
[0076] When determining the distortion coefficient, any object of known shape can be selected to obtain the distortion coefficient. In specific implementation, a checkerboard can be preferably selected to obtain the distortion coefficient. This is because the checkerboard has a regular, high-contrast pattern, which can be very convenient for automated detection and accurately measure the distance between each checkerboard intersection.
[0077] For example, a 12×9 black and white chessboard is used, and a camera to be calibrated (i.e., the vehicle-mounted camera in step 101) is used to capture multiple images containing the calibration plate, such as Figure 3As shown. 1) Use OpenCV's cv2.findChessboardCorners() function to automatically detect the intersections of the four chessboard squares (the intersections of 2 white and 2 black squares). For example, the cv2.findChessboardCorners() function inputs the complete chessboard image taken by the camera and the number of intersections in the horizontal and vertical directions, and then detects the intersections in the image. 2) Use cv2.drawChessboardCorners() function to draw the detection results. 3) After obtaining the intersection detection results, use cv2.calibrateCamera() function to get the camera's distortion coefficient.
[0078] Through step 102-1, the distortion in the track line image obtained in step 101 can be corrected, which helps to improve the accuracy and reliability of track line detection.
[0079] 102-2, performing perspective transformation on the dedistorted image to obtain a perspective transformed image.
[0080] Step 102-2 can be implemented by the following steps:
[0081] 102-2-1, determine the track area in front of the vehicle in the dedistorted image as a region of interest (ROI).
[0082] 102-2-2, determine the coordinates src_points of the four vertices of the target area.
[0083] The target area is the smallest area in the region of interest (ROI) that includes the edges of the left and right track lines.
[0084] The shape of the minimum area can be any shape, for example, the shape of the minimum area is a rectangle, such as the minimum area is a minimum rectangular area. For another example, the shape of the minimum area is a trapezoid, such as the minimum area is a minimum trapezoidal area.
[0085] Taking the shape of the minimum area as a trapezoid as an example, in the region of interest (ROI), the track lines are usually parallel, but perspective distortion may appear in the dedistorted image, making the track lines appear non-parallel. Therefore, in 102-2-2, an image of driving on a straight track can be selected from the dedistorted image, and a trapezoidal area (i.e., the target area, that is, the minimum area in the ROI including the edges of the left and right track lines) is selected along the edges of the left and right track lines, and the coordinates of the four vertices are obtained, which are recorded as src_points.
[0086] 102-2-3, determine the coordinates dst_points of the four vertices in the image after perspective transformation.
[0087] The target area should be a rectangle on the real road, so select a suitable position in the image after perspective transformation, select a rectangular area, and record the four vertices as dst_points.
[0088] 102-2-4, transform the dedistorted image into a perspective transformed image through a projection matrix based on src_points and dst_points.
[0089] After obtaining src_points and dst_points, the projection matrix from src_points to dst_points can be calculated through the function cv2.getPerspectiveTransform(src_points,dst_points) in OpenCV, and then the dedistorted image can be transformed into a perspective transformed image using the projection matrix of src_points and dst_points.
[0090] The final projection result (i.e. the image after perspective transformation) is as follows: Figure 4 As shown, like a "bird's eye view", this will improve the geometry of the track lines, making them easier to detect and track.
[0091] Through step 102 - 2 , the perspective distortion in the dedistorted image can be corrected, and the irregular image area can be projected into a regular image area, thereby improving the geometric structure of the dedistorted image, so as to more accurately detect and track the track line.
[0092] 103, based on the image after perspective transformation, determine the convolution image and the track area.
[0093] Step 103 includes two processes, one is a process of determining a convolution image based on the image after perspective transformation, and the other is a process of determining a track area based on the image after perspective transformation. There is no specific order for the execution of the two processes, that is, the two processes can be executed simultaneously, or one can be executed first and the other later.
[0094] The two processes are described below.
[0095] 1. The process of determining the convolution image based on the image after perspective transformation
[0096] The steps to achieve this process are as follows:
[0097] 201 , performing histogram equalization processing on the image after perspective transformation to obtain a first processed image.
[0098] The first processed image is the image after histogram equalization processing.
[0099] In the specific implementation, it can be achieved through cv2.equalizeHist() function.
[0100] Histogram equalization processing helps to improve the contrast of the perspective transformed image, reduce illumination non-uniformity, reduce noise interference, and improve the visibility of track lines, thereby enhancing the accuracy and robustness of the track line detection method, especially in challenging lighting conditions, which helps to optimize image quality.
[0101] The first processed image can be Figure 5 shown.
[0102] 202, perform Gaussian blur processing on the first processed image to obtain a second processed image.
[0103] The second processed image is the image after Gaussian blur processing.
[0104] In specific implementation, the first processed image (i.e., the image after histogram equalization processing) can be Gaussian blurred by the cv2.GaussianBlur() function to reduce high-frequency noise and details in the first processed image, thereby reducing interference and improving the robustness of the track line detection method.
[0105] The Gaussian blur processing of step 202 can help smooth the first processed image, making the track line more obvious and stable, so as to more accurately detect and track the track line, especially in the case of high noise or poor image quality, which can improve the performance of track line recognition.
[0106] The second processed image can be Figure 6 shown.
[0107] 203 : Fusing the dedistorted image with the second processed image to obtain a third processed image.
[0108] The third processed image is the fused image.
[0109] In addition, when fusion is performed, the input dedistorted image is a black and white image.
[0110] In the specific implementation, the black and white image of the dedistorted image can be fused with the second processed image (i.e. the image after Gaussian blur processing) with appropriate weights through the cv2.addWeighted() function, which can enhance the visibility of the track line, making it easier to detect and segment, thereby improving the accuracy of track line recognition.
[0111] The fusion process in step 203 helps to adjust the image to meet the needs of track line detection under different lighting conditions, thereby enhancing the robustness of the track line detection method.
[0112] The third processed image can be Figure 7 shown.
[0113] 204 , performing convolution processing on the third processed image based on the edge detection convolution kernel to obtain a convolution image.
[0114] Among them, the edge detection convolution kernel is 5×5
[0115] Since the third processed image (i.e., the fused image) is obtained based on the image after perspective transformation, the track is extended vertically in the third processed image, and the horizontal gradient feature is obvious in the third processed image. Step 204 performs convolution processing on the third processed image based on the edge detection convolution kernel, which can realize a horizontal difference operation on the third processed image, and then emphasize the changes in the horizontal direction through the difference operation.
[0116] For example, in step 204, the edge detection convolution kernel is slid on the third processed image, and the pixel value at each position is operated as follows:
[0117] -1×I(x-2,y)+1×I(x-1,y)+0×I(x,y)+1×I(x+1,y)-1
[0118] ×I(x+2,y)
[0119] Wherein, x is the horizontal coordinate of the position of any pixel, y is the vertical coordinate of the position of any pixel, and I(x, y) represents the pixel value of the third processed image at the position (x, y).
[0120] Among them, the horizontal coordinate direction is the horizontal direction in the pixel coordinate system of the image (i.e., the row of pixels in the image), and the vertical coordinate direction is the vertical direction in the pixel coordinate system of the image (i.e., the column in the image). Since the above-mentioned processing (such as the changes in step 102 and step 103) is based on the processing of the track line image, its coordinate system (i.e., the pixel coordinate system) is not transformed during the processing. Therefore, regardless of the image before and after processing, its coordinate system (i.e., the pixel coordinate system) is the same. If a certain pixel point of the track line image is used as the origin (such as the pixel point in the upper left corner of the track line image as the origin, in the actual processing process, other pixel points can also be selected as the origin, such as selecting the pixel point in the lower left corner of the track line image as the origin, or selecting the pixel point in the upper right corner of the track line image as the origin, or selecting the pixel point in the lower right corner of the track line image as the origin, or selecting the pixel point in the center of the track line image as the origin, etc., this embodiment does not limit the selection of the origin), a coordinate system (i.e., the pixel coordinate system) can be obtained. This embodiment and subsequent embodiments all take the pixel point at the lower left corner of the track line image as an example. Figure 8As shown in the pixel coordinate system, the horizontal coordinate value x of any pixel point (x, y) can represent the column (vertical) of the pixel point in the image, and the vertical coordinate value y can represent the row (horizontal) of the pixel point in the image. In other words, the pixel point (x, y) is the first pixel in the image. y The pixel point in the row xth column. If the resolution of the track line image is 1280×720, then there are 720 rows and 1280 columns of pixels in the image. Therefore, there are 1280 horizontal coordinate values in the pixel coordinate system, that is, there are 1280 columns of pixels in the image, and 720 vertical coordinate values, that is, there are 720 rows of pixels in the image.
[0121] It should be noted that the image here is a general term for various images, not a specific image, because all processing does not involve the transformation of the pixel coordinate system, so no matter what kind of image (such as the track line image, or the image after dedistortion, or the image after perspective transformation, or the first processed image, or the second processed image, or the third processed image, or the convolution image), its pixel coordinate system is consistent. That is to say, at any point (x, y) in the track line image, no matter in what kind of image, its coordinates are (x, y).
[0122] 2. The process of determining the track area based on the image after perspective transformation
[0123] The steps to achieve this process are as follows:
[0124] 301, convert the color space of the image after perspective transformation into the HLS color space to obtain an L channel image after space transformation.
[0125] The color space of the image after perspective transformation is RGB color space, and its color space will be converted into HLS color space through step 301. The three channels of HLS color space are hue (H), luminance (L), and saturation (S). By converting the color space of the image after perspective transformation into HLS color space, three channel images can be obtained, namely, H channel image, L channel image, and S channel image. Step 301 obtains the L channel image.
[0126] In addition, color space conversion does not involve coordinate system conversion. Therefore, the coordinate system of the L channel image is consistent with the coordinate system of other images, which are the same pixel coordinate system.
[0127] 302, determine the total number of white pixels in each column of pixels in the L channel image.
[0128] The white color in the L channel image is the track line of the third track, so in step 302, the total number of white pixels in each column of pixels in the L channel image is determined.
[0129] In the pixel coordinate system, based on the horizontal coordinate, the number of white pixels in all pixels with the same horizontal coordinate is determined, and the number is the total number of white pixels in the column corresponding to the horizontal coordinate.
[0130] In actual application, the relative positions between the third rail and the left and right rails are not fixed. Instead, there are two situations: one is that the left and right rails are located on the right side of the third rail, and the other is that the left and right rails are located on the left side of the third rail. Fig. 9 It shows the statistical diagram of white pixels on the right side of the third track on the left and right tracks. Fig.10 The white pixel statistics of the left and right tracks are shown. Fig. 9 still Fig.10 , the image on the upper left is the image after perspective transformation, and its color space is RGB color space; the image on the lower left is the L channel image after the color space of the upper left image is converted to HLS color space; the image on the right is a statistical chart of the total number of white pixels in each column of pixels in the L channel image. Taking the resolution of 1280×720 as an example, 1280 values will be counted, and these values can be drawn into a statistical chart, where the horizontal direction in the statistical chart represents the number of columns of the L channel image (from 1 to 1280), and the vertical direction in the statistical chart represents the total number of white pixels in each column.
[0131] 303 , determine the column with the largest total number of white pixels as the third track line area.
[0132] For example, analysis Fig. 9 or Fig.10 As shown in the statistical chart, find the column with the largest value (that is, the column with the largest total number of white pixels). This column is where the third track is located, that is, the third track line area, thereby realizing the area determination of the track line of the third track.
[0133] 304. Determine left and right track line areas based on the third track line area.
[0134] The implementation process of step 304 is:
[0135] 304-1, clipping the L channel image in the target direction of the third track line area to obtain a driving track area.
[0136] Among them, the target direction is the direction in which the left and right rails are located on the third rail.
[0137] like Fig. 9 As shown, the left and right rails are located on the right side of the third rail. Fig. 9 The target direction is the right direction. Fig.10 As shown, the left and right rails are located on the left side of the third rail. Fig.10In the case of , the target direction is the left direction. That is, the target direction is the direction in which the left and right rails are located on the third rail in actual application.
[0138] In 304-1, the L channel image is cropped according to the third track line area obtained in step 303. Since the third track is outside the running track (i.e., the left and right tracks) (i.e., the left and right tracks are located on the right side of the third track, or the left and right tracks are located on the left side of the third track) and the symmetry of the running track (i.e., the left and right tracks), the running track area obtained by 304-1, i.e., the image of the area of the two tracks, is as shown in FIG. Fig.11 shown.
[0139] 304-2, detecting straight line segments in the driving track area.
[0140] The L channel image is cropped through step 304-1 to obtain a cropped image, which is the image of the driving track area. In step 304-2, a line extraction operation is performed on the cropped image (i.e., the image of the driving track area), such as detecting the line segment in the cropped image (i.e., the image of the driving track area) through the LSD algorithm in OpenCV, and returning the starting point and end point coordinates of the straight line segment, which characterize the straight line segment in the driving track area.
[0141] 304-3, clustering the pixels of each straight line segment based on the column where each pixel in the straight line segment is located.
[0142] After the straight line segments are detected in step 304-2 (actually, the coordinates of the start and end points of the straight line segments are obtained), the horizontal coordinates of these line segments in the pixel coordinate system are extracted, and the straight line segments are divided into left and right groups by a clustering algorithm (such as K-means clustering). The purpose of clustering is to divide the straight line segments into left and right sides according to their positions, so as to facilitate the subsequent analysis of the left and right track line areas of the driving track.
[0143] 304-4, determine the column where the cluster center is located as the left and right track line area.
[0144] Based on the cluster center obtained in step 304-3, the straight line segment is divided into two groups, the left and right groups. The cluster center can be understood as the center position of the straight line segment on the horizontal coordinate in the pixel coordinate system. This position (i.e., the corresponding column) is the left and right track line area. Therefore, the left and right track line area can be obtained according to the horizontal coordinate corresponding to the center position, such as Fig.11 shown.
[0145] Through steps 301 to 304, the areas of the third rail and the left and right running rails can be obtained, which provides a basis for subsequent track line detection.
[0146] 104, based on the track area, slide the sliding window on the convolution image, determine the track edge in the sliding window after each sliding, and perform track line detection based on the track edge.
[0147] This step is implemented based on a sliding window. The implementation details are as follows:
[0148] 104-1, based on the initial pixel point, determine the initial position of the sliding window in the track area.
[0149] If it is the first sliding, the initial pixel point is a preset pixel point. If it is not the first sliding, the initial pixel point is determined based on the last target position of the sliding window.
[0150] In addition, the track area includes two types, one is the third track line area, and the other is the left and right track line area. The size of the sliding window can be pre-set, and the sliding window sizes corresponding to the left and right tracks and the third track can be the same or different (that is, when processing the third track line area, the sliding window size corresponding to the third track is used, and when processing the left and right track line areas, the sliding window size corresponding to the left and right tracks is used). This embodiment and subsequent embodiments are described by taking the sliding window size corresponding to the left and right tracks as 15 pixels high and 60 pixels wide; the sliding window size corresponding to the left and right tracks as 15 pixels high and 100 pixels wide as an example.
[0151] In addition, the initial pixel point is also preset, and this embodiment does not limit it. For example, the pixel point corresponding to the origin position in the pixel coordinate system is set as a preset pixel point, then if step 104-1 is executed for the first time, the pixel point corresponding to the origin position can be used as the initial pixel point, if it is not the first time to execute step 104-1, it means that step 104-1 has been executed before, then when step 104-1 was executed last time, a target position of a sliding window was determined, and the initial pixel point can be determined based on the last target position of the sliding window during this execution, for example, the pixel point at the lower left corner of the sliding window in the last target position of the sliding window is determined as the initial pixel point, or the pixel point at the lower right corner of the sliding window in the last target position of the sliding window is determined as the initial pixel point, etc.
[0152] In step 104-1, the initial position may be used as the position of the upper left corner of the sliding window, thus obtaining an initial sliding window.
[0153] 104-2, based on the column of the track area within the initial position, adjust the initial position to obtain the target position.
[0154] In step 104-2, when adjusting the initial position based on the column of the track area within the initial position, the initial sliding window can be translated left and right based on the horizontal coordinate (i.e., column) of the initial position, on the premise that the vertical coordinate (i.e., row) of the sliding window in the pixel coordinate system remains unchanged, to obtain the target position of the sliding window.
[0155] For example, if the track area is the third track line area, the sliding window obtained in step 104-1 can be translated left and right so that the initial pixel point is located at the lower left corner (or lower right corner) of the sliding window, and the position of the sliding window after adjustment is the target position of the sliding window. If the track area is the left and right track line areas, the sliding window obtained in step 104-1 can be translated left and right so that the initial pixel point is located at the center point of the lowest line of the sliding window, and the position of the sliding window after adjustment is the target position of the sliding window.
[0156] 104-3, the sliding window on the convolution image slides to the target position.
[0157] In step 104-2, the target position of the sliding window can be determined based on the track area, such as the coordinates of all pixels covered by the sliding window. Since the pixel coordinate system does not change in the method provided in this embodiment, the target position of the sliding window in step 104-2 (such as the coordinates of all pixels covered by the sliding window) can be used to obtain the area with the same coordinates on the convolution image, which is the current target position of the sliding window on the convolution image. Therefore, after obtaining the target position in step 104-2, the sliding window on the convolution image can be slid to the target position, thereby realizing the linkage between the track area and the convolution image. Because the track area is an L-channel image, the image has better recognition accuracy, and the target position determined by the track area is more accurate, thereby ensuring the accurate position of the sliding window on the convolution image.
[0158] 104-4, sum the values of each pixel in the sliding window after sliding by column, and determine the column with the maximum sum as the track edge in the sliding window.
[0159] In step 104-4, the values of each pixel in the sliding window in the convolution image are summed by column, and the column with the maximum sum is determined as the track edge in the sliding window, thereby achieving accurate track line positioning, such as Fig.12 shown.
[0160] The track line in the sliding window can be obtained by performing steps 104-1 to 104-4 once each. The track line in each sliding window can be obtained in sections by sliding the sliding window in the track area.
[0161] In a specific implementation, steps 104 - 1 to 104 - 4 are repeatedly executed until the sliding window completely slides over the track area, thereby detecting and obtaining all track lines of the track line image.
[0162] For example, the sizes of the sliding windows of the three tracks are predetermined, such as the sliding window width of the left and right running tracks is 100 pixels and the height is 15 pixels, and the sliding window width of the third track is 100 pixels and the height is 15 pixels. Each time steps 104-1 to 104-4 are executed, the initial position of the sliding window is determined, and then the initial position is adjusted based on the column (i.e., x coordinate) of the initial position to obtain the target position, such as the left and right running tracks use the initial position as the midpoint of the lower edge line of the sliding window, and the third track uses the initial position as the lower left corner of the sliding window. Then the sliding window on the convolution image slides to the target position, and the values of each pixel in the sliding window after sliding in the convolution image are summed by column, and the column with the maximum sum is determined as the track edge in the sliding window. Then steps 104-1 to 104-4 are executed again to determine the initial position of the next sliding window, and then the track edge in the next sliding window is detected until all rows are fully searched. Through this sliding window strategy, the running track line can be effectively tracked and located, and the accurate extraction of the track position can be achieved.
[0163] The track line detection method provided in this embodiment is applicable to a variety of scenarios such as straight lines, curves, and three-track position changes, ensuring accurate detection of track lines, and providing efficient and accurate data feature support for subsequent tasks such as intrusion detection and automatic driving, which helps to improve the intelligence level of urban rail transit systems and plays a key role in the sustainable development of urban rail transit. At the same time, the track line detection method provided in this embodiment can achieve high real-time track line detection, which can improve the safety and reliability of the train automatic driving system.
[0164] This embodiment provides a track line detection method, which acquires a track line image; after dedistorting the track line image, perspective transforms the dedistorted image to obtain a perspective transformed image; based on the perspective transformed image, determines a convolution image and a track area; based on the track area, slides a sliding window on the convolution image, and determines the track edge in the sliding window after each sliding, and performs track line detection based on the track edge, thereby ensuring the real-time performance of the track line detection.
[0165] Based on the same inventive concept of the track line detection method, this embodiment provides a track line detection device, Fig.13 , the device comprises:
[0166] The acquisition module 1301 is used to acquire a track line image.
[0167] The processing module 1302 is used to perform dedistortion processing on the track line image acquired by the acquisition module 1301, and then perform perspective transformation on the dedistorted image to obtain a perspective transformed image.
[0168] The determination module 1303 is used to determine the convolution image and the track area based on the image after the perspective transformation obtained by the processing module 1302.
[0169] The detection module 1304 is used to slide the sliding window on the convolution image based on the track area, determine the track edge in the sliding window after each sliding, and perform track line detection based on the track edge.
[0170] The processing module 1302 is used to determine the track area in front of the vehicle in the dedistorted image as the region of interest. Determine the coordinates src_points of the four vertices of the target area. The target area is the smallest area including the edges of the left and right track lines in the region of interest. Determine the coordinates dst_points of the four vertices in the perspective transformed image. The dedistorted image is transformed into a perspective transformed image by a projection matrix based on src_points and dst_points.
[0171] The determination module 1303 is used to perform histogram equalization processing on the image after perspective transformation to obtain a first processed image. The first processed image is subjected to Gaussian blur processing to obtain a second processed image. The dedistorted image is fused with the second processed image to obtain a third processed image. The third processed image is subjected to convolution processing based on an edge detection convolution kernel to obtain a convolved image.
[0172] Among them, the edge detection convolution kernel is
[0173] Among them, the determination module 1303 is used to convert the color space of the image after perspective transformation into the HLS color space to obtain the L channel image after space transformation. Determine the total number of white pixels in each column of pixels in the L channel image. Determine the column with the largest total number of white pixels as the third track line area. Based on the third track line area, determine the left and right track line areas.
[0174] Wherein, based on the third track line area, determining the left and right track line areas includes: clipping the L channel image in the target direction of the third track line area to obtain the driving track area. Wherein, the target direction is the direction in which the left and right tracks are located on the third track. Detecting the straight line segments in the driving track area. Clustering the pixel points of each straight line segment based on the column where each pixel point in the straight line segment is located. Determining the column where the clustered class center is located as the left and right track line area.
[0175] Among them, the detection module 1304 is used to determine the initial position of the sliding window in the track area based on the initial pixel point. If it is the first sliding, the initial pixel point is a preset pixel point. If it is not the first sliding, the initial pixel point is determined based on the last target position of the sliding window. Based on the column of the track area in the initial position, the initial position is adjusted to obtain the target position. The sliding window on the convolution image slides to the target position. The values of each pixel in the sliding window after sliding are summed by column, and the column with the maximum sum is determined as the track edge in the sliding window.
[0176] The track line detection device provided in this embodiment determines the convolution image and the track area based on the image after perspective transformation; based on the track area, the sliding window is slid on the convolution image, and the track edge in the sliding window after each sliding is determined, and the track line detection is performed based on the track edge, thereby ensuring the real-time performance of the track line detection.
[0177] Based on the same inventive concept of the track line detection method, this embodiment provides an electronic device, the electronic device is as follows Fig.14 As shown, it includes: a memory 1401, a processor 1402, and a computer program.
[0178] The computer program is stored in the memory 1401 and is configured to be executed by the processor 1402 to implement the above-mentioned column track line detection method.
[0179] Specifically,
[0180] Get the track line image.
[0181] After the track line image is dedistorted, the dedistorted image is perspective transformed to obtain a perspective transformed image.
[0182] Based on the perspective transformed image, the convolution image and the track region are determined.
[0183] Based on the track area, the sliding window is slid on the convolution image, and the track edge within the sliding window after each sliding is determined, and the track line detection is performed based on the track edge.
[0184] Optionally, performing perspective transformation on the dedistorted image to obtain a perspective transformed image includes:
[0185] The track area directly in front of the vehicle in the dedistorted image is determined as the region of interest.
[0186] Determine the four vertex coordinates src_points of the target area, where the target area is the smallest area in the region of interest that includes the edges of the left and right track lines.
[0187] Determine the coordinates dst_points of the four vertices in the perspective transformed image.
[0188] The dedistorted image is transformed into a perspective transformed image by a projection matrix based on src_points and dst_points.
[0189] Optionally, determining a convolution image based on the perspective transformed image includes:
[0190] The image after perspective transformation is subjected to histogram equalization processing to obtain a first processed image.
[0191] Gaussian blur processing is performed on the first processed image to obtain a second processed image.
[0192] The dedistorted image is fused with the second processed image to obtain a third processed image.
[0193] The third processed image is convolved based on the edge detection convolution kernel to obtain a convolved image.
[0194] Optionally, the edge detection convolution kernel is
[0195] Optionally, determining the track area based on the perspective transformed image includes:
[0196] The color space of the perspective transformed image is converted into the HLS color space to obtain the L channel image after space conversion.
[0197] Determine the total number of white pixels in each column of pixels in the L channel image.
[0198] The column with the largest total number of white pixels is determined as the third track line area.
[0199] Based on the third track line area, the left and right track line areas are determined.
[0200] Optionally, based on the third track line area, determining the left and right track line areas includes:
[0201] The L channel image is clipped in the target direction of the third track line area to obtain the driving track area, wherein the target direction is the direction in which the left and right tracks are located on the third track.
[0202] Detect straight line segments in the driving track area.
[0203] The pixels of each straight line segment are clustered based on the column where each pixel point in the straight line segment is located.
[0204] The column where the clustered class center is located is determined as the left and right track line area.
[0205] Optionally, based on the track area, sliding the sliding window on the convolution image, and determining the track edge within the sliding window after each sliding, including:
[0206] Based on the initial pixel point, the initial position of the sliding window is determined in the track area. If it is the first sliding, the initial pixel point is a preset pixel point, and if it is not the first sliding, the initial pixel point is determined based on the last target position of the sliding window.
[0207] Based on the columns of the track area within the initial position, the initial position is adjusted to obtain the target position.
[0208] The sliding window on the convolved image slides to the target location.
[0209] The values of each pixel in the sliding window after sliding are summed by column, and the column with the maximum sum is determined as the track edge in the sliding window.
[0210] The electronic device provided by this embodiment has a computer program executed by a processor to determine a convolution image and a track area based on the image after perspective transformation; based on the track area, a sliding window is slid on the convolution image, and the track edge within the sliding window after each sliding is determined, and track line detection is performed based on the track edge, thereby ensuring the real-time performance of the track line detection.
[0211] Based on the same inventive concept of the track line detection method, this embodiment provides a computer-readable storage medium, and a computer program is stored thereon. The computer program is executed by a processor to implement the track line detection method.
[0212] Specifically,
[0213] Get the track line image.
[0214] After the track line image is dedistorted, the dedistorted image is perspective transformed to obtain a perspective transformed image.
[0215] Based on the perspective transformed image, the convolution image and the track region are determined.
[0216] Based on the track area, the sliding window is slid on the convolution image, and the track edge within the sliding window after each sliding is determined, and the track line detection is performed based on the track edge.
[0217] Optionally, performing perspective transformation on the dedistorted image to obtain a perspective transformed image includes:
[0218] The track area directly in front of the vehicle in the dedistorted image is determined as the region of interest.
[0219] Determine the four vertex coordinates src_points of the target area, where the target area is the smallest area in the region of interest that includes the edges of the left and right track lines.
[0220] Determine the coordinates dst_points of the four vertices in the perspective transformed image.
[0221] The dedistorted image is transformed into a perspective transformed image by a projection matrix based on src_points and dst_points.
[0222] Optionally, determining a convolution image based on the perspective transformed image includes:
[0223] The image after perspective transformation is subjected to histogram equalization processing to obtain a first processed image.
[0224] Gaussian blur processing is performed on the first processed image to obtain a second processed image.
[0225] The dedistorted image is fused with the second processed image to obtain a third processed image.
[0226] The third processed image is convolved based on the edge detection convolution kernel to obtain a convolved image.
[0227] Optionally, the edge detection convolution kernel is
[0228] Optionally, determining the track area based on the perspective transformed image includes:
[0229] The color space of the perspective transformed image is converted into the HLS color space to obtain the L channel image after space conversion.
[0230] Determine the total number of white pixels in each column of pixels in the L channel image.
[0231] The column with the largest total number of white pixels is determined as the third track line area.
[0232] Based on the third track line area, the left and right track line areas are determined.
[0233] Optionally, based on the third track line area, determining the left and right track line areas includes:
[0234] The L channel image is clipped in the target direction of the third track line area to obtain the driving track area, wherein the target direction is the direction in which the left and right tracks are located on the third track.
[0235] Detect straight line segments in the driving track area.
[0236] The pixels of each straight line segment are clustered based on the column where each pixel point in the straight line segment is located.
[0237] The column where the clustered class center is located is determined as the left and right track line area.
[0238] Optionally, based on the track area, sliding the sliding window on the convolution image, and determining the track edge within the sliding window after each sliding, including:
[0239] Based on the initial pixel point, the initial position of the sliding window is determined in the track area. If it is the first sliding, the initial pixel point is a preset pixel point, and if it is not the first sliding, the initial pixel point is determined based on the last target position of the sliding window.
[0240] Based on the columns of the track area within the initial position, the initial position is adjusted to obtain the target position.
[0241] The sliding window on the convolved image slides to the target location.
[0242] The values of each pixel in the sliding window after sliding are summed by column, and the column with the maximum sum is determined as the track edge in the sliding window.
[0243] The computer-readable storage medium provided in this embodiment has a computer program executed by a processor to determine a convolution image and a track area based on the image after perspective transformation; based on the track area, a sliding window is slid on the convolution image, and the track edge within the sliding window after each sliding is determined, and track line detection is performed based on the track edge, thereby ensuring the real-time performance of track line detection.
[0244] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code. The scheme in the embodiments of the present application can be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.
[0245] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0246] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0247] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.
[0248] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.
[0249] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.
[0250] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.
Claims
1. A track line detection method, characterized in that: The method comprises: Get track line images; After dedistorting the track line image, perspective transforming the dedistorted image to obtain a perspective transformed image; Determining a convolution image and a track region based on the perspective transformed image; Based on the track area, a sliding window is slid on the convolution image, and a track edge in the sliding window after each sliding is determined, and track line detection is performed based on the track edge.
2. The method according to claim 1, characterized in that The step of performing perspective transformation on the dedistorted image to obtain a perspective transformed image includes: The track area directly in front of the vehicle in the dedistorted image is determined as the region of interest; Determine the four vertex coordinates src_points of the target area; wherein the target area is the smallest area in the region of interest including the edges of the left and right track lines; Determine the coordinates dst_points of the four vertices in the perspective transformed image; The dedistorted image is transformed into a perspective transformed image by a projection matrix based on src_points and dst_points.
3. The method according to claim 1, characterized in that Determining a convolution image based on the perspective transformed image includes: Performing histogram equalization processing on the image after the perspective transformation to obtain a first processed image; Performing Gaussian blur processing on the first processed image to obtain a second processed image; fusing the dedistorted image with the second processed image to obtain a third processed image; The third processed image is convolved based on an edge detection convolution kernel to obtain a convolved image.
4. The method according to claim 3, characterized in that The edge detection convolution kernel is 5. The method according to claim 1, characterized in that Determining a track area based on the perspective transformed image includes: Convert the color space of the perspective transformed image into the HLS color space to obtain an L channel image after space conversion; Determine the total number of white pixels in each column of pixels in the L channel image; The column with the largest total number of white pixels is determined as the third track line area; Based on the third track line area, the left and right track line areas are determined.
6. The method according to claim 5, characterized in that The determining of the left and right track line areas based on the third track line area includes: Clipping the L channel image in the target direction of the third track line area to obtain a running track area; wherein the target direction is the direction in which the left and right tracks are located on the third track; Detecting straight line segments in the driving track area; Clustering the pixel points of each straight line segment based on the column where each pixel point in the straight line segment is located; The column where the clustered class center is located is determined as the left and right track line area.
7. The method according to claim 1, characterized in that The step of sliding the sliding window on the convolution image based on the track area and determining the track edge in the sliding window after each sliding includes: Based on the initial pixel point, determine the initial position of the sliding window in the track area; wherein, if it is the first sliding, the initial pixel point is a preset pixel point, and if it is not the first sliding, the initial pixel point is determined based on the last target position of the sliding window; Based on the columns of the track area within the initial position, adjusting the initial position to obtain a target position; Sliding the sliding window on the convolution image to the target position; The values of each pixel in the sliding window after sliding are summed by column, and the column with the maximum sum is determined as the track edge in the sliding window.
8. A track line detection device, characterized in that: The device comprises: An acquisition module, used for acquiring a track line image; A processing module, configured to perform a dedistortion process on the track line image acquired by the acquisition module, and then perform a perspective transformation on the dedistorted image to obtain a perspective transformed image; A determination module, configured to determine a convolution image and a track area based on the perspective transformed image obtained by the processing module; The detection module is used to slide the sliding window on the convolution image based on the track area, determine the track edge in the sliding window after each sliding, and perform track line detection based on the track edge.
9. An electronic device, characterized in that: include: Memory; processor; as well as Computer programs; The computer program is stored in the memory and is configured to be executed by the processor to implement the method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: A computer program is stored thereon; the computer program is executed by a processor to implement the method according to any one of claims 1 to 7.
