A rail transit line adaptive marking method, computer equipment and storage medium

Abstract

The present invention discloses a rail transit line adaptive marking method, computer equipment and storage medium, wherein the marking method comprises the following steps: step S1: marking the first frame image; step S2: extracting the projection transformation law of the camera according to the marking information of step S1; then, extracting the image frame data in sequence, and performing the following processing: step S3: extracting edge information; step S4: adaptively expanding the edge information along the X direction; step S5: performing a preliminary Hough transform to extract the low-order parameters of the track centerline; step S6: performing a secondary Hough transform to extract the high-order parameters of the track centerline; step S7: calculating the track centerline; step S8: calculating the tracks on both sides of the track center according to the projection mapping law, and realizing the semantic segmentation of rail transit. The present invention makes full use of the stability of the Hough statistical feature calculation and reduces the algorithm complexity of the generalized Hough transform.

Description

Technical Field

[0001] The invention belongs to the technical field of rail transit, and in particular relates to a rail transit line marking method. Background Art

[0002] With the in-depth development of artificial intelligence, intelligent transportation, 5G communication technology and the gradual expansion of urban scale, more and more visual and deep learning technologies are being used in rail transit and unmanned driving industries. Traditional visual detection and lidar technologies are widely used in various occasions, such as rail transit obstacle detection, pedestrian detection in and out of the station, pantograph safety detection, or driving safety monitoring. Among them, the accurate positioning and identification of traffic rail lines is often the premise and basis of the entire driving safety detection, which is extremely important in rail transit safety monitoring and intelligent driving.

[0003] In terms of track recognition and detection, there are many studies at home and abroad. The algorithms mainly focus on various edge extraction operators, such as using the Sobel operator and its improved version to extract the edge information of the track, and then detecting the track by the straight line fitting method. However, this method is only suitable for straight line methods and has large errors in the detection of curved tracks. Some scholars have proposed using image matching technology, Hog features or mathematical morphology to identify tracks. The algorithms are more complex than the previous simple edge detection algorithms, and have achieved good detection results, but such algorithms are often easily affected by the quality of the image.

[0004] Traditional track detection algorithms are often based on simple grayscale or edge information, and often simply consider the detection of straight tracks or nearly straight tracks. The algorithm is not robust enough and the detection is time-consuming. At the same time, due to long-term exposure to the outdoors, the surface of the locomotive track is severely worn, and the collected image data varies greatly. At the same time, when the vehicle is driving in real time, environmental factors will greatly affect the collected data, such as rainy and snowy weather, locomotive vibration, multiple tracks, and various ambient light sources.

[0005] At the same time, with the development of artificial intelligence technology based on deep learning, more and more deep networks have been introduced into the semantic segmentation of rail transit, such as Unet, Polylane and PiNet, which have achieved good segmentation results. However, in some complex areas of track lines, such as areas with many curves, the segmentation effect is easily affected by external factors and is not stable enough. In addition, since the deep network is end-to-end, once the training is completed, the internal parameters are difficult to adjust directly. Summary of the invention

[0006] The technical problem to be solved by the present invention is to provide a rail transit line adaptive marking method, computer equipment and storage medium to accurately estimate the fitting parameters of the track centerline, thereby avoiding the overall deviation of the track calculation caused by local positioning deviation, strong light and other factors.

[0007] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0008] On the one hand, a method for adaptively marking a rail transit line is provided, comprising the following steps:

[0009] Step S1: mark the first frame image, assume that the track line extends along the Y direction of the image, and the direction perpendicular to the Y direction is the X direction. Along the extension direction of the track line, mark the tracks on both sides of the track line. Assume that the track line data on both sides of the Y direction are the left and right sides, which are represented by dot matrices, EdgeL and EdgeR respectively;

[0010] Step S2: Extract the projection transformation law of the camera based on the marking information in step S1. The trajectory line changes with Y and is described by a one-dimensional array EdgeW, as shown in formula (1).

[0011] EdgeW=EdgeR(X)-EdgeL(X) (1)

[0012] Subtract the X coordinate on the left from the value on the right side of the track line, and record the track width corresponding to each Y value in the image; then, extract the image frame data one by one in order and perform the following processing:

[0013] Step S3: extract edge information;

[0014] Step S4: adaptively expanding the edge information along the X direction;

[0015] Step S5: preliminary Hough transform to extract low-order parameters of the track centerline;

[0016] Step S6: secondary Hough transform to extract high-order parameters of the track centerline;

[0017] Step S7: Calculate and obtain the track centerline;

[0018] Step S8: According to the projection mapping rule, the tracks on both sides of the track center are calculated to achieve semantic segmentation of rail transit.

[0019] Preferably, step S3 comprises the following steps:

[0020] 3.1: Assuming that the image of the current frame is G1, the edge information is extracted according to the Canny edge detection algorithm. The image obtained at this time is G2, as shown in formula (2).

[0021] G2 = Canny(G1) (2);

[0022] 3.2: For image G2, irrelevant edge points are filtered out through the pre-set ROI area, and a valid image G3 is generated;

[0023] 3.3: For image G3, assume that the set of all edge points is E1(P), and further filter out irrelevant points through the constraint mechanism; for all points of E1(P), if there are neighboring edge points, the spacing in the X direction is close to EdgeW, then the extracted point set is E2(P), and the remaining point set is E3(P);

[0024] 3.4: For any point in the point set E3(P), if the horizontal distance and vertical distance with some points in the point set E2(P) do not exceed the set range, these points can be regarded as potential track edge points and merged with the initially screened point set E2(P) to generate a new valid track point set E4(P).

[0025] Preferably, step S4 comprises the following steps:

[0026] For any point data Pi in E4(P), expand left and right along the X direction, and the expanded width is 1 / 2 of the track width of the area where the point is located.

[0027] PLi=(Pi.X-Wi / 2,Pi.Y)

[0028] PRi=(Pi.X-Wi / 2,Pi.Y)

[0029] Among them, PLi and PRi are the edge points of Pi extending to the left and right directions respectively, their Y values ​​remain unchanged, while their X values ​​differ from the X value of Pi by Wi / 2, and the track projection width corresponding to the point Wi is the corresponding width value when y=Pi.Y in the EW data;

[0030] With points PLi and PRi as the center, the points are adaptively expanded by 1-3 pixels to the left and right, so that a single point is expanded into 3 to 7 points. At this time, all the expanded points may be located on the center line of the track. These expanded points can be recorded as the new point set E5(P).

[0031] Preferably, in step S5, the track centerline in the image is described by a curve of no more than 3 degrees, as shown in formula (3),

[0032] x=a3y 3 +a2y 2 +a1y+a0 (3)

[0033] The parameters a3 and a2 are relatively small, so they are assumed to be 0 and simplified to the linear equation (4)

[0034] x=a1y+a0 (4)

[0035] At this time, first find the main tangent direction of the target curve, and make appropriate transformations on formula (4) to generate formula (5).

[0036] a0=-a1y+x (5)

[0037] For the point set E5(P), according to formula (5), any point is mapped to a straight line in the Hough space, where a1 is the independent variable and a0 is the dependent variable; then, the coordinates corresponding to the maximum value in the parameter space are obtained after the accumulation of each point and the mapping is completed, which is the desired optimal a1 and a0.

[0038] Preferably, in step S6, when the low-order parameters are known, the high-order parameters of equation (3) are calculated by Hough transform, and equation (3) is written as formula (6),

[0039] a2=-ya3+(-x-a0-a1y) / y 2 (6)

[0040] For the point set E5(P), according to formula (6), any point is mapped to a straight line in the Hough space, where a3 is the independent variable and a2 is the dependent variable; then, the points are accumulated one by one, and after the mapping is completed, the point coordinates corresponding to the maximum value in the two-dimensional parameter space are calculated, which are the desired optimal a2 and a3.

[0041] Preferably, in step S7, based on the track centerline parameters that have been calculated, when the Y value ranges from 0 to H-1, H is the image height, the corresponding X value coordinate is calculated, and the entire track centerline, that is, the point set E6 (P), is obtained.

[0042] Preferably, if there are multiple fitted curves, the average track deviations of the previous and next frames are compared, and the track with the smallest average track deviation is selected as the track line of the current frame;

[0043] The calculation of the offset EdgeBias is shown in formula (7), where E0(P) is the track centerline point set of the previous frame,

[0044]

[0045] Preferably, in step S8, for the track centerline point set E6(P), when the Y value ranges from 0 to H-1, the track width is retrieved from EdgeW; at this time, the segmented area of ​​the track is an area centered on the track centerline and with an extension width of EdgeW / 2 in the X direction.

[0046] On the other hand, a computer device is provided, comprising at least one processor and a memory; the memory stores computer-executable instructions; the at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor implements the method for adaptive marking of rail transit lines.

[0047] In yet another aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer-executable instructions, and when a processor executes the computer-executable instructions, the method for adaptively marking a rail transit line is implemented.

[0048] The present invention adopts a target recognition strategy based on quadratic Hough transform, and locates the train centerline through edge extraction and camera projection transformation. At the same time, the fitting parameters of the track are extracted in batches through quadratic transformation. Based on the above technical solution, the beneficial effects of the present invention are:

[0049] 1. In terms of algorithm strategy, the calculation process of quadratic Hough transform is adopted, which makes full use of the stability of Hough statistical feature calculation and reduces the algorithm complexity of generalized Hough transform.

[0050] 2. It supports both straight track calculation and curved track calculation. Through the correlation calculation between consecutive frames, the interfering rail transit lines can be eliminated.

[0051] 3. In terms of edge computing, grayscale and gradient image features are used to avoid low image detection robustness caused by single grayscale or edge information, which is easily affected by external ambient light and causes calculation deviations.

[0052] 4. In terms of projection mapping, making full use of the first frame of annotated data and analyzing the track width that changes with the Y value in the image can effectively avoid linear calculation of the mapping formula, and is more accurate, reducing the probability of misjudgment.

[0053] The specific technical solutions and beneficial effects of the present invention will be described in detail in the following specific embodiments in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments:

[0055] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is by no means intended to limit the present invention and its application or use. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0057] Aiming at the driving safety requirements of rail transit, the present invention proposes a rail transit line adaptive marking method based on quadratic Hough transform, taking into full consideration that the quality of images collected by high-speed moving vehicles during the detection process is not stable enough and is easily disturbed by various weather conditions, platform vibrations, angle changes and other external factors, and adopts a target recognition strategy based on quadratic Hough transform to locate the train centerline through edge extraction and camera projection transformation. At the same time, through the quadratic transform, the fitting parameters of the track are extracted in batches, which significantly reduces the computational complexity of solving the traditional multi-parameter generalized Hough transform.

[0058] like Figure 1 As shown, a rail transit line adaptive marking algorithm based on quadratic Hough transform specifically includes the following steps:

[0059] Step 1: Manually mark the first frame image.

[0060] The specific process is as follows: along the extension direction of the track line, mark the tracks on both sides of the track line. Generally, the track line generally extends along the Y direction of the image, and the direction perpendicular to the Y direction is the X direction. Assume that the track line data on both sides of the Y direction are the left and right sides, represented by dot matrix, respectively EdgeL and EdgeR.

[0061] Step 2: Extract the camera's projection transformation rule based on the marker information.

[0062] Generally, the camera projection process is similar to pinhole imaging. As the distance increases, its size in the image decreases proportionally. Therefore, as the track line extends, the distance from the camera increases, and the track width gradually decreases. Among them, its track changes with Y and can be described by a one-dimensional array EdgeW, as shown in formula (1). The value on the right side of the track line is subtracted from the X coordinate on the left side, and the track width corresponding to each Y value in the image is recorded.

[0063] EdgeW=EdgeR(X)-EdgeL(X) (1)

[0064] Next, extract the image frame data one by one in order and perform the following processing:

[0065] Step 3: Extract edge information.

[0066] The specific process is as follows:

[0067] 3.1 Assume that the image of the current frame is G1, and extract the edge information based on the Canny edge detection algorithm. The image obtained at this time is G2, as shown in formula (1).

[0068] G2=Canny(G1) (2)

[0069] 3.2 For image G2, irrelevant edge points are filtered out through the preset ROI area, and a valid image G3 is generated.

[0070] 3.3 For image G3, assuming that the set of all edge points is E1(P), we can further filter out irrelevant points through the constraint mechanism. For all points in E1(P), if there are neighbor edge points, the spacing in the X direction is close to EdgeW. At this time, the extracted point set is E2(P), and the remaining point set is E3(P).

[0071] 3.4 For any point in the point set E3(P), if the horizontal distance and vertical distance from some points in the point set E2(P) do not exceed the set range (for example, the deviation in the X direction is less than 10 pixels and the deviation in the Y direction is less than 50 pixels), then these points can be regarded as potential track edge points and combined with the initially screened point set E2(P) to generate a new valid track point set E4(P). This method allows track edge points with incomplete edge information to enter the subsequent calculation and analysis process.

[0072] Step 4: Adaptively expand the edge information along the X direction. The specific process is as follows:

[0073] For any point data Pi in E4(P), it is expanded left and right along the direction of X. The width of the expansion is 1 / 2 of the track width of the area where the point is located.

[0074] PLi=(Pi.X-Wi / 2,Pi.Y)

[0075] PRi=(Pi.X-Wi / 2,Pi.Y)

[0076] Among them, PLi and PRi are the edge points of Pi extending to the left and right directions respectively, their Y values ​​remain unchanged, while their X values ​​differ from the X value of Pi by Wi / 2, and the track projection width corresponding to the point Wi is the corresponding width value when y=Pi.Y in the EW data.

[0077] Considering the influence of lighting and environment, the error of edge computing, or the influence of camera viewing angle, there are errors in the evaluation of the track centerline. Therefore, for points PLi and PRi, we can further expand them to the left and right by 1-3 pixels adaptively, so that a single point can be expanded into 3 to 7 points.

[0078] At this time, all the extension points may be located on the center line of the track, and these extension points can be recorded as a new point set E5(P).

[0079] In the above-mentioned edge computing, grayscale and gradient image features are used to avoid the low robustness of image detection caused by single grayscale or edge information, which is easily affected by external ambient light and causes calculation deviations.

[0080] Step 5: Perform a preliminary Hough transform to extract the low-order parameters of the track centerline.

[0081] In the collected frame data, the simplest track centerline is parallel to the Y axis. Considering the camera's shooting range and the restrictions on turning angles in rail transit, the track centerline in the image can generally be described by a curve of no more than 3 degrees.

[0082] That is, x = a3y 3 +a2y 2 +a1y+a0 (3)

[0083] The parameters a3 and a2 are relatively small, so they are assumed to be 0 and simplified to the linear equation (3)

[0084] x=a1y+a0 (4)

[0085] At this point, the main tangent direction of the target curve can be calculated first.

[0086] Make appropriate transformations on formula (4) to generate formula (5).

[0087] a0=-a1y+x (5)

[0088] For the point set E5(P), any point can be mapped to a straight line in the Hough space according to formula (5), where a1 is the independent variable and a0 is the dependent variable. Then, the points are accumulated one by one. After the mapping is completed, the coordinates corresponding to the maximum value in the parameter space are calculated, which are the optimal a1 and a0 required.

[0089] Step 6. Perform secondary Hough transform to extract high-order parameters of the track centerline.

[0090] When the low-order parameters are known, the high-order parameters of equation (3) are further calculated by Hough transform. At this time, the formula can be written as (6).

[0091] a2=-ya3+(-x-a0-a1y) / y 2 (6)

[0092] For the point set E5(P), any point can be mapped to a straight line in the Hough space according to formula (6), where a3 is the independent variable and a2 is the dependent variable. Then, the points are accumulated one by one. After the mapping is completed, the coordinates of the point corresponding to the maximum value in the two-dimensional parameter space are calculated, which are the optimal a2 and a3 required.

[0093] Step 7: Calculate the track centerline.

[0094] According to the center line parameters calculated previously, when the Y value ranges from 0 to H-1 (H is the image height), the corresponding X value coordinate can be calculated. At this time, the center line of the entire track can be obtained, that is, the point set E6 (P).

[0095] If there are several track lines in the original frame data, there may be multiple local maximum values ​​in the Hough transform process, that is, there may be multiple fitted curves. At this time, compare the average deviation of the tracks of the previous and next frames, and select the track line with the smallest deviation as the track line of the current frame.

[0096] The calculation of the offset EdgeBias is shown in formula (7), where E0(P) is the track centerline point set of the previous frame.

[0097]

[0098] Step 8: According to the projection mapping rules, calculate the tracks on both sides of the track center to achieve semantic segmentation of rail transit.

[0099] For the track centerline point set E6(P), when the Y value is from 0 to H-1 (H is the image height), the track width can be retrieved from EdgeW. At this time, the track segmentation area is the area with the track centerline as the center and the extension width in the X direction is EdgeW / 2.

[0100] In summary, the present invention adopts the calculation process of the quadratic Hough transform, fully utilizes the stability of the Hough statistical feature calculation, and reduces the algorithm complexity of the generalized Hough transform. By calculating the correlation between consecutive frames, the interfering rail transit lines can be eliminated. In terms of projection mapping, the first frame data marked is fully utilized to analyze the track width that changes with the Y value in the image, which can effectively avoid the linear calculation of the mapping formula, and is more accurate, reducing the probability of misjudgment. Therefore, the fitting parameters of the track centerline can be accurately estimated, avoiding the overall deviation of the track calculation caused by local positioning deviation, strong light and other factors, and the system supports the detection of straight and curved tracks.

[0101] Embodiment 2

[0102] A computer device includes at least one processor and a memory; the memory stores computer-executable instructions; the at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes a rail transit line adaptive marking method as described in Example 1.

[0103] Embodiment 3

[0104] A computer-readable storage medium stores computer-executable instructions. When a processor executes the computer-executable instructions, a rail transit line adaptive marking method described in Example 1 is implemented.

[0105] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Those skilled in the art should understand that the present invention includes but is not limited to the contents described in the above specific embodiments. Any modification that does not deviate from the functional and structural principles of the present invention will be included in the scope of the claims.

Claims

1. A rail transit line adaptive marking method, characterized in that: The steps include: Step S1: mark the first frame image, assume that the track line extends along the Y direction of the image, and the direction perpendicular to the Y direction is the X direction. Along the extension direction of the track line, mark the tracks on both sides of the track line. Assume that the track line data on both sides of the Y direction are the left and right sides, which are represented by dot matrices, EdgeL and EdgeR respectively; Step S2: Extract the projection transformation law of the camera based on the marking information in step S1. The trajectory line changes with Y and is described by a one-dimensional array EdgeW, as shown in formula (1). EdgeW=EdgeR(X)-EdgeL(X) (1) Subtract the X coordinate on the left from the value on the right side of the track line, and record the track width corresponding to each Y value in the image; Next, extract the image frame data one by one in order and perform the following processing: Step S3: extract edge information; Step S4: adaptively expanding the edge information along the X direction; Step S5: preliminary Hough transform to extract low-order parameters of the track centerline; Step S6: secondary Hough transform to extract high-order parameters of the track centerline; Step S7: Calculate and obtain the track centerline; Step S8: According to the projection mapping rule, the tracks on both sides of the track center are calculated to achieve semantic segmentation of rail transit.

2. A rail transit line adaptive marking method according to claim 1, characterized in that: Step S3 includes the following steps: 3.1: Assuming that the image of the current frame is G1, the edge information is extracted according to the Canny edge detection algorithm. The image obtained at this time is G2, as shown in formula (2). G2 = Canny(G1) (2); 3.2: For image G2, irrelevant edge points are filtered out through the pre-set ROI area, and a valid image G3 is generated; 3.3: For image G3, assume that the set of all edge points is E1(P), and further filter out irrelevant points through the constraint mechanism; for all points of E1(P), if there are neighboring edge points, the spacing in the X direction is close to EdgeW, then the extracted point set is E2(P), and the remaining point set is E3(P); 3.4: For any point in the point set E3(P), if the horizontal distance and vertical distance with some points in the point set E2(P) do not exceed the set range, these points can be regarded as potential track edge points and merged with the initially screened point set E2(P) to generate a new valid track point set E4(P).

3. A rail transit line adaptive marking method according to claim 2, characterized in that: Step S4 includes the following steps: For any point data Pi in E4(P), expand left and right along the X direction, and the expanded width is 1 / 2 of the track width of the area where the point is located. PLi=(Pi.X-Wi / 2,Pi.Y) PRi=(Pi.X-Wi / 2,Pi.Y) Among them, PLi and PRi are the edge points of Pi extending to the left and right directions respectively, their Y values ​​remain unchanged, while their X values ​​differ from the X value of Pi by Wi / 2, and the track projection width corresponding to the point Wi is the corresponding width value when y=Pi.Y in the EW data; With points PLi and PRi as the center, the points are adaptively expanded by 1-3 pixels to the left and right, so that a single point is expanded into 3 to 7 points. At this time, all the expanded points may be located on the center line of the track. These expanded points can be recorded as the new point set E5(P).

4. A rail transit line adaptive marking method according to claim 3, characterized in that: In step S5, the track centerline in the image is described by a curve of no more than 3 degrees, as shown in formula (3), x=a3y 3 +a2y 2 +a1y+a0 (3) The parameters a3 and a2 are relatively small, so we assume them to be 0 and simplify them to the linear equation (4): x=a1y+a0 (4) At this time, first find the main tangent direction of the target curve, and make appropriate transformations on formula (4) to generate formula (5). a0=-a1y+x (5) For the point set E5(P), according to formula (5), any point is mapped to a straight line in the Hough space, where a1 is the independent variable and a0 is the dependent variable; then, the coordinates corresponding to the maximum value in the parameter space are obtained after the accumulation of each point and the mapping is completed, which is the desired optimal a1 and a0.

5. A rail transit line adaptive marking method according to claim 4, characterized in that: In step S6, when the low-order parameters are known, the high-order parameters of equation (3) are calculated by Hough transform, and equation (3) is written as formula (6): <h2 style=";text-align:left;direction:ltr">a2 = - ya3 + ( - x - a0 - a1y ) / y<h2 style=";text-align:left;direction:ltr"> 2 <h2 style=";text-align:left;direction:ltr"> (6) For the point set E5(P), according to formula (6), any point is mapped to a straight line in the Hough space, where a3 is the independent variable and a2 is the dependent variable; then, the points are accumulated one by one, and after the mapping is completed, the point coordinates corresponding to the maximum value in the two-dimensional parameter space are calculated, which are the desired optimal a2 and a3.

6. A rail transit line adaptive marking method according to claim 5, characterized in that: In step S7, according to the track centerline parameters that have been calculated, when the Y value ranges from 0 to H-1, H is the image height, the corresponding X value coordinate is calculated, and the entire track centerline, that is, point set E6 (P), is obtained.

7. A rail transit line adaptive marking method according to claim 6, characterized in that: If there are multiple fitted curves, compare the average track deviations of the previous and next frames, and select the one with the smallest average track deviation as the track line of the current frame; The calculation of the offset EdgeBias is shown in formula (7), where E0(P) is the track centerline point set of the previous frame, 8. A rail transit line adaptive marking method according to claim 6, characterized in that: In step S8, for the track centerline point set E6(P), when the Y value ranges from 0 to H-1, the track width is retrieved from EdgeW; at this time, the segmented area of ​​the track is an area centered on the track centerline with an extension width of EdgeW / 2 in the X direction.

9. A computer device, characterized in that: It comprises at least one processor and a memory; the memory stores computer-executable instructions; the at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes a rail transit line adaptive marking method as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions. When the processor executes the computer-executable instructions, the rail transit line adaptive marking method as described in any one of claims 1 to 8 is implemented.

Images