Railway turnout opening determination method and system based on visual detection

Abstract

The present invention discloses a railway turnout opening determination method and system based on visual detection, the method comprising: continuously collecting front rail images, performing feature extraction and trapezoidal transformation on the images, and performing turnout positioning by detecting turnout switches; searching the turnout area in the image after trapezoidal transformation according to the turnout positioning, and extracting a binary image of the turnout area track; respectively extracting the track lines on the left and right sides of the turnout in a row-by-row search manner in the binary image, calculating the distance between the two track lines on the left / right side, and fitting the distance change trend between the two track lines on the left / right side of the turnout; judging the opening and closing conditions of the two track lines on the left / right side according to the distance change trend between the two track lines on the left / right side; judging the turnout opening direction and the main track of the vehicle according to the opening and closing conditions of the track lines on the left and right sides of the turnout. The present invention realizes the judgment of the driving direction at the turnout based on visual detection technology, and improves the ability of vehicle assisted driving control.

Description

Technical Field

[0001] The present invention belongs to the technical field of railway visual detection, and in particular relates to a railway turnout opening determination method and system based on visual detection. Background Art

[0002] When a railway vehicle is driving, it is necessary to accurately identify the track conditions ahead to ensure driving safety. Since railway tracks generally cover a wide area and the tracks are intertwined, accurately judging the direction of the railway turnout opening is a basic requirement for safe railway driving.

[0003] Lane recognition based on machine vision has become an important development direction for assisted driving, but the application of machine vision technology in identifying railway track lines is still relatively small. At present, it is mainly focused on railway track line recognition for line detection and early warning, while there is little research on judging the opening direction of railway turnouts based on machine vision technology. The invention patent with publication number CN112004732A discloses a railway track recognition device and a branch track detection unit for detecting whether there is a branch from the detected railway track. However, it can only detect whether there is a branch track and the position of the branch track based on the image, but cannot accurately judge the opening direction of the turnout. Therefore, in order to further provide more useful information for the operation of railway vehicles, it is necessary to propose a railway turnout opening direction judgment method based on machine vision. Summary of the invention

[0004] In view of this, the present invention proposes a railway turnout opening determination method and system based on visual detection, which is used to solve the problem that the existing visual detection technology cannot effectively identify the opening direction of the railway turnout.

[0005] In a first aspect, the present invention discloses a railway turnout opening determination method based on visual detection, the method comprising:

[0006] Continuously collect the images of the rails ahead, perform feature extraction and trapezoidal transformation on the images, and locate the turnout by detecting the turnout switch;

[0007] According to the position information obtained by turnout positioning, the turnout area is searched in the image after trapezoidal transformation, and the binary image of the track in the turnout area is extracted;

[0008] In the binary image, the track lines on the left and right sides of the turnout are extracted in a row-by-row search manner, the distance between the two track lines on the left / right side is calculated, and the distance change trend between the two track lines on the left / right side of the turnout is fitted;

[0009] Determine the opening and closing status of the two track lines on the left / right side of the turnout according to the distance change trend between the two track lines on the left / right side of the turnout;

[0010] The opening direction of the turnout and the main track of the vehicle are determined based on the opening and closing conditions of the track lines on the left and right sides of the turnout.

[0011] On the basis of the above technical solution, preferably, the feature extraction and trapezoidal transformation of the image specifically includes:

[0012] Extract the edge features of the image and obtain the horizontal high-frequency components of the image;

[0013] The image with horizontal high-frequency components is transformed into a trapezoidal shape. The transformation formula is:

[0014]

[0015] factor(i) w =len w / (stp(i,j)-sta(i,j))

[0016] Among them, Img_out i,j is the output result of the trapezoidal transformation, i=1,2,…,h,j=1,2,…,w,h and w are the height and width of the image respectively, imresize(·) is a scaling function used to scale the image after bilinear interpolation, iimg[i,sta(i,j):stp(i,j)] represents the pixel points starting from sta(i,j) and ending at stp(i,j) in the i-th row of the image, and the calculation formulas of sta(i,j) and stp(i,j) are:

[0017] sta(i,j)=floor(k0*(i-search_row_num)+1+j)

[0018] stp(i,j)=floor(k1*(i-search_row_num)+right_ipm-left_ipm+1+j)

[0019] floor(·) is the rounding function, k0 and k1 represent the slopes of the two legs of the trapezoid, search_row_num is the height of the trapezoid in pixels, right_ipm and left_ipm represent the horizontal coordinates of the pixels at the lower right and lower left corners of the trapezoid, respectively, factor(i) w Indicates the image width scaling factor, len w Indicates the width of the image after scaling.

[0020] On the basis of the above technical solution, preferably, the step of positioning the turnout by detecting the turnout switch specifically includes:

[0021] The two turnout switch images are respectively searched and matched in the image after the trapezoidal transformation, and the correlation value Cor1 between the image and the first turnout switch kernel1 is obtained respectively. i,j , and the correlation value Cor2 with the second turnout switch kernel2 i,j :

[0022]

[0023]

[0024] Wherein, i, j represent the i-th row and j-th column of the image pixel, i=1,2,…,h,j=1,2,…,w,h and w are the height and width of the image respectively, imresize(·) represents the scaling function, which is used to scale the turnout switch image after bilinear interpolation, size_init(i,j) is the scaling factor of the width and height of the image img, which is inversely proportional to the distance between the position of the pixel point (i,j) and the actual ground position; img[i,j] is the window area in the image img after trapezoidal transformation corresponding to the scaled turnout switch image, ∑ i,j It means to sum the operation results in the window area with (i, j) as the upper left corner;

[0025] Factor h 、Factor w is the coefficient correction factor:

[0026]

[0027]

[0028] Among them, θ init and θ actual The initial rotation angle and actual rotation angle are obtained by calibrating the camera respectively. Calibrate the camera to obtain the initial pitch angle and actual pitch angle respectively;

[0029] Find the area with the maximum value of the correlation value as the turnout positioning result Index(i,j):

[0030] Index(i,j)=max(Cor1 i,j , Cor2 i,j )

[0031] Among them, max(·) is the maximum value function.

[0032] Based on the above technical solution, preferably, if Index(i,j) is from Cor1 i,jThe search area of ​​the turnout is determined along the track direction on the right side of Index(i,j);

[0033] If Index(i,j) from Cor2 i,j The search area of ​​the turnout is determined along the track direction on the left side of Index(i,j);

[0034] The search area of ​​the turnout includes the search area on the left side of the turnout and the search area on the right side of the turnout, which correspond to the track lines on both sides of the turnout respectively.

[0035] On the basis of the above technical solution, preferably, extracting the track lines on the left and right sides of the turnout in a row-by-row search manner in the binary image, and calculating the distance between the two track lines on the left / right side specifically includes:

[0036] In the search area on the left side of the turnout, search from bottom to top in a row-by-row manner to extract two track lines in the search area on the left side of the turnout;

[0037] In the search area on the right side of the turnout, search from bottom to top in a row-by-row manner to extract two track lines in the search area on the right side of the turnout;

[0038] Calculate the distance between the two track lines on the left side of the turnout and the distance between the two track lines on the right side of the turnout. The distance calculation formula between the two track lines is:

[0039]

[0040] Where dist is a vector of distances, k = 1, 2, ..., N, N is the number of rows of pixels occupied by a single track line, lane left(k) 、lane right(k) are the pixel positions of the kth pixel in the two leftmost and rightmost track lines respectively.

[0041] On the basis of the above technical solution, preferably, judging the opening and closing status of the left / right track line according to the distance change trend between the left / right track lines specifically includes:

[0042] If the distance change trend between the two track lines on the left / right side is a trend that increases from zero, the two track lines on the left / right side are judged to be closed; when the distance change trend between the two track lines on the left / right side satisfies the condition that it decreases first and then increases, the two track lines on the left / right side are judged to be open.

[0043] On the basis of the above technical solution, preferably, the step of determining the turnout opening direction and the main track of the vehicle according to the opening and closing conditions of the track lines on the left and right sides of the turnout specifically includes:

[0044] If the two track lines on the left are closed and the two track lines on the right are open, the turnout opening direction is to the right;

[0045] If the two track lines on the right are closed and the two track lines on the left are open, the turnout opening direction is to the left;

[0046] The main track is determined based on the two track lines on the opening side, where the track section extending from the opening side to the bottom of the image corresponds to the main track of the vehicle. Based on the principle that the track spacing is fixed, the main track of the vehicle on the corresponding side is determined.

[0047] In a second aspect of the present invention, a railway turnout opening determination system based on visual detection is disclosed, the system comprising:

[0048] Turnout positioning module: used to continuously collect the front rail image, perform feature extraction and trapezoidal transformation on the image, and locate the turnout by detecting the turnout switch;

[0049] Turnout search module: used to search the turnout area in the image after trapezoidal transformation according to the position information obtained by turnout positioning, and extract the binary image of the track in the turnout area; in the binary image, the track lines on the left and right sides of the turnout are extracted respectively by searching row by row;

[0050] Turnout judgment module: used to calculate the distance between the two track lines on the left / right side, fit the distance change trend between the two track lines on the left / right side of the turnout; judge the opening and closing status of the two track lines on the left / right side according to the distance change trend between the two track lines on the left / right side; judge the turnout opening direction and the main track of the vehicle according to the opening and closing status of the track lines on the left and right sides of the turnout.

[0051] In a third aspect of the present invention, an electronic device is disclosed, comprising: at least one processor, at least one memory, a communication interface and a bus;

[0052] Wherein, the processor, memory, and communication interface communicate with each other via the bus;

[0053] The memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to implement the method according to the first aspect of the present invention.

[0054] According to a fourth aspect of the present invention, a computer-readable storage medium is disclosed, wherein the computer-readable storage medium stores computer instructions, and the computer instructions enable a computer to implement the method described in the first aspect of the present invention.

[0055] Compared with the prior art, the present invention has the following beneficial effects:

[0056] 1) The present invention searches for the turnout switch based on the visual detection technology to locate the turnout, searches the turnout area according to the turnout location, and extracts the track lines on the left and right sides of the turnout respectively, and judges the opening and closing conditions of the two track lines on the left / right sides according to the distance change trend between the two track lines on the left / right sides, thereby judging the turnout opening direction and the main track of the vehicle. The turnout opening direction and the vehicle track can be judged based on the visual detection technology, and the driving direction can be quickly judged without the help of manual or complex algorithms, thereby improving the vehicle assisted driving control capability;

[0057] 2) The present invention searches and matches the image after the trapezoidal transformation with the two turnout switches respectively to obtain the relevant area of ​​the turnout switch, and finds the area with a larger correlation in the relevant area as the position of the turnout, so that the turnout area can be accurately located from the image, avoiding the turnout positioning error caused by the complex and staggered track lines;

[0058] 3) The present invention extracts the track lines on the left and right sides respectively according to the position of the turnout, and can distinguish the opening and closing conditions of the track lines on the left and right sides of the turnout according to whether the distance change trend between the two track lines is increasing continuously or first decreasing and then increasing, thereby improving the accuracy and convenience of turnout direction judgment, thereby quickly and accurately identifying the main track of the vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0060] Figure 1 The present invention is a flow chart of a railway turnout opening determination method based on visual detection;

[0061] Figure 2 The image after edge feature extraction;

[0062] Figure 3 is the image before trapezoidal transformation;

[0063] Figure 4 is the image after trapezoidal transformation;

[0064] Figure 5 is the image of kernel1;

[0065] Figure 6 is the image of kernel2;

[0066] Figure 7 It is a schematic diagram of the search area on the left side and the search area on the right side of the turnout;

[0067] Figure 8 This is a schematic diagram of the binarization effect of the two tracks on the left side of the turnout;

[0068] Fig. 9 It is a schematic diagram of line-by-line search and corresponding search results;

[0069] Fig.10 Schematic diagram for calculating the distance between the left and right lines;

[0070] Fig.11 It is a schematic diagram of the distance change trend between the two tracks in the open state;

[0071] Fig.12 It is a schematic diagram of the distance change trend between the two tracks in the closed state;

[0072] Fig.13 The schematic diagram of the principle of judging the opening direction of the turnout;

[0073] Fig.14 This is a schematic diagram of the principle of determining the main track based on the open side turnout;

[0074] Fig.15 for Figure 2 Schematic diagram of the vehicle owner's track determination results in the corresponding turnout. DETAILED DESCRIPTION

[0075] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0076] See also Figure 1 The present invention proposes a railway turnout opening determination method based on visual detection, the method comprising:

[0077] S1. Continuously collect the front rail image, extract features and perform trapezoidal transformation on the image, and locate the turnout by detecting the turnout switch.

[0078] In the process of the vehicle running along the track, the present invention uses a camera to obtain continuous image information in front, takes the train travel direction as the positive direction, and performs turnout positioning based on the acquired images. Since each turnout must have a turnout switch on the left or right side of the rail, and the vertical distance between the turnout switch and the rail, and the distance between the two rails are fixed and known, the turnout area positioning can be achieved by detecting the turnout switch.

[0079] Step S1 specifically includes the following sub-steps:

[0080] S11. After collecting the image, extract the edge features of the image.

[0081] like Figure 2 The figure shows the result of feature extraction of a captured image, and the lateral high-frequency component of the image is obtained.

[0082] S12, performing a trapezoidal transformation on the image with horizontal high-frequency components.

[0083] because Figure 2 The rail image in the image presents a perspective effect where objects near are larger than objects far away, which is not conducive to the location and identification of turnouts. Therefore, the image perspective effect is eliminated through trapezoidal transformation.

[0084] The trapezoidal transformation formula is:

[0085]

[0086] factor(i) w =len w / (stp(i,j)-sta(i,j))

[0087] Among them, Img_out i,j is the output result of the trapezoidal transformation, i, j represent the i-th row and j-th column of the image pixels, i=1,2,…,h,j=1,2,…,w, h and w are the height and width of the image respectively, imresize(·) is a scaling function used to scale the image after bilinear interpolation, img[i,sta(i,j):stp(i,j)] represents the pixels in the i-th row of the image starting from sta(i,j) and ending at stp(i,j), and the calculation formulas of sta(i,j) and stp(i,j) are:

[0088] sta(i,j)=floor(k0*(i-search_row_num)+1+j)

[0089] stp(i,j)=floor(k1*(i-search_row_num)+right_ipm-left_ipm+1+j)

[0090] Among them, floor(·) is the rounding function, k0 and k1 represent the slopes of the lines where the two legs of the trapezoid are located, search_row_num is the pixel height of the trapezoid, right_ipm and left_ipm represent the horizontal coordinates of the pixel points at the lower right corner and lower left corner of the trapezoid respectively. factor(i) w Indicates the image width scaling factor, len w Indicates the width of the image after scaling. [1,factor(i)w ] represent the height scaling factor and width scaling factor respectively, imresize(img[i,sta(i,j):stp(ij)],[1,factor(i) w ]) means that the pixel points in the i-th row of the image img starting from sta(i,j) and ending at stp(i,j) are scaled by a factor of 1 in height and factor(i) in width. w The scaling transformation.

[0091] Figure 3 , Figure 4 The results before and after the trapezoidal transformation are shown in Figure 2. Figure 4 The distortion has been eliminated and a regular orbital image has been obtained.

[0092] S13, detecting the turnout switch in the image after the trapezoidal transformation.

[0093] Get two turnout switch images with different orientations, such as Figure 5 The first turnout switch kernel1 is shown. Figure 6 The second turnout switch kernel2 is shown. The two turnout switches have different orientations. The orientation of the first turnout switch kernel1 indicates that the turnout is on its right side, and the orientation of the second turnout switch kernel2 indicates that the turnout is on its left side. Therefore, if the corresponding turnout switch is detected, the area where the turnout is located can be located.

[0094] The two turnout switch images are respectively searched and matched in the image img after the trapezoidal transformation in a sliding window manner, that is, the image of the first turnout switch kernel1 and the image of the second turnout switch kernel2 are used as templates to perform template matching with the image img after the trapezoidal transformation. Each time a window position is slid, a correlation value Cor1 between the image img and the first turnout switch kernel1 is obtained. i,j Or the correlation value Cor2 between the image img and the second switch kernel2 i,j :

[0095]

[0096]

[0097] Where i and j represent the i-th row and j-th column of the image pixels, i = 1, 2, …, h, j = 1, 2, …, w, h and w are the height and width of the image img after the trapezoidal transformation, respectively. imresize(·) represents a scaling function, which is used to scale the image of the turnout switch kernel1 or kernel2 after bilinear interpolation. size_init(i, j) is the initialized height scaling factor and width scaling factor. The size of size_init(i, j) is inversely proportional to the distance between the position of the pixel point (i, j) and the actual ground position. The purpose of scaling the image of kernel1 or kernel2 is to eliminate the distortion of the image of kernel1 or kernel2 so that it accurately matches the turnout switch in the image after the trapezoidal transformation. The area covered by the scaled image of kernel1 or kernel2 on the trapezoidal transformed image img is defined as the window area. img[i, j] is the window area in img corresponding to the scaled turnout switch image. (i, j) can be set to the upper left corner of the window area. ∑ i,j Indicates the sum of the multiplication results of the window area image and the image of kernel1 or kernel2, Cor1 i,j 、Cor2 i,j It is equivalent to calculating image correlation, which is essentially the sum of the multiplications of corresponding pixel values ​​between the template image and the window area image.

[0098] Factor h 、Factor w is the coefficient correction factor:

[0099]

[0100]

[0101] Among them, θ init and θ actual The initial rotation angle and actual rotation angle are obtained by calibrating the camera respectively. Calibrate the camera to obtain the initial pitch angle and actual pitch angle respectively;

[0102] imresize(kernel1,size_init(i,j).*[Factor h ,Factor w ])) means scaling the image of kernel1, and the height scaling factor of the scaling transformation is the height scaling factor initialized in size_init(i,j) multiplied by Factor h, the width scaling factor is the width scaling factor initialized in size_init(i,j) multiplied by Factor w . imresize(kernel2,size_init(i,j).*[Factor h ,Factor w ])))'s scaling transformation is the same.

[0103] From Cor1 i,j 、Cor2 i,j Find the area with the maximum value of the correlation value as the area where the turnout switch is located, and take the area where the turnout switch is located as the turnout positioning result Index(i,j):

[0104] Index(i,j)=max(Cor1 i,j , Cor2 i,j )

[0105] Among them, max(·) is the maximum value function.

[0106] The present invention searches and matches the image after trapezoidal transformation with two turnout switches respectively, obtains the relevant areas of the turnout switches respectively, and finds the area with larger correlation in the relevant areas as the position of the turnout. The turnout area can be accurately located from the image, avoiding the turnout positioning error caused by the complex and intertwined track lines.

[0107] S2. Searching for a turnout area in the image after the trapezoidal transformation according to the position information obtained by turnout positioning, and extracting a binary image of the track in the turnout area.

[0108] Step S2 specifically includes the following sub-steps:

[0109] S21, search for turnout area

[0110] If Index(i,j) from Cor1 i,j The search area of ​​the turnout is determined along the track direction on the right side of Index(i,j).

[0111] Specifically, take the position of Index(i,j) as the origin, take the position of Index(i,j) as the origin, the positive direction of the longitudinal axis y1 forward along the direction of vehicle travel, and the positive direction of the horizontal axis x1 to the right perpendicular to the track to establish the second local coordinate system o1-x1y1. On the left side of the position of Index(i,j), the range of -3m to 6m along the y1 axis is used as the search area for the turnout. In addition, the search range along the x1 axis can be determined according to the vertical distance between the turnout switch and the rail and the width of the rail, so as to further narrow the search area for the turnout. That is, the lower boundary is 3m in the opposite direction of the vehicle's travel direction, and the upper boundary is 6m in front of the vehicle's travel direction; on the right side of the position of Index(i,j), the left and right boundaries are determined according to the vertical distance between the turnout switch and the rail and the width of the rail, and the area surrounded by the lower boundary, upper boundary, left boundary, and the boundary is used as the search area for the turnout;

[0112] If Index(i,j) from Cor2 i,j It is obtained that the search area of ​​the turnout is determined along the track direction on the left side of Index(i,j).

[0113] Specifically, the second local coordinate system o2-x2y2 is established with the position of Index(i,j) as the origin, the positive direction of the longitudinal axis y2 along the vehicle's travel direction, and the horizontal axis x2 perpendicular to the track to the right. On the left side of the position of Index(i,j), the range of -3m to 6m along the y2 axis is used as the search area for the turnout. In addition, the search range along the x2 axis can be determined according to the vertical distance between the turnout switch and the rail and the rail width, further narrowing the search area for the turnout.

[0114] The search area of ​​the turnout can be divided into the search area on the left side of the turnout and the search area on the right side of the turnout, which correspond to the track lines on both sides of the turnout respectively. Figure 7 The two black boxes in the figure show the search areas on the left and right sides of the turnout respectively.

[0115] S22, binary extraction.

[0116] The binary image of the orbit is obtained by using the following binarization formula:

[0117]

[0118] src(i,j) and dst(i,j) are the pixel values ​​of the pixel at (i,j) before and after the binary conversion, and thresh is the set pixel threshold.

[0119] Taking the left side of the turnout as an example, the binary extraction results are as follows Figure 8 The binary image shown.

[0120] S3. In the binary image, the track lines on the left and right sides of the turnout are extracted respectively by searching row by row, the distance between the two track lines on the left / right side is calculated, and the distance change trend between the two track lines on the left / right side of the turnout is fitted.

[0121] like Fig. 9 The figure shows a schematic diagram of line-by-line search and corresponding search results. In the search area on the left side of the turnout, the search is performed from bottom to top in a line-by-line manner, the center lines of the left and right lines are calculated, and the two track lines in the search area on the left side of the turnout are extracted.

[0122] In the search area on the right side of the turnout, search from bottom to top in a row-by-row manner, calculate the center lines of the left and right lines, and extract the two track lines in the search area on the right side of the turnout.

[0123] Calculate the distance between the two track lines on the left side of the turnout and the distance between the two track lines on the right side of the turnout, such as Fig.10 The figure shows a schematic diagram of the distance calculation between two track lines. The distance coordinate calculation formula between the two track lines is:

[0124]

[0125] Where dist is the coordinate vector composed of distances, k = 1, 2, ..., N, N is the number of rows of pixels occupied by a single track line, lane left(k) 、lane right(k) are the pixel positions of the kth pixel in the two leftmost and rightmost track lines respectively.

[0126] The dist coordinates are fitted to obtain the distance change trend graph between the two track lines on the left side of the turnout and the distance change trend graph between the two track lines on the right side.

[0127] S4. Determine the opening and closing conditions of the two track lines on the left / right side according to the distance change trend between the two track lines on the left / right side.

[0128] If the distance between the two track lines on the left / right side changes from zero to larger, the two track lines on the left / right side are considered closed; if the distance between the two track lines on the left / right side changes from zero to larger, the two track lines on the left / right side are considered open. Fig.11 , 12 The distance change trend diagram between the two track lines is shown in Figure 2. Fig.11 The distance between the two track lines changes from zero to larger, so the two track lines are in a closed state. Fig.12 The distance change trend between the two track lines satisfies the condition of first decreasing and then increasing, so these two track lines are in an open state.

[0129] The present invention determines the opening and closing status of a track line according to the distance variation trend between two track lines, provides a reliable basis for judging the direction of a turnout, improves the accuracy and convenience of judging the direction of a turnout, and thus quickly and accurately identifies the main track of the vehicle.

[0130] S5. Determine the direction of the turnout opening and the main track of the vehicle according to the opening and closing conditions of the track lines on the left and right sides of the turnout.

[0131] like Fig.13 The figure shows the schematic diagram of the principle of judging the direction of turnout opening. If the two track lines on the left are closed and the two track lines on the right are open, the direction of turnout opening is to the right; if the two track lines on the right are closed and the two track lines on the left are open, the direction of turnout opening is to the left.

[0132] Determine the main track based on the two track lines on the opening side, such as Fig.14 As shown in the figure, the track extending from the opening side to the bottom of the image corresponds to the main track of the vehicle (i.e. Fig.14 Then, based on the principle that the track spacing is fixed, determine the main track of the vehicle on the corresponding side. Fig.15 Shown Figure 2 The corresponding main track identification result of the vehicle shows that the method of the present invention can accurately identify the direction of the turnout opening and determine the main track of the vehicle.

[0133] The present invention searches for the turnout switch based on the visual detection technology to locate the turnout, searches the turnout area according to the turnout positioning, and respectively extracts the distance change trend between the two track lines on the left / right side of the turnout to judge the opening and closing conditions of the two track lines on the left / right side, thereby judging the turnout opening direction and the main track of the vehicle, and can realize rapid judgment of the driving direction without the help of manual or complex algorithms, thereby improving the vehicle's assisted driving control capability.

[0134] Corresponding to the above method embodiment, the present invention also proposes a railway turnout opening determination system based on visual detection, the system comprising:

[0135] Turnout positioning module: used to continuously collect the front rail image, perform feature extraction and trapezoidal transformation on the image, and locate the turnout by detecting the turnout switch;

[0136] Turnout search module: used to search the turnout area in the image after trapezoidal transformation according to the position information obtained by turnout positioning, and extract the binary image of the track in the turnout area; in the binary image, the track lines on the left and right sides of the turnout are extracted respectively by searching row by row;

[0137] Turnout judgment module: used to calculate the distance between the two track lines on the left / right side of the turnout, fit the distance change trend between the two track lines on the left / right side of the turnout; judge the opening and closing status of the two track lines on the left / right side according to the distance change trend between the two track lines on the left / right side; judge the turnout opening direction and the main track of the vehicle according to the opening and closing status of the track lines on the left and right sides of the turnout.

[0138] The above system embodiments and method embodiments correspond one to one, and for a brief description of the system embodiments, please refer to the method embodiments.

[0139] The present invention also discloses an electronic device, comprising: at least one processor, at least one memory, a communication interface and a bus; wherein the processor, memory, and communication interface communicate with each other through the bus; the memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to implement the aforementioned method of the present invention.

[0140] The present invention also discloses a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, wherein the computer instructions enable the computer to implement all or part of the steps of the method described in the embodiment of the present invention. The storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk, and other media that can store program codes.

[0141] The system embodiment described above is merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be distributed to multiple network units. A person skilled in the art may select some or all of the modules according to actual needs to achieve the purpose of the solution of this embodiment without creative effort.

[0142] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A railway turnout opening determination method based on visual detection, characterized in that: The method comprises: Continuously collect the images of the rails ahead, perform feature extraction and trapezoidal transformation on the images, and locate the turnout by detecting the turnout switch; According to the position information obtained by turnout positioning, the turnout area is searched in the image after trapezoidal transformation, and the binary image of the track in the turnout area is extracted; In the binary image, the track lines on the left and right sides of the turnout are extracted in a row-by-row search manner, the distance between the two track lines on the left / right side is calculated, and the distance change trend between the two track lines on the left / right side of the turnout is fitted; Determine the opening and closing status of the two track lines on the left / right side according to the distance change trend between the two track lines on the left / right side; Determine the turnout opening direction and the main track of the vehicle according to the opening and closing conditions of the track lines on the left and right sides of the turnout; The method of positioning the turnout by detecting the turnout switch specifically includes: Get two turnout switch images with different orientations, perform sliding search and matching on the two turnout switch images in the image after trapezoidal transformation, and obtain the correlation value Cor1 between the image and the first turnout switch kernel1 respectively. i，j , and the correlation value Cor2 with the second turnout switch kernel2 i,j : Wherein, imresize(·) represents a scaling function, which is used to scale the turnout switch image after bilinear interpolation. size_init(i, j) is the scaling factor of the width and height of the image img, which is inversely proportional to the distance between the position of the pixel point (i, j) and the actual ground position. img[i, j] is the window area in the image img after trapezoidal transformation that corresponds to the scaled turnout switch image. i，j Indicates the sum of the operation results in the window area with (i, j) as the upper left corner; Factor h 、Factor w is the coefficient correction factor: Among them, θ init and θ actual The initial rotation angle and actual rotation angle are obtained by calibrating the camera respectively. Calibrate the camera to obtain the initial pitch angle and actual pitch angle respectively; Find the area with the maximum value of the correlation value as the turnout positioning result Index(i, j): Index(i,j)=max(Cor1 i，j ,Cor2 i,j ) Among them, max(·) is the maximum value function.

2. The railway turnout opening determination method based on visual detection according to claim 1 is characterized in that: The feature extraction and trapezoidal transformation of the image specifically include: Extract the edge features of the image and obtain the horizontal high-frequency components of the image; The image with horizontal high-frequency components is transformed into a trapezoidal shape. The transformation formula is: factor(i) w =len w / (stp(i,j)-sta(i,j)) Among them, Img_out i，j is the output result of the trapezoidal transformation, i, j represent the i-th row and j-th column of the image pixels, i = 1, 2, ..., h, j = 1, 2, ..., w, h and w are the height and width of the image respectively, imresize(·) is a scaling function used to scale the image after bilinear interpolation, img[i, sta(i, j): stp(i, j)] represents the pixels in the i-th row of the image starting from sta(i, j) and ending at stp(i, j), and the calculation formulas of sta(i, j) and stp(i, j) are: sta(i,j)=floor(k0*(i-search_row_num)+1+j) stp(i,j)=floor(k1*(i-search_row_num)+right_ipm-left_ipm+1+j) Among them, floor(·) is the rounding function, k0 and k1 represent the slopes of the lines where the two legs of the trapezoid are located, search_row_num is the height of the trapezoid in pixels, right_ipm and left_ipm represent the horizontal coordinates of the pixel points in the lower right corner and lower left corner of the trapezoid, respectively, and factor(i) w Indicates the image width scaling factor, len w Indicates the width of the image after scaling.

3. The railway turnout opening determination method based on visual detection according to claim 1 is characterized in that: The step of searching for a turnout area in the image after the trapezoidal transformation according to the position information obtained by turnout positioning specifically includes: If Index(i, j) from Cor1 i，j The search area of ​​the turnout is determined along the track direction on the right side of Index(i, j); If Index(i, j) from Cor2 i,j The search area of ​​the turnout is determined along the track direction on the left side of Index(i, j); The search area of ​​the turnout includes a search area on the left side of the turnout and a search area on the right side of the turnout, which correspond to the track lines on both sides of the turnout respectively.

4. The railway turnout opening determination method based on visual detection according to claim 3 is characterized in that: The step of extracting the track lines on the left and right sides of the turnout in a row-by-row search manner in the binary image and calculating the distance between the two track lines on the left / right side specifically includes: In the search area on the left side of the turnout, search from bottom to top in a row-by-row manner to extract two track lines in the search area on the left side of the turnout; In the search area on the right side of the turnout, search from bottom to top in a row-by-row manner to extract two track lines in the search area on the right side of the turnout; Calculate the distance between the two track lines on the left side of the turnout and the distance between the two track lines on the right side of the turnout. The distance calculation formula between the two track lines is: Where dist is a vector of distances, k = 1, 2, ..., N, N is the number of rows of pixels occupied by a single track line, lane left(k) 、lane right(k) are the pixel positions of the kth pixel in the two leftmost and rightmost track lines respectively.

5. The railway turnout opening determination method based on visual detection according to claim 1, characterized in that: The method of judging the opening and closing of the left / right track line according to the distance change trend between the left / right track lines specifically includes: If the distance change trend between the two track lines on the left / right side is a trend that increases from zero, the two track lines on the left / right side are judged to be closed; if the distance change trend between the two track lines on the left / right side satisfies the trend of first decreasing and then increasing, the two track lines on the left / right side are judged to be open.

6. The railway turnout opening determination method based on visual detection according to claim 5 is characterized in that: The method of judging the turnout opening direction and the main track of the vehicle according to the opening and closing conditions of the track lines on the left and right sides of the turnout specifically includes: If the two track lines on the left are closed and the two track lines on the right are open, the turnout opening direction is to the right; If the two track lines on the right are closed and the two track lines on the left are open, the turnout opening direction is to the left; The main track is determined based on the two track lines on the opening side, where the track section extending from the opening side to the bottom of the image corresponds to the main track of the vehicle. Based on the principle that the track spacing is fixed, the main track of the vehicle on the corresponding side is determined.

7. A railway turnout opening determination system based on visual detection, which uses the railway turnout opening determination method based on visual detection according to any one of claims 1 to 6, characterized in that: The system comprises: Turnout positioning module: used to continuously collect the front rail image, perform feature extraction and trapezoidal transformation on the image, and locate the turnout by detecting the turnout switch; Turnout search module: used to search the turnout area in the image after trapezoidal transformation according to the position information obtained by turnout positioning, and extract the binary image of the track in the turnout area; in the binary image, the track lines on the left and right sides of the turnout are extracted respectively by searching row by row; Turnout judgment module: used to calculate the distance between the two track lines on the left / right side of the turnout, fit the distance change trend between the two track lines on the left / right side of the turnout; judge the opening and closing status of the two track lines on the left / right side according to the distance change trend between the two track lines on the left / right side; judge the turnout opening direction and the main track of the vehicle according to the opening and closing status of the track lines on the left and right sides of the turnout.

8. An electronic device, characterized in that: include: at least one processor, at least one memory, a communication interface, and a bus; Wherein, the processor, memory, and communication interface communicate with each other via the bus; The memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions enable a computer to implement the method according to any one of claims 1 to 6.

Images