A track area detection and extraction method based on video stream

Abstract

The application relates to a track area detection and extraction method based on a video stream, which comprises the following steps: preprocessing a collected train operation video stream, converting the train operation video stream into a binary image sequence about scene features, and calibrating initial parameters; constructing a state updating function, setting a global variable state parameter, and initializing; searching for a candidate coordinate set based on a sliding window search algorithm or a band-pass search algorithm; fitting a quadratic track curve based on a least square method, storing key parameters into a quadratic curve parameter sequence after the key parameters are obtained; judging the validity of the key parameters, and updating the global variable state parameter; performing filtering and smoothing treatment on the quadratic curve parameter sequence, extracting curve coordinates from the quadratic curve parameter sequence, and performing perspective inverse transformation, so that a track area is drawn in an original video frame sequence. Compared with the prior art, the application can accurately perceive and extract a track area in a complex operation environment, and is thus favorable for improving the speed and accuracy of track area foreign object intrusion detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of rail transit technology, in particular to a rail area detection and extraction method based on video stream. BACKGROUND

[0002] In a rail transit train operation safety monitoring and early warning system, the rail area covered by the two tracks and the area therebetween is the key target area of interest for foreign object intrusion limit detection, and accurately and quickly detecting intruding foreign objects in the rail area is the key to ensuring the safe operation of the train.

[0003] The current mainstream rail foreign object intrusion detection is to use cameras, radars and other sensors to realize non-contact sensing of the train operation environment, and then realize detection of rail foreign object intrusion, which is easily affected by complex environments during train operation, such as bad weather such as rain / snow / fog, extremely dark light environment in tunnels, trackside signal machines, cable hangers and other facilities and equipment, sharp bends, etc. These complex train operation environments can easily affect the reliability of rail transit train operation safety monitoring, and since the rail area cannot be accurately extracted, the speed and accuracy of subsequent rail area foreign object intrusion detection are low, the false alarm rate and the missed alarm rate of the early warning system are high, which is not conducive to train safety. SUMMARY

[0004] The purpose of the present application is to overcome the defects of the prior art and provide a rail area detection and extraction method based on video stream, which can accurately perceive and extract the rail area in complex operating environments, thereby improving the speed and accuracy of rail area foreign object intrusion detection.

[0005] The purpose of the present application can be achieved by the following technical solution: a rail area detection and extraction method based on video stream, comprising the following steps:

[0006] S1, collecting train operation video stream data and pre-processing the video stream to convert the video stream into a binary image sequence about scene features;

[0007] S2, calibrating initialization parameters according to the binary image sequence;

[0008] S3, constructing a state update function, setting a global variable state parameter and initializing;

[0009] S4, searching for a candidate coordinate set based on a sliding window search algorithm or a passband search algorithm according to the calibrated initialization parameters and the global variable state parameter;

[0010] S5, fitting a quadratic rail curve x=a0+a1y+a2y2 based on the least square method and the candidate coordinate set 2After obtaining the key parameters, they are stored in the quadratic curve parameter sequence, wherein the key parameters are α={a0,a1,a2};

[0011] S6. Based on the constructed state update function, determine the validity of key parameters and update the global variable state parameters;

[0012] S7. Perform filtering and smoothing on the quadratic curve parameter sequence;

[0013] S8. Extract curve coordinates from the filtered and smoothed quadratic curve parameter sequence, perform inverse perspective transformation, and draw the track region in the original video frame sequence.

[0014] Further, in step S1, the preprocessing of the video stream specifically involves decomposing the acquired video stream data into a time-ordered video frame sequence F, and performing image preprocessing on each image in the video frame sequence F, including grayscale conversion, scene segmentation, parameter adaptation, perspective transformation, region relocation, and edge detection. After image preprocessing, the video frame sequence is transformed into a binary image sequence B based on scene features. The video stream data is the forward-facing video data of the vehicle's front recorded by the vehicle-mounted camera during operation, and the video frame sequence F = {f1,…,f...} k-1 ,f k ,…,f n}, where f1 is the image corresponding to the starting frame of the video, f k f is the image corresponding to the k-th frame of the video. n This is the image corresponding to the end frame of the video.

[0015] Furthermore, the image preprocessing in step S1 specifically includes the following steps:

[0016] S11. Grayscale Conversion: The video frame sequence F is converted to grayscale to obtain a grayscale image sequence G = {g1,…,g...} k-1 ,g k ,…,g n}, where g1 is the grayscale image of the starting frame of the video, g k Let g be the grayscale image of the k-th frame of the video. n This is a grayscale image of the last frame of the video.

[0017] S12. Scene Division: Calculate the scene features of each image in the grayscale image sequence G to obtain the time series C = {c1,…,c...} of the scene features. k-1 ,c k ,…,c n The calculation formula is as follows:

[0018]

[0019]

[0020] wherein, gh k is the number of pixel points in the kth gray image whose gray value is greater than or equal to the threshold value 0, gl k is the number of pixel points in the kth gray image whose gray value is less than the threshold value d0, c k is the ratio of the two; l k is the scene category label of the kth gray image, which divides the train running scene into "bright light" and "dark light", when the scene is identified as "bright light", l k = 1, when the scene is identified as "dark light", l k = 0, and d1 is the judgment threshold value for dividing the two typical running scenes;

[0021] S13, parameter self-adaption: dividing the running scene, assigning the algorithm parameters separately for different running scenes, to obtain the scene parameter assignment sequence S = {s1, …, s k-1 , s k , …, s n}, s k = {d0, d1, …}, wherein s1 is the parameter assignment set of the gray image of the starting frame of the video, s k is the parameter assignment set of the kth gray image of the video, and s n is the parameter assignment set of the gray image of the ending frame of the video, and the parameters assigned for each scene are fixed;

[0022] S14, perspective transformation: performing perspective transformation on the gray image sequence, selecting an arbitrary quadrilateral region containing the track in the gray image sequence G, and marking the quadrilateral region Or = {(x1, y1), (x2, y2), (x3, y3), (x4, y4)} with the G image coordinate system, which corresponds to the four vertex coordinates of the upper left corner, the upper right corner, the lower right corner and the lower left corner of the quadrilateral in turn;

[0023] The target region after perspective transformation is a rectangular region Pr = {(x'1, y'1), (x'2, y'2), (x'3, y'3), (x'4, y'4)}, a perspective transformation transfer matrix M and an inverse transfer matrix M in of inverse perspective transformation are generated at the same time, and a perspective gray image sequence T = {t1, … t k-1 , t k , …, t n} after perspective transformation is obtained, wherein t1 is the perspective transformation image of the first gray image, t k is the perspective transformation image of the kth gray image, and t n is the perspective transformation image of the last gray image;

[0024] S15, region repositioning: for the perspective gray image sequence T, a mask function is constructed, a rectangular region of the perspective gray image sequence containing only the track is extracted, and the gray values of the pixels at the remaining positions of the image are set to 0, thereby obtaining a region repositioned perspective gray image sequence R={r1,…,rk,…,r k-1 k n} where r1 is the first frame of the region repositioned perspective gray image, rk is the kth frame of the region repositioned perspective gray image, and r is the last frame of the region repositioned perspective gray image. k n

[0025] S16, edge detection: based on the Sobel operator edge detection method, the Sobel operator in the X direction is used to perform convolution operation on the region repositioned perspective gray image sequence R, a 3*3 convolution kernel is selected, and the absolute value of the convolution result obtained for each pixel of the image is taken, i.e., the gray gradient at the position of the pixel is obtained. After obtaining the gray gradient at the position of each pixel, the pixel is subjected to binaryzation processing according to the parameter assignment result of the scene recognition, and finally an edge detection binary image sequence B={b1,…,bk,…,b k-1 k n} is obtained, where b1 is the first frame of the edge detection binary image of the perspective gray image, bk is the kth frame of the edge detection binary image of the perspective gray image, and b is the last frame of the edge detection binary image of the perspective gray image. k n

[0026] Further, the specific process of the binaryzation processing of the pixel in step S16 is as follows: a high threshold maxVal and a low threshold minVal are set, when the gray gradient at the position of the pixel is greater than maxVal, the gray value of the pixel is assigned as 255; when the gray gradient at the position of the pixel is less than minVal, the gray value of the pixel is assigned as 0; when the gray gradient at the position of the pixel is between maxVal and minVal, it is further judged whether there is a pixel with a gray value of 255 adjacent to the pixel, if yes, the gray value of the pixel is assigned as 255; if not, the gray value of the pixel is assigned as 0.

[0027] Further, step S2 specifically includes the following steps:

[0028] S21, the first image b1 in the binary image sequence B is extracted, and the center position coordinate O(LB X Y ) of the track at the lower edge of the image on one side is calibrated.​​​​​​​​​

[0029] S22, calibrate the track spacing R of the two-side tracks under the size condition of sequence B D .

[0030] Further, the step S3 specifically comprises the following steps:

[0031] S31, construct a state update function and a global variable dec, which is used to represent the binarized image track line fitting result, and is assigned as "1" when the fitting is successful, and is assigned as "0" when the fitting fails, and is initialized as "0";

[0032] S32, construct α k of the type of array container R l , and set the container length threshold as A l ;

[0033] S33, initialize the track line fitting result α k ={0,0,0}, and the average value m l of all α k of the type of data in the container R α ={0,0,0}.

[0034] Further, in the step S4, if the current global variable state parameter is "0", a sliding window search algorithm is used to search and generate a candidate coordinate set; if the current global variable state parameter is "1", a band-pass search algorithm is used to search and generate a candidate coordinate set.

[0035] Further, the specific process of the sliding window search algorithm is as follows:

[0036] A1, based on the calibrated image one-side track lower edge center position, determine the starting position of the rectangular sliding window, and the positioning method is: taking the coordinate O pixel point as the midpoint of the sliding window lower frame, and setting the size parameters of the sliding window: the width is w win , the height is h win , and the parameters are adjustable;

[0037] A2, search for white pixel points (gray value = 255) in the rectangular sliding window, and record the image coordinates of these white pixel points to the candidate coordinate set G, and simultaneously count the number N i of white pixel points in the window;

[0038] A3, set the white pixel point number judgment threshold N u , compare the size relationship between N i and N u , to determine the sliding window growth method to the image upper boundary;

[0039] When Ni ≥ N u , calculate the average of the horizontal coordinates of all white pixel points in the current sliding window as the horizontal coordinate of the middle point of the lower edge of the next sliding window; while when N i < N u , the horizontal coordinate of the lower edge of the current sliding window is still taken as the horizontal coordinate of the middle point of the lower edge of the next sliding window.

[0040] After the sliding window grows once from the lower boundary of the image upwards, the lower edge of the new sliding window grown is at the same vertical coordinate as the upper edge of the sliding window before growth.

[0041] A4, iterate, repeat steps A2 and A3 every time the sliding window grows once from the upper boundary of the image upwards, until the upper edge of the rectangular sliding window reaches the upper boundary of the image, and the iteration ends, that is, a complete candidate coordinate set G in the binary image is obtained.

[0042] Further, the specific process of the class passband search algorithm is as follows:

[0043] B1, input the previous frame binary image b k-1 ; k-1 , and determine the position of the fitted quadratic track curve x=a0+a1y+a2y 1， in the current frame binary image b k ; 2

[0044] B2, shift the fitted quadratic track curve of the previous frame in step B1 to the left and right sides of the image by distances d respectively, to form a passband with the fitted track curve of the previous frame as the central axis and the same width in the x-axis direction, and the width of the passband is d m .

[0045] B3, search for the white pixel points covered by the passband generated in step B2 in the binary image, and record the coordinates of these white pixel points in the candidate coordinate set G.

[0046] Further, the step S6 specifically comprises the following steps:

[0047] S61, input the current track line fitting result a k .

[0048] S62, determine whether all three key parameters in a k are 0, if yes, determine that the fitting fails, set dec to “0”, and then execute step S67, otherwise execute step S63;

[0049] S63, determine whether a k ​If the three key parameters do not meet the preset independent determination condition and preset joint determination condition, the fitting is determined to fail, dec is set to "0", and then step S67 is executed, otherwise step S64 is executed;

[0050] The independent determination condition is to set a threshold limit parameter absolute value range;

[0051] The joint determination condition is to set an image area limit fitting curve possible coverage range;

[0052] S64, calculate the container R l All alpha k The average value m of the type data α Determine alpha k If the three parameters and the calculated average value do not meet the separate threshold condition limit, the fitting is determined to fail, dec is set to "0", and then step S67 is executed, otherwise step S65 is executed;

[0053] S65, set dec to "1";

[0054] S66, set the fitting result parameter alpha k Push the array container R l In the process of iteration, when R l Length reaches the threshold A l When the new fitting result parameter is pushed into the tail of the container, the fitting result parameter stored in the head of the container is extracted at the same time, R l Length will remain constant after A l ;

[0055] S67, local optimal correction, when the current fitting result is determined by using R l Stored historical fitting results, in order to avoid falling into local optimum, set the local optimal correction coefficient A C The calculation formula of local optimal correction is as follows:

[0056]

[0057] m α =m α ×A C

[0058] Wherein, ls is the element number of the array container R l , that is, the local optimal correction is executed once every time the state update function is called.

[0059] Compared with the prior art, the present application has the following advantages:

[0060] The application is characterized in that: the video stream of the train operation is preprocessed to be converted into a binary image sequence about scene features, and the algorithm parameter is initialized according to the binary image sequence; then the state update function is constructed, the global variable state parameter is set and initialized; then the candidate coordinate set is searched based on the sliding window search algorithm or the band-pass search algorithm; then the track line parameter is fitted to obtain three key parameters, the parameter fitting result is judged based on the constructed state update function, and the algorithm state parameter is updated; then the quadratic curve parameter sequence obtained by fitting is smoothed, the curve coordinates are extracted and perspective inverse transformation is performed, and the track area is drawn in the original video frame sequence. Thus, the track area in front of the operating train is accurately detected and extracted, the target area range of the foreign object intrusion limit sensing and detection in the train operation process is reduced, the accuracy and detection speed of the foreign object intrusion limit detection are improved, and the train safety is reliably ensured.

[0061] In the application, when the collected video stream data is preprocessed, the collected video stream data is first decomposed into a video frame sequence F arranged in time, and each image in the video frame sequence F is preprocessed: including six steps of gray scale, scene division, parameter self-adaptation, perspective transformation, region relocation and edge detection, which can process different running scenes, ensure that the converted binary image sequence can accurately correspond to different scene features, and thus help to accurately sense the track area in the complex running environment.

[0062] The state update function is designed to update the state parameter according to the track line fitting result, the state parameter is set as a Boolean logic value "1" when the extraction result is judged to be valid, the state parameter is set as "0" when the extraction result is judged to be invalid, the state parameter is set as "0" when initialized, and the subsequent search algorithm is determined by the state parameter value, the sliding window search algorithm is used when the state parameter value is "0", and the band-pass search algorithm is used when the state parameter value is "1", so as to ensure that the candidate coordinate set is accurately searched, and the accuracy of the track line fitting is improved. BRIEF DESCRIPTION OF DRAWINGS

[0063] Figure 1 The figure is a method flowchart of the application;

[0064] Figure 2a The figure is a "bright light" environment running scene division schematic diagram in the embodiment;

[0065] Figure 2b The figure is a "dark light" environment running scene division schematic diagram in the embodiment;

[0066] Figure 3 The figure is a schematic diagram of image preprocessing of the original image in the embodiment;

[0067] Figures 4a to 4d This is a schematic diagram illustrating the process of performing edge detection on the target region of an image undergoing perspective transformation and obtaining a binarized image, as shown in the embodiment. Figure 4a For the original image, Figure 4b It is a grayscale image. Figure 4c For the gray-level gradient of the convolution operation, Figure 4d It is a binary image;

[0068] Figure 5 This is a schematic diagram illustrating the method for binarizing pixels in this embodiment;

[0069] Figure 6 This is a schematic diagram of the sliding pane growth in the sliding pane search algorithm of the embodiment;

[0070] Figure 7 This is the set of candidate coordinates obtained in the binary image based on a sliding pane-like search algorithm in the embodiment;

[0071] Figure 8 This is an illustration of the passband extension in the passband search algorithm in the embodiment;

[0072] Figure 9 This is the set of candidate coordinates obtained in the binary image based on a passband-like search algorithm in the embodiment;

[0073] Figure 10 This is a flowchart illustrating the algorithm logic of constructing a state update function, setting state parameters, and updating parameters based on the trajectory fitting results in this embodiment.

[0074] Figures 11a to 11d This is a schematic diagram of the track region extraction results for several typical operating scenarios in the embodiment, wherein: Figure 11a For straight path extraction results in bright light environments, Figure 11b For the results of curve extraction in bright light environment, Figure 11c The results are extracted based on normal lighting conditions in the tunnel. Figure 11d The results are from the extraction of extremely dark light in the tunnel. Detailed Implementation

[0075] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0076] Example

[0077] like Figure 1 As shown, a method for detecting and extracting track regions based on video streams includes the following steps:

[0078] S1. Collect train operation video stream data and preprocess the video stream to convert it into a binary image sequence based on scene features;

[0079] S2. Based on the binarized image sequence, calibrate the initialization parameters;

[0080] S3, constructing a state update function, setting a global variable state parameter and initializing;

[0081] S4, searching a candidate coordinate set based on a sliding window search algorithm or a passband search algorithm according to the initialized parameters and the global variable state parameter;

[0082] S5, fitting a quadratic orbit curve x=a0+a1y+a2y 2 based on the candidate coordinate set according to a least square method, and obtaining and storing key parameters a={a0, a1, a2} in a quadratic curve parameter sequence;

[0083] S6, judging the validity of the key parameters based on the constructed state update function, and updating the global variable state parameter;

[0084] S7, filtering and smoothing the quadratic curve parameter sequence;

[0085] S8, extracting curve coordinates from the filtered and smoothed quadratic curve parameter sequence, and performing a perspective inverse transformation to draw an orbit region in the original video frame sequence.

[0086] Through the above steps, the front orbit region in the train operation process is extracted based on the train forward video, additional detection equipment is avoided for the train, the complexity of the system is reduced, the scheme can be used as a supplement and optimization of the foreign object intrusion limit detection algorithm, the calculation amount and running time of the algorithm are reduced, the false alarm rate is reduced, the detection algorithm performance is improved, the accuracy of the track region foreign object intrusion detection is improved, the reliability of the system is improved, and the train safety is ensured. In addition to being applied to track foreign object intrusion limit detection, the scheme can also be used for detection of specific objects in the track region, and a more accurate and direct detection range can be provided when target detection is performed.

[0087] Specifically, in step S1, video stream data of a train (subway or EMU) in the operation process is collected as train front video data recorded by a subway or EMU on-board camera during operation;

[0088] The collected video stream data is decomposed into a video frame sequence F arranged in time, that is, each frame of image of the train forward video data is extracted and sorted in time sequence to obtain the video frame sequence F={f1,…,f k-1 ,f k ,…,f n}, wherein f1 is an image corresponding to a video starting frame, f k is an image corresponding to a k-th frame of the video, and f n is an image corresponding to a video ending frame;

[0089] Image preprocessing is performed on each image in the video frame sequence F, including six steps: grayscale conversion, scene segmentation, parameter adaptation, perspective transformation, region relocalization, and edge detection. After image preprocessing, the video frame sequence is transformed into a binary image sequence B based on scene features. The image preprocessing process is as follows:

[0090] (1a) Grayscale conversion: The video frame sequence F is grayscale converted to obtain the train forward video grayscale image sequence G = {g1,…,g k-1 ,g k ,…,g n}, where g1 is the grayscale image of the starting frame of the video, g k Let g be the grayscale image of the k-th frame of the video. n This is a grayscale image of the last frame of the video.

[0091] (1b) Scene segmentation: Calculate the scene features of each image in the train's forward-facing video grayscale image sequence G, obtaining a time series C = {c1,…,c...} of the scene features. k-1 ,c k ,…,c n The calculation formula is as follows:

[0092]

[0093]

[0094] Among them, gh k gl represents the number of pixels in the k-th grayscale image whose grayscale value is greater than or equal to the threshold d0. k c represents the number of pixels in the k-th grayscale image whose grayscale value is less than the threshold d0. k Then it is the ratio of the two; l k To label the scene category of the k-th grayscale image, the train operation scene is divided into two types: "bright light" and "dark light". When the scene is identified as "bright light", let l k =1, when the scene is identified as "dark light" let l k =0, and d1 is the threshold for distinguishing between the two typical operating scenarios;

[0095] (1c) Parameter Adaptation: Divide the running scenarios and assign separate algorithm parameters for different running scenarios to obtain the scenario parameter assignment sequence S = {s1,…,s} k-1 ,s k ,…,s n}, s k ={d0,d1,…};

[0096] Where s1 is the parameter assignment set of the grayscale image of the starting frame of the video, sk Assign a set of parameter values ​​to the grayscale image of the k-th frame of the video, s n A parameter set is assigned to the grayscale image of the last frame of the video. The parameters assigned to each scene are fixed, and the number of parameters is determined according to the needs of algorithm improvement.

[0097] (1d) Perspective Transformation: Perform perspective transformation on the grayscale image sequence G. Select any quadrilateral region in the grayscale image sequence G that includes the track, and mark the quadrilateral region Or = {(x1,y1),(x2,y2),(x3,y3),(x4,y4)} using the coordinate system of image G (the coordinates of the four points are the coordinates of the vertices of the upper left, upper right, lower right, and lower left corners of the quadrilateral, respectively, the same below). The target region after perspective transformation is a rectangular region Pr = {(x′1,y′1),(x′2,y′2),(x′3,y′3),(x′4,y′4)}. At the same time, generate the perspective transformation transition matrix M and the inverse perspective transformation inverse transition matrix M. in And obtain the perspective grayscale image sequence T={t1,…,t after perspective transformation. k-1 ,t k ,…,t n}, where t1 is the perspective transformed image of the first grayscale image, t k Let t be the perspective transformed image of the k-th grayscale image. n The perspective-transformed image of the last grayscale image;

[0098] (1e) Region Relocalization: For a perspective grayscale image sequence, a mask function is constructed to extract a rectangular region containing only the track from the perspective grayscale image sequence, while setting the grayscale of the remaining pixels in the image to 0, resulting in a perspective grayscale image sequence R = {r1,…,r...} after region relocalization. k-1 ,r k ,…,r n}, where r1 is the perspective grayscale image of the first frame region repositioning, r k For the perspective grayscale image of the k-th frame region repositioned, r n A perspective grayscale image for repositioning the region in the last frame;

[0099] (1f) Edge detection: The edge detection method based on the Sobel operator uses the Sobel operator in the X direction to perform convolution operation on the perspective grayscale image sequence R for region relocalization, and selects a 3×3 convolution kernel. The absolute value of the convolution result for each pixel in the image is taken to obtain the gray-level gradient at that pixel location. After obtaining the gray-level gradient at each pixel location, the pixel is binarized according to the parameter assignment results of scene recognition, finally obtaining the edge detection binarized image sequence B = {b1,…,b...}k-1 ,b k ,…,b n}, where b1 is the edge detection binarized image of the first frame perspective grayscale image, b k b is the edge detection binarized image of the k-th frame perspective grayscale image. n Binarize the edge detection image of the last frame of the perspective grayscale image;

[0100] The specific method for pixel binarization is as follows: a high threshold and a low threshold minVal are set. When the gray-level gradient at the location of a pixel is greater than maxVal, the gray-level value of the pixel is assigned to 255. When the gray-level gradient at the location of a pixel is less than minVal, the gray-level value of the pixel is assigned to 0. When the gray-level gradient at the location of a pixel is between maxVal and minVal, it is then determined whether there are any adjacent pixels with a gray-level value of 255 in the surrounding 8 pixels. If there are, the gray-level value of the pixel is assigned to 255. If not, the gray-level value of the pixel is assigned to 0.

[0101] In step S2, based on the binarized image sequence B obtained in step S1, the initialization parameters of the trajectory line extraction algorithm are calibrated, specifically including:

[0102] (2a) Extract the first image b1 from the binarized image sequence B, and calibrate the center position coordinates O(LB) of the track on one side at the bottom edge of the image. X ,LB Y );

[0103] (2b) Calibrate the track spacing R between the two tracks under the condition of sequence B size. D .

[0104] In step S3, the constructed state update function updates the state parameters based on the orbital fitting result. When the orbital fitting result is determined to be valid, the state parameters are set to the Boolean logic value "1", and when the orbital fitting result is determined to be invalid, they are set to "0". During initialization, the state parameters are set to "0".

[0105] In step S4, the search algorithm used is determined by the state parameter value. When the state parameter value is "0", a sliding pane-like search algorithm is used; when the state parameter value is "1", a passband-like search algorithm is used. If a sliding pane-like search algorithm is used, then according to the initialization parameters calibrated in step S2, candidate track pixel coordinates are searched in the binarized image sequence B obtained in step S1, and a candidate coordinate set G is generated. The specific steps are as follows:

[0106] (4a) Based on the calibrated image-side track lower edge center position in step S2, the starting position of the rectangular sliding window is determined. The positioning method is: taking the coordinate O pixel point as the midpoint of the sliding window lower frame, and setting the size parameters of the sliding window: width w win , and height h win , and the parameters are adjustable;

[0107] (4b) Search for white pixel points (gray value = 255) in the rectangular sliding window, and record the image coordinates of these white pixel points to the candidate coordinate set G, and count the number of white pixel points N i in the window;

[0108] (4c) Set the white pixel point number judgment threshold N u in the window, and compare the size relationship between N i and N u to determine the sliding window growth method to the image upper boundary;

[0109] When N i ≥ N u , the average value of the horizontal coordinates of all white pixel points in the window is calculated as the horizontal coordinate of the midpoint of the lower frame of the next sliding window, and when the number of boundary pixel points in the window is too small (N i < N u ), the horizontal coordinate of the lower edge of the current window is still taken as the horizontal coordinate of the midpoint of the lower frame of the next sliding window;

[0110] After the sliding window grows once from the image lower boundary to the upper boundary, the lower frame of the new sliding window grown is the same as the vertical coordinate of the upper frame of the sliding window before growth;

[0111] (4d) Iteration, the sliding window grows once to the image upper boundary, and steps (4b) and (4c) are repeated, until the upper frame of the rectangular sliding window reaches the image upper boundary, the iteration ends, and a complete candidate coordinate set G in the binary image is obtained.

[0112] If the class channel search algorithm is used, the specific steps are:

[0113] (4aa): Input the previous frame binary image b k-1 , and determine the position of the fitted track line x = a0 + a1y + a2y k-1 in the current frame binary image b k , and the track line fitting result parameter a 2 = {a0, a1, a2};

[0114] (4bb): Shift the track line fitted in step (4aa) to the left and right sides of the image by a certain distance A passband of equal width along the x-axis is formed, with the fitted orbital line from the previous frame as its central axis, and the passband width is d. m ;

[0115] (4cc): Search for the white pixels covered by the passband generated in step (4bb) in the binarized image and record the coordinates of these white pixels in the candidate coordinate set G′.

[0116] In step S5, based on the least squares method, the parameter sequence of the fitted quadratic curve is as follows:

[0117] A = {α1,…,α} k-1 ,α k ,…,α n}

[0118] Where α1 is the fitting curve parameter of the first frame of binarized image, α k Let α be the fitting curve parameter of the binarized image of the k-th frame. n These are the parameters for the fitted curve of the last frame of the binarized image.

[0119] In step S6, the state update function constructed in step S3 is used to determine whether the trajectory fitting result is valid and to update the state parameters. The specific steps are as follows:

[0120] (3a): Construct a state update function and a global variable dec. dec is used to represent the fitting result of the trajectory line of the binary image. It is assigned a value of "1" when the fitting is successful and a value of "0" when the fitting fails. dec is set to "0" when the algorithm is initialized.

[0121] (3b): Constructing α k Array container R of type l The container length threshold is set to A. l ;

[0122] (3c): Initialize the orbital fitting result α k ={0,0,0, container R} l All α k The average value m of type data α ={0,0,0};

[0123] (3d): Input trajectory fitting result α k ;

[0124] (3e): Determine α k If all three parameters are 0, the fitting is considered to have failed, and dec is set to "0" to proceed to step (3j). If not, proceed to the next step.

[0125] (3f): Determine α kIf the three parameters do not satisfy the respective independent determination condition and the joint determination condition, the fitting is determined to fail, dec is set to "0", and step (3j) is entered. If the three parameters satisfy the respective independent determination condition and the joint determination condition, the next step is entered.

[0126] The independent determination condition is to set a threshold limit parameter absolute value range.

[0127] The joint determination condition is to set an image region limit fitting curve possible coverage range.

[0128] (3g): Calculate the container R l of all α k type data average value m α , determine α k whether the three parameters and the calculated average value satisfy the separate threshold condition limit. If not, the fitting is determined to fail, dec is set to "0", and step (3j) is entered. If yes, the next step is entered.

[0129] (3h): Set dec to "1";

[0130] (3i): The fitting result parameter α k that determines the fitting to be successful is pushed into the array container R l . In the process of iteration, when the length of R l reaches the threshold A l , the new fitting result parameter is pushed into the tail of the container while the fitting result parameter stored in the head of the container is extracted, and the length of R l will remain constant after reaching A l .

[0131] (3j): Local optimum correction. When using the historical fitting result stored in R l to determine the current fitting result, it may fall into local optimum. Set the local optimum correction coefficient A C to avoid the algorithm falling into local optimum error determination in the process of iteration. The calculation formula of the local optimum correction is as follows:

[0132]

[0133] m α = m α × A C

[0134] Where ls is the number of elements of the array container R l . Each time the state update function is called, the local optimum correction is executed once.

[0135] In step S7, the quadratic curve parameter sequence A obtained in step S5 is filtered based on a Kalman filtering algorithm to obtain a filtered parameter sequence K A = {kα1, kα2, …, kα k-1 , …, kα k , kα n}, wherein kα1 is a Kalman correction of the first frame of the binarized image fitting curve parameter, kα k is a Kalman correction of the kth frame of the binarized image fitting curve parameter, and kα n is a Kalman correction of the last frame of the binarized image fitting curve parameter.

[0136] In step S8, according to the parameter sequence K A filtered in step S7, curve coordinates are extracted in the y-axis direction at equal intervals according to the image size in the binarized image sequence B to obtain a curve coordinate sequence P T , and the inverse transfer matrix M in obtained in step S1 is used to perform perspective inverse transformation on the curve coordinate sequence P T to obtain a quadratic curve coordinate sequence P that conforms to the image size in the original video frame sequence F. The quadratic curve and the area between the curves are labeled according to the coordinate sequence P in the image coordinates of the video frame sequence F, so that the extraction of the track area is realized.

[0137] To verify the effectiveness of the technical solution, this embodiment takes a train operation of a certain subway line as an example for description.

[0138] (I) Extract a front-facing video data of 12 minutes and 46 seconds in length taken by a front camera during the running process of a train on Shanghai Metro Line 11.

[0139] (II) The collected subway train video stream data is decomposed into video frames to obtain a sequence of 22980 consecutive video frames (fps: 30) with a resolution of 1920*1080, i.e., video frame sequence F = {f1, f2, …, f k-1 , f k , …, f 22980}, wherein each element in F is a 1920*1080 matrix.

[0140] (III) The video frame sequence is converted into a binarized image sequence B about scene features, and the collective steps are as follows:

[0141] (1) Perform grayscale processing on the video frame sequence F to obtain a grayscale image sequence G = {g1, g2, …, g k-1 , g k , …, g 22980}, each pixel gray value is calculated according to the RGB value of the pixel point, and the calculation formula is as follows, wherein Gray is the pixel gray value:

[0142] Gray = 0.299 * R + 0.587 * G + 0.144 * B

[0143] (2) as shown in Figure 2a and 2b , set the gray value threshold d0 = 60, calculate the scene feature c of each image in the gray image sequence G using the formula , and obtain the time sequence C = {c1,..., c k-1 ,c k ,…,c 22980} about the running scene feature, set the "bright light" and "dark light" two typical running scene judgment thresholds d1 = 0.7, and obtain the scene category sequence L = {l1,..., l k-1 ,l k ,…,l 22980} about the gray image sequence;

[0144] (3) according to the scene category sequence L obtained in step (2), the gray image sequence G is classified and the algorithm parameter is assigned, and the scene parameter assignment sequence S = {s1,..., s k-1 ,s k ,…,s 22980} is obtained, s k = {d0, d1,...} ;

[0145] (4) as shown in Figure 3 , the gray image sequence G is processed by perspective transformation, and a quadrilateral region containing the track is calibrated in the G image coordinate system:

[0146] Or = {(924, 600), (1065, 600), (1659, 1075), (425, 1075)}

[0147] and set the target region after perspective transformation as a rectangular region:

[0148] Pr = {(0, 0), (617, 0), (617, 525), (0, 525)}

[0149] According to the perspective transformation principle, the perspective transformation transfer matrix and the perspective inverse transformation transfer matrix are obtained, and the perspective gray image sequence T = {t1,..., t k-1 , t k ,…, t 22980} after perspective transformation is obtained, and each element in T is a 618*526 matrix;

[0150] (5) For the perspective grayscale image sequence T, construct a mask function roi_mask to extract the rectangular region containing only the track in the perspective grayscale image sequence T, and obtain the perspective grayscale image sequence R = {r1,…,r} after region relocalization. k-1 ,r k ,…,r 22980 The basic principle is to construct a binary image of size T, with a gray value of 0 in the target area and a gray value of 255 outside the target area. This binary image is then fused with each image in the gray image sequence T, thereby retaining only the image features within the rectangular target area and filtering out the image features outside the target area.

[0151] (6) Figures 4a to 4d As shown, the edge detection method based on the Sobel operator performs convolution operations on the perspective grayscale image sequence R of the region relocalization using the Sobel operator in the X direction to enhance the edge features of the track in the image. A 3×3 convolution kernel is selected. The absolute value of the convolution result for each pixel in the image is taken to obtain the gray-level gradient at that pixel location. After obtaining the gray-level gradient at each pixel location, parameter values ​​are assigned according to the scene category:

[0152] When the scene category is "1", set the high threshold maxVal = 220 and the low threshold minVal = 70;

[0153] When the scene category is "0", set the high threshold maxVal = 30 and the low threshold minVal = 10;

[0154] The pixel is binarized based on the scene parameters, resulting in an edge detection binarized image sequence B = {b1,…,b...} k-1 ,b k ,…,b 22980};

[0155] Figure 5 An example of a pixel binarization method, such as Figure 5 As shown, the grayscale value of the pixels in segment A is assigned to 255. The pixels in segment B are adjacent to those in segment A, so the grayscale value of the pixels in segment B is also assigned to 255. The pixels in segment C have no adjacent pixels, so the grayscale value of their pixels is assigned to 0.

[0156] (iv) Determine the key initialization parameters of the track line extraction algorithm based on the binarized image sequence B, including: the center position coordinates O(LB) of one side of the track at the lower edge of the image. X =322,LB Y =525) and track spacing R D =180.

[0157] (Five) as shown in the figure, when the global variable dec = "0", based on the class sliding window search algorithm, according to the calibration of the initial parameters, in the binary image sequence B search candidate track pixel coordinates and generate candidate coordinate set G, the specific steps are as follows: Figure 6

[0158] (1) positioning sliding window starting position, the coordinates O (322, 525) as the sliding window lower edge midpoint, set the sliding window size parameters: width is 30, height is 21;

[0159] (2) search for white pixel points (gray value = 255) in the rectangular sliding window, and record the image coordinates of these white pixel points to the candidate coordinate set G, while counting the number of white pixel points N i in the window;

[0160] (3) set the white pixel point number judgment threshold N u = 35, compare the size relationship between N i and N u to determine the growth method of the sliding window to the upper boundary of the image;

[0161] (4) iteration, the sliding window grows to the upper boundary of the image once, repeat step (2) and step (3), until the upper edge of the rectangular sliding window reaches the upper boundary of the image, the iteration ends, and a complete candidate coordinate set G in a binary image is obtained;

[0162] Figure 7 is the candidate coordinate set searched by the sliding window in the growth process;

[0163] (Six) as shown in the figure, when the global variable dec = "1", based on the passband search algorithm, search for candidate track pixel coordinates in the binary image sequence B and generate candidate coordinate set G', the specific steps are as follows: Figure 8

[0164] (1) input the previous frame binary image b k-1 Track line fitting result parameters a k-1 = {a0, a1, a2}, and determine the position of the previous frame fitting track line x = a0 + a1yaa2y k in the current frame binary image b 2 ;

[0165] (2) use the track line fitted in step (1) of the previous frame to respectively translate to the left and right sides of the image by a fixed distance of 15, forming a passband with the previous frame fitting track line as the central axis and equal width in x axis direction, the passband width is 30;

[0166] ​​(3) Search the white pixel points covered by the passband in the binary image, and record the coordinates of these white pixel points in the candidate coordinate set G';

[0167] Figure 9 Extend the candidate coordinate set covered by the passband.

[0168] (Seven) Based on the least squares method, a quadratic curve is fitted according to the candidate coordinates, and a quadratic curve parameter sequence A = {a1, …, a k-1 , a k , …, a 22980} is obtained.

[0169] (Eight) As shown in Figure 10 , the state parameters of the track line extraction algorithm are constructed and updated, and the specific steps of the logic block diagram are as follows:

[0170] (1) Construct the state update function update and the global variable dec, dec is used to represent the track line fitting result of the binary image, and is assigned a value of "1" (corresponding to "True" in the diagram) when the fitting is successful, and is assigned a value of "0" (corresponding to "False" in the diagram) when the fitting fails, and dec is set to "0" when the algorithm is initialized;

[0171] (2) Construct an array container R k of type a l , and set the container length threshold to 10;

[0172] (3) Initialize the track line fitting result a k = {0, 0, 0, the average value m l = {0, 0, 0} of all a k type data in the container R α ;

[0173] (4) Input the track line fitting result a k ;

[0174] (5) Determine whether the three parameters of a k are all 0, if so, it is determined that the fitting fails, and dec is set to "0", and step (10) is entered, if not, the next step is entered;

[0175] (6) Determine whether the three parameters of a k satisfy the independent determination condition and the joint determination condition respectively, if not, it is determined that the fitting fails, and dec is set to "0", and step (10) is entered, if so, the next step is entered;

[0176] The independent determination condition is to set a threshold limit parameter to limit the range of possible values of the absolute value;

[0177] The joint determination condition is to set the image region to limit the possible coverage of the fitting curve;

[0178] (7) Calculate the container R l All α k The average value m of the type data α , determine α k Whether the three parameters and the average value meet the separate threshold condition limit, if not, determine that the fitting fails, set dec to "0", and go to step (10), if yes, go to the next step;

[0179] (8) Set dec to "1";

[0180] (9) The fitting result parameter α k Determined to be successful in fitting is pushed into the array container R l , in the process of iteration, when the length of R l Reaches the threshold 10, the new fitting result parameter is pushed into the tail of the container, and the fitting result parameter stored in the head of the container is extracted at the same time, the length of R l Will remain constant after reaching 10;

[0181] (10) Local optimum correction, when using the historical fitting results stored in R l To determine the current fitting result, it may fall into local optimum, set the local optimum correction coefficient A C = 0.9 to avoid the algorithm falling into local optimum error judgment in the process of iteration.

[0182] (Nine) Based on the Kalman filter algorithm, the quadratic curve parameter sequence obtained by fitting is filtered to obtain the filtered parameter sequence K A ={kα1,…,kα k-1 , kα k ,…,kα 22980}.

[0183] (Ten) According to the image size in the binary image sequence B, the curve coordinates are extracted in the y-axis direction at equal intervals of 10 to obtain the curve coordinate sequence P T , and the curve coordinate sequence P in Is inversely transformed according to the inverse transfer matrix M T , to obtain the quadratic curve coordinate sequence P which conforms to the image size in the video frame sequence F, and according to the coordinate sequence P, the quadratic curve and the area between the curves are labeled in the image coordinates in the video frame sequence F, that is, the extraction of the track area is realized. Figures 11a to 11d As shown in the figure, it is the extraction result of the track area of the train in several typical running scenarios, wherein, Figure 11a It is the extraction result of the straight track in bright light environment, Figure 11b It is the extraction result of the curved track in bright light environment,Figure 11c for tunnel normal light extraction result, Figure 11d for tunnel extremely dark light extraction result.

[0184] In summary, the technical scheme does not add additional vehicle-mounted equipment, uses the video stream data collected in the train operation process to perceive and extract the front track area in the train operation process, narrows the target area range of foreign matter intrusion limit perception and detection in the train operation process, is conducive to improving the accuracy and detection speed of subsequent foreign matter intrusion limit detection, can realize accurate perception and extraction of the track area in a complex operation environment, can be used as a supplement and optimization of the foreign matter intrusion limit detection algorithm, reduces the calculation amount and running time of the algorithm, reduces the false alarm rate, improves the detection algorithm performance, can improve the accuracy of track area foreign matter intrusion detection, and improves the reliability of the system and ensures train safety. In addition to being applied to track foreign matter intrusion limit detection, the scheme can also be used for detection of specific objects in the track area, and can provide more accurate and direct detection range when performing target detection.

Claims

1. A method for track area detection and extraction based on video stream, characterized in that, The method comprises the following steps: S1, collecting train operation video stream data, and pre-processing the video stream to convert the video stream into a binary image sequence about scene features; S2, calibrating initial parameters according to the binary image sequence; S3, constructing a state update function, setting a global variable state parameter and initializing; S4, searching for a candidate coordinate set based on a sliding window search algorithm or a band-pass search algorithm according to the calibrated initial parameters and the global variable state parameter; S5, based on the least square method, fitting a quadratic orbit curve according to the candidate coordinate set and obtaining the key parameters and storing them in the quadratic curve parameter sequence, wherein the key parameters are ; S6, judging the validity of a key parameter based on the constructed state update function, and updating the global variable state parameter; S7, performing filtering and smoothing processing on the quadratic curve parameter sequence; S8, extracting curve coordinates from the filtered and smoothed quadratic curve parameter sequence, and performing perspective inverse transformation to draw a track area in the original video frame sequence; Step S6 specifically comprises the following steps: S61, input the current track line fitting result ; S62, judging whether all of the three key parameters are 0, if yes, it is determined that the fitting fails, and the fitting result is set as , then step S67 is executed, otherwise step S63 is executed; ​ S63, judging whether the three key parameters satisfy respective preset independent determination conditions and preset joint determination conditions, and if not, determining that the fitting fails, setting to , and then performing step S67, otherwise performing step S64; The independent determination condition is to set a threshold limit parameter absolute value possible value range; The joint determination condition is to set an image area limit fitting curve possible coverage range; S64, compute container all inside average of type data , determine whether the three parameters and the computed average meet individual threshold condition limits, if not, then determine that the fit failed and set to , then perform step S67, otherwise perform step S65; S65、will be set to ; S66, the fitting result parameter determined to be a successful fitting pushing into an array container , during the iteration, when the length reaches a threshold , pushing a new fitting result parameter into the tail of the container, while pulling out the fitting result parameter stored in the head of the container, the length will remain constant after reaching ​ S67, local optimum correction, in use In order to avoid falling into local optimum, a local optimum correction coefficient is set when the stored historical fitting result is used to determine the current fitting result The calculation formula of the local optimum correction is as follows: wherein is an array container number of elements, each call to the state update function performs a local optimal correction. 2.The track area detection and extraction method based on video stream according to claim 1, wherein, The preprocessing of the video stream in the step S1 is specifically to decompose the collected video stream data into a video frame sequence arranged in time , and to perform image preprocessing on each image in the video frame sequence : including grayscale, scene division, parameter self-adaptation, perspective transformation, region relocation and edge detection. After image preprocessing, the video frame sequence is converted into a binary image sequence about scene features . The video stream data is the frontward video data of a vehicle head recorded by a vehicle-mounted camera during operation, and the video frame sequence , wherein is an image corresponding to a video starting frame, is an image corresponding to a video frame , and is an image corresponding to a video ending frame. 3.The track area detection and extraction method based on video stream according to claim 2, characterized in that, The image preprocessing in step S1 specifically comprises the following steps: S11. Grayscale conversion: Converting a video frame sequence to grayscale. Perform grayscale conversion to obtain a grayscale image sequence. ,in, The grayscale image of the starting frame of the video. For the video number grayscale image of a frame image This is a grayscale image of the last frame of the video. S12, scene division: respectively calculate the running scene feature of each image in the gray image sequence , and obtain the time sequence about the running scene feature , the calculation formula is as follows: wherein, is the number of pixel points in the first gray scale image whose gray scale value is greater than or equal to the threshold value, is the number of pixel points in the first gray scale image whose gray scale value is less than the threshold value, is the ratio of the two, is the number of pixel points in the first gray scale image whose gray scale value is greater than or equal to the threshold value, is the number of pixel points in the first gray scale image whose gray scale value is less than the threshold value, is the ratio of the two, is the scene category label of the first gray scale image, which divides the train running scene into "bright light" and "dark light", when the scene is identified as "bright light", when the scene is identified as "dark light", , , , is the judgment threshold value of the division of the two typical running scenes. S13. Parameter Adaptation: Divide the running scenarios and assign separate algorithm parameter values ​​for different running scenarios to obtain a scenario parameter assignment sequence. , ,in, Assign a set of parameters to the grayscale image of the starting frame of the video. For the video number The parameter assignment set for a frame grayscale image. A set of parameter assignments is provided for the grayscale image of the last frame of the video, and the parameters are fixed for each scene. S14, perspective transformation: on the gray image sequence carries on the perspective transformation processing, in the gray image sequence selects the arbitrary quadrilateral area including track and uses image coordinate system calibration this quadrilateral area , successively corresponds four vertex coordinates for quadrilateral left upper corner, right upper corner, right lower corner and left lower corner. The target region after perspective transformation is a rectangular region a perspective transformation transfer matrix is generated simultaneously and an inverse transfer matrix of the perspective inverse transformation and a perspective grayscale image sequence after perspective transformation is obtained wherein, is a perspective transformation image of the first frame grayscale image, is a perspective transformation image of the frame grayscale image, is a perspective transformation image of the last frame grayscale image; S15, region repositioning: for the perspective gray image sequence , a mask function is constructed, and the rectangular region of the perspective gray image sequence containing only the track is extracted, and the gray values of the pixels at the remaining positions of the image are set to 0, thereby obtaining a region repositioned perspective gray image sequence , wherein, is a first frame of region repositioned perspective gray image, is a frame of region repositioned perspective gray image, is a last frame of region repositioned perspective gray image. S16. Edge Detection: An edge detection method based on the Sobel operator, using the Sobel operator in the X direction to relocalize the region from the perspective grayscale image sequence. Perform convolution operations, selecting a 3×3 kernel. The absolute value of the convolution result for each pixel in the image is taken to obtain the gray-level gradient at that pixel's location. After obtaining the gray-level gradient at each pixel's location, the pixel is binarized according to the parameter assignment results of scene recognition, ultimately resulting in a binarized image sequence for edge detection. ,in, This is the edge detection binarized image of the first frame of the perspective grayscale image. For the first Edge detection and binarization of a frame-perspective grayscale image. This is the edge detection binarization image of the last frame of the perspective grayscale image.

4. The track area detection and extraction method based on video stream according to claim 3, characterized in that, The specific process of the binarization of the pixel point in step S16 is: setting a high threshold value and a low threshold value , when the gray gradient of the position of the pixel point is greater than , the gray value of the pixel point is 255; when the gray gradient of the position of the pixel point is less than , the gray value of the pixel point is 0; when the gray gradient of the position of the pixel point is between and , it is further judged whether there is a pixel point with a gray value of 255 adjacent to the pixel point, if yes, the gray value of the pixel point is 255; if not, the gray value of the pixel point is 0.

5. The method of claim 2, wherein the method further comprises: Step S2 specifically comprises the following steps: S21, extracting a sequence of binary images S21, extracting a sequence of binary images S21, extracting a sequence of binary images S21, extracting a sequence of binary images S22, calibrate both sides of the track in sequence Track spacing under dimensional conditions .

6. The track area detection and extraction method based on video stream according to claim 5, wherein, Step S3 specifically comprises the following steps: S31, configuration state update function and global variable , To represent the binary image track line fitting results, assign value when fitting success , assign value when fitting failure , initialize to Set to ; S32, configure Array containers of type , the container length threshold is set to ; S33, initialize the track line fitting result , the container all within average value of the type data .

7. The method of claim 6, wherein the method further comprises: In step S4, if the current global variable state parameter is "0", a sliding window search algorithm is used to search for a candidate coordinate set; if the current global variable state parameter is "1", a band-pass search algorithm is used to search for a candidate coordinate set. 8.The track area detection and extraction method based on video stream according to claim 7, characterized in that, The specific process of the sliding window search algorithm is as follows: A1, based on the calibrated image side rail lower edge center position, determine the starting position of the rectangular sliding window, positioning method is: the coordinates Pixel as the midpoint of the sliding window lower frame, set the size parameters of the sliding window at the same time: width is , height is , and the parameters are adjustable; A2, search white pixel points (gray value = 255) in the rectangular sliding window, and record the image coordinates of these white pixel points to the candidate coordinate set Meanwhile, count the number of white pixel points in the window ; A3, set the number of white pixel points in the window pane judgment threshold , compare and size relationship to determine the sliding window pane to the image upper boundary growth method; when At that time, calculate the average x-coordinate of all white pixels within the pane. , as the x-coordinate of the midpoint of the bottom border of the next sliding pane; and when If the current pane's bottom edge coordinate is used, then the bottom edge coordinate of the current pane will still be used as the midpoint coordinate of the bottom border of the next sliding pane. After the sliding window grows once from the lower boundary of the image upward, the lower frame of the new sliding window is the same as the upper frame of the sliding window before growth in the vertical coordinate; A4、iteration, sliding window each time to the image upper boundary growth, repeat step A2 and step A3, until the rectangular sliding window upper frame reaches the image upper boundary, the iteration ends, that is, a complete candidate coordinate set in a binary image is obtained .

9. The track area detection and extraction method based on video stream according to claim 7, wherein, The specific process of the band-pass search algorithm is as follows: B1, input previous frame binarization image track line fitting result parameter and determine the location of the fitted quadratic track curve of the previous frame in the current frame binarization image track line fitting result parameter track line fitting result parameter B2, shift the quadratic curve fitted in step B1 to the left and right of the image by a distance , forming a passband of equal width in the axial direction with the track line fitted in the previous frame as the central axis, the passband width being ; and ; B3. Search the white pixel points covered by the passband generated in step B2 in the binary image, and record the coordinates of these white pixel points in the candidate coordinate set in step B2 in the binary image, and record the coordinates of these white pixel points in the candidate coordinate set

Images