A method for detecting the clearance of rail transit

Abstract

The present invention provides a method for detecting the clearance of rail transit. The method includes: processing the acquired image data of the train's operating environment to generate a depth map; using a semantic segmentation algorithm to extract the track area in the depth map, and calculating the three-dimensional spatial coordinates of the track based on the track area and the depth map; performing a convolution optimal transformation process on the three-dimensional spatial coordinates of the track using a track prior constraint equation to form track three-dimensional point cloud data; based on the track three-dimensional point cloud data and prior knowledge, fitting the track centerlines of straight and curved segments by minimizing the sum of squared errors; extracting multi-scene track planes from the track three-dimensional point cloud data by minimizing the variance vector; constructing a three-dimensional solid track clearance bounding box by combining the track centerlines and the track planes to achieve three-dimensional detection of vehicle clearance and building clearance. The method of the present invention realizes the rapid detection of vehicle clearance and building clearance by constructing three-dimensional point cloud data and a clearance bounding box.

Description

Technical Field

[0001] The present invention relates to the field of clearance detection, and particularly to a method for detecting railway traffic clearance. Background Art

[0002] The scope of railway operation is constantly expanding, and railway safety issues have increasingly become the focus of attention. Among them, railway clearance detection, as a key link to ensure the safe operation of trains, has received extensive attention. The so-called railway clearance detection refers to ensuring that the space required for trains and their cargo is not interfered with during the operation of rail transit, including the safety distances on both sides and above the tracks, to prevent trains from colliding with surrounding facilities, vegetation, and other trains.

[0003] Traditional clearance detection methods mainly rely on manual inspections and regular mechanical scans. These methods are not only time-consuming and laborious, but also the detection accuracy is often not satisfactory. With the rapid development of computer vision and artificial intelligence technologies, automated detection technologies have begun to play an important role in railway clearance detection. Through image processing and deep learning algorithms, the railway clearance range can be efficiently and accurately identified and evaluated, thus significantly improving the detection efficiency and safety.

[0004] Current clearance detection technologies mainly rely on vision devices such as lidar, depth cameras, and 3D laser devices, or combinations thereof. However, the number of rail trains in China is huge, and the large-scale application of these high-cost devices will bring huge economic costs. Therefore, it is particularly important to find a clearance detection method with low cost, fast detection speed, high accuracy, and strong adaptability. Summary of the Invention

[0005] The present invention provides a method for detecting railway traffic clearance to effectively detect the railway traffic clearance.

[0006] To achieve the above object, the present invention adopts the following technical solutions.

[0007] According to one aspect of the present invention, there is provided a method for detecting railway traffic clearance, including:

[0008] Collecting image data of the operating environment in front of the train through a camera installed at the front end of the train, processing the image data of the operating environment to generate a depth map;

[0009] Extracting the track area in the depth map by using a semantic segmentation algorithm, and calculating the three-dimensional space coordinates of the track according to the track area and the depth map;

[0010] Performing convolution optimal transformation processing on the three-dimensional space coordinates of the track by using a track prior constraint equation to form track three-dimensional point cloud data;

[0011] Based on the three-dimensional point cloud data of the track and prior knowledge, the center line of the track of the straight line and the curved line is fitted by minimizing the sum of squared errors;

[0012] Extracting multi-scene track planes from the track three-dimensional point cloud data by using a variance vector minimization method;

[0013] A three-dimensional track limit bounding box is constructed by combining the track centerline and track plane to realize three-dimensional detection of vehicle limits and building limits.

[0014] Preferably, the collecting of operating environment image data in front of the train by a camera installed at the front end of the train, processing the operating environment image data to generate a depth map, includes:

[0015] Set up a calibration plate with a checkerboard pattern with a known black and white spacing that can be captured by the camera. Before installing the camera, use the calibration plate to calibrate and calculate the camera's intrinsic parameter matrix K, radial distortion coefficients k1, k2, k3, and tangential distortion coefficients p1, p2. The intrinsic parameter matrix K includes the focal length (f x ,f y ) and the pixel position coordinates corresponding to the camera center (c x ,c y );

[0016] The calibrated camera is installed at the center of the front of the train. The train is constantly moving forward. The camera collects image data of the running environment in front of the train. The device coordinate system is set as follows: the coordinates of the camera device are (0,0,0), the train travel direction is the y-axis, the direction perpendicular to the track plane is the z-axis, and the x-axis is perpendicular to the plane yoz;

[0017] The monocular estimation algorithm based on convolutional neural network is selected to process the collected operating environment image data to obtain each frame of color image I rgb The corresponding initial depth map D pred ;

[0018] For each frame depth map D pred and color image I rgb , use the depth completion algorithm to generate the depth map D completed .

[0019] Preferably, the extracting the track area in the depth map by using a semantic segmentation algorithm and calculating the three-dimensional spatial coordinates of the track according to the track area and the depth map includes:

[0020] The depth map D is segmented using a semantic segmentation algorithm. completed Process, segment the track area, and output the pixel coordinates (u, v) of the track area;

[0021] Obtain the internal parameter matrix and distortion coefficients of the camera, and use formula (1) to undistort the pixel coordinates (u, v) of the track area to obtain the corrected pixel coordinates (u c , v c );

[0022]

[0023] In the formula, r is the radial distance representing the distance from the point to the image center, (u, v), (u c , v c ) are the pixel coordinates before and after correction respectively, (c x , c y ) is the pixel position coordinate corresponding to the camera center, is a parameter included in the internal parameter matrix K, and k1, k2, k3 and p1, p2 are the radial distortion coefficients and tangential distortion coefficients;

[0024] Extract the depth value d corresponding to the pixel coordinates of the track area from the depth map D completed ;

[0025] Use the corrected pixel coordinates (u c , v c ) and the corresponding depth value d, and calculate the three-dimensional space coordinates (X, Y, Z) of the track in the camera coordinate system through formula (2);

[0026]

[0027] Perform visual odometry calculation through feature point matching according to the running environment image data to obtain the external parameter matrix [R|t] of the camera, including the rotation matrix R and the translation vector t;

[0028] Use the external parameter matrix of the camera and the three-dimensional space coordinates (X, Y, Z) in the camera coordinate system to calculate the three-dimensional space coordinates (X w , Y w , Z w ) of the track in the world coordinate system through formula (3);

[0029]

[0030] Preferably, the convolution optimal transformation process of the three-dimensional space coordinates of the track using the track prior constraint equation to form the track three-dimensional point cloud data includes:

[0031] Voxelize the 3D spatial coordinate point data in the world coordinate system on both sides of the track to divide the left and right regions into voxels, obtaining the left track voxel region and the right track voxel region. Calculate the centroid of the points in each voxel, and construct a specific track prior constraint equation through formula (4) to filter out the voxel regions that do not meet the prior constraints. Perform convolution optimal transformation smoothing and alignment processing on the 3D spatial coordinate point data to form track three-dimensional point cloud data;

[0032]

[0033] In the formula, Voxel i represents the spatial range of the i-th voxel, |V i | represents the number of points in the voxel, represents the centroid of the left region of the i-th voxel, represents the distance calculation from the centroid of the left region to the centroid of the right region of the i-th voxel, W 轨 is the known track width, and ε is the allowable error range.

[0034] Preferably, based on the track three-dimensional point cloud data and prior knowledge, the track centerline of straight lines and curve segments is fitted by minimizing the sum of squared errors, including:

[0035] Obtain the point cloud coordinates and track prior knowledge using the track three-dimensional point cloud data. The track prior knowledge includes track straightness, radius of curvature, and track spacing;

[0036] Use low-order polynomials to fit straight line segments, high-order polynomials or specific curve equations to fit curve segments. Define the sum of squared error function E through formulas (5) and (6), determine the track parameters for fitting straight line segments and curve segments through formulas (7) and (8), substitute into formulas (5) and (6) to fit the track centerline of straight lines and curve segments. By using optimization algorithms, make the fitted curvature close to the given prior radius of curvature, and at the same time ensure that the spacing of the fitted curve is consistent with the track design standard;

[0037]

[0038] In the formula, t is a parameter, (x0, y0, z0) is the starting point coordinate of the straight line segment, and a, b, c respectively represent the direction vector components of the straight line in the x, y, z directions;

[0039]

[0040] In the formula, f(t), g(t), h(t) are the quadratic polynomial representations of the x, y, z components;

[0041]

[0042] where t i is the parameter t value corresponding to each point, (x0, y0, z0) is the starting coordinate of the straight line segment, a, b, c respectively represent the direction vector components of the straight line in the x, y, z directions, and f(ti), g(ti), h(ti) are the three-dimensional coordinates of the fitted curve corresponding to the parameter t i under.

[0043] Preferably, the method of extracting the multi-scenario track plane from the three-dimensional point cloud data of the track by minimizing the variance vector includes:

[0044] Using the three-dimensional point cloud data P of the track, the centered point cloud data, and calculating the covariance matrix;

[0045] Based on the covariance matrix, perform eigenvalue decomposition using formula (9), select the normal vector n of the track plane by the method of minimizing the variance vector, and construct the multi-scenario track plane equation using the normal vector n through formula (10);

[0046]

[0047] where λ i is the i-th eigenvalue, v i is the eigenvector corresponding to the i-th eigenvalue, P i =(x i , y i , z i ) is the coordinate of the i-th point, represents the formula for the centered point cloud data, represents the covariance matrix. By performing eigenvalue decomposition on the covariance matrix, the eigenvector corresponding to the smallest eigenvalue is the normal vector n of the plane;

[0048] Construct the multi-scenario track plane equation using the normal vector n through formula (10);

[0049]

[0050] where are the components of the normal vector n.

[0051] Preferably, the construction of the three-dimensional solid track clearance bounding box by combining the track center line and the track plane to achieve the three-dimensional detection of the vehicle clearance and the building clearance includes:

[0052] Using the track center line and the track plane equation, define the width and height of the track clearance bounding box as the safety widths W v , W b and the heights H v , H b, and supports dynamically adjusting the width and height of the track clearance bounding box;

[0053] Using the information of the width and height of the track clearance bounding box, construct the clearance frames of the vehicle clearance and the building clearance for the cross-section perpendicular to the line centerline at any point c(t)=(x(t), y(t), z(t)) on the track centerline through formulas (11) and (12);

[0054]

[0055]

[0056] In the formula, represent the two endpoints of the positive and negative semi-axes in the x-axis direction of the vehicle clearance, represent the two endpoints of the positive and negative semi-axes in the z-axis direction of the vehicle clearance, x(t), z(t) are the coordinates of any point on the track centerline, W v 、W b and H v 、H b are the safety widths and heights of the vehicle clearance and the building clearance;

[0057] Construct a three-dimensional solid track clearance bounding box along the track plane for the clearance frames of the vehicle clearance and the building clearance to realize the three-dimensional detection of the vehicle clearance and the building clearance.

[0058] According to another aspect of the present invention, there is provided a rail transit clearance detection device, including: a depth map generation module, a three-dimensional space coordinate calculation module, a track three-dimensional point cloud data acquisition module, a track centerline acquisition module, and a track clearance detection module;

[0059] The depth map generation module is used to collect the operation environment image data in front of the train through a camera installed at the front end of the train, process the operation environment image data, and generate a depth map;

[0060] The three-dimensional space coordinate calculation module is used to extract the track area in the depth map by using a semantic segmentation algorithm, and calculate the three-dimensional space coordinates of the track according to the track area and the depth map;

[0061] The track three-dimensional point cloud data acquisition module is used to perform convolution optimal transformation processing on the three-dimensional space coordinates of the track by using a track prior constraint equation to form track three-dimensional point cloud data;

[0062] The track centerline acquisition module is used to fit the track centerline of the straight line and the curve segment by minimizing the sum of squared errors based on the track three-dimensional point cloud data and prior knowledge;

[0063] The described track plane acquisition module is used to extract multi-scene track planes from the three-dimensional track point cloud data by means of variance vector minimization;

[0064] The described track clearance detection module is used to construct a three-dimensional solid track clearance bounding box by combining the track center line and the track plane, so as to realize the three-dimensional detection of vehicle clearance and building clearance.

[0065] A non-transitory computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by a processor, the described rail transit clearance detection method is implemented.

[0066] A computer program product includes a computer program. When the computer program runs on one or more processors, it is used to implement the described rail transit clearance detection method.

[0067] An electronic device includes: a processor, a memory, and a computer program; wherein, the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device runs, the processor executes the computer program stored in the memory, so that the electronic device executes and implements the described rail transit clearance detection method.

[0068] As can be seen from the technical solutions provided by the embodiments of the present invention above, the rail transit clearance detection method provided by the present invention is innovative in that it uses a monocular depth estimation algorithm and semantic segmentation technology, combines track prior knowledge, and realizes the rapid detection of vehicle clearance and building clearance by constructing three-dimensional point cloud data and a clearance bounding box. This method not only reduces the detection cost and simplifies the operation process, but also has extremely high adaptability and can achieve efficient detection in different scenarios and track types. Through self-learning and optimization algorithms, the system can continuously improve the detection accuracy, ensure the safe operation of rail transit, and effectively reduce accidents.

[0069] Additional aspects and advantages of the present invention will be given in part in the following description, and these will become apparent from the following description or be understood through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0071] Figure 1 It is a processing flow chart of a rail transit clearance detection method provided by an embodiment of the present invention.

[0072] Figure 2 The schematic diagram of camera imaging coordinate conversion provided by the embodiment of the present invention.

[0073] Figure 3 The schematic diagram of the two-dimensional clearance frames of trains and buildings at any point on the center line in the vertical section.

[0074] Figure 4 The schematic diagram of the bounding boxes of the three-dimensional track vehicle clearance and building clearance provided by the embodiment of the present invention.

[0075] Figure 5 The structural diagram of a rail transit clearance detection device provided by the embodiment of the present invention. Detailed implementation manners

[0076] The following details the implementation manners of the present invention. The examples of the implementation manners are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements with the same or similar functions throughout. The implementation manners described below with reference to the accompanying drawings are exemplary only for explaining the present invention and should not be construed as limiting the present invention.

[0077] Those skilled in the art of the present technology can understand that, unless specifically stated otherwise, the singular forms "a", "an", "the" and "said" used herein may also include the plural forms. It should be further understood that the term "comprising" used in the description of the present invention means the presence of the described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or their groups. It should be understood that when we say an element is "connected" or "coupled" to another element, it can be directly connected or coupled to other elements, or there may also be intermediate elements. In addition, the "connection" or "coupling" used herein may include wireless connection or coupling. The phrase "and / or" used herein includes any and all combinations of one or more of the associated listed items.

[0078] Those skilled in the art of the present technology can understand that, unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meaning as the general understanding of those of ordinary skill in the art to which the present invention belongs. It should also be understood that terms such as those defined in a general dictionary should be understood to have a meaning consistent with the meaning in the context of the prior art and will not be interpreted with an idealized or overly formal meaning unless defined as herein.

[0079] For the convenience of understanding the embodiments of the present invention, the following will further explain with several specific embodiments as examples in conjunction with the accompanying drawings, and each embodiment does not constitute a limitation to the embodiments of the present invention.

[0080] Example 1

[0081] The processing flow of a rail transit clearance detection method provided by an embodiment of the present invention is as Figure 1 shown, including the following processing steps:

[0082] Step S1: Before installing the camera, obtain the internal parameter matrix and distortion coefficient of the camera through calibration plate calibration calculation, and install a single camera or a combination of cameras such as a monocular camera, an infrared camera, a depth camera, and an industrial camera at the central position of the front end of the train. As the train moves forward, collect the running environment image data in front of the train through the cameras installed at the front end of the train. The device coordinate system is set as follows: the coordinates of the camera device are (0, 0, 0), the train running direction is the y-axis, the direction perpendicular to the track plane is the z-axis, and the x-axis is perpendicular to the plane yoz.

[0083] Apply the monocular depth estimation algorithm to process the collected running environment image data in front of the train to generate a depth map.

[0084] Step S2: Use the semantic segmentation algorithm to extract the track area in the depth map, obtain the track depth value according to the above track area and depth map, and calculate the three-dimensional space coordinates of the track based on the track depth value.

[0085] Step S3: Construct a track prior constraint equation, and use the track prior constraint equation to perform convolution optimal transformation processing on the three-dimensional space coordinates of the track to form track three-dimensional point cloud data.

[0086] Step S4: Based on the track three-dimensional point cloud data and prior knowledge, fit the track center line of the straight line and curve segments by the method of minimizing the sum of squared errors.

[0087] Step S5: Extract the multi-scene track plane from the track three-dimensional point cloud data by the method of minimizing the variance vector.

[0088] Step S6: Combine the track center line and the track plane to construct a three-dimensional solid track clearance bounding box to realize the three-dimensional detection of vehicle clearance and building clearance.

[0089] Further, step S1 specifically includes the following steps:

[0090] S11: Before installing the camera, prepare a checkerboard calibration plate with known black and white spacing and that can be photographed by the camera for camera calibration;

[0091] S12: Place the calibration checkerboard on a plane, ensure that the surface of the checkerboard is parallel or perpendicular to the plane, and take multiple calibration plate images at different angles and positions to ensure that the calibration plate covers all regions of the image;

[0092] S13: Detect the corner points in each calibration board image using the findChessboardCorners computer vision algorithm in OpenCV. For the corner points with inaccurate detection, the corner points can be manually marked;

[0093] S14: Use the detected corner points and the actual size of the calibration board, and apply the calibrateCamera camera calibration algorithm in OpenCV to calculate the internal parameter matrix K of the camera, the radial distortion coefficients k1, k2, k3, and the tangential distortion coefficients p1, p2. The internal parameter matrix K includes the focal length (f x , f y ) and the pixel position coordinates (c x , c y ) corresponding to the camera center;

[0094] S15: Install a single camera or a combination of cameras such as a calibrated monocular camera, an infrared camera, a depth camera, and an industrial camera at the central position of the front end of the train. The train moves forward continuously. Through the cameras installed at the front end of the train, collect the image data of the operating environment in front of the train. The device coordinate system is set as: the coordinates of the camera device are (0, 0, 0), the train running direction is the y-axis, the direction perpendicular to the track plane is the z-axis, and the x-axis is perpendicular to the plane yoz;

[0095] S16: Select a monocular estimation algorithm based on the convolutional neural network with the U-Net structure to process the collected image data to obtain the initial depth map D rgb corresponding to each frame of color image I pred ;

[0096] S17: For each frame of depth map D pred and color image I rgb , use the depth completion algorithm based on the Depth CompletionNetwork network to generate a high-precision depth map D completed .

[0097] Furthermore, step S2 specifically includes the following steps:

[0098] S21: Use the semantic segmentation algorithm with the U-Net structure to process the high-precision depth map D completed obtained in step S1, accurately identify and segment the track area, and output the pixel coordinates (u, v) of the track area;

[0099] S22: Obtain the internal parameter matrix and distortion coefficients of the camera obtained in step S1, and use formula (1) to undistort the track pixel coordinates to obtain the corrected pixel coordinates (u c , v c );

[0100]

[0101] In the formula, r is the radial distance representing the distance from the point to the image center, (u, v), (u c , v c ) are the pixel coordinates before and after calibration respectively, (c x , c y ) is the pixel position coordinate corresponding to the camera center, which is a parameter included in the internal parameter matrix K, and k1, k2, k3 and p1, p2 are the radial distortion coefficients and tangential distortion coefficients;

[0102] S23: From the high-precision depth map D obtained in step S1 completed , extract the depth value d corresponding to the track pixel coordinates;

[0103] S24: Using the calibrated pixel coordinates (u c , v c ) and the corresponding depth value d, calculate the three-dimensional space coordinates (X, Y, Z) of the track in the camera coordinate system through formula (2);

[0104]

[0105] S25: Using the image data collected in step S1, perform visual odometry calculation through the method of feature point matching to obtain the external parameter matrix [R|t] of the camera, including the rotation matrix R and the translation vector t;

[0106] S26: Using the external parameter matrix of the camera and the three-dimensional space coordinates (X, Y, Z) in the camera coordinate system obtained in S24, calculate the three-dimensional space coordinates (X w , Y w , Z w ) in the world coordinate system through formula (3). Figure 2 This is a schematic diagram of camera imaging coordinate conversion provided by an embodiment of the present invention;

[0107]

[0108] Further, step S3 specifically includes the following steps:

[0109] S31: Divide the three-dimensional space coordinate point data on both sides of the track into left and right region voxels by means of voxelization;

[0110] S32: Calculate the centroid of the points within each voxel, construct a specific orbital prior constraint equation through formula (4), filter out the voxel regions that do not meet the prior constraints, and perform spatial domain filtering convolution smoothing on the three-dimensional space coordinates using a 3×3 Gaussian convolution kernel. By iteratively executing for each point in the left orbital voxel region that does not meet the prior constraints, find the point with the closest distance in the right orbital voxel region point cloud as its corresponding point, calculate the sum of squared errors of the Euclidean distances between all corresponding point pairs, the rotation and translation transformation matrices, minimize the sum of squared errors, and apply the transformation matrix to a point cloud to update its position to minimize the distance between point clouds to achieve optimal transformation fine alignment processing, forming high-precision orbital three-dimensional point cloud data;

[0111]

[0112] In the formula, Voxel i represents the spatial range of the i-th voxel, |V i | represents the number of points within the voxel, represents the centroid of the left region of the i-th voxel, represents the calculation of the distance from the centroid of the left region of the i-th voxel to the centroid of the right region of the i-th voxel, W 轨 is the known orbital width, and ε is the allowable error range.

[0113] Further, step S4 specifically includes the following steps:

[0114] S41: Utilize the orbital three-dimensional point cloud data obtained in step S3 to acquire the point cloud coordinates and orbital prior knowledge. The orbital prior knowledge includes orbital straightness, curvature radius, and orbital spacing;

[0115] S42: In three-dimensional space, the orbital centerline needs to be described by three coordinates x, y, and z for its geometric shape. For straight segments, during the fitting process of the orbital centerline, the orbit is divided into straight segments and curved segments for separate processing. Use a straight-line equation or a polynomial of lower-order first or second degree to fit the straight-segment orbit using formula (5), and use a high-order polynomial, circular arc, or spline curve equation to fit the curved-segment orbit using formula (6);

[0116]

[0117] In the formula, t is a parameter, (x0, y0, z0) is the starting coordinate of the straight segment, and a, b, c respectively represent the direction vector components of the straight line in the x, y, z directions;

[0118]

[0119] In the formula, f(t), g(t), h(t) are quadratic polynomial representations of the x, y, z components;

[0120] S43: Define the sum of squared error function \(E\), and determine the track parameters of the straight line segment and the curve segment through formulas (7) and (8), and fit the track centerlines of the straight line and the curve segment;

[0121]

[0122] where \(t\) i is the parameter \(t\) value corresponding to each point, \((x_0, y_0, z_0)\) is the starting point coordinates of the straight line segment, \(a\), \(b\), and \(c\) respectively represent the direction vector components of the straight line in the \(x\), \(y\), and \(z\) directions, \(f(t\) i ), \(g(t\) i ), \(h(t\) i ) are the three-dimensional space coordinates of the fitted curve corresponding to the parameter \(t\) i ;

[0123] For the linear straight line fitting problem, use the normal equation of linear algebra to solve it. For the non-linear curve fitting problem, use the gradient descent method or the Newton method iterative optimization algorithm. By using the optimization algorithm, make the curvature of the fit close to the given prior curvature radius, and at the same time ensure that the spacing of the fitted curve is consistent with the track design standard.

[0124] Furthermore, step S5 specifically includes the following steps:

[0125] S51: Use the three-dimensional point cloud data \(P\) of the track obtained in step S3, centralize the point cloud data, and calculate the covariance matrix;

[0126] S52: Based on the covariance matrix, perform eigenvalue decomposition using formula (9), and select the normal vector \(n\) of the track plane by the method of minimizing the variance vector;

[0127]

[0128] where \(\lambda\) i is the \(i\)-th eigenvalue, \(v\) i is the eigenvector corresponding to the \(i\)-th eigenvalue, \(P\) i =(x i , y i , z i ) is the coordinate of the \(i\)-th point, represents the formula for centralizing the point cloud data, represents the covariance matrix;

[0129] Perform eigenvalue decomposition on the covariance matrix, and use the eigenvector corresponding to the smallest eigenvalue obtained from the decomposition as the normal vector \(n\) of the plane, \(v\) iIt represents the eigenvector corresponding to the i-th eigenvalue. By performing eigenvalue decomposition on the covariance matrix, the variance vector is minimized. The eigenvector corresponding to the smallest eigenvalue is the normal vector n of the plane. For example, if there are 5 results calculated for the i-th eigenvalue, there must be a smallest eigenvalue. For instance, if it is the 3rd one, then the eigenvector v3 corresponding to this smallest eigenvalue is the normal vector n we are looking for;

[0130] S53: Use the normal vector n to construct the multi-scenario orbital plane equation through formula (10);

[0131]

[0132] In the formula, are the components of the normal vector n.

[0133] Furthermore, S6 specifically includes the following steps:

[0134] S61: Use the orbital centerline obtained in step S4 and the orbital plane equation constructed in step S5 to define the width and height of the bounding box as the safety widths W v 、W b and height H v 、H b of the vehicle clearance and the building clearance, and support dynamic adjustment to adapt to different track types and scenario requirements;

[0135] S62: Use the information obtained in S61 to construct the bounding boxes of the vehicle clearance and the building clearance of the cross-section perpendicular to the line centerline for any point c(t)=(x(t), y(t), z(t)) on the orbital centerline through formulas (11) and (12). Figure 3 This is a schematic diagram of the bounding boxes of the vehicle clearance and the building clearance of the cross-section perpendicular to any point on the line centerline provided by the embodiment of the present invention.

[0136]

[0137] In the formula, represent the two endpoints of the positive and negative semi-axes in the x-axis direction of the vehicle clearance, represent the two endpoints of the positive and negative semi-axes in the z-axis direction of the vehicle clearance, x(t), z(t) are the coordinates of any point on the orbital centerline, W v 、W b and H v 、H b are the safety widths and heights of the vehicle clearance and the building clearance;

[0138] Construct a three-dimensional orbital clearance bounding box by building the bounding boxes of the vehicle clearance and the building clearance along the orbital plane to achieve three-dimensional detection of the vehicle clearance and the building clearance;

[0139] S63: Figure 4 This is a schematic diagram of the bounding box for the three-dimensional track vehicle clearance and building clearance provided by the embodiments of the present invention. The two-dimensional clearance frame constructed in S62 is used to construct a three-dimensional track clearance bounding box along the track plane to realize the three-dimensional detection of the vehicle clearance and the building clearance. The railway clearance includes the locomotive and rolling stock clearance and the building clearance.

[0140] Embodiment 2

[0141] The structure of a rail transit clearance detection device provided by the embodiments of the present invention is as Figure 5 shown, and it includes the following modules: a depth map generation module 1, a three-dimensional space coordinate calculation module 2, a track three-dimensional point cloud data acquisition module 3, a track center line acquisition module 4, a track plane acquisition module 5, and a track clearance detection module 5.

[0142] Specifically, the above-mentioned depth map generation module 1 is used to set a calibration board with a known black-and-white spacing pattern that can be captured by the camera. Before installing the camera, the internal parameter matrix K, the radial distortion coefficients k1, k2, k3, and the tangential distortion coefficients p1, p2 of the camera are calculated through calibration using the calibration board. The internal parameter matrix K includes the focal lengths (f x , f y ) and the pixel position coordinates (c x , c y ) corresponding to the camera center;

[0143] The calibrated camera is installed at the central position of the front end of the train. As the train moves forward continuously, the running environment image data in front of the train is collected by the camera. The device coordinate system is set as: the coordinates of the camera device are (0, 0, 0), the train running direction is the y-axis, the direction perpendicular to the track plane is the z-axis, and the x-axis is perpendicular to the plane yoz;

[0144] The monocular estimation algorithm based on the convolutional neural network is selected to process the collected running environment image data to obtain the initial depth map D rgb corresponding to each frame of color image I pred ;

[0145] For each frame of depth map D pred and color image I rgb , the depth completion algorithm is used to generate the depth map D completed .

[0146] Specifically, the above-mentioned three-dimensional space coordinate calculation module 2 is used to process the depth map D completed using the semantic segmentation algorithm to segment the track area and output the pixel coordinates (u, v) of the track area;

[0147] Obtain the internal parameter matrix and distortion coefficients of the camera, and use formula (1) to undistort the pixel coordinates (u, v) of the track area to obtain the corrected pixel coordinates (u c , v c );

[0148]

[0149] where r is the radial distance representing the distance from the point to the image center, (u, v), (u c , v c ) are the pixel coordinates before and after correction respectively, (c x , c y ) is the pixel position coordinate corresponding to the camera center, is a parameter included in the internal parameter matrix K, and k1, k2, k3 and p1, p2 are the radial distortion coefficients and tangential distortion coefficients;

[0150] Extract the depth value d corresponding to the pixel coordinates of the track area from the depth map D completed ;

[0151] Use the corrected pixel coordinates (u c , v c ) and the corresponding depth value d, and calculate the three-dimensional space coordinates (X, Y, Z) of the track in the camera coordinate system through formula (2);

[0152]

[0153] Perform visual odometry calculation through feature point matching based on the running environment image data to obtain the external parameter matrix [R|t] of the camera, including the rotation matrix R and the translation vector t;

[0154] Use the external parameter matrix of the camera and the three-dimensional space coordinates (X, Y, Z) in the camera coordinate system to calculate the three-dimensional space coordinates (X w , Y w , Z w ) of the track in the world coordinate system through formula (3);

[0155]

[0156] Specifically, the above-mentioned track three-dimensional point cloud data acquisition module 3 is used to divide the three-dimensional space coordinate point data on both sides of the track into left and right regional voxels through voxelization, obtain the left track voxel area and the right track voxel area, calculate the centroid of the points in each voxel, construct a specific track prior constraint equation through formula (4), filter out the voxel areas that do not meet the prior constraints, and perform convolution optimal transformation smoothing and alignment processing on the three-dimensional space coordinate point data to form the track three-dimensional point cloud data;

[0157]

[0158] In the formula, Voxel i represents the spatial range of the i-th voxel, |V i | represents the number of points inside the voxel, represents the centroid of the left region of the i-th voxel, represents the calculation of the distance from the centroid of the left region of the i-th voxel to the centroid of the right region of the i-th voxel, W 轨 is the known track width, and ε is the allowable error range.

[0159] Specifically, the above-mentioned track centerline acquisition module 4 is used to obtain point cloud coordinates and track prior knowledge by using the track three-dimensional point cloud data, and the track prior knowledge includes track straightness, curvature radius, and track spacing;

[0160] Use a low-order polynomial to fit a straight line segment, a high-order polynomial or a specific curve equation to fit a curve segment. Define the sum of squared error function E through formulas (5) and (6), determine the track parameters of the fitted straight line segment and curve segment through formulas (7) and (8), substitute them into formulas (5) and (6) to fit the track centerline of the straight line and curve segments. By using an optimization algorithm, make the fitted curvature close to the given prior curvature radius, and at the same time ensure that the spacing of the fitted curve is consistent with the track design standard;

[0161]

[0162] In the formula, t is a parameter, (x0, y0, z0) is the starting point coordinate of the straight line segment, and a, b, c respectively represent the direction vector components of the straight line in the x, y, z directions;

[0163]

[0164] In the formula, f(t), g(t), h(t) are quadratic polynomial representations of the x, y, z components;

[0165]

[0166] In the formula, t i is the parameter t value corresponding to each point, (x0, y0, z0) is the starting point coordinate of the straight line segment, a, b, c respectively represent the direction vector components of the straight line in the x, y, z directions, and f(ti), g(ti), h(ti) are the three-dimensional coordinates of the fitted curve corresponding to the parameter t i Specifically, the above-mentioned track plane acquisition module 5 is used to calculate the covariance matrix by using the track three-dimensional point cloud data P and the centered point cloud data;

[0167] Based on the covariance matrix, perform eigenvalue decomposition using formula (9), select the normal vector n of the orbital plane through the method of minimizing the variance vector, and construct the multi-scenario orbital plane equation using the normal vector n through formula (10).

[0168]

[0169] where λ i is the i-th eigenvalue, v i is the eigenvector corresponding to the i-th eigenvalue, P i =(x i , y i , z i ) is the coordinate of the i-th point. Formula for representing the centralized point cloud data represents the covariance matrix. By performing eigenvalue decomposition on the covariance matrix, the eigenvector corresponding to the smallest eigenvalue is the normal vector n of the plane.

[0170] Construct the multi-scenario orbital plane equation using the normal vector n through formula (10).

[0171]

[0172] where are the components of the normal vector n.

[0173] Specifically, the above-mentioned and the track clearance detection module 6 are used to define the width and height of the track clearance bounding box as the safety widths W v , W b and the heights H v , H b of the vehicle clearance and the building clearance using the track center line and the orbital plane equation, and support dynamic adjustment of the width and height of the track clearance bounding box.

[0174] Construct the clearance frames of the vehicle clearance and the building clearance for the cross-section perpendicular to the line center line at any point c(t)=(x(t), y(t), z(t)) on the track center line using the information of the width and height of the track clearance bounding box through formulas (11) and (12).

[0175]

[0176]

[0177] where represent the two endpoints of the positive and negative semi-axes in the x-axis direction of the vehicle clearance, represent the two endpoints of the positive and negative semi-axes in the z-axis direction of the vehicle clearance, and x(t), z(t) are the coordinates of any point on the track center line, Wv , W b and H v , H b are the safety widths and heights of the vehicle clearance and the building clearance;

[0178] Construct a three-dimensional solid track clearance bounding box by building the bounding boxes of the vehicle clearance and the building clearance along the track plane, so as to realize the three-dimensional detection of the vehicle clearance and the building clearance.

[0179] The specific process of using the device of the present invention for rail transit clearance detection is similar to that of the foregoing method embodiment, and will not be elaborated here.

[0180] Embodiment 3

[0181] A non-transitory computer-readable storage medium is used to store computer instructions, and when the computer instructions are executed by a processor, the rail transit clearance detection method described in Embodiment 1 is realized.

[0182] A computer program product includes a computer program, and when the computer program runs on one or more processors, it is used to realize the rail transit clearance detection method described in Embodiment 1.

[0183] An electronic device includes: a processor, a memory, and a computer program; wherein, the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device runs, the processor executes the computer program stored in the memory, so that the electronic device executes the rail transit clearance detection method described in Embodiment 1.

[0184] In summary, by using industrial cameras for data acquisition, compared with traditional detection methods that rely on high-cost devices such as lidar and depth cameras, the present invention significantly reduces the hardware cost, is suitable for the popularization and application of large-scale rail transit systems, and thus significantly reduces the investment in the overall detection system; secondly, by using advanced computer vision technology and artificial intelligence algorithms, the present invention can realize real-time and automated track clearance detection. By using the monocular estimation algorithm and semantic segmentation algorithm in deep learning, a large amount of image data can be quickly processed and analyzed, effectively improving the detection efficiency and reducing the manual operation time and cost; in addition, the present invention combines depth estimation and semantic segmentation technologies, can accurately identify the track area and calculate the three-dimensional space coordinates of the track, and through fine point cloud data alignment and smoothing processing, provides a high-precision track model reconstruction, ensuring the accuracy of clearance detection and the safety of train operation.

[0185] The method of the present invention has strong adaptability, is applicable to various detection environments and track types, can maintain a high detection accuracy rate in complex environments, and can be adjusted according to different requirements to adapt to various track design parameters and clearance standards; in addition, this method simplifies the complexity of equipment calibration and maintenance, and can complete the detection task using a single or combined camera. The flexible camera calibration and installation process enables it to adapt to various train models and configurations, and the operation is convenient; finally, with the help of deep learning and machine learning algorithms, the present invention has self-learning and self-adaptive capabilities. The system can continuously optimize the algorithm according to the collected data, improve the recognition and adaptation capabilities for various scenarios, ensure the detection accuracy and efficiency during long-term use, effectively prevent the collision between the train and obstacles, ensure the operation safety, and reduce the occurrence of accidents.

[0186] Those of ordinary skill in the art can understand that: The drawings are only schematic diagrams of an embodiment, and the modules or processes in the drawings are not necessarily essential for implementing the present invention.

[0187] From the description of the above embodiments, those skilled in the art can clearly understand that the present invention can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present invention, in essence, or the part that makes a contribution to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of the present invention.

[0188] Each embodiment in this specification is described in a progressive manner. The same or similar parts among the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device or system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiments. The device and system embodiments described above are only illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they can be located in one place, or can be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0189] As described above, the above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A rail transit clearance detection method, characterized in that, Including: Collecting the running environment image data in front of the train through a camera installed at the front end of the train, processing the running environment image data to generate a depth map; Extracting the track area in the depth map by using a semantic segmentation algorithm, and calculating the three-dimensional spatial coordinates of the track according to the track area and the depth map; Performing convolution optimal transformation processing on the three-dimensional spatial coordinates of the track by using a track prior constraint equation to form track three-dimensional point cloud data; Based on the track three-dimensional point cloud data and prior knowledge, fitting the track center lines of straight lines and curve segments by minimizing the sum of squared errors; Extracting multi-scene track planes from the track three-dimensional point cloud data by the method of minimizing the variance vector; Constructing a three-dimensional solid track clearance bounding box by combining the track center line and the track plane to realize three-dimensional detection of vehicle clearance and building clearance; The performing convolution optimal transformation processing on the three-dimensional spatial coordinates of the track by using a track prior constraint equation to form track three-dimensional point cloud data includes: Dividing the three-dimensional spatial coordinate point data in the world coordinate system on both sides of the track into left and right regional voxels by a voxelization method to obtain a left track voxel region and a right track voxel region, calculating the centroid of the points in each voxel, constructing a specific track prior constraint equation through formula (1), filtering out the voxel regions that do not conform to the prior constraint, and performing convolution optimal transformation smoothing and alignment processing on the three-dimensional spatial coordinate point data to form track three-dimensional point cloud data; wherein, V i represents the spatial range of the i-th voxel, |V i | represents the number of points within the voxel, represents the centroid of the left region of the i-th voxel, represents the calculation of the distance from the centroid of the left region of the i-th voxel to the centroid of the right region of the i-th voxel, W 轨 is the known track width, and ε is the allowable error range.

2. The method according to claim 1, characterized in that, The collecting the running environment image data in front of the train through a camera installed at the front end of the train, processing the running environment image data to generate a depth map includes: Set up a calibration board with a checkerboard pattern whose black and white spacing is known and can be captured by the camera. Before installing the camera, use the calibration board to calibrate and calculate to obtain the camera internal parameter matrix K, radial distortion coefficients k1, k2, k3, and tangential distortion coefficients p1, p2. The internal parameter matrix K includes the focal length (f x , f y ) and the pixel position coordinates corresponding to the camera center (c x , c y ); Installing the calibrated camera at the central position of the front end of the train. The train moves forward continuously. Collecting the running environment image data in front of the train through the camera. The device coordinate system is set as: the coordinate of the camera device is (0, 0, 0), the train running direction is the y-axis, the direction perpendicular to the track plane is the z-axis, and the x-axis is perpendicular to the plane yoz; Select a monocular estimation algorithm based on a convolutional neural network to process the collected image data of the operating environment, and obtain each frame of color image I rgb The corresponding initial depth map D pred ; For each frame of depth map D pred and color image I rgb , a depth completion algorithm is used to generate depth map D completed .

3. The method according to claim 2, wherein The extracting the track area in the depth map by using a semantic segmentation algorithm, and calculating the three-dimensional spatial coordinates of the track according to the track area and the depth map includes: Process the depth map D using a semantic segmentation algorithm completed to segment the track area and output the pixel coordinates (u, v) of the track area; Obtain the intrinsic matrix and distortion coefficients of the camera, and use formula (2) to undistort the pixel coordinates (u, v) of the track area to obtain the corrected pixel coordinates (u c , v c ); where r is the radial distance, representing the distance from the point to the image center, (u, v), (u c , v c ) are the pixel coordinates before and after calibration respectively, (c x , c y ) is the pixel position coordinate corresponding to the camera center, which is a parameter included in the internal parameter matrix K, and k1, k2, k3 and p1, p2 are the radial distortion coefficients and tangential distortion coefficients; Extract the depth value d corresponding to the pixel coordinates of the track area from the depth map D completed ; Using the calibrated pixel coordinates (u c , v c ) and the corresponding depth value d, the three-dimensional spatial coordinates (X, Y, Z) of the track in the camera coordinate system are calculated through Equation (3); Performing visual odometry calculation by the method of feature point matching according to the running environment image data to obtain the external parameter matrix [R|t] of the camera, including the rotation matrix R and the translation vector t; Calculate the three-dimensional spatial coordinates (X w , Y w , Z w ) of the orbit in the world coordinate system by using the external camera parameter matrix and the three-dimensional spatial coordinates (X, Y, Z) in the camera coordinate system through formula (4); 4. The method according to claim 1, wherein The fitting the track center lines of straight lines and curve segments by minimizing the sum of squared errors based on the track three-dimensional point cloud data and prior knowledge includes: Obtaining the point cloud coordinates and track prior knowledge by using the track three-dimensional point cloud data. The track prior knowledge includes track straightness, curvature radius and track spacing; Use a low-order polynomial to fit a straight line segment, a high-order polynomial or a curve equation to fit a curve segment. Define the function \(E\) that minimizes the sum of squared errors through formulas (5) and (6). Determine the track parameters of the fitted straight line segment and curve segment through formulas (7) and (8), substitute them into formulas (5) and (6) to fit the track centerlines of the straight line and curve segments. By using an optimization algorithm, make the curvature of the fit close to the given prior curvature radius, and at the same time ensure that the spacing of the fitted curve is consistent with the track design standard; In the formula, \(\tau\) is the curve parameter, \((x_0,y_0,z_0)\) is the starting point coordinates of the straight line segment, and \(a\), \(b\), \(c\) respectively represent the direction vector components of the straight line in the \(x\), \(y\), \(z\) direction vectors; In the formula, \(f(\tau)\), \(g(\tau)\), \(h(\tau)\) are the quadratic polynomial representations of the \(x\), \(y\), \(z\) direction vectors; where τ i is the parameter τ value corresponding to each point, (x0, y0, z0) is the starting coordinate of the line segment, and a, b, c respectively represent the direction vector components of the line in the x, y, and z directions. f(τ i ), g(τ i ), h(τi) are the three-dimensional coordinates of the fitting curve corresponding to the parameter τ i .

5. The method according to claim 4, wherein The method of extracting multi-scene track planes from the three-dimensional point cloud data of the track by minimizing the variance vector includes: Use the three-dimensional point cloud data \(P\) of the track, centralize the point cloud data, and calculate the covariance matrix; Based on the covariance matrix, perform eigenvalue decomposition through formula (9). Select the normal vector \(n\) of the track plane by the method of minimizing the variance vector, and construct the multi-scene track plane equation through formula (10) using the normal vector \(n\); where λ i is the i-th eigenvalue, v i is the eigenvector corresponding to the i-th eigenvalue, P i =(x i , y i , z i ) is the coordinate of the i-th point, represents the formula for centering the point cloud data, represents the covariance matrix. By performing eigenvalue decomposition on the covariance matrix, the eigenvector corresponding to the smallest eigenvalue is the normal vector n of the plane; Construct the multi-scene track plane equation through formula (10) using the normal vector \(n\); In the formula, is the component of the normal vector n.

6. The method according to claim 5, characterized in that The method of constructing a three-dimensional solid track clearance bounding box by combining the track centerline and the track plane to realize the three-dimensional detection of vehicle clearance and building clearance includes: Using the track center line and the track plane equation, define the width and height of the track limit bounding box as the safety widths W v , W b and the height H v , H b , and support dynamically adjusting the width and height of the track limit bounding box; Use the information of the width and height of the track clearance bounding box to construct the clearance frames of vehicle clearance and building clearance for any point \(c(t)=(x(t),y(t),z(t))\) on the track centerline through formulas (11) and (12) for the cross-section perpendicular to the line centerline; In the formula, represent the two endpoints of the positive and negative half-axes of the vehicle gauge in the x-axis direction, represent the two endpoints of the positive and negative half-axes of the vehicle gauge in the z-axis direction, x(t), z(t) are the coordinates of any point on the track center line, W v 、W b and H v 、H b are the safety widths and heights of the vehicle gauge and the building gauge; Construct a three-dimensional solid track clearance bounding box by the clearance frames of vehicle clearance and building clearance along the track plane to realize the three-dimensional detection of vehicle clearance and building clearance.

7. A track traffic clearance detection device, characterized in that, Applicable to the rail transit clearance detection method described in any one of claims 1 to 6, including: a depth map generation module, a three-dimensional space coordinate calculation module, a three-dimensional point cloud data acquisition module of the track, a track centerline acquisition module, a track plane acquisition module, and a track clearance detection module; The depth map generation module is used to collect the running environment image data in front of the train through a camera installed at the front end of the train, and process the running environment image data to generate a depth map; The three-dimensional space coordinate calculation module is used to extract the track area in the depth map by using a semantic segmentation algorithm, and calculate the three-dimensional space coordinates of the track according to the track area and the depth map; The three-dimensional point cloud data acquisition module of the track is used to perform convolution optimal transformation processing on the three-dimensional space coordinates of the track by using the track prior constraint equation to form the three-dimensional point cloud data of the track; The track centerline acquisition module is used to fit the track centerlines of straight lines and curve segments by minimizing the sum of squared errors based on the three-dimensional point cloud data of the track and prior knowledge; The track plane acquisition module is used to extract multi-scene track planes from the three-dimensional point cloud data of the track by the method of minimizing the variance vector; The described track clearance detection module is used to construct a three-dimensional solid track clearance bounding box by combining the track center line and the track plane, and realize the three-dimensional detection of vehicle clearance and building clearance.

8. A non-transitory computer-readable storage medium, characterized in that, The described non-transitory computer-readable storage medium is used to store computer instructions, and when the computer instructions are executed by a processor, the rail transit clearance detection method described in any one of claims 1-6 is realized.

9. A computer program product, characterized in that, It includes a computer program, and when the computer program runs on one or more processors, it is used to realize the rail transit clearance detection method described in any one of claims 1-6; An electronic device, comprising: A processor, a memory, and a computer program; wherein, the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device runs, the processor executes the computer program stored in the memory, so that the electronic device executes and realizes the rail transit clearance detection method described in any one of claims 1-6.

Images