Intelligent detection and positioning method for track irregularities and defects

Abstract

In view of the problems of the track irregularity and disease monitoring system on the market, such as the need for manual cooperation, low intelligence and integration, poor dynamic monitoring and sensing effect, high labor cost and the like, the present application discloses a track irregularity and disease intelligent detection and positioning method, which collects and analyzes the data collected by the three-camera camera and GPS sensor installed on the track inspection vehicle, on the one hand, the track is three-dimensionally reconstructed and the track irregularities such as triangular pits, high and low and level are detected and positioned through the elevation analysis of the point cloud; on the other hand, the image target deep learning detection model is used to detect and position the diseases such as peeling, fish scale injury, layer cracking and wave abrasion on the track, so as to realize the accurate and rapid detection and analysis of the track irregularity and disease, and solve the problem of rapid detection and positioning of track diseases during the operation of the track inspection vehicle. Compared with manual detection, the present application is more reliable, efficient and safe.

Description

Technical Field

[0001] This invention belongs to the field of integrated technology of image processing, three-dimensional reconstruction and deep networks, and specifically relates to a method for intelligent detection and localization of track irregularities and defects. Background Technology

[0002] Safety is always a top priority in railway operations. Because high-speed trains travel at high speeds, even minor accidents can have incalculable and serious consequences. A crucial guarantee for high-speed train operation is a highly smooth track. However, over long-term operation, the increasing frequency and magnitude of impacts from high-speed trains on the track structure inevitably affect the smoothness of the high-speed railway track, leading to safety hazards. Furthermore, rail surface damage and fractures caused by natural disasters and usage also pose significant risks to railway track safety. Therefore, detecting and locating track defects and irregularities is a core task that high-speed railway maintenance departments must address.

[0003] Currently, in railway track inspection, the most commonly used method is dynamic inspection using large track inspection vehicles, which is only conducted monthly or quarterly. This method is costly, inefficient, and requires professional personnel to operate. Static inspection, using manual labor or track geometry detectors, is low-cost and simple to operate, but is easily affected by environmental factors. The construction and maintenance of the three-level control network is costly, with a long construction period, large workload, short maintenance windows, and low measurement efficiency. Building a network for low-speed railways is uneconomical. Therefore, there is an urgent need for a new inspection solution with high measurement accuracy and fast operation speed.

[0004] Current rail irregularity and defect monitoring systems on the market suffer from problems such as the need for manual collaboration, low intelligence and integration, poor dynamic monitoring and sensing effects, and high labor costs. Summary of the Invention

[0005] The purpose of this invention is to provide an intelligent detection and location method for track irregularities and defects, which can achieve accurate and rapid detection and analysis of track defects.

[0006] The technical solution adopted in this invention is an intelligent detection and location method for track irregularities and defects, specifically including the following steps:

[0007] Step 1: Set up a data positioning system and a data acquisition system on the T-shaped track inspection vehicle, and use the data positioning system to collect the position information of the track inspection vehicle in the WGS84 coordinate system {(P t B t )}, where P t and B t Here, t represents the longitude and latitude of the track inspection vehicle at time t, respectively, 1≤t≤T, where T is the total collection time and also the number of data points collected;

[0008] Step 2: On the moving track inspection vehicle, a trinocular camera is used to acquire a sequence of track images at time t from three viewpoints. The left, middle, and right viewpoint images of the orbit at time t are respectively, and the orbit image sequences are stored respectively.

[0009] Step 3: Using the orbital image sequence as input, generate a 3D point cloud of the orbital scene at time t using a 3D point cloud generation method. Simultaneously obtain Projection relationship matrix to images at each viewpoint and The homography matrix between the center viewpoint image and the left and right viewpoint images is also obtained. and

[0010] Step 4: Using the mid-viewpoint orbital image at time t. As input, the two side lines of the left track are extracted using a straight line extraction method. and the two side lines of the right track extract and midline exist A point set is formed by uniformly sampling K points. Multiply by the homography matrix between the center viewpoint and the left viewpoint images Obtain the left viewpoint image The set of projection points on Set of points Projection relation matrix and As input, the open-source OpenCV-Python library function `cv2.triangulatePoints()` is called, and the output is... The corresponding 3D coordinate set Extracting Lr t 1 and Lr t 2 The midline Lr t m In Lr t m A point set {pr} is formed by uniformly sampling K points. t m,1 ,…,pr t m,k ,…,pr t m,K}, using the same method, we obtain the point set {pr t m,1 ,…,pr t m,k ,…,prt m,K} corresponding 3D coordinates

[0011] Step 5: Denote the track width variable as W. s Extract from set Centered on each point, with W s A point cloud within a cube with side length [length] that belongs to the scene's 3D point cloud All points within the range constitute a point set. in Therefore The set of points extracted from the center point of the cube. for The nth k N points k for The number of interior points They are respectively X, Y, Z coordinates The initial point cloud for the left track is obtained; the initial point cloud for the right track is obtained in the same way.

[0012] Step 6: The track elevation variable is denoted as H. s The initial point cloud of the left track is height-filtered, and the point cloud of the right track is height-filtered in the same way; the updated point clouds of the left and right tracks obtained after filtering are as follows: and

[0013] Step 7: Analyze the left and right orbital point clouds at time t. and {SPr t 1 ,…,SPr t k ,…,SPr t K Data analysis is performed to determine if there are any irregularities in the track. If irregularities are found, they are categorized as type C. t Unevenness parameter PS t and uneven position P t ns Record it;

[0014] Step 8: Calculate the location of the irregularity P t ns Corresponding GPS location

[0015] Step 9: Using the collected track image data, create a track defect dataset and train a track delamination, fish scale damage, delamination, and corrugation target detector based on the YOLOv7 network structure to obtain the track defect detection model. ill ;

[0016] Step 10: Obtain the orbital image at time t. and disease detection model ill As input, the detection program train.py, located in the Project / yolov7 directory within a Python environment, is called, and the output shows the disease. The position in the middle and categories Will Multiply by homography matrix get Projection point on the left viewpoint image by and And the projection matrix from 3D to 2D and As input, the open-source OpenCV-Python library function `cv2.triangulatePoints()` is called, and the output is... Corresponding 3D coordinates P t ill ;

[0017] Step 11: Calculate the defect location P obtained by the track inspection vehicle at time t using the same method as in Step 8. t ill GPS location corresponding to the 3D coordinates

[0018] Step 12: Output the type of irregularity C detected by the track inspection car at time t. t Unevenness parameter PS t Uneven GPS location The type of defect detected by the output track inspection car at time t and GPS location

[0019] The invention is further characterized by:

[0020] The specific steps for step 4 are as follows:

[0021] Step 4.1: Obtain the track image As input, the OpenCV-Python library function `cv2.imread()` is called. Convert to grayscale

[0022] Step 4.2: Convert the grayscale image As input, the cv2.GaussianBlur() function from the OpenCV-Python library in the Python programming language is called. Perform Gaussian filtering to obtain the filtered image.

[0023] Step 4.3: Using the filtered image For input, the OpenCV-Python library function `cv2.Canny()` is called. Edge detection processing is performed to obtain the edge image.

[0024] Step 4.4: Using edge images For input, the OpenCV-Python library function `cv2.HoughLinesP()` is called to... The track lines were extracted to obtain the left and right sidelines of the left track. and the left and right side lines of the right track. Lr t 2 The four sides are represented by their two endpoints as follows: in, Indicates the endpoint that is closer to the camera. Indicates the endpoint that is farther from the camera;

[0025] Step 4.5: Calculate the centerline of the left track edge. calculate and Midpoint of the two endpoints calculate and Midpoint of the two endpoints by and As input, the line extraction algorithm getLineEquation() is called to obtain the center line of the two tracks on the left track. The parameters (a, b, c) of the general form of the equation of a straight line are the median. Represented as ax + by + c = 0; the center line Lr of the two tracks on the right is obtained using the same method. t m ;

[0026] Step 4.6, in A point set is formed by uniformly sampling K points.

[0027] Step 4.7: Set the points The points within are multiplied by the homography matrix. Obtain its image from the left viewpoint. The projection point set in

[0028] Step 4.8: Set the points on the mid-viewpoint image. The set of projection points on the left viewpoint image Projection relation matrix As input, the open-source OpenCV-Python library function `cv2.triangulatePoints()` is called, and the output is the set of 3D coordinates of the left track.

[0029] Step 4.9, in Lr t m A point set {pr} is formed by uniformly sampling K points. t m,1 ,…,pr t m,k ,…,pr t m,K Repeat steps 4.7-4.8 to output the point set {pr}. t m,1 ,…,pr t m,k ,…,pr t m,K The corresponding 3D coordinate set {Pr} t 1 ,…,Pr t k ,…,Pr t K}

[0030] The height filtering process for the initial point cloud of the left track in step 6 is as follows: If its Z coordinate belongs to [H s -△ h H s +△ h If it is a string, keep it; otherwise, remove it from the set. Delete △ h This represents the allowable variation in the elevation of the orbital point cloud.

[0031] The specific steps for step 7 are as follows:

[0032] Step 7.1: Calculate the point cloud of the left orbit. Each subset of points The mean coordinates of the left orbital point cloud are obtained by taking the mean coordinates of the subset of points, where k represents the position of the subset of points. in For point set Elevation information of interior point clouds, i.e. The elevation set of the left orbital point cloud is then represented as: The same method was used to obtain the point cloud coordinate set {pSPr} after right-track meanization. t 1 ,…,pSPr t k ,…,pSPr t K The elevation set {Hr} of the right orbital point cloud t 1 ,...,Hr t k ,...,Hr t K};

[0033] Step 7.2: Determine whether there are triangular irregularities in the track. If triangular irregularities exist, record the irregularity parameters and locations.

[0034] Step 7.3: Determine if there are any unevennesses in the track. If unevennesses exist, record the unevenness parameters and locations.

[0035] Step 7.4: Determine if there is any horizontal unevenness in the track. If horizontal unevenness exists, record the unevenness parameters and location.

[0036] Step 7.2 is as follows:

[0037] Step 7.2.1: Calculate the point cloud elevation difference at the parallel positions of the left and right tracks, using the following formula: Get the set

[0038] Step 7.2.2, if Meet the conditions or conditions This indicates that at time t during the movement of the track vehicle, there exists a track triangular irregularity, i.e., irregularity type C. t It equals 1, where ε h This refers to the allowable deviation accuracy for the unevenness of the triangular dent. Record the parameters of the unevenness of the triangular dent. The location of the unevenness and defects in the triangular pit is

[0039]

[0040] Step 7.3 is as follows:

[0041] Step 7.3.1: Calculate the elements in the left and right track elevation sets and the standard elevation H respectively. h The absolute difference between them is denoted as the set of left and right elevation differences. and in △Hr tk =|Hr t k -H h |;

[0042] Step 7.3.2 If the conditions are met This indicates that at time t, when the track car is moving, there is an unevenness on the left track, i.e., unevenness type C. t Equal to 2, record the parameter for uneven height. and uneven locations of the disease If the condition △Hr is satisfied t k ≥δ h This indicates that at time t during the movement of the track vehicle, there is an unevenness on the right track, i.e., unevenness type C. t Equal to 2, record the unevenness parameter PS. t =△Hr t k and uneven disease location P t ns =pSPr t k , where δ h To account for the allowable deviation accuracy due to unevenness.

[0043] Step 7.4 is as follows:

[0044] Step 7.4.1: Calculate the absolute difference in elevation between the left and right tracks. The calculation formula is as follows: Obtain the set of absolute differences in elevation of the orbits

[0045] Step 7.4.2 If the conditions are met This indicates that at time t, when the track vehicle is moving, there is a horizontal irregularity on the track, i.e., irregularity type C. t Equal to 3, record the horizontal unevenness parameter. and unevenness of the disease location Where η h To allow for horizontal unevenness accuracy.

[0046] The specific steps for step 8 are as follows:

[0047] Step 8.1 Calculate the irregularity position P of the track inspection car at time t. t The set of point cloud coordinates The mean coordinates (x) of all points t ,y t ,z t According to the transformation relationship from the spatial rectangular coordinate system to the geodetic coordinate system, (x)t ,y t ,z t () represents the relative coordinates in the geodetic coordinate system.

[0048] Step 8.2: Position the track inspection vehicle (P) at time t using the data from the positioning system. t B t The GPS absolute position of the track is used as the absolute GPS position of the track. The sum of this absolute GPS position and the relative coordinates of the track irregularity is then used to obtain the absolute GPS position of the track irregularity. Where P t ns =P t +wP t ns ,

[0049] The specific steps in step 9 are as follows:

[0050] Step 9.1: The YOLOv7 network structure for target detection is selected for track defect detection network structure, and its configuration file yolov7.yaml is modified as follows: the number of classifications is changed to the number of track defect classifications C. ill That is, set the category quantity variable nc to C. ill The value is 4;

[0051] Step 9.2: Acquire images of Class C track defects. The total number of images acquired is N. s The number of images captured for each class is

[0052] Step 9.3: The training samples are constructed as follows: Upload the track defect image data to the data annotation website, and use the annotation tool on the website to annotate the defect type and defect detection ground truth boxes in the track defect images; After all images are annotated, use the website's dataset export function to export the annotated dataset to the file storage path data / train, create a dataset configuration file data.yaml in the path data / , and combine the annotated dataset and the configuration file to form the track defect training dataset railData;

[0053] Step 9.4: Set the network training parameters in the network training file train.py: total number of iterations epoch = 100, number of images in the network batch = 16, initial learning rate lr = 0.01, input image size W×H, optimizer used for network training set to Adam, initial value of the network training iteration variable step = 1, training sample data path is data / train, training device is set to decive = 0 to indicate GPU training, and the output model name is yolo7.pt;

[0054] Step 9.5: Define the network loss function as follows:

[0055]

[0056] Where x n y represents the score for predicting the nth sample as a positive example. n Let δ represent the label of the nth sample, and δ represent the sigmoid function.

[0057] Step 9.6: Run the file train.py in the conda environment to train the network. Training should be performed when the number of training iterations (step) is greater than or equal to 200 or the network loss function (Loss) is less than 10. -1 At that time, the network training ends, and the orbital defect detection model is output. ill .

[0058] The specific process of step 10 is as follows:

[0059] Step 10.1: Detect the image and disease detection model ill As input, run the test file test.py in a Python environment, and output the detected disease types. The rectangle where the disease is located in For diseases in Middle position The coordinates of the center point, and These represent the width and height of the rectangular detection box, respectively.

[0060] Step 10.2: According to formula (6), use the homography matrix Image Location of track defects Projected image to the left viewpoint The location of the lesion on the left viewpoint image is obtained from the above.

[0061]

[0062] Step 10.3: Position the disease in the intermediate viewpoint image. Location of the disease in the left viewpoint image 3D to 2D projection matrix As input, the open-source OpenCV-Python library function `cv2.triangulatePoints()` is called to output the 3D coordinates of the track defects.

[0063] The beneficial effects of this invention are:

[0064] This invention provides an intelligent detection and location method for track defects and irregularities. It performs track defect detection, track 3D coordinate reconstruction, and track irregularity geometric parameter measurement by moving and measuring simultaneously, thereby achieving automatic intelligent track detection. Compared with manual detection, it is more reliable, efficient, and safe, and can achieve accurate and rapid detection and analysis of track defects and irregularities. Attached Figure Description

[0065] Figure 1 This is a schematic diagram of the intelligent detection and location method for track irregularities and defects of the present invention;

[0066] Figure 2 This is a schematic diagram of the line extraction method based on Hough transform used in this invention;

[0067] Figure 3 This is a schematic diagram of the process for detecting track irregularities using a track inspection vehicle employed in this invention.

[0068] Figure 4 This is a schematic diagram illustrating the principle of using a track inspection vehicle to detect track triangular pits and unevenness in this invention.

[0069] Figure 5 This is a schematic diagram illustrating the principle of the track inspection vehicle used in this invention for detecting track unevenness.

[0070] Figure 6 This is a schematic diagram illustrating the principle of the track inspection vehicle used in this invention for detecting track level unevenness defects;

[0071] Figure 7 This is a schematic diagram of the process for solving the actual GPS location of track irregularities and track surface defects used in this invention.

[0072] Figure 8 These are track defect detection images in an embodiment of the present invention. Detailed Implementation

[0073] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0074] Deep learning theory has found new applications in various industries, and it can also be applied to track defect detection. By utilizing the advanced feature extraction mechanism and category learning capability of deep learning networks, track defects can be used as specific targets to train the detection model, thereby enabling rapid detection and location of track defects during the operation of the track inspection vehicle.

[0075] In addition, by using the absolute position of the track inspection vehicle's GPS and the relative position of the point cloud coordinates to calculate the absolute position of the track irregularities and defects, accurate positioning information can be provided to railway maintenance and repair departments.

[0076] Based on this, the intelligent detection and location method for track irregularities and defects of the present invention specifically includes the following steps:

[0077] Step 1: Set up a data positioning system and a data acquisition system on the T-type track inspection vehicle. The equipment used includes: GPS, a three-lens camera, an industrial-grade laptop, and a power bank. The data positioning system is used to collect the position information of the track inspection vehicle in the WGS84 coordinate system. t B t )}, where P t and B t Here, t represents the longitude and latitude of the track inspection vehicle at time t, respectively, 1≤t≤T, where T is the total collection time and also the number of data points collected;

[0078] Step 2: On the moving track inspection vehicle, a trinocular camera is used to acquire a sequence of track images at time t from three viewpoints. Images of the orbit at time t, representing the left, middle, and right viewpoints respectively, and... The files are stored in the specified drive letter (e.g., drive D) of the industrial-grade laptop, at the following paths: data / ill / left, data / ill / middle, and data / ill / right.

[0079] Step 3: Using the orbital image sequence as input, generate a 3D point cloud of the orbital scene at time t using the 3D point cloud generation method (OpenMVG+PMVS). Simultaneously obtain Projection relationship matrix to images at each viewpoint and The homography matrix between the center viewpoint image and the left and right viewpoint images is also obtained. and

[0080] Step 4: Using the mid-viewpoint orbital image at time t. As input, the two side lines of the left track are extracted using a straight line extraction method. and the two side lines of the right track extract and midline exist A point set is formed by uniformly sampling K points. Multiply by the homography matrix between the center viewpoint and the left viewpoint images Obtain the left viewpoint image The set of projection points on Set of points Projection relation matrix and As input, the open-source OpenCV-Python library function `cv2.triangulatePoints()` is called, and the output is... The corresponding 3D coordinate set Extracting Lr t 1 and Lr t 2 midline exist A point set {pr} is formed by uniformly sampling K points. t m,1 ,…,pr t m,k ,…,pr t m,K}, using the same method, we obtain the point set {pr t m,1 ,…,pr t m,k ,…,pr t m,K} corresponding 3D coordinates The specific steps are as follows:

[0081] Step 4.1: Obtain the track image As input, the OpenCV-Python library function `cv2.imread()` is called. Convert to grayscale

[0082] Step 4.2: Convert the grayscale image As input, the cv2.GaussianBlur() function from the OpenCV-Python library in the Python programming language is called. Perform Gaussian filtering to obtain the filtered image.

[0083] Step 4.3: Using the filtered image For input, the OpenCV-Python library function `cv2.Canny()` is called. Edge detection processing is performed to obtain the edge image.

[0084] Step 4.4: Using edge images For input, the OpenCV-Python library function `cv2.HoughLinesP()` is called to... The track lines were extracted to obtain the left and right sidelines of the left track. And the left and right sidelines Lr of the right track t 1 、Lr t 2 The four sides are represented by their two endpoints as follows: in, Indicates the endpoint that is closer to the camera. Indicates the endpoint that is farther from the camera;

[0085] Step 4.5: Calculate the centerline of the left track edge. calculate and Midpoint of the two endpoints calculate and Midpoint of the two endpoints With point and As input, the line extraction algorithm getLineEquation() is called to obtain the center line of the two tracks on the left track. The parameters (a, b, c) of the general form of the equation of a straight line are the median. Represented as ax + by + c = 0; the center line Lr of the two tracks on the right is obtained using the same method. t m The process of the line extraction algorithm used is as follows: Figure 2 As shown;

[0086] The pseudocode for the line extraction algorithm getLineEquation(), which takes points P1 and P2 as input, can be represented as:

[0087]

[0088] Step 4.6, in A point set is formed by uniformly sampling K points.

[0089] Step 4.7: Set the points The points within are multiplied by the homography matrix. Obtain its image from the left viewpoint. The projection point set in

[0090] Step 4.8: Set the points on the mid-viewpoint image. The set of projection points on the left viewpoint image Projection relation matrix As input, the open-source OpenCV-Python library function `cv2.triangulatePoints()` is called, and the output is the set of 3D coordinates of the left track.

[0091] Step 4.9, in A point set {pr} is formed by uniformly sampling K points. t m,1 ,…,pr t m,k ,…,pr t m,K Repeat steps 4.7-4.8 to output the point set {pr}. t m,1 ,…,pr t m,k ,…,pr t m,K The corresponding 3D coordinate set

[0092] Step 5: Denote the track width variable as W. s Extract from set Centered on each point, with W s A point cloud within a cube with side length [length] that belongs to the scene's 3D point cloud All points within the range constitute a point set. in Therefore The set of points extracted from the center point of the cube. for The nth k N points k for The number of interior points They are respectively X, Y, Z coordinates The initial point cloud for the left track is obtained; the initial point cloud for the right track is obtained in the same way.

[0093] Step 6: The track elevation variable is denoted as H. s The initial point cloud of the left orbit is height filtered: If its Z coordinate belongs to [H s -△ h H s +△ h If it is a string, keep it; otherwise, remove it from the set. Delete △ h To determine the allowable elevation variation value for the orbital point cloud, the same method is used to perform elevation filtering on the right orbital point cloud; the updated left and right orbital point clouds obtained after filtering are as follows: and

[0094] Step 7: Analyze the left and right orbital point clouds at time t. and Data analysis is performed to determine if there are any irregularities in the track. If irregularities are found, they are categorized as type C. t Unevenness parameter PS t and uneven position P t ns Record, such as Figure 3 As shown; the specific steps are as follows:

[0095] Step 7.1: Calculate the point cloud of the left orbit. Each subset of points The mean coordinates of the left orbital point cloud are obtained by taking the mean coordinates of the subset of points, where k represents the position of the subset of points. in

[0096] For point set Elevation information of interior point clouds, i.e. The elevation set of the left orbital point cloud is then represented as: The same method was used to obtain the set of point cloud coordinates after right-track meanization. Elevation set of right orbital point clouds

[0097] Step 7.2: Determine if there are triangular irregularities in the track. If triangular irregularities are found, record the irregularity parameters and locations. The specific process is as follows:

[0098] Step 7.2.1: Calculate the point cloud elevation difference at the parallel positions of the left and right tracks, using the following formula: Get the set

[0099] Step 7.2.2, as follows Figure 4 As shown, if Meet the conditions or conditions This indicates that at time t during the movement of the track vehicle, there exists a track triangular irregularity, i.e., irregularity type C. t It equals 1, where ε h The allowable deviation accuracy for the unevenness of the triangular groove is set at 5mm. Record the unevenness parameters of the triangular groove. Uneven location of the triangular pit and other defects

[0100] Step 7.3: Determine if there are any unevennesses in the track. If unevennesses exist, record the unevenness parameters and locations. The specific process is as follows:

[0101] Step 7.3.1: Calculate the elements in the left and right track elevation sets and the standard elevation H respectively. h The absolute difference between them is denoted as the set of left and right elevation differences. and in △Hr t k =|Hr t k -H h |;

[0102] Step 7.3.2, as follows Figure 5 As shown, If the conditions are met This indicates that at time t, when the track car is moving, there is an unevenness on the left track, i.e., unevenness type C. t Equal to 2, record the parameter for uneven height. and uneven locations of the disease If the condition △Hr is satisfied t k ≥δ h This indicates that at time t during the movement of the track vehicle, there is an unevenness on the right track, i.e., unevenness type C. t Equal to 2, record the unevenness parameter PS. t =△Hr t k and uneven disease location P t ns =pSPr t k , where δ h The allowable deviation accuracy for unevenness is set at 6mm.

[0103] Step 7.4: Determine if there are any horizontal irregularities in the track. If horizontal irregularities exist, record the irregularity parameters and locations. The specific process is as follows:

[0104] Step 7.4.1: Calculate the absolute difference in elevation between the left and right tracks. The calculation formula is as follows: Obtain the set of absolute differences in elevation of the orbits

[0105] Step 7.4.2, as follows Figure 6 As shown, If the conditions are met This indicates that at time t, when the track vehicle is moving, there is a horizontal irregularity on the track, i.e., irregularity type C. t Equal to 3, record the horizontal unevenness parameter. and unevenness of the disease location Where η h The allowable deviation accuracy for horizontal unevenness is set at 6 mm.

[0106] Step 8: Calculate the location of the irregularity P t ns Corresponding GPS location like Figure 7 As shown; the specific steps are as follows:

[0107] Step 8.1 Calculate the irregularity position P of the track inspection car at time t. t The set of point cloud coordinates The mean coordinates (x) of all points t ,y t ,z t According to the transformation relationship from the spatial rectangular coordinate system to the geodetic coordinate system, (x) t ,y t ,z t () represents the relative coordinates in the geodetic coordinate system.

[0108] Step 8.2: Position the track inspection vehicle (P) at time t using the data from the positioning system. t B t The GPS absolute position of the track is used as the absolute GPS position of the track. The sum of this absolute GPS position and the relative coordinates of the track irregularity is then used to obtain the absolute GPS position of the track irregularity. Where P t ns =P t +wP t ns ,

[0109] Step 9: Using the collected track image data, create a track defect dataset and train a target detector based on the YOLOv7 network structure to detect track defects such as peeling, fish scale damage, delamination, and corrugation, thus obtaining the track defect detection model. ill The specific steps are as follows:

[0110] Step 9.1: The YOLOv7 network structure for target detection is selected for track defect detection network structure, and its configuration file yolov7.yaml is modified as follows: the number of classifications is changed to the number of track defect classifications C. ill That is, set the category quantity variable nc to C. ill The value is 4;

[0111] The YOLOv7 network structure for object detection is the YOLOv7 network structure for general object detection proposed in the paper "YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors" published by Chien-Yao Wang, Alexey Bochkovskiy et al. in 2022.

[0112] Step 9.2: Acquire images of Class C track defects. The total number of images acquired is N. s The number of images captured for each class is

[0113] Step 9.3: Construction of training samples is as follows: Upload the track defect image data to the data annotation website, and use the annotation tool on the website to annotate the defect types and ground truth bounding boxes of the track defect images; after all images are annotated, use the website's dataset export function to export the annotated dataset to the file storage path data / train, create a dataset configuration file data.yaml in the path data / , and write the following content in the file:

[0114]

[0115] The labeled dataset and configuration file are combined to form the rail defect training dataset railData;

[0116] Step 9.4: Set the network training parameters in the network training file train.py: total number of iterations epoch = 100, number of images in batch = 16, initial learning rate lr = 0.01, input image size W×H, optimizer set to Adam, initial value of training iterations step = 1, training sample data path is data / train, training device is set to decive = 0 (indicating GPU training), and the output model name is yolo7.pt; the specific command line code is as follows:

[0117]

[0118] Step 9.5: Define the network loss function as follows:

[0119]

[0120] Where x n y represents the score for predicting the nth sample as a positive example. nLet δ represent the label of the nth sample, and δ represent the sigmoid function.

[0121] Step 9.6: Run the file train.py in the conda environment to train the network. Training should be performed when the number of training iterations (step) is greater than or equal to 200 or the network loss function (Loss) is less than 10. -1 At that time, the network training ends, and the orbital defect detection model is output. ill .

[0122] Step 10: Obtain the orbital image at time t. and disease detection model ill As input, the detection program train.py, located in the Project / yolov7 directory within a Python environment, is called, and the output shows the disease. The position in the middle and categories Will Multiply by homography matrix get Projection point on the left viewpoint image by and And the projection matrix from 3D to 2D and As input, the open-source OpenCV-Python library function `cv2.triangulatePoints()` is called, and the output is... Corresponding 3D coordinates P t ill The specific process is as follows:

[0123] Step 10.1: Detect the image and disease detection model ill As input, run the test file test.py in a Python environment, and output the detected disease types. The rectangle where the disease is located in For diseases in Middle position The coordinates of the center point, and These represent the width and height of the rectangular detection box, respectively.

[0124] Step 10.2: According to formula (6), use the homography matrix Image Location of track defects Projected image to the left viewpoint The location of the lesion on the left viewpoint image is obtained from the above.

[0125]

[0126] Step 10.3: Position the disease in the intermediate viewpoint image. Location of the disease in the left viewpoint image 3D to 2D projection matrix As input, the open-source OpenCV-Python library function `cv2.triangulatePoints()` is called to output the 3D coordinates of the track defects.

[0127] Step 11: Calculate the defect location P obtained by the track inspection vehicle at time t using the same method as in Step 8. t ill GPS location corresponding to the 3D coordinates

[0128] Step 12: Output the type of irregularity C detected by the track inspection car at time t. t Unevenness parameter PS t Uneven GPS location The type of defect detected by the output track inspection car at time t and GPS location

[0129] Track defect detection images such as Figure 8 As shown, (a), (b), and (c) represent track delamination, track fracture, and track debris, respectively.

[0130] The track defect detection network constructed by the method of this invention was tested on a track image dataset to evaluate its detection performance. The following indicators were used to objectively evaluate the detection results:

[0131] (1) Precision (P). It represents the proportion of samples that have a true result and that the result is correct out of all samples that have a true result.

[0132] (2) Recall (R). It represents the proportion of samples that are true and whose test results are correct out of all samples that are actually true.

[0133] (3) F1Score: The harmonic mean of precision and recall, a comprehensive measure of the model's detection performance, and its formula is as follows:

[0134]

[0135] The performance of the track defect detection model was tested based on the above evaluation indicators, and the results are shown in Table 1.

[0136] Table 1

[0137]

[0138]

[0139] The track irregularity detection method proposed in this invention was subjected to track irregularity detection performance testing. In addition to P, R, and F1 Score, a new objective evaluation index was added: accuracy (Acc), which represents the proportion of correctly detected samples out of the total samples.

[0140] The performance of the track irregularity detection method proposed in this invention was evaluated based on the above evaluation indicators, and the results are shown in Table 2:

[0141] Table 2

[0142]

[0143] As shown in Tables 1 and 2, the method of this invention has achieved relatively accurate detection results in the task of detecting track defects and track irregularities, proving the effectiveness of the method. The above subjective and objective results demonstrate that the method of this invention can effectively detect and identify track defect images and track point clouds, and the detection results reflect the efficiency and accuracy of the method. Furthermore, compared with traditional track detection methods, our method achieves rapid and accurate detection and analysis of track defects and track irregularities while meeting the required detection accuracy, reducing labor costs.

[0144] Through the above-described method, this invention provides an intelligent detection and location method for track irregularities and defects. Addressing the problems of existing track irregularity and defect monitoring systems on the market, such as the need for manual collaboration, low levels of intelligence and integration, poor dynamic monitoring and perception, and high labor costs, this invention designs a mobile measurement method capable of detecting and locating railway track irregularities and defects while moving and measuring. This achieves accurate and rapid detection and analysis of track irregularities and defects. Therefore, this invention collects and analyzes data from a tri-lens camera and GPS sensor installed on a track inspection vehicle. On one hand, it performs three-dimensional reconstruction of the track and uses point cloud elevation analysis to detect and locate track irregularities such as triangular pits, unevenness, and levelness. On the other hand, it employs an image target deep learning detection model to detect and locate defects such as peeling, fish-scale damage, delamination, and corrugation on the track. Compared to traditional detection methods, the stereo vision-based track irregularity detection and location method is fast and efficient, saving manpower and resources to a certain extent, and is also easy to install on a track inspection vehicle.

Claims

1. A method for intelligent detection and location of track irregularities and defects, characterized in that: Specifically comprising implementing according to the following steps: Step 1, build data positioning system and data acquisition system on the T-shaped rail inspection vehicle, and collect the position information of the rail inspection vehicle in the WGS84 coordinate system by using the data positioning system wherein, and are respectively the longitude and latitude values of the rail inspection vehicle at the moment, , is the total collection time Step 2, collecting from three view points by using three cameras on the moving track inspection vehicle track image sequence at time t , are respectively left view point, middle view point and right view point images of the track at time t, and the track image sequence is stored respectively Step 3, generate the 3D point cloud using the track image sequence as input the track scene 3D point cloud at the moment , the projection relationship matrix of the track scene 3D point cloud to each view point image is obtained , and , the homography matrix between the center view point image and the left and right view point images is also obtained ;​​ Step 4, with Mid-viewpoint orbital image at time As input, the two side lines of the left track are extracted using a straight line extraction method. and the two side lines of the right track ,extract and midline ,exist Uniform sampling The points constitute a point set. Multiply by the homography matrix between the center viewpoint and the left viewpoint images. The left viewpoint image is obtained. The set of projection points on ; set of points , Projection relation matrix and As input, call the open-source opencv-python library function. The output is obtained The corresponding 3D coordinate set ;extract and midline ,exist Uniform sampling The points constitute a point set. The point set is obtained using the same method. Corresponding 3D coordinates ; Step 5: Denote the track width variable as... Extract from set Centered on each point in China A point cloud within a cube with side length [length] that belongs to the scene's 3D point cloud All points within the range constitute a point set. ,in Therefore The set of points extracted from the center point of the cube. , for The first in One point, for The number of interior points , They are respectively of coordinate, The initial point cloud for the left track is obtained; the initial point cloud for the right track is obtained in the same way. ; Step 6, the track elevation variable is denoted as The initial point cloud of the left track is filtered by height, and the right track point cloud is filtered by height in the same way; the updated left and right track point clouds after filtering are and ; Step 7, to left and right track point clouds updated in real time and perform data analysis to determine whether there is a track irregularity, if there is an irregularity, record the irregularity type , irregularity parameters and irregularity position ; Step 8, calculate irregularity position corresponding GPS position ; Step 9, using the collected track image data to make track disease data set, training track peeling, fish scale injury, delamination, wave grinding target detector based on YOLOv7 network structure, obtaining track disease detection model ; Step 10, put orbital images at time Disease detection model As input, in Calling file paths in an environment The following detection procedure Output of diseases The position in the middle and categories ,Will Multiply by homography matrix ,get Projection point on the left viewpoint image ;by and And the projection matrix from 3D to 2D and As input, call the open-source opencv-python library function. Output Corresponding 3D coordinates ; Step 11. Calculate track inspection car using the same method as step 8 Disease position obtained at the moment GPS position corresponding to 3D coordinates in ; Step 12, output track inspection car Disorder type detected at the moment Disorder parameter Disorder GPS position ; output track inspection car Disease type detected at the moment and GPS position ; The specific steps of step 7 are as follows: Step 7.1, Compute left orbit point cloud The mean of coordinates of each sub-point set , where k represents the position of the sub-point set, and the mean of the left orbit point cloud coordinate set is obtained , where , , , , , The elevation information of the point cloud in the point set , that is , the elevation set of the left orbit point cloud is represented as , and the right orbit mean point cloud coordinate set and the elevation set of the right orbit point cloud are obtained by the same method; Step 7.2, determine whether there is a triangular pit irregularity on the track, and if there is a triangular irregularity, record the irregularity parameters and position; Step 7.3, determine whether there is a high-low irregularity on the track, and if there is a high-low irregularity, record the irregularity parameters and position; Step 7.4, determine whether there is a horizontal irregularity on the track, and if there is a horizontal irregularity, record the irregularity parameters and position; The specific process of step 7.2 is as follows: Step 7.2.1, calculate the point cloud elevation difference at the parallel position of the left and right tracks, the formula is , get the set ; Step 7.2.2, if , , The conditions are met. or conditions This indicates the movement of the track vehicle. At any given time, there is an unevenness in the track triangle, i.e., an unevenness type. It equals 1, where To determine the allowable deviation accuracy of the triangular dent unevenness, record the triangular dent unevenness parameters. Uneven location of the triangular pit and other defects ; The specific process of step 7.3 is as follows: Step 7.3.

1. Calculate the absolute difference between elements in the left and right track elevation sets and the standard elevation, respectively, denoted as the left and right elevation difference sets and where , ;​ Step 7.3.2, if the condition is satisfied, it is indicated that at the instant at which the railcar is moving, there is a high-low irregularity on the left track, i.e. the irregularity type is equal to 2, the high-low irregularity parameters and the high-low irregularity disease position are recorded; if the condition is satisfied, it is indicated that at the instant at which the railcar is moving, there is a high-low irregularity on the right track, i.e. the irregularity type is equal to 2, the high-low irregularity parameters and the high-low irregularity disease position are recorded, where is the allowable deviation accuracy of the high-low irregularity. The specific process of step 7.4 is as follows: Step 7.4.1, calculate the absolute difference of the elevations of the left track and the right track, the formula is , to get the absolute difference set of the elevations of the tracks ; Step 7.4.2, If the condition is satisfied, it is indicated that a horizontal irregularity is present at the instant of time at which the railcar is moving , i.e. the irregularity type is equal to 3, the horizontal irregularity parameters and the horizontal irregularity disease position are recorded, where is the tolerance accuracy of the horizontal irregularity.

2. The method of claim 1, wherein, The specific steps of step 4 are as follows: Step 4.1, convert track image As input, call the opencv-python library function in Python programming language Convert to grayscale image ; Step 4.2, convert the grayscale image to binary image As input, call the opencv-python library function of Python programming language For Gaussian filtering is performed to obtain a filtered image ; Step 4.3, with the filtered image For input, call the opencv-python library function of the Python programming language For input, call the opencv-python library function of the Python programming language Perform edge detection processing to obtain an edge image ; Step 4.4, with the edge image As input, call the opencv-python library function of Python programming language To Carry out track line extraction to obtain the left and right two edge lines of the left track , And the left and right two edge lines of the right track , , The four edge lines are represented by two end points respectively , , , , Wherein, Indicates the end point closer to the camera, Indicates the end point farther from the camera; Step 4.5, calculate the midline of the left orbit edge line : calculate and the midpoint of the two endpoints , , , calculate and the midpoint of the two endpoints , , , call the straight line extraction algorithm with point and as input respectively get the midline of the two tracks of the left orbit the parameters of the general formula straight line equation , that is, express the midline as ; use the same method to get the midline of the two tracks of the right orbit Step 4.6, in uniformly sampling a point set ; Step 4.7, multiply the points in the set by the homography matrix to get the projected point set in the left view image ; Step 4.8, get the set of projected points on the left view image Step 4.9, get the set of projected points on the right view image Step 4.10, get the projection relationship matrix As input, call the open source opencv-python library function As output, get the set of 3D coordinates of the left track ; Step 4.9, in uniformly sampling a point set consisting of repeating the operations of steps 4.7-4.8, outputting a point set a corresponding set of 3D coordinates .

3. The method of claim 1, wherein the method further comprises: The height filtering process described in step 6 on the initial point cloud of the left track is: , if its coordinates belong to , then it is kept, otherwise it is removed from the set , where is the track point cloud elevation allowed variation value.

4. The method of claim 1, wherein the method further comprises: The specific steps of step 8 are as follows: Step 8.1 Calculate the track inspection car uneven position at the moment point cloud coordinate set where the car is located mean coordinate of all points in the point cloud , according to the conversion relationship from the spatial rectangular coordinate system to the geodetic coordinate system, the mean coordinate is converted into the relative coordinate in the geodetic coordinate system ; Step 8.2, the Rail inspection car position collected by the instant data positioning system GPS absolute position of the track as the GPS absolute position, and the relative coordinates of the track irregularity position are summed up to obtain the GPS absolute position of the track irregularity wherein , .

5. The method of claim 1, wherein the method further comprises: The specific steps in step 9 are as follows: Step 9.1, the track disease detection network structure selects the YOLOv7 network structure of target detection, and the configuration file yolov7.yaml is modified as follows: the number of classifications in it is modified to the number of classifications of track diseases, and the value is 4. , Step 9.2, to Collect images of the class of track diseases, the total number of images collected is , and the number of images collected for each class is ; Step 9.3, the construction of the training sample is as follows: upload the track disease image data to the data labeling website, and use the labeling tool on the website to label the track disease image disease type and disease detection true value box; After all the image annotation is completed, the annotated dataset is exported to a file storage path using the dataset export function of the website Next, a dataset configuration file is created Next, a dataset configuration file is created The annotated dataset and the configuration file are combined to form a track disease training dataset ; Step 9.

4. Set network training parameters in the network training file Total number of iterations for network training Number of images for network batch Initial learning rate for network training Image size for network input Optimizer used for network training is set to Initial value of network training times variable Training sample data path is Working device for training is set to Training using GPU is indicated, and the output model name is ; Step 9.5, define the network loss function as: （5） wherein represents a score predicting the i-th sample as a positive example, represents the label of the i-th sample, represents a sigmoid function;​​ Step 9.6, in running the file under the environment , performing network training, when the network training times variable or the network loss function is less than , ending the network training, and outputting the track disease detection model .

6. The method of claim 1, wherein the method further comprises: The specific process of step 10 is as follows: Step 10.1: Detect the image Disease detection model As input, in Run test files in the environment Output the detected disease types The rectangle where the disease is located ,in For diseases in Middle position The coordinates of the center point, and These represent the width and height of the rectangular detection box, respectively. Step 10.2, according to formula (6), utilize the homography matrix The image The position of the disease on the upper track Project to the left view point image The corresponding disease position on the left view point image is obtained ; （6） Step 10.3, position of the disease in the intermediate view image position of the disease in the left view image projection matrix from 3D to 2D As input, call the open source opencv-python library function output track the 3D coordinates of the disease .

Images