Track elastic strip displacement detection method and system based on target space position deviation

Abstract

The invention provides a track elastic strip displacement detection method and system based on target space position deviation. The method comprises the steps that a track brightness diagram is acquired through a laser contourgraph; performing data enhancement processing on the track brightness diagram, and detecting an elastic strip and a clamping plate in the track brightness diagram by using a target detection algorithm to obtain elastic strip and clamping plate detection data; processing according to the detection data of the elastic strip and the clamping plate to obtain coordinate information of the elastic strip and the clamping plate; according to the coordinate information of the elastic strip and the clamping plate, judging whether the elastic strip intersects with the clamping plate by using an intersection rule algorithm; if yes, elastic strip clamping plate intersection position information of each track brightness diagram is obtained through calculation; obtaining the minimum value h of the vertical coordinate difference value; the minimum value h is compared with a judgment threshold value Th, and whether the elastic strip shifts or not is judged; and carrying out elastic strip displacement drawing and information display. The technical problems that accuracy, reliability and detection efficiency of track elastic strip displacement detection are low, and the degree of dependence on railway track elastic strip displacement defect data is too high are solved.

Description

Technical Field

[0001] The present invention relates to the field of rail transit setting detection, and in particular to a rail spring bar displacement detection method and system based on target spatial position deviation. Background Art

[0002] With the development of railway transportation, the safety and reliability of track equipment have received increasing attention. The fastener clips in railway tracks are key components for maintaining the stability of the track structure. However, due to long-term operation and the influence of the external environment, the fastener clips may become loose or shifted, thus affecting the safety and stability of the railway line. Therefore, it is very important to regularly inspect the fastener clips and detect possible shifting in advance.

[0003] Traditional methods for detecting spring clip displacement usually rely on visual inspection or the use of simple measuring tools. This method has the problems of low efficiency, low accuracy, and inability to fully cover. With the continuous development of computer vision and deep learning, spring clip displacement detection methods based on deep learning have gradually emerged. At present, although the detection methods proposed by some institutions and scholars can detect spring clip displacement to a certain extent, due to the small number of spring clip displacement data samples and the need to continuously correct the algorithm for spring clip displacement detection, the generalization is poor and it is difficult to adapt to the complex and changing working conditions on site. The existing public document "Research on Rail Fastener State Detection Based on Deep Learning" discloses the application of deep learning target detection algorithms to the field of railway fastener defect detection. The researchers created a railway fastener defect dataset and used the Faster R-CNN algorithm and the Mask RCNN algorithm to apply it to rail fastener state detection. Under the same control conditions, experiments were conducted on the rail fastener state dataset and the results were obtained. The experimental results show that the Mask RCNN algorithm performs best in rail fastener state detection. However, the algorithm is slow in detecting the status of rail fasteners, and the data samples are too few, including defect samples, which will result in the research only having good detection results in its data set, but poor results when applied to real scenarios. In addition, the algorithm has not yet been verified in a real environment with a patrol car.

[0004] As well as the existing public literature "Research on Railway Fastener Defect Detection Algorithm Based on Deep Learning", the deep learning target detection algorithm is applied to the field of railway fastener defect detection. The researchers created a railway fastener defect dataset and applied SSD, YOLOv3, YOLOv4, YOLOv5s and Faster R-CNN algorithms to railway fastener defect detection. Under the same control conditions, experiments were conducted on the railway fastener defect dataset and the results were obtained. The experimental results show that the YOLOv5s algorithm performs best in railway fastener defect detection and has a fast detection speed. In addition, the existing technology integrates the GhostNet network and CA attention mechanism on the basis of the YOLOv5 algorithm. Compared with the original YOLOv5 algorithm, the experimental comparison of the improved version on the self-made dataset shows that although the detection accuracy is slightly reduced, the processing speed is significantly improved. Moreover, these improvements have not yet been verified in a real environment with the inspection car.

[0005] With the continuous development of computer vision and three-dimensional imaging technology, track spring bar displacement detection methods based on structured light have gradually come to the fore. Structured light technology forms a structured pattern by projecting light, and by capturing the reflection or deformation of light by the object, it can achieve high-precision measurement of the surface shape of the object. For example, the existing invention patent application document "A method for detecting the displacement of a fastener spring bar" with publication number CN110634122A, the existing method includes: using a three-dimensional imaging system to obtain three-dimensional morphological data on both sides of the track, and converting it into a two-dimensional depth image; determining the position of the fastener in the two-dimensional depth image, and extracting a local two-dimensional depth image of the area where the fastener is located; setting the horizontal coordinate value x in the local two-dimensional depth image of the fastener min 、x max and the ordinate value y min ,y max , extract the image of the significant area of ​​the spring strip displacement; set the threshold T1, perform binarization processing on the image of the significant area of ​​the spring strip displacement, and accurately extract the outline of the spring strip displacement area; based on the minimum ordinate value y of the outline of the spring strip displacement area min and the maximum ordinate value y max , calculate the displacement distance of the spring bar Δs=y max -y min ; Set the threshold value T2, and determine whether there is a risk of spring bar displacement based on the spring bar displacement distance Δs and the threshold value T2.

[0006] However, the aforementioned three-dimensional imaging system is used to convert three-dimensional data into two-dimensional data for processing, which takes a long time. In addition, the performance of detecting and locating the displacement position of the spring bar is poor when extracting the position of the spring bar through the set coordinate value.

[0007] In summary, the existing technology has technical problems such as low accuracy, reliability and detection efficiency of rail spring clip displacement detection, and excessive reliance on railway rail spring clip displacement defect data. Summary of the invention

[0008] The technical problem to be solved by the present invention is: how to solve the technical problems that the accuracy, reliability and detection efficiency of rail spring clip displacement detection in the prior art are low, and the degree of dependence on railway rail spring clip displacement defect data is too high.

[0009] The present invention adopts the following technical solution to solve the above technical problem: a rail spring bar displacement detection method based on target space position deviation comprises:

[0010] S1, obtain the track brightness map through the laser profiler;

[0011] S2, performing data enhancement processing on the track brightness image, using the target detection algorithm to detect the spring bars and splints in the track brightness image, and obtaining the spring bar and splint detection data;

[0012] S3, according to the detection data of the spring bar and the splint, the coordinate information of the spring bar and the splint is obtained by processing;

[0013] S4, judging whether the spring bar and the splint intersect with each other by using the intersection rule algorithm according to the coordinate information of the spring bar and the splint;

[0014] S5, if yes, then calculate the first ordinate difference △h1 and the second ordinate difference △h2 between the spring bar and the splint, and obtain the intersection position information of the spring bar and splint of each track brightness image;

[0015] S6, obtaining the minimum value △h between the first ordinate difference △h1 and the second ordinate difference △h2;

[0016] S7, setting a determination threshold Th, comparing the minimum value △h and the determination threshold Th, determining whether the spring bar is displaced, and obtaining target position information of the displaced spring bar;

[0017] S8. Drawing the displacement of the spring strip and displaying the information are performed according to the target position information of the displacement spring strip.

[0018] The present invention combines target detection technology, detects the spatial coordinate information of the spring clip and the clamp, and realizes the detection and positioning of the spring clip displacement according to the spatial coordinate information deviation, thereby improving the detection accuracy and reliability of the rail spring clip displacement. The spring clip displacement detection and positioning method based on the target spatial position deviation of the present invention can provide a new solution for the accurate detection of the rail spring clip displacement problem.

[0019] The present invention does not change the fastener structure, does not require a large number of spring bar displacement samples, and its detection accuracy and detection speed can well meet the standards for rail track spring bar displacement detection.

[0020] The present invention mainly uses the YOLO V5s algorithm to locate the spring clip and the splint, obtains the spatial coordinate information of the two, improves the detection speed and accuracy, and maintains sufficient shape information, which is helpful for further analysis and processing. The present invention combines the target detection technology and the spatial position deviation technology to make the spring clip displacement detection method real-time, which is suitable for real-time monitoring of the displacement state of the rail spring clip, and provides an effective means for track maintenance and safe operation.

[0021] In a more specific technical solution, S2 includes:

[0022] S21, horizontally rotating the track brightness map to obtain a horizontally rotated track brightness map, and enhancing image data of the track brightness map;

[0023] S22, vertically rotating the track brightness map to obtain a vertically rotated track brightness map, thereby enhancing image data of the track brightness map to obtain an image enhanced track brightness map;

[0024] S23, using a preset annotation tool, perform image annotation operations on the spring bars and splints in each track brightness image, and obtain a corresponding label txt file for each track brightness image; wherein, labels are created according to preset target tasks, image annotation operations are performed, and the annotation information of the preset target tasks is converted into the txt file format required for model training;

[0025] S24, obtain a new label file txt for the corresponding label file txt;

[0026] S25, setting the width of the vertical rotation track brightness image, and obtaining a new label file txt corresponding to the vertical rotation track brightness image;

[0027] S26, storing the image enhancement track brightness map, the horizontal rotation track brightness map and the treatment rotation track brightness map into the preset images file, and storing the corresponding label file txt and the new label file txt into the labels file;

[0028] S27. Select target detection algorithms for spring bars and splints. The target detection algorithms include FasterR-CNN, YOLOv5s, and SSD. The network structure of YOLOv5s includes an input end, a Backbone layer, a Neck network, and an output end, Head.

[0029] Before target detection, the present invention performs horizontal and vertical rotations on the track image acquired by the laser profiler, and processes the corresponding label file to obtain the corresponding horizontally rotated label file and the vertically rotated label file; so as to expand the data set and reduce the labeling time; this may include operations such as labelimg labeling, image rotation, and label file processing, which can ensure that the track image and the label file are expanded correctly, and the track image and the label file can be matched one by one.

[0030] In a more specific technical solution, in S26, the image data in the preset images file and the label data in the labels file are divided to obtain a training set train file and a verification set val file.

[0031] In a more specific technical solution, the Backbone layer in S27 includes: Focus, Conv, C3 modules and SPPF modules, and the Backbone layer is used to extract feature information of the input image to form a global and local fusion feature map;

[0032] Use the Focus module to slice the input image, obtain and pass the slice data to the next convolution Conv module;

[0033] The C3 module includes: standard convolutional layer, Bottleneck module and splicing module. Using the C3 module, feature extraction is performed in the Backbone layer;

[0034] Through pooling of spatial pyramid, the SPPF module is obtained, and the local and global information of the feature map are fused through maximum pooling.

[0035] In a more specific technical solution, the Neck network in S27 includes: feature pyramid FPN and path aggregation network PANet, which are used to perform re-fusion operations on the global and local fusion feature maps to obtain a hierarchical enhanced feature map.

[0036] In a more specific technical solution, in S27, the output end Head is used to classify and predict the hierarchical structure enhanced feature map, correct the position of the candidate box, and obtain a suitable prediction result.

[0037] The present invention adopts the target detection model YOLOv5s, which strikes a balance in terms of speed, accuracy and lightness, and is suitable for various real-time target detection scenarios. The model input end of the present invention includes three functions: data enhancement, adaptive anchor frame and adaptive image scaling, which mainly solve the problems of uneven distribution of data sets, optimal anchor frame calculation and removal of redundant information, and improve the speed and accuracy of the model.

[0038] In a more specific technical solution, S3 includes:

[0039] S31, processing and obtaining the position information of the spring bars and the splints according to the detection data of the spring bars and the splints;

[0040] S32, using rect1, storing the rectangular frame of the elastic bar;

[0041] S33. Using rect2, store the rectangular frame of the splint, and extract the ROI based on the rectangular frame.

[0042] The present invention determines the position information of the spring bar and the splint in the brightness image in the form of a bounding box; the coordinate information of the bounding box can accurately describe the position of the spring bar and the splint in the brightness image. The present invention avoids the requirement on the number of negative samples (track images with spring bar displacement) by extracting the coordinate information of the ROI area of ​​the spring bar and the splint in the track image, and can accurately restore the position of the spring bar and the splint.

[0043] In a more specific technical solution, S4 includes:

[0044] S41, traversing step S3 to obtain the stored position information of the spring bars and the splints, so as to obtain the position information of the spring bars and the splints corresponding to each image;

[0045] S42. Using the intersection rule algorithm, determine whether the spring bar and the splint intersect with the following logic:

[0046] x1 <x2+w2

[0047] x1+w1>x2

[0048] y1 <y2+h2

[0049] y1+h1>y2

[0050] Among them, x1 is the horizontal coordinate of the spring bar, y1 is the vertical coordinate of the spring bar, h1 is the height of the spring bar, x1 is the horizontal coordinate of the splint, y2 is the vertical coordinate of the splint, h2 is the height of the splint, w1 is the width of the rectangular frame of the spring bar, and w2 is the width of the rectangular frame of the splint.

[0051] The present invention can accurately and quickly determine the intersecting spring bars and splints in each track image, and can obtain the position information of the intersecting spring bars and splints.

[0052] In a more specific technical solution, S8 includes:

[0053] S81, performing a shift positioning operation on the elastic bar that sends the shift, drawing a shift rectangular frame for the elastic bar, so as to obtain corresponding elastic bar shift positioning data;

[0054] S82. Draw and display the spring bar defect information according to the spring bar displacement positioning data.

[0055] The present invention obtains the position information of the spring bar in the spring bar displacement, uses the horizontal and vertical coordinates of the upper left corner of the spring bar target frame and the width and length of the target frame, can accurately locate the spring bar displacement, and draw the defect information on the located rectangular frame, thereby realizing the accurate positioning of the spring bar displacement and the marking of the defect information.

[0056] In a more specific technical solution, the rail clip displacement detection system based on the target spatial position deviation includes:

[0057] Track image acquisition module, used to acquire track brightness map through laser profilometer;

[0058] The target detection and data set construction module is used to perform data enhancement processing on the track brightness map, and use the target detection algorithm to detect the spring bars and splints in the track brightness map to obtain the spring bar and splint detection data. The target detection and data set construction module is connected to the track image acquisition module;

[0059] A position information acquisition module is used to process the spring bar and splint detection data to obtain the spring bar and splint coordinate information. The position information acquisition module is connected to the target detection and data set construction module.

[0060] A target intersection judgment module is used to judge whether the spring bar and the splint intersect according to the coordinate information of the spring bar and the splint by using the intersection rule algorithm. The target intersection judgment module is connected to the position information acquisition module;

[0061] The intersection target difference calculation module is used to calculate the first ordinate difference △h1 and the second ordinate difference △h2 between the spring bar and the splint when the spring bar and the splint intersect, and obtain the intersection position information of the spring bar and the splint of each track brightness map. The intersection target difference calculation module is connected to the target intersection judgment module;

[0062] A minimum difference determination module is used to obtain the minimum value △h between the first ordinate difference △h1 and the second ordinate difference △h2, and the minimum difference determination module is connected to the intersection target difference calculation module;

[0063] The spring bar displacement determination module sets a determination threshold Th, compares the minimum value △h with the determination threshold Th, determines whether the spring bar is displaced, and obtains the target position information of the displaced spring bar. The spring bar displacement determination module is connected to the difference minimum value determination module;

[0064] The spring clip displacement position positioning and drawing module is used to draw the spring clip displacement and display the information according to the displacement spring clip target position information. The spring clip displacement position positioning and drawing module is connected to the spring clip displacement determination module.

[0065] Compared with the prior art, the present invention has the following advantages:

[0066] The present invention combines target detection technology, detects the spatial coordinate information of the spring clip and the clamp, and realizes the detection and positioning of the spring clip displacement according to the spatial coordinate information deviation, thereby improving the detection accuracy and reliability of the rail spring clip displacement. The spring clip displacement detection and positioning method based on the target spatial position deviation of the present invention can provide a new solution for the accurate detection of the rail spring clip displacement problem.

[0067] The present invention does not change the fastener structure, does not require a large number of spring bar displacement samples, and its detection accuracy and detection speed can well meet the standards for rail track spring bar displacement detection.

[0068] The present invention mainly uses the YOLO V5s algorithm to locate the spring clip and the splint, obtains the spatial coordinate information of the two, improves the detection speed and accuracy, and maintains sufficient shape information, which is helpful for further analysis and processing. The present invention combines the target detection technology and the spatial position deviation technology to make the spring clip displacement detection method real-time, which is suitable for real-time monitoring of the displacement state of the rail spring clip, and provides an effective means for track maintenance and safe operation.

[0069] Before target detection, the present invention performs horizontal and vertical rotations on the track image acquired by the laser profiler, and processes the corresponding label file to obtain the corresponding horizontally rotated label file and the vertically rotated label file; so as to expand the data set and reduce the labeling time; this may include operations such as labelimg labeling, image rotation, and label file processing, which can ensure that the track image and the label file are expanded correctly, and the track image and the label file can be matched one by one.

[0070] The present invention adopts the target detection model YOLOv5s, which strikes a balance in terms of speed, accuracy and lightness, and is suitable for various real-time target detection scenarios. The model input end of the present invention includes three functions: data enhancement, adaptive anchor frame and adaptive image scaling, which mainly solve the problems of uneven distribution of data sets, optimal anchor frame calculation and removal of redundant information, and improve the speed and accuracy of the model.

[0071] The present invention determines the position information of the spring bar and the splint in the brightness image in the form of a bounding box; the coordinate information of the bounding box can accurately describe the position of the spring bar and the splint in the brightness image. The present invention avoids the requirement on the number of negative samples (track images with spring bar displacement) by extracting the coordinate information of the ROI area of ​​the spring bar and the splint in the track image, and can accurately restore the position of the spring bar and the splint.

[0072] The present invention can accurately and quickly determine the intersecting spring bars and splints in each track image, and can obtain the position information of the intersecting spring bars and splints.

[0073] The present invention obtains the position information of the spring bar in the spring bar displacement, uses the horizontal and vertical coordinates of the upper left corner of the spring bar target frame and the width and length of the target frame, can accurately locate the spring bar displacement, and draw the defect information on the located rectangular frame, thereby realizing the accurate positioning of the spring bar displacement and the marking of the defect information.

[0074] The present invention solves the technical problems existing in the prior art of low accuracy, reliability and detection efficiency of rail spring clip displacement detection and excessive reliance on railway rail spring clip displacement defect data. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Figure 1 Schematic diagram of basic steps of a rail spring bar displacement detection method based on target spatial position deviation according to Embodiment 1 of the present invention;

[0076] Figure 2 It is a side view of the 3D profile measuring instrument of embodiment 1 of the present invention;

[0077] Figure 3 A sample diagram of a trolley equipped with LV2800EH-SP and a schematic diagram of the measurement range according to Example 1 of the present invention;

[0078] Figure 4 This is a schematic diagram of specific steps for detecting the spring bars and the clamping plates in the brightness image according to Embodiment 1 of the present invention;

[0079] Figure 5 This is a schematic diagram comparing a normal spring bar and a spring bar displacement according to Embodiment 1 of the present invention;

[0080] Figure 6a This is a diagram showing the effect of the image before rotation according to Embodiment 1 of the present invention;

[0081] Figure 6b This is a diagram showing the effect of horizontally rotating an image according to Embodiment 1 of the present invention;

[0082] Figure 6c This is a diagram showing the effect of vertically rotating an image according to Embodiment 1 of the present invention;

[0083] Figure 7 It is an algorithm framework diagram of YOLOv5s of Example 1 of the present invention;

[0084] Figure 8 This is a schematic diagram of specific steps for obtaining coordinate information of spring bars and splints according to Embodiment 1 of the present invention;

[0085] Fig. 9 This is a schematic diagram of specific steps for determining whether the spring bar and the splint intersect in Embodiment 1 of the present invention;

[0086] Fig.10 A schematic diagram of obtaining elastic strips and intersections according to Embodiment 1 of the present invention;

[0087] Fig.11 This is a schematic diagram of specific steps for calculating the difference between two ordinates of the spring bar and the splint in Example 1 of the present invention;

[0088] Fig.12 It is a schematic diagram of the vertical coordinate difference of the spring clip of Example 1 of the present invention;

[0089] Fig.13 Schematic diagram of the minimum value of the vertical coordinate difference of the spring clip of Example 1 of the present invention;

[0090] Fig.14 It is a schematic diagram of specific steps of drawing the spring bar displacement and displaying information according to Embodiment 1 of the present invention;

[0091] Fig.15 It is a schematic diagram of the positioning and display of the spring bar displacement according to the first embodiment of the present invention. DETAILED DESCRIPTION

[0092] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in combination with the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0093] Example 1

[0094] like Figure 1 As shown, the rail spring bar displacement detection method based on target spatial position deviation provided by the present invention includes the following basic steps:

[0095] S1, obtain track brightness map through laser profilometer;

[0096] like Figure 2 and Figure 3 As shown, in this embodiment, the camera is adjusted to a suitable angle and position to capture the reflected laser and form a high-quality brightness image; see Figure 2 , the mechanical structure of the profiler is as follows Figure 3 Currently, a two-column installation method is adopted, and the height can be adjusted up and down. The mechanical structure is installed on the front of the trolley, and a profile meter is installed on each side to achieve full coverage of the left and right track surfaces and fasteners.

[0097] The present invention obtains a track brightness map by using a laser profilometer, and detects spring bars and splints in the brightness map in combination with a target detection algorithm, thereby improving the accuracy and precision of the detection.

[0098] S2, performing data enhancement processing on the track brightness image, and using the target detection algorithm to detect the spring bars and splints in the track brightness image;

[0099] like Figure 4 As shown, in this embodiment, the step S2 of detecting the spring bars and the clamping plates in the brightness image further includes the following specific steps:

[0100] S21, horizontally rotating the track brightness map to obtain a horizontally rotated track brightness map, thereby enhancing image data of the track brightness map;

[0101] like Figure 6a and Figure 6b As shown, in this embodiment, the horizontal rotation of the image is a rotation around the image horizontal coordinate, and the image vertical coordinate remains unchanged;

[0102] S22, vertically rotating the track brightness map to obtain a vertically rotated track brightness map, thereby enhancing image data of the track brightness map;

[0103] like Figure 5 , Figure 6a and Figure 6c As shown, in this embodiment, the vertical rotation of the image is a rotation around the image ordinate, and the image abscissa remains unchanged; see Figure 5 , the green box on the right is a schematic diagram of a normal spring bar according to Example 1 of the present invention, and the red box on the left is a schematic diagram of a displacement spring bar according to Example 1 of the present invention;

[0104] S23, using annotation software to annotate the spring bars and splints in each track brightness image, and obtaining a label txt file corresponding to each track image;

[0105] In LabelImg used in this embodiment, labels are created according to the target task, and the image annotation is completed. The annotation information needs to be converted into the txt file format required for model training; in this embodiment, each image requires a corresponding txt file, and the txt file includes but is not limited to: target category and bounding box coordinates;

[0106] S24, processing the label file txt marked with the track brightness diagram;

[0107] In this embodiment, each line in the label file txt represents the corresponding labeled category information, with a total of five columns. Respectively represent the label name (that is, the spring bar or splint), the horizontal coordinate of the upper left corner of the rectangular box, the vertical coordinate of the upper left corner of the rectangular box, the width of the rectangular box, and the length of the rectangular box. After the track brightness image is horizontally rotated, the position information of the target object is changed, and the vertical coordinate, width and length of the target are not changed. Only the width x1 of the target is changed. Therefore, the transformed width x2 is set to 1-x1, and the new label file txt corresponding to the track image after horizontal rotation can be obtained;

[0108] S25. Similarly, after the track brightness image is vertically rotated, the position information of the target object is changed, and the horizontal coordinate, width and length of the target are not changed, only the width y1 of the target is changed, so the transformed width y2 is set to 1-y1, and the new label file txt corresponding to the track image after vertical rotation can be obtained;

[0109] S26, the track image, the horizontally rotated track image, and the vertically rotated track image are all stored in the total images file, and the label files corresponding to the three are stored in the total labels file. The image data in the images file and the label data in the labels file are divided into a training set train file and a validation set val file in a ratio of 8:2.

[0110] S27, selecting a target detection algorithm suitable for detecting spring bars and splints;

[0111] In this embodiment, the deep learning-based algorithms in the aforementioned target detection algorithms include but are not limited to: FasterR-CNN, YOLO and SSD; in this embodiment, the deep learning algorithm performs well in the target detection task due to its ability to learn complex features; in this embodiment, YOLOv5 is selected as the target detection algorithm after repeated comparisons;

[0112] In this embodiment, the core idea of ​​the YOLO series algorithm is to convert the target detection task into a single forward propagation problem, which can detect objects, positions and categories in images in real time; the YOLOv5s used in this embodiment is a lightweight model in the YOLOv5 target detection series, which has achieved a balance in speed, accuracy and lightweight, and is suitable for various real-time target detection scenarios;

[0113] like Figure 7As shown, in this embodiment, the YOLOv5s network structure includes but is not limited to: an input end, a Backbone layer, a Neck network, and an output end Head part; in this embodiment, the operations performed in the input end include but are not limited to: data enhancement, adaptive anchor boxes, and adaptive image scaling, which solve the problems of uneven distribution of data sets, optimal anchor box calculation, and removal of redundant information, and improve the speed and accuracy of the model;

[0114] In this embodiment, the modules of the Backbone layer include but are not limited to: Focus, Conv, C3 modules and SPPF modules, which are used to extract feature information of the input image to form a feature map; in this embodiment, the core function of the Focus module is to slice the input image and pass it to the next convolution Conv module; the C3 module draws on the CSPNet structure, and the C3 module includes but is not limited to: 3 standard convolution layers, multiple Bottleneck modules and splicing modules, which is the core module of Backbone feature extraction; in this embodiment, the SPPF is improved by spatial pyramid pooling, and the maximum pooling is used to realize the fusion of local information and global information of the feature map; Neck network The network is a double-tower structure network composed of a feature pyramid and a path aggregation network, which is used to further process and fuse the features extracted by the Backbone layer to enhance the diversity of target feature information and the robustness of target prediction; in this embodiment, the feature pyramid network FPN can fuse feature information of different depths within the network to improve the detection effect of small targets; the path aggregation network PANet promotes the information flow in the instance segmentation framework based on the region proposal method, realizes the fusion of feature information at different levels, and improves the feature extraction ability of the network; the Neck network combines the two to realize parameter aggregation between different backbone layers and different detection layers to enhance the hierarchical structure of the entire feature map, shortening the information path between the lower layer and the top-level features;

[0115] In this embodiment, the Head part is used to classify and predict the features extracted by the Backbone after Neck compression and fusion, and the position of the candidate box is corrected to obtain a more accurate prediction result; in this embodiment, YOLOv5s can use, for example, CIOULoss as a regression loss function, and finally use weighted non-maximum suppression to screen the target box;

[0116] S3, after detecting the spring bars and the splints, obtaining the coordinate information of the spring bars and the splints;

[0117] like Figure 8 As shown, in this embodiment, the step S3 of obtaining the coordinate information of the spring bar and the clamping plate further includes the following specific steps:

[0118] S31, according to the output result of the target detection algorithm, the position information of the spring bar and the splint in the track brightness map can be obtained;

[0119] In this embodiment, the positions of the spring bars and the splints in the brightness image can be detected by the target detection algorithm YOLO V5s algorithm;

[0120] S32, the rectangular frame of the elastic bar is stored in rect1, and four data are stored in rect1, namely, the horizontal coordinate of the upper left corner of the elastic bar rectangular frame, the vertical coordinate of the upper left corner of the elastic bar rectangular frame, the width of the elastic bar rectangular frame, and the length of the elastic bar rectangular frame;

[0121] S33, the rectangular frame of the splint is stored in rect2, and four data are stored in rect2, namely, the horizontal coordinate of the upper left corner of the splint rectangular frame, the vertical coordinate of the upper left corner of the splint rectangular frame, the width of the splint rectangular frame, and the length of the splint rectangular frame;

[0122] In the image processing and computer vision of this embodiment, ROI means a specific area in an image, which includes but is not limited to: targets and information of interest to users and algorithms; in this embodiment, the purpose of extracting or selecting ROI is to focus attention or conduct further analysis, so as to reduce the amount of calculation when processing large images or focus on specific tasks;

[0123] In this embodiment, the ROI may be, for example, any part of an image, including but not limited to an object, a region, a structure, and other features of interest. In a specific application scenario of this embodiment, the ROI is extracted so as to focus on a specific region in subsequent analysis, thereby improving the efficiency and accuracy of the algorithm.

[0124] In the image processing operation of this embodiment, the method of extracting ROI includes but is not limited to: threshold processing, edge detection and segmentation algorithm; in this embodiment, when the ROI is obtained, the current area is further analyzed, processed and measured. In this embodiment, in computer vision, ROI can be, for example, a specific object; the selection of ROI can depend on, for example, specific applications and task requirements;

[0125] S4, judging whether the spring bar and the splint intersect according to the acquired coordinate information of the spring bar and the splint;

[0126] like Fig. 9 As shown, in this embodiment, step S4 of determining whether the spring bar and the splint intersect according to the acquired coordinate information of the spring bar and the splint, further includes the following specific steps:

[0127] S41, traversing step S3 to obtain the stored position information of the spring bars and the splints, so as to obtain the position information of the spring bars and the splints corresponding to each image;

[0128] In this embodiment, the position information of the spring bar and the splint in each track image can be obtained by the target detection algorithm YOLO V5s, and the position information mainly includes: label name (that is, spring bar or splint), the horizontal coordinate of the upper left corner of the rectangular box, the vertical coordinate of the upper left corner of the rectangular box, the width of the rectangular box, and the length of the rectangular box;

[0129] S42, input the spring bar position information (the horizontal coordinate x1 of the upper left corner of the spring bar rectangular frame, the vertical coordinate y1 of the upper left corner of the spring bar rectangular frame, the width w1 of the spring bar rectangular frame, and the height h1 of the spring bar rectangular frame) and the clamping plate position information (the horizontal coordinate x2 of the upper left corner of the clamping plate rectangular frame, the vertical coordinate y2 of the upper left corner of the clamping plate rectangular frame, the width w2 of the clamping plate rectangular frame, and the height h2 of the clamping plate rectangular frame) in each track image into the following rule to determine how many positions of spring bars and clamping plates in a track image intersect;

[0130] S43, setting an intersection rule algorithm to determine whether the spring bar and the splint intersect;

[0131] In this embodiment, the information passed in S42 is used to determine whether the spring bar and the splint intersect. In this embodiment, the intersection rule algorithm is as follows:

[0132] x1 <x2+w2

[0133] x1+w1>x2

[0134] y1 <y2+h2

[0135] y1+h1>y2

[0136] like Fig.10 As shown, in this embodiment, the red one is the elastic bar rectangular frame, the green one is the clamping plate rectangular frame, and there are four rectangular frames in total, which intersect with each other.

[0137] In this embodiment, the intersecting plates and spring bars in the track image are determined by the position information of the spring bars and the clamps, and the positions of the intersecting spring bars and the clamps can be accurately located, avoiding interference from other spring bars and the clamps, and preparing for the next step.

[0138] S5. If they intersect, calculate the two ordinate differences △h1 and △h2 of the spring bar and the clamping plate;

[0139] like Fig.11 As shown, in this embodiment, calculating the two longitudinal coordinate differences △h1 and △h2 of the spring bar and the clamping plate also includes the following specific steps:

[0140] S51, obtaining position information of the spring bar and the splint where each track image intersects, which are the ordinate y1 of the spring bar, the height h1 of the spring bar, the ordinate y2 of the splint, and the height h2 of the splint;

[0141] In this embodiment, the position information of the intersecting spring bars and clamping plates is output, mainly including the longitudinal coordinate y1 of the spring bars, the height h1 of the spring bars, the longitudinal coordinate y2 of the clamping plates, and the height h2 of the clamping plates;

[0142] S52, calculating the differences △h1 and △h2 between the longitudinal coordinates of the intersecting spring bars and the clamping plates;

[0143] In this example, the differences △h1 and △h2 between the longitudinal coordinates of the intersecting spring bar and the splint can be calculated according to the longitudinal coordinate y1 of the intersecting spring bar and the splint, the height h1 of the spring bar, the longitudinal coordinate y2 of the splint, and the height h2 of the splint;

[0144] S53, △h1 is the absolute value of the difference between the ordinates of the spring bar and the splint, △h2 is the absolute value of the difference between the ordinate of the spring bar plus the corresponding height and the ordinate of the splint plus the corresponding height;

[0145] like Fig.11 As shown, in this embodiment, the absolute value is used to better represent the distance between the spring bar and the clamping plate.

[0146] S6, obtaining the minimum value △h between △h1 and △h2;

[0147] S7, setting a determination threshold Th, and determining whether the spring bar is displaced according to the magnitude relationship between Δh and Th;

[0148] like Fig.12 As shown, in this embodiment, the aforementioned determination threshold Th represents the threshold of the spring bar displacement; the setting of the threshold can be determined according to the specific application scenario, the spring bar displacement property and the system requirements, and is usually a predefined value;

[0149] In this embodiment, if Δh>Th, it means that the minimum value of the distance between the spring bar and the clamping plate is greater than the threshold value, which means that the spring bar is displaced; if Δh≤Th, it is determined that the spring bar is not displaced;

[0150] In this embodiment, by reading the position information of the intersecting spring bars and clamps, the absolute values ​​△h1 and △h2 of the differences between the corresponding vertical coordinates of the spring bars and clamps can be calculated to realize the spatial position deviation relationship of the intersecting spring bars and clamps in the track image.

[0151] In this embodiment, by obtaining the minimum value △h between the absolute values ​​△h1 and △h2 of the difference between the longitudinal coordinates of the spring clip, and setting the determination threshold Th, by comparing the values ​​of △h and Th, a reliable determination of whether the spring clip is displaced is achieved.

[0152] S8, drawing the displacement of the spring clip and displaying the information according to the position information of the spring clip target;

[0153] like Fig.13As shown, in this embodiment, the spring clip displacement drawing and information display are performed according to the position information of the spring clip target, and the following specific steps are also included:

[0154] S81, spring bar displacement positioning;

[0155] like Fig.14 As shown, in this embodiment, according to the comparison between the displacement between the intersecting elastic strip and the clamping plate and the threshold value, it can be determined that the intersecting elastic strip has been displaced, and according to the position information of the elastic strip, the elastic strip position information mainly includes the horizontal coordinate x1, the vertical coordinate y1, the elastic strip rectangular frame width w1 and the elastic strip rectangular frame height h1 of the intersecting elastic strip, and a rectangular frame is drawn for the above-mentioned elastic strip with a line thickness of 2 and a red color, and the corresponding position of the elastic strip displacement is obtained, thereby realizing the displacement positioning of the elastic strip.

[0156] S82, drawing defect information on the location of the spring bar displacement;

[0157] In this embodiment, the horizontal coordinate of the elastic bar shift is kept unchanged, the vertical coordinate is moved upward by 4 pixels, the character name is "ttyw", the color is white, the line thickness is 2, and the "ttyw" information is drawn on the drawn elastic bar shift rectangle.

[0158] In summary, the present invention combines target detection technology, detects the spatial coordinate information of the spring clip and the splint, and realizes the detection and positioning of the spring clip displacement according to the spatial coordinate information deviation, thereby improving the detection accuracy and reliability of the rail spring clip displacement. The spring clip displacement detection and positioning method based on the target spatial position deviation of the present invention can provide a new solution for the accurate detection of the rail spring clip displacement problem.

[0159] The present invention does not change the fastener structure, does not require a large number of spring bar displacement samples, and its detection accuracy and detection speed can well meet the standards for rail track spring bar displacement detection.

[0160] The present invention mainly uses the YOLO V5s algorithm to locate the spring clip and the splint, obtains the spatial coordinate information of the two, improves the detection speed and accuracy, and maintains sufficient shape information, which is helpful for further analysis and processing. The present invention combines the target detection technology and the spatial position deviation technology to make the spring clip displacement detection method real-time, which is suitable for real-time monitoring of the displacement state of the rail spring clip, and provides an effective means for track maintenance and safe operation.

[0161] Before target detection, the present invention performs horizontal and vertical rotations on the track image acquired by the laser profiler, and processes the corresponding label file to obtain the corresponding horizontally rotated label file and the vertically rotated label file; so as to expand the data set and reduce the labeling time; this may include operations such as labelimg labeling, image rotation, and label file processing, which can ensure that the track image and the label file are expanded correctly, and the track image and the label file can be matched one by one.

[0162] The present invention adopts the target detection model YOLOv5s, which strikes a balance in terms of speed, accuracy and lightness, and is suitable for various real-time target detection scenarios. The model input end of the present invention includes three functions: data enhancement, adaptive anchor frame and adaptive image scaling, which mainly solve the problems of uneven distribution of data sets, optimal anchor frame calculation and removal of redundant information, and improve the speed and accuracy of the model.

[0163] The present invention determines the position information of the spring bar and the splint in the brightness image in the form of a bounding box; the coordinate information of the bounding box can accurately describe the position of the spring bar and the splint in the brightness image. The present invention avoids the requirement on the number of negative samples (track images with spring bar displacement) by extracting the coordinate information of the ROI area of ​​the spring bar and the splint in the track image, and can accurately restore the position of the spring bar and the splint.

[0164] The present invention can accurately and quickly determine the intersecting spring bars and splints in each track image, and can obtain the position information of the intersecting spring bars and splints.

[0165] The present invention obtains the position information of the spring bar in the spring bar displacement, uses the horizontal and vertical coordinates of the upper left corner of the spring bar target frame and the width and length of the target frame, can accurately locate the spring bar displacement, and draw the defect information on the located rectangular frame, thereby realizing the accurate positioning of the spring bar displacement and the marking of the defect information.

[0166] The present invention solves the technical problems existing in the prior art of low accuracy, reliability and detection efficiency of rail spring clip displacement detection and excessive reliance on railway rail spring clip displacement defect data.

[0167] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A rail clip displacement detection method based on target spatial position deviation, characterized in that: The method comprises: S1, obtain the track brightness map through the laser profiler; S2, performing data enhancement processing on the track brightness image, using a target detection algorithm to detect spring bars and splints in the track brightness image, and obtaining spring bar and splint detection data; S3, processing and obtaining the coordinate information of the spring bars and the splints according to the detection data of the spring bars and the splints; S4, judging whether the spring bar and the splint intersect with each other by using an intersection rule algorithm according to the coordinate information of the spring bar and the splint; S5, if yes, then calculate the first ordinate difference △h1 and the second ordinate difference △h2 between the spring bar and the clamping plate, and obtain the intersection position information of the spring bar and clamping plate of each track brightness image; S6, obtaining a minimum value △h between the first ordinate difference △h1 and the second ordinate difference △h2; S7, setting a determination threshold Th, comparing the minimum value Δh with the determination threshold Th, determining whether the spring bar is displaced, and obtaining target position information of the displaced spring bar; S8. Drawing the displacement of the spring strip and displaying the information are performed according to the target position information of the displacement spring strip.

2. The rail spring bar displacement detection method based on target spatial position deviation according to claim 1 is characterized in that: The S2 includes: S21, horizontally rotating the track brightness map to obtain a horizontally rotated track brightness map, and enhancing image data of the track brightness map; S22, vertically rotating the track brightness map to obtain a vertically rotated track brightness map, thereby enhancing image data of the track brightness map to obtain an image enhanced track brightness map; S23, using a preset annotation tool, performing an image annotation operation on the spring bar and the splint in each track brightness image, and obtaining a corresponding label txt file of each track brightness image; wherein, creating a label according to a preset target task, performing an image annotation operation, and converting the annotation information of the preset target task into a txt file format required for model training; S24, obtaining a new label file txt for the corresponding label file txt; S25, setting the width of the vertical rotation track brightness map, and obtaining the new label file txt corresponding to the vertical rotation track brightness map; S26, storing the image enhancement track brightness map, the horizontal rotation track brightness map and the treatment rotation track brightness map into a preset images file, and storing the corresponding label file txt and the new label file txt into a labels file; S27. For the spring bar and the splint, select the target detection algorithm; wherein the target detection algorithm includes: FasterR-CNN, YOLOv5s and SSD; the network structure of the YOLOv5s includes: an input end, a Backbone layer, a Neck network and an output end Head.

3. The rail spring bar displacement detection method based on target spatial position deviation according to claim 2 is characterized in that: In S26, the image data in the preset images file and the label data in the labels file are divided to obtain a training set train file and a validation set val file.

4. The rail spring bar displacement detection method based on target spatial position deviation according to claim 2 is characterized in that: The Backbone layer in S27 includes: Focus, Conv, C3 modules and SPPF modules, and the Backbone layer is used to extract feature information of the input image to form a global and local fusion feature map; Using the Focus module, the input image is sliced ​​to obtain and transfer the slice data to the next convolution Conv module; The C3 module includes: a standard convolutional layer, a Bottleneck module and a splicing module, and the C3 module is used to perform feature extraction in the Backbone layer; Pooling is performed through a spatial pyramid to obtain the SPPF module, and local information and global information of the feature map are fused through maximum pooling.

5. The rail clip displacement detection method based on target spatial position deviation according to claim 2 is characterized in that: The Neck network in S27 includes: a feature pyramid FPN and a path aggregation network PANet, which are used to perform a re-fusion operation on the global and local fusion feature maps to obtain a hierarchical structure enhanced feature map.

6. The rail clip displacement detection method based on target spatial position deviation according to claim 2 is characterized in that: In S27, the output end Head is used to classify and predict the hierarchical structure enhanced feature map, correct the position of the candidate box, and obtain a suitable prediction result.

7. The rail clip displacement detection method based on target spatial position deviation according to claim 1 is characterized in that: The S3 includes: S31, processing and obtaining the position information of the spring bars and the splints according to the detection data of the spring bars and the splints; S32, using rect1, storing the rectangular frame of the elastic bar; S33, using rect2, storing the rectangular frame of the splint, and extracting ROI based on the rectangular frame.

8. The rail clip displacement detection method based on target spatial position deviation according to claim 1 is characterized in that: The S4 includes: S41, traversing step S3 to obtain the stored position information of the spring bars and the splints, so as to obtain the position information of the spring bars and the splints corresponding to each image; S42, using the intersection rule algorithm, judging whether the spring bar and the splint intersect with the following logic: x1 <x2+w2 x1+w1>x2 y1 <y2+h2 y1+h1>y2 In the formula, x1 is the horizontal coordinate of the spring bar, y1 is the vertical coordinate of the spring bar, h1 is the height of the spring bar, x1 is the horizontal coordinate of the splint, y2 is the vertical coordinate of the splint, h2 is the height of the splint, w1 is the width of the rectangular frame of the spring bar, and w2 is the width of the rectangular frame of the splint.

9. The rail clip displacement detection method based on target spatial position deviation according to claim 1, characterized in that: The S8 includes: S81, performing a shift positioning operation on the elastic bar that sends the shift, drawing a shift rectangular frame for the elastic bar, so as to obtain corresponding elastic bar shift positioning data; S82. Drawing and displaying spring bar defect information according to the spring bar displacement positioning data.

10. A rail clip displacement detection system based on target spatial position deviation, characterized in that: The system comprises: Track image acquisition module, used to acquire track brightness map through laser profilometer; A target detection and data set construction module, used for performing data enhancement processing on the track brightness map, detecting the spring bars and splints in the track brightness map by using a target detection algorithm, and obtaining the spring bars and splints detection data, wherein the target detection and data set construction module is connected to the track image acquisition module; A position information acquisition module, used to process the spring bar and splint detection data to obtain the spring bar and splint coordinate information, and the position information acquisition module is connected to the target detection and data set construction module; A target intersection judgment module, used to judge whether the spring bar and the splint intersect according to the coordinate information of the spring bar and the splint by using an intersection rule algorithm, and the target intersection judgment module is connected to the position information acquisition module; An intersection target difference calculation module is used to calculate the first ordinate difference △h1 and the second ordinate difference △h2 of the spring bar and the splint when the spring bar intersects with the splint, and obtain the intersection position information of the spring bar and the splint of each track brightness map. The intersection target difference calculation module is connected to the target intersection judgment module; A minimum difference determination module, used for obtaining the minimum value △h between the first ordinate difference △h1 and the second ordinate difference △h2, wherein the minimum difference determination module is connected to the intersection target difference calculation module; The spring bar displacement determination module sets a determination threshold Th, compares the minimum value △h with the determination threshold Th, determines whether the spring bar is displaced, and obtains the target position information of the displaced spring bar. The spring bar displacement determination module is connected to the difference minimum value determination module; The spring bar displacement position positioning and drawing module is used to draw the spring bar displacement and display the information according to the displacement spring bar target position information. The spring bar displacement position positioning and drawing module is connected to the spring bar displacement determination module.

Images