Rail fastener spring strip ectopic visual detection method, system and device

Abstract

The application provides a track fastener spring strip ectopic visual detection method, system and device, relates to the track fastener detection technical field, and the method comprises collecting track panoramic overhead view; according to the standard fastener spacing of pre-set, track panoramic overhead view is cut into unit fastener image;According to the order of fastener in track panoramic overhead view, each fastener is numbered;Based on unit fastener image, through spring strip target detection model, spring strip is identified and position is marked, and anchor frame area is obtained;Based on anchor frame area, through spring strip semantic segmentation model, pixel-level segmentation is carried out, and spring strip mask is obtained;Through computer vision function library, the geometric analysis of spring strip mask is carried out, and the rotation angle of spring strip is obtained.The scheme can accurately calculate the rotation angle of spring strip, so that the severity of ectopia can be intuitively and accurately evaluated by railway maintenance personnel, and then maintenance strategy can be scientifically formulated, and the scientificity and accuracy of maintenance decision are greatly improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of rail fastener detection, in particular to a rail fastener spring strip ectopic visual detection method, system and device. BACKGROUND

[0002] With the rapid development of rail transit, the mileage of railways continues to grow. As a key infrastructure of railways, the rail fastener system is prone to performance degradation under the complex operating conditions of high-frequency wheel-rail friction, vibration and impact of trains, such as fastener loss, breakage, loosening, skewing, spring strip ectopia or fatigue fracture, etc. These defects not only affect the comfort of riding, but also seriously threaten the safety and stability of train operation, and even may cause derailment. Traditional rail fastener detection mainly relies on manual inspection, but this approach has many drawbacks. Manual inspection is labor-intensive, inefficient, and easily influenced by subjective judgment, and is difficult to meet the high-frequency and high-quality detection needs under the conditions of shortened detection window period and complex geological environment, and even poses a risk to personal safety. Especially for high-speed rail, timely detection and control of fastener defects are crucial to ensure safe operation. Therefore, there is an urgent need to develop more advanced, efficient and intelligent automated detection methods to overcome the limitations of manual inspection and ensure the safety of railway operation.

[0003] In recent years, deep learning technology has brought breakthroughs in high-speed rail fastener image detection. In the paper "High-speed rail fastener spring strip defect detection based on improved Faster R-CNN", the author proposes a spring strip defect detection method based on improved Faster R-CNN. This method extracts defect features through a multi-layer convolutional neural network, aiming to improve the network's attention to defect features and reduce the influence of environmental interference and imaging positioning deviation. The process is as follows: first, extract the defect feature map through a multi-layer convolutional neural network; second, design a region candidate network to generate candidate regions and perform pooling to extract specific defect locations; finally, use the fully connected layer of the region proposal network (RPN) to calculate the defect class and precise location. This method can effectively suppress environmental interference, enhance the representation ability of defect features, and simplify the image preprocessing process. The shortcomings of the above scheme are as follows:

[0004] (1) Cannot realize quantitative analysis of ectopic degree. The method in the paper mainly focuses on defect detection and classification, i.e., determining whether the spring strip has defects and its class. However, for the specific defect of "ectopia" of the spring strip, the method can only identify the presence of ectopia, but cannot provide accurate quantitative data of the ectopic degree, such as the specific rotation angle. This limits its application in fine maintenance and fault warning.

[0005] (2) The detection accuracy of micro-ectopia is limited: Although this method claims to suppress environmental interference and enhance defect feature representation, the detection framework based on the region proposal network may be limited by the granularity of feature extraction and region positioning when dealing with micro-ectopia or slight deformation, resulting in insufficient detection accuracy. SUMMARY

[0006] The present application aims to provide a rail fastener spring strip ectopia visual detection method, system and device to solve at least one of the above technical problems in the prior art.

[0007] In a first aspect, to solve the above technical problems, the present application provides a rail fastener spring strip ectopia visual detection method, comprising the following steps:

[0008] Step 1, collect the rail panoramic overhead view.

[0009] In a feasible implementation, step 1 specifically comprises:

[0010] Step 11, based on the two cameras (set at the lower part of the front end of the rail detection vehicle), the rail is continuously imaged by scanning shooting mode to obtain scanning images, so as to ensure that there is sufficient overlap between adjacent scanning images, thereby facilitating high-precision image stitching;

[0011] Step 12, by SIFT (Scale-Invariant Feature Transform) algorithm, the feature points (corner points or texture features) in the scanning images are extracted, the corresponding relationship between the same feature points in different scanning images is established, and the geometric transformation parameters are calculated;

[0012] Step 13, based on the geometric transformation parameters, the scanning images are corrected to eliminate the image deformation and misplacement caused by camera movement and lens distortion;

[0013] Step 14, the scanning images are spliced and fused (seamlessly) to eliminate the brightness difference and splicing gap between the scanning images, and the rail panoramic overhead view is obtained.

[0014] Step 2, according to the preset standard fastener spacing, the rail panoramic overhead view is cropped into a series of unit fastener images; according to the order of the fasteners in the rail panoramic overhead view, each fastener is numbered (uniquely) to facilitate subsequent fault tracking and positioning management.

[0015] Step 3, based on the unit fastener image, the spring strip target detection model is used to identify the spring strip and mark the position to obtain the anchor box region.

[0016] In a feasible implementation, the spring strip target detection model uses an image target detection model to ensure detection speed and accuracy.

[0017] In an embodiment, the specific construction method of the training data set of the elastic strip target detection model includes: first, cutting unit fastener images from the actually collected track panoramic overhead view, which includes not only normal state elastic strip images but also abnormal state elastic strip images; the abnormal state includes elastic strip misplacement, elastic strip (slight) deformation and elastic strip damage, etc.; this can enhance the diversity of the training data set, improve the generalization ability and robustness of the model; then, the elastic strip in each unit fastener image is labeled with a bounding box, the rectangular area of the elastic strip is selected and the corresponding class label (i.e. the elastic strip in different states) is assigned.

[0018] In an embodiment, the specific training method of the elastic strip target detection model includes: inputting the unit fastener images in the training data set into the elastic strip target detection model to identify the elastic strip and mark its (approximate) position by an anchor box; based on the back propagation algorithm and the optimizer, adjusting the model parameters (weights and biases) according to the loss function between the prediction result and the true label; through iterative training, obtaining an elastic strip target detection model that can efficiently and accurately identify the elastic strip and mark its position; this can provide accurate region proposals for subsequent refined semantic segmentation, avoiding the low efficiency problem when performing pixel-level processing on the whole image.

[0019] In an embodiment, the identification method of the elastic strip target detection model includes: first, detecting the input image; if no elastic strip is detected, directly generating the missing information of the elastic strip; if the elastic strip is detected, outputting an anchor box that can surround the elastic strip according to the (approximate) position of the elastic strip to determine the anchor box region.

[0020] Step 4, based on the anchor box region, performing pixel-level segmentation by the elastic strip semantic segmentation model to obtain a (high-precision) elastic strip mask; unlike the traditional detection box, the elastic strip mask can provide accurate contour information of the elastic strip, including its non-standard shape and slight deformation, thereby providing sufficient real geometric shape information for subsequent elastic strip misplacement accurate quantitative analysis.

[0021] In an embodiment, the elastic strip semantic segmentation model takes U-Net as the basic framework, and introduces channel attention mechanism and spatial attention mechanism;

[0022] The channel attention mechanism is used to learn the weights of different feature channels, thereby enhancing the response to key features;

[0023] The spatial attention mechanism is used to focus on the key spatial positions in the anchor box region.

[0024] In an embodiment, the specific construction method of the training data set of the elastic strip semantic segmentation model comprises the following steps: first, unit fastener images are cut from the actually collected track panoramic overhead view, which includes not only normal state elastic strip images but also abnormal state elastic strip images; the abnormal state includes elastic strip misplacement, elastic strip (slight) deformation and elastic strip damage, etc.; this can enhance the diversity of the training data set, improve the generalization ability and robustness of the model; then, pixel-level mask annotation is performed on the elastic strip in each unit fastener image, and each pixel point is classified as an elastic strip or a background, so as to outline the accurate contour of the elastic strip.

[0025] In an embodiment, the specific training method of the elastic strip semantic segmentation model comprises the following steps: the pixel-level mask annotation data in the training data set is input into the elastic strip semantic segmentation model to learn how to classify each pixel as an elastic strip or a background; the training target is to minimize the difference between the predicted elastic strip mask and the real elastic strip mask, and the cross-entropy loss is used as the loss function; through iterative training, an elastic strip semantic segmentation model capable of performing pixel-level segmentation on the elastic strip in the anchor box region is obtained, thereby providing real geometric shape information for subsequent accurate quantitative analysis of elastic strip misplacement.

[0026] Step 5, geometric analysis of the elastic strip mask is performed through a computer vision (CV) function library to obtain the rotation angle of the elastic strip.

[0027] In an embodiment, the specific method of geometric analysis comprises the following steps:

[0028] Step 51, pixel points representing the elastic strip boundary are extracted from the elastic strip mask to form a (continuous) elastic strip contour line;

[0029] Step 52, the smallest circumscribed rectangle enclosing the elastic strip contour line is calculated and fitted; in this way, not only the minimum occupied space of the elastic strip can be provided, but also the actual posture of the elastic strip can be reflected through the long side direction;

[0030] Step 53, the included angle between the long side of the smallest circumscribed rectangle and the preset standard horizontal axis (or standard fastener main shaft) is calculated to obtain the rotation angle of the elastic strip.

[0031] In a second aspect, based on the same inventive concept, the application also provides a track fastener elastic strip misplacement visual detection system, which comprises a data receiving module, a data processing module and a result generating module.

[0032] The data receiving module is configured to receive a track panoramic overhead view.

[0033] The data processing module comprises a unit fastener unit, an anchor frame unit, a spring strip mask unit and a rotation angle unit.

[0034] The unit fastener unit is configured to crop a track panoramic top view into a unit fastener image according to a preset standard fastener spacing, and number each fastener according to the order of the fastener in the track panoramic top view.

[0035] The anchor frame unit is configured to identify a spring strip and mark a position based on the unit fastener image by using a spring strip target detection model, and obtain an anchor frame region.

[0036] The spring strip mask unit is configured to perform pixel-level segmentation on the anchor frame region by using a spring strip semantic segmentation model, and obtain a spring strip mask.

[0037] The rotation angle unit is configured to perform geometric analysis on the spring strip mask by using a computer vision function library, and obtain a rotation angle of the spring strip.

[0038] The result generation module is configured to send out a spring strip ectopia detection result, which comprises missing information and a rotation angle of the spring strip.

[0039] In a third aspect, based on the same inventive concept, the present application also provides a track fastener spring strip ectopia visual detection device, which comprises a processor, a memory and a bus, the memory stores instructions and data read by the processor, the processor is configured to call the instructions and data in the memory to execute the track fastener spring strip ectopia visual detection method as described above, and the bus is connected between each functional component for transmitting information.

[0040] In a feasible implementation, the device further comprises a camera arranged at the lower part of the front end of the track detection vehicle, which is configured to collect a track panoramic top view.

[0041] By using the above technical solution, the present application has the following beneficial effects:

[0042] The track fastener spring strip ectopia visual detection method, system and device provided by the present application can accurately extract the complete contour of the spring strip by introducing a pixel-level semantic segmentation model, and can accurately calculate the rotation angle of the spring strip by combining computer vision geometric analysis. For example, the traditional method can only identify the "spring strip ectopia", but the present application can specifically quantify it as "the spring strip rotates 2 degrees clockwise". This quantitative analysis enables railway maintenance personnel to intuitively and accurately evaluate the severity of ectopia, so that scientific maintenance strategies can be developed, such as repairing seriously ectopic fasteners first or predictive maintenance according to the rotation trend, which greatly improves the scientificity and accuracy of maintenance decisions.

[0043] By adopting the pixel-level semantic segmentation technology, the present scheme can finely identify each pixel belonging to the spring strip, thereby depicting the real and irregular shape of the spring strip. Even if the spring strip only has a slight angular tilt, the change of its contour can be accurately captured by the semantic segmentation model. For example, a slightly skewed spring strip may still be considered normal in target detection, but semantic segmentation can clearly depict its contour deviating from the standard shape. Based on such high-precision contour information, subsequent geometric analysis can sensitively perceive and quantify these minor angular deviations. This greatly improves the system's detection capability for early and slight faults, helping to achieve earlier intervention and avoid small problems from evolving into serious safety hazards. BRIEF DESCRIPTION OF DRAWINGS

[0044] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the drawings needed in the following description of the specific embodiments or the prior art will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0045] Figure 1 A flow chart of a rail fastener spring strip deviation visual detection method provided for an embodiment of the present application;

[0046] Figure 2 A spring strip deviation example diagram provided for an embodiment of the present application;

[0047] Figure 3 A spring strip missing example diagram provided for an embodiment of the present application;

[0048] Figure 4 A detection illustration color diagram of a spring strip target detection model provided for an embodiment of the present application;

[0049] Figure 5 A segmentation illustration diagram of a spring strip semantic segmentation model provided for an embodiment of the present application;

[0050] Figure 6 A spring strip mask example diagram in a spring strip missing state provided for an embodiment of the present application; wherein, a diagram is a spring strip missing example diagram; b diagram is a corresponding spring strip mask example diagram;

[0051] Figure 7 A spring strip mask example diagram in a spring strip deviation state provided for an embodiment of the present application; wherein, a diagram is a spring strip deviation example diagram; b diagram is a corresponding spring strip mask example diagram;

[0052] Figure 8 A rotation angle illustration diagram provided for an embodiment of the present application;

[0053] Figure 9A diagram of a rail fastener spring bar misalignment visual detection system provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0055] In the description of the present invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely to facilitate and simplify the description of the present invention. They are not intended to indicate or imply that the devices or components referred to must have, be constructed, or operate in a specific orientation, and therefore should not be construed as limitations on the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0056] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed, detachable, or integral connections; mechanical or electrical connections; direct or indirect connections through an intermediate medium; and internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on the specific circumstances.

[0057] The present invention will be further explained below with reference to specific embodiments.

[0058] It should also be noted that the following specific embodiments or specific implementations are a series of optimized settings listed in the present invention to further explain the specific content of the invention, and these settings can be combined or used in association with each other.

[0059] Example 1:

[0060] like Figure 1 As shown, this embodiment provides a method for visually detecting misalignment of rail fastener spring bars, comprising the following steps:

[0061] Step 1: Collect a panoramic bird's-eye view of the track.

[0062] Furthermore, the step 1 specifically includes:

[0063] Step 11, based on the two cameras (set at the lower part of the front end of the track detection vehicle), the track is continuously imaged by a scanning shooting method to obtain a scanning image, so as to ensure that there is sufficient overlap between adjacent frames of scanning images, thereby facilitating high-precision image stitching.

[0064] Step 12, by using SIFT (Scale-Invariant Feature Transform) algorithm or the like, feature points (corner points or texture features) in the scanning image are extracted, a corresponding relationship between the same feature points in different scanning images is established, and geometric transformation parameters are calculated.

[0065] Step 13, based on the geometric transformation parameters, the scanning image is corrected in order to eliminate the image deformation and misplacement caused by camera movement (such as shaking, lateral displacement, pitch and yaw, etc.) and lens distortion.

[0066] Step 14, the scanning image is stitched and fused (seamlessly) to eliminate the brightness difference between the scanning images and the stitching gap, and an overall panoramic view of the track is obtained.

[0067] Step 2, according to the preset standard fastener spacing, the overall panoramic view of the track is cropped into a series of unit fastener images; each fastener is numbered (uniquely) according to the order of the fastener in the overall panoramic view of the track, so as to facilitate subsequent fault tracking and positioning management.

[0068] Step 3, based on the unit fastener image, a spring strip target detection model is used to identify the spring strip and mark the position, and an anchor box area is obtained.

[0069] Further, the spring strip target detection model uses an image target detection model (such as YOLOv11-s) to ensure the detection speed and accuracy.

[0070] Further, the specific construction method of the training data set of the spring strip target detection model includes: first, unit fastener images are cropped from the actually collected overall panoramic view of the track, which include not only normal state spring strip images but also abnormal state spring strip images; the abnormal state includes spring strip misplacement, spring strip (slight) deformation and spring strip breakage, etc., as shown in the following specific example images of spring strip misplacement and spring strip missing; in this way, the diversity of the training data set can be enhanced, and the generalization ability and robustness of the model can be improved; then, the spring strip in each unit fastener image is labeled with a bounding box (Bounding Box), and the rectangular area of the spring strip is selected and assigned with a corresponding class label (i.e. the spring strip in different states). Figures 2-3

[0071] ​Further, the specific training method of the elastic strip target detection model comprises: inputting the (640*640 pixel) unit fastener image in the training data set into the elastic strip target detection model to identify the elastic strip and mark its (approximate) position through the anchor box; based on the back propagation algorithm and the optimizer, adjusting the model parameters (weights and biases) according to the loss function between the prediction result and the real label; through iterative training, obtaining the elastic strip target detection model which can efficiently and accurately identify the elastic strip and mark its position; in this way, accurate region proposals can be provided for subsequent fine semantic segmentation, avoiding the problem of low efficiency when performing pixel-level processing on the whole image.

[0072] The loss function comprises a classification loss and a regression loss.

[0073] The classification loss (Classification Loss) is used to measure the accuracy of the elastic strip target detection model in judging whether the anchor box contains an elastic strip, and specifically, a cross-entropy loss can be selected for calculation.

[0074] The regression loss (Regression Loss) is used to measure the degree of fit between the predicted anchor box and the real anchor box of the elastic strip target detection model, and specifically, it can comprise an L1 loss and / or an L2 loss and / or an IoU loss.

[0075] The L1 loss is used to calculate the absolute error between the predicted anchor box coordinates and the real anchor box coordinates.

[0076] The L2 loss is used to calculate the relative error between the predicted anchor box coordinates and the real anchor box coordinates.

[0077] The IoU loss is used to calculate the intersection over union between the predicted anchor box and the real anchor box.

[0078] Further, as shown in Figure 4 The identification method of the elastic strip target detection model comprises: first detecting the input image; if no elastic strip is detected, directly generating the missing information of the elastic strip; if the elastic strip is detected, outputting an (red) anchor box capable of enclosing the elastic strip according to the (approximate) position of the elastic strip to determine the anchor box region.

[0079] Step 4, as shown in Figure 5 Based on the anchor box region, pixel-level segmentation is performed through the elastic strip semantic segmentation model to obtain an (high-precision) elastic strip mask; unlike the traditional detection box, the elastic strip mask can provide accurate contour information of the elastic strip, including its non-standard shape and slight deformation, thereby providing sufficient real geometric shape information for subsequent elastic strip ectopic accurate quantitative analysis.

[0080] Further, the elastic strip semantic segmentation model takes U-Net as a basic framework, and introduces channel attention mechanism and spatial attention mechanism;

[0081] The channel attention mechanism is used to learn the weight of different feature channels, so as to enhance the response to key features.

[0082] The spatial attention mechanism is used to focus on key spatial positions in the anchor frame region.

[0083] Further, the specific construction method of the training data set of the elastic strip semantic segmentation model comprises the following steps: first, unit fastener images are cropped from the actually collected track panoramic overhead view, and the unit fastener images include not only normal state elastic strip images, but also abnormal state elastic strip images; the abnormal state includes elastic strip ectopia, elastic strip (slight) deformation and elastic strip defect, etc., as shown in FIG. 1, Figures 6-7 Figure 6 wherein, Figure 7 FIG. 1a is an elastic strip defect example image, Figure 6 FIG. 1b is an elastic strip mask example image in the elastic strip defect state, Figure 7 FIG. 1b is an elastic strip mask example image in the elastic strip ectopia state.

[0084] Further, the specific training method of the elastic strip semantic segmentation model comprises the following steps: the pixel-level mask annotation data in the training data set is input into the elastic strip semantic segmentation model to learn how to classify each pixel as an elastic strip or background; the training target is to minimize the difference between the predicted elastic strip mask and the real elastic strip mask, and the cross-entropy loss is used as the loss function; through iterative training, an elastic strip semantic segmentation model capable of performing pixel-level segmentation on the elastic strip in the anchor frame region is obtained, so as to provide real geometric shape information for subsequent elastic strip ectopia accurate quantitative analysis;

[0085] The cross-entropy loss can be binary cross-entropy loss , and the specific formula can be:

[0086] ;

[0087] wherein, represents the total number of pixels in the elastic strip mask image; represents the i-th pixel in the elastic strip mask image; and ​1 represents a spring, and 0 represents a background; represents a probability that the i-th pixel is a spring.

[0088] Step 5, performing geometric analysis on the spring mask by using a computer vision (CV) function library to obtain a rotation angle of the spring.

[0089] Further, the specific method of the geometric analysis comprises:

[0090] Step 51, extracting pixel points representing a boundary of the spring from the spring mask to form a (continuous) spring contour line;

[0091] Step 52, calculating and fitting a minimum circumscribed rectangle surrounding the spring contour line; in this way, not only the minimum occupied space of the spring can be provided, but also the actual posture of the spring can be reflected through the long side direction;

[0092] Step 53, calculating an included angle between a long side of the minimum circumscribed rectangle and a preset standard horizontal axis (or a standard fastener main shaft) to obtain a rotation angle of the spring, as shown in Figure 8 .

[0093] Further, in the step 52, the minimum circumscribed rectangle can be calculated by using a cv2.minAreaRect() function in the OpenCV library based on a Rotating Calipers Algorithm; in the function, the input is the spring contour line, and the output is a center point coordinate (x, y) of the minimum circumscribed rectangle, a nominal width height (width, height), and a nominal rotation angle ( ).

[0094] Further, the step 52 further comprises comparing a geometric characteristic parameter of the spring contour with a geometric characteristic parameter of a normal spring after the geometric characteristic parameter of the spring contour is calculated; if the geometric characteristic parameter of the spring contour is lower than a preset proportion, it is determined that the spring is incomplete, and the step 53 is not executed.

[0095] The geometric characteristic parameter comprises a spring contour area, a spring contour perimeter, and a circumscribed rectangle area ratio.

[0096] The circumscribed rectangle area ratio refers to a ratio between the spring contour area and an area of the minimum circumscribed rectangle.

[0097] Further, in the step 53, the specific determination method of the direction of the rotation angle comprises:

[0098] Step 531, comparing a width value and a height value of the minimum circumscribed rectangle:

[0099] ​If the width value is smaller than the height value, the width value represents the short side and the height value represents the long side;

[0100] Otherwise, the width value represents the long side and the height value represents the short side;

[0101] Step 532: If the width value is greater than the height value, The value is used as the final rotation angle; if the width value is less than the height value, The value obtained by adding 90 degrees is used as the final rotation angle;

[0102] Step 533: When the minimum circumscribed rectangle is in a horizontal (or vertical) state, the rotation angle is defined as 0 degrees, and the major axis of the minimum circumscribed rectangle is defined as the standard reference axis;

[0103] Step 534: Based on the standard reference axis, The value is corrected by angle conversion to ensure The value always represents the angle between the long side and the standard reference axis and is determined:

[0104] like A positive value indicates that the spring bar rotates counterclockwise;

[0105] like A negative value means the spring rotates clockwise.

[0106] Example 2:

[0107] like Figure 9 As shown, this embodiment provides a rail fastener spring bar misalignment visual detection system, including a data receiving module, a data processing module and a result generating module;

[0108] The data receiving module is used to receive the panoramic bird's-eye view of the track;

[0109] The data processing module includes a unit fastener unit, an anchor frame unit, a spring bar mask unit and a rotation angle unit;

[0110] The unit fastener unit is used to crop the track panoramic top view into unit fastener images according to a preset standard fastener spacing; and number each fastener according to the order of the fasteners in the track panoramic top view;

[0111] The anchor frame unit is used to identify the spring clip and mark its position based on the unit fastener image through the spring clip target detection model to obtain the anchor frame area;

[0112] The spring bar mask unit is used to perform pixel-level segmentation based on the anchor frame area using the spring bar semantic segmentation model to obtain the spring bar mask;

[0113] The rotation angle unit is configured to perform geometric analysis on the spring strip mask by using a computer vision function library to obtain the rotation angle of the spring strip.

[0114] The result generation module is configured to send out the spring strip ectopia detection result, which includes the missing information and the rotation angle of the spring strip.

[0115] Embodiment three:

[0116] The embodiment provides a track fastener spring strip ectopia visual detection device, which comprises a processor, a memory and a bus. The memory stores instructions and data read by the processor. The processor is configured to call the instructions and data in the memory to execute the track fastener spring strip ectopia visual detection method described above. The bus is connected between various functional components for transmitting information.

[0117] Further, the device further comprises a camera arranged at the lower part of the front end of the track detection vehicle, which is configured to collect a track panoramic overhead view.

[0118] In another embodiment, the device can be realized in the form of an integrated device, which can include one or more hardware modules specially configured to perform the corresponding steps, or realized by a processor configured to perform the corresponding steps, or stored in a computer readable medium for implementation by a processor, or realized by some combination.

[0119] The processor executes the various methods and processes described above. For example, the method embodiments in the present scheme can be realized as a software program, which is tangibly contained in a machine readable medium, such as a memory. In some embodiments, part or all of the software program can be loaded and / or installed via the memory and / or communication interface. When the software program is loaded into the memory and executed by the processor, one or more steps of the above-described methods can be performed. Alternatively, in other embodiments, the processor can be configured to perform one of the above methods by any other appropriate means (for example, by means of firmware).

[0120] The device can be realized by using a bus architecture. The bus architecture can include any number of interconnected buses and bridges, depending on the specific application of the hardware and the overall design constraints. The bus will connect various circuits including one or more processors, memories and / or hardware modules together. The bus can also connect various other circuits such as peripheral devices, voltage regulators, power management circuits, external antennas, etc.

[0121] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like, and can be divided into an address bus, a data bus, a control bus, and the like.

[0122] Finally, it should be noted that the above embodiments are merely used to illustrate the technical solutions of the present application, rather than limit the same; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for detecting the abnormal position of a rail fastener spring, characterized in that, The application relates to a method for detecting a spring strip of a rail fastener, comprising the following steps: Step 1, collecting a rail panoramic overhead view; Step 2, according to a preset standard fastener spacing, cutting the rail panoramic overhead view into unit fastener images; According to the order of the fasteners in the rail panoramic overhead view, numbering each fastener; Step 3, based on the unit fastener images, identifying the spring strip and marking the position through a spring strip target detection model to obtain an anchor box region; Step 4, based on the anchor box region, performing pixel-level segmentation through a spring strip semantic segmentation model to obtain a spring strip mask; Step 5, through a computer vision function library, performing geometric analysis on the spring strip mask to obtain a rotation angle of the spring strip; The specific method of the geometric analysis comprises: Step 51, extracting pixel points representing the boundary of the spring strip from the spring strip mask to form a spring strip contour line; Step 52, calculating and fitting a minimum circumscribed rectangle surrounding the spring strip contour line; Step 53, calculating the included angle between the long side of the minimum circumscribed rectangle and a preset standard horizontal axis to obtain the rotation angle of the spring strip; The specific determination method of the direction of the rotation angle comprises: Step 531, comparing the width value and the height value of the minimum circumscribed rectangle: If the width value is less than the height value, the width value represents the short side and the height value represents the long side; Otherwise, the width value represents the long side and the height value represents the short side; Step 532: If the width value is greater than the height value, The value is used as the final rotation angle; if the width value is less than the height value, The value obtained by adding 90 degrees is used as the final rotation angle; Step 533, when the minimum circumscribed rectangle is in a horizontal state, the rotation angle is defined as 0 degrees, and the long axis of the minimum circumscribed rectangle is defined as a standard reference axis. Step 534, based on the standard reference axis, the angle conversion correction is performed on the value, ensuring that the value always represents the included angle between the long side and the standard reference axis. Step 535, after the angle conversion correction, the value is determined.​ If A positive value indicates that the spring bar rotates counterclockwise. If A negative value indicates that the spring rotates clockwise.

2. The detection method according to claim 1, characterized in that, The step 1 specifically comprises: Step 11, based on a camera, continuously imaging the rail through a scanning shooting mode to obtain scanning images; Step 12, extracting feature points in the scanning images through a SIFT algorithm, establishing the corresponding relationship between the same feature points between different scanning images, and calculating geometric transformation parameters; Step 13, correcting the scanning images based on the geometric transformation parameters; Step 14, splicing and fusing the scanning images to eliminate the brightness difference and splicing gap between the scanning images to obtain the rail panoramic overhead view.

3. The method of claim 1, wherein The specific construction method of the training data set of the spring strip target detection model comprises the following steps: firstly, cutting unit fastener images from the actually collected rail panoramic overhead view, wherein the unit fastener images comprise spring strip images in a normal state and spring strip images in an abnormal state; and secondly, performing boundary box labeling on the spring strip in each unit fastener image, and selecting the rectangular region of the spring strip and assigning a corresponding class label.

4. The detection method according to claim 3, characterized in that, The specific training method of the spring strip target detection model comprises the following steps: inputting the unit fastener images in the training data set into the spring strip target detection model to identify the spring strip and mark the position of the spring strip; based on a back propagation algorithm and an optimizer, adjusting the model parameters according to the loss function between the prediction result and the real labeling; and through iterative training, obtaining the spring strip target detection model for identifying the spring strip and marking the position of the spring strip.

5. The method of claim 1, wherein The identification method of the elastic strip target detection model comprises: firstly detecting the input image; if no elastic strip is detected, directly generating the missing information of the elastic strip; if the elastic strip is detected, outputting an anchor box capable of surrounding the elastic strip according to the position of the elastic strip, and determining the anchor box region.

6. The method of claim 1, wherein The specific construction method of the training data set of the elastic strip semantic segmentation model comprises: firstly, cutting unit fastener images from the actually collected track panoramic overhead view, wherein the unit fastener images include not only elastic strip images in a normal state but also elastic strip images in an abnormal state; and secondly, performing pixel-level mask labeling on the elastic strip in each unit fastener image, and classifying each pixel point as an elastic strip or a background.

7. The detection method according to claim 6, characterized in that, The specific training method of the elastic strip semantic segmentation model comprises: inputting the pixel-level mask labeling data in the training data set into the elastic strip semantic segmentation model, learning to classify each pixel as an elastic strip or a background, taking the difference between the predicted elastic strip mask and the real elastic strip mask as the training target, taking the cross-entropy loss as the loss function, and obtaining the elastic strip semantic segmentation model for pixel-level segmentation of the elastic strip in the anchor box region through iterative training.

8. A rail fastener spring dislocation visual inspection system employing the detection method according to any one of claims 1-7, characterized in that, The system comprises a data receiving module, a data processing module and a result generating module. The data receiving module is configured to receive a track panoramic overhead view. The data processing module comprises a unit fastener unit, an anchor box unit, an elastic strip mask unit and a rotation angle unit. The unit fastener unit is configured to cut the track panoramic overhead view into unit fastener images according to a preset standard fastener spacing, and number each fastener according to the order of the fastener in the track panoramic overhead view. The anchor box unit is configured to identify and mark the position of the elastic strip based on the unit fastener image through the elastic strip target detection model, and obtain an anchor box region. The elastic strip mask unit is configured to perform pixel-level segmentation based on the anchor box region through the elastic strip semantic segmentation model, and obtain an elastic strip mask. The rotation angle unit is configured to perform geometric analysis on the elastic strip mask through a computer vision function library, and obtain the rotation angle of the elastic strip. The result generating module is configured to send out the elastic strip ectopic detection result, wherein the elastic strip ectopic detection result comprises the missing information of the elastic strip and the rotation angle.

9. A rail fastener spring dislocation visual detection device, characterized in that, The system comprises a processor, a memory and a bus, the memory stores instructions and data read by the processor, the processor is configured to call the instructions and data in the memory to execute the detection method in any one of claims 1-7, and the bus is connected between each functional component for transmitting information.

Images