Rail fastener elastic strip fracture visual detection method, system and device

Abstract

The application provides a track fastener elastic strip fracture visual detection method, system and device, relates to the track fastener detection technical field, and the method comprises collecting a track panoramic overhead view; the track panoramic overhead view is pretreated, and a unit fastener image is obtained; based on the elastic strip semantic segmentation model, the unit fastener image is pixel-level segmented, and an elastic strip mask is obtained; based on the elastic strip mask, the skeleton line is extracted through a morphological thinning method; based on the skeleton line, the fracture risk is identified and quantified through a multi-dimensional feature analysis method.The scheme can accurately identify fracture and quantitatively classify, so that railway maintenance personnel can intuitively and accurately evaluate the severity of fracture, so that a maintenance strategy can be scientifically formulated; the scheme significantly reduces the misjudgment rate through a multi-dimensional verification mechanism, thereby improving the reliability of the detection result and reducing the artificial review cost.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of rail fastener detection, in particular to a rail fastener spring strip fracture visual detection method, system and device. BACKGROUND

[0002] With the rapid development of rail transit, the mileage of railways continues to grow. As the core stressed component of railway track structure, the spring strip is extremely prone to fracture defects due to fatigue accumulation or stress concentration under the multiple effects of long-term high-frequency vibration, impact load and complex environmental erosion of trains. This progressive deterioration from fine cracks to complete fracture not only seriously threatens the safety of train operation, but also is difficult to detect. In particular, when the spring strip is fractured, it will directly lead to the failure of rail restraint, causing abnormality of track geometric parameters, and may even cause catastrophic accidents such as train derailment, which is much more harmful than general structural deformation.

[0003] Traditional railway fastener detection mainly relies on manual inspection, while the fracture of spring strip, especially the early micro-cracks, is often hidden and difficult to accurately identify by visual inspection. At the same time, manual inspection is low in efficiency and high in labor intensity, and is difficult to meet the strict requirements of "early detection and early warning" of fracture defects under the background of compressed high-speed rail maintenance window period and increased line complexity.

[0004] Although the existing automatic detection technology has made breakthroughs in macro defect recognition, it still has limitations in detecting spring strip fracture. Conventional methods mostly rely on simple morphological analysis, which is difficult to distinguish surface scratches from real cracks, and has a low detection rate for non-penetrating fractures or micro-cracks, which may easily miss the detection and hide potential safety hazards.

[0005] Therefore, for the key defect of spring strip fracture, it is urgent to develop an intelligent detection technology that can accurately identify fine cracks, distinguish fracture types and achieve early warning, in order to overcome the limitations of traditional detection methods and build a safety barrier for railway tracks.

[0006] In recent years, the application of deep learning technology in the field of rail fastener detection has made significant progress. In related research, some scholars have proposed a rail fastener segmentation model based on deep learning semantic segmentation model Fastener-SSN. This model realizes lightweight design of the network through PFPA feature pyramid module, reduces the number of network parameters and floating point operations, and introduces efficient channel attention mechanism (ECA) to improve the robustness of fastener semantic segmentation. At the same time, the connected domain area and number characteristics are further analyzed to realize the detection of fastener state. The experimental results show that the model performs well in terms of frame rate per second (FPS), mean intersection over union (mIoU) and mean accuracy (mAcc), and the designed fastener state classifier can effectively realize the classification and detection of fastener state, with strong applicability. However, for the detection of broken spring, the above method has obvious shortcomings:

[0007] (1) It is difficult to accurately identify the subtle features of the broken defect. Although this method can realize the semantic segmentation and state classification of the fastener, the analysis method of connected domain area and number characteristics is more suitable for detecting macroscopic defects such as fastener loss and overall structure abnormality. For the identification of subtle defects such as early micro-cracks and non-through cracks, it is easy to be confused with surface scratches and stains, leading to misjudgment or missed detection.

[0008] (2) Lack of special analysis mechanism for broken features. The core feature of broken spring is the destruction of the continuity of the structure, and this method does not establish a special analysis model for the broken shape, which cannot effectively extract the key information such as the position, length and direction of the broken part from the segmentation result, and it is difficult to meet the detection needs of "accurate positioning and quantitative evaluation" of the broken defect, and it is difficult to be used as the basis for judgment of the risk level of the broken defect and the basis for formulating maintenance strategies in practical application. SUMMARY

[0009] The purpose of the present application is to provide a rail fastener spring break visual detection method, system and device to solve at least one of the above technical problems in the prior art.

[0010] In a first aspect, to solve the above technical problems, the present application provides a rail fastener spring break visual detection method, comprising the following steps:

[0011] Step 1, collect the panoramic overhead view of the rail.

[0012] In a feasible embodiment, the step 1 specifically comprises:

[0013] Step 11, based on the camera, the track is continuously imaged by a scanning shooting mode to obtain a scanning image, so as to ensure that there is sufficient overlapping area between adjacent frames of scanning images, thereby facilitating high-precision image stitching;

[0014] Step 12, by means of SIFT (Scale Invariant Feature Transform) algorithm or the like, feature points (corner points or texture features) in the scanning image are extracted, a corresponding relationship between the same feature points between different scanning images is established, and geometric transformation parameters are calculated;

[0015] Step 13, based on the geometric transformation parameters, the scanning image is corrected so as to eliminate image deformation and misplacement caused by camera movement (such as shaking, lateral displacement, pitch and yaw, etc.) and lens distortion;

[0016] Step 14, the scanning image is stitched and fused (seamlessly) to eliminate the brightness difference between the scanning images and the stitching gap, and an overall panoramic view of the track is obtained.

[0017] Step 2, the overall panoramic view of the track is preprocessed to obtain a unit fastener image.

[0018] In a feasible implementation, the step 2 specifically comprises:

[0019] Step 21, based on the track edge recognition method of Hough line detection, the unit fastener image is located and cropped from the overall panoramic view of the track according to the standard spacing and arrangement rule of the track fastener.

[0020] In a feasible implementation, the track edge recognition method comprises: first, the position boundary of the steel rail is recognized as a reference to demarcate the transverse range where the fastener is located; then, the square area of each fastener is framed (accurately) in combination with the distribution interval of the fastener in the longitudinal direction (the direction of the track axis) to crop the unit fastener image of uniform size; the unit fastener image comprises a spring strip region.

[0021] Preferably, the spring strip region can be located at the center of the unit fastener image, thereby reducing the interference of the background image.

[0022] Step 22, the unit fastener image is subjected to gray scale normalization processing.

[0023] In a feasible implementation, the specific method of the gray scale normalization processing comprises: by means of the maximum-minimum normalization method, the gray scale value of the unit fastener image is linearly mapped to the range of 0-255, so that the brightness level of different unit fastener images remains consistent, the response difference of the ambient light during shooting and the camera photosensitive element is eliminated, thereby facilitating subsequent feature extraction and model processing.

[0024] Step 23, noise suppression and edge enhancement are performed on the unit fastener image.

[0025] In an implementable embodiment, the noise suppression includes smoothing by Gaussian filtering, and the specific settings include a Gaussian kernel size of 5*5 and a standard deviation of 1.2; in this way, high-frequency noise can be effectively filtered out while the edge information of the elastic strip is maintained as much as possible.

[0026] In an implementable embodiment, the edge enhancement includes calculating the second derivative of the unit fastener image by a Laplacian operator, so as to highlight the contrast between the elastic strip contour and the possible broken edges and the background, and to provide clearer feature input for subsequent semantic segmentation.

[0027] Step 24, deblurring is performed on the unit fastener image.

[0028] In an implementable embodiment, the specific method of deblurring includes estimating the size and direction of the blur kernel by analyzing the spectral characteristics of the unit fastener image, and then performing deconvolution operation by Wiener filtering, gradually and iteratively optimizing the image clarity, so that the texture and fine cracks on the surface of the elastic strip are clearly presented, and the accuracy of subsequent skeleton extraction is ensured based on the blind deconvolution method.

[0029] Step 3, based on an elastic strip semantic segmentation model (such as a channel-spatial attention U-Net), pixel-level segmentation is performed on the unit fastener image to obtain an elastic strip mask, so as to obtain the complete elastic strip contour and the details of the broken area, and to provide high-precision basic data for subsequent skeleton line extraction and broken analysis.

[0030] In an implementable embodiment, the elastic strip semantic segmentation model takes U-Net as the basic framework, includes an encoder-decoder structure and a skip connection, so as to effectively fuse feature information at different levels, and balance the segmentation accuracy and the ability to capture details;

[0031] The elastic strip semantic segmentation model further includes a channel attention mechanism (Channel Attention Mechanism) and a spatial attention mechanism (Spatial Attention Mechanism);

[0032] The channel attention mechanism is used to learn the weights of different feature channels, so as to enhance the response of key feature channels (such as edges and texture mutation areas) related to elastic strip breaks, suppress the interference of irrelevant background channels, and make the model more focused on the feature information that may exist in the broken area;

[0033] The spatial attention mechanism is used for focusing on key spatial positions in the elastic strip area, thereby improving the sensitivity to the micro-fracture area by strengthening the feature extraction of positions where cracks, breakpoints and the like may occur on the elastic strip surface, and ensuring that the fracture features are not covered by background noise.

[0034] In an implementable embodiment, the training data set of the elastic strip semantic segmentation model comprises:

[0035] Unit fastener images collected from different lines, different service lives and different lighting conditions are used to cover various states such as no defect, micro-crack, partial fracture and complete fracture of the elastic strip, so as to ensure the diversity and representativeness of the data set;

[0036] Each unit fastener image is finely labeled, and the pixels are divided into elastic strip regions and background regions: the elastic strip pixels include complete elastic strip contour and fracture region boundary, so that the pixels of the fracture part are accurately labeled as part of the elastic strip region, and a clear supervision signal is provided for the model to learn the fracture feature;

[0037] Data augmentation is performed on the training data set by using data augmentation strategies such as random rotation, scaling, horizontal flipping, brightness adjustment and adding Gaussian noise, so as to simulate the image changes that may be encountered in actual detection, thereby avoiding model overfitting and improving the generalization ability of the model.

[0038] In an implementable embodiment, the specific training method of the elastic strip semantic segmentation model comprises:

[0039] The preprocessed unit fastener image with a size of 640*640 pixels is input into the elastic strip semantic segmentation model; cross-entropy loss (Cross-Entropy Loss) combined with Dice loss is used as the training loss function; the cross-entropy loss is used to optimize the accuracy of pixel-level classification; the Dice loss is used to calculate the overlap degree of the predicted elastic strip mask and the real elastic strip mask, thereby optimizing the edge segmentation accuracy of the elastic strip contour and the fracture region, and solving the class imbalance problem caused by the small pixel ratio of the fracture region;

[0040] Based on the Adam optimizer, the initial learning rate is set, and a stepwise learning rate decay strategy is adopted;

[0041] In the training process, iteration is performed in a preset batch size, and the total number of iterations is determined according to the model convergence condition; when the average intersection over union (mIoU) of the validation set has no improvement after 10 consecutive iterations, the training is stopped to avoid overfitting;

[0042] In this way, after being fully trained, the strip semantic segmentation model can output a high-precision pixel-level strip mask, thereby accurately distinguishing the strip from the background region, and especially clearly retaining the contour information of the crack and breakpoint features on the surface of the strip, thereby facilitating accurate capture of structural changes (caused by the crack features) in the subsequent skeleton line extraction process, and providing reliable morphological basis for crack determination.

[0043] Step 4. Based on the strip mask, a skeleton line is extracted by a morphological thinning method.

[0044] In an implementable embodiment, the morphological thinning method comprises:

[0045] Based on the Zhan-Suen thinning algorithm, non-key pixels, which are pixels that are not strip endpoints and are removed without destroying the topological structure of the strip, are removed by iteratively eroding the edge pixels of the strip mask, to form a skeleton line with a single-pixel width; and by an endpoint determination rule, strip endpoint features are retained, and branches with a length less than a preset threshold are removed, thereby removing pseudo-branches caused by surface scratches or segmentation noise, to avoid interference with subsequent crack analysis; the endpoint determination rule comprises that a pixel point belongs to the skeleton line (i.e., the pixel value is 1), and only the pixel point belongs to the skeleton line in a preset neighborhood of the pixel point.

[0046] In an implementable embodiment, the morphological thinning method further comprises smoothing the skeleton line by a Bezier curve fitting algorithm, so as to reduce local jitter and ensure that the skeleton line is continuous and does not cover the crack features, and specifically comprises: fitting discrete points (i.e., a series of pixel points on the skeleton line) by the least square method, thereby eliminating pixel-level fluctuations in the form of sawteeth while retaining key nodes caused by cracks, so that significant turning points or mutations caused by real cracks are not ignored or erased in pursuit of extreme smoothness; pseudo-breakpoints caused by segmentation errors are connected and repaired, and real breakpoints are retained; the pseudo-breakpoint refers to a breakpoint at which the directions of branches on both sides of the breakpoint are consistent, and the distance between the breakpoints is less than a preset threshold; the real breakpoint refers to a breakpoint at which the directions of branches on both sides of the breakpoint are inconsistent, and the distance between the breakpoints is greater than a preset threshold.

[0047] Step 5. Based on the skeleton line, a crack risk is identified and quantified by a multi-dimensional feature analysis method.

[0048] In an implementable embodiment, the multi-dimensional feature analysis method specifically comprises:

[0049] Step 51, calculate the total area of the skeleton line and make a judgment: if the total area is less than the lower limit of the standard area, it is judged that the elastic strip is at risk of breaking; the reason for this design is that the complete skeleton line area of the elastic strip (i.e. the total number of pixels) should be in a fixed range in a normal state, and the breaking will cause the skeleton line to be discontinuous, thereby causing the effective area to decrease, for example, when the elastic strip is completely broken, its skeleton line may be separated into multiple parts, causing the total area to decrease significantly.

[0050] Step 52, perform connected domain detection and endpoint detection on the skeleton line to identify suspicious connected domains.

[0051] In a feasible implementation, the connected domain detection includes connected domain number detection: if the number of connected domains is equal to 3, it means that the connected domain detection is normal; if the number of connected domains is greater than 3, it means that the connected domain detection is abnormal, and it is judged that the elastic strip is at risk of breaking; if the number of connected domains is less than 3, it means that the connected domain detection is abnormal, and the skeleton line is partially missing, i.e. an abnormal situation caused by breaking, and it is also judged that the elastic strip is at risk of breaking; the reason for this setting is that the number of connected domains of a standard elastic strip skeleton line is 3, each connected domain has 2 endpoints, and there are a total of 6 endpoints;

[0052] The endpoint detection includes endpoint number detection and endpoint distribution verification.

[0053] The endpoint number detection includes counting the total number of endpoints of the skeleton line; if the total number of endpoints is equal to 6, it means that the endpoint number detection is normal; if the total number of endpoints is greater than 6, it is judged that the elastic strip is at risk of breaking; if the total number of endpoints is less than 6, it means that the skeleton line is partially missing, i.e. an abnormal situation caused by breaking, and it is judged that the elastic strip is at risk of breaking;

[0054] The endpoint distribution verification includes checking whether the number of endpoints of each connected domain is 2: if yes, it is marked as a normal connected domain; if no, it is marked as a suspicious connected domain.

[0055] Step 53, for the suspicious connected domain, perform simplified structure analysis, including key node detection, branch length analysis and global morphological verification.

[0056] In this way, the progressive application of rapid screening (step 52) and fine analysis (step 53) not only maintains the sensitivity to the breaking feature, but also avoids the computational overhead and robustness problems brought by complex template matching, achieving efficient and accurate detection of elastic strip breaking.

[0057] In an implementable embodiment, the key node detection comprises detecting whether the positions and connection relationships of the key nodes conform to the positions and connection relationships of the key nodes on the standard spring strip skeleton line after presetting a plurality of key nodes (e.g. bending points) on the skeleton line; if not (indicating that the key nodes are missing or the connection is abnormal), it is determined that the spring strip has a risk of breakage;

[0058] The branch length analysis comprises calculating the lengths of all branches in the skeleton line and comparing the lengths with a length threshold value; if the lengths are greater than the length threshold value, it is indicated that there is an abnormal branch (possibly a bifurcated structure caused by breakage), and it is determined that the spring strip has a risk of breakage;

[0059] The global morphology verification comprises calculating global parameters such as the total length and the width-to-height ratio of the skeleton line and comparing the global parameters with the global parameters of the standard spring strip; if the deviation of the global parameters exceeds a preset range, it is determined that the spring strip has a risk of breakage;

[0060] Step 54, for the spring strip with a risk of breakage, a breakage level subdivision is performed.

[0061] In an implementable embodiment, the breakage level specifically comprises:

[0062] Micro-cracks: the total area of the skeleton line is in the range of 85-95% of the standard area, and the number of connected domains is 3 or the total number of end points is 6-8;

[0063] Partial breakage: the connected domain detection is abnormal and the number of connected domains is in the range of 2-6, or the total number of end points is 8-12 and the total area of the skeleton line is in the range of 70-85% of the standard area;

[0064] Complete breakage: the number of connected domains exceeds the range of 2-6, or the total number of end points exceeds the range of 4-12 and the total area of the skeleton line is less than 70% of the standard area.

[0065] In a second aspect, based on the same inventive concept, the application also provides a rail fastener spring strip breakage visual detection system, comprising a data receiving module, a data processing module and a result generating module;

[0066] The data receiving module is configured to receive a rail panoramic top view.

[0067] The data processing module comprises a preprocessing unit, a spring strip mask unit, a skeleton line unit and a risk analysis unit.

[0068] The preprocessing unit is configured to pre-process the rail panoramic top view to obtain a unit fastener image.

[0069] The spring strip mask unit performs pixel-level segmentation on the unit fastener image based on a spring strip semantic segmentation model to obtain a spring strip mask.

[0070] The skeleton line unit extracts the skeleton line based on the spring strip mask through a morphological thinning method;

[0071] The risk analysis unit identifies and quantifies the fracture risk based on the skeleton line through a multi-dimensional feature analysis method;

[0072] The result generation module is configured to send the analysis result of the risk analysis unit.

[0073] In a third aspect, based on the same inventive concept, the present application also provides a rail fastener spring strip fracture visual detection device, which comprises a processor, a memory and a bus, the memory stores instructions and data read by the processor, the processor is configured to call the instructions and data in the memory to execute the rail fastener spring strip fracture visual detection method as described above, and the bus is connected between each functional component for transmitting information.

[0074] In a feasible implementation, the device further comprises a camera arranged at the lower part of the front end of the rail detection vehicle, which is configured to collect a panoramic overhead view of the rail.

[0075] By adopting the above technical solution, the present application has the following beneficial effects:

[0076] The rail fastener spring strip fracture visual detection method, system and device provided by the present application can not only accurately identify the fracture, but also classify the fracture into three levels of micro-cracks, partial fracture and complete fracture according to the quantitative indexes. For example, the present application can be quantified as "the spring strip has micro-cracks, the skeleton line area is 90% of the standard area, the number of connected domains is 3, and the total number of end points is 7". This accurate identification and classification enable the railway maintenance personnel to intuitively and accurately evaluate the severity of the fracture, so that the maintenance strategy can be scientifically formulated, such as repairing the completely fractured fastener first and monitoring the micro-cracks, thereby greatly improving the scientificity and accuracy of the maintenance decision.

[0077] Through the multi-dimensional feature analysis of the skeleton line area, connected domains and end points, key nodes and branches, the present application can effectively exclude the interference of non-fracture factors. For example, the scratches on the surface of the spring strip will only affect the contour of the spring strip and will not change the number of connected domains and end points of the skeleton line, which can be excluded by the connected domain and end point judgment; while the real fracture will cause changes such as reduction of the skeleton line area, increase of the number of connected domains and increase of the number of end points, so combined with multi-dimensional analysis, the scratches and fractures can be accurately distinguished. This multi-dimensional verification mechanism significantly reduces the misjudgment rate, improves the reliability of the detection result, and reduces the cost of manual review. BRIEF DESCRIPTION OF DRAWINGS

[0078] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the description of the specific embodiments or prior art will be briefly introduced. Obviously, the accompanying drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort based on these drawings.

[0079] Figure 1 A flow chart of a rail fastener elastic strip fracture visual detection method provided for an embodiment of the present application is shown in the figure.

[0080] Figure 2 An elastic strip mask segmentation illustration provided for an embodiment of the present application is shown in the figure.

[0081] Figure 3 A skeleton line example provided for an embodiment of the present application is shown in the figure.

[0082] Figure 4 A multi-dimension feature analysis flow example provided for an embodiment of the present application is shown in the figure.

[0083] Figure 5 A rail fastener elastic strip fracture visual detection system diagram provided for another embodiment of the present application is shown in the figure. EMBODIMENT

[0084] The technical solutions of the present application will be described clearly and completely below with reference to the accompanying drawings. Obviously, the described embodiments are some of the embodiments of the present application, but not all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative effort belong to the scope of protection of the present application.

[0085] In the description of the present application, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and therefore cannot be understood as indicating or implying that the device or element must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application. In addition, the terms "first", "second", "third" are only for the purpose of description, and cannot be understood as indicating or implying relative importance.

[0086] In the description of the present application, it should be noted that unless otherwise explicitly specified and limited, the terms "mounting", "connection", "connecting" should be understood in a broad sense, for example, it can be fixed connection, or detachable connection, or integrally connected; it can be mechanical connection, or electrical connection; it can be directly connected, or indirectly connected through an intermediate medium, or the internal communication of two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0087] The present application will be further explained in conjunction with the specific embodiments.

[0088] It should be further noted that the following specific examples or specific embodiments are a series of optimized setting modes listed by the present application to further explain the specific invention content, and these setting modes can be used in combination or in association with each other.

[0089] Embodiment one:

[0090] As shown in the figure, the rail fastener spring strip fracture visual detection method provided by the embodiment includes the following steps: Figure 1

[0091] Step 1, collect the rail panoramic overhead view.

[0092] Further, the step 1 specifically includes:

[0093] Step 11, based on the camera (set at the lower part of the front end of the rail detection vehicle), the rail is continuously imaged by scanning shooting mode to obtain the scanning image, so as to ensure that there is sufficient overlapping area between adjacent frames of scanning images, and then it is beneficial to realize high-precision image stitching;

[0094] Step 12, by using SIFT (Scale Invariant Feature Transform) algorithm and the like, the feature points (corner points or texture features) in the scanning image are extracted, the corresponding relationship between the same feature points between different scanning images is established, and the geometric transformation parameters are calculated;

[0095] Step 13, based on the geometric transformation parameters, the scanning image is corrected in order to eliminate the image deformation and misplacement problems caused by camera movement (such as shaking, lateral displacement, pitch and yaw, etc.) and lens distortion;

[0096] Step 14, the scanning images are spliced and fused (seamlessly) to eliminate the brightness difference and splicing gap between the scanning images, and the rail panoramic overhead view is obtained.

[0097] Step 2, pre-process the rail panoramic overhead view to obtain the unit fastener image.

[0098] Further, the step 2 specifically includes:​

[0099] Step 21, from the track panoramic overhead view, according to the standard spacing and arrangement rules of the track fastener, the track edge recognition method based on Hough line detection is used to locate and crop the unit fastener image.

[0100] Further, the track edge recognition method includes: first, identifying the position boundary of the steel rail as a reference to determine the transverse range of the fastener; then, combining the distribution interval of the fastener in the longitudinal direction (the direction of the track axis), (precisely) framing the square region of each fastener, and cropping the unit fastener image with a uniform size (for example, 640*640 pixels); the unit fastener image includes a spring strip region;

[0101] Preferably, the spring strip region can be located in the center of the unit fastener image, thereby reducing the interference of the background image.

[0102] Step 22, performing gray scale normalization processing on the unit fastener image.

[0103] Further, the specific method of the gray scale normalization processing includes: through the maximum-minimum normalization method, the gray scale value of the unit fastener image is linearly mapped to the range of 0-255, so that the brightness level of different unit fastener images remains consistent, eliminating the response difference of the camera photosensitive element and the change of the ambient light during shooting, and thus facilitating subsequent feature extraction and model processing.

[0104] Step 23, performing noise suppression and edge enhancement on the unit fastener image.

[0105] Further, the noise suppression includes smoothing processing through Gaussian filtering, and the specific settings include: the Gaussian kernel size is 5*5, and the standard deviation is 1.2; in this way, the high-frequency noise can be effectively filtered out while the edge information of the spring strip is maintained as much as possible.

[0106] Further, the edge enhancement includes calculating the second derivative of the unit fastener image through the Laplace operator, so as to highlight the contrast between the spring strip profile and the possible broken edges and the background, and thus provide clearer feature input for subsequent semantic segmentation.

[0107] Step 24, performing deblurring processing on the unit fastener image.

[0108] Further, the specific method of the deblurring processing includes: through the analysis of the spectral characteristics of the unit fastener image, the size and direction of the blur kernel are estimated, and then the deconvolution operation is performed through the Wiener filter, and the image clarity is gradually iteratively optimized, so that the texture and fine cracks on the surface of the spring strip are clearly presented, and thus based on the blind deconvolution method, the accuracy of subsequent skeleton extraction is ensured.

[0109] Step 3, asFigure 2 As shown, based on the elastic strip semantic segmentation model (such as channel-spatial attention U-Net), the unit fastener image is segmented at the pixel level to obtain the elastic strip mask, so as to obtain the complete elastic strip contour and the details of the broken area, and then provide high-precision basic data for subsequent skeleton line extraction and broken analysis.

[0110] Further, the elastic strip semantic segmentation model takes U-Net as the basic framework, including an encoder-decoder structure and a skip connection, so as to effectively fuse feature information at different levels, while ensuring segmentation accuracy and considering the ability to capture details;

[0111] The elastic strip semantic segmentation model also includes a channel attention mechanism (Channel Attention Mechanism) and a spatial attention mechanism (Spatial Attention Mechanism);

[0112] The channel attention mechanism is used to learn the weights of different feature channels, so as to enhance the response of key feature channels (such as edges and texture mutation areas) related to elastic strip breakage, suppress the interference of irrelevant background channels, and make the model more focused on feature information that may have breaks;

[0113] The spatial attention mechanism is used to focus on key spatial positions in the elastic strip area, so as to enhance feature extraction at positions where cracks and breakpoints may occur on the surface of the elastic strip, improve the sensitivity to small broken areas, and ensure that the broken features are not covered by background noise.

[0114] Further, the training data set of the elastic strip semantic segmentation model includes:

[0115] Unit fastener images collected from different lines, different operating years, and different lighting conditions cover various states such as no defects, micro-cracks, partial breaks, and complete breaks, to ensure the diversity and representativeness of the data set;

[0116] Each unit fastener image is finely annotated, and the pixels are divided into elastic strip area and background area: the elastic strip pixels include the complete elastic strip contour and the broken area boundary, so as to ensure that the pixels of the broken part are accurately annotated as part of the elastic strip area, and provide a clear supervision signal for the model to learn the broken features;

[0117] Through data enhancement strategies such as random rotation (-15~15°), scaling (0.8~1.2 times), horizontal flip, brightness adjustment (±20%), and adding Gaussian noise (standard deviation 0.01~0.03), the training data set is enhanced, so as to simulate the image changes that may be encountered in actual detection, thereby avoiding model overfitting and improving the generalization ability of the model.

[0118] Further, the specific training method of the elastic strip semantic segmentation model comprises:

[0119] The pre-processed unit fastener image of 640*640 pixels is input into the elastic strip semantic segmentation model; a cross-entropy loss (Cross-Entropy Loss) combined with a Dice loss is used as a training loss function; the cross-entropy loss is used to optimize the accuracy of pixel-level classification; the Dice loss is used to calculate the overlap of the predicted elastic strip mask and the real elastic strip mask, so as to optimize the edge segmentation accuracy of the elastic strip contour and the broken area, and solve the class imbalance problem caused by the small pixel ratio of the broken area.

[0120] Based on the Adam optimizer, the initial learning rate (0.001) is set, and a step learning rate decay strategy is adopted (i.e. after every 10 iterations, the current learning rate is decayed to 10% of the original learning rate).

[0121] During the training process, the preset batch size (for example, 8) is used for iteration, and the total number of iterations is determined according to the model convergence (for example, 100-200 iterations). When the average intersection over union (mIoU) of the validation set does not improve after 10 consecutive iterations, the training is stopped to avoid overfitting.

[0122] In this way, after sufficient training, the elastic strip semantic segmentation model can output a high-precision pixel-level elastic strip mask, thereby accurately distinguishing the elastic strip from the background area, especially clearly retaining the contour information of the crack, breakpoint and other broken features on the surface of the elastic strip, and further facilitating accurate capture of structural changes (caused by broken features) in the subsequent skeleton line extraction process, and providing reliable morphological basis for broken judgment.

[0123] Step 4, based on the elastic strip mask, a skeleton line is extracted by a morphological thinning method, as shown in Figure 3 .

[0124] Further, the morphological thinning method comprises:

[0125] Based on the Zhan-Suen thinning algorithm, the edge pixels of the elastic strip mask are iteratively eroded to remove non-key pixels, forming a single-pixel-width skeleton line, wherein the non-key pixels refer to pixels that are not elastic strip endpoints and are removed without destroying the elastic strip topology; and by an endpoint judgment rule, the elastic strip endpoint features are retained, and branches with a length less than a preset threshold (for example, 5 pixels) are removed, thereby removing pseudo-branches caused by surface scratches or segmentation noise to avoid interference with subsequent broken analysis; the endpoint judgment rule comprises that a pixel point belongs to the skeleton line (i.e. the pixel value is 1), and only the pixel point belongs to the skeleton line in a preset neighborhood (for example, 8 pixels) of the pixel point.

[0126] Further, the morphological refinement method further comprises smoothing the skeleton line by a Bezier curve fitting algorithm to reduce local jitter and ensure that the skeleton line is continuous and does not cover the broken features, specifically comprising: fitting the discrete points (i.e. a series of pixel points on the skeleton line) by the least square method to eliminate pixel-level fluctuations while retaining key nodes caused by broken, so that the pursuit of extreme smoothness will not ignore or erase those significant turning or mutations caused by real broken; connecting the pseudo-breakpoints caused by segmentation errors and retaining the true breakpoints; the pseudo-breakpoint refers to the breakpoint whose two branches have consistent directions and the breakpoint distance is less than a preset threshold (e.g. 3 pixels); the true breakpoint refers to the breakpoint whose two branches have inconsistent directions and the breakpoint distance is greater than the preset threshold.

[0127] Step 5, based on the skeleton line, a multi-dimensional feature analysis method is used to identify and quantify the broken risk.

[0128] Further, as shown in Figure 4 the multi-dimensional feature analysis method specifically comprises:

[0129] Step 51, calculate the total area of the skeleton line and make a judgment: if the total area is less than the lower limit of the standard area (e.g. 85% of the standard area), it is determined that the elastic strip has a broken risk; the reason for this design is that the complete skeleton line area (i.e. the total number of pixels) of the elastic strip should be in a fixed range under normal conditions, and since broken will cause the skeleton line to be discontinuous, thus reducing the effective area, for example, when the elastic strip is completely broken, its skeleton line may be separated into multiple parts, resulting in a significant decrease in the total area.

[0130] Step 52, perform connected component detection and endpoint detection on the skeleton line to identify suspicious connected components.

[0131] Further, the connected component detection includes connected component number detection: if the number of connected components is equal to 3, it means that the connected component detection is normal; if the number of connected components is greater than 3, it means that the connected component detection is abnormal, and it is determined that the elastic strip has a broken risk; if the number of connected components is less than 3, it means that the connected component detection is abnormal, and the skeleton line has partial missing, i.e. an abnormal situation caused by broken, and it is also determined that the elastic strip has a broken risk; the reason for this setting is that the number of connected components of the standard elastic strip skeleton line is 3, each connected component has 2 endpoints, and a total of 6 endpoints;

[0132] The endpoint detection includes endpoint number detection and endpoint distribution verification.

[0133] The endpoint quantity detection includes counting the total number of endpoints of the skeleton line; if the total number of endpoints is equal to 6, it indicates that the endpoint quantity detection is normal; if the total number of endpoints (for example, 8) is greater than 6, it is determined that the elastic strip has a risk of fracture; and if the total number of endpoints is less than 6, it indicates that the skeleton line has a partial loss, that is, an abnormal situation caused by the fracture of the elastic strip, and it is determined that the elastic strip has a risk of fracture.

[0134] The endpoint distribution verification includes checking whether the number of endpoints of each connected domain is 2: if yes, it is marked as a normal connected domain; and if no (for example, only 1 endpoint), it is marked as a suspicious connected domain.

[0135] Step 53, for the suspicious connected domain, a simplified structure analysis is performed, including key node detection, branch length analysis and global morphological verification.

[0136] In this way, the progressive application of the rapid screening (step 52) and the fine analysis (step 53) not only maintains the sensitivity to the fracture characteristics, but also avoids the calculation overhead and the robustness problem caused by the complex template matching, and realizes the efficient and accurate detection of the fracture of the elastic strip.

[0137] Further, the key node detection includes presetting a plurality of (for example, 3) key nodes (for example, bending points) on the skeleton line, detecting whether the positions and connection relationships of the key nodes meet the positions and connection relationships of the key nodes on the standard elastic strip skeleton line, and if not (indicating that the key nodes have a loss or a connection abnormality), determining that the elastic strip has a risk of fracture.

[0138] The branch length analysis includes calculating the lengths of all branches in the skeleton line, and comparing the lengths with a length threshold (for example, 10 pixels): if greater than the length threshold, it indicates that there is an abnormal branch (possibly a bifurcation structure caused by fracture), and it is determined that the elastic strip has a risk of fracture.

[0139] The global morphological verification includes calculating global parameters such as the total length and the width-to-height ratio of the skeleton line, and comparing the global parameters with the global parameters of the standard elastic strip: if the deviation of the global parameters exceeds a preset range (for example, more than 15% reduction in total length), it is determined that the elastic strip has a risk of fracture.

[0140] Step 54, for the elastic strip with a risk of fracture, a fracture level subdivision is performed.

[0141] Further, the fracture level specifically includes:

[0142] Micro-crack: the total area of the skeleton line is in the interval of 85-95% of the standard area, and the number of connected domains is 3 or the total number of endpoints is 6-8.

[0143] Partial fracture: the connected domain detection is abnormal and the number of connected domains is in the interval of 2-6, or the total number of end points is 8-12 and the total area of the skeleton line is in the interval of 70-85% of the standard area;

[0144] Complete fracture: the number of connected domains exceeds the interval of 2-6, or the total number of end points exceeds the interval of 4-12 and the total area of the skeleton line is less than 70% of the standard area.

[0145] Embodiment two:

[0146] As shown in the figure, the embodiment provides a rail fastener spring strip fracture visual detection system, which comprises a data receiving module, a data processing module and a result generating module; Figure 5

[0147] The data receiving module is used to receive a rail panoramic overhead view.

[0148] The data processing module comprises a preprocessing unit, a spring strip mask unit, a skeleton line unit and a risk analysis unit.

[0149] The preprocessing unit is used to pre-process the rail panoramic overhead view to obtain a unit fastener image.

[0150] The spring strip mask unit performs pixel-level segmentation on the unit fastener image based on a spring strip semantic segmentation model to obtain a spring strip mask.

[0151] The skeleton line unit extracts a skeleton line through a morphological thinning method based on the spring strip mask.

[0152] The risk analysis unit identifies and quantifies the fracture risk through a multi-dimensional feature analysis method based on the skeleton line.

[0153] The result generating module is used to publish the analysis results of the risk analysis unit.

[0154] Embodiment three:

[0155] The embodiment provides a rail fastener spring strip fracture visual detection device, which comprises a processor, a memory and a bus, the memory stores instructions and data read by the processor, the processor is used to call the instructions and data in the memory to execute the rail fastener spring strip fracture visual detection method as described above, and the bus is connected between each functional component for transmitting information.

[0156] Further, the device further comprises a camera arranged at the lower part of the front end of the rail detection vehicle, which is used to collect a rail panoramic overhead view.

[0157] ​In yet another implementation form of the solution, the solution can be implemented by means of an integrated device, which can comprise respective modules for performing the respective or several steps of the above-described embodiments. The modules can be one or more hardware modules specially configured to perform the respective steps, or implemented by a processor configured to perform the respective steps, or stored in a computer-readable medium for implementation by a processor, or by some combination.

[0158] The processor performs the various methods and processes described above. For example, the method embodiments in the solution can be implemented as a software program that is tangibly embodied in a machine-readable medium, such as a memory. In some embodiments, portions of the software program can be loaded and / or installed via the memory and / or communication interface. When the software program is loaded into the memory and executed by the processor, one or more of the steps of the above-described methods can be performed. Alternatively, in other embodiments, the processor can be configured to perform one of the above-described methods by any other suitable means, such as by means of firmware.

[0159] The device can be implemented with a bus architecture. The bus architecture can comprise any number of interconnecting buses and bridges, depending on the specific application of the hardware and the overall design constraints. The bus will connect various circuits of the device, including one or more processors, memories, and / or hardware modules. The bus can also connect various other circuits, such as peripheral devices, voltage regulators, power management circuits, external antennas, etc.

[0160] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc.

[0161] Finally, it should be noted that the above-described embodiments are merely intended to illustrate the technical solutions of the present application, and are not intended to limit the present application; even though the above-described embodiments have been described in detail, those skilled in the art should understand that they can still modify the technical solutions recorded in the above-described embodiments, or make equivalent replacements for some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A rail fastener spring strip fracture visual inspection method, characterized by, The method comprises the following steps: Step 1, collecting a track panoramic overhead view; Step 2, preprocessing the track panoramic overhead view to obtain a unit fastener image; Step 3, performing pixel-level segmentation on the unit fastener image based on a spring strip semantic segmentation model to obtain a spring strip mask; Step 4, extracting a skeleton line from the spring strip mask by a morphological thinning method; The morphological thinning method comprises the following steps: based on a Zhan-Suen thinning algorithm, iteratively eroding the edge pixels of the spring strip mask to remove non-key pixels, thereby forming a single-pixel-width skeleton line, wherein the non-key pixels refer to pixels that are not end points of the spring strip and are removed without destroying the topological structure of the spring strip; and by an end point judgment rule, the end point features of the spring strip are retained, and branches with a length less than a preset threshold are removed; the end point judgment rule comprises that a pixel point belongs to the skeleton line, and only the pixel point belongs to the skeleton line in a preset neighborhood of the pixel point; The morphological thinning method further comprises smoothing the skeleton line by a Bezier curve fitting algorithm, specifically comprising the following steps: fitting discrete points by a least squares method; connecting and repairing pseudo-breakpoints caused by segmentation errors to retain true breakpoints; the pseudo-breakpoint refers to a breakpoint at which the directions of branches on both sides of the breakpoint are consistent, and the distance between the breakpoint is less than a preset threshold; the true breakpoint refers to a breakpoint at which the directions of branches on both sides of the breakpoint are inconsistent, and the distance between the breakpoint is greater than a preset threshold; Step 5, identifying and quantifying the fracture risk of the spring strip by a multi-dimensional feature analysis method based on the skeleton line; The multi-dimensional feature analysis method specifically comprises the following steps: Step 51, calculating the total area of the skeleton line and performing judgment: if the total area is less than the lower limit of the standard area, it is determined that the spring strip has a fracture risk; Step 52, performing connected component detection and end point detection on the skeleton line to identify suspicious connected components; Step 53, performing simplified structure analysis on the suspicious connected components, including key node detection, branch length analysis, and global morphological verification; Step 54, performing fracture level subdivision on the spring strip with a fracture risk; The connected component detection comprises connected component number detection: if the number of connected components is equal to 3, it indicates that the connected component detection is normal; if the number of connected components is greater than 3, it indicates that the connected component detection is abnormal, and it is determined that the spring strip has a fracture risk; if the number of connected components is less than 3, it indicates that the connected component detection is abnormal, and part of the skeleton line is missing, and it is determined that the spring strip has a fracture risk; The end point detection comprises end point number detection and end point distribution verification; The end point number detection comprises counting the total number of end points of the skeleton line; if the total number of end points is equal to 6, it indicates that the end point number detection is normal; if the total number of end points is greater than 6, it is determined that the spring strip has a fracture risk; if the total number of end points is less than 6, it indicates that part of the skeleton line is missing, and it is determined that the spring strip has a fracture risk; The end point distribution verification comprises checking whether the number of end points of each connected component is 2: if yes, it is marked as a normal connected component; if no, it is marked as a suspicious connected component.

2. The method of claim 1, wherein, The step 1 specifically comprises the following steps: Step 11, continuously imaging the track by a scanning shooting mode based on a camera to obtain a scanning image; Step 12, by SIFT algorithm, the feature points in the scanning image are extracted, the corresponding relationship between the same feature points in different scanning images is established, and the geometric transformation parameters are calculated; Step 13, based on the geometric transformation parameters, the scanning image is corrected; Step 14, the scanning image is spliced and fused to eliminate the brightness difference and the splicing gap between the scanning images, and the track panoramic overhead view is obtained.

3. The method of claim 2, wherein, The step 2 specifically comprises: Step 21, from the track panoramic overhead view, according to the standard spacing and arrangement rule of the track fastener, the track edge recognition method based on Hough line detection is used to locate and crop the unit fastener image; Step 22, the unit fastener image is subjected to gray scale normalization processing; Step 23, the unit fastener image is subjected to noise suppression and edge enhancement; Step 24, the unit fastener image is subjected to deblurring processing.

4. The method of claim 3, wherein, The track edge recognition method comprises: first, the position boundary of the steel rail is recognized as a reference to delimit the transverse range where the fastener is located; then, the square region of each fastener is framed in combination with the distribution interval of the fastener in the longitudinal direction, and the unit fastener image of uniform size is cropped; the unit fastener image comprises a spring strip region.

5. The method of claim 3, wherein, The specific method of the deblurring processing comprises: the size and direction of the blur kernel are estimated by analyzing the spectral characteristics of the unit fastener image, and then the deconvolution operation is performed through Wiener filtering for gradually iterative optimization of the image definition.

6. The method of claim 1, wherein, The key node detection comprises: after a plurality of key nodes are preset on the skeleton line, whether the positions and connection relationships of the key nodes conform to the positions and connection relationships of the key nodes on the standard spring strip skeleton line are detected; if not, it is determined that the spring strip has a risk of fracture; The branch length analysis comprises: the lengths of all branches in the skeleton line are calculated and compared with a length threshold value; if greater than the length threshold value, it is indicated that there is an abnormal branch, and it is determined that the spring strip has a risk of fracture; The global morphological verification comprises: the global parameters of the skeleton line are calculated and compared with the global parameters of the standard spring strip; if the deviation of the global parameters exceeds a preset range, it is determined that the spring strip has a risk of fracture.

7. A rail fastener clip breakage visual inspection system employing the method of any one of claims 1 to 6, characterised in that, It comprises a data receiving module, a data processing module and a result generating module; The data receiving module is used for receiving the track panoramic overhead view; The data processing module comprises a preprocessing unit, a spring strip mask unit, a skeleton line unit and a risk analysis unit; The preprocessing unit is used for preprocessing the track panoramic overhead view to obtain the unit fastener image; The spring strip mask unit performs pixel-level segmentation on the unit fastener image based on a spring strip semantic segmentation model to obtain a spring strip mask; The skeleton line unit extracts the skeleton line through a morphological thinning method based on the spring strip mask; The risk analysis unit identifies and quantifies the risk of fracture through a multi-dimensional feature analysis method based on the skeleton line; The result generating module is used for publishing the analysis result of the risk analysis unit.

8. A rail fastener spring breakage visual inspection apparatus, characterized by, It comprises a processor, a memory and a bus, the memory stores instructions and data read by the processor, the processor is used to call the instructions and data in the memory to execute the method in any one of claims 1-6, and the bus is connected between each functional component for transmitting information.

Images