Railway fastener defect detection method and system

Abstract

The present application relates to the field of defect detection technology, and specifically discloses a railway fastener defect detection method and system, which controls a high-speed industrial camera to scan a track area to collect track images, performs image preprocessing on the track images, and extracts ROI image blocks containing potential railway fasteners based on template matching. Subsequently, a target detection algorithm is further introduced to accurately locate and segment the fasteners in the ROI image blocks to obtain more refined fastener ROI image blocks. Furthermore, by extracting the visual features of the fastener ROI image blocks and enhancing the significance of their feature spatial distribution, the expression effect of the visual features such as the color, texture, and shape of the fastener surface is enhanced, thereby realizing intelligent recognition of railway fastener defects on this basis. This method can effectively overcome the limitations of traditional detection methods, improve the reliability of railway fastener defect detection in complex environments, and has strong versatility and adaptability.

Description

Technical Field

[0001] The present application relates to the field of defect detection technology, and more specifically, to a railway fastener defect detection method and system. Background Art

[0002] In modern railway transportation systems, the safety of track structures directly impacts train operating efficiency and the safety of life and property. Rail fasteners, as key components connecting track and sleepers, fulfill the core functions of securing rails and buffering vibration and shock. With the continued expansion of the high-speed rail network and the increasing age of existing lines, rail fasteners are prone to defects such as breakage, rust, loosening, and missing parts due to long-term exposure to complex environments. These defects can lead to track geometric instability, deterioration of train dynamics, and even derailment.

[0003] Traditional railway fastener defect detection relies primarily on manual inspections, requiring operators to carry inspection equipment and conduct point-by-point inspections along the track. This is not only inefficient and costly, but also limited by the human eye's ability to discern and subjective experience. The missed detection rate increases significantly in complex scenarios such as nighttime operations, rainy and snowy weather, or when fastener surfaces are covered with stains. In recent years, automated inspection technologies based on machine vision have gradually replaced manual inspections. However, existing automated inspection technologies often use fixed threshold segmentation or edge detection algorithms to extract fastener regions. These methods are less robust to illumination variations and complex background interference. Limited by issues such as ballast texture interference, uneven lighting, and reflective metal components in track scenes, they are prone to false detections due to oil stains, rust, or natural shadows on the rail surface. Furthermore, traditional algorithms often rely on preset fastener templates or a priori parameters, making them incompatible with the diverse morphological characteristics of different fastener models and lacking sensitivity to low-contrast defects such as fine cracks and localized corrosion.

[0004] Therefore, an optimized railway fastener defect detection method and system are desired. Summary of the Invention

[0005] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides a railway fastener defect detection method and system, which controls a high-speed industrial camera to scan the track area to collect track images, performs image preprocessing on the track images, and extracts ROI image blocks containing potential railway fasteners based on template matching. Then, a target detection algorithm is further introduced to accurately locate and segment the fasteners in the ROI image blocks to obtain more refined fastener ROI image blocks. Furthermore, by extracting the visual features of the fastener ROI image blocks and enhancing the significance of their feature space distribution, the expression effect of the visual features such as the color, texture, and shape of the fastener surface is enhanced, and then on this basis, intelligent recognition of railway fastener defects is achieved. This method can effectively overcome the limitations of traditional detection methods, improve the reliability of railway fastener defect detection in complex environments, and has strong versatility and adaptability.

[0006] According to one aspect of the present application, a method for detecting defects in railway fasteners is provided, comprising:

[0007] Control the high-speed industrial camera to scan the track area to obtain the original track image;

[0008] Performing image preprocessing and region of interest extraction on the original track image to obtain an ROI image block containing potential railway fasteners;

[0009] Performing fastener precise positioning and segmentation on the ROI image block containing the potential railway fastener to obtain a fastener ROI image block;

[0010] Extracting railway fastener visual features from the fastener ROI image block to obtain a railway fastener visual feature coding map;

[0011] performing feature space distribution significance marking on the railway fastener visual feature coding map to obtain a railway fastener visual feature space significant coding map;

[0012] Based on the railway fastener visual feature space significant coding map, it is determined whether the fastener has defects.

[0013] According to another aspect of the present application, a railway fastener defect detection system is provided, comprising:

[0014] The original track image acquisition module is used to control the high-speed industrial camera to scan the track area to obtain the original track image;

[0015] an ROI image block acquisition module, configured to perform image preprocessing and region of interest extraction on the original track image to obtain an ROI image block containing potential railway fasteners;

[0016] A fastener precise positioning and segmentation module, configured to perform precise fastener positioning and segmentation on the ROI image block containing potential railway fasteners to obtain a fastener ROI image block;

[0017] a visual feature extraction module, configured to extract visual features of railway fasteners from the fastener ROI image block to obtain a railway fastener visual feature coding map;

[0018] a feature space distribution significance identification module, configured to perform feature space distribution significance identification on the railway fastener visual feature coding map to obtain a railway fastener visual feature space significance coding map;

[0019] A defect detection module is used to determine whether a fastener has defects based on the significant coding map of the railway fastener visual feature space.

[0020] Compared with the existing technology, the railway fastener defect detection method and system provided by this application controls a high-speed industrial camera to scan the track area to collect track images, performs image preprocessing on the track images, and then extracts ROI image blocks containing potential railway fasteners based on template matching. Then, a target detection algorithm is further introduced to accurately locate and segment the fasteners in the ROI image blocks to obtain more refined fastener ROI image blocks. Furthermore, by extracting the visual features of the fastener ROI image blocks and enhancing the significance of their feature spatial distribution, the expression effect of visual features such as the fastener surface color, texture, and shape is enhanced, thereby realizing intelligent identification of railway fastener defects on this basis. This method can effectively overcome the limitations of traditional detection methods, improve the reliability of railway fastener defect detection in complex environments, and has strong versatility and adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and other purposes, features, and advantages of the present application will become more apparent through a more detailed description of the embodiments of the present application in conjunction with the accompanying drawings. The accompanying drawings are intended to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the drawings, the same reference numerals generally represent the same components or steps.

[0022] Figure 1 Flowchart of a railway fastener defect detection method according to an embodiment of the present application.

[0023] Figure 2 Schematic diagram of data flow of a railway fastener defect detection method according to an embodiment of the present application.

[0024] Figure 3 Flowchart of sub-step S2 of the railway fastener defect detection method according to an embodiment of the present application.

[0025] Figure 4 Flowchart of sub-step S5 of the railway fastener defect detection method according to an embodiment of the present application.

[0026] Figure 5 Flowchart of sub-step S53 of the railway fastener defect detection method according to an embodiment of the present application.

[0027] Figure 6 4 is a block diagram of a railway fastener defect detection system according to an embodiment of the present application. DETAILED DESCRIPTION

[0028] As used in this application and the claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" are not intended to refer to the singular but may include the plural. Generally speaking, the terms "comprises" and "include" only indicate the inclusion of the steps and elements specifically identified, and these steps and elements do not constitute an exclusive list. A method or apparatus may also include other steps or elements.

[0029] Although the present application makes various references to certain modules in the system according to embodiments of the present application, any number of different modules can be used and run on the user terminal and / or server. The modules are illustrative only, and different aspects of the system and method can use different modules.

[0030] Flowcharts are used in this application to illustrate the operations performed by the systems according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, the various steps may be processed in reverse order or simultaneously, as needed. Furthermore, other operations may be added to these processes, or one or more operations may be removed from these processes.

[0031] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described herein.

[0032] It is worth noting that in this application, all actions to obtain data are carried out in compliance with the relevant data protection laws and policies of the country where they are located and with the authorization given by the owner of the corresponding device.

[0033] In response to the technical problems described in the above background technology, this application proposes a railway fastener defect detection method, which controls a high-speed industrial camera to scan the track area to collect track images, performs image preprocessing on the track images, and extracts ROI image blocks containing potential railway fasteners based on template matching. Then, a target detection algorithm is further introduced to accurately locate and segment the fasteners in the ROI image blocks to obtain more refined fastener ROI image blocks. Furthermore, by extracting the visual features of the fastener ROI image blocks and enhancing the significance of their feature space distribution, the expression effect of the visual features such as the color, texture, and shape of the fastener surface is enhanced, and on this basis, intelligent identification of railway fastener defects is achieved. This method can effectively overcome the limitations of traditional detection methods, improve the reliability of railway fastener defect detection in complex environments, and has strong versatility and adaptability.

[0034] Figure 1 Flowchart of a railway fastener defect detection method according to an embodiment of the present application. Figure 2 Schematic diagram of data flow of the railway fastener defect detection method according to an embodiment of the present application. Figure 1 and Figure 2 As shown, the railway fastener defect detection method includes the following steps: S1, controlling a high-speed industrial camera to scan a track area to obtain an original track image; S2, performing image preprocessing and region of interest extraction on the original track image to obtain an ROI image block containing potential railway fasteners; S3, performing fastener precise positioning and segmentation on the ROI image block containing potential railway fasteners to obtain a fastener ROI image block; S4, extracting railway fastener visual features from the fastener ROI image block to obtain a railway fastener visual feature coding map; S5, performing feature space distribution significance identification on the railway fastener visual feature coding map to obtain a railway fastener visual feature space significant coding map; S6, determining whether the fastener has a defect based on the railway fastener visual feature space significant coding map.

[0035] In the above-mentioned railway fastener defect detection method, the step S1 controls the high-speed industrial camera to scan the track area to obtain the original track image. It should be understood that the railway fastener defect detection needs to be completed quickly during the train operation interval, and the high frame rate and high resolution characteristics of the high-speed industrial camera can ensure that clear track images are captured during dynamic scanning. At the same time, in order to overcome the problems of uneven natural lighting, metal reflections and low illumination at night during the image acquisition process, the present application controls the high-speed industrial camera to scan the track area while synchronously triggering the LED line light source to provide stable and uniform lighting conditions for image acquisition through active fill light, thereby reducing noise interference in subsequent processing. In this way, the coordinated work of high-speed imaging and active lighting helps to improve the clarity and contrast of the original track image, suppress the light reflection of interference areas such as oil stains and rust on the rail surface, and provide high-quality input data for subsequent processing.

[0036] Specifically, because railway fastener defect inspection must be performed in a dynamic environment, the camera's high frame rate and high resolution become key factors, thus placing strict demands on the selection of high-speed industrial cameras. A high frame rate ensures that sufficiently clear instantaneous images can be captured as a train passes, while high resolution helps improve the recognition of image details, which is particularly important for detecting low-contrast defects such as fine cracks or localized corrosion. Furthermore, considering the specific characteristics of the railway environment, such as varying lighting and complex background interference, it is essential to select industrial cameras with automatic exposure adjustment and high dynamic range (HDR) capabilities. Such cameras can automatically adjust parameters under varying lighting conditions to accommodate a range of conditions, from bright daylight to low nighttime illumination, thereby providing stable and consistent imaging results.

[0037] When it comes to the actual image acquisition process, in addition to relying on high-quality industrial cameras, it is also necessary to consider how to effectively deal with challenges such as uneven natural lighting, metal reflections, and low illumination at night. To this end, while controlling the high-speed industrial camera to scan the track area, it is particularly important to synchronously trigger the LED line light source to provide stable and uniform lighting conditions for image acquisition. In this way, not only can the shadows and uneven lighting caused by the changing position of the sun be compensated during the day, but sufficient lighting can also be provided during night operations to ensure that the quality of image acquisition is not affected. LED line light sources are an ideal choice because of their high efficiency, energy saving, concentrated light, and easy control. The light source can be arranged along the direction of the track to ensure that the entire area to be scanned is covered, reducing noise interference caused by insufficient light, and also avoiding image quality problems caused by overexposure or backlight.

[0038] Furthermore, in order to maximize the use of these hardware facilities, it is also crucial to optimize the shooting angle and distance. According to the actual layout of the track and the location distribution of the fasteners, the angle and height of the camera should be reasonably set to ensure that every fastener can be captured accurately. Usually, this may require multiple field visits and tests to find the best combination of shooting parameters. For example, in some cases, it may be necessary to install the camera at a certain angle to avoid direct sunlight or reduce reflected light from other directions; in other scenarios, it may be necessary to lower or raise the height of the camera to obtain a wider field of view or higher resolution. In short, by precisely adjusting the various parameters of the camera, the quality of the original track image can be significantly improved, laying a solid foundation for subsequent image preprocessing and region of interest extraction.

[0039] At the same time, attention should also be paid to the impact of environmental factors on image acquisition. For example, rainy and snowy weather may cause the track surface to become wet, increasing the possibility of reflections. It may also cause water droplets or snow to adhere to the fastener surface, affecting the accuracy of visual features. Therefore, when designing the system, it is necessary to consider how to still obtain clear and effective images under adverse weather conditions. One solution is to use equipment with a high waterproof and dustproof rating, and combine it with intelligent algorithms to evaluate the current environmental conditions in real time and automatically adjust the working mode of the camera and light source. For example, in the case of high humidity, the light source intensity can be appropriately increased or the camera shutter speed can be adjusted to reduce the blur caused by water droplets; in the case of snow cover, color filtering or other technical means can be used to remove the interference of white snowflakes to maintain the visibility of the fastener area.

[0040] Furthermore, given the vast scale of the railway network, achieving comprehensive coverage is no easy task. This requires that when planning image acquisition routes, not only the specific conditions of existing lines must be considered, but also sufficient flexibility must be reserved to accommodate the addition of new lines in the future. One feasible approach is to establish a management platform based on a geographic information system (GIS) to record key information for each line, including but not limited to track length, fastener type, and its distribution density. With this platform, resources can be flexibly dispatched according to different needs, and reasonable scanning plans can be formulated. For example, for critical sections with high traffic volume and high safety requirements, more frequent inspection cycles can be arranged; while for remote areas or less frequently used branch lines, the inspection interval can be appropriately extended according to actual conditions, ensuring both safety and efficiency.

[0041] In the above-mentioned railway fastener defect detection method, the step S2 performs image preprocessing and region of interest extraction on the original track image to obtain an ROI image block containing potential railway fasteners. It should be understood that since the original track image usually contains a large amount of redundant information and complex background (such as ballast and sleepers), direct fastener detection may lead to problems such as large computational complexity and low detection accuracy. Therefore, the present application further performs image preprocessing on the original track image to improve the image quality, and extracts the region of interest (ROI) image block containing potential railway fasteners from the preprocessed track image based on template matching technology to achieve preliminary positioning of the fasteners. Among them, Figure 3 FIG. 1 is a flow chart of sub-step S2 of the railway fastener defect detection method according to an embodiment of the present application. Figure 3 As shown, step S2 includes the following steps: S21, performing adaptive image enhancement on the original track image based on the Retinex algorithm to obtain an original track enhanced image; S22, performing image distortion correction on the original track enhanced image to obtain an original track re-enhanced image; and S23, extracting the ROI image block containing the potential railway fastener from the original track re-enhanced image based on template matching.

[0042] Specifically, the step S21 performs adaptive image enhancement on the original track image based on the Retinex algorithm to obtain an original track enhanced image. Specifically, although active fill light is used during the image acquisition process, the original track image acquired may still have problems such as uneven illumination and low contrast due to various factors such as hardware conditions, on-site environment and track material. Therefore, the present application adopts the Retinex algorithm to perform adaptive image enhancement processing on the original track image. As an image enhancement method based on the color constancy of the human visual system, the Retinex algorithm separates the illumination component and the reflection component of the original track image, and performs nonlinear stretching on the reflection component to enhance the local contrast of the image and improve the visual effect of the image, thereby obtaining an original track enhanced image with uniform illumination and rich details.

[0043] Specifically, step S22 performs image distortion correction on the original track-enhanced image to obtain a re-enhanced original track image. It should be understood that high-speed industrial cameras may experience image distortion during image acquisition due to factors such as lens distortion and camera installation angle, which can affect the accuracy of subsequent fastener inspection. Therefore, this application employs image distortion correction technology to correct the original track-enhanced image to eliminate image distortion, improve image quality, and obtain a more accurate and realistic re-enhanced original track image. Specifically, based on the Zhang Zhengyou calibration method, a checkerboard calibration plate (grid size 20mm×20mm) can be used to capture multi-view images, calculate camera intrinsic parameters (focal length, principal point) and distortion coefficients (k1, k2, p1, p2), generate a distortion correction mapping table, and apply this mapping table to perform pixel-by-pixel lookup correction on the original track-enhanced image, thereby eliminating distortion phenomena such as barrel distortion and pincushion distortion in the image and obtaining a re-enhanced original track image.

[0044] Specifically, step S23 extracts the ROI image block containing the potential railway fastener from the original track re-enhanced image based on template matching. It should be understood that track images contain a large amount of background (ballast, rails, and sleepers), making direct global detection computationally intensive and susceptible to interference. Therefore, this application first uses template matching to quickly screen suspicious areas in the original track re-enhanced image. Specifically, template matching, a classic image processing algorithm, locates target objects by calculating the similarity between the image to be inspected and a preset template. In this application, multiple railway fastener template images are pre-designed and stored in a database. Then, the template matching algorithm is used to search the original track re-enhanced image for regions similar to the template images. A normalized cross correlation (NCC) algorithm is used for sliding matching within the image. A threshold (e.g., 0.6) is set to select regions above the threshold as ROIs, thereby extracting ROI image blocks containing potential railway fasteners. This process allows for preliminary localization of railway fasteners, narrowing the processing scope of subsequent algorithms and laying the foundation for more refined fastener detection and defect identification.

[0045] In the above-mentioned railway fastener defect detection method, step S3 involves precisely locating and segmenting the ROI image block containing the potential railway fastener to obtain the fastener ROI image block. In a specific example of this application, step S3 includes inputting the ROI image block containing the potential railway fastener into an object detection model based on the Faster R-CNN model to obtain the fastener ROI image block. It should be understood that while template matching technology can initially screen out potential fastener regions, it is susceptible to interference from similar background (such as rail bolts and ballast blocks), resulting in positioning deviations and insufficient segmentation accuracy (pixel-level errors). Therefore, to further refine the positioning of railway fasteners and improve segmentation accuracy, this application introduces an object detection model based on the Faster R-CNN model to precisely locate and segment the ROI image block containing the potential railway fastener. Specifically, the Faster R-CNN model demonstrates strong performance in the field of object detection. By combining the Region Proposal Network (RPN) with ROI Align, it can perform fastener bounding box regression and mask segmentation, eliminating background interference. In this application, the ROI image block containing potential railway fasteners is fed into the Faster R-CNN model, which uses a convolutional neural network to extract high-level semantic features from the image. The model then generates a series of candidate regions using a region proposal network (RPN) and selects fastener candidate regions with high confidence. ROI Align technology then performs feature alignment on the fastener candidate regions to obtain more accurate fastener location information and segmentation masks, ultimately resulting in the fastener ROI image block.

[0046] In the above-mentioned railway fastener defect detection method, step S4 extracts railway fastener visual features from the fastener ROI image block to obtain a railway fastener visual feature encoding map. It should be understood that traditional hand-crafted features (such as SIFT and HOG) have limited ability to represent complex defects (such as microcracks and rust spots) and are difficult to adapt to the characteristic differences of multiple fastener models. Therefore, this application utilizes a lightweight deep network MobileNetV2 model to extract railway fastener visual features from the fastener ROI image block. As an efficient convolutional neural network architecture, the MobileNetV2 model, through designs such as depthwise separable convolution and inverted residual blocks, significantly reduces computational effort and model parameters while maintaining model performance, thereby improving feature extraction efficiency. In this application, the fastener ROI image block is input into the MobileNetV2 model. The model first uses 1x1 convolution to increase the number of channels (expansion), and then uses 3x3 depthwise separable convolution to perform spatial convolution on each feature channel to capture detailed features of the fastener image, such as the basic fastener morphology, subtle texture, and rust distribution. Next, the number of channels is reduced (compression) through 1x1 convolution, and the nonlinear activation function ReLU6 is introduced through the linear bottleneck layer to maintain the high-dimensional representation capability of the features. This extracts rich and discriminative visual features of railway fasteners and generates a railway fastener visual feature encoding map, providing a feature basis for subsequent fastener defect identification.

[0047] In the above-mentioned railway fastener defect detection method, the step S5 performs feature space distribution significance identification on the railway fastener visual feature coding map to obtain a railway fastener visual feature space significant coding map. Specifically, since fastener defects often manifest as abnormal features in local areas (such as cracks, missing blocks), these subtle features may be smoothed or lost due to pooling operations or feature dimensionality reduction during conventional feature extraction, thereby affecting the accuracy of defect recognition. In this regard, the present application proposes a feature space distribution significance enhancement method, which effectively anchors each pixel position in the railway fastener visual feature coding map based on the local receptive field range context information to enhance its feature expression ability by utilizing the local neighborhood context association information, so that each pixel position more accurately expresses the actual state of the fastener, thereby obtaining a railway fastener visual feature space significant coding map, thereby improving the robustness and accuracy of fastener defect recognition. Among them, Figure 4 FIG. 5 is a flow chart of sub-step S5 of the railway fastener defect detection method according to an embodiment of the present application. Figure 4As shown, the step S5 includes the steps of: S51, extracting the channel feature vector of the (i, j)th pixel position from the railway fastener visual feature coding map as the railway fastener pixel-level visual feature vector to be enhanced; S52, based on the feature distribution of the railway fastener pixel-level visual feature vector to be enhanced, performing local receptive field anchoring on the railway fastener visual feature coding map to screen out a set of railway fastener pixel-level visual feature vectors within the local receptive field; S53, based on the set of railway fastener pixel-level visual feature vectors within the local receptive field, performing feature association-guided saliency enhancement on the railway fastener pixel-level visual feature vector to be enhanced to obtain an enhanced railway fastener pixel-level visual feature vector, wherein the enhanced railway fastener pixel-level visual feature vector is the channel feature vector of the (i, j)th pixel position in the railway fastener visual feature space saliency coding map.

[0048] Specifically, in step S51, the channel feature vector at the (i, j)th pixel position is extracted from the railway fastener visual feature coding map as the railway fastener pixel-level visual feature vector to be enhanced, which is expressed as:

[0049]

[0050]

[0051] in, represents the set of real numbers, 、 and Respectively represent the height, width and number of channels of the railway fastener visual feature coding diagram, Represents the visual feature coding diagram of railway fasteners, Indicates the railway fastener visual feature coding diagram ( , ) pixel position channel feature vector, Represents the pixel-level visual feature vector of the railway fastener to be enhanced.

[0052] Specifically, by focusing on fundamental visual features such as color, texture, and shape of individual pixels, we provide refined pixel-level feature units for the subsequent saliency identification of feature space distribution, thereby enhancing the feature expression of local areas on the fastener surface. Specifically, by extracting pixel-by-pixel pixel-level visual feature vectors for railway fasteners to be enhanced, we can accurately capture subtle feature differences on the fastener surface, enhance the saliency distribution of each pixel in the feature space, and effectively improve the sensitivity and robustness of defect detection for fasteners of diverse shapes in complex environments.

[0053] Specifically, step S52 includes: first, compressing the pixel-level visual feature vector of the railway fastener to be enhanced to obtain a distilled encoding vector of the pixel-level visual feature of the railway fastener to be enhanced, which is expressed as follows:

[0054]

[0055] in, represents the calculation norm, Represents the distilled encoding vector of the pixel-level visual features to be enhanced for railway fasteners.

[0056] That is, since the pixel-level visual feature vectors of railway fasteners to be enhanced may have dimensional redundancy or noise interference, this application removes redundant information and extracts core features through an information compression mechanism, thereby avoiding the problem of defect recognition failure caused by feature redundancy. Specifically, by reducing feature dimensions or reducing information redundancy, focusing on core visual features related to fastener defects, and providing efficient and pure feature input for subsequent adaptive adjustment of the receptive field, the compressed pixel-level visual feature distillation encoding vectors of railway fasteners to be enhanced can more accurately retain key information related to the defects and remove interference from irrelevant noise such as uneven illumination, thereby effectively enhancing the model's defect detection capabilities for different types of fasteners in complex environments.

[0057] Secondly, based on the spatial structural characteristics of the feature distribution of the distilled coding vector of the pixel-level visual features to be enhanced for the railway fastener, the size of the feature receptive field of the distilled coding vector of the pixel-level visual features to be enhanced for the railway fastener is determined, which can be expressed as follows:

[0058]

[0059] in, represents the logarithmic function with base 2, Represents the half-side length of the feature receptive field of the distilled encoding vector of the pixel-level visual features to be enhanced for railway fasteners.

[0060] That is, a fixed receptive field can lead to missed detection of subtle defects and false detection of complex backgrounds. Therefore, this application performs feature distribution spatial structure analysis on the distilled encoding vectors of the pixel-level visual features to be enhanced for the railway fasteners, dynamically perceives the density of their feature distribution, and adaptively adjusts the size of the feature receptive field so that the receptive field size can flexibly change according to the complexity of local features. Contextual information is targeted to enhance the semantics and pertinence of feature saliency detection, thereby improving the accuracy of feature space saliency identification and enhancing the model's robustness to different types of fasteners and complex environments.

[0061] Then, feature decoupling is performed on the railway fastener visual feature encoding map along the channel dimension to obtain a set of railway fastener pixel-level visual feature vectors, which can be expressed as follows:

[0062]

[0063] in, 、 、 and They represent the (1,1), (1, ), No. ( ,1) and ( , ) pixel-level visual feature vector of the railway fastener at the pixel position, represents the feature decoupling function.

[0064] Specifically, through channel-wise decoupling, the channel feature vectors for each pixel in the railway fastener visual feature encoding map are isolated, breaking the correlation between channels and reducing channel coupling. This allows for subsequent refined processing of the single channel features of each pixel, thereby extracting more fine-grained feature information. The resulting set of pixel-level visual feature vectors for railway fasteners eliminates cross-channel interference, providing purer, more independent feature units for subsequent feature processing and effectively improving the ability to capture low-contrast defects.

[0065] Finally, based on the size of the feature receptive field, the set of pixel-level visual feature vectors of the railway fasteners within the local receptive field is filtered out from the set of pixel-level visual feature vectors of the railway fasteners, which can be expressed as follows:

[0066]

[0067] in, 、 、 and They represent the first ( , ), No. ( , ), No. ( , ) and ( , ) position of the railway fastener pixel-level visual feature vector, Represents the set of pixel-level visual feature vectors of railway fasteners within the local receptive field.

[0068] That is, according to the dynamically determined feature receptive field range, the pixel-level visual feature vectors of the railway fasteners in the area surrounding the distilled coding vector of the pixel-level visual features to be enhanced are accurately retrieved, a contextual information library containing multi-dimensional information such as texture, edge, and color is constructed, and a collection of pixel-level visual feature vectors of the railway fasteners within the local receptive field is screened out, so that the model can combine contextual semantics when judging defects, avoid misjudgment of single pixel features, and significantly improve the accuracy and robustness of defect detection under complex backgrounds.

[0069] Figure 5 FIG. 5 is a flow chart of sub-step S53 of the railway fastener defect detection method according to an embodiment of the present application. Figure 5 As shown, the step S53 includes the steps of: S531, based on the feature correlation of each local receptive field pixel-level visual feature vector of the railway fastener in the set of the local receptive field pixel-level visual feature vectors relative to the railway fastener pixel-level visual feature vector to be enhanced, performing significance aggregation on the set of the railway fastener pixel-level visual feature vectors in the local receptive field to obtain a significance aggregation coding vector of the railway fastener pixel-level visual features in the local receptive field; S532, fusing the railway fastener pixel-level visual feature significance aggregation coding vector in the local receptive field and the railway fastener pixel-level visual feature vector to be enhanced to obtain the enhanced railway fastener pixel-level visual feature vector.

[0070] More specifically, the step S531 is expressed as follows:

[0071]

[0072]

[0073] in, The first pixel in the set of visual feature vectors of railway fasteners in the local receptive field is represented by , ) position in the local receptive field of the railway fastener pixel-level visual feature vector, represents the transpose of a vector, represents vector multiplication, represents the softmax normalization function, express The significance weight of Represents the saliency aggregation encoding vector of pixel-level visual features of railway fasteners within the local receptive field.

[0074] That is, by quantifying the correlation between the pixel-level visual feature vectors of railway fasteners in each local receptive field and the pixel-level visual feature vectors of railway fasteners to be enhanced, the significantly correlated feature components are amplified and the non-significant components are suppressed. A saliency aggregation coding vector of the pixel-level visual features of railway fasteners in the local receptive field that integrates contextual semantics is constructed, so that the pixel-level visual features of railway fasteners to be enhanced can more accurately focus on the core information related to defects, providing high-purity feature input for subsequent defect judgment.

[0075] In a preferred example of the present application, step S532 includes: first, based on the conformal fusion commutation relation constraint of the set of pixel-level visual feature vectors of the railway fastener within the local receptive field, modulating the weighting coefficients of the railway fastener pixel-level visual feature saliency aggregation coding vector within the local receptive field and the railway fastener pixel-level visual feature vector to be enhanced, which can be expressed as follows:

[0076]

[0077]

[0078]

[0079] in, represents the sine function, A volume space representation vector representing a set of pixel-level visual feature vectors of railway fasteners within the local receptive field, A boundary representation vector representing a set of pixel-level visual feature vectors of railway fasteners within the local receptive field, and represent the weighting coefficients of the railway fastener pixel-level visual feature saliency aggregation coding vector within the local receptive field and the railway fastener pixel-level visual feature vector to be enhanced, respectively. represents the proportional scaling factor, Indicates the calculation of the bi-norm.

[0080] That is, by introducing the conformal fusion commutation relation constraint, the regularity and coordination of the spatial structure of the pixel-level visual feature vectors of railway fasteners within the local receptive field are ensured, thereby amplifying the feature components related to saliency and suppressing the non-saliency components. This makes the feature differences in the defect area more prominent under the conformal constraint, effectively reducing feature confusion in complex scenes such as uneven illumination, and providing structurally coordinated and saliency-enhanced feature input for subsequent defect judgment.

[0081] Then, based on the weighting coefficient, the railway fastener pixel-level visual feature saliency aggregate coding vector within the local receptive field and the railway fastener pixel-level visual feature vector to be enhanced are weightedly fused to obtain the enhanced railway fastener pixel-level visual feature vector, which is expressed as:

[0082]

[0083] in, Represents the pixel-level visual feature vector of enhanced railway fasteners.

[0084] That is, the weighted coefficient is used to fuse the saliency aggregation coding vector of the pixel-level visual features of the railway fasteners in the local receptive field and the pixel-level visual feature vector of the railway fasteners to be enhanced, so as to amplify the feature components related to saliency and suppress or weaken the feature components related to non-saliency, so that the enhanced pixel-level visual feature vector of the railway fasteners can be integrated with the local context information, more accurately reflect the saliency of the pixels, and enhance the expression effect of visual features such as the color, texture, and shape of the fastener surface, thereby effectively overcoming the influence of complex environment, improving the sensitivity of the model to defect detection, and improving the versatility and adaptability of the detection method and the detection reliability in complex scenarios.

[0085] In the above-mentioned railway fastener defect detection method, the step S6 determines whether the fastener has defects based on the railway fastener visual feature space significant coding map. In a specific example of the present application, the step S6 includes: inputting the railway fastener visual feature space significant coding map into a fastener defect recognition module based on a classifier to obtain a detection result, and the detection result is used to indicate whether the fastener has defects. That is, in order to realize the automatic identification of railway fastener defects, the present application constructs a fastener defect recognition module based on a classification algorithm to map the railway fastener visual feature space significant coding map into a defect probability and distinguish between normal fasteners and defective fasteners. Specifically, the railway fastener visual feature space significant coding map is input into a pre-trained classifier, and the classifier learns the feature distribution pattern of the railway fastener visual feature space significant coding map and maps it to a label space of defect or not based on the decision boundary or feature mapping relationship learned during the training process, thereby outputting a detection result. The test results are presented in a binary form, i.e., the fastener is normal or the fastener is defective, thereby providing railway maintenance personnel with intuitive fastener status information, facilitating timely implementation of corresponding maintenance measures to ensure the safety and stability of railway transportation.

[0086] In summary, a railway fastener defect detection method based on an embodiment of the present application is illustrated, which controls a high-speed industrial camera to scan a track area to collect track images, performs image preprocessing on the track images, and extracts ROI image blocks containing potential railway fasteners based on template matching. Subsequently, a target detection algorithm is further introduced to accurately locate and segment the fasteners in the ROI image blocks to obtain more refined fastener ROI image blocks. Furthermore, by extracting the visual features of the fastener ROI image blocks and enhancing the significance of their feature spatial distribution, the expression effect of the visual features such as the color, texture, and shape of the fastener surface is enhanced, thereby realizing intelligent recognition of railway fastener defects on this basis. This method can effectively overcome the limitations of traditional detection methods, improve the reliability of railway fastener defect detection in complex environments, and has strong versatility and adaptability.

[0087] Furthermore, a railway fastener defect detection system is also provided.

[0088] Figure 6 FIG is a block diagram of a railway fastener defect detection system according to an embodiment of the present application. Figure 6 As shown, the railway fastener defect detection system 100 according to an embodiment of the present application includes: an original track image acquisition module 110, which is used to control a high-speed industrial camera to scan a track area to obtain an original track image; an ROI image block acquisition module 120, which is used to perform image preprocessing and region of interest extraction on the original track image to obtain an ROI image block containing potential railway fasteners; a fastener precise positioning and segmentation module 130, which is used to perform fastener precise positioning and segmentation on the ROI image block containing potential railway fasteners to obtain a fastener ROI image block; a visual feature extraction module 140, which is used to extract railway fastener visual features from the fastener ROI image block to obtain a railway fastener visual feature coding map; a feature space distribution significance identification module 150, which is used to perform feature space distribution significance identification on the railway fastener visual feature coding map to obtain a railway fastener visual feature space significant coding map; and a defect detection module 160, which is used to determine whether a fastener has a defect based on the railway fastener visual feature space significant coding map.

[0089] Here, those skilled in the art will appreciate that the specific operations of each module in the above railway fastener defect detection system have been described in detail above. Figures 1 to 5 The description of the railway fastener defect detection method has been described in detail, and therefore, its repeated description will be omitted.

[0090] The basic principles of the present invention have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, and effects mentioned in the present invention are merely illustrative and non-limiting, and should not be construed as necessarily possessed by each embodiment of the present invention. Furthermore, the specific details of the above embodiments are provided for illustrative purposes and to facilitate understanding, and are not intended to be limiting. These details do not necessarily limit the present invention to being implemented using these specific details.

[0091] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, please refer to the relevant description of other embodiments. In the several embodiments provided by the present invention, it should be understood that the disclosed system and method can be implemented in other ways. For example, the system embodiment described above is only schematic. For example, the unit division is only a logical function division, and there may be other division methods in actual implementation. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0092] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the invention can be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims, not the foregoing description, and all variations within the meaning and range of equivalents of the claims are intended to be encompassed therein. Any reference to a figure in a claim should not be construed as limiting the claim to which it relates.

[0093] In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units stated in the system claims can also be implemented by one unit through software or hardware.

[0094] Finally, it should be noted that the above description has been provided for purposes of illustration and description. Furthermore, the above embodiments are intended only to illustrate the technical solutions of the present invention and are not intended to be limiting. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art will appreciate that the technical solutions of the present invention may be modified or replaced with equivalents without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for detecting defects in railway fasteners, characterized in that: include: Control the high-speed industrial camera to scan the track area to obtain the original track image; Performing image preprocessing and region of interest extraction on the original track image to obtain an ROI image block containing potential railway fasteners; Performing fastener precise positioning and segmentation on the ROI image block containing the potential railway fastener to obtain a fastener ROI image block; Extracting railway fastener visual features from the fastener ROI image block to obtain a railway fastener visual feature coding map; performing feature space distribution significance marking on the railway fastener visual feature coding map to obtain a railway fastener visual feature space significant coding map; determining whether the fastener has a defect based on the railway fastener visual feature space saliency coding map; Performing feature space distribution significance marking on the railway fastener visual feature coding map to obtain a railway fastener visual feature space significant coding map, including: Extracting the channel feature vector of the (i, j)th pixel position from the railway fastener visual feature coding map as the railway fastener pixel-level visual feature vector to be enhanced; Based on the feature distribution of the pixel-level visual feature vectors to be enhanced for the railway fastener, anchoring the railway fastener visual feature encoding map to a local receptive field to screen out a set of railway fastener pixel-level visual feature vectors within the local receptive field; Based on the set of pixel-level visual feature vectors of the railway fasteners within the local receptive field, the pixel-level visual feature vectors of the railway fasteners to be enhanced are subjected to saliency enhancement based on feature association guidance to obtain an enhanced pixel-level visual feature vector of the railway fasteners, wherein the enhanced pixel-level visual feature vector of the railway fasteners is the channel feature vector of the (i, j)th pixel position in the saliency coding map of the railway fastener visual feature space.

2. The railway fastener defect detection method according to claim 1, characterized in that: Controlling a high-speed industrial camera to scan a track area to obtain an original track image includes: synchronously triggering an LED line light source while controlling the high-speed industrial camera to scan the track area.

3. The railway fastener defect detection method according to claim 2, characterized in that: Performing image preprocessing and region of interest extraction on the original track image to obtain an ROI image block containing potential railway fasteners includes: Performing adaptive image enhancement on the original track image based on a Retinex algorithm to obtain an original track enhanced image; Performing image distortion correction on the original track enhanced image to obtain an original track re-enhanced image; The ROI image block containing the potential railway fastener is extracted from the original track enhanced image based on template matching.

4. The railway fastener defect detection method according to claim 3, characterized in that: Performing precise fastener positioning and segmentation on the ROI image block containing potential railway fasteners to obtain a fastener ROI image block, including: The ROI image block containing the potential railway fastener is input into a target detection model based on the Faster R-CNN model to obtain the fastener ROI image block.

5. The railway fastener defect detection method according to claim 4, characterized in that: Based on the feature distribution of the pixel-level visual feature vector to be enhanced of the railway fastener, local receptive field anchoring is performed on the railway fastener visual feature encoding map to screen out a set of railway fastener pixel-level visual feature vectors within the local receptive field, including: performing information compression on the pixel-level visual feature vector of the railway fastener to be enhanced to obtain a distilled encoding vector of the pixel-level visual feature of the railway fastener to be enhanced; Determining the size of a feature receptive field of the distilled coding vector of the pixel-level visual features to be enhanced for the railway fastener based on the feature distribution spatial structure characteristics of the distilled coding vector of the pixel-level visual features to be enhanced for the railway fastener; Performing feature decoupling on the railway fastener visual feature encoding map along a channel dimension to obtain a set of railway fastener pixel-level visual feature vectors; Based on the size of the feature receptive field, a set of pixel-level visual feature vectors of railway fasteners within the local receptive field is filtered out from the set of pixel-level visual feature vectors of railway fasteners.

6. The railway fastener defect detection method according to claim 5, characterized in that: Based on the set of pixel-level visual feature vectors of the railway fastener within the local receptive field, performing saliency enhancement on the to-be-enhanced pixel-level visual feature vectors of the railway fastener based on feature association guidance to obtain an enhanced pixel-level visual feature vector of the railway fastener, comprising: performing saliency aggregation on the set of pixel-level visual feature vectors of the railway fastener within the local receptive field based on feature correlations of each pixel-level visual feature vector of the railway fastener within the local receptive field with respect to the pixel-level visual feature vector of the railway fastener to be enhanced, so as to obtain a saliency aggregation coding vector of the pixel-level visual features of the railway fastener within the local receptive field; The enhanced railway fastener pixel-level visual feature vector is obtained by fusing the railway fastener pixel-level visual feature saliency aggregation coding vector within the local receptive field and the railway fastener pixel-level visual feature vector to be enhanced.

7. The railway fastener defect detection method according to claim 6, characterized in that: The enhanced railway fastener pixel-level visual feature vector is obtained by fusing the railway fastener pixel-level visual feature saliency aggregate coding vector within the local receptive field and the railway fastener pixel-level visual feature vector to be enhanced, comprising: Based on the conformal fusion commutation relation constraint of the set of pixel-level visual feature vectors of the railway fastener within the local receptive field, modulating the weighting coefficients of the railway fastener pixel-level visual feature saliency aggregation coding vector within the local receptive field and the railway fastener pixel-level visual feature vector to be enhanced; Based on the weighting coefficient, the railway fastener pixel-level visual feature saliency aggregation coding vector within the local receptive field and the railway fastener pixel-level visual feature vector to be enhanced are weightedly fused to obtain the enhanced railway fastener pixel-level visual feature vector.

8. The railway fastener defect detection method according to claim 7, characterized in that: Determining whether the fastener has a defect based on the railway fastener visual feature space significant coding map includes: The railway fastener visual feature space saliency coding map is input into a classifier-based fastener defect recognition module to obtain a detection result, and the detection result is used to indicate whether the fastener has a defect.

9. A railway fastener defect detection system, characterized in that: include: The original track image acquisition module is used to control the high-speed industrial camera to scan the track area to obtain the original track image; an ROI image block acquisition module, configured to perform image preprocessing and region of interest extraction on the original track image to obtain an ROI image block containing potential railway fasteners; A fastener precise positioning and segmentation module, configured to perform precise fastener positioning and segmentation on the ROI image block containing potential railway fasteners to obtain a fastener ROI image block; a visual feature extraction module, configured to extract visual features of railway fasteners from the fastener ROI image block to obtain a railway fastener visual feature coding map; a feature space distribution significance identification module, configured to perform feature space distribution significance identification on the railway fastener visual feature coding map to obtain a railway fastener visual feature space significance coding map; a defect detection module for determining whether a fastener has defects based on the railway fastener visual feature space significant coding map; The feature space distribution significance identification module is used to: Extracting the channel feature vector of the (i, j)th pixel position from the railway fastener visual feature coding map as the railway fastener pixel-level visual feature vector to be enhanced; Based on the feature distribution of the pixel-level visual feature vectors to be enhanced for the railway fastener, anchoring the railway fastener visual feature encoding map to a local receptive field to screen out a set of railway fastener pixel-level visual feature vectors within the local receptive field; Based on the set of pixel-level visual feature vectors of the railway fasteners within the local receptive field, the pixel-level visual feature vectors of the railway fasteners to be enhanced are subjected to saliency enhancement based on feature association guidance to obtain an enhanced pixel-level visual feature vector of the railway fasteners, wherein the enhanced pixel-level visual feature vector of the railway fasteners is the channel feature vector of the (i, j)th pixel position in the saliency coding map of the railway fastener visual feature space.

Images