Railway fastener defect detection method and system

Abstract

The invention relates to the technical field of defect detection, and particularly discloses a railway fastener defect detection method and system, and the method comprises the steps: controlling a high-speed industrial camera to scan a track region so as to collect a track image, carrying out the image preprocessing of the track image, extracting an ROI image block containing a potential railway fastener based on template matching, and then carrying out the detection of the potential railway fastener. The method further introduces a target detection algorithm to perform fastener accurate positioning and segmentation on the ROI image blocks to obtain finer fastener ROI image blocks, further extracts visual features of the fastener ROI image blocks, and performs feature space distribution saliency enhancement on the visual features to enhance expression effects of the visual features such as surface color, texture and shape of the fastener, so as to improve the image quality of the fastener. Therefore, intelligent identification of railway fastener defects is realized on the basis. According to the method, the limitation of a traditional detection method can be effectively overcome, the reliability of railway fastener defect detection in a complex environment is improved, and the method has relatively high universality and adaptability.

Description

Technical Field

[0001] The present application relates to the technical field of defect detection, and more specifically, to a railway fastener defect detection method and system. Background Art

[0002] In modern railway transportation systems, the safety of track structures is directly related to the efficiency of train operation and the safety of life and property of personnel. Railway fasteners, as key components connecting tracks and sleepers, have the core functions of fixing the position of rails and buffering vibration and impact. With the continuous expansion of the high-speed railway network and the increasing operating years of existing lines, railway fasteners are prone to defects such as breakage, rust, looseness, and missing due to long-term exposure to complex environments. Such defects may lead to instability of track geometry, deterioration of train dynamics, and even derailment accidents.

[0003] Traditional railway fastener defect detection mainly relies on manual inspections. Operators need to carry detection equipment and check point by point along the track. This is not only inefficient and costly, but also limited by the human eye's ability to distinguish and subjective experience. In complex scenarios such as nighttime operations, rainy and snowy weather, or stains on the surface of fasteners, the missed detection rate increases significantly. In recent years, automated detection technology based on machine vision has gradually replaced manual inspections, but existing automated detection technologies mostly use fixed threshold segmentation or edge detection algorithms to extract fastener areas. Such methods have poor robustness to changes in illumination and complex background interference. They are limited by ballast texture interference, uneven illumination, and reflections of metal parts in track scenes, and are prone to false detection due to oil stains, rust, or natural shadows on the rail surface. At the same time, traditional algorithms usually rely on preset fastener templates or prior parameters, which are difficult to be compatible with the diverse morphological features of fasteners of different models, and are not sensitive enough to low-contrast defects such as fine cracks and local corrosion.

[0004] Therefore, an optimized railway fastener defect detection method and system are desired. Summary of the invention

[0005] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides a railway fastener defect detection method and system, which controls a high-speed industrial camera to scan the track area to collect track images, performs image preprocessing on the track images, and extracts ROI image blocks containing potential railway fasteners based on template matching. Then, a target detection algorithm is further introduced to accurately locate and segment the ROI image blocks of fasteners to obtain more refined fastener ROI image blocks, and then, by extracting the visual features of the fastener ROI image blocks and enhancing the significance of their feature space distribution, the expression effect of the visual features of the fastener surface color, texture, shape, etc. is enhanced, and then on this basis, intelligent recognition of railway fastener defects is achieved. This method can effectively overcome the limitations of traditional detection methods, improve the reliability of railway fastener defect detection in complex environments, and has strong versatility and adaptability.

[0006] According to one aspect of the present application, a railway fastener defect detection method is provided, which comprises: Control the high-speed industrial camera to scan the track area to obtain the original track image; Performing image preprocessing and region of interest extraction on the original track image to obtain a ROI image block containing potential railway fasteners; Performing fastener precise positioning and segmentation on the ROI image block containing potential railway fasteners to obtain a fastener ROI image block; Extracting railway fastener visual features from the fastener ROI image block to obtain a railway fastener visual feature coding map; Performing feature space distribution significance marking on the railway fastener visual feature coding map to obtain a railway fastener visual feature space significant coding map; Based on the railway fastener visual feature space significant coding map, it is determined whether the fastener has defects.

[0007] According to another aspect of the present application, a railway fastener defect detection system is provided, comprising: The original track image acquisition module is used to control the high-speed industrial camera to scan the track area to obtain the original track image; An ROI image block acquisition module is used to perform image preprocessing and region of interest extraction on the original track image to obtain an ROI image block containing potential railway fasteners; A fastener precise positioning and segmentation module, used for performing fastener precise positioning and segmentation on the ROI image block containing potential railway fasteners to obtain a fastener ROI image block; A visual feature extraction module, used for extracting railway fastener visual features from the fastener ROI image block to obtain a railway fastener visual feature coding map; A feature space distribution significance identification module is used to perform feature space distribution significance identification on the railway fastener visual feature coding map to obtain a railway fastener visual feature space significance coding map; The defect detection module is used to determine whether the fastener has defects based on the significant coding map of the visual feature space of the railway fastener.

[0008] Compared with the prior art, the railway fastener defect detection method and system provided by the present application controls a high-speed industrial camera to scan the track area to collect track images, performs image preprocessing on the track images, and extracts ROI image blocks containing potential railway fasteners based on template matching. Then, a target detection algorithm is further introduced to accurately locate and segment the ROI image blocks of fasteners to obtain more refined fastener ROI image blocks. Then, by extracting the visual features of the fastener ROI image blocks and enhancing the significance of their feature space distribution, the expression effect of the visual features of the fastener surface color, texture, shape, etc. is enhanced, and then on this basis, intelligent recognition of railway fastener defects is realized. This method can effectively overcome the limitations of traditional detection methods, improve the reliability of railway fastener defect detection in complex environments, and has strong versatility and adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0010] Figure 1 Flow chart of a railway fastener defect detection method according to an embodiment of the present application.

[0011] Figure 2 Schematic diagram of data flow of a railway fastener defect detection method according to an embodiment of the present application.

[0012] Figure 3 Flow chart of sub-step S2 of the railway fastener defect detection method according to an embodiment of the present application.

[0013] Figure 4 Flow chart of sub-step S5 of the railway fastener defect detection method according to an embodiment of the present application.

[0014] Figure 5 Flow chart of sub-step S53 of the railway fastener defect detection method according to an embodiment of the present application.

[0015] Figure 64 is a block diagram of a railway fastener defect detection system according to an embodiment of the present application. DETAILED DESCRIPTION

[0016] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0017] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules can be used and run on the user terminal and / or server. The modules are only illustrative, and different aspects of the system and method can use different modules.

[0018] Flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps may be processed in reverse order or simultaneously as required. Meanwhile, other operations may also be added to these processes, or a certain step or several steps of operations may be removed from these processes.

[0019] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0020] It is worth noting that in this application, all actions to obtain data are carried out in compliance with the relevant data protection laws and policies of the country where the data is located, and with the authorization given by the owner of the corresponding device.

[0021] In response to the technical problems described in the above background technology, this application proposes a railway fastener defect detection method, which controls a high-speed industrial camera to scan the track area to collect track images, performs image preprocessing on the track images, and extracts ROI image blocks containing potential railway fasteners based on template matching. Then, a target detection algorithm is further introduced to accurately locate and segment the ROI image blocks of fasteners to obtain more refined fastener ROI image blocks, and then, by extracting the visual features of the fastener ROI image blocks and enhancing the significance of their feature space distribution, the expression effect of the visual features of the fastener surface color, texture, shape, etc. is enhanced, and then on this basis, intelligent recognition of railway fastener defects is realized. This method can effectively overcome the limitations of traditional detection methods, improve the reliability of railway fastener defect detection in complex environments, and has strong versatility and adaptability.

[0022] Figure 1 Flow chart of a railway fastener defect detection method according to an embodiment of the present application. Figure 2 Schematic diagram of data flow of the railway fastener defect detection method according to an embodiment of the present application. Figure 1 and Figure 2 As shown, the railway fastener defect detection method includes the following steps: S1, controlling a high-speed industrial camera to scan a track area to obtain an original track image; S2, performing image preprocessing and region of interest extraction on the original track image to obtain an ROI image block containing potential railway fasteners; S3, performing fastener precise positioning and segmentation on the ROI image block containing potential railway fasteners to obtain a fastener ROI image block; S4, extracting railway fastener visual features from the fastener ROI image block to obtain a railway fastener visual feature coding map; S5, performing feature space distribution significance marking on the railway fastener visual feature coding map to obtain a railway fastener visual feature space significant coding map; S6, determining whether the fastener has a defect based on the railway fastener visual feature space significant coding map.

[0023] In the above-mentioned railway fastener defect detection method, the step S1 controls the high-speed industrial camera to scan the track area to obtain the original track image. It should be understood that the railway fastener defect detection needs to be completed quickly during the train operation interval, and the high frame rate and high resolution characteristics of the high-speed industrial camera can ensure that a clear track image is captured during dynamic scanning. At the same time, in order to overcome the problems of uneven natural lighting, metal reflections and low illumination at night during image acquisition, the present application controls the high-speed industrial camera to scan the track area while synchronously triggering the LED line light source to provide stable and uniform lighting conditions for image acquisition through active fill light, thereby reducing noise interference in subsequent processing. In this way, the coordinated work of high-speed imaging and active lighting helps to improve the clarity and contrast of the original track image, suppress the reflection of light in interference areas such as oil stains and rust on the rail surface, and provide high-quality input data for subsequent processing.

[0024] Specifically, since railway fastener defect detection needs to be carried out in a dynamic environment, the high frame rate and high resolution characteristics of the camera become key factors. Therefore, strict requirements are required for the selection of high-speed industrial cameras. The high frame rate ensures that a sufficiently clear instantaneous picture can be captured when the train passes, while the high resolution helps to improve the recognition of image details, which is especially important for discovering low-contrast defects such as fine cracks or local corrosion. In addition, considering the particularity of the railway environment, such as lighting changes, complex background interference and other factors, it is also necessary to select an industrial camera with automatic exposure adjustment function and high dynamic range (HDR) capability. This camera can automatically adjust parameters under different lighting conditions to adapt to various situations from strong light during the day to low illumination at night, thereby providing stable and consistent imaging effects.

[0025] When it comes to the actual image acquisition process, in addition to relying on high-quality industrial cameras, it is also necessary to consider how to effectively deal with challenges such as uneven natural lighting, metal reflections, and low illumination at night. To this end, while controlling the high-speed industrial camera to scan the track area, it is particularly important to synchronously trigger the LED line light source to provide stable and uniform lighting conditions for image acquisition. In this way, not only can the shadows and uneven lighting caused by the change in the position of the sun be compensated during the day, but also sufficient lighting can be provided during night operations to ensure that the quality of image acquisition is not affected. LED line light sources are an ideal choice because of their high efficiency, energy saving, concentrated light, and easy control. The light source can be arranged along the direction of the track to ensure that the entire area to be scanned is covered, reducing noise interference caused by insufficient light, and also avoiding image quality problems caused by overexposure or backlight.

[0026] Furthermore, in order to maximize the use of these hardware facilities, it is also crucial to optimize the shooting angle and distance. According to the actual layout of the track and the location distribution of the fasteners, the angle and height of the camera should be set reasonably to ensure that every fastener can be captured accurately. Usually, this may require multiple field visits and tests to find the best combination of shooting parameters. For example, in some cases, it may be necessary to install the camera at a certain angle to avoid direct sunlight or reduce reflected light from other directions; in other scenarios, it may be necessary to lower or raise the height of the camera to obtain a wider field of view or higher resolution. In short, by precisely adjusting the various parameters of the camera, the quality of the original track image can be significantly improved, laying a solid foundation for subsequent image preprocessing and extraction of regions of interest.

[0027] At the same time, attention should also be paid to the impact of environmental factors on image acquisition. For example, rainy and snowy weather may cause the track surface to become wet, increasing the possibility of reflections, and also causing water droplets or snow to adhere to the surface of the fasteners, affecting the accuracy of visual features. Therefore, when designing the system, it is necessary to consider how to still obtain clear and effective images under adverse weather conditions. One solution is to use equipment with a high level of waterproof and dustproof, and combine it with intelligent algorithms to evaluate the current environmental status in real time and automatically adjust the working mode of the camera and light source. For example, in the case of high humidity, appropriately increase the intensity of the light source or adjust the shutter speed of the camera to reduce the blur caused by water droplets; in the case of snow cover, color filtering or other technical means can be used to remove the interference of white snowflakes to maintain visibility of the fastener area.

[0028] In addition, considering the huge scale of the railway network, it is not easy to achieve comprehensive coverage. This requires that when planning the image acquisition route, not only the specific conditions of the existing lines should be taken into account, but also sufficient flexibility should be reserved to accommodate the addition of new lines in the future. A feasible method is to establish a management platform based on a geographic information system (GIS) to record the key information of each line, including but not limited to the length of the track, the type of fasteners and their distribution density. With the help of this platform, resources can be flexibly dispatched according to different needs and a reasonable scanning plan can be formulated. For example, for key sections with high traffic volume and high safety requirements, more frequent inspection cycles can be arranged; for remote areas or branches with low frequency of use, the inspection interval can be appropriately extended according to the actual situation, which not only ensures safety but also improves efficiency.

[0029] In the above-mentioned railway fastener defect detection method, the step S2 performs image preprocessing and region of interest extraction on the original track image to obtain an ROI image block containing potential railway fasteners. It should be understood that since the original track image usually contains a large amount of redundant information and complex background (such as ballast and sleepers), direct fastener detection may lead to problems such as large amount of calculation and low detection accuracy. Therefore, the present application further performs image preprocessing on the original track image to improve the image quality, and extracts the region of interest (ROI) image block containing potential railway fasteners from the preprocessed track image based on template matching technology to achieve preliminary positioning of the fasteners. Among them, Figure 3 FIG. 1 is a flowchart of sub-step S2 of the railway fastener defect detection method according to an embodiment of the present application. Figure 3 As shown, the step S2 includes the steps of: S21, performing adaptive image enhancement on the original track image based on the Retinex algorithm to obtain an original track enhanced image; S22, performing image distortion correction on the original track enhanced image to obtain an original track re-enhanced image; S23, extracting the ROI image block containing the potential railway fastener from the original track re-enhanced image based on template matching.

[0030] Specifically, the step S21, based on the Retinex algorithm, performs adaptive image enhancement on the original track image to obtain an enhanced original track image. Specifically, although active fill light is used during the image acquisition process, the acquired original track image may still have problems such as uneven illumination and low contrast due to various factors such as hardware conditions, on-site environment and track material. Therefore, the present application adopts the Retinex algorithm to perform adaptive image enhancement processing on the original track image. As an image enhancement method based on the color constancy of the human visual system, the Retinex algorithm separates the illumination component and the reflection component of the original track image, and performs nonlinear stretching on the reflection component to enhance the local contrast of the image and improve the visual effect of the image, thereby obtaining an enhanced original track image with uniform illumination and rich details.

[0031] Specifically, in step S22, the original track enhanced image is subjected to image distortion correction to obtain the original track re-enhanced image. It should be understood that, considering that high-speed industrial cameras may cause image distortion due to factors such as lens distortion and camera installation angle when acquiring images, the accuracy of subsequent fastener detection is affected. Therefore, the present application adopts image distortion correction technology to correct the original track enhanced image to eliminate the distortion phenomenon in the image, improve the image quality, and obtain a more accurate and true original track re-enhanced image. Specifically, based on the Zhang Zhengyou calibration method, a chessboard calibration plate (square size 20mm×20mm) can be used to collect multi-view images, calculate the camera internal parameters (focal length, principal point) and distortion coefficients (k1, k2, p1, p2), generate a distortion correction mapping table, and apply the mapping table to perform pixel-by-pixel lookup correction on the original track enhanced image, thereby eliminating distortion phenomena such as barrel distortion and pincushion distortion in the image, and obtaining the original track re-enhanced image.

[0032] Specifically, in step S23, the ROI image block containing potential railway fasteners is extracted from the original track enhanced image based on template matching. It should be understood that the track image contains a large amount of background (ballast, rails, sleepers), and direct global detection has a large amount of calculation and is susceptible to interference. Therefore, the present application first quickly screens the suspicious area in the original track enhanced image by template matching. Specifically, template matching technology is a classic image processing algorithm, which realizes the positioning of the target object by calculating the similarity between the image to be detected and the preset template. In the present application, a variety of railway fastener template images are pre-designed and stored in the database. Then, the template matching algorithm is used to search for areas similar to the template image in the original track enhanced image, and the normalized cross correlation (NCC) algorithm is used for sliding matching in the image. A threshold (such as 0.6) is set to screen the area above the threshold as the ROI, thereby extracting the ROI image block containing potential railway fasteners. Through this process, the preliminary positioning of railway fasteners can be achieved, the processing range of subsequent algorithms can be narrowed, and the foundation for more sophisticated fastener detection and defect identification can be laid.

[0033] In the above-mentioned railway fastener defect detection method, the step S3 performs fastener precise positioning and segmentation on the ROI image block containing potential railway fasteners to obtain the fastener ROI image block. In a specific example of the present application, the step S3 includes: inputting the ROI image block containing potential railway fasteners into a target detection model based on the Faster R-CNN model to obtain the fastener ROI image block. It should be understood that although the template matching technology can preliminarily screen out the potential fastener area, it is easily disturbed by similar backgrounds (such as rail bolts and ballast blocks), resulting in positioning deviations, and the segmentation accuracy is insufficient (pixel-level error). Therefore, in order to further refine the positioning of railway fasteners and improve the segmentation accuracy, the present application introduces a target detection model based on the Faster R-CNN model to perform fastener precise positioning and segmentation on the ROI image block containing potential railway fasteners. Specifically, the Faster R-CNN model shows strong performance in the field of target detection. It combines the region proposal network (RPN) with ROI Align to achieve fastener bounding box regression and mask segmentation, eliminating background interference. In this application, the ROI image block containing potential railway fasteners is input into the Faster R-CNN model, which uses a convolutional neural network to extract high-level semantic features of the image, generates a series of candidate regions through a region proposal network (RPN), and screens out fastener candidate regions with high confidence. Subsequently, the ROI Align technology is used to align the fastener candidate regions to obtain more accurate fastener location information and segmentation masks, and obtain fastener ROI image blocks.

[0034] In the above-mentioned railway fastener defect detection method, the step S4 extracts the railway fastener visual features from the fastener ROI image block to obtain the railway fastener visual feature encoding map. It should be understood that the traditional manual features (such as SIFT, HOG) have limited characterization capabilities for complex defects (such as microcracks, rust spots), and are difficult to adapt to the feature differences of multiple models of fasteners. In this regard, the present application adopts a lightweight deep network MobileNetV2 model to extract the railway fastener visual features in the fastener ROI image block. As an efficient convolutional neural network structure, the MobileNetV2 model can greatly reduce the amount of calculation and model parameters while ensuring the performance of the model through designs such as deep separable convolution and inverted residual blocks, thereby improving the efficiency of feature extraction. In the present application, the fastener ROI image block is input into the MobileNetV2 model, which first uses 1x1 convolution to increase the number of channels (expansion), and then uses 3x3 deep separable convolution to perform spatial convolution on each feature channel to capture the detailed features of the fastener image, such as the basic shape of the fastener, subtle texture, rust distribution, etc. Next, the number of channels is reduced (compression) through 1x1 convolution, and the nonlinear activation function ReLU6 is introduced through the linear bottleneck layer to maintain the high-dimensional representation capability of the features, thereby extracting rich and discriminative visual features of railway fasteners and generating a visual feature encoding map of railway fasteners, providing a feature basis for subsequent fastener defect identification.

[0035] In the above-mentioned railway fastener defect detection method, the step S5 performs feature space distribution significance marking on the railway fastener visual feature coding map to obtain a railway fastener visual feature space significant coding map. Specifically, since fastener defects often manifest as abnormal features in local areas (such as cracks, missing pieces), these subtle features may be smoothed or lost due to pooling operations or feature dimensionality reduction during conventional feature extraction, thereby affecting the accuracy of defect identification. In this regard, the present application proposes a feature space distribution significance enhancement method, which effectively anchors each pixel position in the railway fastener visual feature coding map based on the local receptive field range context information, so as to enhance its feature expression ability by utilizing the local neighborhood context association information, so that each pixel position more accurately expresses the actual state of the fastener, thereby obtaining a railway fastener visual feature space significant coding map, thereby improving the robustness and accuracy of fastener defect identification. Among them, Figure 4 FIG. 5 is a flowchart of sub-step S5 of the railway fastener defect detection method according to an embodiment of the present application. Figure 4As shown, the step S5 includes the steps of: S51, extracting the channel feature vector of the (i, j)th pixel position from the railway fastener visual feature coding map as the railway fastener pixel-level visual feature vector to be enhanced; S52, based on the feature distribution of the railway fastener pixel-level visual feature vector to be enhanced, anchoring the railway fastener visual feature coding map locally to screen out a set of railway fastener pixel-level visual feature vectors within the local receptive field; S53, based on the set of railway fastener pixel-level visual feature vectors within the local receptive field, performing feature association-guided significance enhancement on the railway fastener pixel-level visual feature vector to be enhanced to obtain an enhanced railway fastener pixel-level visual feature vector, wherein the enhanced railway fastener pixel-level visual feature vector is the channel feature vector of the (i, j)th pixel position in the railway fastener visual feature space significance coding map.

[0036] Specifically, in step S51, the channel feature vector at the (i, j)th pixel position is extracted from the railway fastener visual feature coding map as the railway fastener pixel-level visual feature vector to be enhanced, which is expressed by the formula: in, represents the set of real numbers, , and They represent the height, width and number of channels of the railway fastener visual feature coding diagram, Represents the visual feature coding diagram of railway fasteners, Indicates the railway fastener visual feature coding diagram ( , ) channel feature vector at the pixel position, Represents the pixel-level visual feature vector of the railway fastener to be enhanced.

[0037] That is, by focusing on the basic visual features of a single pixel, such as color, texture, and shape, a fine pixel-level feature unit is provided for the subsequent significant identification of the feature space distribution, thereby enhancing the feature expression effect of each local area on the fastener surface. Specifically, by extracting the pixel-level visual feature vector of the railway fastener to be enhanced pixel by pixel, it is possible to accurately capture the subtle feature differences on the fastener surface, enhance the significant distribution of each pixel in the feature space, and effectively improve the sensitivity and robustness of defect detection of fasteners with various shapes in complex environments.

[0038] Specifically, the step S52 includes: first, compressing the information of the railway fastener to be enhanced pixel-level visual feature vector to obtain the railway fastener to be enhanced pixel-level visual feature distillation coding vector, which is expressed by the formula: in, represents the calculation norm, Represents the distilled encoding vector of pixel-level visual features to be enhanced for railway fasteners.

[0039] That is, since the pixel-level visual feature vectors of railway fasteners to be enhanced may have dimensional redundancy or noise interference, the present application removes redundant information and extracts core features through an information compression mechanism to avoid the problem of defect recognition failure caused by feature redundancy. Specifically, by reducing feature dimensions or reducing information redundancy, focusing on core visual features related to fastener defects, and providing efficient and pure feature input for subsequent adaptive adjustment of the receptive field, the compressed pixel-level visual feature distillation coding vectors of railway fasteners to be enhanced can more accurately retain key information related to defects and remove interference from irrelevant noise such as uneven illumination, thereby effectively enhancing the model's defect detection capabilities for fasteners of different models in complex environments.

[0040] Secondly, based on the feature distribution spatial structure characteristics of the distilled coding vector of the pixel-level visual features to be enhanced for the railway fastener, the size of the feature receptive field of the distilled coding vector of the pixel-level visual features to be enhanced for the railway fastener is determined, which is expressed as follows: in, represents the logarithmic function with base 2, Represents the half-side length of the feature receptive field of the distilled encoding vector of the pixel-level visual features to be enhanced for railway fasteners.

[0041] That is, the fixed receptive field may lead to missed detection of minor defects and false detection of complex backgrounds, etc. Therefore, the present application analyzes the spatial structure of the feature distribution of the pixel-level visual feature distillation coding vector to be enhanced for the railway fasteners, so as to dynamically perceive the density of its feature distribution, and adaptively adjust the size of the feature receptive field, so that the size of the receptive field can be flexibly changed according to the complexity of local features, and the context information is used in a targeted manner to enhance the semantics and pertinence of feature saliency detection, thereby improving the accuracy of feature space saliency identification and enhancing the robustness of the model to different types of fasteners and complex environments.

[0042] Then, the railway fastener visual feature encoding map is feature decoupled along the channel dimension to obtain a set of railway fastener pixel-level visual feature vectors, which is expressed as: in, , , and They represent the (1,1), (1, ), No. ( ,1) and ( , ) pixel-level visual feature vector of the railway fastener at the pixel position, represents the feature decoupling function.

[0043] That is, through the channel dimension decoupling operation, the channel feature vector of each pixel position in the railway fastener visual feature encoding map is separated, breaking the correlation between channels and reducing channel coupling, so that the single channel feature of each pixel can be finely processed in the future, and then more fine-grained feature information can be mined. The set of pixel-level visual feature vectors of railway fasteners obtained in this way can eliminate mutual interference between channels, provide purer and independent feature units for subsequent feature processing, and effectively improve the feature capture capability of low-contrast defects.

[0044] Finally, based on the size of the feature receptive field, the set of pixel-level visual feature vectors of the railway fasteners within the local receptive field is screened out from the set of pixel-level visual feature vectors of the railway fasteners, which is expressed by the formula: in, , , and They represent the first ( , ), No. ( , ), No. ( , ) and ( , ) pixel-level visual feature vector of the railway fastener at the position, Represents the set of pixel-level visual feature vectors of railway fasteners within the local receptive field.

[0045] That is, according to the dynamically determined feature receptive field range, the pixel-level visual feature vectors of the railway fasteners in the area surrounding the pixel-level visual feature distillation coding vector to be enhanced are accurately retrieved, a contextual information library containing multi-dimensional information such as texture, edge, color, etc. is constructed, and a collection of pixel-level visual feature vectors of the railway fasteners within the local receptive field is screened out, so that the model can combine contextual semantics when judging defects, avoid misjudgment of single pixel features, and significantly improve the accuracy and robustness of defect detection under complex backgrounds.

[0046] Figure 5 FIG. 5 is a flowchart of sub-step S53 of the railway fastener defect detection method according to an embodiment of the present application. Figure 5As shown, the step S53 includes the steps of: S531, based on the feature correlation of each pixel-level visual feature vector of the railway fastener in the local receptive field with respect to the pixel-level visual feature vector of the railway fastener to be enhanced, performing significance aggregation on the set of pixel-level visual feature vectors of the railway fastener in the local receptive field to obtain a significance aggregation coding vector of the pixel-level visual features of the railway fastener in the local receptive field; S532, fusing the significance aggregation coding vector of the pixel-level visual features of the railway fastener in the local receptive field and the pixel-level visual feature vector of the railway fastener to be enhanced to obtain the enhanced pixel-level visual feature vector of the railway fastener.

[0047] More specifically, the step S531 is expressed by the formula: in, The first ( , ) position in the local receptive field of the railway fastener pixel-level visual feature vector, represents the transpose of a vector, represents vector multiplication, represents the softmax normalization function, express The significance weight of Represents the saliency aggregation encoding vector of pixel-level visual features of railway fasteners within the local receptive field.

[0048] That is, by quantifying the correlation between the pixel-level visual feature vectors of railway fasteners and the pixel-level visual feature vectors of railway fasteners to be enhanced in each local receptive field, the significantly correlated feature components are amplified and the non-significant components are suppressed, and a saliency aggregation coding vector of the pixel-level visual features of railway fasteners in the local receptive field that integrates contextual semantics is constructed, so that the pixel-level visual features of railway fasteners to be enhanced can more accurately focus on the core information related to the defects, thereby providing high-purity feature input for subsequent defect judgment.

[0049] In a preferred example of the present application, the step S532 includes: first, based on the conformal fusion commutation relation constraint of the set of pixel-level visual feature vectors of the railway fasteners in the local receptive field, modulating the weighting coefficients of the pixel-level visual feature saliency aggregation coding vector of the railway fasteners in the local receptive field and the pixel-level visual feature vector to be enhanced of the railway fasteners, which is expressed by the formula: in, represents the sine function, A volume space representation vector representing a set of pixel-level visual feature vectors of railway fasteners within the local receptive field, A boundary representation vector representing a set of pixel-level visual feature vectors of railway fasteners within the local receptive field, and represent the weighting coefficients of the railway fastener pixel-level visual feature saliency aggregation coding vector within the local receptive field and the railway fastener pixel-level visual feature vector to be enhanced, respectively, represents the proportional scaling factor, Represents the calculation of the bi-norm.

[0050] That is, by introducing the conformal fusion commutation relation constraint, the regularity and coordination of the spatial structure of the pixel-level visual feature vector of railway fasteners in the local receptive field are ensured, thereby amplifying the feature components related to saliency and suppressing the non-saliency components. This makes the feature differences in the defective area more prominent under the conformal constraint, effectively reducing feature confusion in complex scenes such as uneven illumination, and providing structurally coordinated and saliency-enhanced feature input for subsequent defect judgment.

[0051] Then, based on the weighting coefficient, the railway fastener pixel-level visual feature saliency aggregation coding vector in the local receptive field and the railway fastener pixel-level visual feature vector to be enhanced are weighted fused to obtain the enhanced railway fastener pixel-level visual feature vector, which is expressed as: in, Represents the pixel-level visual feature vector of enhanced railway fasteners.

[0052] That is, the weighted coefficient is used to fuse the railway fastener pixel-level visual feature saliency aggregation coding vector and the railway fastener pixel-level visual feature vector to be enhanced within the local receptive field, so as to amplify the feature components related to saliency and suppress or weaken the feature components related to non-saliency, so that the enhanced railway fastener pixel-level visual feature vector can fuse the local context information, more accurately reflect the saliency of the pixel, and enhance the expression effect of the visual features such as the color, texture, and shape of the fastener surface, so as to effectively overcome the influence of the complex environment, enhance the sensitivity of the model to defect detection, and improve the versatility and adaptability of the detection method and the detection reliability in complex scenarios.

[0053] In the above-mentioned railway fastener defect detection method, the step S6 determines whether the fastener has defects based on the significant coding map of the railway fastener visual feature space. In a specific example of the present application, the step S6 includes: inputting the significant coding map of the railway fastener visual feature space into a fastener defect recognition module based on a classifier to obtain a detection result, and the detection result is used to indicate whether the fastener has defects. That is, in order to realize the automatic identification of railway fastener defects, the present application constructs a fastener defect recognition module based on a classification algorithm to map the significant coding map of the railway fastener visual feature space into a defect probability and distinguish between normal fasteners and defective fasteners. Specifically, the significant coding map of the railway fastener visual feature space is input into a pre-trained classifier, and the classifier learns the feature distribution pattern of the significant coding map of the railway fastener visual feature space, and maps it to a label space of defect or not based on the decision boundary or feature mapping relationship learned during the training process, thereby outputting a detection result. The test results are presented in the form of two categories, i.e., the fastener is normal or the fastener is defective, so as to provide railway maintenance personnel with intuitive fastener status information, facilitate timely adoption of corresponding maintenance measures, and ensure the safety and stability of railway transportation.

[0054] In summary, the railway fastener defect detection method based on the embodiment of the present application is explained, which controls a high-speed industrial camera to scan the track area to collect track images, performs image preprocessing on the track images, and extracts ROI image blocks containing potential railway fasteners based on template matching. Then, the target detection algorithm is further introduced to accurately locate and segment the ROI image blocks of fasteners to obtain more refined fastener ROI image blocks, and then, by extracting the visual features of the fastener ROI image blocks and enhancing the significance of their feature space distribution, the expression effect of the visual features of the fastener surface color, texture, shape, etc. is enhanced, and then on this basis, intelligent recognition of railway fastener defects is realized. This method can effectively overcome the limitations of traditional detection methods, improve the reliability of railway fastener defect detection in complex environments, and has strong versatility and adaptability.

[0055] Furthermore, a railway fastener defect detection system is also provided.

[0056] Figure 6 FIG. 1 is a block diagram of a railway fastener defect detection system according to an embodiment of the present application. Figure 6As shown, according to the railway fastener defect detection system 100 of the embodiment of the present application, it includes: an original track image acquisition module 110, which is used to control a high-speed industrial camera to scan the track area to obtain an original track image; an ROI image block acquisition module 120, which is used to perform image preprocessing and region of interest extraction on the original track image to obtain an ROI image block containing potential railway fasteners; a fastener precise positioning and segmentation module 130, which is used to perform fastener precise positioning and segmentation on the ROI image block containing potential railway fasteners to obtain a fastener ROI image block; a visual feature extraction module 140, which is used to extract railway fastener visual features from the fastener ROI image block to obtain a railway fastener visual feature coding map; a feature space distribution significance identification module 150, which is used to perform feature space distribution significance identification on the railway fastener visual feature coding map to obtain a railway fastener visual feature space significant coding map; a defect detection module 160, which is used to determine whether a fastener has a defect based on the railway fastener visual feature space significant coding map.

[0057] Here, those skilled in the art will appreciate that the specific operations of each module in the above-mentioned railway fastener defect detection system have been described in detail above. Figures 1 to 5 The description of the railway fastener defect detection method has been described in detail, and therefore, its repeated description will be omitted.

[0058] The basic principle of the present invention is described above in conjunction with specific embodiments. However, it should be pointed out that the advantages, strengths, effects, etc. mentioned in the present invention are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. must be possessed by each embodiment of the present invention. In addition, the specific details of the above embodiments are only for the purpose of illustration and facilitation of understanding, rather than limitation, and the above details do not limit the present invention to being implemented by adopting the above specific details.

[0059] In the above embodiments, the description of each embodiment has its own emphasis. For the parts that are not described or recorded in detail in a certain embodiment, please refer to the relevant description of other embodiments. In the several embodiments provided by the present invention, it should be understood that the disclosed system and method can be implemented in other ways. For example, the system embodiment described above is only schematic. For example, the unit division is only a logical function division, and there may be other division methods in actual implementation. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0060] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference to a figure in a claim should not be considered as limiting the claim to which it relates.

[0061] In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. The multiple units stated in the system claims can also be implemented by one unit through software or hardware.

[0062] Finally, it should be noted that the above description has been given for the purpose of illustration and description. In addition, the above embodiments are only used to illustrate the technical solution of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A railway fastener defect detection method, characterized in that: include: Control the high-speed industrial camera to scan the track area to obtain the original track image; Performing image preprocessing and region of interest extraction on the original track image to obtain a ROI image block containing potential railway fasteners; Performing fastener precise positioning and segmentation on the ROI image block containing potential railway fasteners to obtain a fastener ROI image block; Extracting railway fastener visual features from the fastener ROI image block to obtain a railway fastener visual feature coding map; Performing feature space distribution significance marking on the railway fastener visual feature coding map to obtain a railway fastener visual feature space significant coding map; Based on the railway fastener visual feature space significant coding map, it is determined whether the fastener has defects.

2. The railway fastener defect detection method according to claim 1, characterized in that: Controlling a high-speed industrial camera to scan a track area to obtain an original track image includes: while controlling the high-speed industrial camera to scan the track area, synchronously triggering an LED line light source.

3. The railway fastener defect detection method according to claim 2, characterized in that: The original track image is subjected to image preprocessing and region of interest extraction to obtain a ROI image block containing potential railway fasteners, including: Based on the Retinex algorithm, the original track image is adaptively enhanced to obtain an original track enhanced image; Performing image distortion correction on the original orbit enhanced image to obtain an original orbit re-enhanced image; The ROI image block containing the potential railway fastener is extracted from the original track enhanced image based on template matching.

4. The railway fastener defect detection method according to claim 3, characterized in that: The fasteners are accurately located and segmented on the ROI image block containing potential railway fasteners to obtain fastener ROI image blocks, including: The ROI image block containing potential railway fasteners is input into a target detection model based on the Faster R-CNN model to obtain the fastener ROI image block.

5. The railway fastener defect detection method according to claim 4, characterized in that: The railway fastener visual feature coding map is subjected to feature space distribution significance marking to obtain the railway fastener visual feature space significant coding map, including: Extracting the channel feature vector of the (i, j)th pixel position from the railway fastener visual feature coding map as the railway fastener pixel-level visual feature vector to be enhanced; Based on the feature distribution of the pixel-level visual feature vector to be enhanced of the railway fastener, anchoring the railway fastener visual feature encoding map locally to screen out a set of railway fastener pixel-level visual feature vectors within the local receptive field; Based on the set of pixel-level visual feature vectors of railway fasteners within the local receptive field, the pixel-level visual feature vectors of the railway fasteners to be enhanced are saliency enhanced based on feature association guidance to obtain enhanced pixel-level visual feature vectors of railway fasteners, wherein the enhanced pixel-level visual feature vectors of railway fasteners are the channel feature vectors of the (i, j)th pixel position in the saliency coding map of the railway fastener visual feature space.

6. The railway fastener defect detection method according to claim 5, characterized in that: Based on the feature distribution of the pixel-level visual feature vector to be enhanced for the railway fastener, local receptive field anchoring is performed on the railway fastener visual feature encoding map to screen out a set of pixel-level visual feature vectors of the railway fastener within the local receptive field, including: Performing information compression on the pixel-level visual feature vector of the railway fastener to be enhanced to obtain a distilled encoding vector of the pixel-level visual feature of the railway fastener to be enhanced; Determining the size of the feature receptive field of the distilled coding vector of the pixel-level visual features to be enhanced for the railway fastener based on the feature distribution spatial structure characteristics of the distilled coding vector of the pixel-level visual features to be enhanced for the railway fastener; Performing feature decoupling on the railway fastener visual feature encoding map along the channel dimension to obtain a set of railway fastener pixel-level visual feature vectors; Based on the size of the feature receptive field, a set of pixel-level visual feature vectors of railway fasteners within the local receptive field is filtered out from the set of pixel-level visual feature vectors of railway fasteners.

7. The railway fastener defect detection method according to claim 6, characterized in that: Based on the set of pixel-level visual feature vectors of the railway fastener in the local receptive field, the pixel-level visual feature vectors of the railway fastener to be enhanced are enhanced based on feature association guidance to obtain enhanced pixel-level visual feature vectors of the railway fastener, including: Based on the feature correlation of each pixel-level visual feature vector of the railway fastener in the local receptive field with respect to the pixel-level visual feature vector of the railway fastener to be enhanced, the set of pixel-level visual feature vectors of the railway fastener in the local receptive field is saliency aggregated to obtain a saliency aggregated coding vector of the pixel-level visual feature of the railway fastener in the local receptive field; The railway fastener pixel-level visual feature saliency aggregation coding vector within the local receptive field and the railway fastener pixel-level visual feature vector to be enhanced are fused to obtain the enhanced railway fastener pixel-level visual feature vector.

8. The railway fastener defect detection method according to claim 7, characterized in that: The method of fusing the railway fastener pixel-level visual feature saliency aggregation coding vector in the local receptive field and the railway fastener pixel-level visual feature vector to be enhanced to obtain the enhanced railway fastener pixel-level visual feature vector comprises: Based on the conformal fusion commutation relation constraint of the set of pixel-level visual feature vectors of the railway fastener in the local receptive field, modulating the weighting coefficients of the pixel-level visual feature saliency aggregation coding vector of the railway fastener in the local receptive field and the pixel-level visual feature vector to be enhanced of the railway fastener; Based on the weighting coefficient, the railway fastener pixel-level visual feature saliency aggregation coding vector within the local receptive field and the railway fastener pixel-level visual feature vector to be enhanced are weightedly fused to obtain the enhanced railway fastener pixel-level visual feature vector.

9. The railway fastener defect detection method according to claim 8, characterized in that: Based on the railway fastener visual feature spatial significant coding map, determining whether the fastener has defects includes: The railway fastener visual feature space significant coding map is input into a classifier-based fastener defect recognition module to obtain a detection result, and the detection result is used to indicate whether the fastener has a defect.

10. A railway fastener defect detection system, characterized in that: include: The original track image acquisition module is used to control the high-speed industrial camera to scan the track area to obtain the original track image; An ROI image block acquisition module is used to perform image preprocessing and region of interest extraction on the original track image to obtain an ROI image block containing potential railway fasteners; A fastener precise positioning and segmentation module, used for performing fastener precise positioning and segmentation on the ROI image block containing potential railway fasteners to obtain a fastener ROI image block; A visual feature extraction module, used for extracting railway fastener visual features from the fastener ROI image block to obtain a railway fastener visual feature coding map; A feature space distribution significance identification module is used to perform feature space distribution significance identification on the railway fastener visual feature coding map to obtain a railway fastener visual feature space significance coding map; The defect detection module is used to determine whether the fastener has defects based on the significant coding map of the visual feature space of the railway fastener.

Images