Machine vision detection method and system for railway sleeper supporting fastener missing

Abstract

The invention relates to the technical field of rail transit detection, in particular to a machine vision detection method and system for railway sleeper supporting fastener missing, and the method comprises a data acquisition step, a detection model construction step, an environment judgment step, a model selection step and a defect detection step. And the optimal detection model is selected according to the environment type, so that the problem of long processing time of a single model in a complex environment in a traditional method is avoided, the detection efficiency is improved, potential safety hazards can be fed back in time, and the problems of long missing detection response time and low detection efficiency in the prior art are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of rail transit detection, and specifically relates to a machine vision detection method and system for the absence of railway tie support fasteners. Background Art

[0002] With the rapid development of rail transit, rail detection technology has become an important guarantee for ensuring the safe operation of trains. In recent years, vision-based object recognition methods have been widely used in the field of rail detection. It collects rail images in real time through on-vehicle cameras and combines image processing technology to identify abnormal objects on the rails, such as foreign objects and damage to rail components. However, during the high-speed operation of trains, the existing vision recognition methods still face the following technical bottlenecks: (1) Image blurring leads to processing delays: Due to the high train speed, the images collected by on-vehicle cameras are vulnerable to motion blurring and vibration interference, with low image resolution and blurred edges. Traditional image enhancement and deblurring algorithms require a large amount of computing resources and are difficult to meet the real-time processing requirements. Especially in complex lighting or dynamic environments, the processing time of existing algorithms increases significantly, resulting in lag in object recognition and inability to timely feedback potential safety hazards. (2) Insufficient matching accuracy for blurred images: Existing methods usually rely on a clear image database for feature matching, but it is difficult to extract features from blurred images, and misjudgment or missed detection is likely to occur during direct matching. For example, the contour of the object in the blurred image is deformed or the color is distorted, resulting in inaccurate feature mapping relationships and seriously affecting the reliability of the recognition results. (3) Low efficiency of multi-camera collaborative calibration: To improve the detection coverage rate, existing technologies often use multi-camera systems. However, it is difficult to effectively eliminate the geometric deformation and errors of images caused by differences in installation angles or environmental interference of different cameras. Traditional calibration methods rely on complex hardware adjustments or static calibration and cannot dynamically adapt to the real-time changes during train operation. There are deficiencies in both calibration accuracy and efficiency. In addition, there is a lack of targeted correction strategies for the common-mode error and differential-mode error of the multi-camera system, further reducing the robustness of the system.

[0003] Therefore, to solve the problems existing in the prior art, the present invention proposes a machine vision detection method and system for the absence of railway tie support fasteners. Summary of the Invention

[0004] Aiming at the deficiencies of the prior art, the purpose of the present invention is to provide a machine vision detection method and system for the absence of railway tie support fasteners.

[0005] To achieve the above purpose, the present invention provides the following technical solutions: A machine vision detection method for the absence of railway tie support fasteners includes the following steps: Data acquisition step: Obtain environmental data, train operation data, and image sampling data of sleeper support fasteners. The image sampling data includes a number of support fastener sampling images. Detection model construction step: Construct a number of support fastener missing recognition models according to different detection logics and detection parameters, and calculate the model detection cost of each support fastener missing recognition model. The model detection cost is a comprehensive evaluation index of detection speed and detection accuracy. Environmental judgment step: Analyze the current detection environment type through an environmental judgment model according to the environmental data and train operation data. Model selection step: Calculate the correction cost of each support fastener missing recognition model according to the current detection environment type, and calculate the comprehensive cost in combination with the model detection cost. Select the support fastener missing recognition model with the minimum comprehensive cost as the optimal detection model. Defect detection step: Input the image sampling data of the sleeper support fasteners into the optimal detection model, and output the support fastener missing detection result.

[0006] As a further improvement of the present invention, the support fastener missing recognition model includes a low-brightness environment missing recognition model, a curved track environment missing recognition model, and a straight track missing recognition model. The low-brightness environment missing recognition model completes missing detection by fusing multi-spectral image data and denoised visible light images, and extracting fastener thermal characteristics and geometric distribution rules; the curved track environment missing recognition model completes missing detection by calculating the track curvature to dynamically correct image deformation and matching the dynamic template of fastener arrangement changing with curvature; the straight track missing recognition model completes missing detection by identifying the straight arrangement rule of fasteners in the image and combining edge detection and deep learning algorithms.

[0007] As a further improvement of the present invention, the construction of the environmental judgment model includes setting the light intensity, track type in the environmental data, and vehicle speed in the train operation data as discrete environmental factors, calculating the environmental type judgment value through a supervised learning algorithm according to historical detection data, establishing the mapping relationship between environmental factors and environmental types according to the environmental type judgment value, and judging and outputting the environmental type through a preset judgment value classification threshold. The environmental categories include low-brightness environment, curved track environment, and straight track environment.

[0008] As a further improvement of the present invention, the construction of the low-brightness environment missing recognition model includes registering the visible light image and the infrared image in the image sampling data, and dynamically allocating fusion weights based on the real-time ambient light intensity to complete the fusion of multi-spectral data. The random noise is eliminated by performing a multi-frame superposition denoising algorithm on the visible light image, and the image contrast is enhanced by adaptive histogram equalization to complete low-light enhancement. The fastener area is located based on the metal thermal radiation characteristics of the infrared image, and combined with the preset fastener geometric distribution template of the track, the matching degree between the fastener density in the local area and the template is calculated. If the matching degree is lower than the threshold and the thermal feature contour is broken, it is output that the support fastener is missing.

[0009] As a further improvement of the present invention, the construction of the curved track environment missing recognition model includes calculating the real-time track curvature data of the track according to the support fastener sampling image, performing polar coordinate transformation on the support fastener sampling image according to the real-time track curvature data, mapping the curved track to obtain a virtual straight line image, generating a dynamic template of the fastener arrangement that matches the current real-time track curvature based on a preset curvature fastener spacing mapping table, matching the dynamic template of the fastener arrangement and the actual position of the fastener in the virtual straight line image by the phase correlation method, calculating the deviation value between the adjacent fastener spacing value and the preset ideal spacing value. When the deviation values of the fastener spacing values of three consecutive fasteners exceed the preset threshold, it is output that the support fastener is missing.

[0010] As a further improvement of the present invention, the construction of the straight track missing recognition model includes extracting the fastener edge contour in the support fastener sampling image, detecting the straight arrangement rule of the fastener center points by the Hough transform, generating a fitting straight line of the fastener positions, calculating the lateral offset of each fastener center point from the fitting straight line of the fastener positions. If the lateral offset exceeds the preset offset threshold, the area where the fastener is located is set as an abnormal area, and the local image of the abnormal area is input into a preset convolutional neural network. Using the complete fastener image and the edge detection mask as training data, the fastener missing probability is output. When the fastener missing probability is greater than the preset probability threshold and the edge detection result is abnormal, it is output that the support fastener is missing.

[0011] As a further improvement of the present invention, the defect detection step includes selecting the image sampling data type and performing image processing according to the data type and data format requirements of the selected optimal detection model, performing size reduction, noise elimination, and equalizing the gray distribution on the image sampling data to complete the standardization process, inputting the processed image sampling data into the corresponding support fastener missing recognition model, and performing spatio-temporal consistency analysis on the missing areas detected in multiple consecutive frames of the image sampling data. If the same position is determined to be missing in multiple frames, the final detection result is output as the support fastener is missing.

[0012] As a further improvement of the present invention, the calculation of the model detection cost includes calculating a detection speed score based on the relationship between the single - detection time of the model and the preset maximum allowable time and minimum theoretical time. The detection speed score represents the time duration. The recall rate and precision rate of the model are statistically calculated through the test data set, and a comprehensive precision score is calculated according to the weight distribution rule of giving priority to the recall rate and then the precision rate. The speed score and the precision score are weighted and summed according to the preset weights, and the model detection cost is generated in combination with the type of the support fastener missing recognition model.

[0013] As a further improvement of the present invention, the calculation of the correction cost includes dynamically adjusting the parameters of the support fastener missing recognition model according to the current environment type, calculating the change range of the model detection accuracy before and after the adjustment, statistically calculating the proportion of the extra time consumed by the model parameter adjustment in the total single - detection time, and dynamically allocating weights to the parameter adjustment cost and the real - time cost according to the environmental complexity to generate the final correction cost.

[0014] A machine vision detection system for missing railway sleeper support fasteners, comprising: A data acquisition module that acquires environmental data, train operation data, and image sampling data of the sleeper support fasteners. The image sampling data includes a number of support fastener sampling images; A detection model construction module that constructs a number of support fastener missing recognition models according to different detection logics and detection parameters, and calculates the model detection cost of each support fastener missing recognition model. The model detection cost is a comprehensive evaluation index of detection speed and detection accuracy; An environment judgment module that analyzes the current detection environment type through an environment judgment model according to the environmental data and train operation data; A model selection module that calculates the correction cost of each support fastener missing recognition model according to the current detection environment type, combines the model detection cost to calculate the comprehensive cost, and selects the support fastener missing recognition model with the minimum comprehensive cost as the optimal detection model; A defect detection module that inputs the image sampling data of the sleeper support fasteners into the optimal detection model and outputs the detection result of the missing support fasteners.

[0015] The beneficial effects of the present invention are: (1) The detection efficiency of the missing railway sleeper support fasteners is improved. By constructing detection models for different environment types and selecting the optimal detection model according to the environment type, the problem of long processing time of a single model in a complex environment in the traditional method is avoided, the detection efficiency is improved, and potential safety hazards can be timely feedback.

[0016] (2) The detection accuracy is improved. By adopting targeted image processing and feature extraction methods, more accurate data processing is carried out on the detection models for different environmental types, such as multi-spectral fusion and noise reduction processing in low-light environments, curvature dynamic correction and dynamic template matching in curved track environments, edge detection and deep learning algorithms in straight track environments, etc. It effectively solves problems such as image blurring and difficult feature extraction in different environments, improves the detection accuracy, and reduces misjudgment and missed detection.

[0017] (3) The system robustness is enhanced: The environmental judgment model can accurately judge the current detection environmental type. The model selection module dynamically adjusts the model parameters and selects the optimal model according to the environmental type, enabling the system to adapt to the real-time environmental changes during train operation and enhancing the system robustness. At the same time, the construction of multiple models and the calculation of the comprehensive cost improve the adaptability of the system to different environments and data, further enhancing the system robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is the flowchart of the method of the present invention; Figure 2 is the schematic diagram of the detected image of the present invention; Figure 3 is the schematic diagram of the detected track area of the present invention; Figure 4 is the block diagram of the system of the present invention; Figure 5 is the schematic diagram of the system process of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0019] The present invention will be further described in detail below with reference to the drawings and embodiments. The same components are denoted by the same reference numerals. It should be noted that the terms "front", "rear", "left", "right", "upper" and "lower" used in the following description refer to the directions in the drawings, and the terms "bottom surface" and "top surface", "inner" and "outer" refer to the directions towards or away from the geometric center of a specific component, respectively.

[0020] A machine vision detection method for the lack of railway sleeper support fasteners, as Figures 1 to 5 shown, includes the following steps: The data acquisition step, obtaining environmental data, train operation data and image sampling data of the sleeper support fasteners, and the image sampling data includes a plurality of support fastener sampling images; In practical applications, the data acquisition step uses high-precision vehicle-mounted cameras and a variety of sensors, including light sensors and speed sensors, to obtain environmental data, train operation data, and image sampling data of sleeper support fasteners. The light sensor collects light intensity data in real time, the speed sensor obtains the train operation speed data in real time, and the vehicle-mounted camera collects image sampling data of sleeper support fasteners at a certain frame rate to ensure that the collected data has high accuracy and real-time performance.

[0021] The detection model construction step constructs several missing fastener recognition models according to different detection logics and detection parameters, and calculates the model detection cost of each missing fastener recognition model. The model detection cost is a comprehensive evaluation index of detection speed and detection accuracy. The detection model construction step constructs a missing recognition model in low-light environment, a missing recognition model in curved track environment, and a missing recognition model in straight track environment according to different detection logics and parameters. When constructing the missing recognition model in low-light environment, a visible light camera and an infrared camera are used to collect image data at the same time, and the visible light image and the infrared image are registered through an image registration algorithm. The image matching algorithm uses a feature point-based registration method, including SIFT feature point extraction and matching algorithm, to ensure the spatial consistency of the multi-spectral image. Then, according to the real-time ambient light intensity, the fusion weights of the visible light image and the infrared image are dynamically allocated through an adaptive algorithm. For example, when the ambient light intensity is low, the fusion weight of the infrared image is increased to highlight the thermal characteristics of the fasteners; when the ambient light intensity is high, the fusion weight of the visible light image is increased to utilize the rich details of the visible light image. The visible light image is processed by multi-frame superposition noise reduction, specifically, by averaging multiple consecutive visible light images to eliminate random noise, and then the image contrast is enhanced through an adaptive histogram equalization algorithm to make the fastener features in the image clearer. Based on the metal thermal radiation characteristics of the infrared image, a threshold segmentation algorithm is used to locate the fastener area. The threshold can be determined according to historical data and experiments to determine a suitable threshold range, and then combined with the preset fastener geometric distribution template of the track, the matching degree between the fastener density in the local area and the template is calculated. If the matching degree is lower than the preset threshold and the thermal feature contour is broken, it is determined that the support fastener is missing.

[0022] The environment judgment step analyzes the current detection environment type through an environment judgment model according to the environmental data and the train operation data. The environmental judgment step takes light intensity, track types such as straight tracks and curved tracks, and vehicle speed as discrete environmental factors. The historical detection data includes detection results and corresponding environmental types under different environmental factors. Through a supervised learning algorithm, such as a decision tree algorithm, the historical detection data is trained to establish a mapping relationship between environmental factors and environmental types. According to the real-time collected environmental factor data, the environmental type judgment value is calculated and compared with a preset judgment value classification threshold to judge the current environmental type.

[0023] The model selection step calculates the correction cost of each support fastener missing recognition model according to the current detection environmental type, and combines the model detection cost to calculate the comprehensive cost. The support fastener missing recognition model with the minimum comprehensive cost is selected as the optimal detection model. The model selection step dynamically adjusts the parameters of each support fastener missing recognition model according to the current environmental type. For example, in a low-brightness environment, adjust the fusion weight, noise reduction parameters, etc. of the low-brightness environment missing recognition model; in a curved track environment, adjust the curvature calculation parameters, dynamic template generation parameters, etc. of the curved track environment missing recognition model. Calculate the change range of the model detection accuracy before and after adjustment, and determine a reasonable evaluation index through experiments and statistics to measure the accuracy change, such as the change in recall rate and precision rate. Statistically calculate the proportion of the additional time consumed by the model parameter adjustment in the total time of a single detection. According to the environmental complexity, such as a higher complexity in the low-brightness environment, followed by the curved track environment, and a lower complexity in the straight track environment, dynamically allocate the weights of the parameter adjustment cost and the real-time cost to generate the final correction cost. Combine the model detection cost, and through the test data set, statistically calculate the single detection time, recall rate, and precision rate of the model. Calculate the comprehensive accuracy score according to the weight allocation rule with the recall rate being prioritized and the precision rate being secondary. The speed score and the accuracy score are weighted and summed according to the preset weights to calculate the comprehensive cost, and the model with the minimum comprehensive cost is selected as the optimal detection model.

[0024] The defect detection step inputs the image sampling data of the sleeper support fasteners into the optimal detection model and outputs the detection result of the support fastener missing.

[0025] The defect detection module processes the image sampling data according to the data type and format requirements of the optimal detection model. For example, if the optimal detection model is a missing recognition model for low-light environments, visible light images and infrared images are selected as input data, and size reduction is performed to crop the images to the size required by the model. Noise is eliminated through the median filtering algorithm, and the gray level distribution is equalized through the gray level equalization algorithm to complete the standardization process. The processed images are input into the model, and spatio-temporal consistency analysis is performed on the missing areas detected in consecutive multiple frames of images. Specifically, the positions and detection results of the missing areas in each frame of the image are recorded. If the same position is determined to be missing in consecutive multiple frames, usually 3 frames, the final detection result is output as the missing of the support fastener to reduce false positives caused by accidental factors.

[0026] Specifically, as Figures 1 to 5 shown, the support fastener missing recognition model includes a missing recognition model for low-light environments, a missing recognition model for curved track environments, and a missing recognition model for straight tracks. The missing recognition model for low-light environments completes the missing detection by fusing multi-spectral image data and the denoised visible light image and extracting the thermal characteristics and geometric distribution rules of the fasteners; the missing recognition model for curved track environments completes the missing detection by calculating the track curvature to dynamically correct the image deformation and matching the dynamic template of the fastener arrangement changing with the curvature; the missing recognition model for straight tracks completes the missing detection by identifying the straight arrangement rule of the fasteners in the image and combining edge detection and deep learning algorithms.

[0027] Among them, the implementation of the missing recognition model for low-light environments includes: Image registration: The feature point-based registration algorithm, such as the SIFT algorithm, is used to extract feature points in the visible light image and the infrared image. Through feature point matching and transformation matrix calculation, accurate registration of the two images is achieved.

[0028] Multi-spectral fusion: According to the real-time ambient light intensity, the fusion weights of the visible light image and the infrared image are dynamically adjusted. When the light intensity is low, the fusion weight of the infrared image is increased to highlight the thermal characteristics of the fasteners; when the light intensity gradually increases, the fusion weight of the visible light image is gradually increased to utilize the detailed information of the visible light image.

[0029] Noise reduction and enhancement processing: Multi-frame superposition noise reduction is performed on the visible light image, that is, the continuously acquired multiple frames of visible light images are averaged to eliminate random noise; then adaptive histogram equalization processing is performed. The image is divided into several sub-regions, and histogram equalization is performed on each sub-region separately to enhance the local contrast of the image.

[0030] Fastener positioning and missing detection: Based on the thermal radiation characteristics of metal fasteners in infrared images, the fastener area is located through a threshold segmentation algorithm. Combining with a preset fastener geometric distribution template, calculate the matching degree between the fastener density in the local area and the template, and at the same time analyze whether the thermal feature contour is complete. If the matching degree is lower than the preset threshold and the thermal feature contour is broken, it is determined that the support fasteners in this area are missing.

[0031] The implementation of the missing identification model for the curved track environment includes: Track curvature calculation: Adopt a curvature calculation method based on image processing. For example, by detecting the curve features of the track edge, using the least squares method to fit the curve, and calculate the real-time curvature data of the track.

[0032] Polar coordinate transformation: According to the real-time track curvature data, perform polar coordinate transformation on the sampled image of the support fasteners, map the curved track image into a virtual straight line image, and eliminate the image deformation caused by the track curvature.

[0033] Dynamic template generation: According to the preset curvature-fastener spacing mapping table, find the fastener spacing corresponding to the current real-time track curvature, and generate a dynamic template of the fastener arrangement that matches the current curvature.

[0034] Template matching and missing detection: Search for the dynamic template of the fastener arrangement in the virtual straight line image through the phase correlation method to determine the actual position of the fasteners; calculate the deviation value between the adjacent fastener spacing value and the preset ideal spacing value. When the spacing deviation values of 3 consecutive fasteners all exceed the preset threshold, it is determined that the support fasteners in this area are missing.

[0035] The implementation of the missing identification model for the straight track includes: Edge contour extraction: Adopt the Canny edge detection algorithm to extract the edge contour of the sampled image of the support fasteners to obtain the edge contour information of the fasteners.

[0036] Hough transform to detect straight lines: Use the Hough transform algorithm to detect the straight line arrangement rule of the fastener center points in the edge contour image and generate a fitting straight line for the fastener positions.

[0037] Lateral offset calculation: Calculate the lateral offset of each fastener center point from the fitting straight line of the fastener positions. If the offset exceeds the preset threshold, mark the area where the fastener is located as an abnormal area.

[0038] Deep learning detection: Input the local image of the abnormal area into a preset convolutional neural network. This network uses a large number of complete fastener images and edge detection masks as training data and outputs the fastener missing probability. When the fastener missing probability is greater than the preset probability threshold and the edge detection result shows that the edge is incomplete, it is determined that the support fasteners in this area are missing.

[0039] Specifically, such as Figures 1 to 5As shown, the construction of the environmental judgment model includes setting the light intensity, track type in the environmental data, and vehicle speed in the train operation data as discrete environmental factors, calculating the environmental type judgment value through a supervised learning algorithm based on historical detection data, establishing a mapping relationship between the environmental factors and the environmental type according to the environmental type judgment value, and judging and outputting the environmental type through a preset judgment value classification threshold. The environmental categories include low-brightness environment, curved track environment, and straight track environment.

[0040] The construction of the environmental judgment model uses the decision tree algorithm in supervised learning. First, collect a large amount of historical detection data, including environmental data and corresponding environmental types, low-brightness environment, curved track environment, and straight track environment, under different light intensities, track types, and vehicle speeds. Then, use the light intensity, track type, and vehicle speed as input features and the environmental type as the output label to train the decision tree model. By continuously adjusting the parameters and structure of the decision tree, the model can accurately calculate the environmental type judgment value according to the input environmental factors and establish a mapping relationship between the environmental factors and the environmental type. Finally, set a reasonable judgment value classification threshold. For example, when the environmental type judgment value is less than 0.3, it is determined as a low-brightness environment; when it is between 0.3 and 0.7, it is determined as a curved track environment; when it is greater than 0.7, it is determined as a straight track environment, so as to achieve an accurate judgment of the current detected environmental type.

[0041] Specifically, as Figures 1 to 5 shown, the construction of the low-brightness environment missing recognition model includes registering the visible light image and the infrared image in the image sampling data, and completing the fusion of multi-spectral data by dynamically allocating fusion weights based on the real-time environmental light intensity. Eliminating random noise by performing a multi-frame superposition noise reduction algorithm on the visible light image and enhancing the image contrast through adaptive histogram equalization to complete low-light enhancement. Locating the fastener area based on the metal thermal radiation characteristics of the infrared image, and combining with the preset fastener geometric distribution template of the track, calculating the matching degree between the fastener density in the local area and the template. If the matching degree is lower than the threshold and the thermal feature contour is broken, it is output as a missing support fastener.

[0042] The specific implementation of model construction includes: Multi - spectral image registration and fusion. Using a registration algorithm based on feature point matching such as the SIFT algorithm, scale - invariant feature points are extracted from visible light images and infrared images. After screening out corresponding points through two - way matching and removing mismatched points using the RANSAC algorithm, the homography matrix is calculated to complete image registration, ensuring the alignment of the fastener positions in the two types of images. An ambient light intensity is established. The unit of ambient light intensity is lux, and a mapping table between Lux and the fusion weight is established (for example: when the light intensity < 10Lux, the weight of the infrared image accounts for 70% and the visible light accounts for 30%; when 10Lux ≤ light intensity < 30Lux, the infrared accounts for 50% and the visible light accounts for 50%; when the light intensity ≥ 30Lux, the infrared accounts for 30% and the visible light accounts for 70%). The weight is dynamically adjusted through real - time light intensity sensor data to balance the thermal features and visible light details.

[0043] Pre - processing of visible light images. For multi - frame superposition noise reduction, 5 consecutive visible light images are collected. Superposition noise reduction is performed through pixel mean calculation. The image is divided into sub - blocks of 8×8 pixels, and histogram equalization is performed on each sub - block, and the contrast gain is limited (the threshold is set to 2.0) to avoid noise amplification and improve the image contrast in low - light environments.

[0044] Fastener positioning and missing detection. After performing Gaussian blur (kernel size 3×3) on the infrared image, the high - temperature area (thermal radiation characteristics of metal fasteners) is extracted through Otsu threshold segmentation to generate a fastener candidate area mask. A preset standard fastener arrangement template is set (such as the horizontal spacing is 600mm ± 5mm, and the vertical spacing is 500mm ± 5mm). The local fastener density (the number of fasteners per unit area) is calculated within the candidate area. If the density is less than 80% of the template density and the thermal feature contour has a break of ≥ 2 pixels, it is determined that the fastener is missing.

[0045] Specifically, as Figures 1 to 5 shown, the construction of the bending track environment missing recognition model includes calculating the real - time track curvature data of the track according to the sampled image of the support fastener. The real - time track curvature calculation includes: Performing Canny edge detection on the sampled image of the support fastener to extract the track edge curve; using the least squares method to fit a quadratic curve , and calculating the curvature formula: , taking the mean curvature of 5 points in the central area of the track as the real - time curvature data. According to the real - time track curvature data, polar coordinate transformation is performed on the sampled image of the support fastener to map the bending track to a virtual straight - line image. The polar coordinate transformation for correcting deformation includes taking the center of the track curvature as the pole and converting the bending track image from the Cartesian coordinate system to the polar coordinate system. The formula is: , where (x0, y0) are the coordinates of the center of curvature. After transformation, a virtual straight-line track image is generated to eliminate the influence of bending deformation on the fastener spacing. Based on a preset curvature-fastener spacing mapping table, a dynamic template of fastener arrangement matching the current real-time track curvature is generated. By using the phase correlation method, the dynamic template of fastener arrangement and the actual position of the fasteners are matched in the virtual straight-line image, and the deviation value between the adjacent fastener spacing value and the preset ideal spacing value is calculated. The dynamic template generation and matching are carried out. According to the curvature-spacing mapping table (for example: curvature (k = 0.001 / mm) corresponds to a spacing of 580 mm, (k = 0.002 / mm) corresponds to a spacing of 560 mm), a dynamic template of equidistant fastener arrangement is generated, and the template length covers 3 fastener spacings. In the virtual straight-line image, the translation amount between the template and the actual fastener image is calculated by the phase correlation algorithm to locate the center position of the fasteners; the deviation of the adjacent fastener spacing is calculated. If the spacing deviation of 3 consecutive fasteners exceeds ±15 mm (preset threshold), it is determined that a fastener is missing. When the deviation value of the fastener spacing value of 3 consecutive fasteners from the preset ideal spacing value exceeds the preset threshold, it is output that a support fastener is missing.

[0046] Specifically, as Figures 1 to 5 shown, the construction of the straight-line track missing recognition model includes extracting the edge contour of the fasteners in the sampled image of the support fasteners, detecting the linear arrangement rule of the center points of the fasteners through the Hough transform, generating a fitting straight line for the fastener positions, calculating the lateral offset of each fastener center point from the fitting straight line of the fastener positions. If the lateral offset exceeds the preset offset threshold, the area where the fastener is located is set as an abnormal area. The local image of the abnormal area is input into a preset convolutional neural network. Using the complete fastener image and the edge detection mask as training data, the fastener missing probability is output. When the fastener missing probability is greater than the preset probability threshold and the edge detection result is abnormal, it is output that a support fastener is missing.

[0047] Edge contour extraction and straight-line detection include using the Canny edge detection algorithm (Gaussian kernel size 5×5, high-to-low threshold ratio 3:1) to extract the edge contour of the fasteners. Perform Hough straight-line detection on the edge image, set the polar angle range [-10°, 10°] (corresponding to the straight-line track direction), screen out the straight line where the fastener center points are located, and generate a fitting straight-line equation y = mx + b. For each fastener center point (x i , y i ), calculate its perpendicular distance to the fitting straight line. If the distance exceeds the preset threshold, mark the 30×30 pixel area where the fastener is located as an abnormal area.

[0048] Adopt the ResNet-18 lightweight model. The input is an abnormal area image (size 30×30 pixels, grayscale image), and the output is the probability of fastener missing (0-1). Collect 100,000 fastener-complete images (including edge detection masks) and 20,000 missing images, and enhance the data by randomly rotating (±10°) and scaling (±5%); the loss function uses binary cross-entropy, and the optimizer is Adam (learning rate 1e -4 ). When the CNN output probability > 0.9 and the edge detection mask shows that the edge break length > 10 pixels, it is determined that the fastener is missing.

[0049] Specifically, as Figures 1 to 5 shown, the defect detection steps include, according to the data type and data format requirements of the selected optimal detection model, selecting the image sampling data type and performing image processing, performing size cropping, noise elimination, and equalizing the gray level distribution on the image sampling data to complete the standardization process, inputting the processed image sampling data into the corresponding support fastener missing recognition model, performing spatio-temporal consistency analysis on the missing areas detected in multiple consecutive frames of the image sampling data. If the same position is determined to be missing in multiple frames, the final detection result is output as the support fastener is missing.

[0050] Image standardization process Size cropping: According to the input requirements of the optimal model, crop the original image to 512×512 pixels (retaining the central field of view of the fastener area).

[0051] Noise elimination: Use median filtering (kernel size 3×3) to remove salt-and-pepper noise, and then smooth the image through Gaussian filtering (kernel size 5×5, standard deviation 1.5).

[0052] Gray level equalization: Perform global histogram equalization on the image, expand the gray level value range to [0, 255], and enhance the overall contrast.

[0053] Spatio-temporal consistency analysis Multi-frame detection: Continuously collect 10 frames of images, and record the coordinates of the missing areas detected in each frame.

[0054] Consistency judgment: If the same coordinate area is determined to be missing in ≥6 frames (time window is about 0.2 seconds, assuming a frame rate of 30fps), then confirm that this area is a real missing area and output the detection result; otherwise, it is regarded as a false detection and excluded.

[0055] Specifically, as Figures 1 to 5As shown, the calculation of the model detection cost includes calculating a detection speed score based on the relationship between the single - detection time of the model and the preset maximum allowable time and minimum theoretical time. The detection speed score represents the time duration. The recall rate and precision rate of the model are statistically calculated through the test data set, and a comprehensive precision score is calculated according to the weight allocation rule that gives priority to the recall rate and secondary to the precision rate. The speed score and the precision score are weighted and summed according to the preset weights, and the model detection cost is generated in combination with the type of the support fastener missing recognition model.

[0056] The model detection cost is generated by comprehensively evaluating the detection speed and precision: first, a speed score is calculated based on the single - detection time of the model and the preset time interval; then, the recall rate and precision rate are statistically calculated through the test data and the comprehensive precision score is calculated according to the weights; finally, the speed score and the precision score are weighted and summed according to the preset weights, and the final model detection cost is generated in combination with the model type. The smaller the value, the better the comprehensive performance of the model.

[0057] The calculation configuration of the model detection cost is as follows: ; Among them, C model is the model detection cost, which is a cost index that comprehensively measures the detection speed and precision. The smaller the value, the better the model performance; t is a time parameter representing the single - detection time, and its value range is [t min , t max , where t min is the minimum theoretical time, t max is the maximum allowable time, α is a quantization parameter of the environmental type, calculated by α = 1+log(1 + EnvScore), EnvScore ∈ [0,10] is the normalized value of the environmental sensor data, g(Prec, Rec) is the precision synthesis function, Prec is the precision rate, Rec is the recall rate, and the precision synthesis function is a non - linear combination of the precision rate and the recall rate, g = Rec·tanh(γ·Prec), where γ is a calibration coefficient with a default value of 2, β is a speed penalty coefficient representing the exponential decay weight during the detection time, is the model sensitivity term, representing the sum of the squares of the gradients of the model F with respect to the input image I k and the parameter θ, N is the number of test images, is a small constant used to ensure that the denominator is not zero, F(I k , θ) is the detection model output function, which is the inference result of the deep - learning model, I k is the k - th frame image, θ is the model parameter, used to suppress timeout detection, and the cost increases exponentially with the increase of the detection time.

[0058] Specifically, for example Figures 1 to 5As shown, the calculation of the correction cost includes dynamically adjusting the parameters of the support fastener missing recognition model according to the current environment type, calculating the change range of the model detection accuracy before and after the adjustment, counting the proportion of the extra time consumed by the model parameter adjustment in the total time of a single detection, dynamically allocating weights to the parameter adjustment cost and the real-time cost according to the environmental complexity, and generating the final correction cost.

[0059] The correction cost is generated based on the impact of dynamically adjusting the model parameters according to the current environment type: first, calculate the change range of the model detection accuracy before and after the parameter adjustment, then count the proportion of the parameter adjustment time in the total time of a single detection, and then dynamically allocate the weights of the accuracy change cost and the real-time cost according to the environmental complexity (such as high complexity for low brightness / bent track and low complexity for straight track), and finally generate the correction cost through weighting, reflecting the requirements of the environment for the model adaptability.

[0060] The correction cost calculation is configured with: ; Among them, C adj is the correction cost, is the comprehensive cost of model parameter adjustment, is the environmental parameter domain, representing the multi-dimensional environmental factor integration domain, , where L light is the maximum light intensity, K curve is the maximum track curvature, V speed is the maximum vehicle speed, is the balance coefficient between environmental complexity and real-time performance, =erf(ρ·EnvScore), erf is the error function, ρ is the scaling factor, ∇ θ ΔA is the accuracy gradient change, representing the gradient difference of the model detection accuracy before and after the parameter adjustment, which is calculated through automatic differentiation, is the proportion of adjustment time consumption, which is the ratio of the parameter adjustment time to the total time of a single detection, is the real-time sensitivity coefficient, which is a preset parameter, is the environmental non-linear mapping function, which is used to non-linearly map environmental factors to a high-dimensional space, is the model curvature penalty term, representing the sum of squares of the second derivatives of the loss function L with respect to the parameter θ, reflecting the convergence stability of the model.

[0061] A machine vision detection system for missing support fasteners of railway sleepers, as Figures 1 to 5 shown, includes: A data acquisition module, which acquires environmental data, train operation data, and image sampling data of the sleeper support fasteners. The image sampling data includes several sampling images of support fasteners; The detection model construction module constructs several support fastener missing recognition models according to different detection logics and detection parameters, and calculates the model detection cost of each support fastener missing recognition model. The model detection cost is a comprehensive evaluation index of detection speed and detection accuracy; The environment judgment module analyzes the current detection environment type through the environment judgment model according to the environmental data and train operation data; The model selection module calculates the correction cost of each support fastener missing recognition model according to the current detection environment type, combines the model detection cost to calculate the comprehensive cost, and selects the support fastener missing recognition model with the minimum comprehensive cost as the optimal detection model; The defect detection module inputs the image sampling data of the sleeper support fasteners into the optimal detection model and outputs the detection result of support fastener missing.

[0062] The machine vision detection system for railway sleeper support fastener missing is built based on an on-vehicle distributed computing platform and includes the following core hardware modules: Data acquisition module: Image acquisition unit: A line array camera (resolution ≥ 5000×1000 pixels, frame rate ≥ 100fps) installed at the bottom of the train, equipped with infrared and visible light dual-band lenses, supporting synchronous acquisition of multi-spectral images; auxiliary cameras are installed on both sides of the train to cover the fasteners on both sides of the track.

[0063] Environment perception unit: Integrated light intensity sensor (measurement range 0 - 10000Lux, accuracy ±5%), vehicle speed sensor (CAN bus access to the train operation control system), track curvature sensor (based on inertial navigation and vision fusion algorithm).

[0064] Detection model construction module: Edge computing server: Equipped with NVIDIA Jetson Xavier NX, built-in CUDA acceleration unit, supporting parallel construction of multiple models (low brightness, curved track, straight track models), and optimizing the model inference speed through TensorRT.

[0065] Environment judgment module: Embedded processor: An ARMCortex-A72 architecture chip, pre-deployed with an environment judgment model based on decision tree, real-time processing of data such as light intensity, vehicle speed, and track type, and outputting the environment type (low brightness / curved track / straight track).

[0066] Model selection module: Real-time scheduler: Based on FPGA to achieve high-speed logic control, dynamically switch the optimal model according to the environment type and model cost (detection cost + correction cost), and the switching delay ≤ 10ms.

[0067] Defect Detection Module: Image Preprocessing Unit: Integrated with a dedicated image processing chip (such as Xilinx Zynq), supporting parallel execution of standardized processes such as size cropping, noise reduction, and grayscale equalization; Detection Result Output Unit: Transmits the detection results (including the coordinates of the missing position and the confidence level) to the train monitoring system through an Ethernet interface and triggers an on-vehicle alarm device (such as an audible and visual alarm).

[0068] System Working Process Step 1: Data Synchronous Acquisition The line array camera acquires track images (including visible and infrared bands) at a frame rate of 100fps. The light intensity sensor outputs the ambient light value in real time. The vehicle speed sensor obtains the train running speed through the CAN bus (accuracy ±0.1km / h). The track curvature sensor calculates the track curvature in real time through a vision algorithm (updated every 50ms).

[0069] The data of each module is calibrated with a time stamp through a synchronous clock (accuracy ±1μs) to ensure strict alignment of the image data and the environmental data.

[0070] Step 2: Dynamic Construction of the Detection Model At the initial stage of train startup (or during regular maintenance), the edge computing server offline trains three types of detection models based on historical data (including fastener images under different illuminations and track types): Low-brightness Model: Fuses multi-spectral image features and adopts TripletLoss to optimize the joint expression of thermal features and geometric features during training; Curved Track Model: Based on the track image dataset with different curvatures, trains a dynamic template generation algorithm; Straight Track Model: Trains a CNN network using a large number of straight track fastener images (including complete and missing samples).

[0071] The server calculates the detection cost of each model in real time (a comprehensive evaluation of speed and accuracy) and stores it in the model parameter library.

[0072] Step 3: Real-time Judgment of the Environment Type The embedded processor receives the data of light intensity (L), vehicle speed (V), and track curvature (k), and judges the environment type according to the following rules: Low-brightness Environment: L < 30Lux and V ≤ 80km / h; Curved Track Environment: k > 0.0005 / mm or the track type is marked as "curve section"; Straight Track Environment: Other situations.

[0073] The environment judgment result is output to the model selection module in the form of one-hot encoding.

[0074] Step 4: Dynamic Selection of the Optimal Model The real-time scheduler pre-adjusts the parameters of each model according to the environmental type (for example, preloading high infrared weight parameters for the low-brightness model), and calculates the correction cost (reflecting the impact of parameter adjustment on accuracy and speed); By comprehensively considering the detection cost and the correction cost, select the model with the minimum cost as the currently executed model (for example: in a low-brightness environment, preferentially select the low-brightness model, and if its cost is higher than the cost of the curved track model + correction cost, then make a dynamic switch).

[0075] Step 5: Defect Detection and Result Output The image preprocessing unit performs standardization processing on the input image: crop it to the fastener ROI area (512×512 pixels), reduce noise through median filtering + Gaussian filtering, and then enhance the contrast through histogram equalization; The preprocessed image is input into the optimal model, and the missing probability and position coordinates of the fasteners are output; The defect detection module performs spatio-temporal consistency analysis on the detection results of 10 consecutive frames (confirmation is made if the same position is detected as missing in ≥6 frames). The final result is transmitted to the train monitoring system through Ethernet, and the track mileage and image screenshots of the missing position are displayed on the on-vehicle terminal.

[0076] 3. System Integration and Verification Installation Layout: The line array camera is installed in the middle at the bottom of the train bogie, and the optical axis is perpendicular to the track plane; the auxiliary cameras are installed on both sides at a 45° angle to cover the fasteners at the track edge; the sensors and computing devices are integrated into the on-vehicle cabinet, and vibration interference is reduced through shock-absorbing devices.

[0077] Test Verification: In the laboratory environment, use a simulated track platform (supporting adjustable lighting and curvature) to verify the detection accuracy of each model (recall rate ≥98%, precision rate ≥95%); in the actual line test, the average detection delay of the system for fastener missing is ≤200ms, meeting the real-time requirement.

[0078] 4. Differences from the Existing Technology Multi-modal Fusion Architecture: Different from traditional single-camera vision systems, this system enhances the feature robustness in complex environments through multi-spectral image fusion and multi-camera collaboration; Dynamic Model Scheduling Mechanism: Break through the limitations of the adaptability of static models to environmental changes, and achieve a dynamic balance between detection efficiency and accuracy through a cost-driven model selection strategy; Spatio-temporal Consistency Verification: Through multi-frame data correlation analysis, effectively suppress false detections caused by motion blur and random noise, and reduce the false alarm rate by more than 80% compared with traditional single-frame detection.

[0079] The basic features, principles, and advantages of the present invention have been shown and described above. It should be noted that the present invention is not limited by the above embodiments, which are only partial embodiments. Without departing from the spirit and scope of the present invention, several improvements and supplements made are regarded as the protection scope of the present invention.

Claims

1. A machine vision detection method for the absence of railway sleeper support fasteners, characterized in that, It includes the following steps: Data acquisition step: Obtain environmental data, train operation data, and image sampling data of sleeper support fasteners. The image sampling data includes several support fastener sampling images. Detection model construction step: Construct several support fastener missing recognition models according to different detection logics and detection parameters, and calculate the model detection cost of each support fastener missing recognition model. The model detection cost is a comprehensive evaluation index of detection speed and detection accuracy. Environment judgment step: Analyze the current detection environment type through an environment judgment model according to the environmental data and train operation data. Model selection step: Calculate the correction cost of each support fastener missing recognition model according to the current detection environment type, and calculate the comprehensive cost in combination with the model detection cost. Select the support fastener missing recognition model with the minimum comprehensive cost as the optimal detection model. Defect detection step: Input the image sampling data of the sleeper support fasteners into the optimal detection model and output the support fastener missing detection result.

2. The machine vision detection method for the absence of railway sleeper support fasteners according to claim 1, characterized in that, The support fastener missing recognition model includes a low-brightness environment missing recognition model, a curved track environment missing recognition model, and a straight track missing recognition model. The low-brightness environment missing recognition model completes the missing detection by fusing multi-spectral image data and the visible light image after noise reduction processing, and extracting the fastener thermal characteristics and geometric distribution rules; the curved track environment missing recognition model completes the missing detection by calculating the track curvature to dynamically correct the image deformation and matching the dynamic template of the fastener arrangement changing with the curvature; the straight track missing recognition model completes the missing detection by identifying the straight arrangement rule of the fasteners in the image and combining edge detection and deep learning algorithms.

3. The machine vision detection method for the absence of railway sleeper support fasteners according to claim 1, wherein The construction of the environment judgment model includes setting the light intensity, track type in the environmental data, and vehicle speed in the train operation data as discrete environmental factors, calculating the environmental type judgment value through a supervised learning algorithm according to historical detection data, establishing the mapping relationship between the environmental factors and the environmental type according to the environmental type judgment value, and judging and outputting the environmental type through a preset judgment value classification threshold. The environmental categories include low-brightness environment, curved track environment, and straight track environment.

4. The machine vision detection method for the absence of railway sleeper support fasteners according to claim 2, wherein, The construction of the low-brightness environment missing recognition model includes registering the visible light image and the infrared image in the image sampling data, dynamically allocating the fusion weight based on the real-time ambient light intensity to complete the fusion of multi-spectral data, eliminating random noise through a multi-frame superposition noise reduction algorithm for the visible light image and enhancing the image contrast through adaptive histogram equalization to complete low-light enhancement, locating the fastener area based on the metal thermal radiation characteristics of the infrared image, combining the preset fastener geometric distribution template of the track, calculating the matching degree between the fastener density in the local area and the template. If the matching degree is lower than the threshold and the thermal feature contour is broken, it is output as a missing support fastener.

5. The machine vision detection method for the absence of railway sleeper support fasteners according to claim 2, characterized in that, The construction of the bending track environment missing recognition model includes calculating the real-time track curvature data of the track according to the sampled image of the support fastener, performing polar coordinate transformation on the sampled image of the support fastener according to the real-time track curvature data, mapping the bending track to obtain a virtual straight-line image, generating a dynamic template of the fastener arrangement matching the current real-time track curvature based on a preset curvature-fastener spacing mapping table, matching the dynamic template of the fastener arrangement and the actual position of the fastener in the virtual straight-line image by the phase correlation method, calculating the deviation value between the adjacent fastener spacing value and the preset ideal spacing value, and when the deviation values of the fastener spacing values of three consecutive fasteners exceed the preset threshold, outputting that the support fastener is missing.

6. The machine vision detection method for the absence of railway sleeper support fasteners according to claim 2, characterized in that, The construction of the straight track missing recognition model includes extracting the edge contour of the fastener in the sampled image of the support fastener, detecting the straight arrangement rule of the fastener center points by the Hough transform, generating a fitting straight line of the fastener positions, calculating the lateral offset of each fastener center point from the fitting straight line of the fastener positions, if the lateral offset exceeds the preset offset threshold, setting the area where the fastener is located as an abnormal area, inputting the local image of the abnormal area into a preset convolutional neural network, using the complete image of the fastener and the edge detection mask as training data, outputting the fastener missing probability, and when the fastener missing probability is greater than the preset probability threshold and the edge detection result is abnormal, outputting that the support fastener is missing.

7. The machine vision detection method for the lack of railway sleeper support fasteners according to claim 1, characterized in that, The defect detection step includes selecting the image sampling data type and performing image processing according to the data type and data format requirements of the selected optimal detection model, performing size reduction, noise elimination and equalizing the gray-scale distribution on the image sampling data to complete the standardization process, inputting the processed image sampling data into the corresponding support fastener missing recognition model, performing spatio-temporal consistency analysis on the missing areas detected in multiple consecutive frames of the image sampling data, and if the same position is determined to be missing in multiple frames, outputting the final detection result as the support fastener is missing.

8. A machine vision detection method for the absence of railway sleeper support fasteners according to claim 1, characterized in that, The calculation of the model detection cost includes calculating the detection speed score based on the relationship between the single detection time-consuming of the model and the preset maximum allowable time-consuming and the minimum theoretical time-consuming, where the detection speed score represents the time-consuming duration, statistically calculating the recall rate and precision rate of the model through the test data set, calculating the comprehensive precision score according to the weight allocation rule of giving priority to the recall rate and secondary to the precision rate, performing weighted summation on the speed score and the precision score according to the preset weight, and generating the model detection cost in combination with the type of the support fastener missing recognition model.

9. The machine vision detection method for the absence of railway sleeper support fasteners according to claim 1, characterized in that, The calculation of the correction cost includes dynamically adjusting the parameters of the support fastener missing recognition model according to the current environment type, calculating the change range of the model detection accuracy before and after the adjustment, statistically calculating the proportion of the extra time consumed by the model parameter adjustment in the total single detection time-consuming, and dynamically allocating weights to the parameter adjustment cost and the real-time cost according to the environmental complexity to generate the final correction cost.

10. A machine vision detection system for the absence of railway sleeper support fasteners, applicable to the machine vision detection method for the absence of railway sleeper support fasteners described in any one of claims 1 to 9, characterized in that, including: A data acquisition module that acquires environmental data, train operation data, and image sampling data of the sleeper support fasteners, where the image sampling data includes several sampled images of the support fasteners; The detection model construction module constructs several support fastener missing recognition models according to different detection logics and detection parameters, and calculates the model detection cost of each support fastener missing recognition model. The model detection cost is a comprehensive evaluation index of detection speed and detection accuracy; The environment judgment module analyzes the current detection environment type through the environment judgment model according to the environment data and train operation data; The model selection module calculates the correction cost of each support fastener missing recognition model according to the current detection environment type, combines the model detection cost to calculate the comprehensive cost, and selects the support fastener missing recognition model with the smallest comprehensive cost as the optimal detection model; The defect detection module inputs the image sampling data of the sleeper support fastener into the optimal detection model and outputs the detection result of the support fastener missing.

Images