Railway fastener anomaly detection method and system based on computer vision

Abstract

The invention provides a railway fastener anomaly detection method and system based on computer vision, and the method comprises the steps: a fastener data arrangement step: carrying out the screening and combination in a track area image, and obtaining a fastener data set; a fastener anomaly detection network construction step: matching a threshold value through a random floating-point number, selecting to carry out false anomaly and label generation strategy processing or data enhancement strategy processing on the fastener image to obtain a first image, extracting features in the first image, and reconstructing through a reconstruction network to obtain a second image, obtaining a comparison score of all corresponding associated pixels between the two images through a preset comparison function; a fastener anomaly model training step: substituting the fastener data set into the fastener anomaly detection network for training to obtain a fastener anomaly model; a fastener abnormality output step: substituting the real-time fastener image into the fastener abnormality model to output an abnormal contour mark and a comparison score at an abnormal contour; according to the invention, it can be ensured that the fastener can be detected in different abnormal states, and various different fastener failure forms are covered.

Description

Technical Field

[0001] The present invention relates to the technical field of rail fastener detection, and more particularly to a railway fastener anomaly detection method and system based on computer vision. Background Art

[0002] Railway fasteners are a key part of the railway transportation safety system. They are mainly responsible for fixing the position of the rails and providing elastic support for them. These functions are crucial to ensuring the safety and smoothness of train operation, and will affect the stability and reliability of the entire railway system. Fasteners are susceptible to failure due to bending stress and shear stress from the rails, and are prone to spring bar breakage and loosening of fastening bolts. In addition, due to open-air scenes, rubber pads are prone to aging and foreign matter inclusions. At present, the status detection of railway fasteners mainly relies on manual inspections, which is inefficient and susceptible to subjective judgment. Therefore, it is particularly important to develop a railway line detection system based on advanced technologies such as deep learning.

[0003] In the past decade, researchers have proposed a variety of methods to detect railway fastener anomalies using digital devices such as sensors, including vibration signal-based detection, ultrasonic detection, and acoustic analysis. However, these methods usually require expensive equipment and rely on detailed track geometry parameters to accurately locate and evaluate the fastener status. Therefore, due to the high cost, these methods have not been widely used.

[0004] With the development of computer vision, some new methods have been gradually proposed in the field of railway fastener status detection. Some studies have extracted improved edge gradient features and macro texture features of fasteners, and used image fusion features and Bayesian compressed sensing models to realize automatic identification of rail fastener anomalies. Another method combines improved Canny edge detection and median filtering, and uses curve feature projection templates to match the edge contours of fasteners to achieve rapid detection of missing fasteners. In addition, the Chinese invention patent with application number CN2024107981562 proposes to use a variant YOLO target detection network to realize fastener anomaly detection. Although the above-mentioned computer vision-based detection method improves the detection accuracy to a certain extent, it can only detect fastener anomalies of known abnormal types, and cannot accurately detect abnormal states without negative samples. Summary of the invention

[0005] In view of the deficiencies in the prior art, an object of the present invention is to provide a railway fastener anomaly detection method and system based on computer vision.

[0006] To achieve the above object, the present invention provides the following technical solutions: The railway fastener anomaly detection method based on computer vision includes the following steps: Track image acquisition step, obtaining images of the track area including fasteners in different scenes taken by the camera; The fastener data sorting step is to select fastener images from the track area images through a preset target detection network and combine them to obtain a fastener data set; The step of constructing a fastener anomaly detection network is to match a random floating point number generated by an open source library to a threshold value, and to select a pseudo-anomaly and label generation strategy or a data enhancement strategy to process the fastener image according to the threshold range to obtain a first image, and to reconstruct the features extracted from the first image through a reconstruction network to obtain a second image, and to obtain a comparison score of all corresponding associated pixels between the second image and the fastener image through a preset comparison function for the second image; a fastener anomaly model training step, substituting the fastener data set into the fastener anomaly detection network for reciprocal training, and selecting the network model with the smallest loss value as the fastener anomaly model; In the fastener anomaly output step, the real-time fastener image is substituted into the fastener anomaly model to output the abnormal contour mark and the comparison score at the abnormal contour.

[0007] Furthermore, the pseudo-anomaly and label generation strategy includes randomly selecting image cropping in a preset texture recognition data set and seamlessly fusing it with the fastener image through a fusion algorithm to generate a corresponding label image.

[0008] Furthermore, the pseudo-anomaly and label generation strategy includes a target image cropping step and an image fusion and label generation step. The target image cropping step randomly selects a category and an image ID from a preset texture recognition data set as a first target image, and randomly generates an irregular area whose minimum circumscribed rectangle has a length and a width that are both smaller than the length and the width of the target image, and cuts the irregular area from the target image to generate a new second target image; The image fusion and label generation step uses a fusion algorithm to obtain a seamless fusion of the fastener image and the second target image, and changes the pixels in the irregular area to 1 and the pixel values ​​in other areas to 0 to form a corresponding label image.

[0009] Furthermore, the fusion algorithm is configured as follows: , , , , , in, g is the fastener image, f* is the second target image, For irregular areas, is the boundary of the irregular region, is the interpolation function, and Represent the interpolation function and fastener images In irregular areas The Laplace operator inside, h is the distance between pixels in a two-dimensional image, For a two-dimensional image at pixel points The value on A is a sparse matrix, is a column vector containing the values ​​of all pixels in the irregular area, b is a column vector containing the fastener images g The Laplace filtering result of each pixel in the irregular area Ω is: For fastener images g In pixels Laplace filtering result on .

[0010] Furthermore, the data enhancement strategy includes superimposing three channels of the fastener image to form an image in which the pixel values ​​corresponding to each channel are equal, and randomly performing one or more image processing on it to obtain an enhanced image.

[0011] Furthermore, the image processing includes image flipping, image rotation, image cropping, image scaling, image translation or image dithering, etc.

[0012] Furthermore, a loss function group is provided in the reconstruction network, and the loss function group includes a structural similarity phase index function, a mean square error loss function and a perceptual loss function. The structural similarity phase index function and the mean square error loss function are used to determine the difference between each pixel for calculation and focus on the local differences between the two images, and the perceptual loss function is used to determine the perceptual quality of the image.

[0013] Furthermore, the contrast function includes a multi-scale gradient magnitude similarity function and an ISSM function. The multi-scale gradient magnitude similarity function calculates the gradient magnitude similarity between the fastener image and the second image at different scales through an image pyramid. The ISSM function is used to calculate the brightness similarity, contrast similarity and structural similarity between the fastener image and the second image.

[0014] Furthermore, in the step of constructing the fastener anomaly detection network, the label image or the enhanced image is converted from a spatial domain image to a frequency domain image through Fourier transform, and interference filtering is performed in the frequency domain image to obtain a first image.

[0015] The railway fastener anomaly detection system based on computer vision includes a track image acquisition module to obtain images of the track area including fasteners taken by a camera in different scenes; The fastener data sorting module selects fastener images from the track area images through a preset target detection network and combines them to obtain a fastener data set; The fastener anomaly detection network construction module uses a random floating point number matching threshold generated by an open source library to select a pseudo-anomaly and label generation strategy or a data enhancement strategy to process the fastener image according to the threshold range to obtain a first image, reconstructs the features extracted from the first image through a reconstruction network to obtain a second image, and obtains a comparison score of all corresponding associated pixels between the second image and the fastener image through a preset comparison function for the second image; A fastener anomaly model training module, which substitutes the fastener data set into the fastener anomaly detection network for reciprocal training, and selects the network model with the smallest loss value as the fastener anomaly model; The fastener anomaly output module substitutes the real-time fastener image into the fastener anomaly model to output the abnormal contour mark and the comparison score of the abnormal contour.

[0016] The beneficial effects of the present invention are as follows: by constructing a fastener anomaly detection network, it is possible to ensure that fasteners can be detected under different abnormal conditions and to cover various fastener failure forms. Specifically, by randomly selecting images obtained by image fusion label generation or data enhancement, and further transforming and eliminating interference, the reconstruction network can quickly extract features in the processed image, reconstruct the normal or clear parts of the fasteners, and blur the abnormal areas. When subsequent fastener data sets are substituted into the network, a model with the smallest loss value can be obtained as a fastener anomaly model. In addition, through the comparison function constructed in the fastener anomaly detection network, the comparison scores of all corresponding associated pixels in the fastener image and the reconstructed image can be directly reflected, and the scores are used to judge the possibility of the anomaly, and are attached to the fastener image as display data. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is the overall flow chart of the present invention; Figure 2 It is a framework diagram of the fastener anomaly detection network constructed in the present invention; Figure 3 It is a fastener marking diagram in the present invention; Figure 4 is an image of a fastener entraining foreign matter in the present invention; Figure 5 is the second target image reconstructed by the reconstruction network in the present invention; Figure 6 It is an image displayed abnormally in the present invention. DETAILED DESCRIPTION

[0018] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. The same parts are represented by the same reference numerals. It should be noted that the words "front", "rear", "left", "right", "upper" and "lower" used in the following description refer to directions in the accompanying drawings, and the words "bottom surface" and "top surface", "inner" and "outer" refer to directions toward or away from the geometric center of a specific component, respectively.

[0019] Since the current detection method based on computer vision has improved the detection accuracy to a certain extent, but can only detect the fastener anomalies of known abnormal types, it is impossible to accurately detect the abnormal state without negative samples. Therefore, the present invention designs such a railway fastener anomaly detection method based on computer vision, such as Figure 1 As shown, the following steps are included: The track image acquisition step is to obtain the images of the track area including the fasteners under different scenes taken by the camera. Specifically, a line scan camera mounted on the bottom of an electric passenger car or an engineering vehicle is used to take images of the track area including the fasteners. The line scan camera has a resolution of 4K (up to 4096 lines can be resolved per image height perpendicular to the scanning direction) of an infrared camera module.

[0020] In the fastener data sorting step, the fastener images are screened out from the track area images through the preset target detection network, and the fastener data set is obtained by combining them. Specifically, 5000 images containing fasteners are first collected using a line scan camera, and the fastener target detection data set is annotated and sorted according to the YOLO format. The labeling is performed using LabelImg software, and the labeling format is as follows: Figure 3 As shown in the figure, the categories are marked as blocks, and the training set, test set, and validation set are divided into 8:1:1 ratios; Secondly, build a fastener target detection network and train the target detection network to form a fastener target detection model. Specifically, the preset target detection network uses the YOLOV11 network, which is integrated with the transform feature extraction network. The YOLOV11s model pre-trained in the Images dataset is used as the pre-training model, and is trained in the fastener target detection dataset to achieve iterative tuning of network parameters. During training, data enhancement processing such as image stretching and rotation is performed. The fusion of the transform feature extraction network can further extract the multi-scale texture, color, and logical information of the fasteners. , which helps to optimize the parameters of the YOLOV11 network and realize the accurate recognition of the target. The network is trained for at least 300 rounds, and the model with the smallest loss value is taken as the fastener target detection model, and it is converted into a model in TensorRT format, which can be adapted to the host of the ARM architecture on conventional trains for model reasoning. The model input is an infrared image, which is compressed to 640*640 in size, and the output is a list of n*5, where n represents the number of fasteners detected in the input image. The first dimension consists of the category of the detected target, the x and y coordinates of the upper left corner of the normalized minimum enclosing rectangle, and the length and width; Finally, a line scan camera is used to collect images of the track area including fasteners in normal conditions to create a fastener dataset. Specifically, images of different usage states are collected in tunnels and open-air scenes under different ambient lights. The different ambient lights include: ① Tunnel lights will affect the tunnel scene, and the image brightness of the corresponding position close to the tunnel light is high; the image brightness of the corresponding position far from the tunnel light is low; ② Ambient light in different time periods and weather conditions in open-air environments.

[0021] Different usage states refer to: newly installed fasteners are relatively clean, but as the use time goes by, the surface of the fasteners will be covered with oil, dust, etc., causing the state of the fasteners to change. The normal state refers to that all the components of the fasteners are not deformed, damaged, or have failed, and there are no foreign objects covering the fasteners. Therefore, the collected images must include all the above situations, and the data volume must exceed 20,000 images that meet the above requirements. The fastener target detection model is used to perform fastener target detection on all the above images collected, and the position and size information of the minimum circumscribed rectangle of all detected fasteners on the original image are cut out, and several infrared images containing only fasteners are generated as the fastener data set, and the fastener data set is divided into a training set and a test set at a ratio of 9:1.

[0022] The present invention uses target detection to detect fasteners on the original image and intercept the fastener image, and does not use semantic segmentation to directly segment the fasteners. The reason is that the fasteners have complex shapes and changeable states. The existing semantic segmentation network cannot quickly and accurately segment the correct fastener area, and the number of fasteners on each railway line is large. Considering the image transmission speed and data storage cost, the image containing only the fasteners after interception can effectively filter out the impact of track areas such as rails, ramps, sleepers, etc. on subsequent abnormality detection.

[0023] Since railway fasteners are composed of spring bars, rail pads, threaded spikes, threaded flat washers, double spring washers, buckle plates, iron seats, insulating pads, bolt spikes, nuts, flat washers, etc., their failure forms are diverse, the types of foreign matter coverage are unknown, and negative samples, i.e., images of railway fasteners in a failed state or fasteners covered with foreign matter, are not easy to obtain. The existing classified or supervised detection algorithms can only detect specific types of anomalies, which cannot meet the needs of detecting all anomalies of railway fasteners. Therefore, the present invention includes a fastener anomaly detection network construction step, such as Figure 2 As shown in the figure, it includes a three-stage unsupervised railway fastener anomaly detection network, which only requires fastener data in normal state and can detect all types of fastener anomalies, specifically: First, a random floating point number matching threshold η from 0 to 1 is generated by the numpy open source library. When the threshold η is in the range of 0 to 0.7 (threshold range one), the fastener image is processed by the pseudo-anomaly and label generation strategy to obtain the first image. When the threshold η is in the range of 0.7 to 1 (threshold range two), the fastener image is processed by the data enhancement strategy to obtain the first image. The features extracted from the first image are reconstructed through the reconstruction network to obtain the second image. The second image is passed through a preset contrast function to obtain the contrast scores of all corresponding associated pixels between the second image and the fastener image.

[0024] The fastener anomaly model training step is to substitute the fastener dataset into the fastener anomaly detection network for reciprocating training for at least 800 rounds, and select the network model with the smallest loss value as the fastener anomaly model, that is, convert it into a TensorRT model.

[0025] Fastener abnormality output steps, such as Figure 6As shown, the real-time fastener image is substituted into the fastener anomaly model to output the abnormal contour mark and the contrast score at the abnormal contour. The contrast score is usually between 0 and 10. The larger the score, the greater the probability that the pixel location is abnormal. In the railway fastener anomaly detection, pixels with scores greater than 0.15 are identified as abnormal. The contrast score image is binarized, the binarization boundary is 0.15, the pixel values ​​greater than 0.15 are set to 255, and the remaining pixel values ​​are set to 0. Therefore, the result of railway fastener anomaly detection has been obtained. When the railway fastener is normal, all pixel values ​​of the output image are 0, otherwise, the pixel values ​​of the output image are not all 0. In addition, the findContours function in the OpenCV open source library is used to detect the abnormal contour in the image, and then the minAreaRect function is used to calculate the minimum circumscribed rectangle of the abnormal contour and draw it on the input image, so that the abnormal area can be clearly displayed, so as to facilitate intuitive monitoring in the system background.

[0026] The advantages of the present invention are: by building a fastener anomaly detection network, it is possible to ensure that fasteners can be detected under different abnormal conditions and cover various fastener failure forms. Specifically, the image obtained by randomly selecting image fusion label generation or data enhancement is further transformed and interference eliminated, so that the reconstruction network can quickly extract features in the processed image, reconstruct the normal or clear part of the fastener, and blur the abnormal area. When the subsequent fastener data set is substituted into the network, a model with the smallest loss value can be obtained as the fastener anomaly model. In addition, through the comparison function built in the fastener anomaly detection network, the comparison score of all corresponding associated pixels in the fastener image and the reconstructed image can be directly reflected, and the score is used as the judgment of the possibility of the anomaly, and attached to the fastener image as display data, which is convenient for the system background to perform intuitive monitoring and feedback to the railway maintenance system, so as to facilitate targeted maintenance plans.

[0027] The pseudo-anomaly and label generation strategy includes randomly selecting image cropping in a preset texture recognition data set and seamlessly fusing it with the fastener image through a fusion algorithm to generate a corresponding label image. Specifically, the pseudo-anomaly and label generation strategy includes a target image cropping step and an image fusion and label generation step. The target image cropping step randomly selects a category and an image ID in a preset DTD texture recognition data set as the first target image, and randomly generates an irregular area whose minimum circumscribed rectangle has a length and a width that are smaller than the length and the width of the target image, and cuts the irregular area in the target image to generate a new second target image; In addition to randomly generating irregular areas for cropping, a variety of cropping strategies can also be introduced, such as intelligent cropping based on image content (such as identifying key objects or areas in the image and cropping them), cropping based on image edges (ensuring that the edges of the cropped image are smooth and the information is complete), etc.

[0028] The image fusion and label generation step uses a fusion algorithm to obtain a seamless fusion of the fastener image, the second target image, and the irregular area, and changes the pixels in the irregular area to 1 and the pixel values ​​in other areas to 0 to form a corresponding label image; Specifically, the steps of image fusion and label generation are as follows: the three-channel superposition of the fastener image forms a three-channel image with equal pixel values ​​in each channel, and the fastener image is recorded as g , the second target image is recorded as f * , the area within the irregular shape in the second target image is recorded as , the boundary of the irregular region is recorded as , establish the Poisson equation and find an interpolation function , so that in the fusion area Inside, The gradient and source image g The gradient is consistent, and at the boundary of the fusion area superior, The value of and the target image f * The values ​​are consistent, and the Poisson equation is configured as: , , in, and Represent the interpolation function and fastener images In irregular areas The Laplacian operator in ; Then discretize the Laplace operator. After discretization, the Laplace operator in the two-dimensional image is expressed as: , in, h is the spacing between pixels in a 2D image (usually 1 in a uniform grid), For a two-dimensional image at pixel points The value on After discretizing the above Laplace operator, Poisson's equation is expressed as the following linear equations: , in, Ais a sparse matrix describing the irregular region The Laplace relationship between each pixel in the (fusion area) and its neighboring pixels, is a column vector containing the irregular area The values ​​of all pixels in the (fusion area), b is a column vector containing the fastener image g In irregular areas Laplace filtering result of each pixel in the (fusion area); For irregular areas Each pixel in the (fusion area) , the established equation is: , in, For fastener images g In pixels Laplace filtering result on ; Finally, due to A is a sparse matrix, and the sparse matrix solving algorithm is used to solve the above linear equations To get the interpolation function In irregular areas The values ​​of all pixels in the (fusion area) are solved to obtain the interpolation function Then, use the interpolation function Replace the target image with the value f * The value of the corresponding position in the image is calculated to achieve seamless fusion of the fastener image and the second target image. Then, the irregular area of ​​the second target image is The pixels in the (fusion area) become 1, and the pixel values ​​in other areas become 0, forming the corresponding labels.

[0029] The data enhancement strategy includes superimposing three channels of the fastener image to form an image with equal pixel values ​​in each channel, and randomly performing one or more image processing to obtain an enhanced image to increase the diversity and richness of the data to prevent the model from overfitting during the training process. The image processing includes image flipping, image rotation, image cropping, image scaling, image translation or image dithering. In addition to basic operations such as image flipping, rotation, and cropping, multi-scale transformation can be introduced, that is, scaling and cropping images at different scales. This helps the network learn the features of fasteners at different sizes and improves its generalization ability on images of different resolutions.

[0030] Regardless of whether the fastener image is processed by the pseudo-anomaly and label generation strategy or the data enhancement strategy through random selection, it is necessary to convert the label image or the enhanced image from the spatial domain image to the frequency domain image through Fourier transform, and filter the frequency domain image to filter out the interference of noise such as dust and rust on the railway fastener to obtain the first image. This process pays more attention to texture information than the spatial domain image and is more suitable for feature extraction of infrared images.

[0031] like Figure 4 and Figure 5 As shown in the figure, the backbone network of the reconstruction network adopts the U-Net network, and at the lowest end of the U-Net network, that is, the feature is pulled into one dimension and then connected to the ResNet50 network, so that the U-Net network after connecting to the ResNet50 network has stronger ability in extracting high-dimensional information, and can pay more attention to the position logic information of each component in the fastener. A loss function group is set in the reconstruction network, and the loss function group includes a structural similarity phase index function, a mean square error loss function and a perceptual loss function. The structural similarity phase index function and the mean square error loss function are used to judge the difference between each pixel for calculation and pay attention to the local difference between the two images. The perceptual loss function is used to judge the perceptual quality of the image, which is in line with the human eye's perception of image quality. The combined use is more conducive to the effect of reconstructing the image. The backbone network of the perceptual loss function is composed of VGG-19. The input image size of this network is 512*512*3, and the output image size is also 512*512*3. The purpose of this network is to reconstruct the normal area of ​​the railway fastener well, but for the abnormal area, this network cannot reconstruct it, so that the abnormal area is reconstructed into a fuzzy and irregular image.

[0032] The contrast function includes a multi-scale gradient magnitude similarity function (MSGMS) and an ISSM function. The multi-scale gradient magnitude similarity function calculates the gradient magnitude similarity between the fastener image and the second image at different scales through an image pyramid, specifically including: ① constructing an image pyramid: constructing image pyramids for the fastener image and the second image respectively, and the image at each scale is obtained by smoothing and downsampling the image at the previous scale; ② calculating the gradient: calculating the gradient of the image at each scale; ③ calculating the gradient magnitude similarity; the ISSM function is used to calculate the brightness similarity, contrast similarity and structural similarity between the fastener image and the second image, that is, the difference between the two images is obtained by calculating the mean (brightness), standard deviation (contrast) and covariance (structure) of the images respectively.

[0033] According to the designed detection method, a railway fastener anomaly detection system based on computer vision is designed, including a track image acquisition module to obtain images of the track area including fasteners in different scenes taken by the camera; The fastener data sorting module selects fastener images from the track area images through a preset target detection network and combines them to obtain a fastener data set; The fastener anomaly detection network construction module uses the random floating point number matching threshold generated by the open source library to select the pseudo-anomaly and label generation strategy or data enhancement strategy processing of the fastener image according to the threshold range to obtain the first image, reconstructs the features extracted from the first image through the reconstruction network to obtain the second image, and obtains the comparison score of all corresponding associated pixels between the second image and the fastener image through the preset comparison function; The fastener anomaly model training module substitutes the fastener data set into the fastener anomaly detection network for reciprocal training, and selects the network model with the smallest loss value as the fastener anomaly model; The fastener anomaly output module substitutes the real-time fastener image into the fastener anomaly model to output the abnormal contour mark and the comparison score of the abnormal contour.

[0034] The above are only preferred embodiments of the present invention. The protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A railway fastener anomaly detection method based on computer vision, characterized in that: The steps include: Track image acquisition step, obtaining images of the track area including fasteners in different scenes taken by the camera; The fastener data sorting step is to select fastener images from the track area images through a preset target detection network and combine them to obtain a fastener data set; The step of constructing a fastener anomaly detection network is to match a random floating point number generated by an open source library to a threshold value, and to select a pseudo-anomaly and label generation strategy or a data enhancement strategy to process the fastener image according to the threshold range to obtain a first image, and to reconstruct the features extracted from the first image through a reconstruction network to obtain a second image, and to obtain a comparison score of all corresponding associated pixels between the second image and the fastener image through a preset comparison function for the second image; a fastener anomaly model training step, substituting the fastener data set into the fastener anomaly detection network for reciprocal training, and selecting the network model with the smallest loss value as the fastener anomaly model; In the fastener anomaly output step, the real-time fastener image is substituted into the fastener anomaly model to output the abnormal contour mark and the comparison score at the abnormal contour.

2. The method for detecting anomalies of railway fasteners based on computer vision according to claim 1, characterized in that: The pseudo-anomaly and label generation strategy includes randomly selecting image cropping in a preset texture recognition data set and seamlessly fusing it with the fastener image through a fusion algorithm to generate a corresponding label image.

3. The method for detecting anomalies of railway fasteners based on computer vision according to claim 2, characterized in that: The pseudo-anomaly and label generation strategy includes a target image cropping step and an image fusion and label generation step. The target image cropping step randomly selects a category and an image ID from a preset texture recognition data set as a first target image, and randomly generates an irregular area whose minimum circumscribed rectangle has a length and a width that are both smaller than the length and the width of the target image, and cuts the irregular area from the target image to generate a new second target image; The image fusion and label generation step uses a fusion algorithm to obtain a seamless fusion of the fastener image and the second target image, and changes the pixels in the irregular area to 1 and the pixel values ​​in other areas to 0 to form a corresponding label image.

4. The method for detecting anomalies of railway fasteners based on computer vision according to claim 3, characterized in that: The fusion algorithm is configured as follows: ， ， ， ， ， in, g is the fastener image, f * is the second target image, For irregular areas, is the boundary of the irregular region, is the interpolation function, and Represent the interpolation function and fastener images In irregular areas The Laplace operator inside, h is the distance between pixels in a two-dimensional image, For a two-dimensional image at pixel points The value on A is a sparse matrix, is a column vector containing the values ​​of all pixels in the irregular area, b is a column vector containing the fastener images g The Laplace filtering result of each pixel in the irregular area Ω is: For fastener images g In pixels Laplace filtering result on .

5. The method for detecting anomalies of railway fasteners based on computer vision according to claim 4, characterized in that: The data enhancement strategy includes superimposing three channels of the fastener image to form an image with equal pixel values ​​corresponding to each channel, and randomly performing one or more image processing on it to obtain an enhanced image.

6. The method for detecting anomalies of railway fasteners based on computer vision according to claim 5, characterized in that: The image processing includes image flipping, image rotation, image cropping, image scaling, image translation or image dithering, etc.

7. The method for detecting anomalies of railway fasteners based on computer vision according to claim 1, characterized in that: A loss function group is provided in the reconstruction network, and the loss function group includes a structural similarity phase index function, a mean square error loss function and a perceptual loss function. The structural similarity phase index function and the mean square error loss function are used to determine the difference between each pixel for calculation and focus on the local difference between the two images. The perceptual loss function is used to determine the perceptual quality of the image.

8. The method for detecting anomalies of railway fasteners based on computer vision according to claim 1, characterized in that: The contrast function includes a multi-scale gradient magnitude similarity function and an ISSM function. The multi-scale gradient magnitude similarity function calculates the gradient magnitude similarity between the fastener image and the second image at different scales through an image pyramid. The ISSM function is used to calculate the brightness similarity, contrast similarity and structural similarity between the fastener image and the second image.

9. The method for detecting anomalies of railway fasteners based on computer vision according to claim 5, characterized in that: The step of constructing the fastener anomaly detection network is to convert the label image or the enhanced image from a spatial domain image to a frequency domain image through Fourier transform, and perform interference filtering in the frequency domain image to obtain a first image.

10. Railway fastener anomaly detection system based on computer vision, characterized by: It includes a track image acquisition module to obtain images of the track area including fasteners taken by the camera in different scenes; The fastener data sorting module selects fastener images from the track area images through a preset target detection network and combines them to obtain a fastener data set; The fastener anomaly detection network construction module uses a random floating point number matching threshold generated by an open source library to select a pseudo-anomaly and label generation strategy or a data enhancement strategy to process the fastener image according to the threshold range to obtain a first image, reconstructs the features extracted from the first image through a reconstruction network to obtain a second image, and obtains a comparison score of all corresponding associated pixels between the second image and the fastener image through a preset comparison function for the second image; A fastener anomaly model training module, which substitutes the fastener data set into the fastener anomaly detection network for reciprocal training, and selects the network model with the smallest loss value as the fastener anomaly model; The fastener anomaly output module substitutes the real-time fastener image into the fastener anomaly model to output the abnormal contour mark and the comparison score of the abnormal contour.

Images