A railway fastener abnormal type matching method and system

Abstract

The invention provides a railway fastener abnormal type matching method and system, comprising a fastener looseness calculation step, obtaining environmental data in an inspection section and the use time of sleepers, calculating the deformation rate of sleepers through sleeper deformation, and then calculating the fastener looseness rate by combining the weight of trains passing through the section, the number of train passes, the use time of fasteners and environmental data; an abnormality analysis selection step, selecting fastener fast analysis or multi-sensor fusion analysis according to the fastener looseness rate; a fast fastener analysis step, obtaining an image set in the inspection section, and then obtaining the explicit abnormality type of the fastener through fastener abnormality network analysis; a multi-sensor fusion analysis step, obtaining sensor data collected by each sensor in the inspection section, and then obtaining the latent abnormality type of the fastener through corresponding abnormality analysis strategy analysis; the invention has the advantage of being able to accurately judge the period when the fastener itself is abnormal, selecting the inspection means according to the judgment result, and improving the inspection efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of rail fastener detection, and more particularly to a method and system for matching abnormal types of railway fasteners. Background Art

[0002] Railway fasteners are a key part of the railway transportation safety system. They are mainly responsible for fixing the position of the rails and providing elastic support for them. These functions are crucial to ensuring the safety and stability of train operation, and will affect the stability and reliability of the entire railway system. Fasteners are susceptible to failure due to bending stress and shear stress from the rails, and are prone to spring bar breakage and loosening of fastening bolts. In addition, in open-air scenarios, rubber pads are prone to aging and foreign matter inclusion.

[0003] At present, the status detection of railway fasteners mainly relies on the inspection of patrol vehicles. The vision on the patrol vehicle is used to capture images of each fastener along the driving track in turn, and the variant YOLO target detection network is used to detect fastener anomalies on the collected images. Although the above-mentioned computer vision-based detection method has improved the detection accuracy to a certain extent, it can only detect fastener anomalies of known abnormal types. The inspection section contains thousands of fasteners. The patrol vehicle needs to analyze each fastener image one by one during each inspection, which results in a large amount of analysis data, a long time, and it is difficult to ensure the detection accuracy of many fasteners. In addition, it is impossible to accurately determine the period when the fastener itself has an abnormality, and can only rely on the artificially planned inspection cycle for regular inspections. Summary of the invention

[0004] In view of the deficiencies in the prior art, an object of the present invention is to provide a method and system for matching abnormal types of railway fasteners.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] A method for matching abnormal types of railway fasteners comprises the following steps:

[0007] The fastener looseness calculation step comprises obtaining environmental data and the service life of the sleepers in the inspection section, calculating the sleeper deformation rate through the sleeper deformation, and then obtaining the weight of the trains passing through the section, the number of train passes, the service life of the fasteners, environmental data and the sleeper deformation rate, calculating the fastener looseness rate through the fastener looseness;

[0008] an abnormal analysis selection step, comparing the fastener loosening rate with a preset threshold, and selecting a fastener rapid analysis step or a multi-sensor fusion analysis step according to the comparison result;

[0009] A fast fastener analysis step, obtaining rail images containing fasteners in the inspection section, marking and sorting them in sequence according to the acquisition order to obtain an image set, and analyzing the image set through a preset fastener anomaly network to obtain the fastener explicit anomaly type;

[0010] The multi-sensor fusion analysis step obtains the sensor data collected by each sensor in the inspection section, and analyzes the sensor data through the corresponding abnormality analysis strategy to obtain the hidden abnormality type of the fastener.

[0011] Furthermore, the fastener rapid analysis step includes an abnormal classification strategy, and the abnormal classification strategy includes a foreign body detection sub-step and a fastener deviation detection sub-step.

[0012] The foreign body detection sub-step selects the track image corresponding to the fasteners indicated as dominant abnormalities as the fastener map to be analyzed, analyzes the recognition completeness of the fasteners in the fastener map to be analyzed according to the preset fastener template map, and performs the fastener offset detection step if the recognition completeness is greater than or equal to the standard degree, and outputs the foreign body occlusion type if the recognition completeness is less than the standard degree;

[0013] The fastener offset detection sub-step uses the intersection point of the sleeper and the track in the fastener diagram to be analyzed as a reference point to determine the relative position of the fastener at the reference point. If the relative position is the same as the preset position, a normal instruction is output; otherwise, a fastener offset type is output.

[0014] Furthermore, the sensing data includes a vibration frequency value collected by a vibration sensor and a noise value collected by a noise sensor.

[0015] Furthermore, the abnormality analysis strategy includes a vibration analysis sub-step and a noise analysis sub-step.

[0016] The vibration analysis sub-step is to draw a continuous vibration frequency table according to the time sequence of the acquisition of the vibration frequency values, and analyze the frequency fluctuation according to the vibration frequency table to output whether the fastener is hidden abnormal or normal;

[0017] The noise analysis sub-step is to draw a continuous noise table according to the time sequence of collection after filtering the noise values, and analyze the noise fluctuation according to the noise table to output whether the fastener is hidden abnormal or normal.

[0018] Furthermore, the abnormality analysis strategy also includes a ripple verification sub-step, which draws a continuous point curve diagram between the camera and the preset reference point based on the continuously captured track images, and verifies whether the hidden abnormalities of the fasteners output in the vibration analysis sub-step and the noise analysis sub-step are accurate based on each fluctuation unit analysis in the point curve diagram.

[0019] Furthermore, the sleeper deformation formula is configured as:

[0020] ,

[0021] in, is the deformation rate of the sleeper, is the service life of the sleepers, is the ambient temperature, is the ambient humidity, are the first constant, the second constant, the third constant and the fourth constant respectively, is the time constant related to the sleeper characteristics, is the base of natural logarithms.

[0022] Furthermore, the fastener loosening calculation formula is configured as:

[0023] ,

[0024] in, is the fastener loosening rate, is the deformation rate of the sleeper, is the ambient temperature, is the reference temperature, is the ambient humidity, is the reference humidity, is the weight of the train when it passes for the i-th time, is the number of train passes, The usage time of the fastener. is the base of natural logarithms, They are respectively the first influence coefficient of ambient temperature on the fastener loosening rate and the second influence coefficient of ambient humidity on the fastener loosening rate.

[0025] Furthermore, it also includes an abnormal feedback step, marking the sleeper where the fastener corresponding to the latent abnormal type of the fastener is located as the abnormal position, and counting the number of fastener abnormalities at the abnormal position within a certain period of time. The number of fastener abnormalities includes the explicit abnormal type of the fastener and the latent abnormal type of the fastener. If the number of fastener abnormalities exceeds a preset threshold, feedback is given for warning.

[0026] Furthermore, the track image and sensor data are collected by a patrol vehicle or a previous traveling train.

[0027] A railway fastener abnormal type matching system, comprising:

[0028] The fastener looseness calculation module obtains the environmental data of the inspection section and the service life of the sleepers, calculates the sleeper deformation rate through the sleeper deformation, and then obtains the weight of the train passing through the section, the number of train passes, the service life of the fasteners, the environmental data and the sleeper deformation rate, and calculates the fastener looseness rate through the fastener looseness calculation;

[0029] An abnormal analysis selection module compares the fastener looseness rate with a preset threshold value and selects to perform a fastener rapid analysis step or a multi-sensor fusion analysis step according to the comparison result;

[0030] The fastener rapid analysis module obtains the track images containing fasteners in the inspection section, marks and sorts them in sequence according to the acquisition order to obtain an image set, and analyzes the image set through a preset fastener abnormality network to obtain the fastener explicit abnormality type;

[0031] The multi-sensor fusion analysis module obtains the sensor data collected by each sensor in the inspection section, and analyzes the sensor data through the corresponding abnormality analysis strategy to obtain the hidden abnormality type of the fastener.

[0032] The beneficial effects of the present invention are as follows: by judging the deformation of the sleepers and then combining other reference factors with the deformation of the sleepers to judge the looseness rate of the fasteners, it is possible to accurately judge the period when the fasteners themselves are abnormal, and select the means of inspection based on the results of the judgment, thereby improving the efficiency of the inspection. Specifically, when the looseness rate of the fasteners is too small, only rapid image analysis is required during the inspection process, the purpose of which is to analyze whether there is foreign matter covering the fasteners or whether the fasteners themselves are offset in the horizontal direction. When the looseness rate of the fasteners is too large, multi-sensor fusion analysis is required during the inspection process, the purpose of which is to analyze whether the fasteners themselves are loose. Detailed analysis is performed through vibration sensing and noise sensing, and then the distance between the image analysis camera and the preset reference point is verified to improve the accuracy of the analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is the overall flow chart of the present invention;

[0034] Figure 2 is a flow chart of the abnormal classification strategy in the present invention;

[0035] Figure 3 is a flow chart of the abnormal analysis strategy in the present invention;

[0036] Figure 4 It is a diagram of a fastener of the foreign body shielding type in the present invention;

[0037] Figure 5 It is a normal diagram of the fastener in the present invention. DETAILED DESCRIPTION

[0038] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. The same parts are represented by the same reference numerals. It should be noted that the words "front", "rear", "left", "right", "upper" and "lower" used in the following description refer to directions in the accompanying drawings, and the words "bottom surface" and "top surface", "inner" and "outer" refer to directions toward or away from the geometric center of a specific component, respectively.

[0039] Since the current railway fastener status detection mainly relies on the inspection of patrol vehicles, the images of each fastener are taken along the driving track in sequence through the vision on the patrol vehicle, and the fastener anomaly detection is realized on the collected images using a variant YOLO target detection network. Although the above-mentioned detection method based on computer vision has improved the detection accuracy to a certain extent, it can only detect fastener anomalies of known abnormal types. The inspection section contains thousands of fasteners, and the patrol vehicle needs to analyze each fastener image one by one during each inspection, which results in a large amount of analysis data, a long time consumption, and it is difficult to ensure the detection accuracy of many fasteners. In addition, it is impossible to accurately determine the period when the fastener itself has an abnormality, and can only rely on the artificially planned inspection cycle for regular inspection; therefore, the present invention designs this railway fastener anomaly type matching method, such as Figure 1 As shown, the following steps are included:

[0040] Data storage step: The railway background management system records the environmental data of the inspection section, the usage time of each sleeper, the model of each train passing through the inspection section (including weight), and the usage time of each fastener. The environmental data includes the ambient temperature and humidity of the area. Each sleeper has a number, and the background record corresponds to an installation time. Fasteners also have numbers and installation times.

[0041] The fastener looseness calculation step is to obtain the environmental data of the inspection section and the service life of the sleepers and calculate the sleeper deformation rate through the sleeper deformation;

[0042] The sleeper deformation formula is configured as:

[0043] ,

[0044] in, is the deformation rate of the sleeper, is the service life of the sleepers, is the ambient temperature, is the ambient humidity, are the first constant, the second constant, the third constant and the fourth constant respectively, is the time constant related to the sleeper characteristics, is the base of natural logarithms, The value range of is [0,∞), where 0 means no deformation and a larger value means a higher deformation rate. The longer the sleeper is used, the greater the deformation rate will be.

[0045] Then obtain the weight of trains passing through the section, the number of train passes, the service life of the fasteners, environmental data, and the deformation rate of the sleepers, and calculate the fastener looseness to obtain the fastener looseness rate;

[0046] The fastener loosening calculation formula is configured as:

[0047] ,

[0048] in, is the fastener loosening rate, is the deformation rate of the sleeper, is the ambient temperature, is the reference temperature, is the ambient humidity, is the reference humidity, is the weight of the train when it passes for the i-th time, is the number of train passes, The usage time of the fastener. is the base of natural logarithms, They are the first influence coefficient of ambient temperature on the fastener loosening rate and the second influence coefficient of ambient humidity on the fastener loosening rate. The value range is also [0,∞), where 0 indicates complete tightness and larger values ​​indicate greater looseness.

[0049] The calculation of fastener loosening rate is performed in real time by the background system. When the loosening rate of a fastener or several fasteners is calculated to be too large, an inspection instruction is sent to the inspection personnel. If the fastener loosening rate is within the specified range, the inspection vehicle will conduct regular inspections according to the predetermined inspection cycle.

[0050] Abnormal analysis selection step, according to the fastener loose rate and the preset threshold N1 comparison, if the fastener loose rate <Preset threshold N1, select the fastener quick analysis step, if the fastener loose rate ≥Preset threshold N1 multi-sensor fusion analysis step.

[0051] The fastener analysis step is to obtain the track images containing fasteners in the inspection section, mark and sort them in the acquisition order to obtain an image set, and analyze the image set through the preset fastener anomaly network to obtain the fastener explicit anomaly type, such as Figure 2 As shown, it also includes an abnormal classification strategy, which includes a foreign body detection sub-step and a fastener offset detection sub-step.

[0052] In the foreign body detection sub-step, the track image corresponding to the fasteners indicated as obvious abnormalities is screened out as the fastener map to be analyzed, and the recognition completeness of the fasteners in the fastener map to be analyzed is analyzed according to the preset fastener template map. If the recognition completeness is greater than or equal to the standard degree, the fastener offset detection step is performed. If the recognition completeness is less than the standard degree, the foreign body occlusion type is output, such as Figure 4As shown, specifically, under normal circumstances, it is only necessary to consider whether there are foreign objects wrapped around the fastener or whether the fastener is covered and blocked by foreign objects. Therefore, a standard fastener template image needs to be preset. The degree to which the fastener can be accurately identified and located by the template image matching algorithm is usually matched using familiarity calculation or matching calculation. Next, a standard degree N2 is set. If the fastener recognition completeness of a certain fastener image to be analyzed is greater than or equal to the standard degree, it means that the fastener is not covered by foreign objects or the fastener is not wrapped with foreign objects. The fastener exposed on the surface can be completely detected in the image. On the contrary, when covered or wrapped with foreign objects, the completeness obtained by the fastener image matching will be less than the standard degree. Therefore, it is necessary to identify that there are foreign objects on the fastener, output as foreign object blocking type, and perform acquired feedback marking. The existing marking method is to locate which fastener is by counting sleepers. The present invention can also locate by the order of collecting images.

[0053] The fastener offset detection sub-step takes the intersection of the sleeper and the track in the fastener diagram to be analyzed as a reference point, and determines the relative position of the fastener at the reference point. If the relative position is the same as the preset position, a normal instruction is output, otherwise, a fastener offset type is output. Specifically, the visual camera setting position on the inspection vehicle is fixed, and the inspection vehicle travels along the track. Therefore, under normal circumstances, the position of the fastener is also fixed relative to the camera, and the fastener position in the fastener diagram to be analyzed should be standard. The first step of the present invention is to determine the reference point: by marking a reference point at the intersection of the sleeper and the track. Under normal circumstances, the sleeper and the track are set vertically, so the center point of the vertical position is found as the reference point;

[0054] The second step is to determine the position of the fastener: the identified fasteners are screened for contour points, and the positional relationship (such as distance and angle) of the selected contour points on the fasteners relative to the reference points is calculated;

[0055] The third step is to compare with the preset position: the calculated fastener position is compared with the preset fastener standard position. The preset fastener standard position is determined based on the position of the fastener relative to the intersection of the sleeper and the track in the normal track image. If the actual position of the fastener is the same as the preset position or the error is within the allowable range, the fastener position is considered normal and a normal instruction is output (such as Figure 5 The figure shows the normal fastener);

[0056] Step 4: Offset judgment and output: If the actual position of the fastener deviates significantly from the preset position and exceeds the allowable error range, it is considered that the fastener has offset, mainly in the horizontal direction, but it may not affect the stability of the fastener fixed on the sleeper;

[0057] For the above-mentioned types, whether they are foreign object obstruction or fastener offset, they will be automatically marked to realize data landing, trigger the conditions for further on-site inspection or maintenance operations, and guide inspectors to conduct fixed-point inspections to ensure the safe operation of the track.

[0058] The fastener anomaly network is an abnormal state detection network without negative samples. By building a fastener anomaly detection network, it can ensure that fasteners can be detected in different abnormal states and can cover various fastener failure forms. The main purpose is to detect whether there is an abnormality in the fastener, but the type of abnormality cannot be detected. The purpose of using it in the present invention is to detect whether an abnormality occurs at the fastener;

[0059] The construction of the fastener anomaly network is specifically as follows: track image acquisition steps, obtaining images of the track area including fasteners under different scenes taken by the camera. Specifically, a line scan camera mounted on the bottom of an electric passenger car or an engineering vehicle is used to take images of the track area including fasteners. The line scan camera has a resolution of 4K (up to 4096 lines can be distinguished per image height perpendicular to the scanning direction) of an infrared camera module.

[0060] The fastener data sorting step is to screen out fastener images from the track area images through a preset target detection network and combine them to obtain a fastener data set. Specifically, 5,000 images containing fasteners are first collected using a line scan camera, and the fastener target detection data set is annotated and sorted according to the YOLO format. The labeling is performed using LabelImg software, the category is labeled as block, and the training set, test set, and validation set are divided according to a ratio of 8:1:1;

[0061] Secondly, build a fastener target detection network and train the target detection network to form a fastener target detection model. Specifically, the preset target detection network uses the YOLOV11 network, which is integrated with the transform feature extraction network. The YOLOV11s model pre-trained in the Images dataset is used as the pre-training model, and is trained in the fastener target detection dataset to achieve iterative tuning of network parameters. During training, data enhancement processing such as image stretching and rotation is performed. The fusion of the transform feature extraction network can further extract the multi-scale texture, color, and logical information of the fasteners. , which helps to optimize the parameters of the YOLOV11 network and realize the accurate recognition of the target. The network is trained for at least 300 rounds, and the model with the smallest loss value is taken as the fastener target detection model, and it is converted into a model in TensorRT format, which can be adapted to the host of the ARM architecture on conventional trains for model reasoning. The model input is an infrared image, which is compressed to 640*640 in size, and the output is a list of n*5, where n represents the number of fasteners detected in the input image. The first dimension consists of the category of the detected target, the x and y coordinates of the upper left corner of the normalized minimum enclosing rectangle, and the length and width;

[0062] Finally, a line scan camera is used to collect images of the track area including fasteners in normal conditions to create a fastener dataset. Specifically, images of different usage states are collected in tunnels and open-air scenes under different ambient lights. The different ambient lights include: ① Tunnel lights will affect the tunnel scene, and the image brightness of the corresponding position close to the tunnel light is high; the image brightness of the corresponding position far from the tunnel light is low; ② Ambient light in different time periods and weather conditions in open-air environments.

[0063] Different usage states refer to: newly installed fasteners are relatively clean, but as the use time goes by, the surface of the fasteners will be covered with oil, dust, etc., causing the state of the fasteners to change. The normal state refers to that all the components of the fasteners are not deformed, damaged, or have failed, and there are no foreign objects covering the fasteners. Therefore, the collected images must include all the above situations, and the data volume must exceed 20,000 images that meet the above requirements. The fastener target detection model is used to perform fastener target detection on all the above images collected, and the position and size information of the minimum circumscribed rectangle of all detected fasteners on the original image are cut out, and several infrared images containing only fasteners are generated as the fastener data set, and the fastener data set is divided into a training set and a test set at a ratio of 9:1.

[0064] The present invention uses target detection to detect fasteners on the original image and intercept the fastener image, and does not use semantic segmentation to directly segment the fasteners. The reason is that the fasteners have complex shapes and changeable states. The existing semantic segmentation network cannot quickly and accurately segment the correct fastener area, and the number of fasteners on each railway line is large. Considering the image transmission speed and data storage cost, the image containing only the fasteners after interception can effectively filter out the impact of track areas such as rails, ramps, sleepers, etc. on subsequent abnormality detection.

[0065] Since railway fasteners are composed of spring bars, rail pads, threaded spikes, threaded flat washers, double spring washers, buckle plates, iron seats, insulating pads, bolt spikes, nuts, flat washers, etc., their failure forms are diverse, the types of foreign matter coverage are unknown, and negative samples, i.e., images of railway fasteners in a failed state or fasteners covered with foreign matter, are not easy to obtain. The existing classified or supervised detection algorithms can only detect specific types of anomalies, which cannot meet the needs of detecting all anomalies of railway fasteners. Therefore, the present invention includes a fastener anomaly detection network construction step, including a three-stage unsupervised railway fastener anomaly detection network, which only requires fastener data in a normal state and can detect all types of anomalies of fasteners, specifically:

[0066] First, a random floating point number matching threshold η from 0 to 1 is generated by the numpy open source library. When the threshold η is in the range of 0 to 0.7 (threshold range one), the fastener image is processed by the pseudo-anomaly and label generation strategy to obtain the first image. When the threshold η is in the range of 0.7 to 1 (threshold range two), the fastener image is processed by the data enhancement strategy to obtain the first image. The features extracted from the first image are reconstructed through the reconstruction network to obtain the second image. The second image is passed through a preset contrast function to obtain the contrast scores of all corresponding associated pixels between the second image and the fastener image.

[0067] The fastener anomaly model training step is to substitute the fastener dataset into the fastener anomaly detection network for reciprocating training for at least 800 rounds, and select the network model with the smallest loss value as the fastener anomaly model, that is, convert it into a TensorRT model.

[0068] The step of outputting the abnormal fastener is to substitute the real-time fastener image into the abnormal fastener model to output the abnormal contour mark and the contrast score at the abnormal contour. The contrast score is usually between 0 and 10. The larger the score, the greater the probability that the pixel location is abnormal. In the abnormal detection of railway fasteners, pixels with a score greater than 0.15 are identified as abnormal. The contrast score image is binarized, and the binarization boundary is 0.15. The pixel values ​​greater than 0.15 are set to 255, and the remaining pixel values ​​are set to 0. Therefore, the result of the abnormal detection of railway fasteners has been obtained. When the railway fastener is normal, all pixel values ​​of the output image are 0. Otherwise, the pixel values ​​of the output image are not all 0. In addition, the findContours function in the OpenCV open source library is used to detect the abnormal contour in the image, and then the minAreaRect function is used to calculate the minimum circumscribed rectangle of the abnormal contour and draw it on the input image, so that the abnormal area can be clearly displayed, so as to facilitate intuitive monitoring by the system background.

[0069] The pseudo-anomaly and label generation strategy includes randomly selecting image cropping in a preset texture recognition data set and seamlessly fusing it with the fastener image through a fusion algorithm to generate a corresponding label image. Specifically, the pseudo-anomaly and label generation strategy includes a target image cropping step and an image fusion and label generation step.

[0070] The target image cropping step randomly selects a category and an image ID in a preset DTD texture recognition data set as the first target image, and randomly generates an irregular area whose minimum circumscribed rectangle has a length and a width that are smaller than the length and the width of the target image, and cuts the irregular area in the target image to generate a new second target image;

[0071] In addition to randomly generating irregular areas for cropping, a variety of cropping strategies can also be introduced, such as intelligent cropping based on image content (such as identifying key objects or areas in the image and cropping them), cropping based on image edges (ensuring that the edges of the cropped image are smooth and the information is complete), etc.

[0072] The image fusion and label generation step uses a fusion algorithm to obtain a seamless fusion of the fastener image, the second target image, and the irregular area, and changes the pixels in the irregular area to 1 and the pixel values ​​in other areas to 0 to form a corresponding label image;

[0073] Specifically, the steps of image fusion and label generation are as follows: the three-channel superposition of the fastener image forms a three-channel image with equal pixel values ​​in each channel, and the fastener image is recorded as g , the second target image is recorded as f * , the area within the irregular shape in the second target image is recorded as , the boundary of the irregular region is recorded as , establish the Poisson equation and find an interpolation function , so that in the fusion area Inside, The gradient and source image g The gradient is consistent, and at the boundary of the fusion area superior, The value of and the target image f * The values ​​are consistent, and the Poisson equation is configured as:

[0074] ,

[0075] ,

[0076] in, and Represent the interpolation function and fastener images In irregular areas The Laplacian operator in ;

[0077] Then discretize the Laplace operator. After discretization, the Laplace operator in the two-dimensional image is expressed as:

[0078] ,

[0079] in, h is the spacing between pixels in a 2D image (usually 1 in a uniform grid), For a two-dimensional image at pixel points The value on

[0080] After discretizing the above Laplace operator, Poisson's equation is expressed as the following linear equations:

[0081] ,

[0082] in, A is a sparse matrix describing the irregular region The Laplace relationship between each pixel in the (fusion area) and its neighboring pixels, is a column vector containing the irregular area The values ​​of all pixels in the (fusion area), b is a column vector containing the fastener image g In irregular areas Laplace filtering result of each pixel in the (fusion area);

[0083] For irregular areas Each pixel in the (fusion area) , the established equation is:

[0084] ,

[0085] in, For fastener images g In pixels Laplace filtering result on ;

[0086] Finally, due to A is a sparse matrix, and the sparse matrix solving algorithm is used to solve the above linear equations To get the interpolation function In irregular areas The values ​​of all pixels in the (fusion area) are solved to obtain the interpolation function Then, use the interpolation function Replace the target image with the value of f * The value of the corresponding position in the image is calculated to achieve seamless fusion of the fastener image and the second target image. Then, the irregular area of ​​the second target image is The pixels in the (fusion area) become 1, and the pixel values ​​in other areas become 0, forming the corresponding labels.

[0087] The data enhancement strategy includes superimposing three channels of the fastener image to form an image with equal pixel values ​​in each channel, and randomly performing one or more image processing to obtain an enhanced image to increase the diversity and richness of the data to prevent the model from overfitting during the training process. The image processing includes image flipping, image rotation, image cropping, image scaling, image translation or image dithering.

[0088] In addition to basic operations such as image flipping, rotation, and cropping, multi-scale transformation can be introduced, that is, scaling and cropping images at different scales. This helps the network learn the features of fasteners at different sizes and improves its generalization ability on images of different resolutions.

[0089] Regardless of whether the fastener image is processed by the pseudo-anomaly and label generation strategy or the data enhancement strategy through random selection, it is necessary to convert the label image or the enhanced image from the spatial domain image to the frequency domain image through Fourier transform, and filter the frequency domain image to filter out the interference of noise such as dust and rust on the railway fastener to obtain the first image. This process pays more attention to texture information than the spatial domain image and is more suitable for feature extraction of infrared images.

[0090] The backbone network of the reconstruction network adopts the U-Net network, and at the lowest end of the U-Net network, that is, the features are pulled into one dimension and then connected to the ResNet50 network, so that the U-Net network after connecting to the ResNet50 network has stronger ability in extracting high-dimensional information, and can pay more attention to the position logic information of each component in the fastener. A loss function group is set in the reconstruction network, which includes a structural similarity phase index function, a mean square error loss function and a perceptual loss function. The structural similarity phase index function and the mean square error loss function are used to judge the difference between each pixel for calculation and pay attention to the local difference between the two images. The perceptual loss function is used to judge the perceptual quality of the image, which is in line with the human eye's perception of image quality. The combined use is more conducive to the effect of reconstructing the image. The backbone network of the perceptual loss function is composed of VGG-19. The input image size of this network is 512*512*3, and the output image size is also 512*512*3. The purpose of this network is to reconstruct the normal area of ​​the railway fastener better, but for the abnormal area, this network cannot reconstruct it, so that the abnormal area is reconstructed as a fuzzy and irregular image.

[0091] The contrast function includes a multi-scale gradient magnitude similarity function (MSGMS) and an ISSM function. The multi-scale gradient magnitude similarity function calculates the gradient magnitude similarity between the fastener image and the second image at different scales through an image pyramid, specifically including: ① constructing an image pyramid: constructing image pyramids for the fastener image and the second image respectively, and the image at each scale is obtained by smoothing and downsampling the image at the previous scale; ② calculating the gradient: calculating the gradient of the image at each scale; ③ calculating the gradient magnitude similarity; the ISSM function is used to calculate the brightness similarity, contrast similarity and structural similarity between the fastener image and the second image, that is, the difference between the two images is obtained by calculating the mean (brightness), standard deviation (contrast) and covariance (structure) of the images respectively.

[0092] Since the multi-sensor fusion analysis step is selected after the calculation of the fastener loosening rate, the fastener positions with abnormal calculated fastener loosening rates should be marked at this time. When performing the multi-sensor fusion analysis step, targeted sensor collection should be performed, such as Figure 3As shown, the sensor data collected by each sensor at the marked fastener in the inspection section is obtained. Assuming that the marked fasteners are 2024-111, 2024-112...2024-145, the sensor data collected by the inspection vehicle should include the data at 2024-111 to 2024-145, and the sensor data includes the vibration frequency value collected by the vibration sensor and the noise value collected by the noise sensor. The vibration frequency value reflects the vibration state of the inspection vehicle when it is driving, and the noise value reflects the sound intensity around the inspection vehicle when it is driving. The vibration frequency value and the noise value both include 35 values. The sensor data are analyzed by the corresponding abnormal analysis strategy to obtain the hidden abnormality type of the fastener, wherein the abnormal analysis strategy includes a vibration analysis sub-step and a noise analysis sub-step. The vibration analysis and the noise analysis are performed simultaneously, specifically:

[0093] Vibration analysis sub-steps: 1. Data preprocessing and cleaning: Check the 35 vibration frequency values ​​collected above to see if there are obvious abnormal values ​​or missing values. If the vibration frequency value is too large or too small, it may be caused by sensor failure, environmental interference or data transmission error. The processing method is to choose to delete, replace or smooth. If the vibration frequency value is missing, it may be caused by the sensor failing to successfully record data or data transmission interruption. The processing method can be to use interpolation methods or other data filling strategies to complete it;

[0094] Draw a vibration frequency table: When the above 35 vibration frequency values ​​can be used for subsequent analysis after processing, use the chart tool (line chart) to visualize the vibration frequency values ​​in chronological order to form a continuous vibration frequency table, which helps to intuitively observe the changing trend and periodic characteristics of the vibration frequency; Frequency fluctuation analysis: According to the broken line in the vibration frequency table, judge the periodic change trend of the vibration fluctuation and whether it exceeds the threshold of the vibration fluctuation. If the actual vibration frequency exceeds the threshold, it may indicate that there is a hidden abnormality in the fastener, and the hidden abnormality is reflected as the looseness of the fastener itself. Noise analysis sub-steps, 1. Data preprocessing and noise filtering: The above 35 collected noise values ​​are processed through noise filtering algorithms (such as bandpass filters, low-pass filters, high-pass filters, etc.) to remove background noise and irrelevant factors. The purpose of noise filtering is to improve the signal-to-noise ratio of the noise data so that it can better reflect the changes in the fastener status. When the fastener is loose, there will be a gap between the track and the sleeper, resulting in greater vibration and noise when the inspection vehicle travels here;

[0095] Draw a noise table: visualize the 35 noise values ​​after noise filtering in chronological order to form a continuous noise table, which helps to intuitively observe the changing trend and periodic characteristics of the noise; noise fluctuation analysis: judge the periodic change trend of the noise fluctuation and whether it exceeds the threshold of the noise fluctuation based on the broken line in the noise table. If the actual noise exceeds the threshold, it may indicate that there is a hidden abnormality in the fastener, and the invisible abnormality is reflected in the looseness of the fastener itself. When any one of the vibration analysis and noise analysis above outputs a hidden abnormality, it means that there is a risk of abnormality in the fasteners. The corresponding abnormal analysis strategy also includes a ripple verification sub-step, which draws a continuous point curve diagram between the camera and the preset reference point based on the continuously captured track images, and verifies whether the hidden abnormality of the fasteners output in the vibration analysis sub-step and the noise analysis sub-step is accurate based on each fluctuation unit analysis in the point curve diagram; for example, the invention patent with patent number: 2024112745875 discloses a method and system for detecting abnormality of a wheelset of a rail train, which specifically discloses judging whether there is an abnormality in the wheelset by analyzing the fluctuation unit in the point curve diagram, and in the present invention, it can be judged through the analysis of the fluctuation unit that when the inspection vehicle is running at a loose fastener, the track will vibrate under the pressure of the sleeper, and the preset reference point should be a point on the sleeper.

[0096] The current inspection relies on inspection vehicles, while the track images and sensor data in the present invention are collected by the inspection vehicle or the previous train. After each train has collected the track image, the data is landed and analyzed in the background system, which is less costly than another inspection by the inspection vehicle.

[0097] It also includes an abnormality feedback step, marking the sleeper where the fastener corresponding to the latent abnormality type of the fastener is located as the abnormal position, counting the number of fastener abnormalities at the abnormal position within a certain period of time, and the number of fastener abnormalities includes the explicit abnormality type of the fastener and the latent abnormality type of the fastener. If the number of fastener abnormalities exceeds the preset threshold, feedback is given for warning. If the number of fastener abnormalities on a certain sleeper is too many within a certain period, it means that the fastener itself may not be the cause, and it is necessary to consider whether there are abnormalities in the track and the sleeper as a warning condition for triggering whether the track and the sleeper are abnormal.

[0098] According to the designed detection method, a railway fastener abnormality type matching system is designed, including

[0099] The fastener looseness calculation module obtains the environmental data of the inspection section and the service life of the sleepers, calculates the sleeper deformation rate through the sleeper deformation, and then obtains the weight of the train passing through the section, the number of train passes, the service life of the fasteners, the environmental data and the sleeper deformation rate, and calculates the fastener looseness rate through the fastener looseness calculation;

[0100] The abnormal analysis selection module compares the fastener loosening rate with a preset threshold value and selects a fastener rapid analysis step or a multi-sensor fusion analysis step according to the comparison result;

[0101] The fastener analysis module obtains the track images containing fasteners in the inspection section, marks and sorts them in the acquisition order to obtain an image set, and analyzes the image set through a preset fastener anomaly network to obtain the fastener explicit anomaly type;

[0102] The multi-sensor fusion analysis module obtains the sensor data collected by each sensor in the inspection section, and analyzes the sensor data through the corresponding abnormality analysis strategy to obtain the hidden abnormality type of the fastener.

[0103] The above are only preferred embodiments of the present invention. The protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A method for matching abnormal types of railway fasteners, characterized in that: The steps include: The fastener looseness calculation step comprises obtaining environmental data and the service life of the sleepers in the inspection section, calculating the sleeper deformation rate through the sleeper deformation, and then obtaining the weight of the trains passing through the section, the number of train passes, the service life of the fasteners, environmental data and the sleeper deformation rate, calculating the fastener looseness rate through the fastener looseness; The sleeper deformation calculation formula is configured as: ， in, is the deformation rate of the sleeper, is the service life of the sleepers, is the ambient temperature, is the ambient humidity, are the first constant, the second constant, the third constant and the fourth constant respectively, is the time constant related to the sleeper characteristics, is the base of natural logarithms; The fastener loosening calculation formula is configured as: ， in, is the fastener loosening rate, is the deformation rate of the sleeper, is the ambient temperature, is the reference temperature, is the ambient humidity, is the reference humidity, is the weight of the train when it passes for the i-th time, is the number of train passes, The usage time of the fastener. is the base of natural logarithms, They are respectively the first influence coefficient of ambient temperature on the fastener loosening rate and the second influence coefficient of ambient humidity on the fastener loosening rate; an abnormal analysis selection step, comparing the fastener loosening rate with a preset threshold, and selecting a fastener rapid analysis step or a multi-sensor fusion analysis step according to the comparison result; A fast fastener analysis step, obtaining rail images containing fasteners in the inspection section, marking and sorting them in sequence according to the acquisition order to obtain an image set, and analyzing the image set through a preset fastener anomaly network to obtain the fastener explicit anomaly type; The multi-sensor fusion analysis step obtains the sensor data collected by each sensor in the inspection section, and analyzes the sensor data through the corresponding abnormality analysis strategy to obtain the hidden abnormality type of the fastener.

2. A method for matching abnormal types of railway fasteners according to claim 1, characterized in that: The fastener rapid analysis step includes an abnormal classification strategy, and the abnormal classification strategy includes a foreign body detection sub-step and a fastener deviation detection sub-step. The foreign body detection sub-step selects the track image corresponding to the fasteners indicated as dominant abnormalities as the fastener map to be analyzed, analyzes the recognition completeness of the fasteners in the fastener map to be analyzed according to the preset fastener template map, and performs the fastener offset detection step if the recognition completeness is greater than or equal to the standard degree, and outputs the foreign body occlusion type if the recognition completeness is less than the standard degree; The fastener offset detection sub-step uses the intersection point of the sleeper and the track in the fastener diagram to be analyzed as a reference point to determine the relative position of the fastener at the reference point. If the relative position is the same as the preset position, a normal instruction is output; otherwise, a fastener offset type is output.

3. A method for matching abnormal types of railway fasteners according to claim 1, characterized in that: The sensing data includes a vibration frequency value collected by a vibration sensor and a noise value collected by a noise sensor.

4. A method for matching abnormal types of railway fasteners according to claim 2 or 3, characterized in that: The abnormality analysis strategy includes a vibration analysis sub-step and a noise analysis sub-step. The vibration analysis sub-step is to draw a continuous vibration frequency table according to the time sequence of the acquisition of the vibration frequency values, and analyze the frequency fluctuation according to the vibration frequency table to output whether the fastener is hidden abnormal or normal; The noise analysis sub-step is to draw a continuous noise table according to the time sequence of collection after filtering the noise values, and analyze the noise fluctuation according to the noise table to output whether the fastener is hidden abnormal or normal.

5. A method for matching abnormal types of railway fasteners according to claim 4, characterized in that: The abnormality analysis strategy also includes a ripple verification sub-step, which draws a continuous point curve diagram between the camera and the preset reference point based on the continuously captured track images, and verifies whether the hidden abnormalities of the fasteners output in the vibration analysis sub-step and the noise analysis sub-step are accurate based on each fluctuation unit analysis in the point curve diagram.

6. A method for matching abnormal types of railway fasteners according to claim 1, characterized in that: It also includes an abnormal feedback step, marking the sleeper where the fastener corresponding to the latent abnormal type of the fastener is located as the abnormal position, and counting the number of fastener abnormalities at the abnormal position within a certain period of time. The number of fastener abnormalities includes the explicit abnormal type of the fastener and the latent abnormal type of the fastener. If the number of fastener abnormalities exceeds a preset threshold, feedback is given for warning.

7. A railway fastener abnormal type matching method according to claim 1, characterized in that: The track image and sensor data are collected by a patrol vehicle or a previous traveling train.

8. A railway fastener abnormal type matching system, characterized by: include The fastener looseness calculation module obtains the environmental data of the inspection section and the service life of the sleepers, calculates the sleeper deformation rate through the sleeper deformation, and then obtains the weight of the train passing through the section, the number of train passes, the service life of the fasteners, the environmental data and the sleeper deformation rate, and calculates the fastener looseness rate through the fastener looseness calculation; The sleeper deformation calculation formula is configured as: ， in, is the deformation rate of the sleeper, is the service life of the sleepers, is the ambient temperature, is the ambient humidity, are the first constant, the second constant, the third constant and the fourth constant respectively, is the time constant related to the sleeper characteristics, is the base of natural logarithms; The fastener loosening calculation formula is configured as: ， in, is the fastener loosening rate, is the deformation rate of the sleeper, is the ambient temperature, is the reference temperature, is the ambient humidity, is the reference humidity, is the weight of the train when it passes for the i-th time, is the number of train passes, The usage time of the fastener. is the base of natural logarithms, They are respectively the first influence coefficient of ambient temperature on the fastener loosening rate and the second influence coefficient of ambient humidity on the fastener loosening rate; An abnormal analysis selection module compares the fastener looseness rate with a preset threshold value and selects to perform a fastener rapid analysis step or a multi-sensor fusion analysis step according to the comparison result; The fastener rapid analysis module obtains the track images containing fasteners in the inspection section, marks and sorts them in sequence according to the acquisition order to obtain an image set, and analyzes the image set through a preset fastener abnormality network to obtain the fastener explicit abnormality type; The multi-sensor fusion analysis module obtains the sensor data collected by each sensor in the inspection section, and analyzes the sensor data through the corresponding abnormality analysis strategy to obtain the hidden abnormality type of the fastener.

Images