A deep learning-based rail fastener anomaly recognition method and system
By employing a deep learning-based method for identifying railway fastener anomalies, utilizing a lightweight YOLOv5 network and improved data augmentation and imbalanced learning branches, the problems of slow detection speed and low accuracy of railway fasteners are solved, achieving fast, accurate, and timely identification and feedback of fastener anomalies.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- LI CHUANG ZHI HENG ELECTRONICS TECH CO LTD
- Filing Date
- 2022-03-15
- Publication Date
- 2026-05-12
Smart Images
Figure CN114612833B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of neural network technology, and in particular to a method and system for identifying anomalies in railway rail fasteners based on deep learning. Background Technology
[0002] Fasteners (joints) used to secure railway tracks are crucial for track safety; their condition directly impacts train operation and the potential for safety and economic hazards. With the rapid development of railway technology, the design of automatic track condition detection systems has become a top priority for railway authorities.
[0003] Compared to manual inspection methods that rely on human visual inspection, automated fastener inspection methods are faster and more accurate. Automated fastener inspection typically involves acquiring images of the railway track along its direction and then detecting the status of the fasteners within those images. However, in real-world scenarios, the area containing the fasteners in the acquired track images is often small, making inspection difficult or requiring the inspection system to have high computing power, resulting in problems such as large computational load and high detection latency. Summary of the Invention
[0004] In order to provide a detection method with fast detection speed, high recognition accuracy and timely status feedback, this application provides a method and system for identifying abnormalities in railway fasteners based on deep learning.
[0005] The first aspect of this application provides a method for identifying railway track fastener anomalies based on deep learning, including:
[0006] Obtain the track depth map and the corresponding acquisition location, and input it into the pre-established fastener detection model;
[0007] The fastener detection model responds to the input rail depth map, locates the fastener in the rail depth map, and generates a prediction box image with the fastener and basic information of the prediction box image. The basic information of the prediction box image includes the location of the fastener in the prediction box image, preliminary classification, and rail image acquisition time.
[0008] The predicted bounding box image and its basic information are input into a pre-established fastener abnormality recognition model.
[0009] The fastener abnormality recognition model responds to the prediction box image by performing secondary classification on the fasteners in the prediction box image to obtain the final category of the fasteners.
[0010] In one implementation, the pre-established fastener detection model is established through the following steps:
[0011] Obtain a rail depth map, mark the location and type of fasteners in the rail depth map, and preprocess the rail depth map;
[0012] The preprocessed track depth map is randomly scaled, randomly stitched, color gamut converted, and white noise is added before being stitched together to form a sample image.
[0013] The sample images are input into a lightweight YOLOv5 network to complete a preset period of combined training, which is divided into backbone network training and YOLOv3 detector head training.
[0014] Select sample images in the sample images whose aspect ratio of the fastener area exceeds a preset threshold as second samples. Use the second samples to train the backbone network separately for a preset number of few samples. Then, synchronize the residuals learned by backpropagation of the backbone network's few sample training to the lightweight YOLOv5 network's backbone network with a lower learning rate.
[0015] The YOLOv3 detector head is trained a preset number of times using the second sample alone, and the residuals learned by backpropagation of the YOLOv3 detector head's few-sample training are synchronized to the lightweight YOLOv5 network's YOLOv3 detector head with a lower learning rate.
[0016] In one implementation, the pre-established fastener abnormality identification model is established through the following steps:
[0017] The predicted bounding box image output by the fastener detection model and the basic information of the predicted bounding box image are input into the secondary recognition network. The secondary recognition network includes an imbalanced learning branch based on a reweighted loss function and a data augmentation branch based on an improved Mixup algorithm.
[0018] The data augmentation branch interpolates the initial category of fasteners in the prediction box image and shifts the interpolated category toward the fastener abnormal state category.
[0019] The initial category of the fastener in the prediction box image is shifted to the abnormal state category of the fastener by an imbalanced learning branch.
[0020] The imbalanced learning branch and the data augmentation branch are dynamically weighted to obtain the loss value, and the loss value is backpropagated to update the parameters of the secondary recognition network in a synchronous manner to obtain the fastener abnormal state recognition model.
[0021] In one implementation, the loss function of the imbalanced learning branch is:
[0022]
[0023] Where y is the actual label of the sample, γ is the modulation coefficient, n is the number of samples, and p i To predict the probability that a sample belongs to category i, C is the total number of all categories, β∈[0,1) is a hyperparameter, and E is the number of valid samples of category i. i .
[0024] In one implementation, the loss value is: Loss = αL imb +(1-α)L mu , where L imb L is the loss function for the imbalanced learning branch. mu It is the loss function for the data augmentation branch.
[0025] In one implementation, the step of obtaining the rail depth map and the corresponding acquisition location, and inputting it into a pre-established fastener detection model, specifically includes:
[0026] Obtain the track depth map and the corresponding data collection location;
[0027] The rail depth map is preprocessed, and the preprocessed rail depth map is randomly scaled, randomly stitched, color gamut converted, and white noise is added before stitching to obtain the image to be detected.
[0028] The image to be detected and the corresponding acquisition position are input into the pre-established fastener detection model.
[0029] One implementation also includes:
[0030] The track depth map and its corresponding acquisition location are stored on the server via UDP;
[0031] In addition, UDP communication is used to send the final category of the fastener, the location of the corresponding fastener, and the rail depth map back to the server;
[0032] If the final category of the fastener is an abnormal state, an alarm signal is generated and issued. The abnormal state includes at least a loose or obstructed fastener.
[0033] The second aspect of this application provides a rail fastener anomaly identification system based on deep learning, characterized in that the rail fastener anomaly identification system based on deep learning is used to execute the rail fastener anomaly identification method based on deep learning provided in the first aspect of this application;
[0034] The deep learning-based railway fastener anomaly identification system includes:
[0035] The image acquisition module is used to acquire depth maps of the railway tracks.
[0036] The fastener positioning and detection module is used to determine the acquisition location of the rail depth map;
[0037] Fastener status recognition module, wherein the fastener status recognition module stores a fastener detection model and a fastener abnormal status recognition model;
[0038] The fastener detection model is used to input the rail depth map, locate the fastener in the rail depth map, and generate a prediction box image with the fastener and the basic information of the prediction box image.
[0039] The fastener abnormality recognition model is used to perform secondary classification of the fasteners in the prediction box image based on the prediction box image to obtain the final category of the fastener.
[0040] A third aspect of this application provides a computer device, characterized in that it comprises:
[0041] Memory, used to store computer programs;
[0042] A processor is configured to execute the computer program to implement the steps of a deep learning-based method for identifying abnormal rail fasteners as provided in the first aspect of this application.
[0043] The fourth aspect of this application provides a computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is processed and executed, it implements the steps of the deep learning-based method for identifying abnormal rail fasteners provided in the first aspect of this application.
[0044] This application provides a deep learning-based method for identifying railway fastener anomalies. First, a railway depth map and its corresponding acquisition location are acquired and input into a pre-established fastener detection model. Then, the fastener detection model locates the fastener position in the railway depth map and generates a predicted bounding box image containing the fastener and its basic information. Next, the predicted bounding box image and its basic information are input into a pre-established fastener anomaly identification model. The model responds to the predicted bounding box image by performing secondary classification on the fasteners within it, obtaining the final category of the fastener. By locating fasteners in the railway depth map, extracting predicted bounding box images containing fasteners, and then using the fastener anomaly identification model to identify the final category of the fastener, computational complexity is reduced, detection speed and recognition accuracy are improved, and timely feedback of detection results is ensured. Attached Figure Description
[0045] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0046] Figure 1 A flowchart illustrating a deep learning-based method for identifying anomalies in railway fasteners, provided as an embodiment of this application;
[0047] Figure 2 This is a schematic diagram of the fastener detection model training process provided in the embodiments of this application;
[0048] Figure 3 A schematic diagram of the training process for the fastener abnormal state recognition model provided in this application embodiment;
[0049] Figure 4 This application provides a schematic diagram of three states of the fastener in its embodiments.
[0050] Figure 5 Sample images for training provided in the embodiments of this application;
[0051] Figure 6 The fastener image provided in this application is obtained using mosaic enhancement;
[0052] Figure 7 The fastener image provided in this application embodiment is obtained using improved data enhancement. Detailed Implementation
[0053] To provide a detection method with high speed, high accuracy, and timely status feedback, this application provides a deep learning-based method and system for identifying anomalies in railway fasteners. Figure 1 The diagram shown is a flowchart of a deep learning-based method for identifying anomalies in rail fasteners provided in this application. The deep learning-based method for identifying anomalies in rail fasteners includes steps S101 to S104.
[0054] S101, obtain the rail depth map and the corresponding acquisition location, and input the pre-established fastener detection model.
[0055] In practical applications, the depth map of the railway track is obtained through the image acquisition module. Specifically, an industrial camera is used to acquire the depth information of the railway track to obtain a 16-bit depth 1-channel railway track depth map of 1280×1000. When acquiring the railway track depth map, the corresponding position of the railway track depth map is acquired simultaneously, and the railway track image acquisition time is recorded.
[0056] The process involves preprocessing the rail depth map, and then randomly scaling, randomly stitching, converting the color gamut, and adding white noise to the preprocessed rail depth map to create an image to be detected. The image to be detected and the acquisition locations of the rail depth image are then input into a pre-established fastener detection model.
[0057] See Figure 2This is a schematic diagram of the fastener detection model training process provided in the embodiments of this application. The pre-established fastener detection model is established through S201 to S205:
[0058] S201, Obtain the rail depth map, mark the location and type of fasteners in the rail depth map, and preprocess the rail depth map.
[0059] Since rail depth maps are automatically acquired by equipment, typically through continuous or intermittent image acquisition along the rail, the acquired images represent the entire track. Therefore, the rail depth map contains numerous areas where fasteners are absent. In this embodiment, after obtaining the rail depth map, the location and type of fasteners are marked. Marking the location of fasteners involves circling the area where the fastener is located on the rail depth map. The marked rail depth map is then preprocessed. The fastener categories include at least three states: normal, loose, and obstructed. Figure 4 As shown, the three fastener images arranged from left to right in the figure correspond to the fasteners in three states: normal, loose, and obscured.
[0060] S202, after randomly scaling, randomly stitching, color gamut conversion and adding white noise to the preprocessed rail depth map, the images are stitched together to form a sample image.
[0061] To improve detection speed, the proposed embodiment suggests changing the mosaic enhancement from a 4-image stitching to a 9-image stitching, such as... Figure 5 , Figure 6 and Figure 7 As shown, unlike the scenarios used in mosaic enhancement, the localization and recognition of rail fasteners does not involve small samples or closely spaced samples. Therefore, the improved data augmentation proposed in this embodiment does not employ the random scaling and stitching strategy used in mosaic enhancement, but instead directly performs random cropping and stitching. Specifically, the preprocessed rail depth map is randomly scaled, randomly stitched, color gamut converted, and white noise is added before being stitched into sample images. Furthermore, the size of the rail fastener is approximately 250×250mm, so to improve detection speed, this detection network increases the number of image stitches. This will accelerate the convergence speed of the offline detection network, indirectly increasing the amount of training data fed in, making the training less dependent on the GPU (graphics processing unit) and faster.
[0062] It should be noted that, Figure 5 , Figure 6 and Figure 7 In the image, the image within the black box is the fastener image, and the rectangular black area in the attached image is the corresponding image of the railway track.
[0063] S203, the sample image is input into the lightweight YOLOv5 network to complete the combined training for a preset period. The combined training is divided into backbone network training and YOLOv3 detector head training.
[0064] S204, filter the sample images in the sample images whose aspect ratio of the fastener area exceeds a preset threshold, and use them as the second sample. Use the second sample to train the backbone network separately for a preset number of few samples, and synchronize the residual learned by backpropagation of the backbone network's few sample training to the lightweight YOLOv5 network's backbone network with a lower learning rate.
[0065] S205, the YOLOv3 detector head is trained a preset number of times using the second sample alone, and the residuals learned by backpropagation of the YOLOv3 detector head's few-sample training are synchronized to the lightweight YOLOv5 network's YOLOv3 detector head at a lower learning rate.
[0066] Specifically, in this embodiment, the training cycle for the combined training is 295 cycles. Since most fasteners in normally operating rails are in normal condition in the actual collected rail depth maps, with only a very small number in abnormal condition, after completing 295 cycles of combined training, rail depth maps of fastener targets with an aspect ratio exceeding 2 are used separately for training with few-sample data. Here, the few-sample data refers to fasteners categorized as abnormal or occluded.
[0067] The few-shot training specifically involves: selecting sample images from the sample images whose aspect ratio of the fastener region exceeds 2 as the second sample; using the second sample to perform few-shot training on the backbone network for 5 training epochs; and synchronizing the residuals learned by backpropagation of the few-shot training of the backbone network to the lightweight YOLOv5 network backbone network at a lower learning rate; and using the second sample to perform few-shot training on the YOLOv3 detection head for 5 training epochs; and synchronizing the residuals learned by backpropagation of the few-shot training of the YOLOv3 detection head to the lightweight YOLOv5 network YOLOv3 detection head at a lower learning rate.
[0068] S102, the fastener detection model responds to the input rail depth map, locates the position of the fastener in the rail depth map, and generates a prediction box image with the fastener and basic information of the prediction box image. The basic information of the prediction box image includes the position of the fastener in the prediction box image, preliminary classification, and rail image acquisition time.
[0069] The location of the fastener is its actual coordinates on the rail, which can be preliminarily classified as normal, loose, or obstructed.
[0070] This application proposes a real-time detection network for rail fasteners based on YOLOv5. It improves the recognition capability for few samples by introducing a read-only few-sample mining network. Part of the optimization is achieved by improving the model feeding data method to make the YOLOv5 network lightweight, thereby optimizing the real-time positioning of rail fasteners by the system.
[0071] Due to the real-time requirements of rail fastener detection, this application proposes using a lightweight YOLOv5 network for real-time fastener localization. To address the long-tail problem of data, a few-shot training module is added. For rail fastener states that are easily missed due to the limited number of images during training, these fastener images are used as input to the few-shot training module during offline fastener detection training. The few-shot training module synchronously shares the learned residual information with the regular YOLOv5 network and updates the weights. In the subsequent online detection stage, only the regular network runs; therefore, adding the few-shot training module only improves the system's ability to identify abnormal fasteners without affecting the testing speed of the network model.
[0072] S103, input the predicted bounding box image and its basic information into the pre-established fastener abnormality recognition model.
[0073] Among them, see Figure 3 This is a schematic diagram of the training process of the fastener abnormal state recognition model provided in the embodiment of this application. The pre-established fastener abnormal state recognition model establishment process is as follows: S301 to S304.
[0074] S301, the predicted bounding box image output by the fastener detection model and the basic information of the predicted bounding box image are input into the secondary recognition network. The secondary recognition network includes an imbalanced learning branch based on a reweighted loss function and a data augmentation branch based on an improved Mixup algorithm.
[0075] It should be noted that the fastener abnormal state recognition model mainly addresses the problem of high similarity in fastener states. It performs secondary recognition on fastener states with long-tail distribution. This secondary recognition network is divided into an imbalanced learning branch based on a reweighted loss function and a data augmentation branch based on an improved Mixup algorithm. The loss value is obtained by dynamically weighting the two.
[0076] S302, the preliminary category of the fastener in the prediction box image is interpolated through the data augmentation branch, and the interpolated category is shifted towards the fastener abnormal state category.
[0077] The data-augmented learning channel improves the Mixup algorithm by shifting the class interpolation towards the minority class during the data input phase, thus allocating a larger share (λ) of small sample labels to the mixed samples. By reasonably controlling the degree of shift towards the minority class, a reasonable allocation of the decision boundary is achieved. Here, λ follows a Beta distribution, and λ...y The shift is achieved by the following formula when the total number of samples in the states to which the two samples to be mixed differs by a factor of 3.
[0078] λ~Beta(α,α);
[0079]
[0080] Where α = 1 - T / T max T is the current training cycle number. max The total training cycles (including combined training and few-shot training) are 300, n i Let n be the total number of samples in category i. j Let represent the total number of samples in category j, i and j represent the categories of two randomly selected samples from the mixed sample, and x and y represent the image data and labels of the mixed sample, respectively.
[0081] S303, the initial category of the fastener in the prediction box image is shifted to the abnormal state category of the fastener through an imbalanced learning branch.
[0082] The imbalanced learning branch uses reweighting techniques to improve the network's preference for normal fasteners. The concept of effective sample number is used to reweight the data. Effective sample number is defined as the sample volume, which can be calculated using a simple formula, where n is the number of samples, C is the total number of all classes, and β∈[0,1) is a hyperparameter. The effective sample number E of class i is... i To simplify the calculation, β = (n-1) / n. This paper uses the Focalloss method, and the loss function for the imbalanced learning branch is:
[0083]
[0084] Where y is the actual label of the sample, γ is the modulation coefficient, n is the number of samples, and p i To predict the probability that a sample belongs to category i, C is the total number of all categories, β∈[0,1) is a hyperparameter, and E is the number of valid samples of category i. i .
[0085] S304, dynamically weight the imbalanced learning branch and the data augmentation branch to obtain the loss value, and backpropagate the loss value to update the parameters of the secondary recognition network in a synchronous manner to obtain the fastener abnormal state recognition model.
[0086] Wherein, the loss value is Loss = αL imb +(1-α)L mu L imb L is the loss function for the imbalanced learning branch. mu It is the loss function for the data augmentation branch.
[0087] S104, the fastener abnormal state recognition model responds to the prediction box image, performs secondary classification on the fasteners in the prediction box image, and obtains the final category of the fastener.
[0088] In this embodiment, due to the strong similarity between fasteners, after obtaining the detection box (predicted box image) of the rail fastener, the objects within the box are classified a second time. To address the long-tail phenomenon in the distribution of rail fasteners, a long-tail recognition algorithm for fastener images based on bi-branch learning is proposed. Reweighting techniques are used to improve the bias of the deep learning network towards normal fastener states, and to delineate reasonable decision boundaries for abnormal fastener states that require more monitoring and localization.
[0089] In practical applications, the deep learning-based method for identifying abnormal rail fasteners further includes: storing the rail depth map and the corresponding acquisition location on a server via UDP (User Datagram Protocol); and sending the final category of the fastener, the corresponding location of the fastener, and the rail depth map back to the server using UDP communication; the final category includes normal and abnormal states, and if the final category of the fastener is an abnormal state, an alarm signal is generated and issued, and the abnormal state includes at least fastener loosening or obstruction.
[0090] The deep learning-based method for identifying abnormal rail fasteners proposed in this application achieves image acquisition, fastener detection, status recognition, and alarm feedback before sampling the next image. The detection speed reaches 83 FPS, and the fastener status recognition accuracy is 95.6%.
[0091] In summary, the rail fastener anomaly identification method based on deep learning provided in this application embodiment is based on the following real-world scenarios: (1) The similarity between fastener states is relatively strong, and the appearance of fasteners such as fastener tilting when they are loose is difficult for computers to distinguish. (2) When rail data is collected, there are cases where only a small part of the fastener is collected at the edge of the image. Such incomplete fasteners pose a great challenge to annotation and computer vision-based fastener localization and state recognition. (3) In normally operating rails, most fasteners are in normal state, and only a very small number of fasteners are in abnormal state. At this time, deep learning will tend to extract features based on the majority class (normal fasteners) and will generate a large number of misjudged minority class samples. (4) Rail fastener anomaly identification requires extremely high feedback speed. From the sampling of rail data by the industrial camera suspended under the inspection vehicle to the feedback of the fastener state to the server, a very short time is required to ensure the real-time performance of the system.
[0092] To address the aforementioned challenges, this application embodiment acquires a track depth map and its corresponding acquisition location, and generates a prediction frame image containing fasteners and basic information of the prediction frame image using a pre-established fastener detection model. Then, a fastener abnormality identification model performs secondary classification on the fasteners in the prediction frame image to obtain the final category of the fasteners.
[0093] Corresponding to the aforementioned embodiment of a deep learning-based method for identifying rail fastener anomalies, this application also provides an embodiment of a deep learning-based system for identifying rail fastener anomalies. The deep learning-based system for identifying rail fastener anomalies is used to execute the aforementioned deep learning-based method for identifying rail fastener anomalies. The deep learning-based system for identifying rail fastener anomalies includes:
[0094] The image acquisition module is used to acquire depth maps of the railway tracks.
[0095] The fastener positioning and detection module is used to determine the acquisition location of the rail depth map.
[0096] The fastener status recognition module stores a fastener detection model and a fastener abnormal status recognition model.
[0097] The fastener detection model is used to input the rail depth map, locate the fastener position in the rail depth map, and generate a prediction box image with the fastener and the basic information of the prediction box image.
[0098] The fastener abnormality recognition model is used to perform secondary classification of the fasteners in the prediction box image based on the prediction box image to obtain the final category of the fastener.
[0099] Accordingly, embodiments of this application also provide a computer device, including:
[0100] Memory is used to store computer programs.
[0101] A processor is used to execute the computer program to implement the steps of the deep learning-based rail fastener anomaly identification method described above.
[0102] Accordingly, this application also provides a computer-readable storage medium storing a computer program, which, when processed and executed, implements the steps of the above-described deep learning-based method for identifying abnormal rail fasteners.
[0103] The above specific embodiments further illustrate the purpose, technical solution and beneficial effects of this application. It should be understood that the above are only specific embodiments of this application and are not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solution of this application should be included within the scope of protection of this application.
Claims
1. A method for identifying anomalies in railway rail fasteners based on deep learning, characterized in that, include: Obtain the track depth map and the corresponding acquisition location, and input it into the pre-established fastener detection model; The fastener detection model responds to the input rail depth map, locates the fastener in the rail depth map, and generates a prediction box image with the fastener and basic information of the prediction box image. The basic information of the prediction box image includes the location of the fastener in the prediction box image, preliminary classification, and rail image acquisition time. The predicted bounding box image and its basic information are input into a pre-established fastener abnormality recognition model. The fastener abnormal state recognition model responds to the prediction box image by performing secondary classification on the fasteners in the prediction box image to obtain the final category of the fasteners. The pre-established fastener abnormality identification model is established through the following steps: The predicted bounding box image output by the fastener detection model and the basic information of the predicted bounding box image are input into the secondary recognition network. The secondary recognition network includes an imbalanced learning branch based on a reweighted loss function and a data augmentation branch based on an improved Mixup algorithm. The data augmentation branch interpolates the initial category of fasteners in the prediction box image and shifts the interpolated category toward the fastener abnormal state category. The initial category of the fastener in the prediction box image is shifted to the abnormal state category of the fastener by an imbalanced learning branch. The imbalanced learning branch and the data augmentation branch are dynamically weighted to obtain the loss value, and the loss value is backpropagated to update the parameters of the secondary recognition network in a synchronous manner to obtain the fastener abnormal state recognition model. The loss function of the imbalanced learning branch is: ； in, For the actual labels of the samples, The modulation coefficient, For the sample size, To predict which sample belongs The probability of , where C is the total number of all categories, For hyperparameters, Fastener category Valid sample size ; The loss value is: ,in, It is the loss function of the imbalanced learning branch. It is the loss function for the data augmentation branch.
2. The method for identifying railway track fastener anomalies based on deep learning according to claim 1, characterized in that, The pre-established fastener detection model is established through the following steps: Obtain a rail depth map, mark the location and type of fasteners in the rail depth map, and preprocess the rail depth map; The preprocessed track depth map is randomly scaled, randomly stitched, color gamut converted, and white noise is added before being stitched together to form a sample image. The sample images are input into a lightweight YOLOv5 network to complete a preset period of combined training, which is divided into backbone network training and YOLOv3 detector head training. Select sample images in the sample images whose aspect ratio of the fastener area exceeds a preset threshold as second samples. Use the second samples to train the backbone network separately for a preset number of few samples. Then, synchronize the residuals learned by backpropagation of the backbone network's few sample training to the lightweight YOLOv5 network's backbone network with a lower learning rate. The YOLOv3 detector head is trained a preset number of times using the second sample alone, and the residuals learned by backpropagation of the YOLOv3 detector head's few-sample training are synchronized to the lightweight YOLOv5 network's YOLOv3 detector head with a lower learning rate.
3. The method for identifying railway rail fastener anomalies based on deep learning according to claim 1, characterized in that, The specific steps for obtaining the rail depth map and corresponding acquisition location, and inputting it into the pre-established fastener detection model are as follows: Obtain the track depth map and the corresponding data collection location; The rail depth map is preprocessed, and the preprocessed rail depth map is randomly scaled, randomly stitched, color gamut converted, and white noise is added before stitching to obtain the image to be detected. The image to be detected and the corresponding acquisition position are input into the pre-established fastener detection model.
4. The method for identifying railway rail fastener anomalies based on deep learning according to claim 1, characterized in that, Also includes: The track depth map and its corresponding acquisition location are stored on the server via UDP; In addition, UDP communication is used to send the final category of the fastener, the location of the corresponding fastener, and the rail depth map back to the server; If the final category of the fastener is an abnormal state, an alarm signal is generated and issued. The abnormal state includes at least a loose or obstructed fastener.
5. A deep learning-based system for identifying anomalies in railway rail fasteners, characterized in that, The deep learning-based rail fastener anomaly identification system is used to execute the deep learning-based rail fastener anomaly identification method according to any one of claims 1-4. The deep learning-based railway fastener anomaly identification system includes: The image acquisition module is used to acquire depth maps of the railway tracks. The fastener positioning and detection module is used to determine the acquisition location of the rail depth map; Fastener status recognition module, wherein the fastener status recognition module stores a fastener detection model and a fastener abnormal status recognition model; The fastener detection model is used to locate the fastener in the rail depth map based on the input rail depth map and generate a prediction box image with the fastener and the basic information of the prediction box image. The fastener abnormal state recognition model is used to perform secondary classification of the fasteners in the prediction box image based on the prediction box image to obtain the final category of the fastener. The fastener abnormality identification model is established through the following steps: The predicted bounding box image output by the fastener detection model and the basic information of the predicted bounding box image are input into the secondary recognition network. The secondary recognition network includes an imbalanced learning branch based on a reweighted loss function and a data augmentation branch based on an improved Mixup algorithm. The data augmentation branch interpolates the initial category of fasteners in the prediction box image and shifts the interpolated category toward the fastener abnormal state category. The initial category of the fastener in the prediction box image is shifted to the abnormal state category of the fastener by an imbalanced learning branch. The imbalanced learning branch and the data augmentation branch are dynamically weighted to obtain the loss value, and the loss value is backpropagated to update the parameters of the secondary recognition network in a synchronous manner to obtain the fastener abnormal state recognition model. The loss function of the imbalanced learning branch is: ； in, For the actual labels of the samples, The modulation coefficient, For the sample size, To predict which sample belongs The probability of , where C is the total number of all categories, For hyperparameters, Fastener category Valid sample size ; The loss value is: ,in, It is the loss function of the imbalanced learning branch. It is the loss function for the data augmentation branch.
6. A computer device, characterized in that, include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the steps of the deep learning-based method for identifying abnormal rail fasteners as described in any one of claims 1-4.
7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when processed and executed, implements the steps of the deep learning-based method for identifying abnormal rail fasteners as described in any one of claims 1-4.