System and method for detecting and classifying rail defects by using artificial intelligence

Abstract

The invention provides a system and method for detecting and classifying rail defects by using artificial intelligence, and the system comprises a receiving module which is used for inputting one or more rail images; the preprocessing module is used for preprocessing the rail image; the Transform module is used for carrying out defect detection and false alarm analysis on the preprocessed image by adopting a two-stage Transform model; and the output module is used for outputting and displaying a result containing defect, misinformation and unknown information. The method has the advantages that the method is easy to implement, the false alarm rate of an existing railway track detection system can be reduced, and the defect detection rate is increased. The method is easy to implement, and the risk of accidents or injuries in the defect detection process is reduced. And the service life of the track is prolonged, and the requirement for large-scale maintenance or replacement is reduced to the maximum extent.

Description

Technical Field

[0001] The invention belongs to the field of computer technology and relates to a system and method for detecting and classifying rail defects using artificial intelligence. Background Art

[0002] Rail defect detection usually relies on specialized inspection systems, and the onboard track inspection system (ORIS) is one such system. ORIS can identify track defects while the train is running and is an important tool for ensuring the safety and maintenance of the railway network. However, a significant obstacle encountered by the system is that it is prone to a large number of false alarms. This problem is particularly evident when trying to distinguish between normal, well-maintained track images and defective track images. The high false alarm rate poses a challenge to the efficiency and accuracy of the system. Railway inspection is one of the important tasks to ensure railway safety. If defects can be detected at an early stage, track maintenance or replacement can be carried out in time, and catastrophic failures such as track derailment and rail breakage can be prevented. The ideal defect detection system should be efficient, non-destructive, robust and reliable. Axle box acceleration (ABA) measurement is a common method to identify railway anomalies.

[0003] There are two methods for rail defect detection: non-destructive and destructive. Non-destructive methods involve inspecting, testing or evaluating discontinuities or differences in properties of materials, components or assemblies without destroying the availability of the component or system. Destructive methods, on the other hand, are not generally used for regular maintenance or inspection of tracks because they can cause damage. However, destructive methods can be used in a controlled environment for research, material testing or post-accident analysis of faulty components. In addition, destructive and non-visual inspection techniques, including acoustic, ultrasonic, eddy current and flux leakage testing, are also used to identify internal anomalies such as voids and inclusions.

[0004] Previously, simulated ABA signals and wheel-rail interaction models were analyzed. The signals were processed by short-time Fourier transform, continuous wavelet transform, empirical mode decomposition, and power spectral density to estimate the likelihood and severity of different rail surface defects. By utilizing algorithms of windowed Fourier Transformer and wavelet Transformer, different track defects can be detected from data collected by instrumented wagons. Another common measurement method is the air-coupled ultrasonic rail inspection system. However, the high acoustic impedance mismatch between air and steel severely affects the inspection performance. The industry needs to establish an efficient and non-destructive system and method, and pre-train the model to obtain robust and reliable inspection performance.

[0005] On the other hand, a 2D model is created to predict the performance of the system, which works at different defect sizes at different locations. 2D laser displacement sensors are used to monitor rail corrugation. However, pitch vibrations in the measurement environment may cause measurement errors in the laser sensor. The model takes the deviation angle and offset into account and uses laser ultrasonic technology (LUT) to calibrate the dataset under pitch vibration. LUT can be used for matching pursuit (MP) to develop a non-contact inspection system for rails. It can sense Rayleigh waves generated by laser excitation and detect defects at different locations such as subsurface horizontal plane, subsurface vertical plane, surface horizontal plane, and surface edge. In addition, electromagnetic tomography technology can also facilitate rail crack detection. Back-projection algorithms can be applied to electromagnetic tomography to classify rail cracks. However, the above-mentioned technologies require expensive equipment and only experts can control them competently.

[0006] An effective method to classify or detect rail surface defects is to use computer vision. Computer vision methods, such as those using convolutional neural networks (CNNs) or traditional image processing, are good at finding surface irregularities such as cracks, dents, and ripples due to their obvious visual features. For example, image data is collected by a real-time detection system such as a line scan camera on the rail surface. This method is less time-consuming and more efficient. Traditional machine learning algorithms, such as SVM (Support Vector Machine) and Random Forest, have been applied to identify railway anomalies using computer vision. Previous studies have successfully demonstrated that the use of pre-trained SVMs can identify the differences between sample tie rod plates and reference tie rod plates in a sliding window of machine vision technology. In addition, images can be collected by unmanned aerial vehicles (UAVs) and random forest classification can be performed to detect rail cracks. In this work, the images are noisy and need to be denoised.

[0007] Although the denoising method using wavelet transform and median filtering (WTCMF) can be used for image preprocessing of rail surface defect detection, the solution is to enhance edge detection capability and mitigate the impact of noise. There are two major obstacles in the visual inspection system for rail surface inspection. Images are usually taken under ambient lighting, and the shapes and sizes of defects vary.

[0008] The Coarse-to-Fine Model (CTFM) takes inputs at three different levels: sub-image, region, and pixel. It filters defect-free areas at the region level, locates defective areas using pure phase Fourier transform at the sub-image level, and extracts the shape of defects at the pixel level. The use of line scan cameras can be combined with the Spectral Image Difference Procedure (SIDP) to identify rail cracks and defects on irregular textures.

[0009] Traditional computer vision methods can detect defects and classify defect categories of track images. For example, morphological operation methods involving the use of morphological feature extraction and image processing, combined with Hough transform to detect rail defects have been disclosed. Specific defects such as head inspection breaks, apletiliks, and undulations can be identified. Another disclosed technology is a real-time rail system that uses the H value of a color image for rapid preprocessing and uses morphological processing to eliminate redundancy. It uses the contours of the directional chain code to quickly identify the shape of defects in the system, which can achieve real-time detection. However, traditional image processing methods have a high false alarm rate and require a lot of domain knowledge.

[0010] Among convolutional neural network methods (CNN-based methods), YOLOX involves using the results of various defects that affect train operation and safety. Current defect detection models struggle in the case of poor lighting and insufficient data. Previous literature proposed an enhanced YOLOX model combined with image processing techniques to efficiently highlight and detect railway defects. RetinaNet proposed a deep learning (DTL) framework to detect cracks on the track surface using a limited number of training images. This study used the pre-trained YOLOv3 and RetinaNet models from the COCO dataset, applied these transfer models to rail images, and then adopted an ensemble scheme to enhance the detection effect. When benchmarked with several models and traditional detection methods using a dataset of 102 railway images, the DTL model showed higher recall and precision. YOLOv3 performed well in detecting smaller cracks, while RetinaNet was suitable for detecting larger cracks.

[0011] The technology with Chinese patent document number CN112649513A describes an application of neural networks in defect detection, which provides a railway artificial intelligence defect judgment method based on image recognition. The technology with US patent document number US20230290135A1 may have disclosed a system for robustly representing and generating images, wherein input labels of input images are received and reasoning about the input images is generated based on a visual Transformer (ViT) system, the system including at least one self-attention module for performing label mixing and a channel self-attention module for performing channel processing. However, neither the artificial intelligence defect judgment method based on image recognition nor the self-attention module provides a practical solution to the long-standing problems of high false alarm rates and high computational requirements.

[0012] In addition, Chinese patent documents CN114973199A and CN115661726A disclose a method for detecting obstacles in rail transit trains based on convolutional neural networks and an autonomous video acquisition and analysis method for assembly of rail transit workpieces. However, both technologies lack effective classification modules and cannot accurately classify defects to reduce the false alarm rate.

[0013] False alarms, or incorrect defect identifications, can lead to a variety of problems. They can trigger unnecessary maintenance activities, causing disruptions to the rail network and incurring additional costs. In addition, too many false alarms can overload the task of human operators monitoring the system, which can reduce trust in the system's alarms and can cause legitimate defects to go unnoticed. Therefore, innovative systems and methods are urgently needed to effectively address the false alarm problem in track defect detection. Summary of the invention

[0014] The purpose of the present invention is to overcome the above-mentioned shortcomings and provide a system and method for detecting and classifying rail defects using artificial intelligence, which can solve various problems caused by false alarms or erroneous defect identification in the prior art.

[0015] In order to achieve the above-mentioned purpose, the technical solution adopted by the present invention is: a system for detecting and classifying rail defects using artificial intelligence, characterized in that it includes:

[0016] A receiving module, used for inputting one or more rail images;

[0017] A preprocessing module, used for preprocessing the rail image;

[0018] Transformer module, which is used to perform defect detection and false alarm analysis on preprocessed images using a two-stage Transformer model;

[0019] The output module is used to output the results containing defects, false positives, and unknown information.

[0020] Furthermore, the two-stage Transformer model includes: a first-stage Transformer model and a second-stage Transformer model using a shift window arranged in sequence.

[0021] Furthermore, the two-stage Transformer model has a predetermined confidence threshold, whose value is 0.7.

[0022] Furthermore, the first-level Transformer model uses a pre-trained model on ImageNet.

[0023] Furthermore, the second-level Transformer model has a classification head modified from the first-level Transformer model to output at least one category of defects.

[0024] Further, the at least one category of defects includes: wrong image, wear mark, joint, mark, squat pit, weld, word, head inspection error, missing fastener, rail head abnormality, corrugation.

[0025] Further, the system is adapted to be implemented into existing detection systems, including vehicle-mounted track detection systems.

[0026] Further, the receiving module inputs one or more rail images from an image file, a database with multiple false alarm images, or an image from a sensor camera.

[0027] Another object of the present invention is to provide a method for detecting and classifying rail defects using artificial intelligence, characterized in that it includes:

[0028] Input one or more rail images;

[0029] Preprocessing the rail image;

[0030] A two-stage Transformer model is used to perform defect detection and false alarm analysis on preprocessed images;

[0031] The output shows the results including defects, false positives, and unknown information.

[0032] Furthermore, the inputting one or more rail images includes: acquiring rail defect images from a database, an image file or an image from a sensor camera.

[0033] Furthermore, the rail image is preprocessed, including denoising the rail image.

[0034] Furthermore, a two-stage Transformer model is used to perform defect detection and false alarm analysis on the preprocessed images, including:

[0035] Use the first-level Transformer model to extract feature maps from the input image;

[0036] Apply a mathematical function to the classification head of the first-level Transformer model to predict defects or false positives in each feature map and each image;

[0037] Score each predicted image;

[0038] High-score images with confidence scores exceeding a predetermined threshold are screened out from low-score images with confidence scores below a predetermined threshold.

[0039] Furthermore, a two-stage Transformer model is used to perform defect detection and false alarm analysis on the preprocessed images, including:

[0040] Pass the high-score image in the first-level Transformer model to the second-level Transformer model;

[0041] Extract feature maps from high-resolution images;

[0042] Apply a mathematical function to the classification head of the second-level Transformer model to classify specific defect categories;

[0043] Label images as specific defect classes.

[0044] Furthermore, a two-stage Transformer model is used to perform defect detection and false alarm analysis on the preprocessed images, including:

[0045] Pass the low-score image from the first-level Transformer model to the second-level Transformer model;

[0046] Extract feature maps from low-resolution images;

[0047] Apply a mathematical function to the classification head of the second-level Transformer model to classify the image;

[0048] Label images as "unknown" or "false positive" based on the confidence score.

[0049] Further, the low-scoring images in the first-stage Transformer model are passed to the second-stage Transformer model, including: generating an improved false positive recall rate.

[0050] Further, marking the image as a specific defect category includes: selecting, by a classification head, the specific defect category from a set of predetermined defect categories.

[0051] Further, selecting high-score images with confidence scores exceeding a predetermined threshold from low-score images with confidence scores below a predetermined threshold includes:

[0052] Select the best confidence score as the predetermined threshold;

[0053] initiating optimal threshold selection in the event that a predetermined threshold is selected;

[0054] Images with confidence scores exceeding a predetermined threshold are separated, wherein images with confidence scores exceeding the predetermined threshold are regarded as high-score images, and vice versa.

[0055] Further, outputting the results including defects, false positives, and unknown information includes the following steps:

[0056] Determine whether the image consists of defects, false positives, or unknowns;

[0057] The analysis results are displayed based on the two-stage Transformer model.

[0058] Further, determining whether the image consists of defects, false positives, or unknowns includes: determining according to a classification head of a second-level Transformer model.

[0059] Further, selecting the best confidence score as the predetermined threshold includes: selecting from a range of 0.5 to 0.9 with a step size of 0.05.

[0060] Further, applying a mathematical function to a classification head of the first-level Transformer model to predict defects or false positives in each feature map and each image, including: training the first-level Transformer model using training images containing defects and false positives.

[0061] Compared with the prior art, the present invention has the following advantages:

[0062] First, the false alarm rate of the existing railway track detection system can be reduced and the defect detection rate can be improved. The Swin Transformer used in the present invention has a self-attention mechanism and a hierarchical structure, which can more accurately detect defective rail images, thereby reducing the false alarm rate. In addition, the cascade structure of the first Swin Transformer has an optimal threshold to further improve the prediction, while the second Swin Transformer classifies defects for specific defect categories.

[0063] Second, non-destructive techniques are used to detect defects on the rails. This technique has the advantage of being less expensive than destructive methods because it does not require extensive excavation or replacement of rails. The invention is also easier to implement because it does not require the removal or replacement of rails. This allows defects to be detected more quickly, reducing downtime for repairs.

[0064] Thirdly, it also causes minimal disruption to train services as it does not require track closures or diversions. This ensures smoother operations and avoids inconvenience to passengers. It is also relatively safer for workers as it eliminates physical contact with the rails. This reduces the risk of accidents or injuries during defect detection.

[0065] Fourth, the track can be continuously monitored, allowing defects to be detected in real time or at regular intervals. This proactive approach helps prevent major failures or accidents and ensures timely maintenance and repairs. It also maintains the integrity of the track because it does not require the removal or replacement of any components. This helps extend the service life of the track and minimizes the need for large-scale repairs or replacements.

[0066] Fifth, it is more environmentally friendly because it does not generate excess waste and does not require the disposal of materials on the track. This reduces the carbon footprint associated with defect detection and maintenance activities. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] The features of the present invention will be more readily understood and appreciated when the following detailed description is read in conjunction with the accompanying drawings of preferred embodiments of the present invention, in which:

[0068] Figure 1 A flow chart of the present invention is shown;

[0069] Figure 2 Four embodiments of the defect classification of the present invention are shown;

[0070] Figure 3 A flow chart of an embodiment of a two-stage Transformer model with confidence threshold selection of the present invention is shown. DETAILED DESCRIPTION

[0071] As required, specific embodiments of the present invention are disclosed herein. However, it should be understood that the disclosed embodiments are only examples of the present invention, which can be implemented in a variety of different forms. Therefore, the specific structural and functional details disclosed herein should not be interpreted as limitations, but only as the basis of the claims. It should be understood that the drawings and their detailed description are not used to limit the present invention to the specific forms disclosed herein. On the contrary, the present invention covers all modifications, equivalents and alternatives that fall within the scope defined by the claims. As used throughout this application, the word "may" means optional (i.e., it means possible), rather than mandatory (i.e., it means must). Similarly, the words "include" and "comprise" are meant to include but are not limited to. In addition, unless otherwise mentioned, the word "one" means "at least one" and the word "multiple" means one or more. When using abbreviations or technical terms, these refer to the generally accepted meanings known in the art.

[0072] Traditional image processing techniques often suffer from high false alarm rates due to their sensitivity to variation and lack of adaptability. First, they are very sensitive to various factors such as lighting conditions, shadows, noise, and environmental changes. These variables can severely affect the accuracy and reliability of image processing results, making it very difficult to maintain a low false alarm rate. Second, traditional methods lack adaptability. Unlike deep learning methods, which can be trained on a wide range of data sets to learn and adapt to specific scenarios, traditional techniques cannot evolve or improve their performance without human intervention when faced with new data or new defect types, which limits their usefulness in dynamic real-world applications.

[0073] To address these deficiencies, modern computer vision methods, especially those based on deep learning, have become increasingly popular. These methods can automatically learn and adapt to new data, making them more robust in dealing with variability and reducing false positives. By using neural networks and large-scale datasets, these methods can extract relevant features and patterns from images, making them more general and more adaptable to changing conditions. In addition, deep learning methods can continuously improve their performance over time, making them a valuable tool in a variety of image processing applications, including object recognition, defect detection, and image analysis.

[0074] More specifically, the present invention provides a system for detecting and classifying rail defects using artificial intelligence, the system comprising: a receiving module configured to receive image input; a preprocessing module configured to preprocess the image input; a two-stage Transformer model configured to detect defects or false alarms based on the image input; and an output module for displaying results containing defects, false alarms, or unknown information.

[0075] Due to the recent progress in the field of computer vision, it is believed that the Transformer model is often used because of its self-attention mechanism. Compared with traditional convolutional neural networks, it has better feature extraction capabilities, especially in defect detection tasks. Among various Transformer models, Swin Transformer stands out, which refers to Shifted windowTransformer, a multi-level visual transformation model using shifted windows. Swin Transformer is different from the standard Transformer structure in that it adds a self-attention window. This new feature not only reduces the computational requirements, but also improves the model performance through its layered architecture and shifting window attention mechanism. Taking advantage of its ability to effectively extract features and accurately detect defective track images, the present invention adopts Swin Transformer because of its flexible backbone and architecture.

[0076] In one embodiment of the present invention, a two-stage Transformer model is used, wherein the two-stage Transformer model is at least two Swin Transformer models, and the model further includes a first Swin Transformer model and a second Swin Transformer model arranged in sequence. These cascaded Swin Transformers have the best confidence selection, wherein the cascaded Swin Transformer design is improved to integrate two sequential Swin Transformers.

[0077] In another embodiment of the present invention, the two-stage Transformer model has a predetermined confidence threshold whose value is 0.7.

[0078] In another embodiment of the present invention, the first Swin Transformer model uses a pre-trained model on ImageNet.

[0079] In another embodiment of the present invention, the second Swin Transformer model has a classification head modified from the first Swin Transformer model to output at least one category of defects. The classification head is modified to include twelve categories of defects, which will be further disclosed below.

[0080] In another embodiment of the present invention, at least one category of defects includes but is not limited to errors, images, wear marks, joints, marks, squats, welds, words, head inspection errors, missing fasteners, rail head anomalies, corrugations, or others.

[0081] In addition, the first Swin Transformer model is a binary classifier and the second Swin Transformer model is a multi-classifier. For simplicity, the classifier head of the Swin Transformer model is modified to include twelve categories of defects. Generally, a binary classifier is used to predict the input image. If the prediction confidence is greater than 0.7, a multi-class classifier is used to predict the specific class because these images are not hard samples for the model. If the prediction confidence is not greater than 0.7, a binary classifier is further used to predict the binary class labels of these hard samples. The confidence is selected in the range of 0.5 to 0.9, where 0.7 is the optimal threshold based on false positive recall and precision. Figure 2 The specific image classes for which the defect normal class was determined by the second Swin Transformer model / multi-class classifier are shown.

[0082] In another embodiment of the present invention, the system can be implemented into an existing inspection system, including but not limited to an onboard rail inspection system (ORIS).

[0083] In another embodiment of the present invention, the image input is obtained from an image file, a database with multiple false positive images, or an image from a sensor camera.

[0084] More specifically, the two-stage Swin Transformer in the present invention undertakes different tasks. For example, in the first stage, the first Swin Transformer model is tasked with predicting false alarm and defective track images. To improve these predictions, an optimal threshold selection is applied to filter out low-confidence false alarm and defective track images. Specific defect categories are detected in the second Swin Transformer model. The track image at the input of the second Swin Transformer model is the high-confidence false alarm image selected by the output of the first Swin Transformer model. This approach can achieve a higher defect detection rate and reduce the false alarm rate output by the detection system.

[0085] The present invention provides a method (100) for detecting and classifying rail defects using artificial intelligence, the method comprising: inputting one or more rail images (102); preprocessing the images (104); performing defect detection or false alarm analysis on the images using a two-stage Transformer model (106); and outputting results (108). Figure 1 A description of the method is further described.

[0086] In another embodiment of the present invention, the step of inputting one or more rail images (102) further comprises selecting defect images from a database, an image file, or an image from a sensor camera.

[0087] Another embodiment of the present invention is that the step of pre-processing the image (104) further includes performing a denoising process on the image with rail defects.

[0088] In another embodiment of the present invention, the step of performing defect detection or false alarm analysis on an image using a two-stage Transformer model (106) further includes: extracting a feature map from an input image using a first Transformer model; predicting defects or false alarms in each feature map and each image to a classification head of the first Transformer model; scoring each predicted image; and filtering out high-scoring images having confidence scores exceeding a predetermined threshold from low-scoring images having confidence scores below a predetermined threshold. Figure 3 The choice of confidence threshold is further explained.

[0089] In another embodiment of the present invention, the step of performing defect detection or false alarm analysis on an image using a two-stage Transformer model (106) further includes: transferring a high-score image in a first Transformer model to a second Transformer model; extracting a feature map from the high-score image; applying a mathematical function to a classification head of the second Transformer model to classify a specific defect category; and marking the image as a specific defect category.

[0090] In another embodiment of the present invention, the step of performing defect detection or error analysis on an image using a two-stage Transformer model (106) further includes: passing the low-score image in the first Transformer model to the second Transformer model; extracting a feature map from the low-score image; applying a mathematical function to the classification head of the second Transformer model to classify the image; and marking the image as "unknown" or "false positive" based on a confidence score.

[0091] In another embodiment of the present invention, the step of transferring the low-scoring image from the first Transformer model to the second Transformer model further includes generating an improved false positive recall rate.

[0092] In another embodiment of the present invention, the step of marking the image as a specific defect category further comprises selecting, by a classification head, the specific defect category from a set of predetermined defect categories.

[0093] In another embodiment of the present invention, the step of filtering high-score images with confidence scores exceeding a predetermined threshold from low-score images with confidence scores below a predetermined threshold further includes: selecting the best confidence score as the predetermined threshold; initiating the best threshold selection when the predetermined threshold is selected; and separating images with confidence scores exceeding the predetermined threshold, wherein images with confidence scores exceeding the predetermined threshold are regarded as high-score images, and vice versa.

[0094] In another embodiment of the present invention, the step of outputting the result (108) further includes: determining whether the image is composed of defects, false positives or unknowns; and displaying the result according to the analysis result of the two-stage Transformer model.

[0095] In another embodiment of the present invention, the step of determining whether the image consists of defects, false positives, or unknowns is determined by the classification head of the second Transformer model.

[0096] In another embodiment of the present invention, the step of selecting the best confidence score as the predetermined threshold is selected from the range of 0.5 to 0.9 with a step size of 0.05. This is a threshold selection technique that is applied after training the first SwinTransformer model. The high confidence of the predicted result and the low confidence of the predicted result are separated by the best confidence threshold.

[0097] In another embodiment of the present invention, the step of predicting defects or false positives in each feature map and each image by applying a mathematical function to the classification head of the first Transformer model further includes training the first Transformer model using training images containing defects and false positives.

[0098] The present invention has various advantages, more specifically, it can reduce the false alarm rate of existing railway track detection systems and improve the defect detection rate. The Swin Transformer used in the present invention has a self-attention mechanism and a hierarchical structure, which can more accurately detect defective rail images, thereby reducing the false alarm rate. In addition, the cascade structure of the first Swin Transformer has an optimal threshold to further improve the prediction, while the second Swin Transformer classifies defects for specific defect categories.

[0099] Another advantage of the present invention is that it uses a non-destructive technique to detect defects on rails. This technique has the advantage of being less expensive than destructive methods because it does not require extensive excavation or replacement of rails. The present invention is also easier to implement because it does not require the removal or replacement of rails. This allows defects to be detected more quickly, reducing downtime for repairs.

[0100] The invention also causes minimal disruption to train services as it does not require track closures or diversions. This ensures smoother operations and avoids inconvenience to passengers. It is also relatively safer for workers as it eliminates physical contact with the rails. This reduces the risk of accidents or injuries during defect detection.

[0101] Additionally, the invention allows for continuous monitoring of the track, allowing for the detection of defects in real time or at regular intervals. This proactive approach helps prevent major breakdowns or accidents and ensures timely maintenance and repairs. It also preserves the integrity of the track as it does not require the removal or replacement of any components. This helps extend the life of the track and minimizes the need for major repairs or replacements.

[0102] The invention is also more environmentally friendly as it does not generate excess waste and does not require the disposal of materials on the track. This reduces the carbon footprint associated with defect detection and maintenance activities.

[0103] Table 1 shows the results of the first-level Swin Transformer model. It shows the results of false positive recall and precision at different confidence levels. The column labeled “Number of Detections” indicates the number of predictions whose probability exceeds the confidence level. The column labeled “True Negatives” corresponds to the false positive class, while the column “True Positives” is related to the defect class.

[0104] Table 1

[0105]

[0106] Table 2 illustrates the comparison of our model with other popular models such as Visual Transformer, ResNet152, and Efficientnet. The comparison metric is top1 accuracy.

[0107] Table 2

[0108]

[0109] Table 3 illustrates the prediction results selected from the low confidence threshold.

[0110] Table 3

[0111]

[0112]

[0113] Table 4 illustrates the prediction results selected from the high confidence threshold. This is the multi-class classifier result for the high confidence prediction samples from the binary classifier. Accuracy refers to the ratio of the model predicting the correct multi-class label.

[0114] Table 4

[0115]

[0116] Table 5 shows the results of the cascade model with different confidence levels. This is the result of combining these two levels of Transformers. The results show that the cascade architecture improves the false positive recall rate by 3.7% with a precision loss of 0.7%.

[0117] Table 5

[0118] Confidence Number of tests True Negative True Positive False Negative False Positive precise Accuracy 0.5 901 783 79 18 21 0.949 0.977 0.6 866 771 65 15 15 0.934 0.980 0.7 831 760 54 10 7 0.921 0.987 0.8 799 739 50 6 4 0.895 0.991 0.9 786 732 47 3 4 0.887 0.995

[0119] The above explanation of the present invention is not limited to the aforementioned embodiments and drawings, and it is obvious to those skilled in the art that various substitutions, modifications and changes may be made without departing from the scope of the present invention.

Claims

1. A system for detecting and classifying rail defects using artificial intelligence, characterized in that: include: A receiving module, used for inputting one or more rail images; A preprocessing module, used for preprocessing the rail image; Transformer module, which is used to perform defect detection and false alarm analysis on preprocessed images using a two-stage Transformer model; The output module is used to output the results containing defects, false positives, and unknown information.

2. The system for detecting and classifying rail defects using artificial intelligence according to claim 1, characterized in that: The two-stage Transformer model includes: a first-stage Transformer model and a second-stage Transformer model using a shift window arranged in sequence.

3. The system for detecting and classifying rail defects using artificial intelligence according to claim 1, characterized in that: The two-stage Transformer model has a predetermined confidence threshold of 0.

7.

4. The system for detecting and classifying rail defects using artificial intelligence according to claim 2, characterized in that: The first-level Transformer model uses a pre-trained model on ImageNet.

5. The system for detecting and classifying rail defects using artificial intelligence according to claim 2, characterized in that: The second-level Transformer model has a classification head modified from the first-level Transformer model to output at least one category of defects.

6. The system for detecting and classifying rail defects using artificial intelligence according to claim 5, characterized in that: The at least one category of defects includes: wrong image, wear mark, joint, mark, squat, weld, word, head inspection error, missing fastener, rail head abnormality, corrugation.

7. The system for detecting and classifying rail defects using artificial intelligence according to claim 1, characterized in that: The system is intended to be implemented into existing inspection systems, including vehicle-mounted track inspection systems.

8. The system for detecting and classifying rail defects using artificial intelligence according to claim 1, characterized in that: The receiving module inputs one or more rail images from an image file, a database with multiple false alarm images, or images from a sensor camera.

9. A method for detecting and classifying rail defects using artificial intelligence, characterized in that: include: Input one or more rail images; Preprocessing the rail image; A two-stage Transformer model is used to perform defect detection and false alarm analysis on preprocessed images; The output shows the results including defects, false positives, and unknown information.

10. The method for detecting and classifying rail defects using artificial intelligence according to claim 9, characterized in that: The inputting one or more rail images includes: acquiring rail defect images from a database, an image file or an image from a sensor camera.

11. The method for detecting and classifying rail defects using artificial intelligence according to claim 9, characterized in that: The rail image is preprocessed, including denoising the rail image.

12. The method for detecting and classifying rail defects using artificial intelligence according to claim 9, characterized in that: A two-stage Transformer model is used to perform defect detection and false alarm analysis on preprocessed images, including: Use the first-level Transformer model to extract feature maps from the input image; Apply a mathematical function to the classification head of the first-level Transformer model to predict defects or false positives in each feature map and each image; Score each predicted image; High-score images with confidence scores exceeding a predetermined threshold are screened out from low-score images with confidence scores below a predetermined threshold.

13. The method for detecting and classifying rail defects using artificial intelligence according to claim 12, characterized in that: A two-stage Transformer model is used to perform defect detection and false alarm analysis on preprocessed images, including: Pass the high-score image in the first-level Transformer model to the second-level Transformer model; Extract feature maps from high-resolution images; Apply a mathematical function to the classification head of the second-level Transformer model to classify specific defect categories; Label images as specific defect classes.

14. The method for detecting and classifying rail defects using artificial intelligence according to claim 12, characterized in that: A two-stage Transformer model is used to perform defect detection and false alarm analysis on preprocessed images, including: Pass the low-score image from the first-level Transformer model to the second-level Transformer model; Extract feature maps from low-resolution images; Apply a mathematical function to the classification head of the second-level Transformer model to classify the image; Label the image as "unknown" or "false positive" based on the confidence score.

15. The method for detecting and classifying rail defects using artificial intelligence according to claim 14, characterized in that: Passing low-scoring images from the first-level Transformer model to the second-level Transformer model, including: generating improved false positive recall.

16. The method for detecting and classifying rail defects using artificial intelligence according to claim 13, characterized in that: Marking the image as a specific defect category includes: selecting, by a classification head, the specific defect category from a set of predetermined defect categories.

17. The method for detecting and classifying rail defects using artificial intelligence according to claim 12, characterized in that: Screening out high-score images with confidence scores exceeding a predetermined threshold from low-score images with confidence scores below a predetermined threshold includes: Select the best confidence score as the predetermined threshold; initiating optimal threshold selection in the event that a predetermined threshold is selected; Images with confidence scores exceeding a predetermined threshold are separated, wherein images with confidence scores exceeding the predetermined threshold are regarded as high-score images, and vice versa.

18. The method for detecting and classifying rail defects using artificial intelligence according to claim 9, characterized in that: The output shows the results including defects, false positives, and unknown information, including the following steps: Determine whether the image consists of defects, false positives, or unknowns; The analysis results are displayed based on the two-stage Transformer model.

19. The method for detecting and classifying rail defects using artificial intelligence according to claim 18, characterized in that: Determine whether the image consists of defects, false positives, or unknowns, including: determining based on the classification head of the second-level Transformer model.

20. The method for detecting and classifying rail defects using artificial intelligence according to claim 17, characterized in that: The best confidence score is selected as the predetermined threshold, including: selecting from a range of 0.5 to 0.9 with a step size of 0.

05.

21. The method for detecting and classifying rail defects using artificial intelligence according to claim 12, characterized in that: Applying a mathematical function to a classification head of a first-level Transformer model to predict defects or false positives in each feature map and each image, including: training the first-level Transformer model using training images containing defects and false positives.

Images