Track crack identification method
The orbital crack recognition model is constructed through the PatchCore algorithm, and the downsampling feature library and pixel-level anomaly scores are used to identify orbital cracks, which solves the problems of sample imbalance and complex labeling, and realizes the efficient identification and cold start capability of orbital cracks.
Patent Information
- Application Number
- CN202411759072.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-03
- Publication Date
- 2025-03-28
AI Technical Summary
In the prior art, the orbital crack identification has problems such as extremely uneven sample samples, inability to achieve cold start and complex data sample labeling, and high cost.
The orbital crack recognition model is constructed using the PatchCore algorithm. By inputting the normal orbital sample image into the model for training, the features are extracted and downsampled, and saved to the feature library. During the test, the test image features and feature library are extracted for nearest neighbor search, the pixel-level abnormality score is calculated, and the abnormal pixel area is determined through threshold comparison.
It is realized that only a large number of normal samples are needed to identify cracks in the training stage, solving the problems of sample imbalance and complex labeling, and can quickly start cold in new scenarios, and reduce segmentation errors through downsampling and regional growth methods.
Smart Images
Figure CN119722591A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of track detection, and more specifically to a track crack identification method. Background Art
[0002] Traditional rail crack image recognition algorithms generally include three steps: image preprocessing, feature extraction, and image classification. Image preprocessing is mainly used to improve image quality, highlight target pixels, reduce noise interference, and further facilitate feature extraction; feature extraction is to use a feature vector or a specific value to quantify the description content; image classification is an image processing method that separates objects of different categories based on image features, such as Figure 2 shown.
[0003] In rail crack image recognition, image preprocessing mainly solves problems such as image blur, low contrast, and severe noise interference. For the image blur problem generated when the image acquisition device takes rail images, the commonly used methods include the traditional image deblurring algorithm based on the imaging model and the SRN-DeblurNet based on the convolutional neural network. Affected by problems such as illumination and low quality of the image acquisition camera, the image contrast is too low. Due to the low contrast, the image details are unclear and the grayscale level is not obvious, which is not conducive to feature extraction. For this reason, image enhancement, image sharpening, histogram equalization, grayscale transformation and other methods are commonly used to improve the image contrast. For the noise existing in the image, the commonly used methods include anisotropic diffusion filtering, edge recognition, binarization combined with morphological filtering, wavelet transform, image saliency analysis combined with multi-directional fitting region growth, etc. However, these methods are not ideal for processing images with complex textures, and small crack images are prone to feature loss during the denoising process.
[0004] Feature extraction and feature selection are important components of image recognition. The quality of feature descriptors directly determines the accuracy of classification. Commonly used image features include: color or grayscale statistical features, texture or edge features, algebraic features, and image transformation coefficient features. In complex and diverse bridge inspection environments, for track crack images with difficult background and foreground separation, texture features such as co-occurrence matrix, Tamura texture feature expression, Gabor transform, and wavelet transform are usually used to describe features. Some scholars have found that complex background images are not conducive to crack feature extraction. Therefore, after converting the crack image to the HSV color space, a threshold is set to distinguish the complex image, and the complex image is denoised, and then Gabor and wavelet transforms are used to extract features. Some people use regional multi-scale to extract grayscale, entropy, and texture feature information of cracks and their surroundings. The texture feature description algorithm has a high recognition accuracy, but it has problems such as weak illumination robustness, high requirements for image quality, interference from noise, and difficulty in image preprocessing. In addition to using texture feature description, geometric features are also the most commonly used feature descriptors for cracks due to their slender shape.
[0005] Most of the classification layers of image recognition can be represented by generalized linear models, which can be used to map input images to output categories. Commonly used classifiers include support vector machines and random forests. Support vector machines (SVM) use convolution kernels to "upscale" input features to high-dimensional space, turning them into new feature vectors, and then perform image recognition classification algorithms; random forests are classifiers that contain multiple decision trees, and are essentially an integrated learning. Some scholars have conducted comparative experiments on three classifiers: SVM, Adaboost, and Random Forest. The results show that when the feature descriptors are the same, using different classifiers will produce different classification effects.
[0006] In the prior art, the difficult problems in the track crack identification scenario are as follows:
[0007] (1) The samples are extremely unbalanced. Normal samples are very easy to obtain, but crack samples are very difficult to obtain. First, traditional deep neural network training requires a large number of labeled samples. However, in the track crack scenario, crack defects are not common, while there are many samples of normal rails, resulting in extremely unbalanced samples.
[0008] (2) Existing algorithms cannot achieve cold start in crack recognition in new scenarios. For crack recognition in different backgrounds, the common problem is that there are a lot of normal samples and very few crack samples, but the difference is that the background of crack recognition in different scenarios is different, which makes the model unable to generalize quickly to other fields, or it cannot be cold started in new crack recognition scenarios (that is, no model training is required).
[0009] (3) Data sample labeling is complex and costly. Crack labeling is generally done using semantic segmentation, but the different shapes of cracks make the labeling process very complicated, which not only places higher demands on labelers, but also increases the labeling cost linearly. Summary of the invention
[0010] In order to overcome the defects in the above-mentioned prior art, the present invention discloses a rail crack identification method to solve the problems existing in the prior art, such as the difficulty in obtaining the above-mentioned crack samples, the inability to implement cold start in crack identification in new scenarios, and the high cost of data sample annotation. The present invention draws on the algorithm idea of industrial anomaly detection PatchCore to solve the problem.
[0011] In order to achieve the above objectives, the technical solution adopted by the present invention is:
[0012] A rail crack identification method comprises the following steps:
[0013] S1. Construct rail crack identification model;
[0014] S2, inputting the normal track sample image into the track crack recognition model for model training, extracting normal sample features from the normal track sample image during model training, and saving the extracted normal sample features into a feature library;
[0015] S3, input the rail test image into the rail crack recognition model for model testing. During the model testing, the rail test image features are extracted, and the extracted test image features are compared with the normal sample features in the feature library for nearest neighbor search to obtain the defect area of the rail test image and the pixel-level anomaly score of each pixel in the defect area, and the pixel-level anomaly score is compared with the pixel threshold to obtain the abnormal pixel. All abnormal pixel areas constitute the rail crack area and are highlighted;
[0016] S4. Inputting the track image to be identified into the trained and tested track crack identification model to perform track crack identification.
[0017] 1. Model Construction
[0018] Preferably, the track crack identification model is constructed using the PatchCore algorithm.
[0019] 2. Model Training
[0020] Preferably, during model training, the extracted normal sample features are downsampled, and then the downsampled normal sample features are saved in a feature library.
[0021] In the present invention, it can be known from model training that as the number of normal samples increases, if the extracted features are directly stored in the feature library, the feature library will increase dramatically, so the extracted features need to be downsampled.
[0022] In the present invention, during the training phase, only a large number of normal track sample images are needed, which is easy for the scenario of rail crack recognition, and these samples do not need manual annotation. A large number of normal track sample images are input into the pre-trained neural network model (rail crack recognition model) to extract features, and then the large number of extracted features are downsampled to reduce data redundancy, and finally the features are saved in the feature library.
[0023] Preferably, the down-sampling input includes: rail crack identification model Feature layer j, normal sample X N , step size s, number of blocks p, target length of feature library l, random linear projection The output includes: a block-level feature library M.
[0024] Preferably, the downsampling process includes:
[0025] S21, establishing an empty feature set M;
[0026] S22, taking an image from the normal track sample image and extracting its L-dimensional features, and downsampling the number of features, and calculating the union of the extracted features and the existing feature set M; repeating this step until all normal track sample images are sampled;
[0027] S23, establishing an empty feature set Mc;
[0028] S24, from the first dimension to the Lth dimension of the feature, select a feature m from the feature set M in each dimension, and add the m feature to the feature set Mc; wherein, when selecting feature m, the Euclidean distance between feature m and all the features in the feature set Mc in the dimension is the farthest;
[0029] S25, clear the feature set M, and then store the feature set Mc into the feature set M.
[0030] 3. Model Testing
[0031] In the present invention, during the testing phase, a cracked image is input into a rail crack recognition model (the same as during training) to extract features, and then the features are compared with all normal sample features extracted and saved in the feature library during training for nearest neighbor search. At this time, the background area features of the input image can be successfully matched with the features in the database, while the features at the cracks will not be able to find matching features in the database, thereby obtaining an abnormal score for each pixel point, and highlighting the abnormal pixels by setting a threshold, thereby achieving pixel-level segmentation of rail cracks with zero abnormal samples.
[0032] Preferably, the model test includes: the test image is x test , divide the test image into several blocks m test After that, find each block m of the test image in the feature library test The closest feature m * , and calculate the block m with the maximum distance between all blocks of the test image and the matching features test,* , and further get the test image abnormality score s * =||m test,* -m * ||2, and compare the test image anomaly score with the image threshold. When the test image anomaly score is greater than the image threshold, the block m corresponding to the test image anomaly score is test as the defect area in the test image.
[0033] Preferably, the model test further includes: for the defect area For each pixel, find the pixel feature with the closest Euclidean distance in the feature library Then calculate the defect area Pixel-level anomaly score The pixel-level anomaly score is then compared with the pixel threshold. When the pixel-level anomaly score is greater than the pixel threshold, the pixel area corresponding to the pixel-level anomaly score is a rail crack.
[0034] Preferably, the image threshold is 0.6 and the pixel threshold is 0.5.
[0035] In the present invention, for the abnormal score of the test image (between 0 and 1), a threshold of 0.6 between abnormal and normal is set in the program. If the abnormal score is greater than the threshold, it is considered abnormal, otherwise it is normal. If it is an abnormal image, it is necessary to perform pixel-level abnormal and normal judgments. By setting the abnormal pixel point threshold of 0.5, if it exceeds the threshold, the pixel point is considered to be a crack, otherwise it is not.
[0036] Preferably, the model test further comprises: performing secondary filtering on the abnormal pixel region by a region growing method to obtain the final track crack region and highlight it.
[0037] Preferably, the secondary filtering of the abnormal pixel region by the region growing method comprises the following steps:
[0038] Find the pixel position coordinates (i, j) with the largest pixel-level anomaly score from the abnormal pixel area image;
[0039] The 8-neighborhood of the pixel position coordinate (i, j) is used for growth. The growth criterion is that the difference between the gray value of the test point and the gray value of the neighborhood growth point is less than or equal to 2. This step is repeated until growth is no longer possible, and the regional growth track crack segmentation image is obtained.
[0040] The intersection of the regional growth track crack segmentation image and the abnormal pixel area image is calculated, and the intersection is the final track crack area and it is highlighted.
[0041] IV. Model Application
[0042] In the present invention, after the track crack recognition model is constructed, trained and tested, the track image to be recognized is input into the track crack recognition model. The track crack recognition model uses the previous training and testing ideas to perform track crack recognition on the track image to be recognized, and highlights the recognized cracks.
[0043] Beneficial effects of the present invention:
[0044] The rail crack recognition method provided by the present invention only requires a large amount of normal sample data during the training phase, which is easy for the scenario of rail crack recognition, and these samples do not need to be manually labeled. A large number of normal samples are input into a pre-trained neural network model to extract features, and then the large number of extracted features are downsampled to reduce data redundancy, and finally the features are saved in a feature library.
[0045] The rail crack recognition method provided by the present invention, in the test phase, inputs a cracked image into a pre-trained neural network model to extract features, and then performs nearest neighbor search on the features and all normal sample features extracted and saved in the feature library during training. At this time, the background area features of the input image can be successfully matched with the features in the database, while the features at the cracks cannot find matching features in the database, thereby obtaining the abnormal score of each pixel point, and highlighting the abnormal pixels by setting a threshold, thereby realizing the pixel-level segmentation of rail cracks with zero abnormal samples.
[0046] The rail crack identification method provided by the present invention further performs secondary filtering on the identification result through the traditional region growing method to reduce the segmentation error. BRIEF DESCRIPTION OF THE DRAWINGS
[0047] Figure 1 It is a schematic diagram of the rail crack identification method of the present invention in the training and testing stages;
[0048] Figure 2 This is the traditional rail crack image recognition algorithm flow. DETAILED DESCRIPTION
[0049] The concept, specific structure and technical effects of the present invention will be clearly and completely described below in conjunction with the embodiments and drawings to fully understand the purpose, characteristics and effects of the present invention.
[0050] A rail crack identification method, such as Figure 1 As shown, the following steps are included:
[0051] S1. Construct rail crack identification model;
[0052] S2, inputting the normal track sample image into the track crack recognition model for model training, extracting normal sample features from the normal track sample image during model training, and saving the extracted normal sample features into a feature library;
[0053] S3, input the rail test image into the rail crack recognition model for model testing. During the model testing, the rail test image features are extracted, and the extracted test image features are compared with the normal sample features in the feature library for nearest neighbor search to obtain the defect area of the rail test image and the pixel-level anomaly score of each pixel in the defect area, and the pixel-level anomaly score is compared with the pixel threshold to obtain the abnormal pixel. All abnormal pixel areas constitute the rail crack area and are highlighted;
[0054] S4. Inputting the track image to be identified into the trained and tested track crack identification model to perform track crack identification.
[0055] like Figure 1 As shown, in the present invention, during the training phase, only a large amount of normal sample data is required, which is easy for the scenario of rail crack identification, and these samples do not need to be manually labeled (solving Difficulties 1 and 3 mentioned in the background technology). A large number of normal samples are input into the pre-trained neural network model to extract features, and then the large number of extracted features are downsampled to reduce data redundancy, and finally the features are saved in the feature library.
[0056] During the testing phase, a cracked image is input into the pre-trained neural network model (the same as during training) to extract features, and then the features are compared with all the normal sample features extracted and saved in the feature library during training for nearest neighbor search. At this time, the background area features of the input image can be successfully matched with the features in the database, while the features at the cracks will not be able to find matching features in the database, thereby obtaining the anomaly score for each pixel point. By setting the threshold to highlight the abnormal pixels, the pixel-level segmentation of rail cracks with zero abnormal samples is achieved.
[0057] From the above training and testing process, it can be seen that when migrating to crack recognition in a new scenario, it is only necessary to obtain a large number of normal samples to achieve cold start of crack recognition, and the prediction results are presented in the form of semantic segmentation. This solves the difficulty 2 mentioned in the background technology.
[0058] From the above training process, we can see that as the number of normal samples increases, if the extracted features are directly stored in the feature library, the feature library will increase dramatically, so the extracted features need to be downsampled. The downsampling algorithm is as follows:
[0059] Input includes: Pre-trained neural network model Feature layer j, normal sample X N , step size s, number of blocks p, target length of feature library l, random linear projection The output includes: a block-level feature library M.
[0060] Specifically, the downsampling process includes the following steps:
[0061] Step 1: Create an empty feature set M
[0062] Step 2: Take an image from the normal sample, extract the features of L dimension, and downsample the number of features. Finally, calculate the union with the existing feature set M, and repeat this step until all image samples have been sampled.
[0063] Step 3: Create an empty feature set Mc;
[0064] Step 4: From the 1st dimension to the Lth dimension of the feature, select a feature m from M in each dimension. This feature needs to have the farthest Euclidean distance from all the features in Mc in this dimension, and then add the m feature to Mc. The main purpose is to select the most representative features from M and add them to Mc;
[0065] Step 5: Clear M and then store Mc into M.
[0066] In the testing phase, assume that the test image is x test After dividing the image into blocks, find each block m of the test image in the feature library test The closest feature m * , and calculate the block with the maximum distance between all blocks of the image and the matching features, that is, Further get s * =||m test,* -m * ||2. Thus, the abnormal score s of the image is obtained * In order to segment the defect area, it is necessary to obtain the pixel-level anomaly score. For each image block, find the feature with the closest Euclidean distance in the feature library, so there is Therefore, the abnormal score of this image block is This allows for semantic segmentation of abnormal pixels.
[0067] In this embodiment, for the abnormal score of the test image (between 0 and 1), a threshold of 0.6 between abnormal and normal is set in the program. If the abnormal score is greater than the threshold, it is considered abnormal, otherwise it is normal. If it is an abnormal image, it is necessary to perform pixel-level abnormal and normal judgments. By setting the abnormal pixel threshold of 0.5, if it exceeds the threshold, the pixel is considered to be a crack, otherwise it is not.
[0068] In this embodiment, in addition to the above-mentioned deep learning algorithm for identifying rail cracks, the traditional region growing method is also used to perform secondary filtering on the identification results to reduce segmentation errors. The specific steps are as follows:
[0069] Step 1: Find the pixel position coordinates (i, j) with the largest anomaly score from the pixel-level anomaly segmentation map obtained by the deep neural network;
[0070] Step 2: Using the 8-neighborhood of (i, j), the growth criterion is that the difference between the grayscale value of the test point and the grayscale value of the neighborhood growth point is less than or equal to 2, and this step is repeated until no further growth is possible;
[0071] Step 3: Find the intersection of the rail crack segmentation image obtained by the region growing method and the rail crack segmentation image obtained by the deep neural network. (That is, if a pixel is an abnormal pixel, it must appear in the segmentation result of the region growing method and the segmentation result of the deep neural network at the same time.)
[0072] In this embodiment, after the track crack recognition model is constructed, trained and tested, the track image to be recognized is input into the track crack recognition model. The track crack recognition model uses the previous training and testing ideas to perform track crack recognition on the track image to be recognized, and highlights the recognized cracks.
[0073] The above is a specific description of the implementation mode of the present invention, but the present invention is not limited to the described embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present invention, and these equivalents or substitutions are all included in the scope defined by the claims of the present invention.
Claims
1. A rail crack identification method, characterized in that: The following steps are involved: Construct a rail crack identification model; The normal track sample image is input into the track crack recognition model for model training. During the model training, the normal sample features in the normal track sample image are extracted, and the extracted normal sample features are saved in the feature library; The rail test image is input into the rail crack recognition model for model testing. During the model testing, the rail test image features are extracted, and the extracted test image features are compared with the normal sample features in the feature library for nearest neighbor search to obtain the defect area of the rail test image and the pixel-level anomaly score of each pixel in the defect area. The pixel-level anomaly score is compared with the pixel threshold to obtain the abnormal pixel. All abnormal pixel areas constitute the rail crack area and are highlighted. The track image to be identified is input into the trained and tested track crack identification model for track crack identification.
2. A rail crack identification method as claimed in claim 1, characterized in that: During model training, the extracted normal sample features are downsampled, and then the downsampled normal sample features are saved in the feature library.
3. A rail crack identification method as claimed in claim 2, characterized in that: The downsampled inputs include: Rail crack identification model Feature layer j, normal sample X N , step size s, number of blocks p, target length of feature library l, random linear projection The output includes: a block-level feature library M.
4. A rail crack identification method as claimed in claim 3, characterized in that: The downsampling process includes: Establish an empty feature set M; Take an image from the normal track sample image and extract its L-dimensional features, then downsample the number of features and calculate the union of the extracted features and the existing feature set M; repeat this step until all normal track sample images are sampled; Create an empty feature set Mc; From the first dimension to the Lth dimension of the feature, select a feature m from the feature set M in each dimension and add the m feature to the feature set Mc; when selecting feature m, the Euclidean distance between feature m and all the features in the feature set Mc in this dimension is the farthest; Clear the feature set M, and then store the feature set Mc into the feature set M.
5. A rail crack identification method as claimed in claim 1, characterized in that: Model testing includes: the test image is x test , divide the test image into several blocks m test After that, find each block m of the test image in the feature library test The closest feature m * , and calculate the block m with the maximum distance between all blocks of the test image and the matching features test,* , and further get the test image abnormality score s * =||m test,* -m * ||2, and compare the test image anomaly score with the image threshold. When the test image anomaly score is greater than the image threshold, the block m corresponding to the test image anomaly score is test as the defect area in the test image.
6. A rail crack identification method as claimed in claim 5, characterized in that: Model testing also includes: for defect areas For each pixel, find the pixel feature with the closest Euclidean distance in the feature library Then calculate the defect area Pixel-level anomaly score The pixel-level anomaly score is then compared with the pixel threshold. When the pixel-level anomaly score is greater than the pixel threshold, the pixel area corresponding to the pixel-level anomaly score is a rail crack.
7. A rail crack identification method according to claim 1, characterized in that: The model test also includes: secondary filtering of abnormal pixel areas through the region growing method to obtain the final track crack area and highlight it.
8. A rail crack identification method as claimed in claim 7, characterized in that: The secondary filtering of the abnormal pixel region by the region growing method comprises the following steps: Find the pixel position coordinates (i, j) with the largest pixel-level anomaly score from the abnormal pixel area image; The 8-neighborhood of the pixel position coordinate (i, j) is used for growth. The growth criterion is that the difference between the gray value of the test point and the gray value of the neighborhood growth point is less than or equal to 2. This step is repeated until growth is no longer possible, and the regional growth track crack segmentation image is obtained. The intersection of the regional growth track crack segmentation image and the abnormal pixel area image is calculated, and the intersection is the final track crack area and it is highlighted.
9. A rail crack identification method as claimed in claim 1, characterized in that: The track crack identification model is constructed using the PatchCore algorithm.
10. A rail crack identification method according to claim 6, characterized in that: The image threshold is 0.6 and the pixel threshold is 0.5.