Rail damage ultrasonic positioning and identification method and computer-readable storage medium

Abstract

This application discloses a method for ultrasonically locating and identifying rail damage and a computer-readable storage medium. The method comprises the following steps: S11) converting rail ultrasonic detection signals into ultrasonic time series data and B-display image data; S12) preprocessing the generated ultrasonic time series data and B-display image data; S13) using an image segmentation network to semantically segment the preprocessed B-display image data and generate segmented subregions to localize damage; S14) using a classification network to identify damage categories for targets within the generated segmented subregions; and S15) using expert rules to extract features and post-process the classification results of step S14), refining the damage identification results obtained in step S14. This application addresses the technical issues of existing rail damage ultrasonic detection methods, which require significant manual intervention, have long detection cycles, and exhibit low damage detection rates.

Description

Technical Field

[0001] The present application relates to the technical field of rail flaw detection, and in particular to an ultrasonic flaw detection method and a computer-readable storage medium for intelligently locating and classifying internal rail damage. Through deep learning and image segmentation technology, the position of internal rail damage can be efficiently and accurately located, and the type of damage can be automatically identified. Background Art

[0002] In the field of rail engineering machinery, rail flaw detection vehicles 6 are commonly used as engineering vehicles for detecting internal damage of rails and are widely used in rail flaw detection and maintenance. Figure 1 As shown, an ultrasonic rail flaw detection system typically includes a rail flaw detection system 7 and a rail flaw analysis system 8, mounted on a rail flaw detection vehicle 6. A flaw detection device 9 is also mounted on the vehicle, with a probe wheel 11 mounted on the probe wheel. When rail flaw detection is required, the rail flaw detection system 7 controls the probe wheel 11 to press against the upper surface of a rail 13. This sends an ultrasonic excitation pulse signal to the probe wheel 11. Under the influence of the alternating electric field, the ultrasonic chip 12 within the probe wheel 11 generates mechanical vibrations synchronized with the electric field, thereby emitting an ultrasonic signal. The generated ultrasonic signal passes through the wheel fluid, wheel film, and coupling fluid layer and is incident on the rail 13. Ultrasonic waves emitted by ultrasonic chips 12 at different angles propagate along different paths within the rail 13. As the ultrasonic signal propagates within the rail 13, it returns upon encountering a flaw or the bottom of the rail. The returned ultrasonic signal passes through the rail flaw analysis system 8 to form a flaw image.

[0003] As attached Figure 2 As shown, the ultrasonic rail flaw detection system is equipped with a total of six probe wheels 11 filled with ethylene glycol liquid, with three probe wheels 11 installed on each side. On the same side of the probe wheels, the front and rear probe wheels are installed in mirror-image positions, each equipped with six ultrasonic chips 12, namely a three-chip array with 0 degrees, 45 degrees, and 70 degrees, and a side-shot chip; the middle probe wheel is equipped with two ultrasonic chips with a 70-degree deflection pointing to the inside of the rail 13 and a 0-degree ultrasonic chip. When the rail flaw detection vehicle 6 is running, the probe wheel 11 rolls along the rail 13, and the ultrasonic chips 12 inside it remain stationary. Ultrasonic chips 12 installed at different angles can ensure multi-angle and multi-directional flaw detection of the rail 13.

[0004] The rail damage analysis system 8 of the rail flaw detection vehicle 6 generally has two graphic windows, A-type display and B-type display, for damage judgment. Among them, the A-type display displays the ultrasonic echo simulation signal through an oscilloscope, as shown in the attached figure. Figure 3 As shown in the figure, while the B-type display directly displays the information of the ultrasonic signal reflection points inside the rail through images, as shown in the figure. Figure 4 In the B-type diagram, the reflection surface of the ultrasonic wave is defined as shown in the attached figure. Figure 5The normal direction of the echo point shown is the ultrasonic propagation direction. Each echo point can be detected by the ultrasonic chip 12 at the corresponding angle, and is therefore recorded as the corresponding detection channel number. The rail damage detection system 7 uses multiple detection. A structural feature can be detected multiple times. For example, the features A, B, and C of a bolt hole can be detected by different ultrasonic chips 12, thereby forming an "A"-shaped detection pattern, as shown in the attached figure. Figure 6 shown.

[0005] Rail flaw detection vehicles are widely used in rail flaw inspection operations due to their high efficiency and comprehensive inspection range. However, quickly and accurately identifying internal rail flaws remains an area of ​​urgent need for improvement. The current operating model primarily relies on manual flaw identification through full-line playback by onboard and ground personnel. This not only increases labor costs but also relies heavily on factors such as the experience of the playback personnel, which can easily lead to subjective and human-induced flaw omissions. Furthermore, while some domestic rail flaw detection vehicles' rail flaw detection systems include a flaw recognition function that can automatically locate and classify flaws in measured data, the recognition algorithm employed relies on a logical judgment method based on established rules. The diverse shapes and conditions of rail flaws cannot be fully determined using simple logic. As a result, the accuracy of flaw recognition results is low, with a high incidence of false positives and missed flaws. Currently, intelligent flaw recognition using ultrasonic testing data using methods such as decision tree analysis, artificial feature construction, and support vector machines is becoming a mainstream research area. With the rapid development of deep learning in target recognition and classification, a number of deep learning-based damage identification methods have emerged. For example, these methods use a binary sparse matrix of 16 ultrasonic channels as input, utilize inter-channel distance features to locate the target, and then use the AlexNet network to identify damage. Alternatively, they use RGB color channels to represent the rail head, rail waist, and rail foot information, respectively, and then use the RCNN network for target detection. Although deep learning methods offer many advantages, practical applications in rail damage identification still suffer from poor interpretability and difficulties in parameter optimization.

[0006] In the prior art, the following technical solutions are mainly related to this application:

[0007] Prior Art 1 is a Chinese invention application filed by the Guangzhou Institute of Advanced Technology, Chinese Academy of Sciences on March 3, 2020, and published on July 10, 2020, with publication number CN111402209A. This application discloses a U-Net-based method for high-speed rail rail damage detection, including the following steps: establishing a real-world scenario for high-speed rail rail damage detection and acquiring images of rail damage detection; marking rail damage areas in each acquired original image and saving the annotations; expanding training samples through data augmentation operations such as translation, rotation, and scaling; setting parameters for the rail damage detection network model, inputting an image and its corresponding segmentation map, training the detection network using a deep learning algorithm platform, and modifying the detection network model parameters based on the evaluation metrics of the trained network until the evaluation metrics are obtained; marking the original input image with a rectangular box based on the segmentation map calculated by the U-Net network; if the acquired image contains defects, the relevant processing results are displayed on the computer, and the defective workpiece is identified and stored in the database. This application is applicable to rail damage and crack detection under different conditions, with strong generalization and high accuracy. This application primarily uses a depth camera to collect rail surface data, presenting and analyzing it in pure image form. To compensate for environmental influences on the collected data, additional light sources are required to supplement brightness and other factors. It does not utilize data from other modalities for damage detection. Furthermore, in terms of overall architecture, this application primarily uses image data to extract rail surface damage using a UNet network, ultimately marking the damaged areas with rectangular frames. Its primary focus is on data labeling, data preprocessing, and UNet model parameter adjustment.

[0008] Prior art 2 is a Chinese invention application filed by Shenzhen Biyibi Network Technology Co., Ltd. on September 24, 2021, and published on January 4, 2022, with publication number CN113888488A. This application discloses a rail defect detection method and system based on a deep residual shrinkage network, comprising the following steps: S1) preprocessing and cleaning the damaged image data of existing rails; S2) extracting features from the cleaned damaged image data to obtain a feature vector of the corresponding damage; S3) training and learning the features of the damaged image to obtain an optimal classification model; S4) obtaining the damaged rail image data, and obtaining a damage list after classification using the optimal classification model. This application does not require manual intervention, reducing labor costs, and has a high damage detection rate, but it is difficult to process noise and lacks edge shape information. First, this application addresses the problem of inaccurate damage judgment based on damage data. It uses B-display image data instead of data from other modalities, and focuses more on the damage judgment scenario. At the same time, the application skips the target positioning step of the entire line data, and instead directly determines the specific category of damage based on the target identification of the ultrasonic data.

[0009] Prior art 3 is the paper "Rail Surface Damage Detection Method Based on Improved U-Net Convolutional Neural Network" published by Liang Bo et al. in "Laser & Optoelectronics Progress" Issue 2, 2021, pp. 326-332. This paper plays a very important role in promoting the automated detection of rail surface damage based on the deep learning method of convolutional neural network, and proposes a rail surface damage detection method based on convolutional neural network. First, adding a branch network between the contraction path and the expansion path of the classic U-Net can assist U-Net in outputting an ideal segmentation map. Then, the Type-I RSDDs high-speed railway track dataset is used as a test sample, and the test sample is amplified by data enhancement and then placed in the improved U-Net for training and testing. Finally, the proposed method is evaluated using evaluation indicators. After testing, the detection accuracy of the method proposed in this paper reached 99.76%, which is 6.74 percentage points higher than the highest level of other methods, which can significantly improve the detection accuracy. This paper mainly uses the Type-I RSDDs high-speed railway track dataset as a test sample. According to the examples in the paper, it is composed of images of the rail surface taken with a high-speed camera. The difference in input data will affect the subsequent technical route selected and a series of methods for processing the data itself. In addition, the paper aims to use a modified U-Net convolutional neural network to segment and locate damage on the rail surface. Its advantage lies in adding a branch unit before the expansion path, named GRASP network. This branch network takes the output result of the contraction path (downsampling part) as input, and the prediction result after processing by the branch network as output and combines it with the final output result of the expansion path to optimize the accuracy of target segmentation. Therefore, it is itself an improvement on the UNet network, without adding other processing methods.

[0010] Prior art 4 is a paper published by Luo Jiangping et al. in Locomotive Electric Transmission, Issue 2, 2021, pp. 100-107

[0011] "Intelligent Rail Damage Identification System Based on Deep Learning and Support Vector Machines." This paper proposes a technical solution for an intelligent rail damage identification system based on deep learning and support vector machines. The system uses a combination of deep separable convolution and selective search for target location. Based on manually constructed multi-dimensional features, it uses a support vector machine (SVM) approach to classify damage images. The effectiveness of the proposed system was verified by testing it with manually annotated samples from actual line test data. Test results demonstrate excellent performance across various technical indicators, with a damage detection rate of 99.8%, a false alarm rate reduced to 12%, and a classification accuracy exceeding 95%. The paper uses selective search to provide candidate boxes and employs a separable convolutional network to identify the target with the highest target score within these boxes. After the target location is determined, 90-dimensional features are manually designed, and a support vector machine (SVM) model is trained. The SVM then determines the final damage classification. However, the use of a separable convolutional network results in discontinuous kernel processing, which loses information correlation. Furthermore, the paper's use of manually selected features introduces a degree of subjectivity. Summary of the Invention

[0012] In view of this, the purpose of this application is to provide a method for ultrasonic positioning and identification of rail damage and a computer-readable storage medium to solve the technical problems that the existing ultrasonic detection method for rail damage requires more manual intervention, has a long detection cycle and a low damage detection rate.

[0013] In order to achieve the above-mentioned invention objectives, the present application specifically provides a technical implementation scheme of a method for ultrasonically locating and identifying rail damage, which includes the following steps:

[0014] S11) converting the rail ultrasonic detection signal into ultrasonic time series data and visually visible B-display image data;

[0015] S12) preprocessing the ultrasonic time series data and B-display image data generated in step S11);

[0016] S13) using an image segmentation network to perform semantic segmentation on the B-display image data processed in step S12) and generate segmented sub-regions to achieve damage localization;

[0017] S14) using a classification network to identify damage categories of targets within the segmented sub-regions generated in step S13);

[0018] S15) Using expert rules to perform feature extraction and post-processing on the classification results of step S14), and refine the damage identification results obtained in step S14).

[0019] Furthermore, the step S11) further includes the following process:

[0020] The ultrasonic detection signals within a plurality of unit pulse lengths are used as the data contained in a frame of image, thereby converting the original ultrasonic detection signal data into a plurality of image data, and continuously inputting the data into step S12 in the form of a data stream for data preprocessing.

[0021] Furthermore, the data preprocessing process in step S12) is to perform processing including enhancement, supplementation or denoising on the ultrasonic time series data and B-display image data. Through data preprocessing, on the basis of balanced data distribution, the distance within the data class becomes smaller and the distance between the data classes becomes larger.

[0022] Furthermore, the pre-processing process in step S12) includes the following steps:

[0023] Based on the ultrasonic time series data, the data where the bottom wave disappears are directly deleted.

[0024] Furthermore, the pre-processing process in step S12) includes the following steps:

[0025] Based on the damage type determined by the existing ultrasonic detection signal, the echo point data of the damage is intercepted from the original ultrasonic time series data according to the coordinate area. According to the scope of the damage, the vertical coordinate representing the depth is kept unchanged, and the horizontal coordinate representing the distance is randomly drawn onto a new preset background image. This simulates damage data under different backgrounds and thus performs data amplification. Alternatively, according to the extension law of the damage, partial data is extracted and reconstructed through data segmentation to form a new damage, which is presented in the form of a B-display image, thereby achieving data amplification.

[0026] Furthermore, the pre-processing process in step S12) includes the following steps:

[0027] Masks of 3*3, 5*5 and 7*7 sizes are used respectively to mask the content of the B-display image data in a sliding window manner according to the mask step size, and it is ensured that the masked part of the selected size does not affect the judgment of damage.

[0028] Furthermore, the pre-processing process in step S12) includes the following steps:

[0029] Perform Hilbert transform on ultrasonic time series data to obtain analytical signal, retain the signal unilateral spectrum to draw the signal envelope, and reduce the noise interference of time series. Hilbert transform is expressed as x a =F -1 (F(x)2U), where x a represents the Hilbert analytic signal, F(·) and F -1(·) represents the Fourier transform and inverse Fourier transform, U represents the unit step signal, and x represents the original signal. After the envelope of the analytical signal is determined, the mean of the envelope at each time point is calculated using the values ​​of the upper and lower envelopes.

[0030] Furthermore, the image segmentation network in step S13) adopts a UNet network.

[0031] Furthermore, the step S13) includes the following process:

[0032] S131) ​​The B-display image data processed in step S12) is used as the input feature map, which is first scaled to a size of 572×572×1, and then convolved twice with a 3×3×64 convolution kernel to a size of 568×568×64. Then, a 2×2 maximum pooling operation is performed to a size of 284×284×64, completing a downsampling operation.

[0033] According to the above step S131), the 3×3 convolution and 2×2 pooling operations are repeated three times. After each maximum pooling operation, the number of convolution kernels of the first 3×3 convolution operation is doubled.

[0034] S132) After reaching the fourth maximum pooling at the bottom layer, the feature map becomes 32×32×512 in size, and then performs two 3×3×1024 convolution operations, finally becoming 28×28×1024 in size.

[0035] S133) The upsampling operation is performed on a feature map based on a size of 28×28×1024. First, a 2×2×512 deconvolution operation is performed to change the feature map to a size of 56×56×512. Then, the image before the corresponding maximum pooling layer is copied and cropped, and the image obtained by deconvolution is spliced ​​with the channel dimension to obtain a feature map of size 56×56×1024. Then, a 3×3×512 convolution operation is performed to complete an upsampling operation.

[0036] According to the above step S133), the 2×2 deconvolution and 3×3 convolution operations are repeated three times, and the number of convolution kernels in the first 3×3 convolution operation after each splicing is reduced exponentially.

[0037] S134) After reaching the fourth deconvolution at the top layer, the feature map becomes 392×392×64 in size, is copied and cropped, and then concatenated to a size of 392×392×128, and then two 3×3×64 convolution operations are performed.

[0038] S135) obtains a feature map of size 388×388×64, and finally performs a 1×1×2 convolution operation to obtain the segmentation area information of the B-display image.

[0039] Furthermore, according to the segmented area information obtained in step S13), each individual target block is obtained as an irregularly shaped B-display image block, the segmented B-display block is drawn into a black background to generate a new sub-block image, and then the final damage type is identified by the designed secondary classifier.

[0040] Furthermore, the step S14) includes:

[0041] After damage classification using the classification network, each detection sample of the sub-block image is given a category confidence, that is, the probability value of the detection sample belonging to a certain target category, denoted as v obj_conf , preset the probability value v of the detection sample obj_conf ≥thresh, the classification network can make accurate judgments, and the thresh value can be customized according to different situations. obj_conf When the confidence level is less than thresh, the sample with low confidence is classified as junk, indicating that the target category of the current sample needs additional information to assist in the judgment. Then, the target category is judged based on the expert rules for the junk class to accurately determine the target category.

[0042] Furthermore, the classification network in step S14) uses a ResNet network as a secondary classifier.

[0043] Furthermore, the step S14) includes the following process:

[0044] S141) The sub-block image of the segmented sub-region is scaled to a size of 224×224×3, and a convolution operation is first performed. The convolution operation has a convolution kernel of 3×3, a stride of 2, a padding of 3, an output channel of 64, and an output feature map size of 112×112×64.

[0045] S1421) The first residual structure combination includes two residual base structures. The output feature map first undergoes two 3×3×64 convolution operations, where the convolution kernel is 3×3, the step size is 1, the padding is 1, and the number of channels is 64. The output feature map is added to the input feature map of the residual base structure in a channel-by-channel addition manner. The second 3×3×64 convolution operation is performed twice, where the convolution kernel is 3×3, the step size is 1, the padding is 1, and the number of channels is 64. The output feature map is added to the input feature map of the residual base structure in a channel-by-channel addition manner. At this time, the feature map output by the first residual structure combination has a size of 56×56×64.

[0046] S1422) The second residual structure combination includes two residual basic structures. The feature map output by the first residual structure combination is first subjected to two 3×3×128 convolution operations. The convolution kernel of the convolution operation is 3×3, the step size is 1, the padding is 1, and 128 kernel functions are used. A 1×1×128 convolution operation is performed between the two convolution operations, with a step size of 2 and a padding of 0 to reduce the size of the feature map by half, and at the same time, it is added to the input feature map of the residual basic structure in a channel addition manner. Two more 3×3×128 convolution operations are performed, with a convolution kernel of 3×3, a step size of 1, a padding of 1, and 128 kernel functions, and at the same time, it is added to the input feature map of the residual basic structure in a channel addition manner. At this time, the feature map size output by the second residual structure combination is 28×28×128.

[0047] S1423) The third residual structure combination includes two residual basic structures, and the feature map output by the second residual structure combination is first subjected to two 3×3×256 convolution operations, with a convolution kernel of 3×3, a step size of 1, a padding of 1, and 256 kernel functions. A 1×1×256 operation is performed between the two convolution operations, with a step size of 2 and a padding of 0 to reduce the size of the feature map by half, and at the same time, it is added to the input feature map of the residual basic structure in a channel addition manner. The second time, two 3×3×256 convolution operations are performed again, with a convolution kernel of 3×3, a step size of 1, a padding of 1, and a 256 kernel function, and at the same time, it is added to the input feature map of the residual basic structure in a channel addition manner. At this time, the output feature map size of the third residual structure combination is 14×14×256;

[0048] S1424) The fourth residual structure combination includes two residual basic structures, and the feature map output by the third residual structure combination is first subjected to two 3×3×512 convolution operations, the convolution kernel of the convolution operation is 3×3, the step size is 1, the padding is 1, and 512 kernel functions are used. A 1×1×512 operation is performed between the two convolution operations, with a step size of 2 and a padding of 0 to reduce the size of the feature map by half, and at the same time, it is added to the input feature map of the residual basic structure in a channel addition manner. The second time, two 3×3×512 convolution operations are performed again, the convolution kernel of the convolution operation is 3×3, the step size is 1, the padding is 1, and 512 kernel functions are used, and at the same time, it is added to the input feature map of the residual basic structure in a channel addition manner. At this time, the output feature map size of the fourth residual structure combination is 7×7×512;

[0049] S143) The feature map outputted by the step S141) is average-pooled after being combined with the four residual structures, and the output data size is 1×1×512. Finally, the output is outputted through the fully connected layer to several categories corresponding to the classification, and the damage category and probability value of the target area are finally outputted through the activation function.

[0050] Furthermore, the input of the expert rules is image data or ultrasonic time series data. The expert rules can be used to combine the basic ultrasonic time series data and image data to jointly determine the final damage category for garbage objects that are difficult to classify by the classification network.

[0051] Furthermore, the expert rules in step S15) include:

[0052] Calculate the fitting line of all channel echo points in the target area based on the ultrasonic time series data. After determining the fitting line, calculate the total slope Among them, Δy and Δx represent the vertical coordinate increment and horizontal coordinate increment of the damage echo point in the target area respectively;

[0053] Based on the ultrasonic time series data, the fitting line of the echo point of a channel in the target area is calculated. After the fitting line is determined, the slope of a channel is calculated. Δy c and Δx c They respectively represent the horizontal coordinate increment and vertical coordinate increment of the damage echo point belonging to a certain channel;

[0054] Only when the slope k of each channel c When the total slope k of all channels does not meet any of the judgment conditions of rail head core damage, rail waist damage, and rail bottom damage, the current target is judged to be noise under the trend condition.

[0055] Furthermore, the expert rules in step S15) include:

[0056] For scattered point noise, the density and 1 / 4 density of the current target area are obtained based on the B-display image data, which are recorded as Where s represents the area of ​​the current target region, and m represents the number of channel echo points within the current target region. By dividing the current target region into four sub-regions and calculating the corresponding density distribution for each, we determine whether there is damage and optimize the damaged area for secondary positioning.

[0057] Furthermore, the expert rules in step S15) include:

[0058] For targets with similar shapes, mask matching and channel echo point position matching are used based on B-display image data to calculate and analyze whether the target conforms to the echo point distribution pattern. First, a binarization method is used to separate the target and background parts of the target image data. Then, a mask is slid across the sub-target image to determine whether the size of the unmatched target is abnormal.

[0059] Furthermore, the expert rules in step S15) include:

[0060] For screw hole cracks, based on the B-display image data, the positional relationship between the 45-degree channel echo point and the 0-degree channel echo point in the front and rear directions is determined, as well as whether the echo point appears at the crack position of the screw hole, thereby determining whether there is any abnormality in the screw hole.

[0061] The present application further specifically provides a technical implementation solution of a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned ultrasonic positioning and identification method for rail damage is implemented.

[0062] By implementing the above-mentioned rail damage ultrasonic positioning and identification method and the technical solution of the computer-readable storage medium provided by this application, the following beneficial effects are achieved:

[0063] (1) The ultrasonic positioning and identification method for rail damage and the computer-readable storage medium of the present application use an image segmentation network to perform semantic segmentation on the B-display image, and use a classification network to determine the damage category of the formed segmented sub-regions. This method can obtain relatively accurate target location and category results without human intervention, overcoming the problem that the existing ultrasonic detection method for rail damage requires manual feature design and manual decision-making on B-display data segmentation, greatly improving the damage detection rate and shortening the detection cycle of rail ultrasonic flaw detection;

[0064] (2) The ultrasonic positioning and identification method for rail damage and the computer-readable storage medium of the present application use an image segmentation network to directly locate internal rail damage from the perspective of image processing, and use a classification network to determine the damage category, thereby achieving end-to-end rail ultrasonic flaw detection data recognition and accurately identifying the nature of the damage by combining multi-layer features. This solves the technical problems of existing rail damage ultrasonic detection methods, such as poor interpretability and difficulty in parameter optimization, inability to accurately analyze internal rail damage, and difficulty in processing noise and edge shape information;

[0065] (3) The ultrasonic positioning and identification method for rail damage and the computer-readable storage medium of the present application convert the original ultrasonic echo signal data (B-display data) into time series data, introduce the concept of time through time series data, and can show the correlation between echoes at different times; at the same time, in the scope of time series data, the statistical characteristics of the past and current moments are used to predict and guide the data change trend at future moments, and it conforms to the objective development law of things; similarly, while converting the original ultrasonic echo signal data (B-display data) into time series data, the time series data itself can have richer properties, such as periodicity, stationarity, time-frequency domain characteristics, etc.; in addition, the time series allows tolerance of certain random variables, which increases the generalization ability of the prediction model;

[0066] (4) The ultrasonic positioning and identification method for rail damage and the computer-readable storage medium of the present application use image vision to perform target positioning and identification of ultrasonic rail detection data. The rail ultrasonic detection data is first converted into visually visible image data, and a data preprocessing method in a special visual field is selected to perform data enhancement, supplementation, and denoising. This can effectively solve the problems of unbalanced distribution of original data and susceptibility of data to noise interference;

[0067] (5) The present invention's ultrasonic positioning and identification method for rail damage and its computer-readable storage medium newly classifies garbage when determining damage categories, and can distinguish between damage and non-damage or a small number of confusing samples at the edge of noise classification, thus resolving the technical problem that some targets and noise are difficult to distinguish using classification networks. Setting noise categories can prevent the classification network from segmenting scattered noise and B-display echo points that do not conform to the damage extension law due to inaccurate segmentation;

[0068] (6) The ultrasonic positioning and identification method for rail damage and the computer-readable storage medium of the present application, by constructing expert rules through artificially designed features, can combine the basic time series and image information to determine the final damage category for targets that are difficult for the classifier to distinguish in the garbage category, thereby refining the damage identification results and ensuring that the identification results are more in line with actual operational requirements. At the same time, through multimodal data processing, it can not only make up for the technical problems of difficulty in removing noise from image data and the insensitivity of the secondary classifier to noise, but also make up for the problem of difficulty in manually designing accurate features by relying solely on time series, resulting in abnormal cutting of the target area and the lack of clear boundary surfaces. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] To more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other embodiments can be derived from these drawings without inventive effort.

[0070] Figure 1 This is a schematic diagram of the structure of an existing rail damage ultrasonic detection system;

[0071] Figure 2 It is a schematic diagram of the prior art ultrasonic wafer detection process;

[0072] Figure 3 This is a waveform diagram of the 0-degree ultrasonic chip excitation signal and the A-type display signal of the existing rail damage ultrasonic detection system;

[0073] Figure 4 This is a schematic diagram of the B-type display interface of the existing rail damage ultrasonic detection system;

[0074] Figure 5 This is a schematic diagram of the definition of channel echo points in the B-type diagram of rail damage detection in the prior art;

[0075] Figure 6 This is a schematic diagram of the "A"-shaped detection pattern of the bolt hole in the B-type diagram of the existing rail damage detection technology;

[0076] Figure 7 This is a schematic block diagram of the structure of a specific embodiment of the rail damage ultrasonic positioning and identification system based on the method of the present application;

[0077] Figure 8 This is a flowchart of a specific embodiment of the method for ultrasonic positioning and identification of rail damage in the present application;

[0078] Figure 9 It is a waveform display diagram showing the ultrasonic echo signal pulse position presentation form and time series presentation form;

[0079] Figure 10 This is a flow chart of the data preprocessing process in a specific embodiment of the rail damage ultrasonic positioning and identification method of the present application;

[0080] Figure 11 This is a flow chart of the data preprocessing process in another specific embodiment of the rail damage ultrasonic positioning and identification method of the present application;

[0081] Figure 12 This is a flow chart of the data preprocessing process in another specific embodiment of the method for ultrasonic positioning and identification of rail damage in the present application;

[0082] Figure 13 This is a flow chart of the data preprocessing process in another specific embodiment of the rail damage ultrasonic positioning and identification method of the present application;

[0083] Figure 14 This is a flow chart of the time series data preprocessing process in a specific embodiment of the rail damage ultrasonic positioning and identification method of the present application;

[0084] Figure 15 This is a block diagram of the structure of a damage location module in a specific embodiment of the rail damage ultrasonic location and identification system on which the method of this application is based;

[0085] Figure 16 This is a flow chart of a damage location process in a specific embodiment of the rail damage ultrasonic location and identification method of the present application;

[0086] Figure 17 This is a block diagram of the structural composition of the classification network unit (residual basic structure) of the damage classification module in a specific embodiment of the rail damage ultrasonic positioning and identification system based on the method of the present application;

[0087] Figure 18 This is a flow chart of the damage classification process in a specific embodiment of the rail damage ultrasonic positioning and identification method of the present application;

[0088] Figure 19 This is a block diagram of the structure of a damage classification module in a specific embodiment of the rail damage ultrasonic positioning and identification system on which the method of this application is based;

[0089] Figure 20 This is a flowchart of the expert rule judgment process in a specific embodiment of the rail damage ultrasonic positioning and identification method of the present application;

[0090] Figure 21 This is a flowchart of the expert rule judgment process in another specific embodiment of the rail damage ultrasonic positioning and identification method of the present application;

[0091] Figure 22 This is a flowchart of the expert rule judgment process in another specific embodiment of the rail damage ultrasonic positioning and identification method of the present application;

[0092] In the figure: 1-signal conversion module, 2-data preprocessing module, 3-damage location module, 4-damage classification module, 5-expert rule judgment module, 6-ultrasonic rail flaw detection vehicle, 7-rail damage detection system, 8-rail damage analysis system, 9-flaw detection device, 10-rail damage ultrasonic location and identification system, 11-probe wheel, 12-ultrasonic chip, 13-rail, 14-residual basic structure, 15-first residual structure combination, 16-second residual structure combination, 17-third residual structure combination, 18-fourth residual structure combination. DETAILED DESCRIPTION

[0093] To make the purpose, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0094] As attached Figure 1 To the attached Figure 22 As shown, a specific embodiment of the rail damage ultrasonic positioning and identification method and computer-readable storage medium of the present application is given. The present application is further explained below in conjunction with the drawings and specific embodiments.

[0095] This application mainly relates to deep learning and computer image segmentation technology, and in particular to an intelligent positioning and classification method for internal rail damage. Through deep learning image segmentation technology, the position of internal rail damage can be efficiently and accurately located, and the category of damage can be automatically identified. This application provides a rail damage ultrasonic positioning and identification method based on UNet image segmentation and precise classification, and a computer-readable storage medium. The artificial intelligence model is used to accurately divide the damage and then determine the damage. This end-to-end automatic positioning and identification method does not require excessive manual intervention, and can improve the damage detection rate in rail flaw detection operations and shorten the detection cycle. Compared with existing rail ultrasonic flaw detection methods, it has the advantages of a short flaw detection cycle, a high damage detection rate, and accurate and fast positioning, which plays an important role in the safe maintenance and operation of railways.

[0096] Example 1

[0097] As attached Figure 8 As shown, an embodiment of the method for ultrasonic positioning and identification of rail damage of the present application specifically includes the following steps:

[0098] S11) converting the rail ultrasonic detection signal into ultrasonic time series data and visually visible B-display image data;

[0099] S12) preprocessing the ultrasonic time series data and B-display image data generated in step S11);

[0100] S13) using an image segmentation network to perform semantic segmentation on the B-display image data processed in step S12) and generate segmented sub-regions to achieve damage localization;

[0101] S14) using a classification network to identify damage categories of targets within the segmented sub-regions generated in step S13);

[0102] S15) Using expert rules to perform feature extraction and post-processing on the classification results of step S14), and refine the damage identification results obtained in step S14).

[0103] The ultrasonic positioning and identification method for rail damage in this application first converts the rail ultrasonic detection signal into an ultrasonic time series and B-display image data. During data processing, data preprocessing is performed by combining image data and ultrasonic time series. The reason for using ultrasonic time series for data preprocessing is that time series information contains a lot of easily segmented information, which facilitates operations such as data interception during the data preprocessing process. After data preprocessing is completed, the UNet network is used to generate an image for target segmentation. The segmentation results are then cut into blocks and passed through a secondary classifier in the form of regional sub-graphs to finally determine the damage category represented by the regional sub-graph.

[0104] The input data in step S11) is the ultrasonic echo signal data collected during the rail flaw detection operation. The B-display image data and time series data are data forms that are presented after secondary processing based on the original ultrasonic echo signal data, and can be generated at the same time. The original ultrasonic echo signal is expressed in the form of (x, y) coordinates, which only indicates where the pulse is located and at what depth there is an echo point, without any correlation before and after time. It is unknown whether there is data for the previous pulse. The time series introduces the concept of time on this basis, indicating that the echo signal at a certain position is correlated with the time points before and after it. As shown in the attached figure, Figure 9 As shown, the difference between the two data presentation forms of the original ultrasonic echo signal and the time series is demonstrated. When presented in the form of a time series, the horizontal axis represents the moment. Only then can the current moment be associated with the previous moment, thereby measuring whether the ultrasonic echo is stable.

[0105] Step S11) further includes the following process:

[0106] Ultrasonic detection signals within several unit pulse lengths are used as the data contained in a frame of image, so as to convert the original ultrasonic detection signal data into several image data, and continuously input them into the subsequent steps for data preprocessing in the form of data stream.

[0107] The large-scale rail flaw detection vehicle 6 continuously scans the inside of the rail 13 through ultrasonic chips 12 with different emission directions and frequencies, and detects damage based on the principle that ultrasonic waves will be reflected when encountering an inhomogeneous medium. In order to facilitate playback personnel to locate the damage position, the reflection state inside the rail 13 is usually displayed in the form of a B-display image. This application uses images to intuitively display the specific situation of the rail B-display data. In order to ensure the quality of the data, in the process of segmenting the original ultrasonic echo signal data, it is taken into account that in the process of converting the ultrasonic signal into image data, the image size cannot be too large, because a large image size will lead to the problem of consuming too many computer resources for image generation and reading. Therefore, this application specifically uses ultrasonic signals within a pulse length of 960 units (0.6 meters actual length) as the data contained in a frame of B-display image, that is, a B-display image only contains data within the range of 960 pulse lengths.

[0108] The data preprocessing step can effectively increase the data volume and balance the data distribution to a certain extent. The data preprocessing process in step S12) includes processing of the ultrasonic time series data and B-display image data, including enhancement, supplementation, or denoising. Through data preprocessing, the distance within the data class is reduced and the distance between classes is increased on the basis of balancing the data distribution, which can significantly improve the data quality. Due to the single background of the rail ultrasonic echo B-display image, this application mainly adopts the following methods for B-display image data preprocessing in accordance with the actual needs of the rail flaw detection task:

[0109] The first pre-processing method in step S12) further comprises the following steps:

[0110] The data where the bottom wave disappears is directly deleted based on the original ultrasonic echo signal data and ultrasonic time series data. In the B-display image data, the bottom wave disappearance channel is very similar to the 0-degree channel, and the bottom wave disappearance is only an auxiliary condition for damage judgment. In the subsequent model processing process, the bottom wave disappearance class and the horizontal crack class are easily misclassified, which reduces the accuracy of damage classification. Therefore, it is necessary to delete the bottom wave disappearance channel information. There are two ways to delete: First, since the B-display image is a visual image data drawn according to the coordinates based on the actual ultrasonic signal, the ultrasonic signal data can be directly segmented, the bottom wave disappearance data can be deleted, and then the B-display image data can be redrawn. Second, the bottom wave disappearance data can be erased by using a mask or other method on the drawn B-display image data, but the mask design and erasure operations are cumbersome and redundant. Therefore, it will be more convenient to directly delete the signal from the data end and then redraw the image. At the same time, deleting the bottom wave disappearance channel can avoid misclassification with horizontal cracks. The specific operation process example is shown in the attached figure. Figure 10 shown.

[0111] Since in the actual environment, the number of damages in the rail 13 is relatively small, while other objects such as normal screw holes and joints are relatively large, some damage data is collected manually. For example, the second pre-processing method in step S12) can further include the following process:

[0112] This application determines the damage type based on the existing ultrasonic detection signal, and cuts off the damage echo point from the original time-series ultrasonic sequence according to the coordinate area. According to the range of the damage echo point, the vertical coordinate (depth) is constant and the horizontal coordinate (distance) is random, and the whole is drawn into the preset background image to simulate the damage data under different backgrounds, thereby realizing data amplification. The specific example is shown in the attached figure. Figure 11 In the preprocessing method, some damage can also be simulated and created according to the extension law of the damage, and part of the data can be extracted and reconstructed by data segmentation to form new damage, which is then drawn on the background canvas according to the legend of the channel echo point. This is equivalent to performing related cutting and copying operations on the time series, and finally presenting it in the form of a B-display image, thereby achieving data amplification. A specific example is shown in the attached Figure 12 As shown in the figure, this method makes it difficult to successfully perform operations such as channel shifting and adding signal points using a single B-mode image. This is because echo point signals are often clustered together in B-mode echo signal data, making them difficult to separate. Therefore, it is more convenient to redraw the B-mode image using coordinate points and echo channel information based on time series signals.

[0113] The third pre-processing method in step S12) further comprises the following steps:

[0114] Use masks of 3*3, 5*5 and 7*7 sizes respectively and use a sliding window method to mask the content of the B-display image data according to the mask step size, and ensure that the masked part of the selected size does not affect the judgment of the damage. In order to enhance the generalization ability of the later model, this application adopts a mosaic method, that is, use masks of 3*3, 5*5, and 7*7 sizes respectively and use a sliding window method to mask the content of the B-display image according to the mask step size. The masked part of the selected size will not affect the judgment of the damage. The final B-display image is presented in a mosaic effect similar to a chessboard. The method of partially blocking area blocks can diversify the data and balance the distribution of the data. At the same time, it can allow the model to understand more advanced semantic information. Specific examples are shown in the attached. Figure 13 shown.

[0115] For the data preprocessing steps specific to time series, the most important consideration is the stationarity of the time series signal and minimizing the noise interference of the time series. The noise in the time series presented by the ultrasonic echo signal is mainly manifested as unexplainable echo points with no statistical regularity. The time series presented by the ultrasonic echo signal is stationary, which is the data property determined by the underlying ultrasonic acquisition process. In reducing the impact of noise on the time series signal, the present application provides an envelope averaging method to further reduce the impact of noise, that is, performing a fast Fourier transform (FFT) on the signal and retaining the unilateral spectrum of the signal for drawing the envelope of the signal. The fourth preprocessing method in step S12) further includes the following steps:

[0116] Perform Hilbert transform on ultrasonic time series data to obtain analytical signal, retain the signal unilateral spectrum to draw the signal envelope, and reduce the noise interference of time series. Hilbert transform can be expressed as x a =F -1 (F(x)2U), where x a represents the Hilbert analytic signal, F(·) and F -1 (·) represents Fourier transform and inverse Fourier transform, U represents the unit step signal, and x represents the original signal. After the envelope of the analytical signal is determined, the mean value of the envelope at each time point is calculated using the values ​​of the upper envelope and the lower envelope at each time point. After this processing, some long-term continuous noise echoes will be suppressed by the envelope averaging method. The specific operation example is shown in the attached figure. Figure 14 shown.

[0117] Step S13) mainly completes UNet image segmentation and mask acquisition. The image segmentation network in step S13) further adopts UNet network. UNet network uses B display image as input data. UNet network is an image segmentation network based on CNN (convolutional neural network). Its UNet network structure is as shown in the attached figure. Figure 15 As shown, it mainly consists of a convolutional layer, a maximum pooling layer (downsampling), a deconvolution layer (upsampling) and a ReLU nonlinear activation function. Step S13) further includes the following process:

[0118] i) Maximum pooling layer, downsampling process:

[0119] S131) ​​The B-display image data processed in step S12) is used as the input feature map, which is first scaled to a size of 572×572×1, and then convolved twice with a 3×3×64 convolution kernel to a size of 568×568×64, and then subjected to a 2×2 maximum pooling operation to a size of 284×284×64 (the 3×3 convolution is generally followed by a ReLU nonlinear activation function, attached). Figure 15For the convenience of description (not shown), a downsampling operation is completed.

[0120] According to the above step S131), the 3×3 convolution and 2×2 pooling operations are repeated three times. After each maximum pooling operation, the number of convolution kernels of the first 3×3 convolution operation is doubled.

[0121] S132) After reaching the fourth maximum pooling at the bottom layer, the feature map becomes 32×32×512 in size, and then performs two 3×3×1024 convolution operations, finally becoming 28×28×1024 in size.

[0122] ii) Deconvolution layer, upsampling process:

[0123] S133) The upsampling operation is performed on a feature map based on a size of 28×28×1024. First, a 2×2×512 deconvolution operation is performed to change the feature map to a size of 56×56×512. Then, the image before the corresponding maximum pooling layer is copied and cropped (copy and crop), and the image obtained by deconvolution is spliced ​​according to the channel dimension to obtain a feature map of size 56×56×1024. Then, a 3×3×512 convolution operation is performed to complete an upsampling operation.

[0124] According to the above step S133), the 2×2 deconvolution and 3×3 convolution operations are repeated three times, and the number of convolution kernels in the first 3×3 convolution operation after each splicing is reduced exponentially.

[0125] S134) After reaching the fourth deconvolution at the top layer, the feature map becomes 392×392×64 in size, is copied and cropped, and then concatenated to a size of 392×392×128, and then two 3×3×64 convolution operations are performed.

[0126] S135) obtains a feature map of size 388×388×64, and finally performs a 1×1×2 convolution operation to obtain the segmentation area information of the B-display image.

[0127] The UNet is a network model that uses a U-shaped structure to capture contextual and positional information. In the first half of the UNet, it performs four downsampling steps using a convolution (Conv) + max pooling method. This downsampling process extracts shallow features such as texture and shape. As the convolution proceeds, higher-level semantic features of the B-display image are continuously extracted. The highest convolutional layer essentially describes the most interesting regions of the original B-display image. In the second half of the UNet, layer-by-layer upsampling is used to gradually restore the regions of interest back to the original B-display image. The convolution step can be viewed as repeated processing of the data, continuously extracting useful information, while upsampling can be considered the inverse of downsampling. Therefore, the UNet can also be viewed as an "encoder-decoder" operation.

[0128] In the UNet network, the image's edge texture information is continuously lost through the downsampling step of "convolution + max pooling." Although the upsampling step is the inverse of downsampling, the interpolated values ​​may contain spurious values. Therefore, in order to obtain refined edge information, the feature maps from the downsampling step are fused using a channel-wise splicing method during each upsampling process to obtain accurate edge information. This channel-wise splicing method provides richer descriptive features, which can better adapt to pixel-level segmentation tasks. However, this richness in features also increases computational resource and memory consumption. After analysis by the UNet network, a mask corresponding to the initial image is generated. This mask information can be used to identify objects in the original B-display image data. However, UNet region segmentation may result in edge discontinuities. Therefore, the mask is first dilated and then eroded to remove black cores and obtain clear edges. The corresponding objects are then segmented in the original B-display image to form object information of varying sizes.

[0129] As attached Figure 16 In the UNet network structure shown, the downsampling processes in layers 1-4 can be considered as a single entity, differing only in the size and information contained in the feature maps after downsampling. The entire downsampling process continuously extracts high-level image semantic features, moving from perceiving information such as color and texture within the image to perceiving specific object categories within the image. The four-layer upsampling process in the latter half can also be considered as a single entity, continuously interpolating and perceiving image segmentation edges. The direct connection from downsampling to upsampling feature maps primarily serves to make segmentation edges more specific and consistent with the actual state of the original image. The upsampling process is equivalent to continuously inserting pseudo-values ​​into the feature map from the previous step, resulting in blurred edges that indicate the extent of the segmentation area.

[0130] Although as a preferred embodiment, this application specifically uses the UNet network to implement image segmentation, the image segmentation network can also use MaskRCNN, FCN and other network forms. MaskRCNN is an RCNN (i.e., candidate region

[0131] +CNN) divides the network head into two parts: one is to use FCN to segment the recommended area, and the other is the head for classification. The original design of RCNN uses a two-step classification network. First, it recommends the region of interest where the target exists from the entire image, and then performs relevant target recognition on the region of interest. The entire network structure is wide and deep. Processing an image takes 2 to 3 times the time of the UNet segmentation network. In addition, the background of the B-display image data in the current intelligent identification task of rail damage is not complicated. The use of MaskRCNN will result in many blank background areas when providing candidate areas, which wastes the computer's computing power. The task difficulty does not require the use of MaskRCNN to implement the segmentation task. The FCN network obtains a feature map through a series of convolution and pooling operations, that is, the high-level semantics of the image are restored to the segmented area of ​​the image through deconvolution. In order to obtain accurate image segmentation edges, the last three layers of feature maps are retained by cascading them forward and backward. However, this method does not preserve details well because it always loses information during the convolution and pooling processes. In addition, the FCN network does not consider the relationship between pixels at all. Compared with the UNet network structure, it ignores the spatial regularization step commonly used in pixel-based classification segmentation methods and lacks spatial consistency.

[0132] According to the segmented area information obtained in step S13), each individual target block is obtained as an irregularly shaped B-display image block. The segmented B-display block is drawn onto a black background (white is used as the background and black as the image in the accompanying figure for a more intuitive expression) to generate a new sub-block image. The final damage type is then identified through the designed secondary classifier.

[0133] The classification network in step S14) uses the ResNet network as a secondary classifier. The secondary classifier uses B-display image data, and the time series data will be used together as the basis for determining the type of damage in the subsequent expert rule judgment part. This application uses the ResNet18 network as a secondary classifier. The ResNet18 network is a lightweight model with a fast forward reasoning speed. In addition, the ResNet18 network introduces a residual connection method, which can avoid the gradient disappearance problem during the training process. The network is designed as: H(x) = F(x) + x, where x is the output of the previous layer, and H(x) is the output after residual processing. At this time, as long as F(x) = 0, an identity mapping H(x) = x is formed. Here, F(x) is the residual, and its structure is as shown in the attached figure. Figure 17As shown. The input x is the feature map output by the previous layer, and H(x) is the output of the residual structure (also called residual block) connected by shortcuts. At this time, the basic information of the input x is included, which ensures that the feature extraction and optimization process will not be worse than the input x due to information loss. At the same time, the F(x) residual block is added, where the Weight Layer represents the convolution layer, ReLU is the activation function layer for nonlinear mapping, and Batch Normalization is the batch normalization layer. F(x) is used to provide richer features and push the network training to the optimal loss situation. The actual data processing steps of the ResNet18 network and the situations to be avoided are shown in the attached figure. Figure 18 shown.

[0134] As attached Figure 19 As shown, step S14) further includes the following process:

[0135] S141) The sub-block image of the segmented sub-region is scaled to a size of 224×224×3, and a convolution operation is first performed. The convolution operation has a convolution kernel of 3×3, a stride of 2, a padding of 3, an output channel of 64, and an output feature map size of 112×112×64.

[0136] The output data is then processed by the residual network several times, with the change in the number of feature map channels as a large residual structure combination, and the output dimension of each part is calculated with a large residual structure combination.

[0137] S1421) The first residual structure combination 15 includes two residual base structures 14. The output feature map first undergoes two 3×3×64 convolution operations, with a convolution kernel of 3×3, a stride of 1, a padding of 1, and a number of channels of 64. The output feature map is added to the input feature map of the residual base structure 14 in a channel-by-channel addition manner. The second 3×3×64 convolution operation is performed twice, with a convolution kernel of 3×3, a stride of 1, a padding of 1, and a number of channels of 64. The output feature map is added to the input feature map of the residual base structure 14 in a channel-by-channel addition manner. At this time, the feature map output by the first residual structure combination 15 has a size of 56×56×64.

[0138] S1422) The second residual structure combination 16 is composed of two residual basic structures 14. The feature map output by the first residual structure combination 15 is first subjected to two 3×3×128 convolution operations. The convolution kernel of the convolution operation is 3×3, the step size is 1, the padding is 1, and 128 kernel functions are used. A 1×1×128 convolution operation is performed between the two convolution operations, with a step size of 2 and a padding of 0 to reduce the size of the feature map by half, and at the same time, it is added to the input feature map of the residual basic structure 14 in a channel-by-channel addition manner. Two more 3×3×128 convolution operations are performed, with a convolution kernel of 3×3, a step size of 1, a padding of 1, and a 128 kernel function. At the same time, it is added to the input feature map of the residual basic structure 14 in a channel-by-channel addition manner. At this time, the dimension of the feature map output by the second residual structure combination 16 is 28×28×128;

[0139] S1423) The third residual structure combination 17 includes two residual basic structures 14. The feature map output by the second residual structure combination 16 is first subjected to two 3×3×256 convolution operations. The convolution kernel of the convolution operation is 3×3, the step size is 1, the padding is 1, and 256 kernel functions are used. A 1×1×256 operation is performed between the two convolution operations, with a step size of 2 and a padding of 0 to reduce the size of the feature map by half, and at the same time, it is added to the input feature map of the residual basic structure 14 in a channel addition manner. The second time, two 3×3×256 convolution operations are performed again. The convolution kernel of the convolution operation is 3×3, the step size is 1, the padding is 1, and 256 kernel functions are used. At the same time, it is added to the input feature map of the residual basic structure 14 in a channel addition manner. At this time, the output feature map size of the third residual structure combination 17 is 14×14×256.

[0140] S1424) The fourth residual structure combination 18 is composed of two residual basic structures 14. The feature map output by the third residual structure combination 17 is first subjected to two 3×3×512 convolution operations. The convolution kernel of the convolution operation is 3×3, the step size is 1, the padding is 1, and 512 kernel functions are used. A 1×1×512 operation is performed between the two convolution operations, with a step size of 2 and a padding of 0 to reduce the size of the feature map by half, and at the same time, it is added to the input feature map of the residual basic structure 14 in a channel addition manner. The second time, two 3×3×512 convolution operations are performed again. The convolution kernel of the convolution operation is 3×3, the step size is 1, the padding is 1, and 512 kernel functions are used. At the same time, it is added to the input feature map of the residual basic structure 14 in a channel addition manner. At this time, the output feature map size of the fourth residual structure combination 18 is 7×7×512;

[0141] S143) The feature map outputted in step S141) is average-pooled after being combined with four residual structures, and the output data size is 1×1×512. Finally, the output is outputted through a fully connected layer to several categories to which the classification belongs, and the damage category and its probability value of the target area are finally outputted through an activation function.

[0142] Although as a preferred embodiment, this application specifically uses the ResNet network to achieve target recognition, the classification network can also use networks such as Googlenet and Vggnet. Googlenet expands the network width to the extreme, and uses a modular form to divide the entire network into a small block. However, due to its huge width, Googlenet has a particularly large number of neurons. When the amount of data is small, the use of Googlenet usually causes gradient vanishing or direct overfitting. Vggnet basically uses a direct connection method to expand the depth of the network model, but it does not have a residual structure similar to the ResNet network, which may cause useful features to slowly disappear in the ever-deepening convolution, resulting in the problem of gradient vanishing. Moreover, for problems with simple content, the execution efficiency of the Vggnet network will be 2 to 3 times lower than that of the ResNet network.

[0143] During sample sorting, the initial sample categories set include damage within the rail 13 and frequently occurring targets. Furthermore, a noise category is specifically defined in this embodiment. This is to prevent the UNet network from segmenting out scattered noise and B-display image echo points that do not conform to the damage extension pattern due to inaccurate segmentation. However, in image processing research and real-world environments, noise quantification is extremely difficult, and noise modeling is typically not performed. However, establishing noise categories is essential because deep convolutional neural networks have a large number of neurons, each of which can represent a feature expression. Therefore, although noise quantification and modeling are difficult, through the continuous weighted fusion of numerous neurons, a hyperplane can be found to distinguish between noise and useful targets with significantly different feature descriptions. The goal of this application at this stage is to reduce noise interference, not to completely eliminate it. Any remaining difficult-to-analyze noise and targets should be analyzed with the assistance of other processing methods. The purpose of adding noise categories in this application is to distinguish obvious noise and to guide the subsequent processing to focus on data that is difficult to distinguish between noise and targets. In order to further solve the technical problem that some targets and noise are difficult to distinguish by using classification networks, this application specially divides a new "junk class" during the data sorting process. The distinction between the "junk class" refers to a small number of confused samples on the edge of the classification of damage and non-damage or noise categories. In the above steps, obvious noise has been distinguished, but the noise that is difficult to distinguish may contain useful signals. After using the ResNet18 network for damage classification, each detection sample can obtain a category confidence, that is, the probability value of the detection sample belonging to a certain detection category. In order to ensure the classification accuracy of the ResNet18 network, samples with low confidence values ​​are classified as "junk class", which is equivalent to using the scoring rules to first determine that the ResNet18 network can definitely detect the correct samples, and then use expert rules to judge the target category for the "junk class", thereby making more accurate target category determination. In the "junk class" determined by this application, it is generally samples that contain noise signals when the UNet network performs image segmentation, resulting in the real target to be identified being covered by noise. During forward inference, the ResNet18 network incorporates noise visual features into the learning iterations, resulting in a discrepancy between the learned features and the true target feature distribution, making inference difficult. This also illustrates that the "junk" classification task is not only performed by the ResNet18 network, but also requires the assistance of some practical rules to assist in the classification process. Step S14 further includes:

[0144] After damage classification using the ResNet18 network, each sub-block image detection sample is given a category confidence, that is, the probability value of the detection sample belonging to a certain target category, denoted as v obj_conf, preset the probability value v of the detection sample obj_conf ≥thresh, the classification network can make accurate judgments, and the thresh value can be customized according to different situations. obj_conf When the confidence level is less than thresh, the sample with low confidence is classified as junk, indicating that the target category of the current sample needs additional information to assist in the judgment. Then, the target category is judged based on the expert rules for the junk class to accurately determine the target category.

[0145] The expert rules used in this application only determine the damage category from a visual perspective in the process of determining the damage location by the UNet network and the damage category by the ResNet18 network, which may not meet the actual operational requirements. Therefore, this application also requires manual design of features and construction of expert rules to further refine the classification of the detection results of each type of damage. The input of the expert rules is further image data or ultrasonic time series data. Through the expert rules, the garbage targets that are difficult to classify by the classification network are combined with the basic ultrasonic time series data and image data to jointly determine the final damage category.

[0146] Because noise often occurs in the rail head channel, it is easy to confuse the judgment of core damage and noise. Taking the judgment of rail head core damage as an example, the expert rules in step S15) further include:

[0147] Based on the ultrasonic time series data, the fitting straight line of all channel echo points in the target area is calculated (the fitting straight line indicates that the sum of the Euclidean distances of all channel echo points to the straight line is the smallest). After the fitting straight line is determined, the total slope is calculated. Wherein, Δy and Δx represent the vertical coordinate increment and the horizontal coordinate increment of the damage echo point in the target area, respectively.

[0148] Based on the ultrasonic time series data, the fitting line of the echo point of a certain channel in the target area is calculated (the fitting line indicates that the sum of the Euclidean distances of all channel echo points to the line is the smallest). After determining the fitting line, the slope of a certain channel is calculated. Δy c and Δx c They respectively represent the horizontal and vertical increments of the damage echo point belonging to a certain channel.

[0149] Only when the slope k of each channel c When the total slope k of all channels does not meet any of the judgment conditions of rail head core damage, rail waist damage, and rail bottom damage, the current target is judged to be noise under the trend condition.

[0150] Here, k and k for each channel must be calculated c The reason is that if a singular k cWhen the value is large, k is likely to be affected by the value; and each channel in the target frame may have anomalies, so in order to make up for their respective defects, k and the k of each channel must be c Calculate it.

[0151] The specific steps for obtaining the fitting line and slope are as follows: let the original echo signal be p = (x i ,y i ),i=1,2,···,n. Let the unit vector in the direction of the straight line be I, and the unit vector in the direction of the normal be n. Assume that a point (x0,y0) on the fitting line can be taken as the origin of the coordinates, and the coordinate value p after translation is obtained i =(x i -x0,y i -y o ). Since I⊥n, the sum of the squares of the perpendicular distances from the point to the line is expressed as ∑||p i || 2 =∑[(n T p i ) 2 +(I T p i ) 2 ]=∑n T p i p i T n+∑I T p i p i T I=n T ∑p i p i T n+I T ∑p i p i T I, let S = ∑p i p i T , then ∑||p i || 2 =n T Sn+I T SI. Where n T Sn is the sum of the squares of the vertical distances from the point to the line, I T SI is the sum of the squares of the projection lengths of a point onto a line. The goal is to find I and n so that n T Sn is the smallest and I T SI is the largest.

[0152] Since S is a second-order symmetric positive definite matrix, there are two positive eigenvalues ​​λ max ≥λ min According to the properties of the second-order symmetric positive definite matrix, we know that nT Sn=λ min And I T SI=λ max When , the direction of the fitted line meets the requirements.

[0153] To determine the location of the origin (x0, y0), the square sum of the perpendicular distances from the point to the line can be rewritten as Among them, when and When ∑||p i || 2 Minimum, that is to say, the origin of the coordinate can be placed at the center point of the n coordinate points. After determining the fitting line, the overall slope k of the target can be calculated according to the increment of the ordinate Δy / the increment of the abscissa Δx. Similarly, the k of each channel can also be calculated. c .

[0154] In the rail head damage analysis example, damage determination requires a continuously downward trajectory in the rail head channel echo direction corresponding to the channel of the echo signal. If the slope is horizontal or does not meet the slope range for the rail head channel, then the current target trend does not meet the trend conditions for damage determination.

[0155] In addition, for scattered noise, the expert rules in step S15) further include:

[0156] Based on the B-display image data, the density and 1 / 4 density of the current target area are calculated and recorded as Where s represents the area of ​​the current target area, and m represents the number of channel echo points in the current target area. The reason for choosing 1 / 4 density is that the echo points may be unevenly distributed in the area after the B-display image is segmented. Therefore, the current target area is divided into four sub-areas, and the corresponding density distribution is calculated for each sub-area. The specific example is shown in the attached figure. Figure 20 As shown. By dividing the current target area into four sub-areas, the corresponding density distribution is calculated respectively to determine whether it is damaged, and the damaged area is optimized for secondary positioning. The specific method is that in the process of damage judgment, the trend along the channel direction is an important basis for judgment. The density of the current target area is calculated, and then the 1 / 4 density is calculated. If the density calculated in each small block divided by the 1 / 4 density is not much different, it means that the echo points are discretely present in each area of ​​the target frame, and there is no complete corresponding damage trend. If the density in the upper and lower small blocks on the right side of the target frame already accounts for a large part of the density of the entire target frame, then it means that the echo points are concentrated on the right side of the target frame, and the initial target frame may have a large blank area, that is, the target frame is too large. This example only represents a selection condition for density judgment. In actual applications, other density calculation methods can be selected according to actual needs.

[0157] In addition, for targets with similar shapes, such as screw holes, this application also uses morphological analysis methods, using mask matching and channel echo point position matching methods to calculate whether fixed-shaped targets such as screw holes meet the echo point rules and analyze whether they meet the target echo point shape rules. The expert rules in step S15) further include:

[0158] Based on the B-display image data, mask matching and channel echo point position matching are used to calculate and analyze whether the target conforms to the echo point distribution pattern. First, a binarization method is used to separate the target and background components of the target image data. A mask is then slid across the sub-target image to determine whether the size of the unmatched target is abnormal. For example, an "A"-shaped mask can be used to determine whether a screw hole is a "normal screw hole," and an inverted "eight"-shaped mask can be used to determine whether a weld is a "normal weld." For example, for an abnormal screw hole with a lower crack, the mask can be selected as a "normal screw hole" mask for sliding. When the mask is slid to a certain area where the "A"-shaped area of ​​a normal screw hole matches the upper mask, the remaining image portion is calculated. If a large number of rail waist channel echo points still exist below, this indicates an abnormality below and, under the current conditions, it can be determined to be an abnormal screw hole suspected of a lower crack.

[0159] In addition, there are obvious differences in the echo point positions between abnormal screw holes and normal screw holes. By judging the positional relationship between the echo points of the 45-degree and 0-degree channels in two directions, the oblique cracks in the screw holes can be determined. Specific examples are shown in the attached figure. Figure 21 As shown. The expert rules in step S15) further include:

[0160] For screw hole cracks, based on the B-display image data, the positional relationship between the 45-degree channel echo point and the 0-degree channel echo point in the front and rear directions is determined, as well as whether the echo point appears at the crack position of the screw hole to determine whether the screw hole is abnormal. Figure 21 For example, the abnormal screw holes in the image have strict echo point restrictions, whether they are oblique cracks, oblique cracks, or horizontal cracks. For example, in the case of oblique cracks, first determine the normal echo of the normal screw hole "A" shape, which is always 45 degrees backward on the left and 45 degrees forward on the right. There is an intersection between the 0-degree channel and the 45-degree channels forward and backward or their extensions. By judging whether there is a 45-degree forward echo point at the intersection of 0 and 45 degrees backward, the attached Figure 21 The circled portion of the abnormal screw hole is located in the middle. Similarly, the other half can be used to determine whether the screw hole has an upward oblique crack based on whether the echo point meets the echo conditions for damage.

[0161] Expert rule judgment can be used to determine the final damage category of the "junk" class that is difficult for the classifier to distinguish, combined with basic time series information and image graphic information, as shown in the attached figure. Figure 22 As shown. This multimodal data processing method adopted in this application can overcome the technical problems of difficulty in removing noise from image data and the insensitivity of the secondary classifier itself to noise. At the same time, it can also overcome the technical problems of difficulty in accurately establishing artificial features based solely on time series, resulting in abnormal target segmentation and the lack of clear boundary surfaces.

[0162] The ultrasonic positioning and identification method for rail damage described in Example 1 of the present application is based on UNet image segmentation and ResNet combined with expert rule judgment to achieve accurate classification, including steps such as data preprocessing, UNet image segmentation, damage positioning, and damage category determination. The present application first converts the rail ultrasonic detection data into visually visible image data, and then uses visual images to perform target positioning and identification of the ultrasonic rail detection data. In order to solve problems such as unbalanced distribution of raw data and susceptibility to noise interference, data enhancement, supplementation, and denoising are performed through data preprocessing. In order to achieve end-to-end detection of damage within the rail and reduce manual intervention, the UNet network is used to perform semantic segmentation on the B-display image data, and the ResNet network is used to determine the category of targets within the area based on the segmented sub-regions. Relatively accurate target location and category results can be obtained without manual intervention. The present application uses the UNet network for image segmentation, which can obtain more refined segmentation edge results. By utilizing image edge information, some overlapping damages at the same position can be accurately distinguished. At the same time, target recognition technology is used to locate and classify targets, and accurate segmentation contours are not required. In order to ensure that the recognition results are more in line with the requirements of actual operations, this application further uses expert rules to extract features and post-process existing damage, and refine the damage results. From the perspective of image processing, this application uses a deep learning image segmentation network to directly locate damage within the rail, and uses a secondary classifier to determine the damage category, achieving end-to-end rail ultrasonic flaw detection data recognition, and can combine multiple layers of features to identify the nature of the damage. At the same time, this application also overcomes the technical problem that most existing rail damage ultrasonic detection methods require manual feature design and manual decision-making for B-display image data segmentation, greatly shortening the detection cycle of rail ultrasonic flaw detection.

[0163] Example 2

[0164] An embodiment of a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for ultrasonic positioning and identification of rail damage described in Example 1 is implemented.

[0165] Example 3

[0166] As attached Figure 7 As shown, an embodiment of a rail damage ultrasonic positioning and identification system 10 used in the method described in Example 1 specifically includes:

[0167] Data preprocessing module 2, preprocesses the input ultrasonic time series data and B-display image data;

[0168] The damage localization module 3 uses an image segmentation network to perform semantic segmentation on the B-display image data processed by the data preprocessing module 2 and generates segmented sub-regions to achieve damage localization;

[0169] The damage classification module 4 uses a classification network to determine the damage category of the target in the segmented sub-region generated by the damage localization module 3;

[0170] The expert rule judgment module 5 uses expert rules to perform feature extraction and post-processing on the judgment result of the damage classification module 4, and refines the damage identification result obtained by the damage classification module 4.

[0171] The rail damage ultrasonic location and identification system 10 also includes a signal conversion module 1, which converts the rail ultrasonic detection signal into ultrasonic time series data and visually visible B-display image data. The signal conversion module 1 uses the ultrasonic detection signal within a certain unit pulse length as the data contained in a frame of image. This module converts the raw ultrasonic detection signal data into a plurality of image data, which are continuously input into the data preprocessing module 2 in the form of a data stream.

[0172] The data preprocessing module 2 performs processing including enhancement, supplementation, or denoising on the ultrasonic time series data and B-display image data. Through data preprocessing, the distance within the data class becomes smaller and the distance between the data classes becomes larger on the basis of balancing the data distribution. The data preprocessing module 2 mainly preprocesses the data in the following ways:

[0173] The first method is to directly delete the data where the bottom wave disappears based on the ultrasonic time series data, as shown in the attached figure. Figure 10 shown.

[0174] The second method is to extract the damage point data from the original ultrasonic time series data according to the coordinate area based on the damage category determined by the existing ultrasonic detection signal. According to the damage range, the vertical coordinate representing the depth remains unchanged, and the horizontal coordinate representing the distance is randomly drawn into a new preset background map. The damage data under different backgrounds are simulated to perform data amplification, as shown in the attached figure. Figure 11 As shown; or according to the extension law of the damage, partial data is extracted and reconstructed by data segmentation to form new damage and present it in the form of B-display images, thereby achieving data amplification, as shown in the attached Figure 12 shown.

[0175] The third method is to use 3*3, 5*5 and 7*7 size masks respectively and mask the content of the B-display image data in a sliding window manner according to the mask step size, and ensure that the masked part of the selected size does not affect the damage determination, as shown in the attached figure. Figure 13 shown.

[0176] The fourth method is to perform Hilbert transform on ultrasonic time series data to obtain analytical signals, retain the single-sided spectrum of the signal to draw the envelope of the signal, and reduce the noise interference of the time series. The Hilbert transform is expressed as x a =F -1 (F(x)2U), where x a represents the Hilbert analytic signal, F(·) and F -1 (·) represents Fourier transform and inverse Fourier transform, U represents the unit step signal, and x represents the original signal. After the envelope of the analytical signal is determined, the mean value of the envelope at each time point is calculated using the value of the upper envelope and the value of the lower envelope, as shown in the attached figure. Figure 14 shown.

[0177] As a preferred embodiment of the present application, the image segmentation network of the damage location module 3 further specifically adopts a UNet network. The UNet network uses the B-display image as input data. The UNet network is an image segmentation network based on CNN (convolutional neural network). The UNet network structure is shown in the attached figure. Figure 15As shown in the figure, the network mainly consists of a convolutional layer, a maximum pooling layer (downsampling), a deconvolution layer (upsampling), and a ReLU nonlinear activation function. The preprocessed B-display image data is input as a feature map into the damage localization module 3. It is first scaled to a size of 572×572×1, then convolved twice with a 3×3×64 kernel to a size of 568×568×64. Then, a 2×2 maximum pooling operation is performed to a size of 284×284×64, completing a downsampling operation. Following this downsampling process, three 3×3 convolutions and 2×2 pooling operations are repeated. After each maximum pooling operation, the number of convolution kernels in the first 3×3 convolution operation is doubled. After reaching the fourth maximum pooling operation at the bottom layer, the feature map is reduced to 32×32×512. Two more 3×3×1024 convolutions are performed, bringing the final size to 28×28×1024. The upsampling operation is performed on a 28×28×1024 feature map. First, a 2×2×512 deconvolution operation is performed to reduce the feature map size to 56×56×512. The image before the corresponding max pooling layer is then copied and cropped, and concatenated with the deconvolved image along the channel dimension to obtain a 56×56×1024 feature map. A 3×3×512 convolution operation is then performed to complete the upsampling operation. This upsampling process is repeated three times for 2×2 deconvolution and 3×3 convolution. The number of convolution kernels in the first 3×3 convolution after each concatenation is reduced exponentially. After the fourth deconvolution at the top layer, the feature map is reduced to 392×392×64. It is then copied, cropped, and concatenated to 392×392×128, and two more 3×3×64 convolution operations are performed. A feature map of size 388×388×64 is obtained, and finally a 1×1×2 convolution operation is performed to obtain the segmentation area information of the B-display image.

[0178] The damage classification module 4 obtains each individual target block as an irregularly shaped B-display image block based on the segmented area information output by the damage positioning module, uses the smallest circumscribed pure black rectangular image of the B-display image block as the background, draws the segmented area onto the black background to generate a new sub-block image, and then uses the designed secondary classifier to determine the final damage type.

[0179] After damage classification using the classification network, each sub-block image detection sample is given a category confidence, that is, the probability value of the detection sample belonging to a certain target category, denoted as v obj_conf , preset the probability value v of the detection sample obj_conf ≥thresh, the classification network can make accurate judgments, and the thresh value can be customized according to different situations. obj_confWhen the confidence score is less than thresh, the damage classification module 4 classifies the sample with low confidence as spam, indicating that the target category of the current sample requires additional information to determine. Expert rules are then used to determine the target category for the spam class, resulting in an accurate determination of the target category. As a preferred embodiment of the present application, the classification network of the damage classification module 4 further specifically uses a ResNet18 network as a secondary classifier.

[0180] As a typical embodiment of the present application, as shown in the attached Figure 19 As shown in FIG, the damage classification module 4 further specifically includes: a convolutional layer, a first residual structure combination 15, a second residual structure combination 16, a third residual structure combination 17, a fourth residual structure combination 18, a pooling layer and a fully connected layer. Since the network structure of the damage classification module 4 is fixed, the size of each layer of feature map will change with the size of the input image data. Therefore, in the attached Figure 17 、 18and 19 do not specifically limit the input image size and the size of the feature map after each layer of processing. Data scaling and feature map size are only described in the embodiments of this application. The specific structure of the damage classification module 4 is as follows: the sub-block image of the segmented sub-region is scaled to a size of 224×224×3, and is first input into the convolution layer for a convolution operation. The convolution kernel of the convolution operation is 3×3, the step size is 2, the padding is 3, the output channel is 64, and the output feature map size is 112×112×64. The first residual structure combination 15 includes two residual basic structures 14. The output feature map first undergoes two 3×3×64 convolution operations. The convolution kernel of the convolution operation is 3×3, the step size is 1, the padding is 1, and the number of channels is 64. It is added to the input feature map of the residual basic structure 14 in a channel addition manner. The second time, two 3×3×64 convolution operations are performed. The convolution kernel of this convolution operation is 3×3, the stride is 1, the padding is 1, and the number of channels is 64. It is added to the input feature map of the residual basic structure 14 in a channel-by-channel addition manner. At this time, the feature map output by the first residual structure combination 15 is 56×56×64 in size. The second residual structure combination 16 includes two residual basic structures 14. The feature map output by the first residual structure combination 15 first undergoes two 3×3×128 convolution operations. The convolution kernel of this convolution operation is 3×3, the stride is 1, the padding is 1, and 128 kernel functions are used. Between the two convolution operations, a 1×1×128 convolution operation is performed with a stride of 2 and padding of 0 to reduce the size of the feature map by half. At the same time, it is added to the input feature map of the residual basic structure 14 in a channel-by-channel addition manner. Two more 3×3×128 convolution operations are performed, with a convolution kernel of 3×3, a stride of 1, and a padding of 1. 128 kernel functions are used, and the input feature map of the residual base structure 14 is added in a channel-wise manner. At this time, the feature map output by the second residual structure combination 16 is 28×28×128 in size. The third residual structure combination 17 includes two residual base structures 14. The feature map output by the second residual structure combination 16 first undergoes two 3×3×256 convolution operations, with a convolution kernel of 3×3, a stride of 1, a padding of 1, and a 256 kernel function. Between the two convolution operations, a 1×1×256 operation is performed with a stride of 2 and a padding of 0 to reduce the feature map size by half, and the feature map is added in a channel-wise manner to the input feature map of the residual base structure 14. The second time, two 3×3×256 convolution operations are performed. The convolution kernel of the convolution operation is 3×3, the step size is 1, the padding is 1, and 256 kernel functions are used. At the same time, the input feature map of the residual basic structure 14 is added in a channel-by-channel manner. At this time, the output feature map size of the third residual structure combination 17 is 14×14×256.The fourth residual structure combination 18 includes two residual base structures 14. The feature map output by the third residual structure combination 17 first undergoes two 3×3×512 convolution operations. The convolution kernel of this convolution operation is 3×3, the stride is 1, the padding is 1, and 512 kernel functions are used. Between the two convolution operations, a 1×1×512 operation is performed with a stride of 2 and a padding of 0 to reduce the size of the feature map by half. At the same time, it is added to the input feature map of the residual base structure 14 in a channel-by-channel addition manner. The second 3×3×512 convolution operation is performed twice. The convolution kernel of this convolution operation is 3×3, the stride is 1, the padding is 1, and 512 kernel functions are used. At the same time, it is added to the input feature map of the residual base structure 14 in a channel-by-channel addition manner. At this time, the output feature map size of the fourth residual structure combination 18 is 7×7×512. The feature map output by the convolutional layer is combined through four residual structures and then output to the pooling layer for average pooling operation. The output data size is 1×1×512. Finally, it is output through the fully connected layer to several categories to which the classification belongs. The activation function finally outputs the damage category and probability value of the target area.

[0181] The input of the expert rule is image data or ultrasonic time series data. The expert rule judgment module 5 uses the expert rule to combine the garbage target that is difficult to classify by the classification network with the basic ultrasonic time series data and image data to jointly determine the final damage category, as shown in the attached figure. Figure 22 shown.

[0182] The expert rules in the expert rule judgment module 5 further include:

[0183] Calculate the fitting line of all channel echo points in the target area based on the ultrasonic time series data. After determining the fitting line, calculate the total slope Wherein, Δy and Δx represent the vertical coordinate increment and the horizontal coordinate increment of the damage echo point in the target area, respectively.

[0184] Based on the ultrasonic time series data, the fitting line of the echo point of a channel in the target area is calculated. After the fitting line is determined, the slope of a channel is calculated. Δy c and Δx c They respectively represent the horizontal and vertical increments of the damage echo point belonging to a certain channel.

[0185] Only when the slope k of each channel c When the total slope k of all channels does not meet any of the judgment conditions of rail head core damage, rail waist damage, and rail bottom damage, the current target is judged to be noise under the trend condition.

[0186] The expert rules in the expert rule judgment module 5 further include:

[0187] For scattered point noise, the density and 1 / 4 density of the current target area are obtained based on the B-display image data, which are recorded as Where s represents the area of ​​the current target area, and m represents the number of channel echo points in the current target area. By dividing the current target area into four sub-areas, the corresponding density distribution is calculated respectively to determine whether it is damaged, and the damaged area is optimized for secondary positioning, as shown in the attached figure. Figure 20 shown.

[0188] The expert rules in the expert rule judgment module 5 further include:

[0189] For targets with similar shapes, mask matching and channel echo point position matching are used based on B-display image data to calculate and analyze whether the target conforms to the echo point distribution pattern. First, a binarization method is used to separate the target and background parts of the target image data. Then, a mask is slid across the sub-target image to determine whether the size of the unmatched target is abnormal.

[0190] The expert rules in the expert rule judgment module 5 further include:

[0191] For screw hole cracks, based on the B-display image data, the positional relationship between the 45-degree channel echo point and the 0-degree channel echo point in the front and rear directions is determined, as well as whether the echo point appears at the crack position of the screw hole. Figure 21 shown.

[0192] Compared with the comparative document 1, the application scenarios of the ultrasonic positioning and identification method of rail damage and the computer-readable storage medium described in the specific embodiment of this application are different, and the difference in application scenarios is also reflected in the difference in data properties and presentation forms. The main application object of this application is the ultrasonic data collected during the actual flaw detection operation, which reflects the internal situation of the rail and is converted into a B-display image through visualization. The ultrasonic positioning and identification method of rail damage and the computer-readable storage medium proposed in this application can simultaneously process multi-modal data, and generate the final damage location based on the special data properties of each mode. This application differs from the comparative document 1 in terms of data collection and presentation, data form diversity and data processing methods. In addition, in terms of the overall architecture, the main framework of this application focuses on setting data preprocessing, using the UNet model for initial damage positioning, and designing a secondary classification network to determine the category of the target. At the same time, an expert rule module is assisted in the design. The aforementioned multi-data and multi-means approach is used to make the damage detection results more realistic.

[0193] Compared to Reference 2, the rail damage ultrasonic positioning and identification method and computer-readable storage medium described in the specific embodiments of this application utilize a UNet network to first identify all targets along the entire line based on instance segmentation, while also considering the collaborative analysis of B-display image data and time series signals. Furthermore, the expert rules designed in this application are effective in discriminating noise within the original time series signals, and they comprehensively consider the properties of multimodal data, making the resulting results more consistent with actual operational conditions.

[0194] Compared to Reference 3, the rail damage ultrasonic location and identification method and computer-readable storage medium described in the specific embodiments of this application use time-series data and B-display data to reflect the internal conditions of the rail. However, these differences in data can affect the technical approach used for subsequent processing and the methods used to process the data. For example, in this application, because the B-display data is based on the rail ultrasonic acquisition data and does not involve the influence of image brightness during the acquisition process, brightness equalization methods are not used to enhance the data during the image preprocessing step.

[0195] Compared with Comparative Document 4, the ultrasonic positioning and identification method for rail damage and the computer-readable storage medium described in the specific embodiment of this application use UNet instance segmentation technology to determine the location of abnormal targets in the rails, and use ResNet to determine the category of damage, avoiding the participation of artificial features in the positioning and identification parts. Artificial features must be strictly reliable in the task, and extracting suitable features is not easy. At the same time, this application does not involve a fully connected layer in the UNet network, so there will be no problem of large-scale dimensionality reduction in the processing of data, and the segmentation edges are preserved relatively completely. In addition, in order to make the final damage more realistic, this application also designs an expert rule module based on multimodal data after the target category is confirmed.

[0196] In the description of this application, it should be noted that when an element is referred to as being "fixed to" or "disposed on" another element, it may be directly disposed on the other element or indirectly disposed on the other element. When an element is referred to as being "connected to" another element, it may be directly connected to the other element or indirectly connected to the other element.

[0197] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Thus, a feature specified as "first" or "second" may explicitly or implicitly include one or more of the features. Throughout the description of this application, "plurality" or "several" means two or more, unless otherwise specifically defined.

[0198] Professionals may further appreciate that the units and steps of each example described in conjunction with the embodiments disclosed in the specific embodiments of this application can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0199] The methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly using hardware, software modules executed by a processor, or a combination of the two. The software module can be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, various programmable logic devices, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. The processor that executes the software module can be a central processing unit (CPU), an embedded processor, a microcontroller (MCU), a digital signal processor (DSP), a single chip microcomputer, a system on a chip (SOC), a programmable logic device, and any other form of device with control and processing functions known in the art.

[0200] It should be noted that the structures, proportions, sizes, etc. illustrated in the drawings of this specification are only used to match the contents disclosed in the specification for people familiar with this technology to understand and read, and are not used to limit the conditions under which this application can be implemented. Therefore, they have no technical significance. Any structural modification, change in proportional relationship or adjustment of size should still fall within the scope of the technical content disclosed in this application without affecting the efficacy and purpose that can be achieved by this application.

[0201] By implementing the technical solutions of the rail damage ultrasonic positioning and identification method and computer-readable storage medium described in the specific embodiments of this application, the following technical effects can be achieved:

[0202] (1) The rail damage ultrasonic positioning and identification method and computer-readable storage medium described in the specific embodiments of the present application use an image segmentation network to perform semantic segmentation on the B-display image, and use a classification network to determine the damage category of the formed segmented sub-regions. This method can obtain relatively accurate target location and category results without human intervention, overcoming the problem that the existing rail damage ultrasonic detection method requires manual feature design and manual decision-making on B-display data segmentation, greatly improving the damage detection rate and shortening the detection cycle of rail ultrasonic flaw detection;

[0203] (2) The rail damage ultrasonic positioning and identification method and computer-readable storage medium described in the specific embodiments of the present application directly locate the internal damage of the rail using an image segmentation network from the perspective of image processing, and determine the damage category using a classification network, thereby achieving end-to-end rail ultrasonic flaw detection data recognition and accurately identifying the nature of the damage by combining multi-layer features. This solves the technical problems of the existing rail damage ultrasonic detection method, such as poor interpretability and difficulty in parameter optimization, inability to accurately analyze the internal damage of the rail, and difficulty in processing noise and edge shape information;

[0204] (3) The rail damage ultrasonic positioning and identification method and computer-readable storage medium described in the specific embodiments of the present application convert the original ultrasonic echo signal data (B-display data) into time series data. The concept of time is introduced through the time series data, which can show the correlation between echoes at different times. At the same time, in the scope of time series data, it is allowed to use the statistical characteristics of the past and current moments to predict and guide the data change trend at future moments, and it conforms to the objective development law of things. Similarly, when converting the original ultrasonic echo signal data (B-display data) into time series data, the time series data itself can have richer properties, such as periodicity, stationarity, time-frequency domain characteristics, etc. In addition, the time series allows tolerance of certain random variables, which increases the generalization ability of the prediction model.

[0205] (4) The rail damage ultrasonic positioning and identification method and computer-readable storage medium described in the specific embodiments of the present application use image vision to perform target positioning and identification of ultrasonic rail detection data. The rail ultrasonic detection data is first converted into visually visible image data, and a data preprocessing method in a special visual field is selected to perform data enhancement, supplementation, and denoising. This can effectively solve the problems of unbalanced distribution of original data and susceptibility of data to noise interference;

[0206] (5) The rail damage ultrasonic positioning and identification method and computer-readable storage medium described in the specific embodiments of the present application newly classify the damage into garbage when determining the damage category, and can distinguish between damage and non-damage or a small number of confusing samples at the edge of the noise category classification, thereby solving the technical problem that some targets and noise are difficult to distinguish by the classification network. Setting the noise category can prevent the classification network from segmenting the scattered noise and the B-display echo points that do not conform to the damage extension law due to inaccurate segmentation;

[0207] (6) The rail damage ultrasonic positioning and identification method and computer-readable storage medium described in the specific embodiments of this application, through the construction of expert rules by manually designing features, can combine the basic time series and image graphic information to determine the final damage category for targets that are difficult for the classifier to distinguish in the garbage category, thereby refining the damage identification results and ensuring that the identification results are more in line with actual operational requirements. At the same time, this multimodal data processing method can make up for the technical problems of difficulty in removing noise from image data and the insensitivity of the secondary classifier to noise. It can also make up for the problem of difficulty in manually establishing accurate features based solely on time series, resulting in abnormal cutting of the target area and the lack of clear boundary surfaces.

[0208] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0209] The above description is only a preferred embodiment of the present application and does not constitute any formal limitation to the present application. Although the present application has been disclosed as above with a preferred embodiment, it is not intended to limit the present application. Any technician familiar with the art can use the above-disclosed methods and technical contents to make many possible changes and modifications to the technical solution of the present application, or modify it into an equivalent embodiment with equivalent changes, without departing from the spirit and technical solution of the present application. Therefore, any simple modification, equivalent replacement, equivalent change and modification made to the above embodiments based on the technical essence of the present application without departing from the content of the technical solution of the present application still falls within the scope of protection of the technical solution of the present application.

Claims

1. A method for ultrasonic positioning and identification of rail damage, characterized in that: The following steps are involved: S11) converting the rail ultrasonic detection signal into ultrasonic time series data and visually visible B-display image data; S12) preprocessing the ultrasonic time series data and B-display image data generated in step S11); S13) using an image segmentation network to perform semantic segmentation on the B-display image data processed in step S12) and generate segmented sub-regions to achieve damage localization; S14) using a classification network to identify damage categories of targets within the segmented sub-regions generated in step S13); S15) performing feature extraction and post-processing on the classification result of step S14) using expert rules to refine the damage identification result obtained in step S14); The image segmentation network in step S13) adopts a UNet network; The step S14) includes: After damage classification using the classification network, each detection sample of the sub-block image is given a category confidence, that is, the probability value of the detection sample belonging to a certain target category, which is recorded as , preset the probability value of the detection sample When , the classification network can make accurate judgments. The value can be customized according to different situations; when the probability value of the detection sample When , the detection sample with low confidence is classified as garbage, indicating that the target category of the current detection sample still needs other information to assist in judgment; then the target category is judged by expert rules for the garbage class to make an accurate target category determination; The classification network in step S14) uses a ResNet network as a secondary classifier; The input of the expert rules is image data or ultrasonic time series data; through the expert rules, garbage targets that are difficult to classify by the classification network are combined with basic ultrasonic time series data and image data to jointly determine the final damage category.

2. The rail damage ultrasonic positioning and identification method according to claim 1, characterized in that: The step S11) further includes the following process: The ultrasonic detection signals within a plurality of unit pulse lengths are used as the data contained in a frame of image, thereby converting the original ultrasonic detection signal data into a plurality of image data, and continuously inputting the data into step S12 in the form of a data stream for data preprocessing.

3. The rail damage ultrasonic positioning and identification method according to claim 1 or 2, characterized in that: The data preprocessing process in step S12) is to perform processing including enhancement, supplementation or denoising on the ultrasonic time series data and B-display image data. Through data preprocessing, the distance within the data class becomes smaller and the distance between the data classes becomes larger on the basis of balancing the data distribution.

4. The rail damage ultrasonic positioning and identification method according to claim 3, characterized in that: The pre-processing process in step S12) includes the following steps: Based on the ultrasonic time series data, the data where the bottom wave disappears are directly deleted.

5. The rail damage ultrasonic positioning and identification method according to claim 3, characterized in that: The pre-processing process in step S12) includes the following steps: Based on the damage type determined by the existing ultrasonic detection signal, the echo point data of the damage is intercepted from the original ultrasonic time series data according to the coordinate area. According to the scope of the damage, the vertical coordinate representing the depth is kept unchanged, and the horizontal coordinate representing the distance is randomly drawn onto a new preset background image. This simulates damage data under different backgrounds and thus performs data amplification. Alternatively, according to the extension law of the damage, partial data is extracted and reconstructed through data segmentation to form a new damage, which is presented in the form of a B-display image, thereby achieving data amplification.

6. The rail damage ultrasonic positioning and identification method according to claim 3, characterized in that: The pre-processing process in step S12) includes the following steps: Masks of 3*3, 5*5 and 7*7 sizes are used respectively to mask the content of the B-display image data in a sliding window manner according to the mask step size, and it is ensured that the masked part of the selected size does not affect the judgment of damage.

7. The rail damage ultrasonic positioning and identification method according to claim 3, characterized in that: The pre-processing process in step S12) includes the following steps: Perform Hilbert transform on ultrasonic time series data to obtain analytical signals, retain the unilateral spectrum of the signal to draw the envelope of the signal and reduce the noise interference of the time series; Hilbert transform is expressed as ,in represents the Hilbert analytic signal, and represent Fourier transform and inverse Fourier transform respectively, represents a unit step signal, Represents the original signal; after the envelope of the analytical signal is determined, the mean value of the envelope at each time point is calculated using the value of the upper envelope and the value of the lower envelope at each time point.

8. The method for ultrasonic positioning and identification of rail damage according to claim 1, 2, 4, 5, 6 or 7, characterized in that: Step S13) The following processes are included: S131) ​​The B-display image data processed in step S12) is used as an input feature map, which is first scaled to a size of 572×572×1, and then convolved twice with a 3×3×64 convolution kernel to a size of 568×568×64. Then, a 2×2 maximum pooling operation is performed to a size of 284×284×64, completing a downsampling operation. Repeat step S131 for three more 3×3 convolutions and 2×2 pooling operations. After each maximum pooling operation, the number of convolution kernels in the first 3×3 convolution operation is doubled. S132) After reaching the fourth max pooling at the bottom layer, the feature map becomes 32×32×512 in size, and then undergoes two 3×3×1024 convolution operations, finally becoming 28×28×1024 in size; S133) An upsampling operation is performed on a feature map of size 28×28×1024. First, a 2×2×512 deconvolution operation is performed to change the feature map to a size of 56×56×512. Then, the image before the corresponding maximum pooling layer is copied and cropped, and the image obtained by deconvolution is concatenated with the image according to the channel dimension to obtain a feature map of size 56×56×1024. Then, a 3×3×512 convolution operation is performed to complete one upsampling operation. Repeat the 2×2 deconvolution and 3×3 convolution operations three times according to the above step S133, and the number of convolution kernels in the first 3×3 convolution operation after each splicing is reduced exponentially; S134) After reaching the fourth deconvolution at the top layer, the feature map becomes 392×392×64 in size, is copied and cropped, and then concatenated to 392×392×128 in size, and then two 3×3×64 convolution operations are performed; S135) obtains a feature map of size 388×388×64, and finally performs a 1×1×2 convolution operation to obtain the segmentation area information of the B-display image.

9. The rail damage ultrasonic positioning and identification method according to claim 8, characterized in that: According to the segmented area information obtained in step S13), each individual target block is obtained as an irregularly shaped B-display image block, and the segmented B-display block is drawn onto a black background to generate a new sub-block image. The final damage type is then identified by the designed secondary classifier.

10. The method for ultrasonic positioning and identification of rail damage according to claim 1, 2, 4, 5, 6, 7 or 9, characterized in that: Step S14) The following processes are included: S141) scaling the sub-block image of the segmented sub-region to a size of 224×224×3, first performing a convolution operation with a convolution kernel of 3×3, a stride of 2, a padding of 3, an output channel of 64, and an output feature map size of 112×112×64; S1421) The first residual structure combination includes two residual base structures, and the output feature map is first subjected to two 3×3×64 convolution operations, where the convolution kernel is 3×3, the stride is 1, the padding is 1, and the number of channels is 64, and then added to the input feature map of the residual base structure in a channel-by-channel addition manner; The second time, two 3×3×64 convolution operations are performed. The convolution kernel of this convolution operation is 3×3, the stride is 1, the padding is 1, and the number of channels is 64. It is added to the input feature map of the residual basic structure in a channel-by-channel addition manner. At this time, the feature map output by the first residual structure combination is 56×56×64. S1422) The second residual structure combination includes two residual base structures. The feature map output by the first residual structure combination is first subjected to two 3×3×128 convolution operations, where the convolution kernel is 3×3, the stride is 1, the padding is 1, and a 128 kernel function is used; between the two convolution operations, a 1×1×128 convolution operation is performed with a stride of 2 and a padding of 0 to reduce the size of the feature map by half, and the feature map is added to the input feature map of the residual base structure in a channel-wise addition manner; and then two 3×3×128 convolution operations are performed with a 3×3 kernel, a stride of 1, a padding of 1, and a 128 kernel function, and the feature map is added to the input feature map of the residual base structure in a channel-wise addition manner; at this time, the feature map output by the second residual structure combination has a size of 28×28×128; S1423) The third residual structure combination includes two residual base structures. The feature map output by the second residual structure combination is first subjected to two 3×3×256 convolution operations, where the convolution kernel is 3×3, the stride is 1, the padding is 1, and a 256 kernel function is used; between the two convolution operations, a 1×1×256 operation is performed with a stride of 2 and padding of 0 to reduce the size of the feature map by half, and the feature map is added to the input feature map of the residual base structure in a channel-wise addition manner; The second time, two 3×3×256 convolution operations are performed. The convolution kernel of this convolution operation is 3×3, the stride is 1, the padding is 1, and 256 kernel functions are used. At the same time, the input feature map of the residual basic structure is added in a channel-by-channel manner. At this time, the output feature map of the third residual structure combination is 14×14×256. S1424) The fourth residual structure combination includes two residual base structures. The feature map output by the third residual structure combination is first subjected to two 3×3×512 convolution operations, where the convolution kernel is 3×3, the stride is 1, the padding is 1, and a 512 kernel function is used; between the two convolution operations, a 1×1×512 operation is performed with a stride of 2 and a padding of 0 to reduce the size of the feature map by half, and the feature map is added to the input feature map of the residual base structure in a channel-wise addition manner; the second 3×3×512 convolution operation is performed twice, where the convolution kernel is 3×3, the stride is 1, the padding is 1, and a 512 kernel function is used, and the feature map is added to the input feature map of the residual base structure in a channel-wise addition manner; at this time, the output feature map size of the fourth residual structure combination is 7×7×512; S143) The feature map outputted in step S141) is combined with the four residual structures and then average pooled. The output data size is 1×1×512. Finally, the output is mapped to several categories through a fully connected layer. The damage category and probability value of the target area are finally outputted through an activation function.

11. The method for ultrasonic positioning and identification of rail damage according to claim 1, 2, 4, 5, 6, 7 or 9, characterized in that: The expert rules in step S15) include: Calculate the fitting line of all channel echo points in the target area based on the ultrasonic time series data. After determining the fitting line, calculate the total slope ,in, and They represent the vertical coordinate increment and the horizontal coordinate increment of the damaged echo point in the target area respectively; Based on the ultrasonic time series data, the fitting line of the echo point of a channel in the target area is calculated. After the fitting line is determined, the slope of a channel is calculated. , and They respectively represent the horizontal coordinate increment and vertical coordinate increment of the damage echo point belonging to a certain channel; Only when the slope of each channel and the total slope of all channels When none of the judgment conditions of rail head core damage, rail waist damage, and rail bottom damage are met, the current target is judged to be noise under the trend condition.

12. The method for ultrasonic positioning and identification of rail damage according to claim 1, 2, 4, 5, 6, 7 or 9, characterized in that: The expert rules in step S15) include: For scattered point noise, the density and 1 / 4 density of the current target area are obtained based on the B-display image data, which are recorded as ,in, Indicates the area of ​​the current target area. Indicates the number of channel echo points in the current target area. By dividing the current target area into four sub-areas, the corresponding density distribution is calculated to determine whether it is damaged, and the damaged area is optimized for secondary positioning.

13. The method for ultrasonic positioning and identification of rail damage according to claim 1, 2, 4, 5, 6, 7 or 9, characterized in that: The expert rules in step S15) include: For targets with large similarities in shape, mask matching and channel echo point position matching are used based on B-display image data to calculate and analyze whether the target conforms to the echo point distribution law. First, the target and background parts are separated by binarization method for the target image data, and then a mask is slid on the sub-target image to determine whether the size of the unmatched target is abnormal.

14. The method for ultrasonic positioning and identification of rail damage according to claim 1, 2, 4, 5, 6, 7 or 9, characterized in that: The expert rules in step S15) include: For screw hole cracks, based on the B-display image data, the positional relationship between the 45-degree channel echo point and the 0-degree channel echo point in the front and rear directions is determined, as well as whether the echo point appears at the crack position of the screw hole, thereby determining whether there is any abnormality in the screw hole.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the rail damage ultrasonic positioning and identification method according to any one of claims 1 to 14 is implemented.

Images