Floating slab ballast bed sealing strip defect detection method and system based on multi-modal data

Abstract

According to the floating slab ballast bed sealing strip defect detection method and system based on the multi-modal data, efficient and accurate detection of floating slab ballast bed sealing strip defects is achieved through a multi-modal data fusion technology. The system integrates a linear array camera, an acceleration sensor, a pressure sensor and a controllable vibration source, a multi-dimensional data acquisition system is constructed, and the limitation of a single detection means is effectively overcome. Through collaborative analysis of image data and vibration and pressure data, the accuracy and robustness of defect recognition are remarkably improved. According to the defect evaluation model based on the CNN-LSTM hybrid architecture, implicit features in multi-modal data can be deeply mined, and the defect confidence is accurately quantified. Meanwhile, by combining the preset characteristic parameter threshold value and the classification model, intelligent classification of the deformation defect and the foreign matter defect is realized, and a clear basis is provided for subsequent maintenance.

Description

Technical Field

[0001] The present invention relates to the field of railway detection, and specifically relates to a method and system for detecting defects of floating slab track bed sealing strips based on multimodal data. Background Art

[0002] As a key component of the rail transit system, the sealing performance of the floating slab track bed sealing strip directly affects the stability of the track structure, the vibration reduction and noise reduction effect, and the safety of train operation. The sealing strip is long-term exposed to a complex environment and is easily affected by factors such as train loads, temperature and humidity changes, and foreign object intrusion, resulting in defects such as deformation, cracking or shedding. Timely and accurately detecting the defects of the sealing strip is crucial for ensuring the reliable operation of the track system.

[0003] At present, the detection of defects of floating slab track bed sealing strips mainly relies on manual inspections, and evaluations are carried out through visual observation or simple tool measurements. However, manual detection has the following limitations: low detection efficiency, difficult to meet the periodic detection requirements of large-scale lines; strong subjectivity, and the experience differences of different detection personnel are prone to misjudgment or missed judgment; for hidden defects such as internal cracks and foreign object embedding, they cannot be effectively identified. In addition, a single detection method is difficult to comprehensively reflect the state characteristics of the sealing strip, resulting in insufficient accuracy and robustness of the detection results.

[0004] Therefore, in order to solve the above problems, the present invention proposes a method and system for detecting defects of floating slab track bed sealing strips based on multimodal data. Summary of the Invention

[0005] Aiming at the deficiencies of the existing technology, the purpose of the present invention is to provide a method and system for detecting defects of floating slab track bed sealing strips based on multimodal data.

[0006] To achieve the above purpose, the present invention provides the following technical solutions: A method for detecting defects of floating slab track bed sealing strips based on multimodal data, comprising the following steps: A data acquisition step, collecting image data of the track bed sealing strip through a line array camera installed at the bottom of the train, acquiring track bed vibration data through an acceleration sensor, and acquiring floating slab contact pressure data through a pressure sensor; A preliminary defect detection step, calculating the parallelism parameter between the sealing strip and the floating slab of the track bed based on the image data, and triggering a defect depth detection when the parallelism parameter is higher than a preset threshold; A defect depth detection step, starting a controllable vibration source at the tail of the train to emit a lateral vibration signal, synchronously collecting vibration response data and pressure response data of the defect area, inputting them into a multimodal data fusion defect evaluation model to calculate the defect confidence level, and determining that a defect exists when the defect confidence level exceeds the threshold; Defect type judgment step, based on the frequency domain characteristics of the vibration signal and the contact pressure change curve, combined with the parallelism parameter, outputs deformation defects or foreign object defects through the defect feature judgment strategy.

[0007] As a further improvement of the present invention, the data acquisition step further includes eliminating environmental noise from the vibration data. After decomposing the vibration signal into a number of intrinsic mode functions, calculating the kurtosis coefficient of the intrinsic mode functions, and superimposing the components of the intrinsic mode functions with a kurtosis coefficient greater than the preset kurtosis threshold to obtain the vibration signal after eliminating environmental noise.

[0008] As a further improvement of the present invention, the parallelism parameter is configured as: ; where P is the parallelism parameter, representing the parallel degree between the sealing strip and the floating slab of the track bed. The larger the P value, the lower the parallelism. W i is the width value of the i-th sampling point on the fitting surface of the sealing strip and the floating slab in the image, is the average width value; the image data uses the Hough transform to detect the edge of the sealing strip, fits the reference line through cubic spline interpolation, and sets a number of uniformly spaced sampling points.

[0009] As a further improvement of the present invention, the multi-modal data fusion defect assessment model uses a CNN-LSTM hybrid architecture to extract the time-frequency characteristics of the vibration signal. The convolutional neural network includes 5 convolutional layers, with a convolutional kernel size of 3×3 for each layer and a stride of 1, and also includes 3 max-pooling layers, with a pooling window size of 2×2 for each layer.

[0010] As a further improvement of the present invention, the defect depth detection step further includes subtracting the vibration response data and the pressure response data from the initially collected track bed vibration data and contact pressure data to obtain the vibration difference and the pressure difference. Combining the vibration difference and the pressure difference with the lateral vibration signal parameters emitted by the controllable vibration source and the train running speed, the defect confidence is output through the multi-modal data fusion defect assessment model.

[0011] As a further improvement of the present invention, the multi-modal data fusion defect assessment model is configured as: ; where C is the defect confidence, representing the possibility of judging that there is a defect in the sealing strip. The smaller the value of C, the lower the possibility that there is a defect in the sealing strip; V d (t) is the vibration difference function, representing the vibration difference at time t; P d(t) is a function of the pressure difference, representing the pressure difference at time t; T is the total duration of data acquisition, λ is the vibration attenuation coefficient, representing the coefficient of vibration difference decay over time, β is the coefficient of pressure difference change over time, S h (i) is the value of the i-th lateral vibration signal continuously emitted by the controllable vibration source, n represents the number of sampling points of the lateral vibration signal, v is the train speed, v e is the standard speed. The greater the difference between the vehicle speed and the standard speed, the lower the defect confidence level.

[0012] As a further improvement of the present invention, the defect type determination step includes performing a fast Fourier transform on the vibration signal to obtain frequency domain features, identifying specific frequency peaks in the frequency domain features. At the same time, analyzing the slope, amplitude, and change period of the contact pressure change curve, and combining with the parallelism parameter to construct a multi-dimensional feature vector. Analyzing the multi-dimensional feature vector through a pre-trained classification model to output a determination result of deformation defect or foreign object defect.

[0013] As a further improvement of the present invention, the defect feature determination strategy includes setting multiple groups of feature parameter thresholds. When the specific frequency peak of the vibration signal is higher than the first threshold, and the slope of the contact pressure change curve exceeds the second threshold, and at the same time the parallelism parameter is greater than the third threshold, it is determined as a deformation defect; when the specific frequency peak of the vibration signal is lower than the fourth threshold, and the contact pressure change curve shows irregular fluctuations, and at the same time the parallelism parameter is within a specific range, it is determined as a foreign object defect.

[0014] A floating slab track seal defect detection system based on multi-modal data, comprising: A data acquisition module that acquires track seal image data through a linear array camera installed at the bottom of the train, acquires track vibration data through an acceleration sensor, and acquires floating slab contact pressure data through a pressure sensor; A preliminary defect detection module that calculates the parallelism parameter between the seal and the floating slab of the track based on the image data, and triggers defect depth detection when the parallelism parameter is higher than a preset threshold; A defect depth detection module that activates a controllable vibration source at the tail of the train to emit a lateral vibration signal, synchronously acquires vibration response data and contact pressure data of the defect area, and inputs them into a multi-modal data fusion defect assessment model to calculate the defect confidence level. When the confidence level exceeds the threshold, it is determined that there is a defect; A defect type determination module that outputs a deformation defect or a foreign object defect based on the frequency domain features of the vibration signal and the contact pressure change curve, combined with the parallelism parameter through the defect feature determination strategy.

[0015] The beneficial effects of the present invention are as follows: Through the multi-modal data fusion technology, the present invention realizes the efficient and accurate detection of the defects of the floating slab track bed sealing strips. The system integrates a linear array camera, an acceleration sensor, a pressure sensor and a controllable vibration source to construct a multi-dimensional data acquisition system, effectively overcoming the limitations of a single detection method. The collaborative analysis of image data with vibration and pressure data significantly improves the accuracy and robustness of defect recognition. The defect evaluation model based on the CNN-LSTM hybrid architecture can deeply mine the hidden features in multi-modal data and accurately quantify the defect confidence. At the same time, through the combination of preset feature parameter thresholds and classification models, the intelligent classification of deformation defects and foreign object defects is realized, providing a clear basis for subsequent maintenance. This method not only reduces the subjectivity and missed detection rate of manual detection, but also improves the detection efficiency through real-time data acquisition and online analysis, meeting the dynamic monitoring requirements in rail transit operation and having significant engineering application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is the flowchart of the method for detecting defects of the floating slab track bed sealing strip based on multi-modal data of the present invention.

[0017] Figure 2 is the flowchart of data acquisition and processing of the method for detecting defects of the floating slab track bed sealing strip based on multi-modal data of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0018] The present invention will be further described in detail below with reference to the drawings and embodiments. The same components are denoted by the same reference numerals. It should be noted that the terms "front", "rear", "left", "right", "upper" and "lower" used in the following description refer to the directions in the drawings, and the terms "bottom surface" and "top surface", "inner" and "outer" refer to the directions towards or away from the geometric center of a specific component, respectively.

[0019] The present invention relates to a method for detecting defects of a floating slab track bed sealing strip based on multi-modal data, aiming to solve the problem of how to detect the defects of the floating slab track bed sealing strip. In the prior art, the defect detection of the floating slab track bed sealing strip mainly relies on manual visual inspection, which has problems such as low detection efficiency, strong subjectivity, high missed detection and false detection rates. To solve these problems, the present invention proposes a method for detecting defects of a floating slab track bed sealing strip based on multi-modal data. By collecting data through multiple sensors and combining multi-modal data fusion technology, the efficient and accurate detection of the sealing strip defects is realized, as Figure 1 shown, including the following steps: Data acquisition step: Image data of the track bed seal strip is collected by a linear array camera installed at the bottom of the train. The linear array camera is set at the front of the train or the first half of the train close to the front to ensure that after the system processes the image data, when the train passes through the position where abnormal images are collected, the track bed vibration data can be obtained again. The track bed vibration data is obtained through an acceleration sensor. A number of acceleration sensors are evenly set at the bottom of the train, and the contact pressure data of the floating slab is obtained through a pressure sensor. A number of pressure sensors are set at the bottom of the train. The collected seal strip image data is used for direct judgment of crack defects or for judgment of the parallelism between the seal strip and the track bed. The collected vibration data and contact pressure data are used for further judgment of the defect situation of the floating slab track bed seal strip and judgment of the defect type in the follow-up.

[0020] Initial defect detection step: Calculate the parallelism parameter between the seal strip and the floating slab of the track bed based on the image data. When the parallelism parameter is higher than the preset threshold, trigger the defect depth detection; when the parallelism parameter is higher than the preset threshold, it indicates that there is a non-parallel situation between the floating slab of the track bed and the seal strip in the image data at this time. Normally, since the seal strip is tightly sealed against the floating slab track bed, the seal strip and the floating slab track bed are parallel in the absence of abnormalities. However, when the seal strip is deformed, extruded by foreign objects, etc., the seal strip will obviously no longer be parallel to the floating slab track bed.

[0021] Defect depth detection step: Start the controllable vibration source at the tail of the train to emit a lateral vibration signal, synchronously collect the vibration response data and pressure response data of the defect area, input them into the multi-modal data fusion defect assessment model to calculate the defect confidence level. When the defect confidence level exceeds the threshold, it is determined that there is a defect; Defect type judgment step: Based on the frequency domain characteristics of the vibration signal and the contact pressure change curve, combined with the parallelism parameter, output the deformation defect or foreign object defect through the defect feature judgment strategy.

[0022] In the data acquisition step, the image data, vibration data, and contact pressure data of the track bed seal strip are obtained through a linear array camera, an acceleration sensor, and a pressure sensor. These data provide basic information for subsequent defect detection. In the initial defect detection step, the parallelism parameter in the image data is calculated to preliminarily judge whether there is a defect in the seal strip. When the parallelism parameter is higher than the preset threshold, the defect depth detection is triggered. In the defect depth detection step, a lateral vibration signal is emitted by the controllable vibration source and the vibration response data and pressure response data of the defect area are synchronously collected. The multi-modal data fusion defect assessment model is used to calculate the defect confidence level. When the defect confidence level exceeds the threshold, it is determined that there is a defect. In the defect type judgment step, based on the frequency domain characteristics of the vibration signal and the contact pressure change curve, combined with the parallelism parameter, the deformation defect or foreign object defect is output through the defect feature judgment strategy.

[0023] Through the fusion and analysis of multi-modal data, the deficiencies in the prior art are effectively overcome, and the efficient and accurate detection of the defects of the floating slab track bed seal strip is realized. In particular, through the comprehensive analysis of image data, vibration data, and pressure data, the defect type of the seal strip can be accurately judged, providing a reliable basis for subsequent maintenance and repair.

[0024] Further, the data acquisition step further includes, as Figure 2 shown, eliminating the environmental noise of the vibration data. After decomposing the vibration signal into several intrinsic mode functions, calculating the kurtosis coefficient of the intrinsic mode functions, and superimposing the intrinsic mode function components with kurtosis coefficients greater than the preset kurtosis threshold, the vibration signal after eliminating the environmental noise is obtained. Through the above technical means, the present application solves the problem of environmental noise interference in the vibration data, ensuring the accuracy and reliability of the vibration data used in the subsequent defect depth detection step, thereby improving the accuracy and stability of the defect detection.

[0025] The present application decomposes the vibration signal into several intrinsic mode functions, calculates the kurtosis coefficient of each intrinsic mode function, screens out the intrinsic mode function components with kurtosis coefficients greater than the preset kurtosis threshold for superposition, and finally obtains the vibration signal after eliminating the environmental noise. In this way, the environmental noise interference in the vibration signal can be effectively removed, and the effective vibration signal components can be retained. Specifically, this process includes the following steps: In practical applications, an acceleration sensor is used to collect the vibration signal of the floating slab track bed. The selection of the sensor is crucial, and parameters such as its range, accuracy, and sampling frequency need to be considered. For example, an acceleration sensor with a range of ±50g, an accuracy of 0.01g, and a sampling frequency of 10kHz can be selected to ensure that the details of the vibration signal can be accurately captured. The collected vibration signal is usually stored in the form of a time series, and each time point corresponds to a vibration amplitude.

[0026] Before subsequent processing, it is also necessary to conduct a preliminary inspection of the collected original vibration signal to ensure the integrity and accuracy of the data. If missing values or outliers are found in the data, methods such as linear interpolation or median filtering can be used for processing.

[0027] After that, the collected vibration signal is subjected to empirical mode decomposition (EMD) to obtain several intrinsic mode functions (IMFs): The given vibration signal x(t) is respectively fitted with a cubic spline interpolation function to all local maximum points and local minimum points of the signal to obtain the upper envelope e up (t) and the lower envelope e low (t), calculate the mean of the upper envelope and the lower envelope , subtract the mean of the envelope from the original vibration signal to obtain h1(t) = x(t) - m(t). h1(t) is the IMF component that we need to determine whether it can be extracted. Judge whether h1(t) satisfies the IMF conditions, that is, the number of local extreme points and zero-crossing points is equal or at most differs by 1, and the upper and lower envelopes are locally symmetric about the time axis. If not, use h1(t) as the new signal and repeat the above steps until h 1k (t) that satisfies the conditions. At this time, the first IMF component c1(t) can be extracted. Subtract the first IMF component from the original vibration signal to obtain the remaining signal r1(t) = x(t) - c1(t). Use r1(t) as the new signal and repeat the above steps to extract all IMF components until the remaining signal becomes a monotonic function or a constant.

[0028] Finally, the original vibration signal after empirical mode decomposition is expressed as , where c i (t) is the i-th IMF component, and r n (t) is the remaining component.

[0029] After that, calculate the kurtosis coefficient. The kurtosis coefficient is a statistic used to measure the distribution shape of a signal, and it can reflect the sharpness or impulse characteristics of the signal. For each IMF component c i (t), its kurtosis coefficient calculation formula is , where, is the expectation of the difference between each IMF component and the mean , is the mean of c i (t), is the standard deviation of c i (t). Since the true probability distribution of the vibration signal cannot be obtained, the sample kurtosis coefficient is used here for approximate estimation. For the IMF component c i (t) with length N, its sample kurtosis coefficient ; where, is the sample mean of c i (t), and c (j) is the amplitude of c i (t) at the j-th time point. i

[0030] Set a preset kurtosis threshold. The setting of the preset kurtosis threshold needs to be adjusted according to specific application scenarios and experimental data. Generally speaking, by analyzing a large number of vibration signals under normal working conditions, the kurtosis coefficient distribution of their IMF components can be statistically analyzed, and then a suitable threshold can be selected so that the kurtosis coefficients of the IMF components of most noise components are less than this threshold, while the kurtosis coefficients of the IMF components of effective signal components are greater than this threshold. Under normal working conditions, the kurtosis coefficients of the IMF components of noise components mostly concentrate between 1 and 3, while the kurtosis coefficients of the IMF components of effective signal components are usually greater than 3. Therefore, the preset kurtosis threshold is set to 3 here. According to the set kurtosis threshold, the IMF components with kurtosis coefficients greater than this threshold are screened out, and these components are combined into a new set. The screened IMF components are superimposed to obtain the vibration signal after removing environmental noise.

[0031] The empirical mode decomposition in the above steps is an adaptive signal processing method that can decompose complex signals into several intrinsic mode functions with physical meanings. The calculation of the kurtosis coefficient is often used in the field of signal processing and can effectively distinguish signals containing impact components. By setting a reasonable kurtosis threshold, environmental noise can be effectively removed and useful vibration signal components can be retained.

[0032] Furthermore, the parallelism parameter is configured as follows: ; where P is the parallelism parameter, representing the parallel degree between the sealing strip and the floating slab of the ballast bed. The larger the P value, the lower the parallelism. W i is the width value of the i-th sampling point on the fitting surface of the sealing strip and the floating slab in the image, is the average width value; the image data is used to detect the edge of the sealing strip by Hough transform, fit the reference line by cubic spline interpolation, and set a number of sampling points with uniform spacing.

[0033] The technical features included in this application are the calculation formula of the parallelism parameter, the processing method of image data, and the setting of sampling points, which play an important role in solving the quantification problem of the parallelism between the sealing strip of the floating slab and the floating slab of the ballast bed.

[0034] The calculation formula of the parallelism parameter quantifies the parallelism between the sealing strip and the floating slab of the ballast bed by averaging the width values of multiple sampling points. The image data processing method uses the Hough transform to detect the edge of the sealing strip and fits the reference line through cubic spline interpolation, thereby ensuring the accuracy and uniform distribution of the sampling points. These technical features cooperate with each other to effectively solve the problem of quantifying the parallelism between the floating slab sealing strip and the floating slab of the ballast bed. Next, a number of sampling points with uniform spacing are set, the width value Wi of each sampling point is calculated, and the average value W of these width values is obtained. Finally, the parallelism parameter P is calculated using the formula. The larger the P value, the lower the parallelism, thus effectively quantifying the parallelism between the floating slab sealing strip and the floating slab of the ballast bed and ensuring the accuracy and uniform distribution of the sampling points. Compared with the prior art, the present application provides a more accurate and reliable parallelism quantification method by introducing the parallelism parameter and the image data processing method. This method can better judge the parallelism between the sealing strip and the floating slab of the ballast bed, thereby improving the accuracy and reliability of defect detection.

[0035] Furthermore, the multi-modal data fusion defect assessment model uses a CNN-LSTM hybrid architecture to extract the time-frequency features of the vibration signal. The convolutional neural network includes 5 convolutional layers, each with a convolutional kernel size of 3×3 and a stride of 1, and also includes 3 max-pooling layers, each with a pooling window size of 2×2.

[0036] The multi-modal data fusion defect assessment model adopts a CNN-LSTM hybrid architecture. The CNN part includes 5 convolutional layers, each with a convolutional kernel size of 3×3 and a stride of 1. In addition, it also includes 3 max-pooling layers, each with a pooling window size of 2×2. The combination of the convolutional layer and the pooling layer is used to extract the spatial features of the vibration signal. The LSTM part is used to extract the temporal features. This hybrid architecture effectively extracts the time-frequency features of the vibration signal by combining the spatial feature extraction ability of the convolutional neural network and the temporal feature extraction ability of the long short-term memory network, thereby realizing multi-modal data fusion defect assessment.

[0037] The convolutional neural network (CNN) can effectively extract the spatial features in the vibration signal through the combination of multiple convolutional layers and pooling layers. The convolutional kernel size of each convolutional layer is 3×3 and the stride is 1, which can ensure sufficient capture of the details of the vibration signal. The pooling window size of the max-pooling layer is 2×2. By downsampling the feature map, it reduces the size of the feature map and the computational complexity. The long short-term memory network (LSTM) part can capture the temporal features in the vibration signal through the memory and forgetting mechanisms, especially suitable for processing signals with time dependence.

[0038] Specifically, the CNN part first processes the input vibration signal, and through the combination of 5 convolutional layers and 3 max-pooling layers, extracts the spatial features of the vibration signal layer by layer. Then, the extracted features are input into the LSTM part for the extraction of temporal features. The LSTM part, through its unique gating mechanism, can capture the long-term dependencies in the vibration signal, further improving the accuracy of feature extraction.

[0039] In this embodiment, ResNet-LSTM can also be used to construct the model framework. Specifically, ResNet (Residual Network), as a deep convolutional neural network, has significant advantages in dealing with complex feature extraction tasks. Combining it with LSTM (Long Short-Term Memory Network) can more effectively analyze multimodal data and achieve more accurate defect assessment.

[0040] The core advantage of the ResNet part lies in its unique residual block structure. The residual block, by introducing skip connections, allows the network to more easily learn the identity mapping during training, thus alleviating the common problems of gradient vanishing and gradient explosion in deep neural networks, enabling the network to be constructed deeper to capture richer spatial features. In this model, ResNet can be stacked by multiple residual blocks. For example, we can construct a ResNet architecture with 18 layers, 34 layers or deeper levels, and the specific number of layers can be adjusted according to the complexity of the actual data and computing resources. Each residual block contains a convolutional layer, a batch normalization layer (Batch Normalization) and an activation function layer. The size of the convolutional kernel of the convolutional layer can be set according to actual needs. For example, a 3×3 convolutional kernel can control the number of model parameters while ensuring the capture of feature details. The batch normalization layer can accelerate the convergence speed of the model and improve the stability of the model. The activation function layer introduces non-linearity into the network, enhancing the expressive power of the model.

[0041] For the processing of vibration signals, the ResNet part first performs preliminary feature extraction on the input vibration signal. Through a series of convolutional operations and residual blocks, it gradually excavates the complex spatial features in the signal. These spatial features reflect the distribution of the vibration signal at different frequencies and amplitudes, which are very crucial for identifying abnormal patterns and features in the signal.

[0042] The LSTM part further explores the temporal features of the signal based on the spatial features extracted by ResNet. LSTM has a unique gating mechanism, including an input gate, a forget gate, and an output gate. The forget gate determines which information needs to be discarded from the cell state, the input gate determines which new information needs to be added to the cell state, and the output gate determines the output content based on the cell state. This gating mechanism enables LSTM to effectively process sequence data with time dependence and capture the changing trends and long-term dependencies of vibration signals at different time points.

[0043] In the entire model process, the vibration signal in the input multimodal data first undergoes spatial feature extraction by ResNet. ResNet extracts high-level spatial features of the signal layer by layer through its deep network structure and residual blocks. Then, these extracted spatial features are passed as input to the LSTM part. When processing these features, LSTM takes into account the order and changes of the features in the time dimension, and screens and processes the information through its gating mechanism, thereby extracting more representative temporal features. Finally, the results integrating spatial and temporal features are used for multimodal data fusion defect assessment, which can more accurately identify potential defect information in the data and improve the accuracy and reliability of the assessment.

[0044] Compared with the previously adopted CNN-LSTM hybrid architecture, the ResNet-LSTM architecture has stronger feature extraction capabilities when processing complex multimodal data. In the CNN-LSTM hybrid architecture, the CNN part extracts the spatial features of the vibration signal through 5 convolutional layers (each convolutional kernel size is 3×3, and the stride is 1) and 3 max-pooling layers (each pooling window size is 2×2). Due to its residual block structure and deeper network levels, ResNet can dig deeper into the spatial features in the signal and is more stable during the training process. At the same time, the LSTM part plays a key role in extracting temporal features in both architectures, but in the ResNet-LSTM architecture, it processes the results after more advanced spatial feature extraction, which can better capture the temporal changes of the signal and further improve the performance of the model. Through the above hybrid architecture, this application can effectively extract the time-frequency features of vibration signals, thereby realizing multimodal data fusion defect assessment. Compared with the prior art, this application has significant advantages in the accuracy and efficiency of feature extraction, can more accurately evaluate the defects of the floating slab track sealing strip, and improves the reliability and accuracy of defect detection.

[0045] Further, the defect depth detection step further includes obtaining a vibration difference and a pressure difference by subtracting the vibration response data and the pressure response data from the initially collected track bed vibration data and contact pressure data, and outputting a defect confidence level through a multi-modal data fusion defect assessment model by combining the vibration difference and the pressure difference with the lateral vibration signal parameters emitted by a controllable vibration source and the train running speed.

[0046] In the defect depth detection step, the vibration difference and the pressure difference are obtained by comparing the differences between the vibration response data and the pressure response data and the initial data. These differences, combined with the lateral vibration signal parameters emitted by the controllable vibration source and the train running speed, are used to calculate the defect confidence level through a multi-modal data fusion defect assessment model. The calculation of the vibration difference and the pressure difference can reflect the response changes of the sealing strip under controlled vibration. Combining the train running speed and the lateral vibration signal parameters can more accurately evaluate the defect confidence level of the sealing strip and ensure the reliability of the detection results.

[0047] Specifically, the vibration difference is obtained by comparing the current vibration response data with the initial track bed vibration data. Similarly, the pressure difference is obtained by comparing the current pressure response data with the initial contact pressure data. These differences reflect the changes of the sealing strip under the action of the controllable vibration source. The lateral vibration signal parameters emitted by the controllable vibration source and the train running speed are input into the multi-modal data fusion defect assessment model as supplementary information. The model outputs a defect confidence level value by comprehensively integrating these multi-modal data. The higher the defect confidence level value, the greater the possibility that the sealing strip has defects.

[0048] The vibration difference and the pressure difference can be analyzed by signal processing techniques such as the fast Fourier transform (FFT) or the short-time Fourier transform (STFT) to extract useful frequency domain features. The lateral vibration signal parameters of the controllable vibration source can include information such as vibration frequency and amplitude. The train running speed can be obtained in real time through a speed sensor. The multi-modal data fusion defect assessment model can adopt deep learning methods, such as a hybrid architecture of a convolutional neural network (CNN) and a long short-term memory network (LSTM), to extract and fuse various data features, so as to accurately evaluate the defect confidence level.

[0049] Thus, by comprehensively integrating the vibration difference, the pressure difference, the lateral vibration signal parameters and the train running speed, the method of the present application can more comprehensively and accurately evaluate the defect confidence level of the floating slab track bed sealing strip. Compared with the prior art, the method of the present application improves the accuracy and reliability of defect detection and can effectively reduce false alarms and missed detections.

[0050] Further, the multi-modal data fusion defect assessment model is configured with: ; Among them, C is the defect confidence level, indicating the possibility of determining that there are defects in the sealing strip. The smaller the value of C, the lower the possibility of defects in the sealing strip; V d d(t) is the vibration difference function, indicating the vibration difference at time t; P d d(t) is the pressure difference function, indicating the pressure difference at time t; T is the total duration of data acquisition, λ is the vibration attenuation coefficient, indicating the coefficient of vibration difference decay over time, β is the coefficient of pressure difference change over time, S h h(i) is the value of the i-th lateral vibration signal continuously emitted by the controllable vibration source, n represents the number of sampling points of the lateral vibration signal, v is the train speed, v e e is the standard speed. The greater the difference between the train speed and the standard speed, the lower the defect confidence level.

[0051] This application calculates the defect confidence level of the floating slab sealing strip through a multi-modal data fusion defect evaluation model to improve the accuracy of defect detection. Specifically, the model uses the vibration difference function, the pressure difference function, the value of the lateral vibration signal of the controllable vibration source, and the difference between the train speed and the standard speed to calculate the defect confidence level. The vibration difference function and the pressure difference function respectively represent the vibration and pressure changes at specific time points. These data are accumulated over the total duration of data acquisition through integration to reflect the overall changes in vibration and pressure. The value of the lateral vibration signal of the controllable vibration source is calculated through the average value, reflecting the overall characteristics of the vibration signal. The difference between the train speed and the standard speed is used to adjust the confidence level. The greater the speed difference, the lower the confidence level. Through the fusion of the above multiple data, the model can more accurately evaluate the defect confidence level of the sealing strip, thereby improving the reliability of defect detection.

[0052] The specific implementation of the vibration difference function Vd(t) and the pressure difference function Pd(t) can be to collect vibration and pressure data in real time through sensors, and calculate the difference at each time point through a data processing algorithm. The value of the lateral vibration signal Sh(i) of the controllable vibration source can be to emit a stable lateral vibration signal through a preset vibration source and collect the signal value through a sensor. The train speed v and the standard speed ve can be obtained by real-time monitoring of the train's speed sensor. The total duration T of data acquisition can be set according to actual needs, usually one train operation cycle. The vibration attenuation coefficient λ and the coefficient β of pressure difference change over time can be fitted or preset through experimental data.

[0053] The technical solution of this application can more comprehensively consider various factors affecting the defects of the sealing strip through a multi-modal data fusion defect evaluation model, improving the accuracy and reliability of defect detection. Compared with the prior art, this application not only considers vibration and pressure changes, but also introduces the lateral vibration signal of the controllable vibration source and the difference between the train speed and the standard speed, further improving the precision and adaptability of the model. Thus, through the fusion and comprehensive evaluation of multi-modal data, this application provides a more reliable and accurate method for detecting defects in floating slab sealing strips.

[0054] Further, the defect type judgment step includes performing a fast Fourier transform on the vibration signal to obtain frequency domain features, identifying specific frequency peaks in the frequency domain features. At the same time, analyzing the slope, amplitude, and change period of the contact pressure change curve, and combining them with the parallelism parameter to construct a multi-dimensional feature vector. Through a pre-trained classification model, analyze the multi-dimensional feature vector and output the determination result of deformation defect or foreign object defect.

[0055] By analyzing the vibration signal and the contact pressure change curve in detail and combining with the parallelism parameter, a multi-dimensional feature vector can be constructed. Using a pre-trained classification model to analyze these multi-dimensional feature vectors can accurately determine whether the defect type of the sealing strip is a deformation defect or a foreign object defect. This technical solution obtains frequency domain features through a fast Fourier transform, combines the contact pressure change curve and the parallelism parameter to construct a multi-dimensional feature vector, and uses a classification model for determination, solving the technical problem of judging the defect type of the sealing strip.

[0056] Specifically, the fast Fourier transform of the vibration signal can convert the time domain signal into a frequency domain signal, making it easier to identify the peaks of specific frequencies. The slope, amplitude, and change period of the contact pressure change curve can reflect the response characteristics of the sealing strip under different pressure conditions. The parallelism parameter provides information about the geometric relationship between the sealing strip and the floating slab of the track bed. By combining these features into a multi-dimensional feature vector, the state of the sealing strip can be more comprehensively described.

[0057] As a preferred implementation, a hybrid architecture combining a convolutional neural network (CNN) and a long short-term memory network (LSTM) can be used to construct the classification model. The convolutional neural network can effectively extract the time-frequency features of the vibration signal and the contact pressure change curve, while the long short-term memory network is suitable for processing the time series information of these features. In addition, the dimension of the feature vector and the structure of the classification model can be adjusted according to the actual application situation to improve the accuracy and robustness of the determination result.

[0058] Thus, by constructing a multi-dimensional feature vector and using a pre-trained classification model for analysis, this application achieves an accurate determination of the defect types of the sealing strip. Compared with the prior art, the advantage of this application is that it can make more comprehensive use of multi-modal data, combined with advanced signal processing and machine learning techniques, improving the accuracy and reliability of defect type judgment.

[0059] Further, the defect feature judgment strategy includes setting multiple groups of feature parameter thresholds. When the specific frequency peak of the vibration signal is higher than the first threshold, the slope of the contact pressure change curve exceeds the second threshold, and the parallelism parameter is greater than the third threshold, it is determined as a deformation defect; when the specific frequency peak of the vibration signal is lower than the fourth threshold, the contact pressure change curve shows irregular fluctuations, and the parallelism parameter is within a specific range, it is determined as a foreign object defect.

[0060] The technical solution of this application can comprehensively evaluate the vibration signal, the contact pressure change curve, and the parallelism parameter by setting the thresholds of multiple feature parameters, thereby accurately judging the defect types of the sealing strip. Specifically, by setting the thresholds of multiple feature parameters, the vibration signal, the contact pressure change curve, and the parallelism parameter can be comprehensively evaluated, thereby accurately judging the defect types of the sealing strip. These technical features cooperate with each other to play a role in solving the problem of how to judge the deformation defect and foreign object defect of the floating slab sealing strip through multiple groups of feature parameter thresholds.

[0061] The implementation method of the defect feature judgment strategy can be various. For example, the specific frequency peak of the vibration signal can be extracted by fast Fourier transform (FFT), the slope of the contact pressure change curve can be calculated by numerical differentiation, and the parallelism parameter can be obtained by image processing technology. Specifically, the parallelism parameter can be obtained by detecting the edge of the sealing strip through the Hough transform, fitting the reference straight line by cubic spline interpolation, setting several uniformly spaced sampling points, calculating the width values of these sampling points and taking the average.

[0062] This application can accurately judge the deformation defect and foreign object defect of the floating slab sealing strip by setting multiple groups of feature parameter thresholds. Compared with the prior art, the method of this application has higher accuracy and reliability, can effectively reduce the situation of misjudgment and missed judgment, and improves the efficiency and accuracy of the defect detection of the floating slab sealing strip.

[0063] A defect detection system for the floating slab track bed sealing strip based on multi-modal data includes: A data acquisition module that acquires the image data of the track bed sealing strip through a linear array camera installed at the bottom of the train, acquires the track bed vibration data through an acceleration sensor, and acquires the floating slab contact pressure data through a pressure sensor; The preliminary defect detection module calculates the parallelism parameter between the sealing strip and the floating slab of the track bed based on the image data, and triggers the defect depth detection when the parallelism parameter is higher than the preset threshold; The defect depth detection module starts the controllable vibration source at the tail of the train to emit lateral vibration signals, synchronously collects the vibration response data and contact pressure data of the defect area, inputs them into the multi-modal data fusion defect evaluation model to calculate the defect confidence, and determines that there is a defect when the confidence exceeds the threshold; The defect type judgment module outputs deformation defects or foreign object defects through the defect feature judgment strategy based on the frequency domain characteristics of the vibration signal and the contact pressure change curve, combined with the parallelism parameter.

[0064] The above shows and describes the basic features, principles and advantages of the present invention. It should be noted that the present invention is not limited by the above embodiments, but only some embodiments. Without departing from the spirit and scope of the present invention, several improvements and supplements made are regarded as the protection scope of the present invention.

Claims

1. A floating slab ballast sealing strip defect detection method based on multimodal data, characterized in that: The steps include: Data collection step, collecting the image data of the roadbed sealing strip by a linear array camera installed at the bottom of the train, obtaining the vibration data of the roadbed by an acceleration sensor, and obtaining the contact pressure data of the floating plate by a pressure sensor; A preliminary defect detection step is to calculate the parallelism parameter between the sealing strip and the ballast floating plate based on the image data, and trigger the defect depth detection when the parallelism parameter is higher than a preset threshold; Defect depth detection step: start the controllable vibration source at the rear of the train to emit lateral vibration signals, synchronously collect vibration response data and pressure response data of the defect area, input them into the multimodal data fusion defect assessment model to calculate the defect confidence, and determine that there is a defect when the defect confidence exceeds the threshold; The defect type judgment step outputs deformation defects or foreign body defects through defect feature judgment strategies based on the frequency domain characteristics of the vibration signal and the contact pressure change curve in combination with the parallelism parameters.

2. The floating slab ballast sealing strip defect detection method based on multimodal data according to claim 1 is characterized in that: The data collection step also includes eliminating environmental noise from the vibration data by decomposing the vibration signal into a number of intrinsic mode functions, calculating the kurtosis coefficient of the intrinsic mode function, and superimposing the intrinsic mode function components whose kurtosis coefficient is greater than a preset kurtosis threshold to obtain a vibration signal after eliminating environmental noise.

3. The floating slab ballast sealing strip defect detection method based on multimodal data according to claim 1 is characterized in that: The parallelism parameter configurations are: ； Among them, P is the parallelism parameter, which indicates the parallelism between the sealing strip and the floating plate of the roadbed. The larger the P value, the lower the parallelism. i is the width value of the i-th sampling point on the bonding surface between the sealing strip and the floating plate in the image, is the average width value; the image data uses Hough transform to detect the edge of the sealing strip, fits the reference straight line through cubic spline interpolation, and sets a number of sampling points with uniform spacing.

4. The floating slab ballast sealing strip defect detection method based on multimodal data according to claim 1 is characterized in that: The multimodal data fusion defect assessment model adopts a CNN-LSTM hybrid architecture to extract the time-frequency features of vibration signals, wherein the convolutional neural network includes 5 convolutional layers, each with a convolution kernel size of 3×3 and a step size of 1, and also includes 3 maximum pooling layers, each with a pooling window size of 2×2.

5. The floating slab ballast sealing strip defect detection method based on multimodal data according to claim 1 is characterized in that: The defect depth detection step also includes subtracting the vibration response data and pressure response data from the initially collected ballast vibration data and contact pressure data to obtain a vibration difference and a pressure difference, and combining the vibration difference and the pressure difference with the lateral vibration signal parameters emitted by the controllable vibration source and the train speed to output a defect confidence through a multimodal data fusion defect assessment model.

6. The floating slab ballast sealing strip defect detection method based on multimodal data according to claim 1 is characterized in that: The multimodal data fusion defect assessment model is configured with: ； Where C is the defect confidence, which indicates the possibility of judging whether the sealing strip has defects. The smaller the value of C, the lower the possibility of the sealing strip having defects. d (t) is the vibration difference function, which represents the vibration difference at time t; P d (t) is the pressure difference function, which indicates the pressure difference at time t; T is the total duration of data acquisition, λ is the vibration attenuation coefficient, which indicates the coefficient of vibration difference attenuation over time, β is the coefficient of pressure difference changing over time, S h (i) is the i-th lateral vibration signal value continuously emitted by the controllable vibration source, n is the number of sampling points of the lateral vibration signal, v is the train speed, v e is the standard vehicle speed. The greater the difference between the vehicle speed and the standard vehicle speed, the lower the defect confidence.

7. The floating slab ballast sealing strip defect detection method based on multimodal data according to claim 1 is characterized in that: The defect type judgment step includes performing fast Fourier transform on the vibration signal to obtain frequency domain features, identifying specific frequency peaks in the frequency domain features, and at the same time, analyzing the slope, amplitude and change period of the contact pressure change curve, and combining them with the parallelism parameter to construct a multidimensional feature vector. The multidimensional feature vector is analyzed through a pre-trained classification model to output a judgment result of a deformation defect or a foreign body defect.

8. The floating slab ballast sealing strip defect detection method based on multimodal data according to claim 1 is characterized in that: The defect characteristic judgment strategy includes setting multiple sets of characteristic parameter thresholds. When the specific frequency peak of the vibration signal is higher than the first threshold, and the slope of the contact pressure change curve exceeds the second threshold, and the parallelism parameter is greater than the third threshold, it is judged as a deformation defect; when the specific frequency peak of the vibration signal is lower than the fourth threshold, and the contact pressure change curve shows irregular fluctuations, and the parallelism parameter is within a specific range, it is judged as a foreign body defect.

9. A floating slab roadbed sealing strip defect detection system based on multimodal data, applicable to the floating slab roadbed sealing strip defect detection method based on multimodal data as claimed in any one of claims 1 to 8, characterized in that: include: The data acquisition module collects the image data of the roadbed sealing strip through the linear array camera installed at the bottom of the train, obtains the roadbed vibration data through the acceleration sensor, and obtains the floating plate contact pressure data through the pressure sensor; A defect preliminary detection module calculates the parallelism parameter between the sealing strip and the ballast floating plate based on the image data, and triggers defect depth detection when the parallelism parameter is higher than a preset threshold; The defect depth detection module starts the controllable vibration source at the rear of the train to emit lateral vibration signals, synchronously collects vibration response data and contact pressure data of the defect area, and inputs them into the multimodal data fusion defect assessment model to calculate the defect confidence. When the confidence exceeds the threshold, it is determined that there is a defect; The defect type judgment module outputs deformation defects or foreign body defects through defect feature judgment strategies based on the frequency domain characteristics of the vibration signal and the contact pressure change curve combined with the parallelism parameters.

Images