Floating slab roadbed sealing strip defect detection method and system based on multimodal data

Abstract

The present invention provides a floating slab roadbed sealing strip defect detection method and system based on multimodal data, and multimodal data fusion technology to achieve efficient and accurate detection of floating slab roadbed sealing strip defects. The system integrates linear array cameras, acceleration sensors, pressure sensors and controllable vibration sources to construct a multi-dimensional data acquisition system, effectively overcoming the limitations of a single detection method. The collaborative analysis of image data with vibration and pressure data significantly improves the accuracy and robustness of defect identification. The defect assessment model based on the CNN-LSTM hybrid architecture can deeply mine the implicit features in multimodal data and accurately quantify the confidence of defects. At the same time, through the combination of preset feature parameter thresholds and classification models, intelligent classification of deformation defects and foreign body defects is achieved, providing a clear basis for subsequent maintenance.

Description

Technical Field

[0001] The present invention relates to the field of railway detection, and in particular to a method and system for detecting defects in a floating slab roadbed sealing strip based on multimodal data. Background Art

[0002] As a key component of rail transit systems, the sealing performance of floating slab roadbed sealing strips directly impacts the stability of the track structure, vibration and noise reduction, and train safety. Exposed to complex environments for extended periods, these strips are susceptible to factors such as train loads, temperature and humidity fluctuations, and foreign object intrusion, leading to defects such as deformation, cracking, and detachment. Timely and accurate detection of sealing strip defects is crucial to ensuring reliable rail system operation.

[0003] Currently, defect detection for floating slab roadbed sealing strips relies primarily on manual inspections, conducted through visual observation or simple tool measurement. However, manual inspections have the following limitations: low efficiency, making them difficult to adapt to the periodic inspection needs of large-scale lines; high subjectivity, with differences in inspector experience often leading to misjudgments or omissions; and inability to effectively identify hidden defects such as internal cracks and embedded foreign matter. Furthermore, a single inspection method cannot fully characterize the condition of the sealing strip, resulting in insufficient accuracy and robustness in the test results.

[0004] Therefore, in order to solve the above problems, the present invention proposes a floating slab roadbed sealing strip defect detection method and system based on multimodal data. Summary of the Invention

[0005] In view of the shortcomings of the prior art, the object of the present invention is to provide a floating slab roadbed sealing strip defect detection method and system based on multimodal data.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] The method for detecting defects in a floating slab roadbed sealing strip based on multimodal data comprises the following steps:

[0008] The data acquisition step involves collecting trackbed sealing strip image data using a linear array camera installed on the bottom of the train, obtaining trackbed vibration data using an acceleration sensor, and obtaining floating plate contact pressure data using a pressure sensor.

[0009] A preliminary defect detection step calculates a parallelism parameter between the sealing strip and the trackbed floating plate based on the image data, and triggers defect depth detection when the parallelism parameter is higher than a preset threshold;

[0010] In the defect depth detection step, the controllable vibration source at the rear of the train is activated to emit a lateral vibration signal, and the vibration response data and pressure response data of the defect area are simultaneously collected. The data are input into the multimodal data fusion defect assessment model to calculate the defect confidence level. When the defect confidence level exceeds a threshold, a defect is determined to exist.

[0011] The defect type judgment step outputs deformation defects or foreign body defects through the defect feature judgment strategy based on the frequency domain characteristics of the vibration signal and the contact pressure change curve combined with the parallelism parameters.

[0012] As a further improvement of the present invention, the data acquisition step also includes eliminating environmental noise from the vibration data by decomposing the vibration signal into a number of intrinsic mode functions, calculating the kurtosis coefficient of the intrinsic mode function, and superimposing the intrinsic mode function components whose kurtosis coefficient is greater than a preset kurtosis threshold to obtain a vibration signal after eliminating environmental noise.

[0013] As a further improvement of the present invention, the parallelism parameter configuration includes:

[0014] ;

[0015] Among them, P is the parallelism parameter, which indicates the parallelism between the sealing strip and the roadbed floating plate. The larger the P value, the lower the parallelism. i is the width of the i-th sampling point on the bonding surface between the sealing strip and the floating plate in the image, is the average width value; the image data uses Hough transform to detect the edge of the sealing strip, and the reference straight line is fitted by cubic spline interpolation, and a number of sampling points with uniform spacing are set.

[0016] As a further improvement of the present invention, the multimodal data fusion defect assessment model adopts a CNN-LSTM hybrid architecture to extract the time-frequency features of the vibration signal, wherein the convolutional neural network includes 5 convolution layers, each with a convolution kernel size of 3×3 and a step size of 1, and also includes 3 maximum pooling layers, each with a pooling window size of 2×2.

[0017] As a further improvement of the present invention, the defect depth detection step also includes subtracting the vibration response data and pressure response data from the initially collected roadbed vibration data and contact pressure data to obtain a vibration difference and a pressure difference, and combining the vibration difference and the pressure difference with the lateral vibration signal parameters emitted by the controllable vibration source and the train speed to output the defect confidence through a multimodal data fusion defect assessment model.

[0018] As a further improvement of the present invention, the multimodal data fusion defect assessment model is configured with:

[0019] ;

[0020] Among them, C is the defect confidence, which indicates the possibility of judging whether the sealing strip has defects. The smaller the value of C, the lower the possibility of the sealing strip having defects. d (t) is the vibration difference function, which represents the vibration difference at time t; P d (t) is the pressure difference function, which represents the pressure difference at time t; T is the total time of data acquisition, λ is the vibration attenuation coefficient, which represents the coefficient of vibration difference attenuation over time, β is the coefficient of pressure difference changing with time, S h (i) is the i-th lateral vibration signal value continuously emitted by the controllable vibration source, n is the number of lateral vibration signal sampling points, v is the train speed, v e is the standard vehicle speed. The greater the difference between the vehicle speed and the standard vehicle speed, the lower the defect confidence.

[0021] As a further improvement of the present invention, the defect type judgment step includes performing a fast Fourier transform on the vibration signal to obtain frequency domain features, identifying the frequency peak in the frequency domain features, and at the same time, analyzing the slope, amplitude and change period of the contact pressure change curve, and combining them with the parallelism parameters to construct a multidimensional feature vector. The multidimensional feature vector is analyzed through a pre-trained classification model to output the judgment result of deformation defect or foreign body defect.

[0022] As a further improvement of the present invention, the defect feature judgment strategy includes setting multiple sets of characteristic parameter thresholds. When the frequency peak of the vibration signal is higher than the first threshold, and the slope of the contact pressure change curve exceeds the second threshold, and the parallelism parameter is greater than the third threshold, it is judged as a deformation defect; when the frequency peak of the vibration signal is lower than the fourth threshold, and the contact pressure change curve shows irregular fluctuations, and the parallelism parameter is less than the third threshold, it is judged as a foreign body defect.

[0023] The floating slab roadbed sealing strip defect detection system based on multimodal data includes:

[0024] The data acquisition module collects trackbed sealing strip image data through a linear array camera installed on the bottom of the train, obtains trackbed vibration data through an acceleration sensor, and obtains floating plate contact pressure data through a pressure sensor;

[0025] A preliminary defect detection module calculates the parallelism parameter between the sealing strip and the trackbed floating plate based on the image data, and triggers defect depth detection when the parallelism parameter exceeds a preset threshold;

[0026] The defect depth detection module activates the controllable vibration source at the rear of the train to emit lateral vibration signals, synchronously collects vibration response data and contact pressure data of the defect area, and inputs them into the multimodal data fusion defect assessment model to calculate the defect confidence level. When the confidence level exceeds a threshold, a defect is determined to exist.

[0027] The defect type judgment module outputs deformation defects or foreign body defects through defect feature judgment strategies based on the frequency domain characteristics of the vibration signal and the contact pressure change curve, combined with the parallelism parameters.

[0028] The beneficial effects of the present invention are as follows: the present invention realizes efficient and accurate detection of defects in the sealing strips of floating slab roadbeds through multimodal data fusion technology. The system integrates linear array cameras, acceleration sensors, pressure sensors and controllable vibration sources to construct a multi-dimensional data acquisition system, which effectively overcomes the limitations of a single detection method. The collaborative analysis of image data with vibration and pressure data significantly improves the accuracy and robustness of defect identification. The defect assessment model based on the CNN-LSTM hybrid architecture can deeply mine the implicit features in multimodal data and accurately quantify the confidence of defects. At the same time, through the combination of preset feature parameter thresholds and classification models, intelligent classification of deformation defects and foreign body defects is achieved, providing a clear basis for subsequent maintenance. This method not only reduces the subjectivity and missed detection rate of manual detection, but also improves detection efficiency through real-time data acquisition and online analysis. It is suitable for dynamic monitoring needs in rail transit operations and has significant engineering application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a flow chart of the floating slab roadbed sealing strip defect detection method based on multimodal data of the present invention.

[0030] Figure 2 It is a data collection and processing flow chart of the floating slab roadbed sealing strip defect detection method based on multimodal data of the present invention. DETAILED DESCRIPTION

[0031] The present invention will be described in further detail below with reference to the accompanying drawings and embodiments. Identical components are denoted by the same reference numerals. It should be noted that the terms "front," "rear," "left," "right," "upper," and "lower" used in the following description refer to directions in the accompanying drawings, and the terms "bottom," "top," "inner," and "outer" refer to directions toward or away from the geometric center of a component, respectively.

[0032] The present invention relates to a method for detecting defects in sealing strips of floating slab roadbeds based on multimodal data, aiming to solve the problem of how to detect defects in sealing strips of floating slab roadbeds. In the prior art, defect detection of sealing strips of floating slab roadbeds mainly relies on manual visual inspection, which has the problems of low detection efficiency, strong subjectivity, high missed detection and false detection rates. In order to solve these problems, the present invention proposes a method for detecting defects in sealing strips of floating slab roadbeds based on multimodal data, which collects data through multiple sensors and combines multimodal data fusion technology to achieve efficient and accurate detection of sealing strip defects, such as Figure 1 As shown, the following steps are included:

[0033] The data acquisition step is to collect the image data of the roadbed sealing strip through a linear array camera installed on the bottom of the train. The linear array camera is set at the front of the train, or the front half of the train close to the front of the train, to ensure that after the system completes processing the image data, the roadbed vibration data can be obtained again when the train passes the position where the abnormal image is collected. The roadbed vibration data is obtained through an acceleration sensor. Several acceleration sensors are evenly set at the bottom of the train, and the floating plate contact pressure data is obtained through a pressure sensor. Several pressure sensors are set at the bottom of the train. The collected sealing strip image data is used for direct judgment of crack defects, or for judgment of the parallelism between the sealing strip and the roadbed. The collected vibration data and contact pressure data are used for subsequent further judgment of the defect condition and defect type of the floating plate roadbed sealing strip.

[0034] The preliminary defect detection step is to calculate the parallelism parameter between the sealing strip and the floating plate of the roadbed based on the image data. When the parallelism parameter is higher than the preset threshold, the defect depth detection is triggered; when the parallelism parameter is higher than the preset threshold, it means that there is a non-parallel situation between the floating plate of the roadbed and the sealing strip in the image data at this time. Under normal circumstances, since the sealing strip is tightly attached to the floating plate roadbed for sealing, the sealing strip and the floating plate roadbed are parallel in the absence of abnormalities. However, when the sealing strip is deformed, squeezed by foreign objects, etc., the sealing strip will obviously no longer be parallel to the floating plate roadbed.

[0035] In the defect depth detection step, the controllable vibration source at the rear of the train is activated to emit a lateral vibration signal, and the vibration response data and pressure response data of the defect area are simultaneously collected. The data are input into the multimodal data fusion defect assessment model to calculate the defect confidence level. When the defect confidence level exceeds a threshold, a defect is determined to exist.

[0036] The defect type judgment step outputs deformation defects or foreign body defects through the defect feature judgment strategy based on the frequency domain characteristics of the vibration signal and the contact pressure change curve combined with the parallelism parameters.

[0037] The data acquisition step acquires the image data, vibration data and contact pressure data of the roadbed sealing strip through a linear array camera, an accelerometer and a pressure sensor. These data provide basic information for subsequent defect detection. The preliminary defect detection step preliminarily determines whether the sealing strip has defects by calculating the parallelism parameter in the image data. When the parallelism parameter is higher than the preset threshold, the defect depth detection is triggered. The defect depth detection step emits a lateral vibration signal through a controllable vibration source and synchronously collects the vibration response data and pressure response data of the defect area. The defect confidence is calculated using a multimodal data fusion defect assessment model. When the defect confidence exceeds the threshold, it is determined that a defect exists. The defect type judgment step is based on the frequency domain characteristics of the vibration signal and the contact pressure change curve, combined with the parallelism parameter, and outputs deformation defects or foreign body defects through a defect feature judgment strategy.

[0038] By integrating and analyzing multimodal data, the system effectively overcomes the shortcomings of existing technologies and achieves efficient and accurate detection of defects in floating slab roadbed sealing strips. In particular, the comprehensive analysis of image, vibration, and pressure data enables accurate identification of the type of sealing strip defect, providing a reliable basis for subsequent repair and maintenance.

[0039] Furthermore, the data collection step also includes: Figure 2 As shown, the vibration data is subjected to environmental noise elimination. After decomposing the vibration signal into several intrinsic mode functions, the kurtosis coefficients of the intrinsic mode functions are calculated, and the intrinsic mode function components with kurtosis coefficients greater than a preset kurtosis threshold are superimposed to obtain a vibration signal after environmental noise elimination. Through the above technical means, the present application solves the problem of environmental noise interference in vibration data, ensures the accuracy and reliability of the vibration data used in the subsequent defect depth detection step, and thus improves the accuracy and stability of defect detection.

[0040] This application decomposes the vibration signal into several intrinsic mode functions (IMFs), calculates the kurtosis coefficient of each IMF, selects IMF components with kurtosis coefficients greater than a preset kurtosis threshold, and superimposes them to ultimately obtain a vibration signal after eliminating environmental noise. In this way, environmental noise interference in the vibration signal can be effectively removed, retaining the effective vibration signal components. Specifically, this process includes the following steps:

[0041] In practical applications, accelerometers are used to collect vibration signals from the floating slab trackbed. The selection of the sensor is crucial, and parameters such as its range, accuracy, and sampling frequency must be considered. For example, an accelerometer with a range of ±50g, an accuracy of 0.01g, and a sampling frequency of 10kHz can be selected to accurately capture the details of the vibration signal. The collected vibration signals are typically stored as a time series, with each time point corresponding to a vibration amplitude.

[0042] Before further processing, the collected raw vibration signals need to be initially checked to ensure data integrity and accuracy. If missing or outliers are found in the data, methods such as linear interpolation or median filtering can be used to correct them.

[0043] The collected vibration signal is then subjected to empirical mode decomposition (EMD) to obtain several intrinsic mode functions (IMFs): the given vibration signal x(t) is fitted with all local maximum and local minimum points of the signal using the cubic spline interpolation function to obtain the upper envelope of the signal e up (t) and the lower envelope e low (t), calculate the mean of the upper envelope and the lower envelope , subtract the mean of the envelope from the original vibration signal to obtain h1(t)=x(t)-m(t), h1(t) is the IMF component we want to determine whether it can be extracted, and determine whether h1(t) meets the IMF condition, that is, the number of local extreme points and zero crossing points is equal or differs by at most 1, and the upper and lower envelopes are locally symmetrical about the time axis. If not, take h1(t) as the new signal and repeat the above steps until h1(t) meets the conditions. 1k (t), the first IMF component c1(t) can be extracted at this time, and the first IMF component is subtracted from the original vibration signal to obtain the residual signal r1(t)=x(t)-c1(t). Take r1(t) as the new signal and repeat the above steps to extract all IMF components until the residual signal becomes a monotonic function or a constant.

[0044] Finally, the original vibration signal after empirical mode decomposition is expressed as , where c i (t) is the i-th IMF component, r n (t) is the remaining component.

[0045] Then the kurtosis coefficient is calculated. The kurtosis coefficient is a statistic used to measure the distribution of the signal, which can reflect the sharpness or impulse characteristics of the signal. For each IMF component c i (t), whose kurtosis is

[0046] The calculation formula for the number is ,in, For each IMF component and mean

[0047] The expectation of the difference, c i The mean of (t), c i Since the true probability distribution of the vibration signal cannot be obtained, the sample kurtosis coefficient is used here to approximate the standard deviation of (t). For an IMF with a length of N

[0048] Component c i (t), its sample kurtosis coefficient ;in, c i (t)

[0049] The mean value, c i (j) is c i (t) is the amplitude at the jth time point.

[0050] A preset kurtosis threshold is set. The setting of the preset kurtosis threshold needs to be adjusted according to the specific application scenario and experimental data. Generally speaking, by analyzing a large number of vibration signals under normal operating conditions, the kurtosis coefficient distribution of their IMF components can be statistically analyzed. Then, an appropriate threshold is selected so that the kurtosis coefficients of the IMF components of most noise components are less than the threshold, while the kurtosis coefficients of the IMF components of the effective signal components are greater than the threshold. Under normal operating conditions, the kurtosis coefficients of the IMF components of noise components are mostly concentrated between 1 and 3, while the kurtosis coefficients of the IMF components of the effective signal components are usually greater than 3. Therefore, the preset kurtosis threshold is set to 3 here. Based on the set kurtosis threshold, IMF components with kurtosis coefficients greater than the threshold are screened out. These components are formed into a new set, and the screened IMF components are superimposed to obtain the vibration signal after the environmental noise is eliminated.

[0051] The empirical mode decomposition (EMD) method described above is an adaptive signal processing method that decomposes complex signals into several physically meaningful intrinsic mode functions (IMFs). Calculating the kurtosis coefficient is commonly used in signal processing and can effectively distinguish signals containing impulse components. By setting an appropriate kurtosis threshold, environmental noise can be effectively removed while retaining the useful vibration signal components.

[0052] Furthermore, the parallelism parameter configuration includes:

[0053] ;

[0054] Among them, P is the parallelism parameter, which indicates the parallelism between the sealing strip and the roadbed floating plate. The larger the P value, the lower the parallelism. i is the width of the i-th sampling point on the bonding surface between the sealing strip and the floating plate in the image, is the average width value; the image data uses Hough transform to detect the edge of the sealing strip, and the reference straight line is fitted by cubic spline interpolation, and a number of sampling points with uniform spacing are set.

[0055] The technical features included in this application are the calculation formula of parallelism parameters, the processing method of image data and the setting of sampling points, which play an important role in solving the quantification problem of parallelism between the floating plate sealing strip and the roadbed floating plate.

[0056] The calculation formula for the parallelism parameter quantifies the parallelism between the sealing strip and the trackbed floating plate by averaging the width values at multiple sampling points. The image data processing method uses a Hough transform to detect the edges of the sealing strip and fits a reference line through cubic spline interpolation, thereby ensuring the accuracy and uniform distribution of the sampling points. These technical features work together to effectively solve the problem of quantifying the parallelism between the floating plate sealing strip and the trackbed floating plate. Next, several evenly spaced sampling points are set, the width value Wi of each sampling point is calculated, and the average value W of these width values is calculated. Finally, the parallelism parameter P is calculated using a formula. A larger P value indicates a lower parallelism, thus effectively quantifying the parallelism between the floating plate sealing strip and the trackbed floating plate, ensuring the accuracy and uniform distribution of the sampling points. Compared with the existing technology, this application provides a more accurate and reliable parallelism quantification method by introducing the parallelism parameter and image data processing method. This method can better determine the parallelism between the sealing strip and the trackbed floating plate, thereby improving the accuracy and reliability of defect detection.

[0057] Furthermore, the multimodal data fusion defect assessment model adopts a CNN-LSTM hybrid architecture to extract the time-frequency features of the vibration signal, wherein the convolutional neural network includes 5 convolution layers, each with a convolution kernel size of 3×3 and a step size of 1, and also includes 3 maximum pooling layers, each with a pooling window size of 2×2.

[0058] The multimodal data fusion defect assessment model utilizes a CNN-LSTM hybrid architecture. The CNN layer consists of five convolutional layers, each with a 3×3 kernel size and a stride of 1. It also includes three max pooling layers, each with a 2×2 pooling window size. The combination of convolutional and pooling layers extracts spatial features from vibration signals. The LSTM layer extracts temporal features. This hybrid architecture combines the spatial feature extraction capabilities of convolutional neural networks with the temporal feature extraction capabilities of long-short-term memory networks to effectively extract the time-frequency features of vibration signals, thereby enabling multimodal data fusion defect assessment.

[0059] Convolutional neural networks (CNNs), through a combination of multiple convolutional and pooling layers, can effectively extract spatial features from vibration signals. Each convolutional layer uses a 3×3 kernel with a stride of 1, ensuring that the details of the vibration signal are fully captured. The maximum pooling layer uses a 2×2 pooling window, which downsamples the feature map to reduce its size and computational complexity. The long short-term memory (LSTM) network, through its memory and forgetting mechanisms, can capture temporal features in vibration signals, making it particularly suitable for processing time-dependent signals.

[0060] Specifically, the CNN first processes the input vibration signal, extracting its spatial features layer by layer through a combination of five convolutional layers and three max pooling layers. These extracted features are then fed into the LSTM layer for temporal feature extraction. Through its unique gating mechanism, the LSTM layer captures long-term dependencies in the vibration signal, further improving feature extraction accuracy.

[0061] In this embodiment, ResNet-LSTM can also be used to construct the model framework. Specifically, ResNet (residual network), as a deep convolutional neural network, has significant advantages in processing complex feature extraction tasks. Combining it with LSTM (long short-term memory network) can more effectively analyze multimodal data and achieve more accurate defect assessment.

[0062] The core advantage of ResNet lies in its unique residual block structure. By introducing skip connections, residual blocks allow the network to more easily learn the identity mapping during training, thereby alleviating the vanishing and exploding gradient problems common in deep neural networks. This allows the network to be built deeper and capture richer spatial features. In this model, ResNet can be constructed by stacking multiple residual blocks. For example, a ResNet architecture with 18, 34, or even deeper layers can be constructed. The specific number of layers can be adjusted based on the complexity of the actual data and computing resources. Each residual block contains a convolutional layer, a batch normalization layer, and an activation function layer. The convolutional kernel size of the convolutional layer can be customized according to actual needs. For example, a 3×3 convolution kernel ensures the capture of feature details while limiting the number of model parameters. The batch normalization layer accelerates model convergence and improves model stability. The activation function layer introduces nonlinearity to the network, enhancing its expressive power.

[0063] To process vibration signals, the ResNet component first performs preliminary feature extraction on the input vibration signal. Through a series of convolution operations and residual blocks, it gradually mines the complex spatial features in the signal. These spatial features reflect the distribution of the vibration signal across different frequencies and amplitudes and are crucial for identifying abnormal patterns and characteristics in the signal.

[0064] The LSTM layer further exploits the temporal characteristics of the signal based on the spatial features extracted by ResNet. LSTM has a unique gating mechanism, including an input gate, a forget gate, and an output gate. The forget gate determines which information to discard from the cell state, the input gate determines which new information to add to the cell state, and the output gate determines the output based on the cell state. This gating mechanism enables LSTM to effectively process time-dependent sequence data, capturing the changing trends and long-term dependencies of the vibration signal at different time points.

[0065] Throughout the model process, the vibration signal in the input multimodal data first passes through the ResNet to extract spatial features. Through its deep network structure and residual blocks, ResNet extracts high-level spatial features of the signal layer by layer. These extracted spatial features are then passed as input to the LSTM component. When processing these features, the LSTM considers their order and variation in time. Using its gating mechanism, it filters and processes the information to extract more representative temporal features. Ultimately, these results, combining spatial and temporal features, are used in multimodal data fusion defect assessment, enabling more accurate identification of potential defects in the data and improving the accuracy and reliability of the assessment.

[0066] Compared to the previously used CNN-LSTM hybrid architecture, the ResNet-LSTM architecture boasts stronger feature extraction capabilities when processing complex multimodal data. In the CNN-LSTM hybrid architecture, the CNN component extracts spatial features from vibration signals through five convolutional layers (each with a 3×3 kernel size and a stride of 1) and three max pooling layers (each with a 2×2 pooling window size). ResNet, with its residual block structure and deeper network layers, is able to more deeply explore spatial features in the signal and is more stable during training. While the LSTM component plays a key role in extracting temporal features in both architectures, the ResNet-LSTM architecture processes the results of more advanced spatial feature extraction, better capturing temporal variations in the signal and further improving model performance.

[0067] Through this hybrid architecture, this application effectively extracts the time-frequency features of vibration signals, enabling multimodal data fusion for defect assessment. Compared to existing technologies, this application offers significant advantages in feature extraction accuracy and efficiency, enabling more accurate assessment of defects in floating slab roadbed sealing strips, improving the reliability and accuracy of defect detection.

[0068] Furthermore, the defect depth detection step also includes subtracting the vibration response data and pressure response data from the initially collected roadbed vibration data and contact pressure data to obtain a vibration difference and a pressure difference, and combining the vibration difference and the pressure difference with the lateral vibration signal parameters emitted by the controllable vibration source and the train speed to output the defect confidence through a multimodal data fusion defect assessment model.

[0069] During the defect depth detection step, vibration and pressure difference values are calculated by comparing the difference between the vibration and pressure response data and the initial data. These differences are combined with the lateral vibration signal parameters emitted by the controllable vibration source and the train speed, and the defect confidence level is calculated using a multimodal data fusion defect assessment model. The calculation of vibration and pressure difference values reflects the response changes of the sealing strip under controlled vibration. Combined with the train speed and lateral vibration signal parameters, the defect confidence level of the sealing strip can be more accurately assessed, ensuring the reliability of the inspection results.

[0070] Specifically, the vibration difference is calculated by comparing the current vibration response data with the initial trackbed vibration data. Similarly, the pressure difference is calculated by comparing the current pressure response data with the initial contact pressure data. These differences reflect the changes in the sealing strip under the influence of the controlled vibration source. The lateral vibration signal parameters emitted by the controlled vibration source and the train speed are input into the multimodal data fusion defect assessment model as supplementary information. By integrating this multimodal data, the model outputs a defect confidence value. A higher defect confidence value indicates a greater likelihood of a sealing strip defect.

[0071] Vibration and pressure differences can be analyzed using signal processing techniques such as fast Fourier transform (FFT) or short-time Fourier transform (STFT) to extract useful frequency domain features. Lateral vibration signal parameters from a controllable vibration source can include information such as vibration frequency and amplitude. Train speed can be acquired in real time using speed sensors. Multimodal data fusion defect assessment models can utilize deep learning methods, such as a hybrid architecture of convolutional neural networks (CNN) and long short-term memory networks (LSTM), to extract and fuse multiple data features for accurate defect confidence assessment.

[0072] Therefore, by integrating vibration difference, pressure difference, lateral vibration signal parameters, and train speed, the method of this application can more comprehensively and accurately assess the confidence level of defects in the floating slab roadbed sealing strip. Compared with existing technologies, the method of this application improves the accuracy and reliability of defect detection and can effectively reduce false positives and missed positives.

[0073] Furthermore, the multimodal data fusion defect assessment model is configured with:

[0074] ;

[0075] Among them, C is the defect confidence, which indicates the possibility of judging whether the sealing strip has defects. The smaller the value of C, the lower the possibility of the sealing strip having defects. d (t) is the vibration difference function, which represents the vibration difference at time t; P d (t) is the pressure difference function, which represents the pressure difference at time t; T is the total time of data acquisition, λ is the vibration attenuation coefficient, which represents the coefficient of vibration difference attenuation over time, β is the coefficient of pressure difference changing with time, S h (i) is the i-th lateral vibration signal value continuously emitted by the controllable vibration source, n is the number of lateral vibration signal sampling points, v is the train speed, v e is the standard vehicle speed. The greater the difference between the vehicle speed and the standard vehicle speed, the lower the defect confidence.

[0076] This application uses a multimodal data fusion defect assessment model to calculate the defect confidence of the floating plate sealing strip to improve the accuracy of defect detection. Specifically, the model uses the vibration difference function, the pressure difference function, the lateral vibration signal value of the controllable vibration source, and the difference between the train speed and the standard speed to calculate the defect confidence. The vibration difference function and the pressure difference function represent the vibration and pressure changes at a time point, respectively. These data are accumulated by integration within the total duration of data acquisition to reflect the overall changes in vibration and pressure. The lateral vibration signal value of the controllable vibration source is calculated by the average value, reflecting the overall characteristics of the vibration signal. The difference between the train speed and the standard speed is used to adjust the confidence. The greater the speed difference, the lower the confidence. Through the fusion of the above-mentioned multiple data, the model can more accurately evaluate the defect confidence of the sealing strip, thereby improving the reliability of defect detection.

[0077] The vibration difference function Vd(t) and the pressure difference function Pd(t) can be implemented by collecting vibration and pressure data in real time through sensors, and calculating the difference at each time point using a data processing algorithm. The lateral vibration signal Sh(i) of the controllable vibration source can be generated by a preset vibration source to generate a stable lateral vibration signal, and the signal value is collected by sensors. The train speed v and the standard speed ve can be obtained by monitoring the train's speed sensor in real time. The total data collection time T can be set according to actual needs, typically one train operation cycle. The vibration attenuation coefficient λ and the time-varying pressure difference coefficient β can be fitted or preset using experimental data.

[0078] The technical solution of this application, through a multimodal data fusion defect assessment model, can more comprehensively consider the various factors affecting sealing strip defects, thereby improving the accuracy and reliability of defect detection. Compared with the existing technology, this application not only considers vibration and pressure changes, but also introduces the lateral vibration signal of a controllable vibration source and the difference between the train speed and the standard speed, further improving the accuracy and adaptability of the model. Thus, through the fusion and comprehensive evaluation of multimodal data, this application provides a more reliable and accurate method for floating plate sealing strip defect detection.

[0079] Furthermore, the defect type judgment step includes performing fast Fourier transform on the vibration signal to obtain frequency domain features, identifying frequency peaks in the frequency domain features, and at the same time, analyzing the slope, amplitude and change period of the contact pressure change curve, and combining them with the parallelism parameters to construct a multidimensional feature vector. The multidimensional feature vector is analyzed through a pre-trained classification model to output the judgment result of deformation defect or foreign body defect.

[0080] By analyzing the vibration signal and contact pressure curve in detail, combined with the parallelism parameter, a multidimensional feature vector can be constructed. Using a pre-trained classification model, these multidimensional feature vectors can be analyzed to accurately determine whether the sealing strip defect is a deformation defect or a foreign object defect. This technical solution uses fast Fourier transform to obtain frequency domain features. Combining the contact pressure curve and parallelism parameters, a multidimensional feature vector is constructed and determined using the classification model, solving the technical problem of determining the type of sealing strip defect.

[0081] Specifically, the Fast Fourier Transform (FFT) of the vibration signal converts the time-domain signal into the frequency-domain, making it easier to identify frequency peaks. The slope, amplitude, and period of the contact pressure curve reflect the response characteristics of the sealing strip under different pressure conditions. The parallelism parameter provides information on the geometric relationship between the sealing strip and the trackbed floating plate. By combining these features into a multidimensional feature vector, a more comprehensive description of the sealing strip's condition can be achieved.

[0082] As a preferred implementation, a hybrid architecture combining a convolutional neural network (CNN) and a long short-term memory network (LSTM) can be used to construct the classification model. The CNN effectively extracts the time-frequency features of vibration signals and contact pressure curves, while the LSTM is suitable for processing the time series information of these features. Furthermore, the dimension of the feature vector and the structure of the classification model can be adjusted according to the actual application to improve the accuracy and robustness of the judgment results.

[0083] Therefore, this application accurately determines the type of sealing strip defect by constructing a multidimensional feature vector and analyzing it using a pre-trained classification model. Compared with existing technologies, this application's advantage lies in its ability to more comprehensively utilize multimodal data, combined with advanced signal processing and machine learning techniques, to improve the accuracy and reliability of defect type judgment.

[0084] Furthermore, the defect characteristic judgment strategy includes setting multiple sets of characteristic parameter thresholds. When the frequency peak of the vibration signal is higher than the first threshold, and the slope of the contact pressure change curve exceeds the second threshold, and the parallelism parameter is greater than the third threshold, it is judged as a deformation defect; when the frequency peak of the vibration signal is lower than the fourth threshold, and the contact pressure change curve shows irregular fluctuations, and the parallelism parameter is less than the third threshold, it is judged as a foreign body defect.

[0085] The technical solution of this application, by setting thresholds for multiple characteristic parameters, enables a comprehensive evaluation of the vibration signal, contact pressure variation curve, and parallelism parameters, thereby accurately determining the type of sealing strip defect. Specifically, by setting thresholds for multiple characteristic parameters, a comprehensive evaluation of the vibration signal, contact pressure variation curve, and parallelism parameters is enabled, thereby accurately determining the type of sealing strip defect. These technical features, working in conjunction with each other, play a role in solving the problem of how to determine deformation defects and foreign matter defects in floating plate sealing strips using multiple sets of characteristic parameter thresholds.

[0086] Defect feature detection strategies can be implemented in a variety of ways. For example, the frequency peak of the vibration signal can be extracted using a fast Fourier transform (FFT), the slope of the contact pressure curve can be calculated using numerical differentiation, and parallelism parameters can be obtained using image processing techniques. Specifically, the parallelism parameters can be obtained by detecting the edge of the sealing strip using a Hough transform, fitting a reference line using cubic spline interpolation, setting several evenly spaced sampling points, calculating the width of these sampling points, and averaging them.

[0087] By setting multiple sets of characteristic parameter thresholds, this application can accurately identify deformation defects and foreign matter defects in floating slab sealing strips. Compared to existing technologies, this method has higher accuracy and reliability, effectively reducing misjudgments and missed detections, and improving the efficiency and precision of floating slab sealing strip defect detection.

[0088] The floating slab roadbed sealing strip defect detection system based on multimodal data includes:

[0089] The data acquisition module collects trackbed sealing strip image data through a linear array camera installed on the bottom of the train, obtains trackbed vibration data through an acceleration sensor, and obtains floating plate contact pressure data through a pressure sensor;

[0090] A preliminary defect detection module calculates the parallelism parameter between the sealing strip and the trackbed floating plate based on the image data, and triggers defect depth detection when the parallelism parameter exceeds a preset threshold;

[0091] The defect depth detection module activates the controllable vibration source at the rear of the train to emit lateral vibration signals, synchronously collects vibration response data and contact pressure data of the defect area, and inputs them into the multimodal data fusion defect assessment model to calculate the defect confidence level. When the confidence level exceeds a threshold, a defect is determined to exist.

[0092] The defect type judgment module outputs deformation defects or foreign body defects through defect feature judgment strategies based on the frequency domain characteristics of the vibration signal and the contact pressure change curve, combined with the parallelism parameters.

[0093] The above shows and describes the basic features, principles, and advantages of the present invention. It should be noted that the present invention is not limited to the above embodiments, which are only some embodiments. Without departing from the spirit and scope of the present invention, various improvements and supplements made are considered to be within the scope of protection of the present invention.

Claims

1. A floating slab roadbed sealing strip defect detection method based on multimodal data, characterized in that: The steps include: The data acquisition step involves collecting trackbed sealing strip image data using a linear array camera installed on the bottom of the train, obtaining trackbed vibration data using an acceleration sensor, and obtaining floating plate contact pressure data using a pressure sensor. A preliminary defect detection step calculates a parallelism parameter between the sealing strip and the trackbed floating plate based on the image data, and triggers defect depth detection when the parallelism parameter is higher than a preset threshold; In the defect depth detection step, the controllable vibration source at the rear of the train is activated to emit a lateral vibration signal, and the vibration response data and pressure response data of the defect area are simultaneously collected. The data are input into the multimodal data fusion defect assessment model to calculate the defect confidence level. When the defect confidence level exceeds a threshold, a defect is determined to exist. Defect type judgment step: Based on the frequency domain characteristics of the vibration signal and the contact pressure change curve, combined with the parallelism parameter, the defect characteristic judgment strategy outputs deformation defects or foreign body defects; The defect type determination step includes performing a fast Fourier transform on the vibration signal to obtain frequency domain features, identifying frequency peaks in the frequency domain features, analyzing the slope, amplitude, and change period of the contact pressure change curve, combining the slope, amplitude, and change period with the parallelism parameter, constructing a multidimensional feature vector, analyzing the multidimensional feature vector using a pre-trained classification model, and outputting a determination result of a deformation defect or a foreign matter defect; The defect characteristic judgment strategy includes setting multiple sets of characteristic parameter thresholds. When the frequency peak of the vibration signal is higher than the first threshold, the slope of the contact pressure change curve exceeds the second threshold, and the parallelism parameter is greater than the third threshold, it is judged as a deformation defect; when the frequency peak of the vibration signal is lower than the fourth threshold, the contact pressure change curve shows irregular fluctuations, and the parallelism parameter is less than the third threshold, it is judged as a foreign body defect.

2. The floating slab roadbed sealing strip defect detection method based on multimodal data according to claim 1 is characterized in that: The data collection step also includes eliminating environmental noise from the vibration data by decomposing the vibration signal into a plurality of intrinsic mode functions, calculating the kurtosis coefficients of the intrinsic mode functions, and superimposing the intrinsic mode function components whose kurtosis coefficients are greater than a preset kurtosis threshold to obtain a vibration signal after eliminating environmental noise.

3. The floating slab roadbed sealing strip defect detection method based on multimodal data according to claim 1 is characterized in that: The parallelism parameter configurations are: ； Among them, P is the parallelism parameter, which indicates the parallelism between the sealing strip and the roadbed floating plate. The larger the P value, the lower the parallelism. i is the width of the i-th sampling point on the bonding surface between the sealing strip and the floating plate in the image, is the average width value; the image data uses Hough transform to detect the edge of the sealing strip, and the reference straight line is fitted by cubic spline interpolation, and a number of sampling points with uniform spacing are set.

4. The floating slab roadbed sealing strip defect detection method based on multimodal data according to claim 1 is characterized in that: The multimodal data fusion defect assessment model adopts a CNN-LSTM hybrid architecture to extract the time-frequency features of vibration signals, wherein the convolutional neural network includes five convolutional layers, each with a convolution kernel size of 3×3 and a step size of 1, and also includes three maximum pooling layers, each with a pooling window size of 2×2.

5. The floating slab roadbed sealing strip defect detection method based on multimodal data according to claim 1 is characterized in that: The defect depth detection step also includes subtracting the vibration response data and pressure response data from the initially collected track bed vibration data and contact pressure data to obtain a vibration difference value and a pressure difference value, and combining the vibration difference value and the pressure difference value with the lateral vibration signal parameters emitted by the controllable vibration source and the train speed to output a defect confidence level through a multimodal data fusion defect assessment model.

6. The floating slab roadbed sealing strip defect detection method based on multimodal data according to claim 1 is characterized in that: The multimodal data fusion defect assessment model is configured with: ； Among them, C is the defect confidence, which indicates the possibility of judging whether the sealing strip has defects. The smaller the value of C, the lower the possibility of the sealing strip having defects. d (t) is the vibration difference function, which represents the vibration difference at time t; P d (t) is the pressure difference function, which represents the pressure difference at time t; T is the total time of data acquisition, λ is the vibration attenuation coefficient, which represents the coefficient of vibration difference attenuation over time, β is the coefficient of pressure difference changing with time, S h (i) is the i-th lateral vibration signal value continuously emitted by the controllable vibration source, n is the number of lateral vibration signal sampling points, v is the train speed, v e is the standard vehicle speed. The greater the difference between the vehicle speed and the standard vehicle speed, the lower the defect confidence.

7. A floating slab roadbed sealing strip defect detection system based on multimodal data, applicable to the floating slab roadbed sealing strip defect detection method based on multimodal data according to any one of claims 1 to 6, characterized in that: include: The data acquisition module collects trackbed sealing strip image data through a linear array camera installed on the bottom of the train, obtains trackbed vibration data through an acceleration sensor, and obtains floating plate contact pressure data through a pressure sensor; A preliminary defect detection module calculates the parallelism parameter between the sealing strip and the trackbed floating plate based on the image data, and triggers defect depth detection when the parallelism parameter exceeds a preset threshold; The defect depth detection module activates the controllable vibration source at the rear of the train to emit lateral vibration signals, synchronously collects vibration response data and contact pressure data of the defect area, and inputs them into the multimodal data fusion defect assessment model to calculate the defect confidence level. When the confidence level exceeds a threshold, a defect is determined to exist. The defect type judgment module outputs a deformation defect or a foreign body defect based on the frequency domain characteristics of the vibration signal and the contact pressure variation curve in combination with the parallelism parameter through a defect feature judgment strategy. The module performs a fast Fourier transform on the vibration signal to obtain the frequency domain characteristics and identifies the frequency peak in the frequency domain characteristics. At the same time, the module analyzes the slope, amplitude, and variation period of the contact pressure variation curve and combines them with the parallelism parameter to construct a multidimensional feature vector. The module analyzes the multidimensional feature vector using a pre-trained classification model and outputs the result of the deformation defect or foreign body defect judgment. The defect characteristic judgment strategy includes setting multiple sets of characteristic parameter thresholds. When the frequency peak of the vibration signal is higher than the first threshold, the slope of the contact pressure change curve exceeds the second threshold, and the parallelism parameter is greater than the third threshold, it is judged as a deformation defect; when the frequency peak of the vibration signal is lower than the fourth threshold, the contact pressure change curve shows irregular fluctuations, and the parallelism parameter is less than the third threshold, it is judged as a foreign body defect.

Images