Roadbed defect online intelligent detection method based on knowledge distillation ultralight network

Abstract

The invention discloses an online intelligent roadbed defect detection method based on a knowledge distillation ultra-light network. The method comprises the following steps: S1, collecting and processing actual data of a vibration field in a roadbed; s2, building, training and deploying a roadbed defect detection neural network model; wherein the roadbed defect detection neural network is composed of a lightweight network for roadbed defect detection and a roadbed defect detection network based on a large model; and S3, carrying out roadbed defect online intelligent detection based on the roadbed defect detection neural network model. The invention further discloses a corresponding system, electronic equipment and a computer readable storage medium, the distributed sensors are arranged on the road, vehicle driving is used as an excitation source, and under the condition that the normal operation state of the road is not affected, rapid detection of roadbed defects can be achieved, and normal operation of traffic is guaranteed.

Description

Technical Field

[0001] The present invention relates to the technical field of subgrade defect state detection, and particularly relates to an online intelligent detection method and system for subgrade defects based on a knowledge distillation ultra-light network. Background Art

[0002] In recent years, with the continuous rapid development of China's economy, various infrastructure constructions have been gradually improved, among which transportation construction occupies an important position. The subgrade has been bearing the influence of dynamic traffic loads, climate environment changes, and geological conditions for a long time, and is prone to defects such as cavities, voids, or loose bodies. If these defects are not discovered and treated in time, they may cause subgrade instability and even lead to traffic accidents or economic losses. Therefore, subgrade detection work has also received extensive attention. Although the traditional geological drilling method has high reliability, it has high costs and will cause a certain degree of damage to the highway. Therefore, the gradual development and application of non-destructive testing technologies have become a major trend. Common non-destructive testing technologies for subgrade defects mainly include ground penetrating radar method and seismic reflection wave method. The ground penetrating radar method provides high-resolution information by emitting high-frequency electromagnetic waves, but it is easily affected by the water content of the formation and electromagnetic noise. The seismic reflection wave method, as a geophysical prospecting method, detects the underground formation structure by artificially exciting seismic waves on the ground surface by using the difference in wave impedance of the media above and below the formation interface. Although these methods have strong non-destructiveness and remarkable effects, data analysis and judgment rely on professionals, and at the same time, the exploration process may require temporary road closure, increasing the complexity of use.

[0003] With the development of seismic motion measurement technology, by using a moving vehicle as an active vibration source and a sensor as a receiving source, the echo response of the underground medium to the vibration wave can be obtained, and then a time profile of the vibration field can be constructed. By combining a neural network to perform neural network modeling on the time profile, intelligent detection of subgrade defects can be realized. This method does not require damage to the road surface and will not affect the normal operation of the road. It can continuously monitor under normal traffic operation conditions, improving the intelligent level of operation and maintenance. Therefore, this technology has become a research hotspot in the field of intelligent transportation. However, there are still the following problems in using neural networks for subgrade defect detection:

[0004] (1) Low accuracy in subgrade defect recognition and poor generalization ability

[0005] The subgrade defect detection neural network relies on the time profile of the vibration field to identify defects such as cavities, voids, and loose bodies. However, due to the lack of diversity in samples (time profiles of the vibration field) and the high annotation difficulty, the model has significant deficiencies in accuracy and generalization ability. At the same time, the acquisition process of the time profile of the vibration field is complex and costly, limiting the scale of the dataset and making it difficult for the model to handle complex geological conditions, especially with low recognition accuracy in special scenarios. In addition, there are few samples of some defect types (such as loose bodies), and the model has weak detection ability for these diseases, further reducing the recognition accuracy. Finally, the lack of diverse samples with different sampling frequencies, vibration amplitudes, and propagation paths makes it difficult for the model to adapt to subgrade defect detection under different working conditions.

[0006] (2) The subgrade defect detection network model is complex, and it is difficult to deploy at the edge, resulting in insufficient detection real-time performance

[0007] In the subgrade defect detection task, the classification of the time profile of the vibration field and the defect judgment usually rely on complex deep learning models. These models often have a large number of parameters and high computational requirements. However, due to limited hardware resources, the edge computing platform is difficult to meet the computational requirements of complex models, mainly manifested in the following aspects: First, the edge computing platform needs to quickly process the time profile data of the vibration field collected from the front end, but the inference time of high-complexity models is long, unable to meet the real-time requirements, resulting in system response delays. Second, complex models have high requirements for computing power, storage, and memory, while edge devices usually only have limited GPU computing power and memory capacity, making it difficult to load and run the complete model, thus affecting the actual use effect. Summary of the Invention

[0008] The purpose of the present invention is to provide an online intelligent detection method and system for subgrade defects based on a knowledge distillation ultra-light network. By deploying distributed sensors on the road and using vehicle driving as an excitation source, it is possible to achieve rapid detection of subgrade defects without affecting the normal operation state of the highway and ensure the normal operation of traffic. The method mainly includes two major parts: the acquisition and processing of subgrade internal vibration field data and the subgrade defect detection network. The acquisition and processing of subgrade internal vibration field data are used to obtain road vibration information and construct a dataset, and the subgrade defect detection network classifies defects according to the time profile of the vibration field and the instantaneous frequency map.

[0009] The first aspect of the present invention lies in providing an online intelligent detection method for subgrade defects based on a knowledge distillation ultra-light network, including:

[0010] S1, collecting and processing the actual data of the subgrade internal vibration field;

[0011] S2. Build, train and deploy a subgrade defect detection neural network model; wherein, the subgrade defect detection neural network consists of a lightweight network for subgrade defect detection and a subgrade defect detection network based on a large model;

[0012] S3. Conduct on-line intelligent detection of subgrade defects based on the subgrade defect detection neural network model.

[0013] Preferably, the S1 includes:

[0014] S11. Collect the actual data of the vibration field inside the subgrade, including:

[0015] (1) Arrange a row along the road corresponding to the subgrade to form a vibration sensor array composed of multiple distributed vibration sensor nodes;

[0016] (2) Use the vibration generated by a single vehicle driving at a constant speed as the excitation source, and obtain the vibration signal by the distributed vibration sensor nodes;

[0017] (3) Transmit the vibration signal to the edge computing platform based on the wireless local area network;

[0018] S12. Obtain the actual vibration field time profile and the actual instantaneous frequency map based on the actual data of the vibration field inside the subgrade, including integrating the data of the vibration sensor array in the edge computing platform to obtain the actual vibration field time profile; and converting the time-domain data of the vibration sensor array into the frequency-domain data of the vibration sensor array based on the Wigner-Ville distribution algorithm, and obtaining the actual instantaneous frequency map based on the frequency-domain data;

[0019] S13. Generate the simulated vibration field time profile and the simulated instantaneous frequency map under different geological conditions based on the subgrade finite element model.

[0020] Preferably, the step S13 includes:

[0021] (1) Establish a subgrade finite element model, including: establish a complete subgrade finite element model according to the "Technical Standard for Comprehensive Detection and Risk Assessment of Urban Underground Disease Bodies" according to different geological conditions, defect types and parameter configurations;

[0022] (2) Generate the simulated vibration field time profile and the simulated instantaneous frequency map covering various source depths, sampling frequencies and propagation paths on the subgrade finite element model by the finite difference method; wherein, the simulated vibration field time profile and the simulated instantaneous frequency map are used to provide the subgrade defect vibration field time profile data set and the instantaneous frequency map data set under different geological conditions, and the subgrade defect vibration field time profile data set and the instantaneous frequency map data set under different geological conditions are used to provide training samples for the subgrade defect detection neural network model.

[0023] Preferably, S2 includes:

[0024] S21, constructing a lightweight network for subgrade defect detection; the lightweight network consists of multiple feature extraction networks, a cross-attention module, a weighted fusion module, a linear layer, and an activation function layer. There are six feature extraction networks in total, with three for each of the vibration field time profile and the instantaneous frequency map. The three feature extraction networks are connected in series. The lightweight network for subgrade defect detection takes the simulated vibration field time profile and the instantaneous frequency map as inputs and processes them through two parallel branches, each branch consisting of the three feature extraction networks. The cross-attention module performs interaction and fusion on the features extracted by the two parallel branches to capture the correlations between different modality features. Then, the weighted fusion module performs a weighting operation on the output of the cross-attention module to further integrate the extracted features. The linear layer in the classification part performs a linear transformation on the fused features, maps them to an appropriate dimension, and finally, the activation function layer converts the output of the linear layer into a probability distribution to achieve the output of class prediction;

[0025] S22, constructing a subgrade defect detection network based on a large model; the subgrade defect detection network based on a large model is a multi-scale NiNformer network, including two-dimensional convolution 1, two-dimensional convolution 2, two-dimensional convolution 3, a NiN feature extraction module, a multi-scale fusion layer of the subgrade defect detection network, a weighted fusion module of the subgrade defect detection network, a linear layer of the subgrade defect detection network, a SoftMax activation function layer of the subgrade defect detection network, and an output layer of the SoftMax activation function layer. Taking the simulated vibration field time profile and the simulated instantaneous frequency map as inputs, the two-side branch structures are symmetric. The two-side branches first go through two-dimensional convolution 1, then are processed by the NiN feature extraction module, followed by two-dimensional convolution 2 and the NiN feature extraction module for another operation, then go through two-dimensional convolution 3 and the NiN feature extraction module to obtain features. After that, each branch undergoes multi-scale fusion through the multi-scale fusion module, and then the results of the two-side branches are weighted and fused through the weighted fusion module of the subgrade defect detection network, and finally, after passing through the linear layer of the subgrade defect detection network and the SoftMax activation function layer of the subgrade defect detection network, the class is output through the output layer of the SoftMax activation function layer;

[0026] S23, performing the training process of the subgrade defect detection network based on a large model and the lightweight network;

[0027] S24, deploying the trained subgrade defect detection neural network model to an edge computing platform, and the edge computing platform is an NVIDIA Jetson Nano platform.

[0028] Preferably, S21 includes:

[0029] (1) Establish a feature extraction network, which includes a two-dimensional convolutional layer, a ReLU activation function layer, a max pooling layer, a batch normalization layer, and an output layer; the feature extraction network extracts features through the two-dimensional convolutional layer and the ReLU activation function, and reduces the data dimension through the max pooling layer and the batch normalization layer to accelerate training; the normalized data is output through the output layer;

[0030] (2) Establish a cross-attention module, including multiple input modules, linear layer modules corresponding to the multiple input modules, a scaled dot-product attention module, and an attention output module;

[0031] (3) Construct a weighted fusion module, which is used to perform weighted fusion on the output of the cross-attention module to further integrate the features;

[0032] (4) Construct a SoftMax activation function layer and an output layer.

[0033] Preferably, the S22 includes:

[0034] (1) Establish a NiN feature extraction module, which is used to extract and process features of pictures, including a feedforward neural network, a transpose module, a gated unit, a normalization module, a linear layer, and a multi-layer hybrid perceptron;

[0035] (2) Establish a multi-scale fusion layer of the subgrade defect detection network, which is used to perform multi-scale fusion on the results after branch feature extraction, and comprehensively integrate different scale feature information to enhance the feature expression ability.

[0036] Preferably, the S23 includes:

[0037] (1) Take the simulated vibration field time profile and the simulated instantaneous frequency map as inputs, and input them into a lightweight network and a subgrade defect detection network based on a large model for training respectively;

[0038] (2) After the training is completed, perform the training process of orthogonal projection knowledge distillation through the orthogonal projection knowledge distillation technology, including:

[0039] Step 1, perform orthogonal normalization on the features output by the weighted fusion module of the subgrade defect detection network, including:

[0040] First, normalize the features Z t output by the weighted fusion module of the subgrade defect detection network into a normalized output feature with a mean of 0 and a variance of 1, and the formula is shown as formula (4) below:

[0041]

[0042] Among them, μ and σ respectively represent the t mean and standard deviation of Z; represents the standardized output feature;

[0043] Secondly, perform whitening on the standardized output feature to eliminate the correlation between features. The formula is as shown in Equation (5) below:

[0044]

[0045] Among them, W = ∑ -1 / 2 , and ∑ represents the feature covariance matrix;

[0046] Step 2: Perform orthogonal projection on the features output by the weighted fusion module of the lightweight network, including:

[0047] First, project the features Z s output by the weighted fusion module of the lightweight network onto the subspace of the subgrade defect detection network features based on the large model The formula is as shown in Equation (6) below:

[0048]

[0049] Among them, P represents the orthogonal basis matrix generated by the subgrade defect detection network based on the large model, which is used to maintain geometric alignment; represents minimizing the lightweight network features;

[0050] Secondly, define the loss function L of the distillation loss OPD , and minimize the difference between the lightweight network features and the subgrade defect detection network features based on the large model. The formula is as shown in Equation (7) below:

[0051]

[0052] Step 3: Optimization and training, including:

[0053] First, calculate the loss function L of the fusion loss. The loss function L of the fusion loss is the weighted sum of the loss function L of the distillation loss OPD and the network classification loss L class of the lightweight network, as shown in Equation (8):

[0054] L = αL class + βL OPD (8);

[0055] Among them, α and β are hyperparameters that respectively control the distillation loss L OPDand the classification loss L class weight;

[0056] Secondly, during the training process, the high-quality simulated vibration field time profiles generated by the finite element and the actual collected data are used to jointly train the subgrade defect detection network and the lightweight network based on the large model, and the weights of the lightweight network for subgrade defect detection are adjusted by optimizing the loss function L of the fusion loss, so that its performance is close to that of the subgrade defect detection network based on the large model;

[0057] (3) Fine-tuning the trained lightweight network based on the actual vibration field time profile and the actual instantaneous frequency map, including: using the actually collected vibration field time profile as the input of the network, during the training process, the loss function is cross-entropy, the optimizer is AdamW, and the batch is adjusted.

[0058] Preferably, the S24 includes:

[0059] (1) Compressing and optimizing the lightweight network to adapt to the hardware resource limitations of the NVIDIA Jetson Nano platform;

[0060] (2) Using TensorRT for model acceleration to improve the inference speed and reduce the latency, while ensuring the efficient operation of the network;

[0061] (3) Invoking the CUDA acceleration function of the NVIDIA Jetson Nano platform to improve the image processing and computing performance and achieve real-time subgrade defect detection.

[0062] Preferably, the S3 includes:

[0063] S31, receiving the real-time subgrade vibration data of the vibration sensor as a vibration signal;

[0064] S32, on the NVIDIA Jetson Nano platform, the subgrade defect detection neural network model effectively processes the vibration signal from the vibration sensor, and quickly outputs the subgrade defect identification result while ensuring accuracy, meeting the requirements of online intelligent and fast detection in practical applications.

[0065] The second aspect of the present invention lies in providing an online intelligent detection system for subgrade defects based on a knowledge distillation ultra-light network for implementing the method of the first aspect, including:

[0066] A data acquisition and processing module (101) for acquiring and processing the actual data of the vibration field inside the subgrade;

[0067] Subgrade defect detection neural network model establishment module (102) is used to build, train and deploy a subgrade defect detection neural network model; wherein, the subgrade defect detection neural network is composed of a lightweight network for subgrade defect detection and a subgrade defect detection network based on a large model;

[0068] Subgrade defect online intelligent detection module (103) is used to perform online intelligent detection of subgrade defects based on the subgrade defect detection neural network model.

[0069] A third aspect of the present invention provides an electronic device, including a processor and a memory, the memory stores multiple instructions, and the processor is used to read the instructions and execute the method as described in the first aspect.

[0070] A fourth aspect of the present invention provides a computer-readable storage medium, the computer-readable storage medium stores multiple instructions, and the multiple instructions can be read and executed by the processor to execute the method as described in the first aspect.

[0071] Advantages of the method and system of the present invention:

[0072] (1) Aiming at the poor generalization ability of the subgrade defect detection neural network model, the present invention proposes a method combining a finite element model and actual collected data. First, based on the measured data, a dataset of time profiles and instantaneous frequency maps of subgrade defect vibration fields under different geological conditions is generated through a finite element model, providing rich training samples for the neural network model and improving the network generalization ability; second, the model parameters of the neural network are fine-tuned using the actual collected data to further improve the model's adaptability to real scenarios.

[0073] (2) To solve the problems of low subgrade defect recognition accuracy, complex model and poor real-time performance. First, the time profile of the vibration field and the instantaneous frequency map are combined as the input of the neural network. The time profile of the vibration field can accurately present the spatial characteristics of seismic wave propagation, and the abnormal area is characterized by different vibration intensities or propagation modes. The instantaneous frequency map can assist in judging the defect type, thus improving the accuracy of the model; second, a subgrade defect detection network based on a large model is adopted to extract subgrade data features from different scales to improve the recognition accuracy of subgrade defects; then, the lightweight network for subgrade defect detection is trained through orthogonal projection knowledge distillation technology, so that the lightweight network for subgrade defect detection can maintain a high recognition accuracy while being lightweight. In this way, the lightweight network for subgrade defect detection can also approach the performance of the subgrade defect detection network based on a large model on resource-constrained devices, thus taking into account both accuracy and real-time performance. Brief Description of the Drawings

[0074] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the related art, the following will briefly introduce the drawings required for use in the description of the specific embodiments or the related art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0075] Figure 1 It is a flowchart of an online intelligent detection method for subgrade defects based on a knowledge distillation ultra-light network according to an embodiment of the present invention;

[0076] Figure 2 It is a schematic layout diagram of arranging a row of vibration sensors along the road corresponding to the subgrade to form multiple distributed vibration sensor nodes according to an embodiment of the present invention;

[0077] Figure 3 (a) and Figure 3 (b) are a time profile diagram of the vibration field and an instantaneous frequency diagram according to an embodiment of the present invention;

[0078] Figure 4 It is a schematic diagram of the subgrade defect detection network structure according to an embodiment of the present invention;

[0079] Figure 5 It is a schematic diagram of the lightweight network structure for subgrade defect detection according to an embodiment of the present invention;

[0080] Figure 6 It is a schematic diagram of the feature extraction network structure according to an embodiment of the present invention;

[0081] Figure 7 It is a schematic diagram of the cross-attention module structure according to an embodiment of the present invention;

[0082] Figure 8 It is a schematic diagram of the subgrade defect detection network based on a large model according to an embodiment of the present invention;

[0083] Figure 9 It is a schematic diagram of the NiN feature extraction module structure according to an embodiment of the present invention;

[0084] Figure 10 It is a schematic diagram of the structure of the multi-scale fusion layer of the subgrade defect detection network according to an embodiment of the present invention;

[0085] Figure 11 It is an architecture diagram of an online intelligent detection system for subgrade defects based on a knowledge distillation ultra-light network according to an embodiment of the present invention;

[0086] Figure 12 It is a structure diagram of an electronic device according to an embodiment of the present invention. Detailed implementation manners

[0087] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0088] In the description of the present invention, it should be noted that the orientation or positional relationship indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. is based on the orientation or positional relationship shown in the accompanying drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus cannot be construed as a limitation of the present invention. In addition, the terms "first", "second", and "third" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance.

[0089] In the description of the present invention, it should be noted that unless otherwise clearly specified and limited, the terms "installation", "connection", and "connection" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations.

[0090] Embodiment 1

[0091] As Figure 1 shown, the first aspect of the present invention is to provide an online intelligent detection method for subgrade defects based on a knowledge distillation ultra-light network, including:

[0092] S1, collecting and processing the actual data of the vibration field inside the subgrade;

[0093] As a preferred implementation manner, the S1 includes:

[0094] S11, collecting the actual data of the vibration field inside the subgrade;

[0095] In this embodiment, S11 includes:

[0096] (1) As Figure 2 shown, a vibration sensor array composed of a series of distributed vibration sensor nodes is arranged along the road corresponding to the subgrade; in this embodiment, the detection area of the road corresponding to each section of the subgrade is 100 m, the spacing is 0.5 m, and a total of 200 vibration sensors are arranged;

[0097] (2) Use the vibration generated by a single vehicle traveling at a constant speed as the excitation source, and obtain the vibration signal by the distributed vibration sensor nodes.

[0098] (3) Transmit the vibration signal to the edge computing platform based on the wireless local area network.

[0099] S12. Obtain the actual vibration field time profile and the actual instantaneous frequency map based on the actual data of the vibration field inside the subgrade, including integrating the data of the vibration sensor array in the edge computing platform to obtain the actual vibration field time profile; and converting the time-domain data of the vibration sensor array into the frequency-domain data of the vibration sensor array based on the Wigner-Ville distribution algorithm, and obtaining the actual instantaneous frequency map based on the frequency-domain data.

[0100] During the subgrade defect detection process, the vibration field time profile can accurately represent the spatial characteristics of the seismic wave propagation. The abnormal areas of the subgrade usually show differences in local vibration intensity or propagation mode. The instantaneous frequency map reveals the abnormal patterns of frequency changes, which helps to identify the abnormalities during the seismic wave propagation, thus providing key support for judging the types of subgrade defects.

[0101] The vibration field time profile and the instantaneous frequency map are respectively as Figure 3 (a) and Figure 3 (b) shown.

[0102] S13. Generate the simulated vibration field time profile and the simulated instantaneous frequency map under different geological conditions based on the subgrade finite element model.

[0103] In this embodiment, step S13 includes:

[0104] (1) Establish a subgrade finite element model, including: establish a complete subgrade finite element model according to the Technical Standard for Comprehensive Detection and Risk Assessment of Urban Underground Disease Bodies, according to different geological conditions, defect types (such as cavities, voids, loose bodies, etc.) and parameter configurations.

[0105] (2) Through the finite difference method, generate the simulated vibration field time profile and the simulated instantaneous frequency map covering various source depths, sampling frequencies and propagation paths on the subgrade finite element model; wherein, the simulated vibration field time profile and the simulated instantaneous frequency map are used to provide the subgrade defect vibration field time profile dataset and the instantaneous frequency map dataset under different geological conditions, and the subgrade defect vibration field time profile dataset and the instantaneous frequency map dataset under different geological conditions are used to provide training samples for the subgrade defect detection neural network model.

[0106] S2. Build, train, and deploy a subgrade defect detection neural network model. Among them, the subgrade defect detection neural network consists of a lightweight network for subgrade defect detection and a subgrade defect detection network based on a large model, and its structure is as shown in Figure 4 shown.

[0107] As a preferred implementation, S2 includes:

[0108] S21. Construct a lightweight network for subgrade defect detection;

[0109] The lightweight network for subgrade defect detection is a self-built network model, and its structure schematic diagram is as shown in Figure 5 shown. It consists of multiple feature extraction networks, a cross-attention module, a weighted fusion module, a linear layer, and an activation function layer. There are a total of six feature extraction networks, three for each of the vibration field time profile diagram and the instantaneous frequency diagram. The three feature extraction networks are connected in series. The lightweight network for subgrade defect detection takes the simulated vibration field time profile diagram and the instantaneous frequency diagram as inputs and processes them through two parallel branches respectively. Each branch is composed of the three feature extraction networks. The cross-attention module conducts interaction and fusion on the features extracted by the two parallel branches to capture the correlation between different modal features. Then, the weighted fusion module performs a weighted operation on the output of the cross-attention module to further integrate the extracted features. The linear layer in the classification part performs a linear transformation on the fused features, maps them to an appropriate dimension, and finally uses the activation function layer (in this embodiment, the SoftMax activation function layer) to convert the output of the linear layer into a probability distribution to achieve the output of class prediction.

[0110] In this embodiment, the lightweight network for subgrade defect detection inputs a one-dimensional image of 256×256. The output of the input image after passing through the feature extraction network three times is 4×4×4096. After the cross-attention module and the weighted fusion of the features of the vibration field time profile diagram and the instantaneous frequency diagram, it is output to the linear layer, transformed into 1×4 through the linear layer, and the probabilities of each category are output through the SoftMax layer, so as to output the category (normal, cavity, void, loose body).

[0111] In this embodiment, S21 includes:

[0112] (1) Establish a feature extraction network. The feature extraction network includes a two-dimensional convolutional layer, a ReLU activation function layer, a max pooling layer, a batch normalization layer, and an output layer. The feature extraction network extracts features through the two-dimensional convolutional layer and the ReLU activation function, and reduces the data dimension through the max pooling layer and the batch normalization layer to accelerate training. The normalized data is output through the output layer. The structure schematic diagram of the feature extraction network is as shown in Figure 6As shown, where the two-dimensional convolutional layer has a convolutional kernel of 3×3, a stride of 1, and a padding of 1, and the max pooling layer is max pooling of 4×4.

[0113] (2) Establish a cross-attention module; the cross-attention module is used to fuse multi-source information, and by dynamically focusing on the correlation between different feature channels, improve the richness and accuracy of feature representation. Its network structure diagram is as Figure 7 shown, including multiple input modules, linear layer modules corresponding to the multiple input modules, scaled dot-product attention modules, and attention output modules; among them, the input data (in this embodiment, there are input 1 and input 2) first passes through three different said linear layers (Linear) respectively. The three different linear layers correspond to query (Query, marked as "Q"), key (Key, marked as "K"), and value (Value, marked as "V") respectively. Each linear layer performs a linear transformation on the input to obtain different representations; cross-input 1 and the obtained Q1 and Q2 of input 2. Then, the transformed query (Q), key (K), and value (V) are input into the scaled dot-product attention module. In the scaled dot-product attention module, the dot product of the query and all keys is calculated, then divided by the scaling factor (usually the square root of the key vector dimension), then the attention weights are obtained through the Softmax function, and finally the attention weights are multiplied by the corresponding value vectors of the values and summed to obtain the attention output and output through the attention output module.

[0114] For inputs 1 and 2, the formula of the cross-attention module is shown as formula (1) below:

[0115]

[0116] where, F O1 、F O2 represent the features of the output 1 of the cross-attention module and the features of the output 2 of the cross-attention module, Q1 and Q2 represent the query vectors of input 1 and input 2, K1 and K2 represent the key vectors of input 1 and input 2, V1 and V2 represent the value vectors of input 1 and input 2, and d k represents the dimension of the key vector.

[0117] (3) Construct a weighted fusion module

[0118] The weighted fusion module is used to perform weighted fusion on the output of the cross-attention module and further integrate the features; the weighted fusion principle of the weighted fusion module is shown as formula (2):

[0119] F = ω1F O1 + ω2F O2 (2);

[0120] where, F O1 、FO2 respectively represent the features of the output 1 of the cross-attention module and the features of the output 2 of the cross-attention module; F represents the features output by the weighted fusion layer; ω1 and ω2 are the weights of the output 1 and output 2 of the cross-attention module respectively, and ω1 and ω2 are hyperparameters obtained through training.

[0121] (4) Construct a SoftMax activation function layer and an output layer.

[0122] S22, construct a subgrade defect detection network based on a large model; the subgrade defect detection network based on a large model is intended to adopt a multi-scale NiNformer network, and the specific network structure diagram is as Figure 8 shown. The subgrade defect detection network based on a large model includes two-dimensional convolution 1, two-dimensional convolution 2, two-dimensional convolution 3, a NiN feature extraction module, a multi-scale fusion layer of the subgrade defect detection network, a weighted fusion module of the subgrade defect detection network, a linear layer of the subgrade defect detection network, a SoftMax activation function layer of the subgrade defect detection network, and an output layer of the SoftMax activation function layer. Taking the simulated vibration field time profile diagram and the simulated instantaneous frequency diagram as inputs, the two-side branch structures are symmetric; the two-side branches first go through two-dimensional convolution 1, and then are processed by the NiN feature extraction module, then two-dimensional convolution 2 and the NiN feature extraction module operate again, then features are obtained through two-dimensional convolution 3 and the NiN feature extraction module, and then each branch undergoes multi-scale fusion through the multi-scale fusion module, and then the results of the two-side branches are weighted and fused through the weighted fusion module of the subgrade defect detection network (similar to the weighted fusion module in S21), and then pass through the linear layer of the subgrade defect detection network and the SoftMax activation function layer of the subgrade defect detection network, and finally the category is output through the output layer of the SoftMax activation function layer.

[0123] As a preferred embodiment, S22 includes:

[0124] (1) Establish a NiN feature extraction module, and the NiN feature extraction module is used for feature extraction and processing of pictures, and its network structure is as Figure 9As shown in the figure, it includes a feedforward neural network, a transpose module, a gating unit, a normalization module, a linear layer, and a multi-layer hybrid perceptron. After the input features are "normalized" for the first time by the normalization module, the gating unit can regulate the information of the input picture after preliminary processing, determine which information enters the subsequent "feedforward neural network", thereby affecting the feature extraction and processing direction of the entire network for the input picture. The multi-layer hybrid perceptron is a neural network structure containing multiple hidden layers. It can perform complex non-linear transformations on the input data, further extract and fuse information from the "linear layer" and the "previous layer", mine more advanced and abstract features in the data, and provide more valuable information for subsequent processing. These features have a crucial impact on the final output result.

[0125] (2) Establish a multi-scale fusion layer for the subgrade defect detection network. The multi-scale fusion layer of the subgrade defect detection network is used to perform multi-scale fusion on the results after branch feature extraction, and comprehensively integrate feature information of different scales to enhance the feature expression ability. The structure of the multi-scale fusion layer of the subgrade defect detection network is as Figure 10 shown in the figure. Among them, the parameters of the two-dimensional convolutional layer of the multi-scale fusion layer are: convolution kernel: 4×4, stride: 4, no padding. F3, F4, and F5 respectively represent the features of feature 1, feature 2, and feature 3. F6 represents the feature output by the multi-scale fusion module. The processing processes of feature 4, feature 5, and feature 6 are the same. The formula for the weighted fusion part is shown in Equation (3):

[0126] F6 = ω3F3 + ω4F4 + ω5F5 (3);

[0127] Among them, ω3, ω4, and ω5 are the weights of feature 1, feature 2, and feature 3 respectively. ω3 = 0.5, ω4 = 0.3, ω5 = 0.2.

[0128] The subgrade defect detection network based on the large model inputs a 256×256 two-dimensional picture. The time profile of the vibration field is divided into non-overlapping image blocks of 64×64 through two-dimensional convolution 1 (convolution kernel 4×4, stride 4, no padding), and the output is 64×64×16. The NiN feature extraction module only performs feature extraction without changing the size of the feature map. After passing through two-dimensional convolutional layer 2 (convolution kernel 4×4, stride 4, no padding), it gets 16×16×256. After passing through the NiN feature extraction module and two-dimensional convolutional layer 3 (convolution kernel 4×4, stride 4, no padding) again, the output is 4×4×4096. The outputs of the three two-dimensional convolutional layers are output as 4×4×4096 through the multi-dimensional convolutional layer. After the weighted fusion of the features of the time profile of the vibration field and the features of the instantaneous frequency map, it is output to the linear layer, transformed into 1×4 through the linear layer, and the probabilities of each category are output through the SoftMax layer, thereby outputting the category (normal, cavity, void, loose body).

[0129] S23. Conduct the training process of the subgrade defect detection network and the lightweight network based on the large model.

[0130] As a preferred implementation, the S23 includes:

[0131] (1) Take the simulated vibration field time profile and the simulated instantaneous frequency map as inputs, and input them into the lightweight network and the subgrade defect detection network based on the large model for training respectively.

[0132] In this embodiment, a deep learning environment is constructed based on the PyTorch framework, and 2 4080Ti GPUs and CUDA parallel computing are used to accelerate image processing. During the training process of the subgrade defect detection network and the lightweight network based on the large model, cross-entropy is used as the loss function, the optimizer is selected as AdamW, 256 batches of samples are used for each training, the learning rate is set to 0.001, and the training is iterated 500 rounds.

[0133] (2) After the training is completed, conduct the training process of orthogonal projection knowledge distillation through the orthogonal projection knowledge distillation technology, which is suitable for improving the performance of the lightweight network model for subgrade defect detection in scenarios with limited computing power. The orthogonal projection distillation technology aims to improve the learning efficiency and performance of the lightweight network for subgrade defect detection through orthogonal projection and orthogonal normalization, so that the features of the lightweight network gradually approach the features of the subgrade defect detection network based on the large model.

[0134] Step 1. Conduct orthogonal normalization on the features output by the weighted fusion module of the subgrade defect detection network, including:

[0135] First, standardize the features Z output by the weighted fusion module of the subgrade defect detection network t to adjust them into a standardized output feature with a mean of 0 and a variance of 1. The formula is as shown in formula (4) below:

[0136]

[0137] where μ and σ respectively represent the mean and standard deviation of Z t ; represents the standardized output feature;

[0138] Second, perform whitening on the standardized output feature to eliminate the correlation between features. The formula is as shown in formula (5) below:

[0139]

[0140] where W = ∑ -1 / 2 , and ∑ represents the feature covariance matrix.

[0141] Step 2, perform orthogonal projection on the features output by the weighted fusion module of the lightweight network, including:

[0142] First, project the features Z output by the weighted fusion module of the lightweight network s onto the subspace of the features of the roadbed defect detection network based on the large model The formula is as shown in formula (6) below:

[0143]

[0144] where P represents the orthogonal basis matrix generated by the roadbed defect detection network based on the large model, which is used to maintain geometric alignment; represents minimizing the lightweight network features;

[0145] Secondly, define the loss function L of the distillation loss OPD , and minimize the difference between the lightweight network features and the features of the roadbed defect detection network based on the large model . The formula is as shown in formula (7) below:

[0146]

[0147] Step 3, optimization and training, including:

[0148] First, calculate the loss function L of the fusion loss. The loss function L of the fusion loss is the weighted sum of the loss function L of the distillation loss OPD and the network classification loss L of the lightweight network class , as shown in formula (8):

[0149] L = αL class + βL OPD (8);

[0150] where α and β are hyperparameters that respectively control the weights of the distillation loss L OPD and the classification loss L class ;

[0151] Secondly, during the training process, jointly train the roadbed defect detection network based on the large model and the lightweight network with the high-quality simulated vibration field time profile diagrams generated by the finite element and the actual collected data, and adjust the weights of the lightweight network for roadbed defect detection by optimizing the loss function L of the fusion loss to make its performance close to that of the roadbed defect detection network based on the large model.

[0152] (3) Fine-tune the trained lightweight network based on the actual vibration field time profile diagrams and actual instantaneous frequency diagrams, so as to improve its generalization ability, enhance the accuracy, and better adapt to the applications in real scenarios.

[0153] In this embodiment, during the fine-tuning process of the lightweight network for subgrade defect detection, the actually collected vibration field time profile is used as the input of the network. During the training process, the loss function is cross-entropy, the optimizer is AdamW, but the batch size is adjusted to 64, the learning rate is kept at 0.001, and the iteration is 400 rounds. Through parameter adjustment, its adaptability to the real scene is enhanced. By adjusting the parameters of the lightweight network, it can better cope with noise, interference, and geological changes, thereby improving the accuracy of subgrade defect identification.

[0154] S24. Deploy the trained subgrade defect detection neural network model to the edge computing platform, including: In order to apply the deep learning model to the edge computing device, the present invention deploys the lightweight network for subgrade defect detection with fine-tuned parameters to the NVIDIA Jetson Nano platform.

[0155] As a preferred implementation manner, S24 includes:

[0156] (1) During the deployment process, first compress and optimize the lightweight network to adapt to the hardware resource limitations of the NVIDIA Jetson Nano platform;

[0157] (2) Use TensorRT for model acceleration to improve the inference speed and reduce the latency, while ensuring the efficient operation of the network;

[0158] (3) Invoke the CUDA acceleration function of the NVIDIA Jetson Nano platform to further improve the image processing and computing performance and achieve real-time subgrade defect detection.

[0159] S3. Perform online intelligent detection of subgrade defects based on the subgrade defect detection neural network model, including:

[0160] S31. Receive the real-time subgrade vibration data of the vibration sensor;

[0161] S32. On the NVIDIA Jetson Nano platform, the subgrade defect detection neural network model effectively processes the real-time subgrade vibration data from the vibration sensor and quickly outputs the subgrade defect identification result while ensuring accuracy, meeting the requirements of online intelligent and fast detection in practical applications.

[0162] Embodiment 2

[0163] As Figure 11 shown, this embodiment provides an online intelligent detection system for subgrade defects based on a knowledge distillation ultra-light network, which is used to implement the method of Embodiment 1 and includes:

[0164] The data acquisition and processing module 101 is used to acquire and process the actual data of the vibration field inside the roadbed;

[0165] The roadbed defect detection neural network model establishment module 102 is used to build, train and deploy the roadbed defect detection neural network model; wherein, the roadbed defect detection neural network is composed of a lightweight network for roadbed defect detection and a roadbed defect detection network based on a large model;

[0166] The roadbed defect online intelligent detection module 103 is used to perform online intelligent detection of roadbed defects based on the roadbed defect detection neural network model.

[0167] The present invention also provides a memory storing multiple instructions for implementing the method as in Embodiment 1.

[0168] As Figure 12 shown, the present invention also provides an electronic device, including a processor 301 and a memory 302 connected to the processor 301. The memory 302 stores multiple instructions that can be loaded and executed by the processor, enabling the processor to execute the method as in Embodiment 1.

[0169] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An online intelligent detection method for subgrade defects based on a knowledge distillation ultra-light network, characterized in that, Including: S1, collecting and processing the actual data of the vibration field inside the roadbed; S2, building, training and deploying a neural network model for roadbed defect detection; wherein, the neural network for roadbed defect detection is composed of a lightweight network for roadbed defect detection and a roadbed defect detection network based on a large model; S3, performing online intelligent detection of roadbed defects based on the neural network model for roadbed defect detection.

2. The on-line intelligent detection method for subgrade defects based on a knowledge distillation ultra-light network according to claim 1, characterized in that The S1 includes: S11, collecting the actual data of the vibration field inside the roadbed, including: (1) Arranging a row of vibration sensor arrays composed of multiple distributed vibration sensor nodes along the road corresponding to the roadbed; (2) Using the vibration generated by a single vehicle traveling at a constant speed as the excitation source, and obtaining vibration signals by the distributed vibration sensor nodes; (3) Transmitting the vibration signals to the edge computing platform based on the wireless local area network; S12, obtaining the actual vibration field time profile and the actual instantaneous frequency map based on the actual data of the vibration field inside the roadbed, including integrating the data of the vibration sensor array in the edge computing platform to obtain the actual vibration field time profile; and converting the time-domain data of the vibration sensor array into the frequency-domain data of the vibration sensor array based on the Wigner-Ville distribution algorithm, and obtaining the actual instantaneous frequency map based on the frequency-domain data; S13, generating the simulated vibration field time profile and the simulated instantaneous frequency map under different geological conditions based on the roadbed finite element model.

3. The on-line intelligent detection method for subgrade defects based on a knowledge distillation ultra-light network according to claim 2, characterized in that The step S13 includes: (1) Establishing a roadbed finite element model, including: establishing a complete roadbed finite element model according to the Technical Standard for Comprehensive Detection and Risk Assessment of Urban Underground Disease Bodies, according to different geological conditions, defect types and parameter configurations; (2) Generating the simulated vibration field time profile and the simulated instantaneous frequency map covering various source depths, sampling frequencies and propagation paths on the roadbed finite element model by the finite difference method; wherein, the simulated vibration field time profile and the simulated instantaneous frequency map are used to provide the roadbed defect vibration field time profile data set and the instantaneous frequency map data set under different geological conditions, and the roadbed defect vibration field time profile data set and the instantaneous frequency map data set under different geological conditions are used to provide training samples for the neural network model for roadbed defect detection.

4. The on-line intelligent detection method for subgrade defects based on a knowledge distillation ultra-light network according to claim 3, characterized in that, The S2 includes: S21. Construct a lightweight network for subgrade defect detection. The lightweight network consists of multiple feature extraction networks, a cross-attention module, a weighted fusion module, a linear layer, and an activation function layer. There are a total of six feature extraction networks, with three each for the time profile of the vibration field and the instantaneous frequency map, and the three feature extraction networks are connected in series. The lightweight network for subgrade defect detection takes the simulated time profile of the vibration field and the instantaneous frequency map as inputs and processes them through two parallel branches, each branch consisting of the three feature extraction networks. The cross-attention module conducts interaction and fusion on the features extracted by the two parallel branches to capture the correlations between different modal features. Then, the weighted fusion module performs a weighting operation on the output of the cross-attention module to further integrate the extracted features. The linear layer in the classification part performs a linear transformation on the fused features, mapping them to an appropriate dimension, and finally, the activation function layer converts the output of the linear layer into a probability distribution to achieve category prediction output. S22. Construct a subgrade defect detection network based on a large model. The subgrade defect detection network based on a large model is a multi-scale NiNformer network, including two-dimensional convolution 1, two-dimensional convolution 2, two-dimensional convolution 3, a NiN feature extraction module, a multi-scale fusion layer of the subgrade defect detection network, a weighted fusion module of the subgrade defect detection network, a linear layer of the subgrade defect detection network, a SoftMax activation function layer of the subgrade defect detection network, and an output layer of the SoftMax activation function layer. Taking the simulated time profile of the vibration field and the simulated instantaneous frequency map as inputs, the two-side branch structure is symmetric. The two-side branches first go through two-dimensional convolution 1, then are processed by the NiN feature extraction module, followed by two-dimensional convolution 2 and the NiN feature extraction module for another operation, then go through two-dimensional convolution 3 and the NiN feature extraction module to obtain features. After that, each branch undergoes multi-scale fusion through the multi-scale fusion module, and then the results of the two-side branches are weighted and fused through the weighted fusion module of the subgrade defect detection network, pass through the linear layer of the subgrade defect detection network and the SoftMax activation function layer of the subgrade defect detection network, and finally, the output layer of the SoftMax activation function layer outputs the category. S23. Conduct the training process of the subgrade defect detection network based on a large model and the lightweight network. S24. Deploy the trained subgrade defect detection neural network model to an edge computing platform, and the edge computing platform is the NVIDIA Jetson Nano platform.

5. The on-line intelligent detection method for subgrade defects based on knowledge distillation ultra-light network according to claim 4, characterized in that The S21 includes: (1) Establish a feature extraction network. The feature extraction network includes a two-dimensional convolutional layer, a ReLU activation function layer, a max pooling layer, a batch normalization layer, and an output layer. The feature extraction network extracts features through the two-dimensional convolutional layer and the ReLU activation function, and reduces the data dimension through the max pooling layer and the batch normalization layer to accelerate training. The normalized data is output through the output layer. (2) Establish a cross-attention module, including multiple input modules, linear layer modules corresponding to the multiple input modules, scaled dot-product attention modules, and attention output modules; (3) Construct a weighted fusion module, which is used to perform weighted fusion on the output of the cross-attention module to further integrate the features; (4) Construct a SoftMax activation function layer and an output layer.

6. The on-line intelligent detection method for subgrade defects based on knowledge distillation ultra-light network according to claim 5, characterized in that The S22 includes: (1) Establish a NiN feature extraction module, which is used to extract and process features of pictures, including a feed-forward neural network, a transpose module, a gated unit, a normalization module, a linear layer, and a multi-layer hybrid perceptron; (2) Establish a multi-scale fusion layer for the subgrade defect detection network, which is used to perform multi-scale fusion on the results after branch feature extraction, and comprehensively integrate different scale feature information to enhance the feature expression ability.

7. An online intelligent detection method for subgrade defects based on a knowledge distillation ultra-light network according to claim 6, characterized in that, The S23 includes: (1) Take the simulated vibration field time profile and the simulated instantaneous frequency map as inputs, and input them into a lightweight network and a subgrade defect detection network based on a large model for training respectively; (2) After training, conduct an orthogonal projection knowledge distillation training process through orthogonal projection knowledge distillation technology, including: Step 1, perform orthogonal normalization on the features output by the weighted fusion module of the subgrade defect detection network, including: First, standardize the feature Z output by the weighted fusion module of the subgrade defect detection network t to adjust it into a standardized output feature with a mean of 0 and a variance of 1. The formula is shown in the following formula (4): where μ and σ represent the mean and standard deviation of Z t respectively; represents the standardized output feature; Secondly, perform whitening processing on the standardized output features to eliminate the correlation between features. The formula is as shown in the following formula (5): where W = ∑ -1 / 2 , and ∑ represents the characteristic covariance matrix; Step 2, perform orthogonal projection on the features output by the weighted fusion module of the lightweight network, including: First, project the feature Z output by the weighted fusion module of the lightweight network s onto the subspace of the features of the subgrade defect detection network based on the large model The formula is as shown in the following formula (6): Among them, P represents the orthogonal basis matrix generated by the subgrade defect detection network based on the large model, which is used to maintain geometric alignment; represents minimizing the lightweight network features; Secondly, define the loss function \(L\) of the distillation loss OPD , and minimize the features of the lightweight network and the features of the subgrade defect detection network based on the large model . The difference is shown in Equation (7) below: Step 3, optimization and training, including: First, calculate the loss function L of the fusion loss. The loss function L of the fusion loss is the loss function L of the distillation loss OPD and the network classification loss L of the lightweight network class which is the weighted sum as shown in Equation (8): L = αL class + βL OPD (8); where α and β are hyperparameters that control the weights of the distillation loss L OPD and the classification loss L class respectively; Secondly, during the training process, use high-quality simulated vibration field time profiles generated by finite element and actual collected data to jointly train the subgrade defect detection network based on a large model and the lightweight network, and adjust the weights of the lightweight network for subgrade defect detection by optimizing the loss function L of the fusion loss to make its performance close to that of the subgrade defect detection network based on a large model; (3) Fine-tune the trained lightweight network based on the actual vibration field time profile and actual instantaneous frequency map, including: using the actually collected vibration field time profile as the input of the network, during the training process, the loss function is cross-entropy, the optimizer is AdamW, and the batch is adjusted.

8. An online intelligent detection method for subgrade defects based on a knowledge distillation ultra-light network according to claim 7, characterized in that, The S24 includes: (1) Compress and optimize the lightweight network to adapt to the hardware resource limitations of the NVIDIA Jetson Nano platform; (2) Use TensorRT to accelerate the model, improve the inference speed and reduce the latency, while ensuring the efficient operation of the network; (3) Invoke the CUDA acceleration function of the NVIDIA Jetson Nano platform to improve the image processing and computing performance, and achieve real-time subgrade defect detection.

9. The on-line intelligent detection method for subgrade defects based on a knowledge distillation ultra-light network according to claim 8, characterized in that The S3 includes: S31, receive the real-time subgrade vibration data of the vibration sensor as a vibration signal; S32. On the NVIDIA Jetson Nano platform, the subgrade defect detection neural network model can effectively process the vibration signals from the vibration sensors, and quickly output the subgrade defect recognition results while ensuring accuracy, meeting the requirements of online intelligent and rapid detection in practical applications.

10. An online intelligent detection system for subgrade defects based on a knowledge distillation ultra-light network, which is used to implement the method according to any one of claims 1-9, and is characterized in that, It includes: A data acquisition and processing module (101) for acquiring and processing the actual data of the vibration field inside the subgrade; A subgrade defect detection neural network model establishment module (102) for building, training, and deploying a subgrade defect detection neural network model; wherein, the subgrade defect detection neural network is composed of a lightweight network for subgrade defect detection and a subgrade defect detection network based on a large model; A subgrade defect online intelligent detection module (103) for performing online intelligent detection of subgrade defects based on the subgrade defect detection neural network model.

Images