An online intelligent detection method for roadbed defects based on knowledge distillation ultra-light network

Abstract

The present invention discloses an online intelligent detection method for roadbed defects based on a knowledge distillation ultra-light network, comprising: S1, collecting and processing actual data on the roadbed's internal vibration field; S2, building, training, and deploying a roadbed defect detection neural network model; wherein the roadbed defect detection neural network comprises a lightweight roadbed defect detection network and a roadbed defect detection network based on a large model; and S3, performing online intelligent detection of roadbed defects based on the roadbed defect detection neural network model. Also disclosed are corresponding systems, electronic devices, and computer-readable storage media. By utilizing distributed sensors deployed on the road and using vehicle travel as an excitation source, this method can achieve rapid detection of roadbed defects without disrupting normal highway operation, thereby ensuring the smooth operation of traffic.

Description

Technical Field

[0001] The present invention relates to the technical field of roadbed defect status detection, and in particular to an online intelligent detection method and system for roadbed defects based on a knowledge distillation ultra-light network. Background Art

[0002] In recent years, my country's economy has continued to grow at a rapid pace, and infrastructure of all kinds has been gradually improved, with transportation infrastructure playing a crucial role. Roadbeds are constantly subjected to dynamic traffic loads, climate and environmental changes, and geological conditions, making them prone to cavities, voids, and loose body defects. If these defects are not promptly detected and addressed, they can lead to roadbed instability, even resulting in traffic accidents and economic losses. Consequently, roadbed inspection has garnered widespread attention. Traditional geological drilling methods, while highly reliable, are costly and can cause some damage to highways. Consequently, the gradual development and application of non-destructive testing technologies has become a major trend. Common non-destructive testing techniques for roadbed defects include ground-penetrating radar (GPR) and seismic reflection wave methods. GPR emits high-frequency electromagnetic waves to provide high-resolution information, but is susceptible to influences such as stratum water content and electromagnetic noise. Seismic reflection wave methods, as a geophysical exploration method, exploit the difference in wave impedance between the media above and below the stratum interface to artificially induce seismic waves at the surface to detect subsurface structures. While these methods are highly non-destructive and effective, data analysis and interpretation require specialized personnel, and the exploration process may require temporary road closures, increasing their complexity.

[0003] With the development of seismic measurement technology, using moving vehicles as active vibration sources and sensors as receiving sources, the echo response of the underground medium to the vibration wave can be obtained, and then a time profile of the vibration field can be constructed. The time profile is then modeled by a neural network, realizing intelligent detection of roadbed defects. This method does not require damage to the road surface and does not affect the normal operation of the road. It can continuously monitor under normal traffic conditions and improve the level of intelligent operation and maintenance. Therefore, this technology has become a research hotspot in the field of intelligent transportation. However, the use of neural networks for roadbed defect detection still has the following problems:

[0004] (1) Low roadbed defect recognition accuracy and poor generalization ability

[0005] Neural networks for roadbed defect detection rely on vibration field time profiles to identify defects such as cavities, voids, and loose bodies. However, due to the lack of sample diversity (vibration field time profiles) and the difficulty of labeling, the model suffers from significant deficiencies in accuracy and generalization. Furthermore, the complex and costly process of acquiring vibration field time profiles limits the size of the dataset, making it difficult for the model to cope with complex geological conditions. In particular, recognition accuracy is low in special scenarios. Furthermore, the limited sample size of some defect types (such as loose bodies) weakens the model's ability to detect these defects, further reducing recognition accuracy. Finally, the lack of diverse samples with varying sampling frequencies, vibration amplitudes, and propagation paths makes the model difficult to adapt to roadbed defect detection under diverse operating conditions.

[0006] (2) The roadbed defect detection network model is complex and edge deployment is difficult, resulting in insufficient real-time detection.

[0007] In roadbed defect detection tasks, the classification and defect assessment of vibration field time profiles typically rely on complex deep learning models. These models often have numerous parameters and high computational requirements. However, edge computing platforms, due to their limited hardware resources, struggle to meet the computational demands of these complex models. This is primarily manifested in the following aspects: First, edge computing platforms must rapidly process the vibration field time profile data collected from the front-end. However, the inference time for highly complex models is long, failing to meet real-time requirements and resulting in system response delays. Second, complex models require high computing power, storage, and memory, while edge devices typically have limited GPU computing power and memory capacity, making it difficult to load and run complete models, thus impacting actual performance. Summary of the Invention

[0008] The present invention aims to provide an online intelligent detection method and system for roadbed defects based on a knowledge distillation ultra-light network. By deploying distributed sensors along the road and using vehicle motion as an excitation source, this method enables rapid detection of roadbed defects without disrupting normal highway operation, thereby ensuring the smooth operation of traffic. The method primarily comprises two components: a data acquisition and processing section for the roadbed's internal vibration field, and a roadbed defect detection network. The latter is used to acquire road vibration information and construct a data set, while the latter classifies defects based on the vibration field's time profile and instantaneous frequency graph.

[0009] A first aspect of the present invention is to provide an online intelligent detection method for roadbed defects based on a knowledge distillation ultra-light network, comprising:

[0010] S1, collects and processes the actual data of the vibration field inside the roadbed;

[0011] S2, building, training, and deploying a roadbed defect detection neural network model; wherein the roadbed defect detection neural network is composed of a lightweight roadbed defect detection network and a roadbed defect detection network based on a large model;

[0012] S3, performing online intelligent detection of roadbed defects based on the roadbed defect detection neural network model.

[0013] Preferably, the S1 includes:

[0014] S11, collects actual data of the vibration field inside the roadbed, including:

[0015] (1) Arrange a row of vibration sensor arrays consisting of multiple distributed vibration sensor nodes along the road corresponding to the roadbed;

[0016] (2) The vibration generated by a single vehicle traveling at a constant speed is used as the excitation source, and the distributed vibration sensor nodes obtain the vibration signal;

[0017] (3) transmitting the vibration signal to an edge computing platform based on a wireless local area network;

[0018] S12, obtaining an actual vibration field time profile and an actual instantaneous frequency graph based on actual vibration field data within the roadbed, including integrating data from the vibration sensor array in an edge computing platform to obtain the actual vibration field time profile; and converting the time domain data of the vibration sensor array into frequency domain data of the vibration sensor array based on a Wigner-Will distribution algorithm, and obtaining the actual instantaneous frequency graph based on the frequency domain data;

[0019] S13, based on the roadbed finite element model, generates simulated vibration field time profiles and simulated instantaneous frequency diagrams under different geological conditions.

[0020] Preferably, the step S13 includes:

[0021] (1) Establishing a finite element model of the roadbed, including: establishing a complete finite element model of the roadbed according to different geological conditions, defect types and parameter configurations in accordance with the Technical Standards for Comprehensive Detection and Risk Assessment of Urban Underground Defective Bodies;

[0022] (2) By using the finite difference method, a simulated vibration field time profile and a simulated instantaneous frequency diagram covering a variety of earthquake source depths, sampling frequencies and propagation paths are generated on the roadbed finite element model; wherein, the simulated vibration field time profile and the simulated instantaneous frequency diagram are used to provide a roadbed defect vibration field time profile dataset and an instantaneous frequency diagram dataset under different geological conditions, and the roadbed defect vibration field time profile dataset and the instantaneous frequency diagram dataset under different geological conditions are used to provide training samples for a roadbed defect detection neural network model.

[0023] Preferably, the S2 includes:

[0024] S21, constructing a lightweight network for roadbed defect detection; the lightweight network is composed of multiple feature extraction networks, cross-attention modules, weighted fusion modules, linear layers and activation function layers. There are six feature extraction networks in total, three of which are set for the vibration field time profile and the instantaneous frequency map respectively, and the three feature extraction networks are connected in series; the lightweight network for roadbed defect detection takes the simulated vibration field time profile and the instantaneous frequency map as input, and processes them respectively through two parallel branches, each branch is composed of the three feature extraction networks; the cross-attention module interacts and fuses the features extracted by the two parallel branches to capture the correlation between different modal features; then the weighted fusion module performs a weighted operation on the output of the cross-attention module to further integrate the extracted features; the linear layer of the classification part performs a linear transformation on the fused features and maps them to a suitable dimension, and finally the activation function layer is used to convert the output of the linear layer into a probability distribution to achieve category prediction output;

[0025] S22, constructing a roadbed defect detection network based on a large model; the roadbed defect detection network based on the large model is a multi-scale NiNformer network, including two-dimensional convolution 1, two-dimensional convolution 2, two-dimensional convolution 3, NiN feature extraction module, roadbed defect detection network multi-scale fusion layer, roadbed defect detection network weighted fusion module, roadbed defect detection network linear layer, roadbed defect detection network SoftMax activation function layer and SoftMax activation function layer output layer; the simulated vibration field time profile and the simulated instantaneous frequency map are used as input, and the branch structures on both sides are symmetrical; the branches on both sides are first subjected to two-dimensional convolution 1, then processed by the NiN feature extraction module, and then the two-dimensional convolution 2 and the NiN feature extraction module are operated again, and then the features are obtained by the two-dimensional convolution 3 and the NiN feature extraction module, and then each branch is subjected to multi-scale fusion by the multi-scale fusion module, and then the branch results on both sides are weighted fused by the roadbed defect detection network weighted fusion module, and finally output categories by the roadbed defect detection network linear layer, the roadbed defect detection network SoftMax activation function layer, and finally the SoftMax activation function layer output layer;

[0026] S23, training the roadbed defect detection network and lightweight network based on the large model;

[0027] S24, deploying the trained roadbed defect detection neural network model to an edge computing platform, wherein the edge computing platform is an NVIDIA Jetson Nano platform.

[0028] Preferably, the S21 includes:

[0029] (1) Establishing a feature extraction network, the feature extraction network includes a two-dimensional convolution layer, a ReLU activation function layer, a maximum pooling layer, a batch normalization layer and an output layer; the feature extraction network extracts features through the two-dimensional convolution layer and the ReLU activation function, and reduces the data dimension through the maximum pooling layer and the batch normalization layer to accelerate training; and outputs the normalized data through the output layer;

[0030] (2) Establish a cross-attention module, including multiple input modules, linear layer modules corresponding to the multiple input modules, scaled dot product attention modules, and attention output modules;

[0031] (3) Constructing a weighted fusion module, which is used to perform weighted fusion on the output of the cross-attention module to further integrate the features;

[0032] (4) Construct the SoftMax activation function layer and output layer.

[0033] Preferably, the S22 includes:

[0034] (1) Establishing a NiN feature extraction module, which is used to extract and process features of images and includes a feedforward neural network, a transposition module, a gating unit, a normalization module, a linear layer, and a multi-layer hybrid perceptron;

[0035] (2) Establishing a multi-scale fusion layer of the roadbed defect detection network, the multi-scale fusion layer of the roadbed defect detection network is used to perform multi-scale fusion on the results after branch feature extraction, and integrate feature information of different scales to enhance the feature expression capability.

[0036] Preferably, the S23 includes:

[0037] (1) The simulated vibration field time profile and the simulated instantaneous frequency map are input as input to a lightweight network and a roadbed defect detection network based on a large model for training respectively;

[0038] (2) After the training is completed, the orthogonal projection knowledge distillation training process is carried out using the orthogonal projection knowledge distillation technology, including:

[0039] Step 1, performing orthogonal normalization on the features output by the weighted fusion module of the roadbed defect detection network, including:

[0040] First, the features output by the weighted fusion module of the roadbed defect detection network are Standardize and adjust the output features to a mean of 0 and a variance of 1. The formula is shown in the following formula (4):

[0041] (4);

[0042] in, and Respectively The mean and standard deviation of represents the standardized output features;

[0043] Secondly, the standardized output features Perform whitening to eliminate the correlation between features. The formula is shown in the following formula (5):

[0044] (5);

[0045] in, , represents the feature covariance matrix;

[0046] Step 2, performing orthogonal projection on the features output by the weighted fusion module of the lightweight network, comprising:

[0047] First, the features output by the weighted fusion module of the lightweight network are

[0048] Projection to the subspace of network features for roadbed defect detection based on large models ∈V k ( n ), the formula is shown in the following formula (6):

[0049] (6);

[0050] in, represents the orthogonal basis matrix generated by the large model-based roadbed defect detection network to maintain geometric alignment; Represents the minimized lightweight network features;

[0051] Second, define the loss function of distillation loss , minimize lightweight network features Network characteristics of roadbed defect detection based on large model The difference is shown in the following formula (7):

[0052] (7);

[0053] Step 3, optimization and training, includes:

[0054] First, calculate the loss function of the fusion loss , the loss function of the fusion loss The loss function is the distillation loss and network classification loss of lightweight network The weighted sum of is as shown in formula (8):

[0055] (8);

[0056] in, and are hyperparameters that control the distillation loss and classification loss The weight of

[0057] Secondly, during the training process, the high-quality simulated vibration field time profile generated by finite element and the actual collected data are used to jointly train the roadbed defect detection network based on the large model and the lightweight network, and the loss function of the fusion loss is optimized. Adjust the weights of the lightweight network for roadbed defect detection to make its performance close to that of the roadbed defect detection network based on the large model;

[0058] (3) Fine-tune the trained lightweight network based on the actual vibration field time profile and the actual instantaneous frequency map, including: using the actual collected vibration field time profile as the input of the network. During the training process, the loss function adopts cross entropy, the optimizer is AdamW, and the batch is adjusted.

[0059] Preferably, the S24 includes:

[0060] (1) compressing and optimizing the lightweight network model to adapt to the hardware resource limitations of the NVIDIA Jetson Nano platform;

[0061] (2) Use TensorRT to accelerate the model, improve inference speed and reduce latency, while ensuring efficient operation of the network;

[0062] (3) Invoke the CUDA acceleration function of the NVIDIA Jetson Nano platform to improve image processing and computing performance and achieve real-time roadbed defect detection.

[0063] Preferably, the S3 includes:

[0064] S31, receiving real-time roadbed vibration data from a vibration sensor as a vibration signal;

[0065] S32, on the NVIDIA Jetson Nano platform, the roadbed defect detection neural network model effectively processes the vibration signal from the vibration sensor and quickly outputs the roadbed defect identification result while ensuring accuracy, meeting the requirements of online intelligent rapid detection in practical applications.

[0066] A second aspect of the present invention is to provide an online intelligent detection system for roadbed defects based on a knowledge distillation ultra-light network, which is used to implement the method of the first aspect, comprising:

[0067] A data acquisition and processing module (101) is used to acquire and process actual data of the vibration field inside the roadbed;

[0068] A roadbed defect detection neural network model building module (102) is used to build, train and deploy a roadbed defect detection neural network model; wherein the roadbed defect detection neural network is composed of a lightweight roadbed defect detection network and a roadbed defect detection network based on a large model;

[0069] The roadbed defect online intelligent detection module (103) is used to perform online intelligent detection of roadbed defects based on the roadbed defect detection neural network model.

[0070] A third aspect of the present invention provides an electronic device, comprising a processor and a memory, wherein the memory stores a plurality of instructions, and the processor is configured to read the instructions and execute the method described in the first aspect.

[0071] A fourth aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a plurality of instructions, and the plurality of instructions can be read by a processor to execute the method described in the first aspect.

[0072] Beneficial effects of the method and system of the present invention:

[0073] (1) To address the poor generalization capabilities of neural network models for roadbed defect detection, this paper proposes a method based on a finite element model combined with actual collected data. First, based on the measured data, a finite element model is used to generate a dataset of time profiles and instantaneous frequency diagrams of roadbed defect vibration fields under different geological conditions, providing a rich set of training samples for the neural network model and improving the network's generalization capabilities. Second, the actual collected data is used to fine-tune the model parameters of the neural network, further improving the model's adaptability to real-world scenarios.

[0074] (2) The present invention aims to solve the problems of low roadbed defect recognition accuracy, complex models, and poor real-time performance. First, the vibration field time profile and instantaneous frequency map are combined as neural network inputs. The vibration field time profile can accurately present the spatial characteristics of seismic wave propagation. The abnormal area is manifested as different vibration intensity or propagation mode. The instantaneous frequency map can assist in determining the defect type, which can improve the accuracy of the model. Secondly, a roadbed defect detection network based on a large model is used to extract roadbed data features from different scales to improve the recognition accuracy of roadbed defects. Then, the lightweight network for roadbed defect detection is trained through orthogonal projection knowledge distillation technology, so that the lightweight network for roadbed defect detection maintains high recognition accuracy while being lightweight. In this way, the lightweight network for roadbed defect detection can also approach the performance of the roadbed defect detection network based on a large model on resource-constrained equipment, thereby taking into account both accuracy and real-time performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in related technologies, the following briefly introduces the drawings required for use in the specific embodiments or related technical descriptions. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0076] Figure 1 A flow chart of an online intelligent detection method for roadbed defects based on a knowledge distillation ultra-light network according to an embodiment of the present invention;

[0077] Figure 2 A schematic diagram of a layout structure of a plurality of distributed vibration sensor nodes formed by arranging a row of vibration sensors along a road corresponding to a roadbed according to an embodiment of the present invention;

[0078] Figure 3 (a) and (b) are time cross-sectional diagrams and instantaneous frequency diagrams of the vibration field provided according to an embodiment of the present invention;

[0079] Figure 4 A schematic diagram of a roadbed defect detection network structure provided according to an embodiment of the present invention;

[0080] Figure 5 A schematic diagram of a lightweight network structure for roadbed defect detection according to an embodiment of the present invention;

[0081] Figure 6 A schematic diagram of a feature extraction network structure provided according to an embodiment of the present invention;

[0082] Figure 7 A schematic diagram of the structure of a cross attention module according to an embodiment of the present invention;

[0083] Figure 8 A schematic diagram of a network structure for roadbed defect detection based on a large model according to an embodiment of the present invention;

[0084] Figure 9 A schematic diagram of the structure of a NiN feature extraction module according to an embodiment of the present invention;

[0085] Figure 10 A schematic diagram of the structure of a multi-scale fusion layer of a roadbed defect detection network according to an embodiment of the present invention;

[0086] Figure 11 This is an architecture diagram of an online intelligent detection system for roadbed defects based on a knowledge distillation ultra-light network according to an embodiment of the present invention;

[0087] Figure 12 A structural diagram of an electronic device provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0088] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0089] In the description of the present invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely to facilitate and simplify the description of the present invention. They are not intended to indicate or imply that the devices or components referred to must have, be constructed, or operate in a specific orientation, and therefore should not be construed as limitations on the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0090] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed, detachable, or integral connections; mechanical or electrical connections; direct or indirect connections through an intermediate medium; and internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on the specific circumstances.

[0091] Example 1

[0092] like Figure 1As shown, the first aspect of the present invention is to provide an online intelligent detection method for roadbed defects based on a knowledge distillation ultra-light network, comprising:

[0093] S1, collects and processes the actual data of the vibration field inside the roadbed;

[0094] As a preferred embodiment, the S1 includes:

[0095] S11, collecting actual data of the vibration field inside the roadbed;

[0096] In this embodiment, S11 includes:

[0097] (1) If Figure 2 As shown, a vibration sensor array consisting of multiple distributed vibration sensor nodes is arranged along the road corresponding to the roadbed. In this embodiment, the road detection area corresponding to each section of the roadbed is 100m, with a spacing of 0.5 meters, and a total of 200 vibration sensors are arranged.

[0098] (2) The vibration generated by a single vehicle traveling at a constant speed is used as the excitation source, and the distributed vibration sensor nodes obtain the vibration signal;

[0099] (3) Transmitting the vibration signal to the edge computing platform based on the wireless local area network.

[0100] S12, obtaining an actual vibration field time profile and an actual instantaneous frequency graph based on actual vibration field data within the roadbed, including integrating data from the vibration sensor array in an edge computing platform to obtain the actual vibration field time profile; and converting the time domain data of the vibration sensor array into frequency domain data of the vibration sensor array based on a Wigner-Will distribution algorithm, and obtaining the actual instantaneous frequency graph based on the frequency domain data;

[0101] During roadbed defect detection, vibration field time profiles accurately represent the spatial characteristics of seismic wave propagation. Abnormal areas in the roadbed typically manifest as differences in localized vibration intensity or propagation patterns. Instantaneous frequency plots reveal unusual patterns of frequency variation, helping to identify anomalies in seismic wave propagation and providing critical support for determining the type of roadbed defect.

[0102] The time section diagram and instantaneous frequency diagram of the vibration field are shown as follows: Figure 3 As shown in (a) and (b).

[0103] S13, based on the roadbed finite element model, generates simulated vibration field time profiles and simulated instantaneous frequency diagrams under different geological conditions;

[0104] In this embodiment, step S13 includes:

[0105] (1) Establishing a finite element model of the roadbed, including: establishing a complete finite element model of the roadbed according to the "Technical Standards for Comprehensive Detection and Risk Assessment of Urban Underground Defective Bodies", different geological conditions, defect types (such as voids, voids, loose bodies, etc.) and parameter configurations;

[0106] (2) By using the finite difference method, a simulated vibration field time profile and a simulated instantaneous frequency diagram covering a variety of earthquake source depths, sampling frequencies and propagation paths are generated on the roadbed finite element model; wherein, the simulated vibration field time profile and the simulated instantaneous frequency diagram are used to provide a roadbed defect vibration field time profile dataset and an instantaneous frequency diagram dataset under different geological conditions, and the roadbed defect vibration field time profile dataset and the instantaneous frequency diagram dataset under different geological conditions are used to provide training samples for a roadbed defect detection neural network model.

[0107] S2, build, train and deploy a roadbed defect detection neural network model; wherein the roadbed defect detection neural network consists of a lightweight network for roadbed defect detection and a roadbed defect detection network based on a large model, and its structure is as follows Figure 4 shown.

[0108] As a preferred embodiment, the S2 includes:

[0109] S21, building a lightweight network for roadbed defect detection;

[0110] The lightweight network for roadbed defect detection is a self-built network model, and its structural diagram is as follows: Figure 5 As shown. It consists of multiple feature extraction networks, cross-attention modules, weighted fusion modules, linear layers, and activation function layers. There are six feature extraction networks in total, three for each of the vibration field time profile and instantaneous frequency graph, and the three feature extraction networks are connected in series. The lightweight network for roadbed defect detection takes the simulated vibration field time profile and the instantaneous frequency graph as input and processes them through two parallel branches, each of which is composed of the three feature extraction networks. The cross-attention module interacts and fuses the features extracted by the two parallel branches to capture the correlation between different modal features. The weighted fusion module then performs a weighted operation on the output of the cross-attention module to further integrate the extracted features. The linear layer of the classification part performs a linear transformation on the fused features and maps them to the appropriate dimension. Finally, the activation function layer (the SoftMax activation function layer is used in this embodiment) is used to convert the output of the linear layer into a probability distribution to achieve category prediction output.

[0111] In this embodiment, the lightweight network for roadbed defect detection inputs a 256×256 one-dimensional image. The input image is output from the feature extraction network three times to a 4×4×4096 image. After the cross-attention module and the weighted fusion of the features of the vibration field time profile and the instantaneous frequency map, the image is output to the linear layer, transformed to 1×4 by the linear layer, and output through the SoftMax layer to output the probability of each category, thereby outputting the category (normal, void, void, loose body).

[0112] In this embodiment, the S21 includes:

[0113] (1) Establish a feature extraction network, which includes a two-dimensional convolution layer, a ReLU activation function layer, a maximum pooling layer, a batch normalization layer, and an output layer; the feature extraction network extracts features through the two-dimensional convolution layer and the ReLU activation function, and reduces the data dimension through the maximum pooling layer and the batch normalization layer to accelerate training; the normalized data is output through the output layer. The feature extraction network structure diagram is shown in FIG. Figure 6 As shown in the figure, the two-dimensional convolution layer has a convolution kernel of 3×3, a stride of 1, and a padding of 1, and the maximum pooling layer has a maximum pooling of 4×4.

[0114] (2) Establish a cross attention module; the cross attention module is used to fuse multi-source information and improve the richness and accuracy of feature expression by dynamically focusing on the correlation between different feature channels. Its network structure is shown in the figure below: Figure 7 As shown, it includes multiple input modules, linear layer modules corresponding to the multiple input modules, scaled dot product attention modules, and attention output modules; wherein the input data (this embodiment has input 1 and input 2) first passes through three different linear layers (Linear), and the three different linear layers correspond to the query (Query, marked as "Q"), the key (Key, marked as "K"), and the value (Value, marked as "V"), each linear layer performs a linear transformation on the input to obtain a different representation; the cross input 1 and the input 2 are obtained and ,Then, the transformed query (Q), key (K) and value (V) are input into the scaled dot product attention module, in which the dot product of the query and all keys is calculated, and then divided by the scaling factor (usually the square root of the key vector dimension), and then the attention weight is obtained by the Softmax function. Finally, the attention weight is multiplied with the value vector corresponding to the value and summed to obtain the attention output and output through the attention output module.

[0115] For input 1 and input 2, the formula of the cross attention module is as follows (1):

[0116] , (1);

[0117] in, 、 Represents the features of the cross-attention module output 1 and the features of the cross-attention module output 2, 、 represents the query vector of input 1 and input 2, 、 represents the key vector of input 1 and input 2, 、 A vector representing the values ​​of input 1 and input 2, Indicates the dimension of the key vector.

[0118] (3) Constructing a weighted fusion module

[0119] The weighted fusion module is used to perform weighted fusion on the output of the cross-attention module to further integrate the features. The weighted fusion principle of the weighted fusion module is shown in formula (2):

[0120] (2);

[0121] in, 、 Represent the features of cross-attention module output 1 and cross-attention module output 2 respectively; Represents the features output by the weighted fusion layer; 、 are the weights of output 1 and output 2 of the cross attention module, 、 are the hyperparameters obtained through training.

[0122] (4) Construct the SoftMax activation function layer and output layer.

[0123] S22, build a roadbed defect detection network based on a large model; the roadbed defect detection network based on a large model is proposed to use a multi-scale NiNformer network. The specific network structure is shown in the figure below. Figure 8As shown. The large-scale model-based roadbed defect detection network includes two-dimensional convolution 1, two-dimensional convolution 2, two-dimensional convolution 3, a NiN feature extraction module, a roadbed defect detection network multi-scale fusion layer, a roadbed defect detection network weighted fusion module, a roadbed defect detection network linear layer, a roadbed defect detection network SoftMax activation function layer, and a SoftMax activation function layer output layer. The simulated vibration field time profile and the simulated instantaneous frequency map are used as input, and the two branches are symmetrically structured. The two branches first undergo two-dimensional convolution 1, then are processed by the NiN feature extraction module, followed by two-dimensional convolution 2 and the NiN feature extraction module, and then undergo two-dimensional convolution 3 and the NiN feature extraction module to obtain features. Each branch then undergoes multi-scale fusion in the multi-scale fusion module. The results of the two branches are then weightedly fused in the roadbed defect detection network weighted fusion module (similar to the weighted fusion module in S21). The results are then passed through the roadbed defect detection network linear layer, the roadbed defect detection network SoftMax activation function layer, and finally the SoftMax activation function layer output layer to output the category.

[0124] As a preferred embodiment, the S22 includes:

[0125] (1) Establish a NiN feature extraction module, which is used to extract and process features of images. Its network structure is as follows: Figure 9 As shown, it includes a feedforward neural network, a transposition module, a gating unit, a normalization module, a linear layer, and a multi-layer hybrid perceptron. After the input features are first "normalized" by the normalization module, the gating unit can regulate the information of the input image after preliminary processing, and decide which information enters the subsequent "feedforward neural network", thereby affecting the feature extraction and processing direction of the input image by the entire network. The multi-layer hybrid perceptron is a neural network structure containing multiple hidden layers. It can perform complex nonlinear transformations on the input data, further extract and fuse information from the "linear layer" and "previous layer", and mine higher-level and more abstract features in the data, providing more valuable information for subsequent processing. These features have a key impact on the final output results.

[0126] (2) Establish a multi-scale fusion layer for roadbed defect detection network. The multi-scale fusion layer is used to perform multi-scale fusion on the results after branch feature extraction, and to integrate feature information of different scales to enhance the feature expression capability. The structure of the multi-scale fusion layer is as follows: Figure 10 As shown in the figure, the parameters of the two-dimensional convolution layer of the multi-scale fusion layer are convolution kernel: 4×4, stride 4, and no padding. 、 and Represent the characteristics of feature 1, feature 2 and feature 3 respectively, It represents the features output by the multi-scale fusion module. The processing of features 4, 5, and 6 is the same. The weighted fusion formula is shown in formula (3):

[0127] (3);

[0128] in, 、 、 are the weights of feature 1, feature 2, and feature 3 respectively. =0.5, =0.3, =0.2.

[0129] The large-scale model-based roadbed defect detection network inputs a 256×256 two-dimensional image. The vibration field time profile is divided into non-overlapping 64×64 image blocks through two-dimensional convolution layer 1 (convolution kernel 4×4, stride 4, no padding), resulting in an output of 64×64×16. The NiN feature extraction module only performs feature extraction without changing the size of the feature map. After passing through two-dimensional convolution layer 2 (convolution kernel 4×4, stride 4, no padding), the output is 16×16×256. After passing through the NiN feature extraction module and two-dimensional convolution layer 3 (convolution kernel 4×4, stride 4, no padding), the output is 4×4×4096. The outputs of the three two-dimensional convolution layers are output through a multidimensional convolution layer to a 4×4×4096 output. After the features of the vibration field time profile and the instantaneous frequency map are weighted and fused, the output is sent to the linear layer, transformed to 1×4, and passed through the SoftMax layer to output the probability of each category, thereby outputting the category (normal, void, void, loose body).

[0130] S23, training the roadbed defect detection network and lightweight network based on the large model;

[0131] As a preferred embodiment, the S23 includes:

[0132] (1) The simulated vibration field time profile and the simulated instantaneous frequency map are input as input to a lightweight network and a roadbed defect detection network based on a large model for training respectively;

[0133] In this example, a deep learning environment was built based on the PyTorch framework, using two 4080Ti GPUs and CUDA parallel computing to accelerate image processing. Cross-entropy was used as the loss function for the large-scale model-based roadbed defect detection network and the lightweight network, and the AdamW optimizer was selected. Each training session used 256 batches of samples, a learning rate of 0.001, and 500 training iterations.

[0134] (2) After the training is completed, the orthogonal projection knowledge distillation technology is used to perform the training process. This process is suitable for improving the performance of the lightweight network model for roadbed defect detection in scenarios with limited computing power. The orthogonal projection distillation technology aims to improve the learning efficiency and performance of the lightweight network for roadbed defect detection through orthogonal projection and orthogonal normalization, so that the characteristics of the lightweight network gradually approach the characteristics of the roadbed defect detection network based on the large model.

[0135] Step 1, performing orthogonal normalization on the features output by the weighted fusion module of the roadbed defect detection network, including:

[0136] First, the features output by the weighted fusion module of the roadbed defect detection network are Standardize and adjust the output features to a mean of 0 and a variance of 1. The formula is shown in the following formula (4):

[0137] (4);

[0138] in, and Respectively The mean and standard deviation of represents the standardized output features;

[0139] Secondly, the standardized output features Perform whitening to eliminate the correlation between features. The formula is shown in the following formula (5):

[0140] (5);

[0141] in, , represents the feature covariance matrix.

[0142] Step 2, performing orthogonal projection on the features output by the weighted fusion module of the lightweight network, comprising:

[0143] First, the features output by the weighted fusion module of the lightweight network are Projection to the subspace of network features for roadbed defect detection based on large models ∈V k ( n ), the formula is shown in the following formula (6):

[0144] (6);

[0145] in, represents the orthogonal basis matrix generated by the large model-based roadbed defect detection network to maintain geometric alignment; Represents the minimized lightweight network features;

[0146] Second, define the loss function of distillation loss , minimize lightweight network features Network characteristics of roadbed defect detection based on large model The difference is shown in the following formula (7):

[0147] (7);

[0148] Step 3, optimization and training, includes:

[0149] First, calculate the loss function of the fusion loss , the loss function of the fusion loss The loss function is the distillation loss and network classification loss of lightweight network The weighted sum of is as shown in formula (8):

[0150] (8);

[0151] in, and are hyperparameters that control the distillation loss and classification loss The weight of

[0152] Secondly, during the training process, the high-quality simulated vibration field time profile generated by finite element and the actual collected data are used to jointly train the roadbed defect detection network based on the large model and the lightweight network, and the loss function of the fusion loss is optimized. The weights of the lightweight network for roadbed defect detection are adjusted to make its performance close to that of the roadbed defect detection network based on the large model.

[0153] (3) Fine-tune the trained lightweight network based on the actual vibration field time profile and the actual instantaneous frequency map to improve its generalization ability, enhance its accuracy, and better adapt to the application of real scenarios.

[0154] In this example, during the fine-tuning of a lightweight network for roadbed defect detection, actual collected vibration field time profiles were used as network input. During training, cross-entropy was used as the loss function, AdamW was used as the optimizer, the batch size was adjusted to 64, the learning rate was maintained at 0.001, and 400 iterations were performed. Parameter adjustment enhanced the network's adaptability to real-world scenarios. Adjusting the lightweight network parameters improved its ability to better cope with noise, interference, and geological changes, thereby improving the accuracy of roadbed defect identification.

[0155] S24, deploying the trained roadbed defect detection neural network model to an edge computing platform, including: in order to apply the deep learning model to edge computing devices, the present invention deploys a lightweight network for roadbed defect detection that has undergone parameter fine-tuning to the NVIDIA Jetson Nano platform.

[0156] As a preferred embodiment, the S24 includes:

[0157] (1) During the deployment process, the lightweight network is first compressed and optimized to adapt to the hardware resource limitations of the NVIDIA Jetson Nano platform;

[0158] (2) Use TensorRT to accelerate the model, improve inference speed and reduce latency, while ensuring efficient operation of the network;

[0159] (3) The CUDA acceleration function of the NVIDIA Jetson Nano platform is used to further improve image processing and computing performance and realize real-time roadbed defect detection.

[0160] S3, performing online intelligent detection of roadbed defects based on the roadbed defect detection neural network model, including:

[0161] S31, receiving real-time roadbed vibration data from a vibration sensor;

[0162] S32, on the NVIDIA Jetson Nano platform, the roadbed defect detection neural network model effectively processes the real-time roadbed vibration data from the vibration sensor, and quickly outputs roadbed defect identification results while ensuring accuracy, meeting the requirements of online intelligent rapid detection in practical applications.

[0163] Example 2

[0164] like Figure 11 As shown, this embodiment provides an online intelligent detection system for roadbed defects based on a knowledge distillation ultra-light network, which is used to implement the method of embodiment 1, including:

[0165] The data acquisition and processing module 101 is used to acquire and process the actual data of the vibration field inside the roadbed;

[0166] A subgrade defect detection neural network model building module 102 is used to build, train, and deploy a subgrade defect detection neural network model; wherein the subgrade defect detection neural network is composed of a lightweight subgrade defect detection network and a subgrade defect detection network based on a large model;

[0167] The roadbed defect online intelligent detection module 103 is used to perform online intelligent detection of roadbed defects based on the roadbed defect detection neural network model.

[0168] The present invention also provides a memory storing a plurality of instructions, wherein the instructions are used to implement the method as in the first embodiment.

[0169] like Figure 12 As shown, the present invention also provides an electronic device, including a processor 301 and a memory 302 connected to the processor 301, the memory 302 stores multiple instructions, and the instructions can be loaded and executed by the processor to enable the processor to execute the method as in embodiment 1.

[0170] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An online intelligent detection method for roadbed defects based on knowledge distillation ultra-light network, characterized by: include: S1, collects and processes the actual data of the vibration field inside the roadbed; S2, building, training, and deploying a roadbed defect detection neural network model; wherein the roadbed defect detection neural network is composed of a lightweight roadbed defect detection network and a roadbed defect detection network based on a large model; S3, performing online intelligent detection of roadbed defects based on the roadbed defect detection neural network model; Said S1 comprises: S11, collects actual data of the vibration field inside the roadbed, including: (1) Arrange a row of vibration sensor arrays consisting of multiple distributed vibration sensor nodes along the road corresponding to the roadbed; (2) The vibration generated by a single vehicle traveling at a constant speed is used as the excitation source, and the distributed vibration sensor nodes obtain the vibration signal; (3) transmitting the vibration signal to an edge computing platform based on a wireless local area network; S12, obtaining an actual vibration field time profile and an actual instantaneous frequency graph based on actual vibration field data within the roadbed, including integrating data from the vibration sensor array in an edge computing platform to obtain the actual vibration field time profile; and converting the time domain data of the vibration sensor array into frequency domain data of the vibration sensor array based on a Wigner-Will distribution algorithm, and obtaining the actual instantaneous frequency graph based on the frequency domain data; S13, based on the roadbed finite element model, generates simulated vibration field time profiles and simulated instantaneous frequency diagrams under different geological conditions; The step S13 includes: (1) Establishing a finite element model of the roadbed, including: establishing a complete finite element model of the roadbed according to different geological conditions, defect types and parameter configurations; (2) By using the finite difference method, a simulated vibration field time profile and a simulated instantaneous frequency diagram covering a variety of earthquake source depths, sampling frequencies and propagation paths are generated on the roadbed finite element model; wherein, the simulated vibration field time profile and the simulated instantaneous frequency diagram are used to provide a roadbed defect vibration field time profile dataset and an instantaneous frequency diagram dataset under different geological conditions, and the roadbed defect vibration field time profile dataset and the instantaneous frequency diagram dataset under different geological conditions are used to provide training samples for a roadbed defect detection neural network model.

2. The method for online intelligent detection of roadbed defects based on knowledge distillation ultra-light network according to claim 1 is characterized in that: The S2 includes: S21, constructing a lightweight network for roadbed defect detection; the lightweight network is composed of multiple feature extraction networks, cross-attention modules, weighted fusion modules, linear layers and activation function layers. There are six feature extraction networks in total, three of which are set for the vibration field time profile and the instantaneous frequency map respectively, and the three feature extraction networks are connected in series; the lightweight network for roadbed defect detection takes the simulated vibration field time profile and the instantaneous frequency map as input, and processes them respectively through two parallel branches, each branch is composed of the three feature extraction networks; the cross-attention module interacts and fuses the features extracted by the two parallel branches to capture the correlation between different modal features; then the weighted fusion module performs a weighted operation on the output of the cross-attention module to further integrate the extracted features; the linear layer of the classification part performs a linear transformation on the fused features and maps them to a suitable dimension, and finally the activation function layer is used to convert the output of the linear layer into a probability distribution to achieve category prediction output; S22, constructing a roadbed defect detection network based on a large model; the roadbed defect detection network based on the large model is a multi-scale NiNformer network, including a first two-dimensional convolution (1), a second two-dimensional convolution (2), a third two-dimensional convolution (3), a NiN feature extraction module, a roadbed defect detection network multi-scale fusion layer, a roadbed defect detection network weighted fusion module, a roadbed defect detection network linear layer, a roadbed defect detection network SoftMax activation function layer and a SoftMax activation function layer output layer; taking the simulated vibration field time profile and the simulated instantaneous frequency map as input, the two side branches The structure is symmetrical; the branches on both sides are first processed by the first two-dimensional convolution (1), and then processed by the NiN feature extraction module, and then the second two-dimensional convolution (2) and the NiN feature extraction module are operated again, and then the features are obtained by the third two-dimensional convolution (3) and the NiN feature extraction module, and then each branch is multi-scale fused by the multi-scale fusion layer, and then the results of the branches on both sides are weighted fused by the roadbed defect detection network weighted fusion module, and finally the categories are output by the SoftMax activation function layer output layer; S23, training the roadbed defect detection network and lightweight network based on the large model; S24, deploying the trained roadbed defect detection neural network model to an edge computing platform, wherein the edge computing platform is an NVIDIA Jetson Nano platform.

3. The method for online intelligent detection of roadbed defects based on knowledge distillation ultra-light network according to claim 2 is characterized in that: The S21 includes: (1) Establishing a feature extraction network, the feature extraction network includes a two-dimensional convolution layer, a ReLU activation function layer, a maximum pooling layer, a batch normalization layer and an output layer; the feature extraction network extracts features through the two-dimensional convolution layer and the ReLU activation function, and reduces the data dimension through the maximum pooling layer and the batch normalization layer to accelerate training; and outputs the normalized data through the output layer; (2) Establish a cross-attention module, including multiple input modules, linear layer modules corresponding to the multiple input modules, scaled dot product attention modules, and attention output modules; (3) Constructing a weighted fusion module, which is used to perform weighted fusion on the output of the cross-attention module to further integrate the features; (4) Construct the SoftMax activation function layer and output layer.

4. The method for online intelligent detection of roadbed defects based on knowledge distillation ultra-light network according to claim 3 is characterized in that: The S22 includes: (1) Establishing a NiN feature extraction module, which is used to extract and process features of images and includes a feedforward neural network, a transposition module, a gating unit, a normalization module, a linear layer, and a multi-layer hybrid perceptron; (2) Establishing a multi-scale fusion layer of the roadbed defect detection network, the multi-scale fusion layer of the roadbed defect detection network is used to perform multi-scale fusion on the results after branch feature extraction, and integrate feature information of different scales to enhance the feature expression capability.

5. The method for online intelligent detection of roadbed defects based on knowledge distillation ultra-light network according to claim 4 is characterized in that: The S23 includes: (1) The simulated vibration field time profile and the simulated instantaneous frequency map are input as input to a lightweight network and a roadbed defect detection network based on a large model for training respectively; (2) After the training is completed, the orthogonal projection knowledge distillation training process is carried out using the orthogonal projection knowledge distillation technology, including: Step 1, performing orthogonal normalization on the features output by the weighted fusion module of the roadbed defect detection network, including: First, the features output by the weighted fusion module of the roadbed defect detection network are Standardize and adjust the output features to a mean of 0 and a variance of 1. The formula is shown in the following formula (4): （4）； in, and Respectively The mean and standard deviation of represents the standardized output features; Secondly, the standardized output features Perform whitening to eliminate the correlation between features. The formula is shown in the following formula (5): （5）； in, , represents the feature covariance matrix; Step 2, performing orthogonal projection on the features output by the weighted fusion module of the lightweight network, comprising: First, the features output by the weighted fusion module of the lightweight network are Projection to the subspace of network features for roadbed defect detection based on large models ∈V k ( n ), the formula is shown in the following formula (6): （6）； in, represents the orthogonal basis matrix generated by the large model-based roadbed defect detection network to maintain geometric alignment; Represents the minimized lightweight network features; Second, define the loss function of distillation loss , minimize lightweight network features Network characteristics of roadbed defect detection based on large model The difference is shown in the following formula (7): （7）； Step 3, optimization and training, includes: First, calculate the loss function of the fusion loss , the loss function of the fusion loss The loss function is the distillation loss and network classification loss of lightweight network The weighted sum of is as shown in formula (8): （8）； in, and are hyperparameters that control the distillation loss and classification loss The weight of Secondly, during the training process, the high-quality simulated vibration field time profile generated by finite element and the actual collected data are used to jointly train the roadbed defect detection network based on the large model and the lightweight network, and the loss function of the fusion loss is optimized. Adjust the weights of the lightweight network for roadbed defect detection to make its performance close to that of the roadbed defect detection network based on the large model; (3) Fine-tune the trained lightweight network based on the actual vibration field time profile and the actual instantaneous frequency map, including: using the actual collected vibration field time profile as the input of the network. During the training process, the loss function adopts cross entropy, the optimizer is AdamW, and the batch is adjusted.

6. The method for online intelligent detection of roadbed defects based on knowledge distillation ultra-light network according to claim 5 is characterized in that: The S24 includes: (1) compressing and optimizing the lightweight network model to adapt to the hardware resource limitations of the NVIDIA Jetson Nano platform; (2) Use TensorRT to accelerate the model, improve inference speed and reduce latency, while ensuring efficient operation of the network; (3) Invoke the CUDA acceleration function of the NVIDIA Jetson Nano platform to improve image processing and computing performance and achieve real-time roadbed defect detection.

7. The method for online intelligent detection of roadbed defects based on knowledge distillation ultra-light network according to claim 6 is characterized in that: The S3 includes: S31, receiving real-time roadbed vibration data from a vibration sensor as a vibration signal; S32, on the NVIDIA Jetson Nano platform, the roadbed defect detection neural network model effectively processes the vibration signal from the vibration sensor and quickly outputs the roadbed defect identification result while ensuring accuracy, meeting the requirements of online intelligent rapid detection in practical applications.

8. An online intelligent detection system for roadbed defects based on a knowledge distillation ultra-light network, used to implement the method described in any one of claims 1 to 7, characterized in that: include: A data acquisition and processing module (101) is used to acquire and process actual data of the vibration field inside the roadbed; A roadbed defect detection neural network model building module (102) is used to build, train and deploy a roadbed defect detection neural network model; wherein the roadbed defect detection neural network is composed of a lightweight roadbed defect detection network and a roadbed defect detection network based on a large model; The roadbed defect online intelligent detection module (103) is used to perform online intelligent detection of roadbed defects based on the roadbed defect detection neural network model.

Images