A pipeline magnetic flux leakage detection method based on double teacher distillation and adaptive pruning

Abstract

The application discloses a pipeline magnetic flux leakage detection method based on double-teacher distillation and adaptive pruning, and belongs to the technical field of industrial nondestructive testing and deep learning. The method comprises the following steps: constructing and preprocessing an MFL detection dataset; constructing a first teacher network for extracting global frequency domain features, a second teacher network for extracting fine-grained spatial features, and a lightweight student network; training the student network through a double-teacher distillation loss function, and adjusting the weights of the double-teacher network through a knowledge gate module; pruning redundant channels based on an adaptive pruning strategy of a BN layer scaling factor; fine-tuning the pruned model to obtain a final FFDT-APNet model; and realizing end-to-end defect detection. The application also provides a system for realizing the method, comprising a data acquisition module, a preprocessing module, a lightweight model module and a result output module. The application solves the problems of incomplete feature extraction and large pruning precision loss of the existing lightweight model.

Description

Technical Field

[0001] This invention belongs to the field of interdisciplinary technology of industrial nondestructive testing and deep learning. Specifically, it relates to a lightweight deep learning method and system suitable for magnetic flux leakage (MFL) detection of pipelines, especially suitable for real-time defect identification and localization of ultra-high-definition MFL detection data, and can be deployed in embedded inspection equipment. Background Technology

[0002] As the core transport carrier for strategic energy sources such as oil and natural gas, the operational safety of pipelines is directly related to energy supply and public safety. Magnetic flux leakage (MFL) testing, with its non-contact, high sensitivity, and strong anti-interference capabilities, has become the mainstream technology for internal and external pipeline inspection.

[0003] With the widespread application and industrialization of ultra-high-definition MFL inspection equipment, the amount of magnetic signal data generated by a single pipeline inspection is growing exponentially, placing higher demands on the real-time processing capabilities and high-precision recognition capabilities of inspection algorithms. However, while deep model structures can improve expressive performance, they also significantly increase the difficulty and time cost of deployment in industrial scenarios.

[0004] To achieve lightweight detection models, existing technologies mainly employ two approaches: knowledge distillation-based and static pruning-based compression strategies. However, both have significant limitations in practical applications. On the one hand, traditional knowledge distillation methods typically rely on a single teacher model to convey feature information, making it difficult to simultaneously integrate the global structural features of the MFL signal in the frequency domain and the fine-grained features in the spatial domain, thus limiting the lightweight model's ability to extract multi-dimensional features. On the other hand, existing static pruning methods often trim convolutional channels based on fixed thresholds, easily removing feature channels crucial for identifying magnetic flux leakage signals, leading to a significant decrease in model accuracy, typically exceeding 5%.

[0005] Therefore, how to optimize both the number of model parameters and the inference speed while ensuring the accuracy of MFL defect detection, and build a lightweight detection model that combines "high precision and high efficiency", is a key technical problem that urgently needs to be solved in the field of industrial nondestructive testing. Summary of the Invention

[0006] This invention addresses the problems of incomplete feature extraction, significant pruning accuracy loss, and difficulty in embedded deployment in existing lightweight MFL detection models. It provides a pipeline magnetic flux leakage detection method based on frequency domain fine-grained dual-teacher distillation and adaptive pruning. Through complementary knowledge transfer from dual teachers and dynamic pruning strategies, the method reduces the number of model parameters to below 3.5M and compresses the inference time to within 2.5ms while maintaining a detection accuracy of over 92%, achieving real-time and lightweight deployment of ultra-high-definition MFL detection.

[0007] This invention adopts a full-process technical path of "dual-teacher distillation-adaptive pruning optimization-fine-tuning deployment". The core includes a pipeline magnetic flux leakage detection method based on frequency domain fine-grained dual-teacher distillation and adaptive pruning. The specific technical solution is as follows: A pipeline magnetic flux leakage detection method based on frequency domain fine-grained dual-teacher distillation and adaptive pruning, comprising the following steps: Step 1: MFL detection dataset construction and preprocessing; Step 2: Construction of a fine-grained frequency domain dual-teacher network and a lightweight student network; Step 3: Online pre-training for students using dual-teacher knowledge distillation; Step 4: Adaptive pruning optimization based on BN layer scaling factor; Step 5: Fine-tuning and performance optimization of the pruning model; Step 6: End-to-end inspection of MFL defects.

[0008] As a preferred embodiment, step 1 of the present invention specifically includes the following steps: Step 1.1: Collect raw data from ultra-high-definition pipeline MFL inspection, covering radial, axial, and circumferential defect data, and construct a dataset containing 5000 samples; Step 1.2: Normalize the original MFL signal using the following formula: Where x is the original signal value, The sample mean. The standard deviation is the sample standard deviation. Step 1.3: Divide the dataset into training, validation and test sets in a ratio of 7:1:2, and perform data augmentation processing on the training set samples by randomly flipping and adding Gaussian noise to improve the model's generalization ability.

[0009] As another preferred embodiment, step 2 of the present invention specifically includes the following steps: Step 2.1: First Teacher Network (T1) - Frequency Domain Feature Extraction Network: An improved ResNet-101 is used as the basic framework. Short-time Fourier Transform (STFT) and Discrete Cosine Transform (DCT) are added to the backbone layer of the network. Combined with frequency domain consistency loss, the noise resistance is enhanced, and the unique frequency components and energy spectrum patterns of the defects are captured. Step 2.2: Second Teacher Network (T2) – Fine-grained Spatial Feature Extraction Network: Spatial-Channel Collaborative Attention (SCA) modules are inserted after stage 2, stage 3, and stage 4 of the ResNet-101 network. This module simultaneously learns the spatial attention map. and channel attention vector Attention weights are used to enhance the extraction of fine-grained spatial features such as defect edges and micro-cracks; Step 2.3: Student Network (S) - Lightweight Base Network: Construct a lightweight backbone network based on ResNet18, named FFDT-APNet basic framework; Step 2.4: Gating Adjustment Module Gated networks typically consist of global average pooling layers and fully connected layers, used to dynamically adjust distillation weights from two teachers to achieve adaptive knowledge fusion. During training, the gated network trains alongside the student network and determines which teacher's knowledge is more useful to the student network based on the features of the current sample. For example, for samples with significant frequency domain features, the gated network might assign more weights to the student network. Assign higher weights; while for samples with rich details, it may be better to give higher weights. Assign higher weights. , These represent the weights of frequency domain knowledge and fine-grained knowledge, respectively, satisfying... + , It is the regularization loss of the gated network. This is the regularization coefficient. A gated network is a lightweight network that takes the intermediate features of the student network as input and outputs the fused weights of the two teachers, which can be represented as:

[0010] in, A measure of the frequency domain components of student characteristics; Step 2.5: Knowledge Recovery Mechanism To avoid knowledge loss caused by one-time pruning, minimize the difference in student network feature distribution before and after pruning, and ensure that the function of the pruned filter can be quickly compensated by the remaining network structure to prevent a sharp performance degradation, this invention employs a progressive pruning and knowledge recovery mechanism. A differentiated pruning plan is formulated based on the network depth. The knowledge recovery mechanism can be expressed as follows: .

[0011] Where 'l' represents the network layer. These represent the characteristics of the student network before and after the restoration.

[0012] As another preferred embodiment, in step 2.1 of this invention, the DCT and STFT undergo frequency domain transformation and splicing before entering the convolutional layer; specifically, this is expressed as follows: Splicing and fusion: The DCT spectrum and STFT spectrum are directly spliced ​​(Concat) at the channel dimension to form multi-scale frequency domain features; Channel attention weighting: Global average pooling is performed on the fused frequency domain features to obtain channel weights, and the feature map is weighted channel by channel to enhance the effective frequency domain fingerprint; Residual network extraction: The weighted frequency domain features are fed into ResNet-101, and spatial-frequency domain joint feature extraction is completed through convolution and residual structure without additional convolution dimensionality reduction.

[0013] As another preferred embodiment, the STFT of this invention uses a Hanning window. To balance resolution and spatial locality, a window length of 64 points is adopted, which can be adjusted appropriately according to the increase of probe density and the number of sampling points. The frequency axis dimension is 33 (generally the window length + 1). Taking into account the computational load and spectral continuity, the parameter selection step size is 32, the repetition rate is 50%, and the number of frames is 2. The number of FFT points is 64. DCT uses DCT-II to perform two-dimensional DCT on the entire image or blocks.

[0014] As another preferred embodiment, step 2.2 of this invention, spatial attention map:

[0015] The input to the SCA module is the feature map output from the corresponding stage of the fine-grained teacher network: (C represents the number of channels, H and W represent the feature map dimensions), the output is an attention-enhanced feature map. Specifically, it is divided into three parts: channel attention branch, spatial attention branch, and feature fusion. Among them, the channel attention branch (generates channel attention vectors) The core of this approach is to quantify the contribution of different channels to defect identification, filter out channels strongly correlated with defect features, and suppress channels corresponding to background noise. This involves processing the input feature map... F Perform Global Average Pooling (GAP) and Global Max Pooling (GMP) respectively to obtain two channel statistical vectors.

[0016]

[0017] Global average pooling reflects the channel c The global average characteristic response. Global max pooling reflects the channel... c The local maximum feature response is obtained by concatenating the two vectors.

[0018] Spatial attention branch (generating spatial attention graph) The core of this approach is to locate the spatial region where the defect is located, enhance the features of key positions such as the defect edges and tips, and suppress interference from the background region. Specifically, this includes channel-dimensional pooling operations: performing channel-dimensional average pooling and channel-dimensional max pooling on the input feature map F respectively to compress channel-dimensional information, preserve spatial features, and then concatenating the two spatial feature maps to obtain the final feature.

[0019] This feature map employs a 3×3 convolutional layer to reduce the dimensionality of the concatenated feature map (utilizing the local receptive field of convolution to capture spatial context information), and then generates a spatial attention map through Sigmoid activation. The feature fusion part follows the design principle of spatial-channel collaborative enhancement. Through Hadamard product operations, the original feature map is first multiplied element-wise with the spatial attention map to strengthen the feature response of the spatial region where the defect is located. Subsequently, the resulting feature map is subjected to a second Hadamard product operation with the channel attention vector to further filter out channel features highly correlated with defect identification. The final enhanced feature map achieves dual optimization of spatial location and channel dimension, effectively improving the model's ability to perceive and distinguish subtle features of magnetic flux leakage defects.

[0020] ; The combined results of the frequency domain teacher network (T1) are shown in the table below: ; Fine-grained teacher network: ;

[0021] As another preferred embodiment, step 2.3 of this invention employs an adaptive pruning module to further simplify the network, including the following parts: Dynamic filter evaluation: Quantifying the contribution of each filter based on a multi-dimensional sensitivity evaluation matrix. For the... Layer Filters Its sensitivity is defined as:

[0022] This sensitivity comprises frequency domain sensitivity, fine-grained sensitivity, and covariance penalty, among which... Indicates the first Except for the first layer All filters other than the one specified filter, and redundant filters whose covariance term is highly correlated with other filters. This is the penalty coefficient.

[0023] Dynamic sensitivity correction is applied to this sensitivity by introducing a decay factor. This gives higher weight to the sensitivity of recent assessments, and the adjusted sensitivity can be expressed as:

[0024] in, Indicates the first After the update, the first Layer The cumulative sensitivity value of each filter, This is the attenuation factor or smoothing coefficient, with a value range of [0,1]. Based on sensitivity analysis, the filter is divided into... ; Phased, gradual pruning: Develop differentiated pruning plans based on network depth and training progress. The training process is divided into multiple stages, and a pruning operation is performed after each stage: ; Differential pruning thresholds based on network depth: .

[0025] As another preferred embodiment, step 3 of the present invention specifically includes the following steps: Step 3.1 Train two teacher networks separately: using cross-entropy loss as the task loss, and the temperature coefficient of cross-entropy loss. =3, using the Adam optimizer, with a learning rate of 1e-4, and 50 training rounds, until the detection accuracy of the two teacher networks on the validation set converges; Step 3.2 Construct the dual-teacher distillation loss function. The total loss is the weighted sum of student network loss and knowledge recovery loss, as shown in the formula:

[0026]

[0027] in: This represents the distillation loss relative to the frequency domain teacher. This indicates the distillation loss compared to fine-grained teachers. It is the regularization loss of the gated network. It is the regularization coefficient. This is a balance coefficient for knowledge recovery, used to control the importance of feature consistency recovery relative to regular knowledge learning, and is typically... Between, the greater the pruning intensity, Higher values ​​indicate a higher pruning ratio, leading to more severe feature distribution shifts. This necessitates increasing the weight of the knowledge recovery loss to force feature alignment, particularly in the early stages of training. The initial value of μ is relatively low, gradually increasing in the later stages: In the early stages, the model has not fully learned the teacher's knowledge, and overemphasizing feature recovery will inhibit knowledge absorption; in the later stages, the feature distribution tends to stabilize, and the recovery constraint can be gradually strengthened. A strategy combining grid search and early stopping verification is used to determine the optimal μ.

[0028] Step 3.3 Using the pre-trained dual-teacher network as a guide, perform distillation training on the student network: set the learning rate to 5e-5, the training rounds to 50, and the batch size to 32, until the detection accuracy of the student network on the validation set converges to 92.5%.

[0029] As another preferred embodiment, in step 3.2 of this invention, candidate branches are set according to the pruning stages (progressive pruning has three stages). Value set: Phase 1 (Initial Phase, Unpruned):

[0030] Phase 2 (First pruning, pruning ratio 10%):

[0031] Phase 3 (Second pruning, pruning ratio 15%):

[0032] Validation and evaluation: for each candidate Train on the validation set for 10 epochs (early stopping threshold), and record: Validation set accuracy

[0033] Differences in characteristic distribution before and after pruning

[0034] Loss Convergence Rounds

[0035] Overall Score: Calculate the overall score ,in , , , .

[0036] Optimal choice: Select the one with the highest overall score. As a balance coefficient for the current stage.

[0037] Result confirmed: Through the above process, the optimal solution is found at each stage. The following is confirmed: ; Experiments show that using this adaptive method... The method was determined, and the model maintained stable convergence performance at each stage, resulting in a relatively consistent accuracy on the final test set. Time to improve The difference in characteristic distribution is reduced .

[0038] As another preferred embodiment, in step 3.3 of this invention, the network hyperparameters are configured as follows: ; The specific verification standards are as follows: .

[0039] As another preferred solution, in order to accurately quantify the contribution of each convolutional channel to the magnetic flux leakage defect identification task and avoid mis-pruning caused by a single indicator, step 4 of the present invention specifically includes the following steps: Step 4.1 Channel Importance Assessment: For the student network, based on the sensitivity matrix mentioned in Step 2.3, calculate the absolute value |γ| of the scaling factor γ of the batch normalization (BN) layer after each convolutional layer. Use |γ| as the feature importance score of the corresponding convolutional channel. The higher the score, the greater the contribution of the channel to the defect detection task. The sensitivity values ​​of all convolutional channels of the student network are calculated through the multidimensional sensitivity assessment matrix. Taking a typical training phase as an example, the sensitivity distribution was statistically obtained as follows: ; The sensitivity values ​​are generally distributed between 0 and 0.08, with about 38% of the channels having a sensitivity below 0.015, which are potentially redundant channels that can be pruned.

[0040] Step 4.2 Adaptive Threshold Determination: This invention proposes an adaptive threshold determination method that combines accuracy-pruning rate inflection point detection with gating weight guidance, avoiding accuracy loss or insufficient pruning problems caused by a fixed threshold. The specific process is as follows: Step 4.2.1 Candidate Threshold Generation Based on the sensitivity of all filters calculated in step 4.1 Construct a sensitivity distribution histogram and generate a candidate threshold set based on the percentiles of sensitivity values:

[0041] in The first digit representing the sensitivity distribution Percentiles. This setting ensures that the pruning percentage is from... Gradually rise to It covers the range of light to heavy pruning.

[0042] Step 4.2.2 Inflection Point Detection and Threshold Selection For each candidate threshold Perform the following operations: Pruning: Deleting sensitivity All filters are used to obtain the temporary pruning model. ; Fast evaluation: compute on the validation set Precision loss And record the pruning ratio. (That is, the proportion of the number of clipped filters to the total number). Constructing the curve: based on the pruning ratio For the x-axis, there is a loss of precision. Plot the accuracy-pruning rate curve with the vertical axis as the ordinate.

[0043] Determining inflection points using the maximum curvature method: Calculate the curvature at each point on the curve. The point of maximum curvature corresponds to the inflection point:

[0044] Pruning ratio corresponding to the inflection point and its threshold This is the optimal pruning threshold. This method automatically balances the pruning effect with the loss of accuracy, avoiding subjective settings.

[0045] Step 4.2.3 Threshold Fine-Tuning Guided by Gated Weights To make pruning more aligned with the needs of knowledge integration, the dynamic weights output by the knowledge gating module are utilized. (The thresholds are fine-tuned, corresponding to the fusion weights of frequency domain knowledge and fine-grained knowledge, respectively.)

[0046] in This is the adjustment coefficient. If the current sample relies more on frequency domain features ( If the threshold is too high (more aggressive pruning), then the threshold should be increased accordingly; otherwise, it should be decreased (more detailed feature channels should be preserved). This fine-tuning aligns the pruning strategy with the direction of knowledge transfer, further improving the overall performance of the lightweight model.

[0047] Step 4.2.4 Verification and Iteration During training, a candidate threshold set is generated according to steps 4.2.1–4.2.2. Perform pruning sequentially and record the pruning ratio and accuracy loss: ; After plotting the accuracy-pruning rate curve, calculate the curvature at each point. Curvature at threshold 0.01: 0.82; Curvature at threshold 0.015: 1.46 (peak); Curvature at threshold 0.02: 0.91. The curvature reaches its maximum at 0.015, corresponding to an inflection point. After this point, for every 1% increase in the pruning ratio, the accuracy loss accelerates, indicating that 0.015 is the optimal balance point between accuracy and compression efficiency. To verify the effectiveness of 0.015, the overall performance at different thresholds was compared on the validation set. ; At a threshold of 0.015, the model achieves an optimal balance between reducing the number of parameters to 3.5M and the inference time to 2.1ms, with an accuracy decrease of only 0.2%. Step 4.3 Redundant Channel Pruning: Based on the optimal threshold, batch prune the convolutional layer channels of the student network, deleting the convolutional kernels, BN layer parameters, and associated parameters of subsequent feature fusion layers corresponding to redundant channels. See Step 3.2 for details.

[0048] As another preferred embodiment, step 5 of the present invention specifically includes the following steps: Step 5.1 involves fine-tuning the pruned model obtained in Step 4 to recover the minor accuracy loss caused by pruning. The specific training parameters are configured as follows: Optimizer: Adam optimizer is used, momentum parameter , The weight decay coefficient is set to The batchsizes are set to 32 to prevent overfitting. Learning rate: The initial learning rate is set to A cosine annealing decay strategy is employed, gradually decaying the temperature over 50 training rounds. To avoid drastic disturbances to the parameters of the retained channels during fine-tuning. Batch size: Set to 32, consistent with the distillation training phase; Loss function: using total distillation loss ,in , This ensures that the fine-tuning process is still guided by the knowledge of both teachers, while maintaining consistency in the distribution of features before and after pruning; Validation strategy: Evaluate the model accuracy on the validation set every 5 rounds, and save the model with the highest accuracy on the validation set as the final output.

[0049] Step 5.2 Selective Parameter Update: To update only the parameters of the reserved channels, this invention employs a gradient masking mechanism, the specific steps of which are as follows: Generate the mask matrix: After the pruning operation is completed, a binary mask matrix with the same dimensions as the model parameters is generated based on the channel indices retained in step 4.3. The parameter position corresponding to the retained channel is set to 1, and the parameter position corresponding to the pruned channel is set to 0. Binding the mask and optimizer: Before fine-tuning training begins, the mask matrix is... Bind to the optimizer, perform a mask multiplication operation after each gradient backpropagation and before the parameter update:

[0050] in This indicates element-wise multiplication, ensuring that the gradient of the clipped channel is set to zero, and its parameter value always remains zero and is not updated. Freezing Pruning Channels: Through the above mechanism, the convolution kernels, BN layer parameters, and associated parameters of the subsequent feature fusion layer corresponding to the pruning channels are always frozen during the fine-tuning process and do not participate in gradient updates, thereby ensuring the consistency between the model structure and the pruning decisions.

[0051] This mechanism ensures that the fine-tuning process only optimizes the parameters of the retained channels through the dual protection of mask binding and gradient zeroing, thus avoiding the risk of parameters being accidentally activated after pruning.

[0052] Step 5.3 Performance Index Test Conditions and Benchmarks: To verify the overall performance of the fine-tuned model, this invention evaluates it under the following unified test conditions: Training environment: Intel Core i9-12900K processor, NVIDIA RTX 3090 graphics card (24GB VRAM), 64GB DDR5 memory, Ubuntu 20.04 operating system, PyTorch 2.0 deep learning framework; Deployment environment: The embedded chip is NVIDIA Jetson Nano (4G VRAM), the operating system is Ubuntu 18.04, and the inference framework is TensorRT 8.0; Data acquisition equipment: ultra-high-definition magnetic flux leakage sensor (resolution 0.1mm), data acquisition card (sampling frequency 10kHz), pipeline inspection robot (moving speed 0.5m / s).

[0053] Dataset composition: As described in step 1, it contains 5,000 data samples with three types of defects.

[0054] The results before and after the fine-tuning are as follows: .

[0055] As another preferred embodiment, step 6 of the present invention specifically includes the following steps: The ultra-high-definition MFL signal to be detected is input into the final FFDT-APNet model, and the model outputs the defect and the coordinates of the defect location, realizing real-time and high-precision detection of pipeline magnetic flux leakage defects.

[0056] A pipeline magnetic flux leakage detection system that implements the above method The system includes a data acquisition module, a data preprocessing module, a lightweight detection model module, and a result output module. These modules are interconnected sequentially, and the specific structure is as follows: Data acquisition module: Employs an ultra-high-definition magnetic flux leakage sensor and data acquisition card to acquire magnetic flux leakage signal data from the pipe surface at a sampling frequency of 10kHz, and outputs raw MFL signal data; Data preprocessing module: Communicates with the data acquisition module, performs the normalization, data partitioning and enhancement operations in step 1 of claim 1, and outputs standardized MFL detection samples; Lightweight Detection Model Module: This module incorporates the aforementioned FFDT-APNet final model, communicates with the data preprocessing module, receives standardized samples, performs defect detection, and outputs defect categories and location coordinates. This module is deployed on an embedded chip (such as NVIDIA Jetson Nano) and supports real-time inference. Results output module: Communicates with the lightweight detection model module, including a display screen and an audible and visual alarm, displays the defect detection results in real time, and triggers an audible and visual alarm if a defect is detected.

[0057] Secondly, the first teacher network of this invention is an improved ResNet-101 with added STFT and DCT, used to extract global frequency domain features of the MFL signal; the second teacher network is a ResNet-101 with added channel spatial attention mechanism, used to extract fine-grained spatial features of the MFL signal; and the student network is an adaptive pruning lightweight network based on ResNet-18. Furthermore, this invention first trains the dual-teacher network until accuracy convergence, then constructs a dual-teacher distillation loss function, using the dual-teacher network as a guide to train the student network; the formula for the dual-teacher distillation loss function is:

[0058]

[0059] in,

[0060] N is the batch size, and C is the number of categories. For indicator functions, For network prediction samples Category The probability, It is the weighting coefficient that balances the two loss terms.

[0061]

[0062] in, For temperature parameters, For the sample The true label, The variance coefficient is used to ensure that the similarity distribution of the sample is sufficiently different from that of all prototypes, thus avoiding trivial solutions.

[0063]

[0064] in, is the normalized distance between samples i and j in the original image space.

[0065] The beneficial effects of this invention.

[0066] Compared with the prior art, the present invention has the following outstanding substantive features and significant progress: Comprehensive improvement in feature extraction: For the first time, a fine-grained dual-teacher distillation strategy in the frequency domain is proposed, which simultaneously transfers the global frequency domain features and fine-grained spatial features of the MFL signal, solving the problem of incomplete feature extraction by single distillation. This enables the detection accuracy of the lightweight model to reach 92.1%, which is better than the traditional single-teacher distillation model.

[0067] Significant model lightweighting effect: Based on the adaptive pruning strategy of BN layer scaling factor, the number of model parameters is reduced from 11.7M to 3.5M and the inference time is compressed from 4.8ms to 2.1ms (specific implementation step 6) while controlling the accuracy loss ≤0.4%, which fully meets the real-time deployment requirements of embedded devices.

[0068] Highly practical for engineering applications: The methods and processes are standardized, and the system can be directly deployed in industrial scenarios such as portable pipeline inspection robots and fixed pipeline inspection stations, and is compatible with batch processing of ultra-high-definition MFL inspection data. Attached Figure Description

[0069] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. The scope of protection of the present invention is not limited to the following description.

[0070] Figure 1 This is a flowchart illustrating the overall process of the pipeline magnetic flux leakage detection method of the present invention.

[0071] Figure 2 This is a schematic diagram of the structure of the frequency domain fine-grained dual-teacher network and student network of the present invention.

[0072] Figure 3 This is a comparison diagram of different methods based on feature extraction capabilities according to the present invention.

[0073] Figure 4 This is a schematic diagram of the module structure of the pipeline magnetic flux leakage detection system of the present invention.

[0074] Figure 5 This is a confusion matrix diagram of the recognition results of different models in the embodiments of the present invention. Detailed Implementation

[0075] Hardware and software environment Training environment: Intel Core i9-12900K processor, NVIDIA RTX 3090 graphics card (24GB VRAM), 64GB DDR5 memory, Ubuntu 20.04 operating system, PyTorch 2.0 deep learning framework; Deployment environment: The embedded chip is NVIDIA Jetson Nano (4G VRAM), the operating system is Ubuntu 18.04, and the inference framework is TensorRT 8.0; Data acquisition equipment: ultra-high-definition magnetic flux leakage sensor (resolution 0.1mm), data acquisition card (sampling frequency 10kHz), pipeline inspection robot (moving speed 0.5m / s).

[0076] Specific implementation steps According to the method described in this invention, the following steps are completed sequentially: dataset construction, network construction, distillation training, pruning optimization, fine-tuning deployment, and actual detection. The specific process is as follows: Step 1. MFL detection dataset construction and preprocessing MFL (Medium-to-Fluid Fiber) inspection data were collected from a Φ1016 (outer diameter of 1016 mm) long-distance oil pipeline. The dataset contains complete defect types of different sizes and shapes. Data acquisition was based on the controlled variable method. While keeping the defect width and depth constant, the length was adjusted within the range of 10 mm to 120 mm, and the integer value was changed in 10 mm intervals. While keeping the defect length and depth constant, the width was dynamically adjusted within the range of 10 mm to 120 mm. Finally, with the width and length fixed, the depth was gradually increased from 1 mm to the pipe wall thickness. The dataset consists of three parts: 366 sets of industrial field measured signals from Φ1016 large-diameter natural gas pipelines (including 8–20 dB noise and channel damage), 2,500 sets of signals generated by MATLAB magnetic dipole model simulation based on 122 actual defect geometries (covering the parameter space from 10–120 mm in length and 1 mm in depth to the full wall thickness), and 2,500 sets of data augmentation samples generated by adding random noise and channel damage to the real defect data. The magnetic flux density of all samples was uniformly controlled at 1.8T±0.05T, the lift-off value was fixed at 2mm, and the signal-to-noise ratio (SNR) was not less than 15dB.

[0077] Step 2. Construction of a fine-grained frequency domain dual-teacher network and a lightweight student network Step 2.1 First Teacher Network T1: The original spatial signal is subjected to Discrete Cosine Transform (DCT) and Short Time Fourier Transform (STFT) is calculated to capture local features; the two are concatenated along the channel dimension to convert into a frequency domain spectrum as input, so that the ResNet-101 teacher can directly learn the frequency domain features.

[0078] Step 2.2 Second Teacher Network T2: After stage 2, stage 3, and stage 4 of the fine-grained teacher's ResNet-101 network, a Spatial-Channel Collaborative Attention (SCA) module is inserted. This module simultaneously learns the spatial attention map. and channel attention vector Fine-grained representations are obtained by fusing multi-scale features through a Feature Pyramid Network (FPN). Finally, the prototype comparison function is used to adjust the fine-grained features and the stable, gradual differences between the prototype and the model.

[0079] Step 2.3 Student Network S: ResNet18 is used as the basic framework. The student network is optimized for lightweighting through adaptive pruning. At the same time, a knowledge recovery mechanism is introduced to avoid pruning necessary layers of the student network.

[0080] Step 3. Student online pre-training through dual-teacher knowledge distillation Step 3.1 Training a dual-teacher network: Adam optimizer, weight decay coefficient of 5e-4, learning rate using cosine annealing strategy (initial 1e-4, final 1e-6), after 50 training rounds, T1 achieved a detection accuracy of 94.2% on the validation set, and T2 achieved 93.8%; Step 3.2 Distillation training of the student network: Using the above dual-teacher loss function, T=3, Adam optimizer, learning rate 5e-5, after 50 training rounds, the detection accuracy of the student network on the validation set converged to 92.5%, the number of parameters remained at 8.2M, and the inference time was 4.8ms.

[0081] Step 4. Adaptive pruning optimization based on BN layer scaling factor Step 4.1 Calculate the absolute value of the scaling factor γ of each convolutional layer and BN layer in the student network. The statistical distribution of γ is 0-0.08. Step 4.2 Iterate through the thresholds 0.005, 0.01, 0.015, and 0.02 on the validation set. The corresponding precision losses are 0.8%, 0.6%, 0.4%, and 1.2%, respectively. Determine the optimal threshold as 0.015. Step 4.3 Remove redundant channels at a threshold of 0.015, deleting a total of 128 convolutional channels to obtain the preliminary pruned model FFDT-APNet. The number of parameters is reduced to 3.5M, the inference time is reduced to 2.3ms, and the validation set accuracy is 92.1%.

[0082] Step 5. Fine-tuning and performance optimization of the pruning model Fine-tuning of the FFDT-APNet model: learning rate 1e-5, training for 50 epochs, updating only the parameters of the retained channels. After fine-tuning, the model's accuracy on the validation set recovered to 92.5%, and the accuracy on the test set was 92.1%. The inference time was further optimized to 2.1ms (after TensorRT acceleration).

[0083] Step 6. Verification using actual pipeline magnetic flux leakage detection The final FFDT-APNet model was deployed on an NVIDIA Jetson Nano embedded chip and mounted on a pipeline inspection robot to conduct actual inspections on a section of an oil pipeline. Performance comparisons with different models showed that the proposed method achieved the best detection results, with an overall accuracy of 0.921. Across all categories, class Z achieved 100% recognition, class Y achieved 91.0%, and only class X showed a small number of misclassifications. This method enhances the ability to distinguish easily confused features in classes X and Z through multi-dimensional knowledge fusion and precise pruning strategies. The results fully validate the effectiveness and superiority of the proposed framework.

[0084] Table 1 shows the different model complexities and number of parameters. It can be seen that the present invention achieves lightweighting of network model parameters and has achieved significant results in optimizing inference time and computational complexity.

[0085] Table 1 Performance Comparison of Different Models ; It is understood that the above specific description of the present invention is only for illustrating the present invention and is not limited to the technical solutions described in the embodiments of the present invention. Those skilled in the art should understand that modifications or equivalent substitutions can still be made to the present invention to achieve the same technical effect; as long as the use needs are met, they are all within the protection scope of the present invention.

Claims

1. A pipeline magnetic flux leakage detection method based on dual-teacher distillation and adaptive pruning, characterized in that... Includes the following steps: Step 1: MFL detection dataset construction and preprocessing; Step 2: Construction of a fine-grained frequency domain dual-teacher network and a lightweight student network; Step 3: Online pre-training for students using dual-teacher knowledge distillation; Step 4: Adaptive pruning optimization based on BN layer scaling factor; Step 5: Fine-tuning and performance optimization of the pruning model; Step 6: End-to-end inspection of MFL defects.

2. The pipeline magnetic flux leakage detection method based on dual-teacher distillation and adaptive pruning according to claim 1, characterized in that... Step 1 specifically includes the following steps: Step 1.1: Collect raw data from ultra-high-definition pipeline MFL inspection, covering radial, axial, and circumferential defect data, and construct a dataset containing samples; Step 1.2: Normalize the original MFL signal using the following formula: Where x is the original signal value, The sample mean. The standard deviation is the sample standard deviation. Step 1.3: Divide the dataset into training, validation and test sets according to the proportions, and perform data augmentation processing on the training set samples by randomly flipping and adding Gaussian noise to improve the model's generalization ability.

3. The pipeline magnetic flux leakage detection method based on dual-teacher distillation and adaptive pruning according to claim 1, characterized in that... Step 2 specifically includes the following steps: Step 2.1: First Teacher Network T1 - Frequency Domain Feature Extraction Network: An improved ResNet-101 is used as the basic framework. Short Time Fourier Transform (STFT) and Discrete Cosine Transform (DCT) are added to the backbone layer of the network. Combined with frequency domain consistency loss, the noise resistance is enhanced, and the unique frequency components and energy spectrum patterns of defects are captured. Step 2.2: Second Teacher Network T2 – Fine-grained Spatial Feature Extraction Network: Spatial-Channel Collaborative Attention (SCA) modules are inserted after ResNet-101 network stages 2, 3, and 4, respectively. This module simultaneously learns the spatial attention map. and channel attention vector Attention weights are used to enhance the extraction of fine-grained spatial features such as defect edges and micro-cracks; Step 2.3: Student Network S - Lightweight Base Network: Construct a lightweight backbone network based on ResNet18, named FFDT-APNet basic framework; Step 2.4: Gating Adjustment Module During training, the gating network is trained together with the student network. , These represent the weights of frequency domain knowledge and fine-grained knowledge, respectively, satisfying... + , It is the regularization loss of the gated network. It is the regularization coefficient; the gated network is a lightweight network that takes the intermediate features of the student network as input and outputs the fused weights of the two teachers, expressed as: ； in, A measure of the frequency domain components of student characteristics; Step 2.5: Knowledge Recovery Mechanism A progressive pruning and knowledge recovery mechanism is adopted, and a differentiated pruning plan is formulated based on the network depth. The knowledge recovery mechanism is represented as follows: ； Where 'l' represents the network layer. These represent the characteristics of the student network before and after the restoration.

4. The pipeline magnetic flux leakage detection method based on dual-teacher distillation and adaptive pruning according to claim 3, characterized in that... In step 2.1, the DCT and STFT undergo frequency domain transformation and splicing before entering the convolutional layer; specifically, this is expressed as follows: Splicing and fusion: The DCT spectrum and STFT spectrum are directly spliced ​​and concatted in the channel dimension to form multi-scale frequency domain features; Channel attention weighting: Global average pooling is performed on the fused frequency domain features to obtain channel weights, and the feature map is weighted channel by channel to enhance the effective frequency domain fingerprint; Residual network extraction: The weighted frequency domain features are fed into ResNet-101, and spatial-frequency joint feature extraction is completed through convolution and residual structure without additional convolution dimensionality reduction; Step 2.2, Spatial Attention Map: ； The input to the SCA module is the feature map output from the corresponding stage of the fine-grained teacher network: C represents the number of channels, H and W represent the feature map dimensions, and the output is an attention-enhanced feature map. Specifically, it is divided into three parts: channel attention branch, spatial attention branch, and feature fusion. The channel attention branch quantifies the contribution of different channels to defect recognition, filters out channels strongly correlated with defect features, and suppresses channels corresponding to background noise. The input feature map... F Perform global average pooling (GAP) and global max pooling (GMP) respectively to obtain two channel statistical vectors; ； Global average pooling reflects the channel c The global average characteristic response; global max pooling reflects the channel c The local maximum feature response is obtained by concatenating the two vectors. ; The spatial attention branch locates the spatial region where the defect is located, enhances the features of the defect edges and key points, and suppresses interference from the background region. Specifically, this includes channel-dimensional pooling operations: performing channel-dimensional average pooling and channel-dimensional max pooling on the input feature map F respectively to compress channel-dimensional information, preserve spatial features, and then concatenating the two spatial feature maps to obtain the final feature. ； This feature uses convolutional layers to reduce the dimensionality of the concatenated feature map, and then generates a spatial attention map through Sigmoid activation. The feature fusion part follows the design principle of spatial-channel collaborative enhancement. Through Hadamard product operation, the original feature map is first multiplied element-wise with the spatial attention map to enhance the feature response of the spatial region where the defect is located. Then, the obtained feature is subjected to a second Hadamard product operation with the channel attention vector to further filter out the channel features that are highly related to defect recognition. The final output enhanced feature map achieves dual optimization of spatial location and channel dimension, effectively improving the model's ability to perceive and distinguish subtle features of magnetic leakage defects. Step 2.3 employs an adaptive pruning module to further simplify the network, including the following components: Dynamic filter evaluation: Quantifying the contribution of each filter based on a multidimensional sensitivity evaluation matrix; for the ... Layer Filters Its sensitivity is defined as: ； This sensitivity comprises frequency domain sensitivity, fine-grained sensitivity, and covariance penalty, among which... Indicates the first Except for the first layer All filters other than the one specified filter, and redundant filters whose covariance term is highly correlated with other filters. This is the penalty coefficient; Dynamic sensitivity correction is applied to this sensitivity by introducing a decay factor. This gives higher weight to the sensitivity of recent assessments, and the adjusted sensitivity can be expressed as: ； in, Indicates the first After the update, the first Layer The cumulative sensitivity value of each filter, This is the attenuation factor or smoothing coefficient, with a value range of [0,1]. Based on sensitivity analysis, the filter is divided into... ； Phased progressive pruning: Develop differentiated pruning plans based on network depth and training progress; The training process is divided into multiple stages, and a pruning operation is performed after each stage: ； Differential pruning thresholds based on network depth: 。 5. The pipeline magnetic flux leakage detection method based on dual-teacher distillation and adaptive pruning according to claim 1, characterized in that... Step 3 specifically includes the following steps: Step 3.1 Train two teacher networks separately: use cross-entropy loss as the task loss and adopt the Adam optimizer until the detection accuracy of the two teacher networks on the validation set converges. Step 3.2 Construct the dual-teacher distillation loss function. The total loss is the weighted sum of student network loss and knowledge recovery loss, as shown in the formula: ； ； in: This represents the distillation loss relative to the frequency domain teacher. This indicates the distillation loss compared to fine-grained teachers. It is the regularization loss of the gated network. It is the regularization coefficient; This is a balance coefficient for knowledge recovery, used to control the importance of feature consistency recovery relative to regular knowledge learning; Step 3.3: Using the pre-trained dual-teacher network as a guide, perform distillation training on the student network.

6. The pipeline magnetic flux leakage detection method based on dual-teacher distillation and adaptive pruning according to claim 1, characterized in that... Step 4 specifically includes the following steps: Step 4.1 Channel Importance Assessment: For the student network, based on the sensitivity matrix mentioned in Step 2.3, calculate the absolute value |γ| of the scaling factor γ of the post-batch normalized BN layer for each convolutional layer. Use |γ| as the feature importance score for the corresponding convolutional channel. The higher the score, the greater the contribution of the channel to the defect detection task. The sensitivity values ​​of all convolutional channels of the student network are calculated through a multi-dimensional sensitivity assessment matrix. ; Step 4.2 Adaptive threshold determination; Step 4.2.1 Candidate threshold generation; Based on the sensitivity of all filters calculated in step 4.1 Construct a sensitivity distribution histogram and generate a candidate threshold set based on the percentiles of sensitivity values: ； in The first digit representing the sensitivity distribution percentile; Step 4.2.2 Inflection Point Detection and Threshold Selection For each candidate threshold Perform the following operations: Pruning: Deleting sensitivity All filters are used to obtain the temporary pruning model. ; Fast evaluation: compute on the validation set Precision loss And record the pruning ratio. ; Constructing the curve: based on the pruning ratio For the x-axis, there is a loss of precision. Plot the accuracy-pruning rate curve with the vertical axis as the ordinate; Determining inflection points using the maximum curvature method: Calculate the curvature at each point on the curve. The point of maximum curvature corresponds to the inflection point: ； Pruning ratio corresponding to the inflection point and its threshold This is the optimal pruning threshold; Step 4.2.3 Threshold fine-tuning guided by gating weights; Utilizing the dynamic weights output by the knowledge gating module Fine-tune the threshold: in This is the adjustment coefficient; if the current sample relies more on frequency domain features... If the threshold is too high, then the threshold should be raised appropriately; otherwise, the threshold should be lowered. Step 4.2.4 Verification and Iteration; During training, a candidate threshold set is generated according to steps 4.2.1–4.2.

2. Perform pruning sequentially and record the pruning ratio and accuracy loss: After plotting the accuracy-pruning rate curve, calculate the curvature at each point. The curvature is at its maximum, corresponding to the inflection point. After the inflection point, the accuracy loss increases rapidly with each 1% increase in the pruning ratio, indicating that this is the optimal balance point between accuracy and compression efficiency. To verify the effectiveness, the overall performance under different thresholds is compared on the validation set. Step 4.3 Redundant Channel Pruning: Based on the optimal threshold, batch prune the convolutional layer channels of the student network, deleting the convolutional kernels, BN layer parameters, and associated parameters of subsequent feature fusion layers corresponding to redundant channels.

7. The pipeline magnetic flux leakage detection method based on dual-teacher distillation and adaptive pruning according to claim 1, characterized in that... Step 5 specifically includes the following steps: Step 5.1 involves fine-tuning the pruned model obtained in Step 4 to recover the minor accuracy loss caused by pruning; the specific training parameters are configured as follows: The Adam optimizer is used. A cosine annealing attenuation strategy is adopted; The batch size remains consistent with the distillation training phase. Loss function: using total distillation loss ,in ; Validation strategy: Evaluate the model accuracy on the validation set every 5 rounds, and save the model with the highest accuracy on the validation set as the final output; Step 5.2 Selective Parameter Update: To update only the parameters of the reserved channels, a gradient masking mechanism is used. The specific steps are as follows: Generate the mask matrix: After the pruning operation is completed, a binary mask matrix with the same dimensions as the model parameters is generated based on the channel indices retained in step 4.

3. The parameter position corresponding to the retained channel is set to 1, and the parameter position corresponding to the pruned channel is set to 0. Binding the mask and optimizer: Before fine-tuning training begins, the mask matrix is... Bind to the optimizer, perform a mask multiplication operation after each gradient backpropagation and before the parameter update: ； in This indicates element-wise multiplication, ensuring that the gradient of the clipped channel is set to zero, and its parameter value always remains zero and is not updated. Freezing the pruning path: Through the above mechanism, the convolution kernel, BN layer parameters and related parameters of the subsequent feature fusion layer corresponding to the pruning path are always frozen during the fine-tuning process and do not participate in gradient updates, thereby ensuring the consistency between the model structure and the pruning decision. Step 5.3 Performance Index Test Conditions and Benchmarks: To verify the overall performance of the fine-tuned model, an evaluation is conducted under unified test conditions.

8. The pipeline magnetic flux leakage detection method based on dual-teacher distillation and adaptive pruning according to claim 1, characterized in that... Step 6 specifically includes the following steps: The ultra-high-definition MFL signal to be detected is input into the final FFDT-APNet model, and the model outputs the defect and the coordinates of the defect location, realizing real-time and high-precision detection of pipeline magnetic flux leakage defects.

9. The pipeline magnetic flux leakage detection method based on dual-teacher distillation and adaptive pruning according to claim 3, characterized in that... The first teacher network is an improved ResNet-101 with added STFT and DCT, used to extract the global frequency domain features of the MFL signal; The second teacher network is a ResNet-101 with added channel spatial attention mechanism, used to extract fine-grained spatial features of the MFL signal; The student network is an adaptive pruning lightweight network based on ResNet-18.

10. The pipeline magnetic flux leakage detection method based on dual-teacher distillation and adaptive pruning according to claim 1, characterized in that... First, train the dual-teacher network until accuracy convergence, then construct the dual-teacher distillation loss function, and train the student network using the dual-teacher network as guidance; the formula for the dual-teacher distillation loss function is: ； ； in, ； N is the batch size, and C is the number of categories. For indicator functions, For network prediction samples Category The probability, These are the weighting coefficients that balance the two loss terms; ； in, For temperature parameters, For the sample The true label, The variance coefficient is used to ensure that the similarity distribution between the sample and all prototypes is sufficiently different, thus avoiding trivial solutions. ； in, is the normalized distance between samples i and j in the original image space.

Images