An adaptive noise reduction method, system, and medium for pipeline magnetic flux leakage detection data.

Abstract

This invention discloses an adaptive denoising method, system, and medium for pipeline magnetic flux leakage (MFL) detection data, belonging to the field of pipeline nondestructive testing data processing technology. The method includes acquiring an MFL data matrix, which includes sensor channel dimensions and pipeline axial sampling dimensions; dividing the MFL data matrix into multiple data blocks with overlapping regions along the pipeline axial sampling dimension; determining amplitude indices and amplitude thresholds; generating strong and weak signal region masks based on the amplitude indices and amplitude thresholds; inputting the data blocks into a pre-trained MFL denoising model to obtain denoised data blocks; fusing the original data blocks and denoised data blocks according to the masks; and smoothly stitching together the overlapping regions of adjacent fused data blocks to obtain the denoised MFL data matrix. This invention solves the technical problem in existing pipeline MFL detection data denoising processes where a uniform denoising strategy easily leads to over-smoothing of strong signal regions suspected of defects.

Description

Technical Field

[0001] This invention relates to the field of pipeline non-destructive testing data processing technology, and in particular to an adaptive noise reduction method, system and medium for pipeline magnetic flux leakage testing data. Background Technology

[0002] Magnetic flux leakage (MFL) testing is a common non-destructive testing method for pipeline inspection. During the testing process, the MFL sensor moves along the pipeline axis and collects magnetic field response data, thus forming pipeline MFL data. This data can typically be represented as an MFL data matrix, where one dimension corresponds to multiple sensor channels and the other dimension corresponds to the axial sampling points of the pipeline. In actual testing, MFL data is easily affected by factors such as sensor noise, pipeline surface roughness, lift-off effect, electromagnetic interference, equipment range differences, and sampling bias, resulting in background noise in the collected MFL data. Noise reduces the reliability of subsequent defect detection, defect location, defect depth estimation, or defect region identification.

[0003] Existing MFL data denoising methods typically include low-pass filtering, median filtering, wavelet thresholding, empirical mode decomposition (EMD), and neural network-based denoising methods. Traditional filtering methods are relatively simple to implement, but they tend to smooth out peaks, edges, or local morphologies in defect areas while suppressing noise. Wavelet thresholding and EMD methods rely on parameter selection and have limited adaptability to data from different inspection devices, pipe diameters, and sampling conditions. While neural network-based denoising methods can learn the mapping relationship between noisy signals and target signals, uniformly applying denoising to all MFL data may smooth out local strong signals in suspected defect areas as noise, resulting in defect amplitude attenuation or loss of defect edge details.

[0004] Furthermore, pipeline inspection data typically has a long axial sampling length. To adapt to the model input size, it is often necessary to perform block processing along the pipeline axial sampling dimension. If the denoising results of each data block are directly stitched together after block processing, discontinuities can easily occur at the window boundaries, affecting the overall continuity of the denoised MFL data.

[0005] Therefore, it is necessary to provide an adaptive denoising method for pipeline magnetic flux leakage detection data that can combine the amplitude characteristics of MFL data, the type of detection equipment, and the window overlap stitching mechanism, so as to reduce noise in weak signal areas, reduce excessive smoothing in strong signal areas of suspected defects, and improve the continuity at window boundaries. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides an adaptive denoising method, system, and medium for pipeline magnetic flux leakage detection data. The aim is to solve the technical problem that in the existing pipeline magnetic flux leakage detection data denoising process, a uniform denoising strategy can easily lead to over-smoothing of strong signal areas suspected of defects.

[0007] To achieve the above objectives, this application provides the following solution: In a first aspect, this application provides an adaptive denoising method for pipeline magnetic flux leakage detection data, including acquiring an MFL data matrix formed by a pipeline magnetic flux leakage detection sensor, wherein the MFL data matrix includes sensor channel dimensions and pipeline axial sampling dimensions; The MFL data matrix is ​​divided into multiple data blocks with overlapping regions along the sampling dimension of the pipe axis; The amplitude index is determined based on the amplitude variation of the sampled values ​​of the sensor channels in each data block; the amplitude threshold is determined based on the type of detection equipment; and a strong / weak signal region mask is generated based on the amplitude index and amplitude threshold. The data block is input into a pre-trained MFL denoising model to obtain a denoised data block; the original data block and the denoised data block are then fused according to the mask. In this process, the strong signal region is selected from the corresponding data in the original data block, and the weak signal region is selected from the corresponding data in the denoised data block; the overlapping regions of adjacent fused data blocks are smoothly spliced ​​to obtain the denoised MFL data matrix.

[0008] Secondly, this application provides an adaptive noise reduction system for pipeline magnetic flux leakage detection data, including a data block module for acquiring an MFL data matrix formed by a pipeline magnetic flux leakage detection sensor, and dividing the MFL data matrix into multiple data blocks with overlapping regions along the pipeline axial sampling dimension. The amplitude analysis module is used to determine an amplitude index based on the amplitude changes of the sampled values ​​of the sensor channels in each data block; determine an amplitude threshold based on the type of detection equipment; and generate a strong / weak signal region mask based on the amplitude index and amplitude threshold. The denoising inference module is used to input the data block into the pre-trained MFL denoising model to obtain the denoised data block. An adaptive fusion module is used to fuse the original data block and the denoised data block according to the mask, wherein the strong signal region selects the corresponding data in the original data block, and the weak signal region selects the corresponding data in the denoised data block. The smooth stitching module is used to smoothly stitch together the overlapping areas of adjacent fused data blocks to obtain a denoised MFL data matrix.

[0009] Thirdly, this application provides a computer-readable medium having a computer program / instructions stored thereon, which, when executed by a processor, implement the steps of the method described above.

[0010] The beneficial effects of this invention through the above technical solutions are as follows: Strong and weak signal region masks are generated based on amplitude indicators and amplitude thresholds, enabling different processing strategies for strong and weak signal regions in the MFL data. For strong signal regions, corresponding data from the original data block is selected to reduce the smoothing of suspected defect signals during model denoising; for weak signal regions, corresponding data from the denoised data block is selected to reduce background noise interference. Furthermore, this invention determines the amplitude threshold according to the type of detection equipment and smoothly stitches together overlapping areas of adjacent fused data blocks, thereby improving the consistency of data processing from different devices and the axial continuity of the denoising results. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention, and the embodiments in the accompanying drawings do not constitute any limitation on the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 A flowchart of an adaptive noise reduction method for pipeline magnetic flux leakage detection data provided in an embodiment of this application; Figure 2 This is a schematic diagram of an adaptive fusion strategy provided in an embodiment of this application; Figure 3 This is a schematic diagram of sliding window segmentation and splicing provided in an embodiment of this application; Figure 4 This is a schematic diagram of dual-path prediction denoising provided in an embodiment of this application; Figure 5 This is a structural diagram of an adaptive noise reduction system for pipeline magnetic flux leakage detection data provided in an embodiment of this application; Figure 6 This is a diagram illustrating the overall architecture of a noise reduction system provided in an embodiment of this application. Figure 7 This is a schematic diagram of the structure of a computer device provided in one embodiment of this application; The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0013] It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of the application. Rather, these embodiments are provided to make the disclosure more thorough and complete, and to fully convey the scope of the disclosure to those skilled in the art.

[0014] The foregoing and other technical contents, features and effects of the present invention are described in conjunction with the appendix below. Figure 1-7 The detailed description of the embodiments will make this clear. All structural details mentioned in the following embodiments are based on the accompanying drawings.

[0015] Exemplary embodiments of the present invention will now be described with reference to the accompanying drawings.

[0016] In one exemplary embodiment, such as Figure 1 As shown, an adaptive denoising method for pipeline magnetic flux leakage detection data is provided, the method including: S101, acquire the MFL data matrix formed by the pipeline magnetic flux leakage detection sensor, the MFL data matrix including the sensor channel dimension and the pipeline axial sampling dimension; S102, the MFL data matrix is ​​divided into multiple data blocks with overlapping regions along the sampling dimension of the pipe axis; S103, determine the amplitude index based on the amplitude change of the sampled values ​​of the sensor channels in each data block; determine the amplitude threshold based on the type of detection equipment; generate a strong and weak signal region mask based on the amplitude index and amplitude threshold; S104, Input the data block into the pre-trained MFL denoising model to obtain a denoised data block; Fuse the original data block and the denoised data block according to the mask; S105, where the strong signal region is selected from the corresponding data in the original data block, and the weak signal region is selected from the corresponding data in the denoised data block; the overlapping regions of adjacent fused data blocks are smoothly spliced ​​to obtain the denoised MFL data matrix.

[0017] In step S101, the pipeline magnetic flux leakage (MFL) detection sensor moves axially within the pipeline and collects MFL response signals. The signal values ​​collected by multiple sensor channels at different axial sampling points collectively form an MFL data matrix. The MFL data matrix can be represented as a two-dimensional matrix, with one dimension corresponding to the sensor channel and the other dimension corresponding to the axial sampling point of the pipeline.

[0018] For step S102, since pipeline magnetic flux leakage (MFL) detection data typically has a long axial sampling length, directly inputting the complete MFL data matrix into the model for processing may be limited by the model input size or computational resources. Therefore, this embodiment divides the MFL data matrix into blocks along the pipeline axial sampling dimension, so that each data block has a size suitable for the input of the MFL denoising model. Overlapping regions are set between adjacent data blocks to facilitate smooth splicing of the processing results of adjacent data blocks in subsequent steps.

[0019] For step S103, for each data block, the amplitude change of the sampled values ​​of each sensor channel is calculated to obtain the corresponding amplitude index. The amplitude index is used to characterize the degree of signal change of the sensor channel within the current data block. Simultaneously, a corresponding amplitude threshold is determined based on the type of detection device. The amplitude index is compared with the amplitude threshold to generate a strong / weak signal region mask. The strong / weak signal region mask is used to indicate which regions in the data block belong to strong signal regions and which belong to weak signal regions.

[0020] In step S104, the pre-trained MFL denoising model is used to perform denoising inference on the input data block and output a denoised data block corresponding to the original data block. Subsequently, based on the strong and weak signal region mask, the original data block and the denoised data block are selectively fused to form a fused data block.

[0021] In step S105, for data marked as strong signal regions by the mask, the corresponding data in the original data block is selected as the data in the fused data block; for data marked as weak signal regions by the mask, the corresponding data in the denoised data block is selected as the data in the fused data block. This allows for noise suppression in weak signal regions while reducing the risk of over-smoothing of suspected defective strong signal regions by the denoising model. Subsequently, overlapping areas of adjacent fused data blocks are smoothly stitched together to obtain a complete denoised MFL data matrix. This embodiment selectively fuses the original data block and the denoised data block using strong and weak signal region masks, avoiding the uniform application of model denoising results to all MFL data, thus achieving a balance between noise suppression and defect signal preservation.

[0022] In one specific embodiment, the rows of the MFL data matrix correspond to the sensor channels of the pipeline magnetic flux leakage detection sensor, and the columns correspond to the axial sampling points of the pipeline; the MFL data matrix is ​​divided into multiple data blocks with overlapping regions along the axial sampling dimension of the pipeline, including: Sliding windows are divided along the axial sampling dimension of the pipeline according to a preset window width, and adjacent windows have a preset overlap width. When the data length corresponding to the last window is less than the preset window width, the last window is edge-filled, and the fill data generated by the edge filling is cropped after smooth splicing.

[0023] The rows of the MFL data matrix correspond to the sensor channels of the pipeline magnetic flux leakage detection sensor, and the columns correspond to the axial sampling points of the pipeline. That is, the MFL data matrix can be represented as an N×M matrix, where N represents the number of sensor channels and M represents the number of axial sampling points of the pipeline.

[0024] For example, the data in the nth row and mth column of the matrix represents the magnetic flux leakage (MFL) detection signal value collected by the nth sensor channel at the mth axial sampling point of the pipe. This matrix form preserves the spatial correspondence of MFL data along the sensor channel direction and the pipe axial direction.

[0025] When dividing the MFL data matrix into multiple data blocks with overlapping regions along the pipe axial sampling dimension, a sliding window segmentation can be performed according to a preset window width. Specifically, if the preset window width is lineWidth and the preset overlap width is overlap, then the width of each data block along the pipe axial sampling dimension is lineWidth, and adjacent data blocks overlap by overlap sampling points along the pipe axial sampling dimension.

[0026] In one example, lineWidth can be 200 and overlap can be 10. It should be understood that the above values ​​are only one specific implementation, and the preset window width and preset overlap width can also be set according to the input size of the MFL denoising model, the length of the detection data, the sampling interval, or the computing resources.

[0027] When the data length corresponding to the last window is less than the preset window width, edge padding is applied to the last window. Edge padding can be achieved by repeating the value of the last sampling point, ensuring that the last data block meets the preset window width requirement. After denoising, fusion, and smooth stitching, the padding data generated by edge padding is trimmed so that the final output denoised MFL data matrix has the same length as the original MFL data matrix in the pipe axial sampling dimension. Through the above segmentation and padding methods, pipe magnetic flux leakage detection data of variable length can be converted into fixed-length data blocks suitable for model processing, while the overlapping regions provide a data foundation for subsequent smooth stitching.

[0028] In one specific embodiment, the amplitude index is determined based on the amplitude change of the sampled values ​​of the sensor channels in each data block, including: For the nth sensor channel in the jth data block, determine the maximum and minimum sample values ​​of the sensor channel within the jth data block, and determine the difference between the maximum and minimum sample values ​​as the amplitude index of the nth sensor channel within the jth data block.

[0029] The sampled value sequence can be denoted as: X(j,n)={x1,x2,...,xw}. Here, w represents the number of sampling points in the axial sampling dimension of the data block. The maximum sampled value max(X(j,n)) and the minimum sampled value min(X(j,n)) in this sampled value sequence are determined, and the difference between the maximum and minimum sampled values ​​is defined as the amplitude index of the nth sensor channel within the jth data block. amp(j,n) = max(X(j,n)) - min(X(j,n)). Here, amp(j,n) represents the amplitude index corresponding to the nth sensor channel in the j-th data block.

[0030] Performing the above calculations on each sensor channel in the j-th data block yields the amplitude index array corresponding to that data block: Amp(j)={amp(j,1),amp(j,2),...,amp(j,N)}. Here, N represents the number of sensor channels.

[0031] Defective regions in MFL data typically cause changes in local magnetic flux leakage response, resulting in significant amplitude variations in the corresponding sensor channel within a certain axial sampling range. Characterizing these amplitude variations by the difference between the maximum and minimum sampled values ​​can provide a basis for subsequent identification of strong and weak signal regions.

[0032] In one specific embodiment, determining the amplitude threshold based on the type of detection device includes: Read the device identifier field from the MFL data file header. The device identifier field is used to characterize the type, series, or data format version of the detection device from which the MFL data originates. The device identifier field can be stored in the header area of ​​the MFL data file, or in a configuration file or metadata associated with the MFL data.

[0033] The system determines the type of device to be detected based on the device identification field. For example, the system can parse the string, version number, or device code in the device identification field and match the parsing result with a preset device type list to determine the type of device to be detected corresponding to the current MFL data.

[0034] The amplitude threshold is determined based on a preset mapping relationship between device type and amplitude threshold. This mapping relationship can be pre-stored in a configuration file, database, parameter table, or program configuration item. Different detection devices may have different signal ranges, bias parameters, or signal amplitude distributions; therefore, setting corresponding amplitude thresholds for different detection devices helps improve the consistency of strong and weak signal region judgment.

[0035] For example, the mapping relationship between device type and amplitude threshold can include: a first type of detection device corresponding to a first amplitude threshold; a second type of detection device corresponding to a second amplitude threshold; and a third type of detection device corresponding to a third amplitude threshold. After determining the detection device type, the system reads the corresponding amplitude threshold from the mapping relationship and uses this amplitude threshold for subsequent generation of strong and weak signal region masks.

[0036] In one specific embodiment, the device identification field includes an mflVersion field. The mflVersion field is used to characterize the detection device type or data format version corresponding to the MFL data file.

[0037] When the mflVersion field indicates a SY813 device, the current detection device type is determined to be a SY813 device, and the amplitude threshold is set to 3000. When the mflVersion field indicates an HD device, MFL device, ZY device, or ZHT device, the amplitude threshold is set to 200.

[0038] In one implementation, it can be determined whether the mflVersion field begins with a character or string used to characterize the SY813 device; if so, an amplitude threshold of 3000 is used; if not, it can be further determined whether it belongs to the device identifier corresponding to the HD device, MFL device, ZY device or ZHT device, and an amplitude threshold of 200 is used.

[0039] The aforementioned thresholds can be used as specific configurations in the preset mapping relationship between device type and amplitude threshold. By determining the amplitude threshold based on the mflVersion field, the strong and weak signal regions of MFL data can be determined to match the signal amplitude characteristics of different detection devices.

[0040] In one specific embodiment, the strong / weak signal region mask is a channel-window level mask. A channel-window level mask refers to generating a mask value for each sensor channel in each data block, indicating whether that sensor channel belongs to a strong signal region or a weak signal region within that data block.

[0041] Specifically, for the nth sensor channel in the jth data block, the amplitude index amp(j,n) corresponding to the sensor channel is obtained, and the amplitude index is compared with the amplitude threshold.

[0042] When amp(j,n)>threshold, the data in the j-th data block of this sensor channel is marked as a strong signal region. At this time, the corresponding mask value can be set to the first value, such as True, 1, or other identifiers used to represent strong signal regions.

[0043] When amp(j,n)≤threshold, the data in the j-th data block of this sensor channel is marked as a weak signal region. At this time, the corresponding mask value can be set to a second value, such as False, 0, or other identifiers used to represent weak signal regions.

[0044] If both the original data block and the denoised data block are N×W matrices, where N represents the number of sensor channels and W represents the window width, then the channel-window level mask can be represented as a vector of length N. The nth element of this vector is used to control the data fusion method of the nth sensor channel within the current window.

[0045] During the fusion process, for sensor channels whose mask values ​​represent strong signal regions, the data for that channel in the fusion data block is taken from the original data block; for sensor channels whose mask values ​​represent weak signal regions, the data for that channel in the fusion data block is taken from the denoised data block.

[0046] By using channel-window level masks, it is possible to distinguish the signal strength of different sensor channels within different windows while maintaining low computational complexity, thereby achieving differentiated noise reduction processing.

[0047] In one specific embodiment, the data block is input into a pre-trained MFL denoising model to obtain a denoised data block, including the following process.

[0048] First, the data blocks are normalized according to the type of detection equipment to obtain normalized data blocks. Different detection equipment may have different signal ranges, bias parameters, or data encoding methods. Through normalization, the MFL data collected by different equipment can be converted to a numerical range suitable for processing by the MFL denoising model.

[0049] In one implementation, the correspondence between device types and normalization parameters can be pre-configured. After the system determines the detection device type, it reads the corresponding normalization parameters and performs normalization processing on the data blocks.

[0050] Then, the normalized data block is input into the pre-trained MFL denoising model to obtain a normalized denoised data block. The MFL denoising model can be a neural network model, whose input is the normalized data block and whose output is a normalized denoised data block with the same dimensions as the input data block.

[0051] Subsequently, the normalized denoised data block is denormalized to obtain a denoised data block with the same amplitude scale as the original data block. The denormalization process corresponds to the aforementioned normalization process and is used to restore the model output to the amplitude scale of the original MFL data.

[0052] Furthermore, the pre-trained MFL denoising model can be trained using noisy MFL data blocks and corresponding target MFL data blocks. The model structure can adopt an encoder-decoder structure, or other network structures capable of denoising MFL data. This embodiment does not limit the specific number of layers, convolutional kernels, training framework, or hardware environment of the MFL denoising model.

[0053] In one specific embodiment, the overlapping region of adjacent fused data blocks is smoothly stitched together, including the following process. Let the width of the overlapping region of adjacent fused data blocks be overlap. The value at the i-th sampling point of the left fused data block in the overlapping region is denoted as left, and the value at the i-th sampling point of the right fused data block in the overlapping region is denoted as right, where i=0,1,...,overlap-1.

[0054] For the i-th sampling point in the overlapping region, the splicing value `mixed` is determined according to the following formula: mixed=((overlap-1-i)×left+(i+1)×right) / overlap Wherein, overlap represents the width of the overlapping region, left represents the value of the left fused data block at the i-th sampling point, and right represents the value of the right fused data block at the i-th sampling point.

[0055] As shown in the formula above, when i is small, the weight corresponding to left is larger, and the weight corresponding to right is smaller; as i gradually increases, the weight corresponding to left gradually decreases, and the weight corresponding to right gradually increases. This method allows for a smooth transition from the left to the right fused data block in the overlapping region.

[0056] For non-overlapping regions, data from the corresponding fused data blocks can be directly used for splicing. If the last window contains edge-filled data, after splicing all fused data blocks, the edge-filled data is trimmed to ensure that the denoised MFL data matrix has the same pipe axial sampling length as the original MFL data matrix.

[0057] In one specific embodiment, such as Figure 2 As shown, for each data block obtained by segmentation by the MFL data matrix, a strong and weak signal region mask is first generated based on the signal amplitude change of the data block, and then the original data block and the denoised data block are selectively fused based on the mask.

[0058] Specifically, the data block can be represented as a two-dimensional matrix, where each row of the matrix corresponds to a pipeline leakage magnetic flux detection sensor channel, and each column of the matrix corresponds to a pipeline axial sampling point. For the j-th data block, assuming it contains N sensor channels and W axial sampling points, the data block can be represented as block(j)∈R^(N×W).

[0059] Calculate the signal amplitude index ampArray: Calculate the signal amplitude index ampArray for each data block. The amplitude index is the difference between the maximum and minimum values ​​of each row in the data block. Specifically, for the nth sensor channel in the j-th data block, obtain the sequence of sampled values ​​of that sensor channel within the window range: X(j,n)={x1,x2,...,xW}. Where W is the number of sampling points of the data block in the axial sampling dimension of the pipe.

[0060] Determine the maximum sample value max(X(j,n)) and the minimum sample value min(X(j,n)) in the sample value sequence, and use the difference between the two as the amplitude index of the nth sensor channel in the jth data block: ampArray[n]=max(X(j,n))-min(X(j,n)).

[0061] Perform the above calculations on each sensor channel in the data block to obtain the amplitude index array corresponding to the data block: ampArray={ampArray[1],ampArray[2],...,ampArray[N]}. Where N is the number of sensor channels.

[0062] This amplitude index is used to characterize the degree of signal amplitude variation of the corresponding sensor channel within the current data block. Since pipeline defects may cause enhanced local leakage magnetic response or peak-valley variations, channel regions with large amplitude variations can be considered as strong signal regions suspected of containing defect signals; channel regions with small amplitude variations are usually dominated by background noise or weak signals.

[0063] Setting an adaptive threshold: An adaptive threshold is set based on the detection device type. Specifically, the `mflVersion` field in the MFL data file header is read, the detection device type is determined based on this field, and the corresponding amplitude threshold is determined based on the determination result. The recognition logic of the adaptive threshold is as follows: When the `mflVersion` field begins with the lowercase string "sy813", it is determined that the current MFL data originates from a SY813 device, and the adaptive threshold is set to 3000; when the `mflVersion` field does not begin with the lowercase string "sy813", it is determined that the current MFL data originates from a conventional device, including HD devices, MFL devices, ZY devices, or ZHT devices, and the adaptive threshold is set to 200. In other words, the adaptive threshold can be expressed as: `threshold = 3000` if `mflVersion` begins with "sy813"; `threshold = 200` if `mflVersion` does not begin with "sy813". By using the above method, different thresholds can be selected according to the signal amplitude range or data encoding characteristics of different detection devices, thus avoiding misjudgment of strong and weak signal areas when different devices share the same threshold.

[0064] Generate a mask for strong and weak signal regions: Compare the obtained ampArray with the determined threshold to generate a mask for strong and weak signal regions. Specifically, for the nth sensor channel in the jth data block: when ampArray[n] > threshold, set mask[n] to the first value, and mark the data of this sensor channel in the jth data block as a strong signal region; the strong signal region may contain defect signals, and the corresponding data in the original data block is retained during subsequent fusion.

[0065] When ampArray[n] ≤ threshold, mask[n] is set to the second value, and the data of this sensor channel in the j-th data block is marked as a weak signal region. The weak signal region is mainly composed of noise or weak background signal, and the corresponding data in the denoised data block is used in subsequent fusion. Here, mask can be a channel-window level mask, and its length is consistent with the number of sensor channels N. The n-th element in mask is used to indicate the fusion strategy of the n-th sensor channel in the current data block.

[0066] This can be expressed as: mask[n]=True if ampArray[n]>threshold; mask[n]=False if ampArray[n]≤threshold.

[0067] Mask-based adaptive fusion: After obtaining the mask, the original data block is input into the pre-trained MFL denoising model to obtain a denoised data block with the same dimension as the original data block. Subsequently, the original data block and the denoised data block are selectively fused according to the mask. Specifically, for the nth sensor channel in the jth data block: if mask[n]=True, the data of the nth sensor channel in the fused data block is set as the data of the nth sensor channel in the original data block; if mask[n]=False, the data of the nth sensor channel in the fused data block is set as the data of the nth sensor channel in the denoised data block.

[0068] Right now: fusedBlock[n,:]=rawBlock[n,:], if mask[n]=True; fusedBlock[n,:]=denoisedBlock[n,:], if mask[n]=False.

[0069] Wherein, rawBlock represents the original data block, denoisedBlock represents the denoised data block output by the MFL denoising model, and fusedBlock represents the fused data block.

[0070] By employing the aforementioned adaptive fusion strategy, for strong signal regions that may contain defect signals, retaining the original MFL detection data can reduce the impact of the denoising model on the smoothing of defect peaks, edges, or local morphology. For weak signal regions dominated by noise, using the denoising model output can suppress background noise and sensor noise. Thus, while reducing MFL data noise, the fidelity of defect signals can be improved.

[0071] In one specific embodiment, such as Figure 3 As shown, the MFL data matrix is ​​divided into sliding window segments along the sampling dimension of the pipeline axis. After denoising and adaptive fusion of each data block, the overlapping areas of adjacent windows are linearly weighted and smoothly spliced ​​to obtain the complete denoised MFL data matrix.

[0072] Specifically, the rows of the MFL data matrix correspond to sensor channels, and the columns correspond to sampling points along the pipe axis. A preset window width (lineWidth) and an overlap width (overlap) are set along the pipe axis sampling dimension, and the MFL data matrix is ​​divided into multiple data blocks using a sliding window method. Adjacent data blocks have an overlap area with a width of overlap.

[0073] For example, in one implementation, the window width (lineWidth) can be set to 200 sampling points, and the overlap width (overlap) can be set to 10 sampling points. In this case, window 1 can cover axial sampling points 0 to 199, window 2 can cover axial sampling points 190 to 389, and window 3 can cover axial sampling points 380 to 579. The overlap between window 1 and window 2 is axial sampling points 190 to 199, and the overlap between window 2 and window 3 is axial sampling points 380 to 389. It should be understood that the above values ​​are only one specific example, and the window width and overlap width can be adjusted according to the MFL denoising model input size, pipeline axial sampling interval, or computational resources.

[0074] After performing MFL denoising model inference and adaptive fusion based on strong and weak signal region masks for each window data block, multiple fused data blocks are obtained. For two adjacent fused data blocks, the corresponding data in the overlapping area of ​​the left fused data block is denoted as 'left', and the corresponding data in the overlapping area of ​​the right fused data block is denoted as 'right'. To avoid signal abrupt changes at the window boundaries caused by direct splicing, linear weighted smooth splicing is used for the overlapping areas of adjacent windows.

[0075] Linear weighted smoothing is applied to the overlapping region: Let the width of the overlapping region between adjacent windows be *overlap*. For the *i*th sampling point in the overlapping region, *i* = 0, 1, ..., *overlap-1*, the stitching value *mixed* is determined according to the following formula: mixed = ((overlap-1-i)×left+(i+1)×right) / overlap; where left represents the fused data value of the left window at the i-th sampling point in the overlapping area, right represents the fused data value of the right window at the i-th sampling point in the overlapping area, and mixed represents the data value of that sampling point in the stitching result.

[0076] As shown in the formula above, at the beginning of the overlapping region, the left window has a larger weight, while the right window has a smaller weight. As i gradually increases, the weight of the left window gradually decreases, while the weight of the right window gradually increases. This achieves a smooth transition from the fusion result of the left window to the fusion result of the right window within the overlapping region, reducing boundary jumps caused by direct splicing.

[0077] In a specific example, when overlap=10: When i=0, mixed=(9×left[0]+1×right[0]) / 10; When i=5, mixed=(4×left[5]+6×right[5]) / 10; When i=9, mixed=(0×left[9]+10×right[9]) / 10; Left weight: 0.9 → 0.0; Right weight: 0.1 → 1.0. Therefore, the splicing result gradually transitions from being closer to the output of the left window to being closer to the output of the right window.

[0078] For non-overlapping regions, data is directly stitched together using the data from the corresponding fused data blocks in adjacent windows. Specifically, for the non-overlapping portion of the left fused data block located before the overlapping region, the data from the left fused data block is directly retained; for the non-overlapping portion of the right fused data block located after the overlapping region, the data from the right fused data block is directly retained. This avoids unnecessary weighting of regions that do not require smooth transition, while preserving the fusion results of non-overlapping regions in each window.

[0079] Taking window 1 and window 2 as examples, window 1 covers the 0th to 199th sampling points, window 2 covers the 190th to 389th sampling points, and the overlap is 10. Then the fusion result of window 1 can be directly used for the 0th to 189th sampling points, the linear weighted smooth stitching result is used for the 190th to 199th sampling points, and the fusion result of window 2 can be directly used for the 200th to 389th sampling points.

[0080] Trim edge-filled data: If edge filling was performed for the last window in the aforementioned data segmentation step, then after all the fused data blocks are spliced, the filling portion generated by edge filling is trimmed off, so that the length of the final denoised MFL data matrix in the pipe axial sampling dimension is consistent with the original MFL data matrix.

[0081] Specifically, when the number of remaining sampling points corresponding to the last window is less than the preset window width, edge padding can be used to fill the last window to the preset window width. This padding data is only used to meet the model input size requirements and does not correspond to actual sampling points. Therefore, after completing window fusion and stitching, this padding data should be trimmed from the end of the stitching result to restore the original data length.

[0082] By using the above sliding window segmentation and stitching method, while adapting to the input size of the MFL denoising model, the linear weighted smooth stitching of the overlapping area can reduce the discontinuity at the window boundary and improve the continuity of the long-distance MFL data denoising results in the pipeline axial sampling dimension.

[0083] In one specific embodiment, the pre-trained MFL denoising model can be trained using supervised learning. The MFL denoising model receives noisy MFL data blocks and outputs denoised data blocks corresponding to the noisy MFL data blocks. The training method includes dataset construction, data normalization, model training, and model evaluation.

[0084] Dataset Construction: Noisy MFL detection data is collected as training input. This noisy MFL detection data can originate from actual data collected by pipeline magnetic flux leakage (MFL) detection equipment. The data can be represented as an MFL data matrix, which includes sensor channel dimensions and pipeline axial sampling dimensions. When constructing training samples, the noisy MFL detection data can be divided into blocks along the pipeline axial sampling dimension, resulting in multiple noisy MFL data blocks. Each noisy MFL data block can serve as the training input for the MFL denoising model.

[0085] Furthermore, a target MFL data block corresponding to the noisy MFL data block is generated, and the target MFL data block is used as the training target. The target MFL data block can be obtained through manual annotation, manual correction, simulation data generation, or other methods.

[0086] In one implementation, noisy MFL data can be manually corrected based on defect areas, noise areas, and background areas in the MFL detection data to obtain the corresponding target MFL data block. In another implementation, simulated magnetic flux leakage response data can be generated based on pipeline defect geometry parameters, magnetization conditions, sensor arrangement, and sampling parameters, and the simulated low-noise or noise-free magnetic flux leakage response data can be used as the target MFL data block.

[0087] Therefore, data pairs are constructed between noisy MFL data blocks and target MFL data blocks to form a supervised learning dataset. Each data pair can be represented as (inputBlock, targetBlock), where inputBlock represents the noisy MFL data block and targetBlock represents the target MFL data block corresponding to the noisy MFL data block. The supervised learning dataset can be further divided into a training set, a validation set, and a test set. The training set is used to update model parameters, the validation set is used to select model weights or adjust the training process, and the test set is used to evaluate the denoising effect of the model.

[0088] Data normalization: Before model training, the noisy MFL data blocks and their corresponding target MFL data blocks are normalized according to the detection device type, converting the data collected by different detection devices to a numerical range suitable for model training. Different detection devices may have different signal ranges, bias parameters, or data encoding methods; therefore, different normalization methods can be used for different device types. The following normalization process is performed based on the detection device type: When the detection device type is HD or MFL, the data ds is normalized according to the following formula: ds_norm = ds / 4096. Where 4096 represents the measurement range parameter corresponding to this type of device.

[0089] When the detection equipment type is SY, the data ds is normalized according to the following formula: ds_norm=(ds+32768-mean(ds)) / 65535. Where 65535 represents the range parameter corresponding to this type of equipment, 32768 represents the bias parameter, and mean(ds) represents the mean of the data ds.

[0090] When the detection equipment type is ZY, the data ds is normalized according to the following formula: ds_norm=(ds+1024) / 4096. Where 4096 represents the range parameter corresponding to this type of equipment, and 1024 represents the bias parameter.

[0091] When the detection equipment type is ZHT equipment, the data ds is normalized according to the following formula: ds_norm=(ds+1024) / 2700. Where 2700 represents the range parameter corresponding to this type of equipment, and 1024 represents the bias parameter.

[0092] The above normalization process reduces the data distribution differences caused by variations in range, bias, or encoding methods between different detection devices, enabling the MFL denoising model to learn the mapping relationship between noisy MFL data blocks and target MFL data blocks within a unified numerical range. During the model inference phase, the same normalization rules as in the training phase can be used to normalize the input data blocks, and the corresponding inverse normalization process can be performed after obtaining the model output, thereby obtaining denoised data blocks with the same amplitude scale as the original MFL data.

[0093] Model Training: After completing dataset construction and normalization, the MFL denoising model is trained using a supervised learning dataset. The MFL denoising model can employ an encoder-decoder structure. The encoder extracts spatial or channel features from noisy MFL data blocks, and the decoder reconstructs the denoised MFL data blocks based on the extracted features. The model input is the normalized noisy MFL data block, and the model output is the normalized denoised data block with the same dimensions as the input.

[0094] During training, the mean squared error loss function (MSE) or the mean absolute error loss function (MAE) can be used to measure the difference between the model output and the target MFL data block.

[0095] When the mean squared error loss function is used, the loss function can be expressed as: Loss_MSE=(1 / K)Σ(y_pred-y_true)^2.

[0096] When the mean absolute error loss function is used, the loss function can be expressed as: Loss_MAE=(1 / K)Σ|y_pred-y_true|.

[0097] Where y_pred represents the normalized denoised data block output by the MFL denoising model, y_true represents the corresponding normalized target MFL data block, and K represents the number of data points involved in the loss calculation.

[0098] The Adam optimizer can be used to iteratively update the model parameters, with a learning rate set to 0.001. The batch size can be dynamically adjusted based on GPU memory, for example, from 32 to 128. The number of training epochs can be determined based on model convergence, for example, from 50 to 200 epochs, or training can be stopped when the loss on the validation set no longer decreases.

[0099] During training, the validation loss can be calculated using the validation set after each round or a preset training interval, and the model weights with the smallest validation loss can be saved as the final model weights. By saving the model weights with the smallest validation loss, the risk of model overfitting can be reduced, and the model's generalization ability on MFL data that was not used in training can be improved.

[0100] To prevent the training process from consuming all GPU resources, GPU memory can be limited. For example, GPU memory usage can be limited to half the total video memory capacity minus 1.5GB to reserve some video memory for system or other processes. It should be understood that the GPU memory limitation method is only one engineering implementation method and does not constitute a limitation on the scope of protection of this invention.

[0101] After training, the trained model weights and structure can be saved as a model file. In one specific implementation, the model can be packaged in tar.gz format, for example, the package file name is model.noiseRemoval.tgz. It should be understood that the model file format and file name are only one specific implementation method, and the model can also be saved in other formats that can be loaded by the inference program.

[0102] Model Evaluation: After model training is complete, the denoising performance of the MFL denoising model is evaluated using a test set. The data in the test set is not used for model parameter training and is used to evaluate the model's generalization performance on unseen MFL data.

[0103] In one implementation, noisy MFL data blocks from the test set can be input into a trained MFL denoising model to obtain denoised data blocks output by the model. These denoised data blocks are then compared with corresponding target MFL data blocks. Evaluation metrics may include signal-to-noise ratio (SNR) improvement and structural similarity. The SNR improvement is used to evaluate the degree of signal quality improvement before and after denoising. The SNR of the noisy MFL data block relative to the target MFL data block, and the SNR of the denoised data block relative to the target MFL data block, can be calculated separately, and the difference between the two can be used as the SNR improvement value.

[0104] Structural similarity is used to evaluate the consistency in structural morphology between the denoised data block and the target MFL data block. For MFL detection data, structural similarity reflects whether the denoised data retains the spatial structure, local peak-valley morphology, or morphological characteristics of the original magnetic flux leakage response. In addition to signal-to-noise ratio improvement and structural similarity, root mean square error, mean absolute error, defect region amplitude retention rate, or subsequent defect detection accuracy can be used as auxiliary evaluation indicators. Specific evaluation indicators can be selected based on the actual detection task requirements.

[0105] The model evaluation described above determines whether the MFL denoising model meets the requirements for subsequent adaptive fusion denoising processes. If the evaluation is successful, the MFL denoising model can be used as a pre-trained MFL denoising model to generate denoised data blocks in the aforementioned adaptive denoising method for pipeline magnetic flux leakage detection data.

[0106] In one specific embodiment, such as Figure 4 As shown, to further reduce the window boundary effect caused by the single sliding window partitioning method, a dual-path prediction denoising method can be used to process the MFL data matrix. The dual-path prediction denoising method includes a first-path prediction, a second-path prediction, and the fusion of the two prediction results.

[0107] First-path sliding window denoising prediction: The first path divides the original MFL data into sliding windows along the pipe axis sampling dimension with a fixed step size lineWidth, and performs denoising prediction on the data blocks corresponding to each window to obtain the first-path prediction result dsNew1.

[0108] Specifically, assuming a window width of lineWidth, the first prediction path starts from the initial sampling point along the axis of the original MFL data and sequentially extracts multiple window data blocks. For example, the first window may include: W1, covering sampling points 0 to 199; W2, covering sampling points 190 to 389; W3, covering sampling points 380 to 579; and W4, covering sampling points 570 to 769.

[0109] Each window's data block is input into the MFL denoising model for denoising prediction, or processed according to the aforementioned adaptive denoising process, including amplitude index calculation, threshold determination, mask generation, denoising inference, and adaptive fusion. The processing results of each window are concatenated along the sampling dimension of the pipeline axis to obtain the first prediction result dsNew1. In the first prediction result dsNew1, the central region of each window usually has good prediction stability, while the regions at both ends of the window may be affected by window boundary effects.

[0110] Second-path offset sliding window denoising prediction: The second path divides the original MFL data into sliding windows along the pipe axis sampling dimension with lineWidth / 2 as the starting offset, and performs denoising prediction on the data blocks corresponding to each window to obtain the second-path prediction result dsNew2.

[0111] Specifically, the second prediction does not start from the initial sampling point along the axis of the original MFL data, but rather begins windowing from a position offset by lineWidth / 2. For example, when lineWidth is 200, the starting offset of the second prediction is 100 sampling points, and the second window can include: W1', covering sampling points 100 to 299; W2', covering sampling points 290 to 489; W3', covering sampling points 480 to 679; and W4', covering sampling points 670 to 869.

[0112] Each offset window data block is input into the MFL denoising model for denoising prediction, or processed according to the aforementioned adaptive denoising process. The processing results of each offset window are concatenated along the sampling dimension of the pipe axis to obtain the second prediction result dsNew2. Since the second window is offset by half a window width relative to the first window, the boundary regions in the first window may be located in or near the center region in the second window. Therefore, the second prediction result can compensate for the boundary regions in the first prediction result.

[0113] Fusion of two prediction results: After obtaining the first prediction result dsNew1 and the second prediction result dsNew2, the two prediction results are fused to obtain the final prediction result dsNew3.

[0114] The first half of the second prediction result dsNew2 and the second half of the first prediction result dsNew1 are concatenated to obtain the final prediction result dsNew3. That is, within the axial interval corresponding to each window, for the first lineWidth / 2 sampling points, the data at the corresponding position in the second prediction result is used; for the last lineWidth / 2 sampling points, the data at the corresponding position in the first prediction result is used. This fusion method can be expressed as: dsNew3 = concatenate(dsNew2[:, :lineWidth / 2], dsNew1[:, lineWidth / 2:]). Here, concatenate indicates concatenation along the sampling dimension of the pipe axis, dsNew2[:, :lineWidth / 2] represents the first half of the second prediction result, and dsNew1[:, lineWidth / 2:] represents the second half of the first prediction result.

[0115] In another implementation, cyclic fusion can also be performed according to window segments. For each axial interval with lineWidth as the period, the first half of the corresponding interval in the second prediction result and the second half of the corresponding interval in the first prediction result are spliced ​​together to form the fusion result of the corresponding interval; the fusion results of multiple intervals are spliced ​​together sequentially to obtain the complete dsNew3.

[0116] In another implementation, dsNew1 and dsNew2 can be weighted and fused based on the distance of each sampling point from the center of its window. Prediction results closer to the window center are assigned higher weights, while those closer to the window boundary are assigned lower weights. This method can also reduce the impact of unstable prediction results at the window boundary on the final result.

[0117] In one specific embodiment, the denoised MFL data matrix obtained by the present invention is input into a feature extractor, which extracts spatial features. These spatial features are then input into a depth estimation model for defect depth prediction. This cascaded architecture of denoising before estimation reduces the interference of noise on defect depth prediction and improves the stability of the depth estimation results.

[0118] In one specific embodiment, the denoised MFL data matrix obtained by the present invention is input into a semantic segmentation model, which outputs pixel-level classification results of defect regions, including positive defects, negative defects, and background. By performing adaptive denoising before semantic segmentation, missegmentation caused by noise can be reduced, while preserving the edge and amplitude features of strong signal regions suspected of defects, thereby improving the accuracy of defect region identification.

[0119] Based on the same inventive concept, this application also provides a method. The solution provided by this method is similar to the solution described in the above system. Therefore, the specific limitations in one or more method embodiments provided below can be found in the limitations described above, and will not be repeated here.

[0120] In one exemplary embodiment, such as Figure 5As shown, an adaptive denoising system for pipeline magnetic flux leakage detection data is provided, comprising: The data segmentation module is used to acquire the MFL data matrix formed by the pipeline magnetic flux leakage detection sensor, and to divide the MFL data matrix into multiple data blocks with overlapping regions along the pipeline axial sampling dimension. The amplitude analysis module is used to determine the amplitude index based on the amplitude changes of the sampled values ​​of the sensor channels in each data block, and to generate a mask for strong and weak signal regions. The denoising inference module is used to input the data block into the pre-trained MFL denoising model to obtain the denoised data block. An adaptive fusion module is used to fuse the original data block and the denoised data block according to the mask, wherein the strong signal region selects the corresponding data in the original data block, and the weak signal region selects the corresponding data in the denoised data block. The smooth stitching module is used to smoothly stitch together the overlapping areas of adjacent fused data blocks to obtain a denoised MFL data matrix.

[0121] In one specific embodiment, such as Figure 7 The illustration shows a specific implementation of an adaptive noise reduction system for pipeline magnetic flux leakage detection data. This embodiment is used to illustrate one implementation process of the method of the present invention in a computer device and should not be construed as limiting the scope of protection of the present invention.

[0122] I. Implementation Environment: In this embodiment, the denoising system can be deployed in a computer device with graphics processing capabilities. The computer device includes a processor, memory, and a graphics processor. As an example, the hardware environment can use an NVIDIA GeForce RTX 2080 Ti GPU, the deep learning framework can use TensorFlow and Keras, and the programming language can use Python 3.x.

[0123] The denoising system loads a pre-trained MFL denoising model, which can be stored as a model file, for example, model.noiseRemoval.tgz. This model file can be a tar.gz compressed archive, which, after decompression, is in TensorFlow / Keras SavedModel format. The system loads this model file during denoising inference and sets the model to inference mode. In inference mode, model parameters are no longer updated; for example, the model's trainable attribute can be set to False.

[0124] It should be noted that the GPU model, deep learning framework, programming language, and model file format mentioned above are only one specific implementation. The implementation of this invention is not limited to specific hardware, specific deep learning frameworks, specific programming languages, or specific model file formats. As long as a pre-trained MFL denoising model can be loaded and MFL data denoising inference can be performed, the technical solution of this invention can be achieved.

[0125] II. Denoising Model Structure: In this embodiment, the MFL denoising model adopts an encoder-decoder structure. The encoder is used to extract signal features from noisy MFL data blocks, and the decoder is used to reconstruct the denoised MFL data blocks based on the extracted features. The model file is packaged as model.noiseRemoval.tgz (tar.gz format), and after decompression, it is in TensorFlow / KerasSavedModel format, as shown in the table below:

[0126] Table 1. Denoising Model The model structure parameters described above are only a typical embodiment. In actual implementation, the number of encoding layers, decoding layers, filters, convolution kernel size, activation function type, and bottleneck layer unit number can all be adjusted according to the training data scale, detection device type, MFL data dimension, and hardware conditions. As long as the model can receive noisy MFL data blocks and output corresponding denoised data blocks, it can be used in the adaptive denoising process of this invention.

[0127] III. Core Parameter Configuration: In this embodiment, window parameters and amplitude thresholds are configured according to the type of detection device, as shown in the table below:

[0128] Table 2. Core Parameters The parameters described above are preferred configurations in specific embodiments and do not constitute a limitation on the scope of protection of this invention. In practical applications, lineWidth, overlap, and threshold can be adjusted according to the model input dimensions, pipe axial sampling interval, detection equipment range, equipment offset parameters, or historical data amplitude distribution.

[0129] IV. Smooth stitching algorithm and edge filling: In this embodiment, the system performs a sliding window segmentation of the MFL data matrix along the sampling dimension of the pipeline axis to obtain multiple data blocks. After MFL denoising model inference and adaptive fusion, each data block yields a corresponding fused data block. To restore the complete denoised MFL data matrix from multiple fused data blocks, the system concatenates adjacent fused data blocks.

[0130] For adjacent merged data blocks, let the left merged data block be 'left', the right merged data block be 'right', and the width of the overlapping region be 'overlap'. The non-overlapping portion of the left merged data block that is before the overlapping region is denoted as 'netLeft', and the non-overlapping portion of the right merged data block that is after the overlapping region is denoted as 'netRight'.

[0131] The smooth stitching algorithm can include the following steps: First, take the non-overlapping portion of the left fused data block excluding the last overlapping sampling points to obtain netLeft; then, perform linear weighted mixing on each sampling point in the overlapping region. For the i-th sampling point in the overlapping region, i=0,1,...,overlap-1, calculate the stitching value mixed according to the following formula: mixed=((overlap-1-i)×left+(i+1)×right) / overlap; where left represents the value of the left fused data block at the i-th sampling point in the overlapping region, and right represents the value of the right fused data block at the i-th sampling point in the overlapping region; then, stitch each mixed value sequentially along the sampling dimension of the pipe axis to netLeft; finally, stitch the non-overlapping portion netRight of the right fused data block excluding the first overlapping sampling points to the smooth mixing result of the overlapping region to obtain the stitched data.

[0132] The above process can be described as follows: for non-overlapping regions, data from the corresponding fused data block is directly used; for overlapping regions, data is gradually transitioned from the left fused data block to the right fused data block using linear weights. As i increases, the weights corresponding to the left fused data block gradually decrease, while the weights corresponding to the right fused data block gradually increase, thereby reducing abrupt changes at the window boundaries.

[0133] When the remaining data in the last window is less than `lineWidth`, edge padding can be used to fill the last window to `lineWidth`. Edge padding can use array edge values, that is, repeating the value of the last valid sample point. For example, it can be done as follows: `np.pad(ds, ((0,0), (0, lineWidth-overlap-leftover)), mode='edge')`. Here, `ds` represents the data to be filled, `leftover` represents the number of remaining valid sample points in the last window, and `mode='edge'` indicates that edge values ​​are used for padding.

[0134] Since the data generated by edge padding is not the actual sampled data, after all the fused data blocks are spliced, the system trims off the padding portion generated by edge padding, so that the output denoised MFL data matrix is ​​restored to the same axial sampling length as the original MFL data matrix.

[0135] V. Dual-path prediction fusion: In this embodiment, to further reduce the boundary effect caused by the single window partitioning method, a dual-path prediction fusion method can be adopted.

[0136] The first prediction starts at offset 0, and a sliding window is used to segment the data along the pipe axis according to the sampling dimension of the window width lineWidth. Denoising prediction and adaptive fusion are then performed on the data blocks corresponding to each window to obtain the first prediction result dsNew1.

[0137] The second prediction uses lineWidth / 2 as the starting offset, and performs sliding window segmentation along the pipe axis sampling dimension according to the window width lineWidth. Then, it performs denoising prediction and adaptive fusion on the data blocks corresponding to each window to obtain the second prediction result dsNew2.

[0138] Because the second-path window is offset by half the window width relative to the first-path window, sampling points located at the window edge in the first-path prediction result may be located near the window center in the second-path prediction process. Therefore, the second-path prediction result can be used to compensate for boundary regions in the first-path prediction result.

[0139] By using dual-path prediction fusion, each sampling point can be covered by the central region of at least one window as much as possible, reducing the impact of prediction quality degradation at the window boundaries on the final result and improving the axial continuity of the denoised MFL data.

[0140] VI. Specific Application Cases: In one application case, the method is used for noise reduction of conventional pipeline magnetic flux leakage (MFL) detection data. The detection device can be an HD508, the pipe outer diameter can be 508 mm, and the original MFL data dimension can be 40 channels × 50,000 sampling points. The system uses window parameters lineWidth=200 and overlap=10, and an amplitude threshold of threshold=200 based on the device type. After processing, noise in weak signal areas is suppressed, while suspected defect signals in strong signal areas are preserved.

[0141] In another application case, the method is used for denoising MFL data acquired by the SY813 device. The inspection device can be the SY813, the pipe outer diameter can be 813mm, and the original MFL data dimension can be 80 channels × 100,000 sampling points. The system uses window parameters lineWidth=200, overlap=10, and an amplitude threshold of threshold=3000 based on the device type. After processing, the background noise of the denoised MFL data is reduced, and subsequent defect detection can be based on more stable data input.

[0142] The above-mentioned testing equipment models, pipe diameters, and data dimensions are only used to illustrate the application of the method of the present invention and are not intended to limit the scope of protection of the present invention.

[0143] VII. Cascading Applications: In one implementation, the denoised MFL data can be further input into a semantic segmentation model. Specifically, the system first performs the adaptive denoising process described in this embodiment to obtain a denoised MFL data matrix; then, the denoised MFL data matrix is ​​input into a semantic segmentation model, such as model.segmentation.tgz; the semantic segmentation model outputs pixel-level classification results for the defect region. The pixel-level classification results can include positive defects, negative defects, and background. Since the data input to the semantic segmentation model has undergone adaptive denoising, missegmentation caused by noise can be reduced, while preserving defect edges and amplitude features in strong signal regions.

[0144] In another implementation, the denoised MFL data can be further input into a feature extractor, and the features output by the feature extractor can be input into a depth estimation model. Specifically, the system first performs adaptive denoising processing to obtain a denoised MFL data matrix; then, the denoised MFL data matrix is ​​input into the feature extractor, which extracts spatial features related to defects; finally, the spatial features are input into the depth estimation model to predict the depth of pipeline defects. Because the denoised MFL data reduces noise interference, the stability of the depth estimation results can be improved.

[0145] It should be noted that the specific network structure, training method, and output format of the semantic segmentation model, feature extractor, and depth estimation model can be set according to the actual detection task. This embodiment is not limited to a specific model structure.

[0146] VIII. Comparison of experimental data: In an exemplary test, the denoising method provided in this embodiment can be compared with no denoising, moving average filtering, wavelet thresholding denoising, and adaptive median filtering. Evaluation metrics may include signal-to-noise ratio improvement, root mean square error reduction ratio, and defect retention rate.

[0147] The noise reduction effects are compared in the table below:

[0148] Table 3. Noise Reduction Effect The impact on downstream tasks is shown in the table below:

[0149] Table 4. Impact on downstream tasks IX. Computational Complexity Analysis: In this embodiment, the MFL data matrix is ​​assumed to include N sensor channels and M axial sampling points, with a window width of W, a number of windows of K, and a model parameter scale of F.

[0150] The time complexity of sliding window segmentation can be expressed as O(N×M). Amplitude analysis requires calculating the maximum and minimum values ​​for each sensor channel in each data block, and its time complexity can be expressed as O(N×W×K). The time complexity of model inference is related to the size of the input data and the size of the model parameters, and can be expressed as O(N×W×F). The smooth stitching process requires sampling along the pipe axis to recover the complete data, and its time complexity can be expressed as O(N×M).

[0151] In a specific test environment, the total time to complete denoising processing for 100,000 axial sample points of MFL data on an NVIDIA GeForce RTX 2080 Ti GPU is approximately 2.5 seconds. In an environment without a GPU, inference can also be performed using the CPU; for example, denoising 100,000 axial sample points takes approximately 25 seconds.

[0152] In terms of space complexity, the encoder-decoder model can have approximately 2MB of parameters, memory usage during inference can be approximately 50MB, and peak GPU memory usage during batch inference can be approximately 200MB. The above values ​​are only examples of a specific implementation; actual time consumption, memory usage, and GPU memory usage will vary depending on the model structure, data size, hardware conditions, and batch processing settings.

[0153] In one deployment method, the minimum configuration can use an Intel i5 processor, 8GB of memory, and an NVIDIA GTX 1060 GPU; the recommended configuration can use an Intel i7 processor, 16GB of memory, and an NVIDIA RTX 2080 Ti GPU. It should be noted that the above hardware configuration is merely an example of how to implement this embodiment and does not constitute a limitation on the scope of protection of this invention.

[0154] This invention uses adaptive signal amplitude judgment to retain original data in strong signal areas (defect areas) and denoise output in weak signal areas, avoiding excessive smoothing of defect details. The denoising threshold is automatically adjusted according to the type of detection equipment to adapt to the signal characteristics of different devices. An overlapping window + linear weighted smoothing stitching strategy is adopted to effectively eliminate discontinuities at window boundaries. The dual-path prediction method further improves the stitching quality. The entire process from raw noisy data to denoised results is automated, eliminating the need for manual setting of filtering parameters. The denoising module can be used independently or cascaded with modules such as feature extraction, depth estimation, and semantic segmentation, allowing for flexible combinations to adapt to different application scenarios.

[0155] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 7 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and databases. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media to run. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection.

[0156] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0157] In one exemplary embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0158] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0159] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0160] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (Read-Only Memory). Memory includes ROM, magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0161] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0162] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0163] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. An adaptive noise reduction method for pipeline magnetic flux leakage detection data, characterized in that, The methods include: Obtain the MFL data matrix formed by the pipeline magnetic flux leakage detection sensor, wherein the MFL data matrix includes the sensor channel dimension and the pipeline axial sampling dimension; The MFL data matrix is ​​divided into multiple data blocks with overlapping regions along the sampling dimension of the pipe axis; The amplitude index is determined based on the amplitude variation of the sampled values ​​of the sensor channels in each data block; the amplitude threshold is determined based on the type of detection equipment; and a strong / weak signal region mask is generated based on the amplitude index and amplitude threshold. The data block is input into a pre-trained MFL denoising model to obtain a denoised data block; the original data block and the denoised data block are then fused according to the mask. Among them, the strong signal region is selected from the corresponding data in the original data block, and the weak signal region is selected from the corresponding data in the denoised data block; The overlapping areas of adjacent fused data blocks are smoothly stitched together to obtain the denoised MFL data matrix.

2. The method according to claim 1, characterized in that, The rows of the MFL data matrix correspond to the sensor channels of the pipeline magnetic flux leakage detection sensor, and the columns correspond to the axial sampling points of the pipeline. The MFL data matrix is ​​divided into multiple data blocks with overlapping regions along the axial sampling dimension of the pipeline, including: Sliding windows are divided along the axial sampling dimension of the pipeline according to a preset window width, and adjacent windows have a preset overlap width. When the data length corresponding to the last window is less than the preset window width, the last window is edge-filled, and the fill data generated by the edge filling is cropped after smooth splicing.

3. The method according to claim 1, characterized in that, The amplitude index is determined based on the amplitude variation of the sampled values ​​from the sensor channels in each data block, including: For the nth sensor channel in the jth data block, determine the maximum and minimum sample values ​​of the sensor channel within the jth data block, and determine the difference between the maximum and minimum sample values ​​as the amplitude index of the nth sensor channel within the jth data block.

4. The method according to claim 1, characterized in that, The amplitude threshold is determined based on the type of detection equipment, including: Read the device identifier field in the MFL data file header; determine the detection device type based on the device identifier field; determine the amplitude threshold based on the preset mapping relationship between device type and amplitude threshold.

5. The method according to claim 4, characterized in that, The device identification field includes the mflVersion field; when the mflVersion field represents a SY813 device, the amplitude threshold is 3000; when the mflVersion field represents an HD device, MFL device, ZY device, or ZHT device, the amplitude threshold is 200.

6. The method according to claim 3, characterized in that, The strong and weak signal region mask is a channel-window level mask; when the amplitude index corresponding to the nth sensor channel in the jth data block is greater than the amplitude threshold, the data of that sensor channel in the jth data block is marked as a strong signal region; when the amplitude index is not greater than the amplitude threshold, the data of that sensor channel in the jth data block is marked as a weak signal region.

7. The method according to claim 1, characterized in that, The data block is input into a pre-trained MFL denoising model to obtain a denoised data block, including: The data block is normalized according to the type of detection equipment to obtain a normalized data block; The normalized data block is input into the pre-trained MFL denoising model to obtain the normalized denoised data block. The normalized denoised data block is denormalized to obtain a denoised data block with the same amplitude scale as the original data block.

8. The method according to claim 1, characterized in that, Smoothly stitching together the overlapping areas of adjacent fused data blocks, including: Let the width of the overlapping region between adjacent fused data blocks be denoted as 'overlap'. For the i-th sampling point in the overlapping region, the splicing value 'mixed' is determined according to the following formula: mixed=((overlap-1-i)×left+(i+1)×right) / overlap; Where i = 0, 1, ..., overlap-1, left represents the value of the left fused data block at the i-th sampling point, and right represents the value of the right fused data block at the i-th sampling point.

9. An adaptive noise reduction system for pipeline magnetic flux leakage detection data, characterized in that, include: The data segmentation module is used to acquire the MFL data matrix formed by the pipeline magnetic flux leakage detection sensor, and to divide the MFL data matrix into multiple data blocks with overlapping regions along the pipeline axial sampling dimension. The amplitude analysis module is used to determine the amplitude index based on the amplitude changes of the sampled values ​​of the sensor channels in each data block; determine the amplitude threshold based on the type of detection equipment; and generate a strong and weak signal region mask based on the amplitude index and amplitude threshold. The denoising inference module is used to input the data block into the pre-trained MFL denoising model to obtain the denoised data block. An adaptive fusion module is used to fuse the original data block and the denoised data block according to the mask, wherein the strong signal region selects the corresponding data in the original data block, and the weak signal region selects the corresponding data in the denoised data block. The smooth stitching module is used to smoothly stitch together the overlapping areas of adjacent fused data blocks to obtain a denoised MFL data matrix.

10. A computer-readable medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 8.

Images