Pipeline ray flaw detection defect recognition method and system based on image processing

Abstract

The present application relates to the technical field of defect detection, and particularly relates to a pipeline radiographic inspection defect recognition method and system based on image processing, which comprises the following steps: acquiring an image training set and a defect real label frame, extracting the size of the defect real label frame for logarithmic normalization, converting an angle into a sine component and a cosine component, constituting a four-dimensional feature vector, clustering the four-dimensional feature vector by using a Gaussian mixture model, in the training stage, using a defect probability heat map probability as a weight to construct a comprehensive matching degree index covering an intersection-over-union ratio and a scale angle difference corresponding reverse similarity function value, implementing differentiated weight distribution on different probability areas to divide positive and negative samples, improving the rationality of sample matching, and helping the model to converge, converting the Gaussian component identification of the anchor frame template into an embedded vector and candidate area feature fusion, supplementing prior morphological features for a classification regression network, and improving the accuracy of pipeline defect class recognition and position positioning.

Description

Technical Field

[0001] This invention belongs to the field of defect detection technology, and in particular relates to a method and system for identifying defects in pipeline radiographic testing based on image processing. Background Technology

[0002] Pipelines are crucial infrastructure for energy transmission and industrial production, and their operational safety directly impacts production continuity and accident prevention. Under long-term operating conditions such as corrosion, pressure fluctuations, temperature changes, and erosion by complex media, pipelines are prone to damage defects such as cracks, corrosion, and porosity. Therefore, regular radiographic testing is an important means of ensuring safe pipeline operation. In recent years, deep learning-based target detection technologies have been increasingly applied to pipeline flaw detection, especially two-stage detection algorithms relying on region proposal networks, which offer advantages due to their high detection accuracy. However, pipeline radiographic images often have complex backgrounds, and defects frequently exhibit characteristics such as large scale spans, extreme aspect ratios, and random orientation angles. These defects are easily confused with pipe wall textures, weld structures, and imaging noise, leading to problems such as low recognition rates, high false negative rates, and inaccurate boundary localization in general detection models.

[0003] In related technologies, for example, Chinese patent application document with publication number CN111091538A discloses an automatic identification and defect detection method and device for pipeline welds. It uses a training sample set to train weld extraction, text recognition and defect localization respectively. By acquiring radiographic images of welds and inputting the preprocessed images into a well-built and trained deep neural network model, it obtains each weld region in the image and its corresponding category, center position C, confidence level and identifies the width and height of the weld. This realizes a convenient, feasible, accurate and efficient automated weld identification and defect detection technology.

[0004] Existing methods typically generate anchor frame scales and aspect ratios to fit the dataset through clustering, or add angle parameters to the anchor frames to support rotating target detection, and use a fixed-step, dense sliding window to deploy candidate regions on the feature map. However, traditional anchor frame clustering relies heavily on geometric dimensions and does not adequately model the joint statistical relationship between defect scale, aspect ratio, and angle. It also lacks a measure of the uncertainty of anchor frame morphology, resulting in insufficient matching between the anchor frame template and the actual defect morphology. At the same time, existing anchor frame deployment often uses uniform laying across the entire image without considering the spatial probability distribution of defects, and it is difficult to adaptively adjust the deployment density according to the stability of different defect morphologies. This results in redundant computation of a large number of background areas and may reduce the coverage accuracy of specific stable defect morphologies. In addition, the assignment of positive and negative samples usually relies on a single intersection-union threshold, without integrating local probability priors and anchor frame clustering prior information, which limits the feature representation ability of candidate regions and thus affects the accuracy of defect classification and boundary regression. Summary of the Invention

[0005] To address the problems of insufficient matching between existing anchor frame templates and the actual shape of defects, redundant anchor frame layout with low coverage accuracy, coarse positive and negative sample allocation, and limited accuracy of defect classification and boundary regression, this invention provides a pipeline radiographic flaw detection defect identification method and system based on image processing.

[0006] In the first aspect, the present invention provides a pipeline X-ray flaw detection defect identification method based on image processing, comprising: S1: acquiring an image training set and real defect annotation boxes, extracting the dimensions of the real defect annotation boxes and performing logarithmic normalization, converting the angles into sine and cosine components to form a four-dimensional feature vector, using a Gaussian mixture model to cluster the four-dimensional feature vector, analyzing the mean and covariance matrix of the Gaussian components, generating anchor frame templates containing scale, aspect ratio, and angle through inverse mapping, and normalizing the determinant of the covariance matrix as a measure of template cluster morphological uncertainty; S2: inputting the image to be identified into a backbone network to extract feature maps and generate a defect probability heatmap, wherein when the probability value is higher than a preset threshold, Anchor boxes are placed in the feature map region, and the placement step size of each anchor box template is determined based on the uncertainty measure of template cluster morphology; S3: During the network training phase, the intersection-union ratio (IUR) and scale angle difference between the defect ground truth annotation box and the anchor box are calculated. The average probability of the defect probability heatmap in the anchor box region is used as the weight to construct the matching degree index. The IUR weight is increased in high probability areas, and the weight of the inverse similarity function value corresponding to the scale angle difference is increased in low probability areas. Positive and negative samples are assigned accordingly. After the region proposal network outputs the defect candidate region, the Gaussian component identifier of the anchor box template corresponding to the defect candidate region is converted into an embedding vector and fused with the feature map of the defect candidate region as an enhancement feature. The input is used to classify and regress the network to output the defect category and location. Compared to existing technologies where anchor frame clustering does not consider joint statistical relationships and suffers from computational redundancy and insufficient matching due to blind uniform placement across the entire image, this invention, in the context of pipeline X-ray flaw detection, constructs four-dimensional features based on defect size and angle and uses Gaussian mixture model clustering to generate anchor frame templates containing morphological uncertainty measures. Combined with defect probability heatmaps, it achieves adaptive density placement of anchor frames in suspected defect areas. During training, it dynamically allocates positive and negative samples by fusing local probabilities and scale-angle differences. Finally, it introduces Gaussian priors as shape embedding features for enhancement. This not only significantly reduces the unnecessary background computation overhead but also effectively improves the recognition rate and accurate positioning capability of various deformed defects under complex pipe wall texture interference.

[0007] Furthermore, the method for obtaining the template cluster morphological uncertainty measure includes: extracting the width and height parameters of all real defect bounding boxes in the image training set, performing logarithmic normalization, and converting the rotation angle parameters into sine and cosine components, combining them to form a four-dimensional feature vector; using the expectation-maximization algorithm to iteratively fit the four-dimensional feature vector to a Gaussian mixture model until the log-likelihood function converges; extracting the mean vector of each Gaussian component and performing inverse mapping as the initial configuration of the anchor frame template's scale, aspect ratio, and angle; calculating the covariance matrix of each Gaussian component, and after normalizing the determinant value of the covariance matrix, using it as the template cluster morphological uncertainty measure for the corresponding anchor frame template. By extracting the determinant of the covariance matrix and performing normalization processing to quantify the morphological uncertainty measure of the template cluster, a precise mathematical representation of the fluctuation range of different types of defect shape changes is provided. This enables the system to densely deploy anchor frames for defects with fixed shapes and sparsely deploy them for defects with varied shapes, thereby further reducing the number of redundant candidate boxes while ensuring the coverage accuracy of specific defects.

[0008] Furthermore, the image to be identified is input into a backbone network to extract feature maps and generate a defect probability heatmap. This includes: extracting multi-scale deep feature maps of the image to be identified through the backbone network; inputting the top-level feature map into a probability prediction branch consisting of a 1×1 convolutional layer and a sigmoid activation function, outputting a single-channel initial probability map; and resampling the initial probability map to the same spatial resolution as the selected target feature map used for anchor frame placement using a bilinear interpolation algorithm to generate a pixel-level defect probability heatmap. Generating a pixel-level defect probability heatmap by combining deep feature maps with a bilinear interpolation algorithm provides high-precision spatial location prior guidance for subsequent anchor frame placement, effectively filtering out large areas of definitively defect-free normal pipe walls, significantly reducing the false positive rate in the region proposal stage, and accelerating the overall image processing.

[0009] Furthermore, positive and negative samples are assigned accordingly, including: calculating the intersection-union ratio (IU) of the actual defect bounding box and all deployed anchor boxes, and using the normalized differences in width, height, and angle (including periodic corrections) as the scale-angle difference; calculating the average probability value of the area covered by each anchor box in the defect probability heatmap; using the average probability value as an interpolation coefficient, and performing linear weighted fusion with the average probability value as the weight of the IU and the average probability value as the weight of the inverse similarity function corresponding to the scale-angle difference (1 minus the weight of the inverse similarity function corresponding to the scale-angle difference), to calculate the comprehensive matching index of each anchor box; wherein, the inverse similarity function value is obtained by converting the scale-angle difference through a monotonically decreasing function; anchor boxes with a comprehensive matching index greater than a first preset threshold are selected as positive samples, and anchor boxes with a comprehensive matching index less than a second preset threshold are selected as negative samples. By constructing a composite matching index that comprehensively considers the IU, scale-angle difference, and local probability heatmap distribution, and adaptively tilting the matching weight in different probability confidence regions, the division of positive and negative samples is made more consistent with the distribution logic of real defects, effectively reducing the risk of missing detection of difficult samples such as blurred edges and accelerating model convergence.

[0010] Furthermore, after the region proposal network outputs defect candidate regions, it transforms the Gaussian component identifiers of the anchor box templates corresponding to the defect candidate regions into embedding vectors. This includes: recording the generated anchor box templates corresponding to each defect candidate region output by the region proposal network and obtaining the Gaussian component category indexes corresponding to the anchor box templates; constructing a learnable embedding dictionary, the size of which is equal to the total number of Gaussian components; and mapping the Gaussian component category indexes to dense continuous vectors of fixed dimensions through dictionary lookup operations, serving as shape prior embedding vectors for the defect candidate regions. By introducing a learnable embedding dictionary mechanism, the discrete Gaussian component category indexes are transformed into high-dimensional continuous shape prior embedding vectors, giving the network the ability to pre-understand the macroscopic morphological orientation of different types of defects before fully resolving microscopic pixel details, greatly enhancing the model's implicit feature representation of the geometric topological distribution of defects.

[0011] Furthermore, the feature maps of the defect candidate regions are fused together as enhanced features and input into the classification and regression network to output the defect category and location. This process includes: extracting local feature maps of the defect candidate regions from the backbone network using region-of-interest pooling and flattening them into one-dimensional feature vectors; concatenating the shape prior embedding vector with the one-dimensional feature vector; inputting the concatenated features into a multilayer perceptron for feature fusion and dimensionality reduction to generate enhanced features; and inputting the enhanced features into the fully connected classification head and regression head respectively, outputting the defect category confidence and refined bounding box position bias. Before the features are input into the prediction head, the extracted local visual features and shape prior embedding vectors are subjected to multimodal deep fusion and dimensionality reduction in the channel dimension. This ensures that the enhanced features used for classification and regression possess both microscopic texture details and macroscopic statistical morphological cues, thereby effectively reducing the boundary localization error of pipe crack targets with large deformation differences.

[0012] Furthermore, the angle is converted into sine and cosine components, including using sine and cosine functions to convert the rotation angle values ​​into sine and cosine components. Utilizing sine and cosine functions to perform feature transformation on the defect's rotation angle eliminates the numerical discontinuity of the original angle parameters at specific periodic nodes due to abrupt changes, ensuring a smooth transition of the rotational morphology representation within the feature space and maintaining the stability of network training and loss backpropagation.

[0013] Furthermore, the dimensions of the actual defect bounding boxes are extracted and logarithmically normalized. This includes extracting the width and height of each actual defect bounding box, dividing the width and height by the overall image width and height respectively using a logarithmic function, and then taking the logarithm to complete the logarithmic normalization operation. Using a logarithmic function to normalize the actual width and height of the bounding boxes effectively smooths out the long-tail effect of data distribution caused by the extreme aspect ratios and huge scale spans often exhibited by pipe defects. This ensures that the model can perform balanced and efficient feature learning for both micropores and ultra-long cracks within the same scale space.

[0014] Furthermore, multi-scale deep feature maps of the image to be identified are extracted through a backbone network, including: constructing a ResNet50 residual backbone network based on the PyTorch deep learning framework, and performing multi-layer convolution processing on the input image to be identified to extract multi-scale deep feature maps. By selecting a backbone network with a residual structure for multi-layer convolution processing, the gradient decay problem that is easily caused in the training of deep networks is effectively overcome, and key multi-scale defect semantic information can be extracted more specifically and stably from radiographic inspection images full of imaging noise and complex weld interference.

[0015] In a second aspect, the present invention also provides a pipeline radiographic testing defect identification system based on image processing, including a processor and a memory, the memory storing a computer program, and the processor executing the computer program to implement the pipeline radiographic testing defect identification method based on image processing described above.

[0016] Beneficial Effects: This invention performs dimensionless processing on the size and angle features of defect annotation boxes and uses Gaussian mixture model clustering to generate anchor frame templates that include scale, aspect ratio, and angle, thus adapting to the complex shapes of pipeline defects. By combining defect probability heatmaps with anchor frame placement only in high-probability regions and adjusting the placement step size according to the uncertainty of template cluster morphology, the placement density is increased in regions corresponding to templates with lower morphological uncertainty, reducing unnecessary computational redundancy and improving detection efficiency and coverage accuracy. During the training phase, the probability of the defect probability heatmap is used as weights to construct a comprehensive matching index covering the cross-union ratio and the inverse similarity function values ​​corresponding to scale and angle differences. Differentiated weight allocation is applied to different probability regions to distinguish positive and negative samples, improving the rationality of sample matching and aiding model convergence. The Gaussian component identifiers of the anchor frame templates are converted into embedding vectors and fused with candidate region features, supplementing the classification and regression network with prior morphological features and improving the accuracy of pipeline defect category identification and location. Attached Figure Description

[0017] Figure 1 This is a flowchart of the defect identification method in an embodiment of the present invention.

[0018] Figure 2 This is a schematic diagram of the cluster distribution of the anchor frame template in an embodiment of the present invention.

[0019] Figure 3 This is a schematic diagram of the comprehensive matching index curve in an embodiment of the present invention.

[0020] Figure 4 This is a comparative chart evaluating the performance improvement of algorithms at each stage from the basic model to the complete solution. Detailed Implementation

[0021] In the first embodiment, the present invention proposes a pipeline radiographic flaw detection defect identification method based on image processing, such as... Figure 1 As shown, it includes:

[0022] S1. Obtain the image training set and the true defect annotation boxes. Extract the size of the true defect annotation boxes and perform log normalization. Convert the angles into sine and cosine components to form a four-dimensional feature vector. Use a Gaussian mixture model to cluster the four-dimensional feature vectors. Analyze the mean and covariance matrix of the Gaussian components. Generate anchor box templates containing scale, aspect ratio and angle through inverse mapping. Normalize the determinant of the covariance matrix and use it as a measure of the uncertainty of the template cluster shape.

[0023] The OpenCV vision computing library is used to read the training set of pipeline defect images and data annotation files. The width, height and rotation angle values ​​of each defect real annotation box are extracted. The logarithmic function is used to divide the width and height by the overall width and height of the image respectively and then calculate the logarithm to complete the logarithmic normalization operation of the size. The rotation angle values ​​are converted into sine and cosine components using sine and cosine functions. The log-normalized width, height, and sine and cosine components are concatenated to form a four-dimensional feature vector. The Gaussian Mixture model algorithm module in the Scikit-Learn machine learning library is used to perform clustering on the four-dimensional feature vectors of all bounding boxes. The covariance type of the algorithm parameter is set to the English full covariance matrix. The expectation-maximization algorithm is used for iterative optimization calculation until the parameters converge. After the clustering is completed, the mean vector attribute and covariance matrix attribute of each Gaussian component are extracted. The natural exponent of the first two dimensions of the mean vector is obtained by using the exponential function and multiplied by the original image size to obtain the reference width and reference height of the anchor box template. The aspect ratio is determined by the reference width and reference height. The arctangent value is obtained by using the arctan2 arctangent function on the last two dimensions of the mean vector. The reference angle of the anchor box template is obtained by restoring it according to the encoding method of the angle component. Thus, the inverse mapping is completed to generate an anchor box template containing scale, aspect ratio and angle. Calculate the determinant of the covariance matrix corresponding to each Gaussian component, and use the maximum-minimum normalization algorithm to map the determinant values ​​of all Gaussian components to a value range of 0.1 to 1. Use the normalized value as a measure of the uncertainty of the template cluster shape of each anchor frame template.

[0024] In an optional embodiment, the dimensions of the actual defect annotation boxes are extracted and logarithmically normalized. The angles are converted into sine and cosine components to form a four-dimensional feature vector. A Gaussian mixture model is used to cluster the four-dimensional feature vectors. The mean and covariance matrix of the Gaussian components are analyzed. An anchor frame template containing scale, aspect ratio, and angle is generated through inverse mapping. The determinant of the covariance matrix is ​​normalized and used as a measure of the uncertainty of the template cluster shape, including: The width and height parameters of all defect ground truth bounding boxes in the image training set are extracted and log-normalized. The rotation angle parameters are converted into sine and cosine components and combined to form a four-dimensional feature vector. The four-dimensional feature vector is iteratively fitted with a Gaussian mixture model using the expectation-maximization algorithm until the log-likelihood function converges. The mean vector of each Gaussian component is extracted and inversely mapped to serve as the initial configuration of the scale, aspect ratio, and angle of the anchor frame template. The covariance matrix of each Gaussian component is calculated, and the determinant value of the covariance matrix is ​​normalized and used as a measure of the template cluster morphological uncertainty of the corresponding anchor frame template.

[0025] In the process of extracting all ground truth bounding boxes for defects from the image training set, the width W and height H of each bounding box are obtained. The width W and height H are then logarithmically normalized using the following formula: and ,in and The width and height of the input image, for example, a typical input resolution of 512×512 pixels; and These are the width and height after logarithmic normalization. The rotation angle parameter is within the range of 0 to 180°. The continuous sinusoidal components are calculated through transformation. Sum and cosine components This avoids data discontinuities caused by abrupt changes in angle periodicity. The above four parameters are combined to form a four-dimensional feature vector. These are used as input samples for the Gaussian mixture model. During the Gaussian mixture model fitting stage, the optimal range for the total number of Gaussian components K is set to 5 to 9, for example, K=6. The expectation-maximization algorithm is used for iteration. Initially, the mean, covariance matrix, and mixture weights are randomly initialized. The maximum number of iterations is set to 1000, or when the difference between two consecutive iterations of the log-likelihood function is less than a threshold, such as... The model is considered convergent at this point. After convergence, the mean vector of the K Gaussian components is extracted. and the 4×4 covariance matrix The mean vector is restored by inverse mapping and used as the anchor frame template. The width and height are determined by... and Obtained, aspect ratio is The angle is calculated and then obtained through... Calculated.

[0026] For example, if the result after extracting a set of inverse mappings is... =32、 =64、 =45°, calculate the determinant of the covariance matrix of each Gaussian component. The determinant values ​​of all components are scaled to within the range of 0.1 to 1.0 using the max-min normalization method. This normalized value serves as a measure of the uncertainty in the template cluster shape. A smaller value, such as close to 0.1, indicates that the anchor frame template shape is more fixed and the range of shape variation is smaller. A smaller step size, such as 4 pixels, can be used during layout. A larger value indicates that the shape of this type of defect is more variable, and the layout step size, such as 8 pixels or 16 pixels, can be increased to reduce the number of redundant anchor frames. Figure 2 This is a schematic diagram of the cluster distribution of anchor frame templates.

[0027] S2, input the image to be identified into the backbone network to extract feature maps and generate a defect probability heat map, place anchor frames in the feature map regions with probability values ​​higher than a preset threshold, and determine the placement step size of each anchor frame template based on the template cluster morphology uncertainty measure.

[0028] A ResNet50 residual backbone network based on the PyTorch deep learning framework is constructed to perform multi-layer convolutional processing on the input image to be identified in order to extract multi-scale deep feature maps. After the deepest layer of the backbone network, a probability prediction branch network containing a 1×1 convolutional kernel structure and a Sigmoid activation function is connected to generate a defect probability heatmap with one channel. The value of each pixel in the defect probability heatmap represents the probability of a defect existing in the corresponding receptive field region. The set of two-dimensional coordinate points with pixel probability values ​​greater than a preset threshold of 0.5 in the defect probability heatmap is extracted as the placement center region. For each anchor frame template generated above, a direct linear mapping function is established between the placement stride of the anchor frame template and the uncertainty measure of the template cluster shape. A base step size constant is set, plus the product of the template cluster morphological uncertainty measure and the step size adjustment coefficient, as the placement step size of the template. This ensures that the anchor frame template with a smaller template cluster morphological uncertainty measure has a smaller corresponding placement step size. For the calculated placement step size, the meshgrid function of the PyTorch framework is called to generate a two-dimensional coordinate grid with pixel spacing in the feature map region where the placement center is located. Anchor frame space instances are generated at each coordinate grid node according to the scale, aspect ratio, and angle of the corresponding anchor frame template. This increases the placement density of anchor frame templates with small shape changes (i.e., low template cluster morphological uncertainty measure) in the image space, and reduces the number of redundant candidate boxes generated by templates with large morphological change ranges in low-value areas.

[0029] Specifically, the determination method for the basic step size constant and the step size adjustment coefficient is as follows: The basic step size constant is determined based on the downsampling step size of the target feature map in the backbone network where anchor boxes are placed. To ensure the coverage accuracy of minor defects, the value range of the basic step size constant is set to 2 to 8 pixels, preferably half of the downsampling step size of the target feature map. For example, when the downsampling factor of the target feature map relative to the original image is 8, such as in the P3 layer of ResNet, the basic step size constant is 4 pixels. The step size adjustment coefficient is used to control the step size variation caused by morphological uncertainty. Its value is preset according to the constraints of computing resources and coverage requirements. The value range of the step size adjustment coefficient is 8 to 24. The determination logic is that when the template cluster morphological uncertainty measure reaches the maximum value of 1.0, the maximum placement step size (the sum of the basic step size constant and the step size adjustment coefficient) should not exceed the receptive field radius corresponding to the target feature map to avoid detection blind spots.

[0030] In an optional embodiment, the image to be identified is input into a backbone network to extract feature maps and generate a defect probability heatmap, including: extracting multi-scale deep feature maps of the image to be identified through the backbone network; inputting the top-level feature map into a probability prediction branch consisting of a 1×1 convolutional layer and a Sigmoid activation function, and outputting a single-channel initial probability map; and resampling the initial probability map to the same spatial resolution as the selected target feature map used for setting anchor boxes using a bilinear interpolation algorithm to generate a pixel-level defect probability heatmap.

[0031] A 512×512 pixel image to be identified is input into a backbone network based on ResNet-50 or DarkNet-53. This backbone network structure includes initial convolutional layers, a combination of multiple stages of residual layers, and downsampling layers. Feature extraction is performed through stacked convolutional operations and cross-layer identity mapping. The input to this backbone network is a 512×512 pixel tensor containing three channels of the image to be identified. The output of this backbone network is a multi-scale deep feature map obtained by downsampling layer by layer as the network depth increases. , , Specifically, as the backbone network depth increases, the receptive field of the features continuously expands and is downsampled layer by layer, extracting multi-scale deep feature maps with downsampling step sizes of 8, 16, and 32 in sequence. , , Select the top-level feature map with the strongest semantic information. Top-level feature map The input is 16×16 with 2048 channels and is fed into a specially constructed probabilistic prediction branch network.

[0032] The structure of this probabilistic prediction branch network consists of a 1×1 convolutional layer with a stride of 1 and 1 output channel, and a sigmoid activation function layer. The input to this probabilistic prediction branch network is the top-level feature map with the strongest semantic information. Furthermore, the output of this probabilistic prediction branch network is a single-channel initial probability map that smoothly maps numerical values ​​to the 0-1 range. This branch network utilizes 1×1 convolutional layers for cross-channel information exchange and dimensionality reduction, compressing the feature dimension from 2048 to 1, resulting in an unnormalized feature map. This map is then connected to a Sigmoid activation function, generating a 16×16×1 single-channel initial probability map. Each value in the initial probability map represents the initial confidence that a defect exists in that region. To provide pixel-level guidance at the feature map scale for anchor box placement, a target placement layer is extracted from multi-scale feature maps, for example, selecting a stride of 16. The feature map, with a spatial resolution of 32×32, uses bilinear interpolation to upsample the spatial resolution of the 16×16 single-channel initial probability map. During the upsampling process, the coordinates of each target pixel are mapped back to the initial probability map, and the probability values ​​of its four neighboring pixels are used for weighted calculation to smoothly transition the grid edges generated after resampling. This outputs a 32×32 pixel-level defect probability heatmap that fits the target deployment layer size. On this defect probability heatmap, if the pixel probability value at a certain coordinate exceeds a preset optimal threshold (e.g., 0.5), the receptive field center corresponding to that region is activated as the anchor frame deployment point, thereby filtering out a large number of background regions that are determined to be defect-free, reducing computational overhead and lowering the false positive rate in the region proposal stage.

[0033] S3. During the network training phase, the intersection-union ratio (IU) and scale angle difference between the ground truth defect bounding boxes and anchor boxes are calculated. The average probability of the defect probability heatmap in the anchor box region is used as the weight to construct a matching index. The IU weight is increased in high probability regions, and the weight of the inverse similarity function value corresponding to the scale angle difference is increased in low probability regions. Positive and negative samples are assigned accordingly. After the region proposal network outputs the defect candidate region, the Gaussian component label of the anchor box template corresponding to the defect candidate region is converted into an embedding vector and fused with the feature map of the defect candidate region as an enhancement feature. The input is then used to classify and regress the network to output the defect category and location.

[0034] During the forward propagation training of the neural network, the rotation bounding box intersection-union ratio (OCU) calculation function is used to calculate the OCU values ​​of all currently generated anchor boxes and their corresponding defect ground truth boxes. The absolute values ​​of the relative differences in width and height between the anchor boxes and the ground truth boxes, as well as the cosine distance of the angle differences between the two, are calculated. The sum of the above two items is used as the scale angle difference value. The average probability value of the defect falling within the pixel coordinate range of each anchor box in the defect probability heatmap is calculated. A matching weight formula is constructed, and the weight coefficient of the OCU is set to the average probability value. The weight coefficient of the scale angle similarity value obtained by converting the scale angle difference through the inverse similarity function is set to 1 minus the average probability value. The above two weights are multiplied by the corresponding index and then summed to obtain the comprehensive matching index.

[0035] Based on the calculated comprehensive matching index, the top 5 anchor boxes with the highest numerical ranking for each ground truth bounding box are assigned as positive samples, and anchor boxes with values ​​below 0.3 are assigned as negative samples for subsequent loss function calculation and backpropagation gradient update. After the region proposal network outputs a defect candidate region containing foreground regression coordinates using the Non-Maximum Suppression (NMS) algorithm, the Gaussian mixture model component clustering index number to which the anchor box template used to generate the defect candidate region belongs is extracted as the Gaussian component identifier. The PyTorch framework's Embedding layer module is then used to map the discrete integer Gaussian component clustering index number into a continuous high-dimensional feature embedding vector. The dimension of the high-dimensional feature embedding vector can be set according to the network size. The dimensions are set to 64, 128, or 256. The multi-channel feature regions corresponding to the defect candidate regions on the backbone feature map are cropped and uniformly scaled to a standard size of 7×7 using the Region of Interest (ROIAlign) pooling technique. The resulting high-dimensional feature embedding vectors are tensor-concatenated along the feature channel dimensions through a spatial broadcasting mechanism and appended to the uniformly scaled feature map to form a fused and enhanced feature representation. This fused and enhanced feature representation is flattened along the dimensions and input into a classification and regression sub-network structure containing multiple layers of Linear fully connected layers and Softmax activation function modules. The fully connected layer branches independently calculate and output the specific category prediction confidence of the defect, as well as the refined bounding box center point and width, height, and angle offset coordinates.

[0036] In an optional embodiment, during the network training phase, the intersection-union ratio (IU) and scale angle difference between the ground truth defect bounding boxes and anchor boxes are calculated. A matching degree index is constructed using the average probability of the defect probability heatmap within the anchor box region as weights. The IU weight is increased in high-probability regions, and the weight of the inverse similarity function value corresponding to the scale angle difference is increased in low-probability regions. Positive and negative samples are then assigned accordingly, including: Calculate the intersection-union ratio of the actual defect annotation box and all deployed anchor boxes, and the normalized differences between the two in width, height and angle including periodic correction as scale angle differences; calculate the average probability value of the area covered by each anchor box in the defect probability heatmap. The average probability value is used as the interpolation coefficient. The average probability value is used as the weight of the intersection-union ratio, and the average probability value is used as the weight of the inverse similarity function value corresponding to the scale angle difference. The average probability value is subtracted from the average probability value as the weight of the inverse similarity function value corresponding to the scale angle difference. The inverse similarity function value is obtained by converting the scale angle difference through a monotonically decreasing function. Anchor frames with an average matching index greater than a first preset threshold are selected as positive samples, and anchor frames with an average matching index less than a second preset threshold are selected as negative samples.

[0037] When calculating the similarity between the actual annotation frame G and the deployed anchor frame A, not only is the traditional rotation intersection-union ratio used to calculate the ratio of their intersection area to their merged area, but their width W, height H, and angle are also calculated. To determine the dimensional differences, calculate the absolute value of the width-to-height difference and divide it by the image diagonal length to normalize, thus obtaining the normalized width difference. and the high difference after normalization The angle parameter is corrected using a periodic correction function. In the formula, To correct and normalize the angle differences, This represents the rotation angle of the actual annotation box G. To determine the rotation angle of anchor frame A; to handle the continuous annular feature of the angle, take the arithmetic mean of the above three items, or take the L2 norm and divide by . Normalization is performed to generate scale angle differences with values ​​ranging from 0 to 1. The smaller this value, the better the size and rotation attitude match. Simultaneously, a region integration operation is used to extract the feature sub-region on the defect probability heatmap corresponding to the current anchor frame coverage position. The probability values ​​of all pixels within this feature sub-region are summed and averaged to obtain an average probability value between 0 and 1. This is used to represent the network's prior knowledge of the degree of suspected defects in the region. Based on this prior knowledge, a matching mechanism is constructed, and the average probability value is used to represent the network's prior knowledge of the degree of suspected defects in the region. As interpolation coefficients. For scale angle differences. Constructing the reverse similarity function Its function value is located in the range of 0 to 1, and the larger the value, the higher the morphological matching degree. Then, the crossover ratio and the reverse similarity function value are weighted to obtain the formula. The overall matching degree index M is obtained, where IoU is the intersection-union ratio of the actual bounding box and the anchor box.

[0038] Continuing from above, if in a high-confidence region of the defect probability heatmap, such as... =0.85, indicating that the formula places greater emphasis on the intersection-union ratio term, which is dominated by intersection characteristics, with a weight as high as 0.85. However, in low-confidence regions, such as areas with blurred edges of corrosion defects, and... =0.2. To avoid losing potential real targets, the focus is on the inverse similarity function value corresponding to the consistency of shape and size between the two, and the weight of the inverse similarity function value is increased to 0.8 for matching discovery. By setting the preferred value of the first preset threshold to 0.6 and the preferred value of the second preset threshold to 0.3, it is determined that when the comprehensive matching degree M≥0.6, the anchor box is judged as a positive sample containing real defects; when M≤0.3, the anchor box is marked as a negative sample containing only background; anchor boxes between the two are ignored samples and do not participate in the network loss calculation in the current iteration. Figure 3As shown, the high-probability prior curve has a steeper slope as the intersection-union ratio (IU) increases, while the low-probability prior curve changes more gradually with IU.

[0039] In an optional embodiment, after the region proposal network outputs defect candidate regions, it converts the Gaussian component identifiers of the anchor box templates corresponding to the defect candidate regions into embedding vectors, including: The region proposal network outputs the generated anchor box template corresponding to each defect candidate region, and obtains the Gaussian component category index corresponding to the anchor box template; a learnable embedding dictionary is constructed, the size of which is equal to the total number of Gaussian components; through dictionary lookup operations, the Gaussian component category index is mapped to a dense continuous vector of fixed dimension, which serves as the shape prior embedding vector of the defect candidate region.

[0040] Since each preset anchor box is deterministically assigned to a specific Gaussian component after clustering analysis by the Gaussian mixture model during the generation stage, the source information of the original anchor box template on which the defect candidate region is based is inherited and recorded during the process of the region proposal network performing preliminary regression and classification output to obtain a large number of defect candidate regions. The structure of the region proposal network includes a sliding window convolutional layer and parallel classification prediction layer and bounding box regression layer. The input of the region proposal network is the multi-scale deep feature map extracted by the backbone network, and the output of the region proposal network is the category confidence score and bounding box coordinate offset of a large number of defect candidate regions. The Gaussian component category index ID to which the template belongs is extracted. If the total number of Gaussian components is set to the preferred value K=6, then ID∈{0,1,2,3,4,5}. This index explicitly represents the initial shape, proportion, and pose cluster characteristics of the current candidate region in a statistical sense at the micro level.

[0041] In the initialization phase, a learnable embedding dictionary network module is constructed. The specific structure of this embedding dictionary network module is a learnable embedding matrix. The matrix has rows that are strictly equal to the total number of Gaussian components, K. The number of columns, D, is configured to the preset output vector dimension, for example, D=256. All parameters in the dictionary are initialized following a normal distribution with a mean of 0 and a variance of 0.02. The input to this embedding dictionary network module is a Gaussian component category index scalar, and the output is a fixed-dimensional dense continuous real-valued vector, i.e., a shape prior embedding vector. After successfully obtaining the Gaussian component category index, the model uses standard embedding lookup table operations in the underlying framework, such as the nn.Embedding interface in PyTorch, with this index as the lookup key to extract the matrix. The process maps the originally isolated Gaussian category scalars, which lack semantic distance correlation, into 256-dimensional high-dimensional dense and continuous real-valued spatial features. The 256-dimensional embedding vectors update their weights synchronously with the backpropagation of the end-to-end loss of the overall detection network, gradually learning implicit features that can map the potential geometric distribution topological information of each Gaussian template. The feature sequence output by the lookup table is saved as the shape prior embedding vector of the corresponding defect candidate region, so that the subsequent recognition network can pre-know the macroscopic morphological guidance clues of different types of flaw detection defects, such as longitudinal cracks and point pores, before observing pixel details.

[0042] In an optional embodiment, the feature map of the defect candidate region is fused as an enhancement feature, and the input to the classification and regression network outputs the defect category and location, including: Local feature maps of candidate defect regions are extracted from the backbone network using region-of-interest pooling and flattened into one-dimensional feature vectors. The shape prior embedding vector is concatenated with the one-dimensional feature vector. The concatenated features are then input into a multilayer perceptron for feature fusion and dimensionality reduction to generate enhanced features. The enhanced features are then input into a fully connected classification head and a regression head, respectively, to output the defect category confidence and a refined bounding box position bias.

[0043] For all defect candidate regions of varying scales and angles generated by the region proposal network, based on the predicted coordinate information of the defect candidate regions, the defect candidate regions are projected onto the multi-scale fusion feature layer corresponding to the feature pyramid. Then, a rotational region of interest pooling operation with angle parameters is deployed, and gridded sampling points are deployed within each defect candidate region. Pixel-level features are accurately obtained through bilinear interpolation, and feature regions of different scales are fixed to a consistent standard size, for example, a pooling resolution of 7×7 pixels is preferably set. The 7×7×256-channel tensor obtained after pooling is then subjected to a multi-dimensional flattening operation to obtain a one-dimensional visual feature vector with a dimension length of 12544. At the specified data splicing dimension, the extracted one-dimensional visual feature vector is directly spliced ​​with the fixed-length 256-dimensional shape prior embedding vector output from the previous stage, merging to generate a high-dimensional cascaded feature vector with a length of 12800 dimensions. This achieves multimodal fusion of deep image appearance features and external statistical shape topology features at the channel level. Finally, the 12800-dimensional cascaded feature is fed into the multilayer perceptron network module in the subsequently constructed classification and regression network.

[0044] The structure of this multilayer perceptron network module comprises two cascaded dense fully connected layers, each equipped with 1024 output neurons. Batch normalization layers and ReLU nonlinear activation layers are embedded between each layer. The input to this multilayer perceptron network module is a 12800-dimensional high-dimensional cascaded feature vector generated by merging deep image appearance features and shape prior embedding vectors. The output of this multilayer perceptron network module is a high-order condensed, dimensionality-reduced enhanced feature representation vector. This network module performs spatial mapping and channel dimensionality reduction compression on the complex features, outputting a high-order condensed 1024-dimensional enhanced feature representation vector. This batch of 1024-dimensional enhanced feature representation vectors is simultaneously distributed to the dual-branch prediction network structure of the classification and regression network. The end of this dual-branch prediction network structure contains parallel fully connected classification heads and fully connected regression heads. The classification head uses a single fully connected layer and a Softmax activation function, while the regression head uses a linear fully connected layer. The connection layer, the input of this dual-branch prediction network structure is the aforementioned 1024-dimensional enhanced feature representation vector. The output of this dual-branch prediction network structure includes the probability confidence distribution of the number of flaw detection target categories and the 5-dimensional geometric correction parameters under the corresponding predicted category. The fully connected classification head, with the help of a single fully connected layer with Softmax activation, maps the features to the probability confidence distribution of the number of flaw detection target categories including the background category. The parallel fully connected regression head is a linear output architecture, outputting the 5-dimensional geometric correction parameters under the corresponding predicted category. The 5-dimensional geometric correction parameters include the center point lateral offset, longitudinal offset, width scale scaling logarithm, height scale scaling logarithm, and angle rotation increment. Through the above method, relying on the assistance of geometric prior parameters in the multilayer perceptron network module, the regression compensation process helps to reduce the positioning error boundary of pipeline crack targets with large deformation differences.

[0045] The experiment was conducted using an industrial pipeline defect rotating target detection dataset, which contains 1200 pipeline ray images with a resolution of 512×512 pixels, covering typical flaw detection defects such as longitudinal cracks and point pores. The dataset was trained for 100 epochs using a stochastic gradient descent optimizer with an initial learning rate of 0.01. The evaluation metrics were average accuracy across all classes and frames per second. A standard rotating region proposal detection network without integrated optimization modules was used as the base control model.

[0046] It should be noted that the basic model has an average precision of 73.5% and processes 15 frames per second. After introducing Gaussian mixture model clustering and defect probability heatmap-guided anchor frame placement modules into the basic model, the average precision improved to 76.2% and the number of frames per second increased to 24. Building upon this, further adding a comprehensive matching index module based on average probability weighted fusion of intersection-union ratio and scale-angle difference corresponding to the inverse similarity function value, the model's average precision improved to 79.4%, and the number of frames per second stabilized at 23. After fusing Gaussian component shape prior embedding vectors, the complete solution achieved an average precision of 82.1% and maintained a number of frames per second of 21. Figure 4 As shown, the defect probability heatmap guides the placement of anchor boxes, which can reduce redundant candidate regions and improve inference efficiency. The comprehensive matching index further improves the sample allocation quality, while the shape prior embedding vector helps to improve the accuracy of defect classification and rotating bounding box regression, thus achieving a good balance between detection accuracy and processing efficiency.

[0047] The performance improvement is attributed to the Gaussian clustering and defect probability heatmap-guided deployment module, which generates anchor box templates based on the statistical characteristics of defect morphology and filters out a large number of defect-free background areas through a defect probability heatmap threshold activation mechanism, reducing the number of redundant anchor boxes and computational overhead, thereby improving inference speed. The comprehensive matching index uses the average probability of the defect probability heatmap as an interpolation coefficient to dynamically adjust the weights of the cross-union ratio and the inverse similarity function value corresponding to scale angle differences, which helps improve the matching effect of difficult samples such as blurred edges and obvious morphological changes, and reduces the risk of missed detections. The shape prior embedding vector module fuses the statistical morphological information represented by Gaussian components with the visual features of candidate regions, supplementing the classification and regression network with defect morphological priors, thereby further improving the positioning accuracy of rotated bounding boxes and overall detection performance.

[0048] The present invention also provides a pipeline radiographic flaw detection defect identification system based on image processing. The system includes a processor and a memory. The memory stores computer program instructions. When the computer program instructions are executed by the processor, the pipeline radiographic flaw detection defect identification method based on image processing according to the first aspect of the present invention is implemented.

[0049] The system also includes other components well known to those skilled in the art, such as communication buses and communication interfaces, the settings and functions of which are known in the art and therefore will not be described in detail here.

Claims

1. A method for identifying defects in pipe radiographic testing based on image processing, characterized in that, include: S1: Obtain the image training set and the true defect annotation boxes, extract the size of the true defect annotation boxes and log normalize it, convert the angle into sine and cosine components to form a four-dimensional feature vector, use the Gaussian mixture model to cluster the four-dimensional feature vector, analyze the mean and covariance matrix of the Gaussian components, generate anchor box templates containing scale, aspect ratio and angle through inverse mapping, and use the normalized determinant of the covariance matrix as a measure of the uncertainty of the template cluster shape. S2: Input the image to be identified into the backbone network to extract feature maps and generate a defect probability heat map. Place anchor frames in feature map regions where the probability value is higher than the preset threshold. Determine the placement step size of each anchor frame template based on the template cluster morphology uncertainty measure. S3: During the network training phase, the intersection-union ratio (IU) and scale angle difference between the ground truth defect bounding boxes and anchor boxes are calculated. The average probability of the defect probability heatmap in the anchor box region is used as the weight to construct a matching index. The IU weight is increased in high probability regions, and the weight of the inverse similarity function value corresponding to the scale angle difference is increased in low probability regions. Positive and negative samples are assigned accordingly. After the region proposal network outputs the defect candidate region, the Gaussian component label of the anchor box template corresponding to the defect candidate region is converted into an embedding vector and fused with the feature map of the defect candidate region as an enhancement feature. The input is then used to classify and regress the network to output the defect category and location.

2. The pipeline radiographic flaw detection defect identification method based on image processing according to claim 1, characterized in that, Methods for obtaining template cluster morphological uncertainty measures include: The width and height parameters of all defect ground truth bounding boxes in the image training set are extracted and log-normalized. The rotation angle parameters are converted into sine and cosine components and combined to form a four-dimensional feature vector. The four-dimensional feature vector is iteratively fitted with a Gaussian mixture model using the expectation-maximization algorithm until the log-likelihood function converges. The mean vector of each Gaussian component is extracted and inversely mapped to serve as the initial configuration of the anchor frame template's scale, aspect ratio, and angle. The covariance matrix of each Gaussian component is calculated, and the determinant value of the covariance matrix is ​​normalized and used as a measure of the template cluster morphological uncertainty of the corresponding anchor frame template.

3. The pipeline radiographic flaw detection defect identification method based on image processing according to claim 1, characterized in that, The image to be identified is input into the backbone network to extract feature maps and generate a defect probability heatmap, including: Multi-scale deep feature maps of the image to be identified are extracted through a backbone network; the top-level feature map is input into a probability prediction branch consisting of a 1×1 convolutional layer and a Sigmoid activation function, and a single-channel initial probability map is output; the initial probability map is resampled to the same spatial resolution as the selected target feature map used to lay out the anchor box using a bilinear interpolation algorithm, generating a pixel-level defect probability heatmap.

4. The pipeline radiographic flaw detection defect identification method based on image processing according to claim 1, characterized in that, Based on this, positive and negative samples are assigned, including: The intersection-union ratio (IU) of the actual defect annotation box and all deployed anchor boxes is calculated, along with the normalized differences in width, height, and angle (including periodic corrections) as scale angle differences. The average probability value of the area covered by each anchor box in the defect probability heatmap is calculated. The average probability value is used as an interpolation coefficient, and a linear weighted fusion is performed with the average probability value as the weight of the IU and the average probability value as the weight of the inverse similarity function corresponding to the scale angle difference (1 minus the average probability value) to calculate the comprehensive matching index of each anchor box. The inverse similarity function value is obtained by converting the scale angle difference through a monotonically decreasing function. Anchor boxes with a comprehensive matching index greater than a first preset threshold are selected as positive samples, and anchor boxes with a comprehensive matching index less than a second preset threshold are selected as negative samples.

5. The pipeline radiographic flaw detection defect identification method based on image processing according to claim 1, characterized in that, After the region proposal network outputs defect candidate regions, it converts the Gaussian component identifiers of the anchor box templates corresponding to the defect candidate regions into embedding vectors, including: The region proposal network outputs the generated anchor box template corresponding to each defect candidate region, and obtains the Gaussian component category index corresponding to the anchor box template; a learnable embedding dictionary is constructed, the size of which is equal to the total number of Gaussian components; through dictionary lookup operations, the Gaussian component category index is mapped to a dense continuous vector of fixed dimension, which serves as the shape prior embedding vector of the defect candidate region.

6. The pipeline radiographic flaw detection defect identification method based on image processing according to claim 1, characterized in that, The feature map is fused with the defect candidate region feature map as an enhancement feature. This feature is then input into the classification and regression network, where the output includes the defect category and location, including: Local feature maps of candidate defect regions are extracted from the backbone network using region-of-interest pooling and flattened into one-dimensional feature vectors. The shape prior embedding vector is concatenated with the one-dimensional feature vector. The concatenated features are then input into a multilayer perceptron for feature fusion and dimensionality reduction to generate enhanced features. The enhanced features are then input into a fully connected classification head and a regression head, respectively, to output the defect category confidence and a refined bounding box position bias.

7. The pipeline radiographic flaw detection defect identification method based on image processing according to claim 1, characterized in that, Converting angles to sine and cosine components includes: The rotation angle values ​​are converted into sine and cosine components using sine and cosine functions.

8. The pipeline radiographic flaw detection defect identification method based on image processing according to claim 1, characterized in that, Extract the dimensions of the actual defect annotation boxes and perform logarithmic normalization, including: Extract the width and height of the true annotation box for each defect, and use the logarithmic function to divide the width and height by the overall width and height of the image respectively to obtain the logarithm, thus completing the logarithmic normalization operation of the size.

9. The pipeline radiographic flaw detection defect identification method based on image processing according to claim 3, characterized in that, Multi-scale deep feature maps of the image to be identified are extracted using a backbone network, including: A ResNet50 residual backbone network based on the PyTorch deep learning framework is constructed to perform multi-layer convolution processing on the input image to be identified in order to extract multi-scale deep feature maps.

10. A pipeline radiographic flaw detection defect identification system based on image processing, comprising a processor and a memory, characterized in that, The memory stores a computer program, and the processor executes the computer program to implement the image processing-based pipeline radiographic defect identification method as described in any one of claims 1-9.

Images