A power transmission line fitting crimping quality defect identification method based on X-ray detection imaging

Abstract

The utility model provides a kind of transmission line hardware crimping quality defect identification method based on X-ray detection imaging, it is related to transmission line detection field. The original imaging data of transmission line hardware is obtained using X-ray detection equipment;It is transmitted to computer system for processing by sensor;Original image data is carried out denoising processing, standardization processing;Synthetic defect image is generated using diffusion model that is pre-trained to complete, and data set is enlarged;Feature extraction is carried out to the image after pre-processing using convolutional neural network;Preliminary identification of hardware crimping quality defect is carried out using defect classification model that is pre-trained, and the defect area is boxed using target detection;The identification result is visualized, and labeled image is generated;Detailed report of defect type, position and severity is output.The utility model can identify the defect of transmission line hardware crimping quality before failure, solve the line safety problem caused by bad crimping quality, and improve the safety and stability of line.

Description

Technical Field

[0001] This invention belongs to the field of power grid technology and relates to the field of transmission line inspection, specifically a method for identifying quality defects in transmission line fittings based on X-ray inspection imaging. Background Technology

[0002] The crimping quality of tension clamps and splicing pipes in transmission lines is crucial to line safety. Numerous line breaks have occurred nationwide due to poor crimping quality. Once the process is out of control, problems such as crimping position deviations are difficult to detect, thus jeopardizing line safety. Therefore, identifying defects in the crimping quality of transmission line hardware before a fault occurs is essential to reduce or even eliminate line safety issues caused by poor crimping quality, and improving the safety and stability of lines has significant social implications.

[0003] Currently, the main technology for identifying quality defects in the crimping of transmission line fittings is through X-rays, because X-rays can detect all crimping defects. Therefore, in recent years, both the State Grid and the Southern Power Grid have widely adopted visualization detection technology based on X-ray data imaging.

[0004] Currently, the determination of images and defects after detection mainly relies on manual methods. While some use deep learning models to construct data image and defect determination tools, such as using a YOLO-Nano-based object detection model as the foundation for defect recognition, these methods generally have certain technical drawbacks.

[0005] The disadvantages of manually judging images and defects after inspection are as follows: manual judgment of X-ray images requires high professional skills and experience from personnel, and requires long-term professional training. More importantly, it consumes a lot of time and human resources. In addition, it takes a long time to complete the analysis, evaluation and issue the analysis results. Furthermore, manual interpretation is prone to misjudgment, omission and wrong judgment, which can affect the construction progress and quality of the project.

[0006] The disadvantages of the method based on the YOLO-Nano object detection model to determine the detection of images and defects are: 1. Insufficient dataset: Firstly, the dataset is too small. Some individual defects only have a few dozen real sample images, which is definitely insufficient for deep learning algorithms. With too small a dataset, the learning ability of the deep learning model cannot be fully utilized, and it cannot accurately identify defective lines and accurately label the defect locations; 2. High cost of dataset labeling: Although manually labeling images can increase the size of the dataset for training the model, the manual cost is huge, so this method seems unacceptable.

[0007] 3. The target detection task is too simplistic: Defect detection involves more than just outlining the defects in hardware. It first requires determining if the X-ray image shows a quality defect (if there is no defect, this detection is unnecessary). For hardware with defects, target detection is needed to mark the defect locations. After marking the defect locations, it's also necessary to identify the number of defect locations. This approach reduces manual labor costs and allows deep learning to effectively detect defects. Summary of the Invention

[0008] The purpose of this invention is to provide a method for identifying defects in the crimping quality of transmission line hardware based on X-ray detection imaging. Multiple tasks are performed in parallel, and different tasks can be optimized for each other. This invention can identify defects in the crimping quality of transmission line hardware before a fault occurs, solve line safety problems caused by poor crimping quality, and improve the safety and stability of the line.

[0009] The technical solution of this invention is as follows:

[0010] Step 1. Data Acquisition:

[0011] a. Obtain raw imaging data of power transmission line fittings using X-ray inspection equipment;

[0012] b. The data is transmitted from the sensors to the computer system for processing;

[0013] Step 2. Data Preprocessing:

[0014] a. Denoise the original image data using Gaussian filtering or median filtering methods;

[0015] b. Standardize the image to ensure consistency in brightness and contrast;

[0016] c. Diffusion model training: Using a pre-trained diffusion model to generate synthetic defect images to expand the dataset;

[0017] Step 3. Feature Extraction:

[0018] a. Use a convolutional neural network to extract features from the preprocessed image;

[0019] b. Select appropriate convolutional and pooling layers to ensure that the extracted features are representative;

[0020] Step 4: Defect Identification

[0021] a. Use a pre-trained defect classification model with a swin-transformer backbone network to perform preliminary identification of fitting crimping quality defects.

[0022] b. The defective images identified are further processed using a pre-trained defect detection model with a swin-base backbone network (DERT), and the defective regions are bounded out using object detection.

[0023] Step 5. Output Results:

[0024] a. Visualize the recognition results and generate labeled images;

[0025] b. Output a detailed report of the defect type, location, and severity (number).

[0026] This invention involves numerous backbone networks such as Swin-base and Swin-transformer. The backbone network is part of the model and is primarily responsible for extracting features from the raw input. These features are crucial for subsequent tasks (such as classification and object detection). The backbone network is typically a deep convolutional neural network (CNN) or other similar architectures (Swin-base, Swin-transformer). The model itself is a complete, end-to-end trainable / inference system that includes the backbone network, training parameters, several other modules such as task heads (classification head, detection head, counting head), and a decoder.

[0027] The steps involved in using the diffusion model (using a trained diffusion model) in step c of the data preprocessing in step two of this invention are as follows:

[0028] (1). Forward diffusion:

[0029] Forward diffusion is a predefined image noise-adding mechanism and a non-learning Markov chain noise-adding mechanism used to generate multi-scale noise samples during the model training phase. Starting with the acquired original defect image, this process is used to construct the correspondence between the noisy image and the real noise. Gaussian noise is injected sequentially through multiple time steps, and finally a complete noise evolution sequence is output, which helps the pre-trained model to better restore the specific hardware defects from the noise.

[0030] Forward diffusion process:

[0031] - Input: Raw image data;

[0032] - Processing: Gradually add noise to generate a sequence of noisy images;

[0033] Output: Noisy image sequence;

[0034] Processing steps:

[0035] The forward diffusion process (i.e., the noise addition process simulated during the training phase) starts from the real data and gradually adds Gaussian noise. The formula for calculating the noise at time t is as follows:

[0036]

[0037] in, It is a real, original image. This represents the data generated at time t. The activation function (usually tanh or sigmoid) ensures that the output value range is reasonable. It is Gaussian noise. It is the diffusion coefficient that increases with increasing t. This indicates that the original image retention factor decreases as t increases;

[0038] (2). Backdiffusion:

[0039] Backdiffusion is an iterative denoising mechanism based on deep neural networks, used to gradually reconstruct high-quality images from purely noisy data during the model inference stage. This process uses a pre-trained denoising network as its core. After providing initial noise, it performs reverse operations over time steps to estimate and remove noise components introduced at each stage, ultimately generating new samples that conform to the distribution of real data. Unlike forward diffusion during the training stage, backdiffusion does not involve parameter updates during runtime; it only performs forward inference calculations of the neural network and is a purely generative process.

[0040] Reverse diffusion process:

[0041] Input: A sequence of noisy images;

[0042] - Processing: Stepwise noise reduction to reconstruct a high-quality image;

[0043] - Output: The reconstructed image;

[0044] Processing steps:

[0045] When generating new data, start with an initial noisy image that follows a standard Gaussian distribution. Starting with a pre-trained denoising neural network, the reverse process of diffusion is applied step by step, that is, the reverse order from t to 0, to finally reconstruct a high-quality image. This reverse process typically requires the use of sampling methods, such as Monte Carlo sampling.

[0046]

[0047] in, It is the inverse function of the activation function, derived from the generated data. The reverse generation of noise data ;

[0048] (3). Sampling process:

[0049] The sampling process is a data augmentation strategy for generating diverse new images. It expands the sample by executing the reverse diffusion process multiple times independently. The difference between it and reverse diffusion is that each sampling starts with a different initial noise and uses the same set of trained denoising models to generate semantically reasonable and detailed images. This process does not require retraining the model. It only needs to repeatedly call the reverse diffusion process to efficiently produce a large amount of synthetic data that conforms to the distribution of the target domain. It is suitable for model training and testing in small sample scenarios.

[0050] Input: Noisy image;

[0051] - Processing: Sample from noise to generate a new image;

[0052] Output: The newly generated image;

[0053] Processing steps:

[0054] To generate new data with diversity, the above reverse process can be performed multiple times, each time from different noise samples. To begin; this can be achieved through multiple samplings;

[0055]

[0056] Where k is the number of samples, each These are all noise samples generated through a reverse diffusion process;

[0057] Save the newly generated sample data.

[0058] The technical solution for generating synthetic defect images to expand the dataset using a pre-trained diffusion model in step c of the data preprocessing step two of this invention is as follows:

[0059] The following are technical solutions for using images to augment datasets:

[0060] (1) Module loading: Load the denoising generation module of the pre-trained diffusion model, which has the ability to reconstruct the mapping of defective images from noise;

[0061] (2) Noise sampling: Randomly sample the initial noise image from the standard Gaussian distribution as input data;

[0062] (3) Reverse denoising generation: The initial noisy image is input into the denoising generation module, and a multi-step reverse denoising iterative process is performed to gradually remove noise and reconstruct a clear synthetic defect image;

[0063] (4) Dataset construction: Repeat steps (2) to (3) multiple times, each time using different random initial noise to obtain a diverse set of synthetic defect samples, and merge them with the original real images to form an expanded training dataset.

[0064] Step a of feature extraction in step three of this invention includes:

[0065] Input: Preprocessed image data;

[0066] Processing: Feature extraction is performed using the Swin Transformer convolutional neural network; the Swin Transformer convolutional neural network adopts a hierarchical window attention mechanism on the basis of the Swin-base network, which segments the image into non-overlapping patches and performs local attention operations on these patches to achieve global perception of the entire image.

[0067] The core architecture of the Swin Transformer network consists of several shift-window-based modules; each module contains two sub-modules:

[0068] Local Window Attention: This module captures local information by performing self-attention operations on local tiles. Its formula is as follows:

[0069]

[0070] Where Q, K, and V represent query, key, and value, respectively. It is the dimension of attention head;

[0071] Global Attention Module: Captures global information by performing global self-attention operations on the entire image. Its formula is expressed as:

[0072]

[0073] The Swin-Base model is used as the baseline for the backbone network.

[0074] Output: Feature vector.

[0075] Step b of defect identification in step four of this invention includes three sub-tasks: classification, target detection, and counting.

[0076] The method for classifying sub-tasks includes:

[0077] Data preprocessing: Prepare a dataset containing training images and corresponding labels, and divide the dataset into training, validation, and test sets; the dataset contains both defective and defect-free images;

[0078] Input image feature vector: Preprocess the dataset, assuming the input image is... Where H is the image height, W is the image width, and C is the number of channels. Taking swin-Base as an example, the image needs to be of a fixed size of (H, W, C) (224, 224, 3).

[0079] Handling fully connected layers and Softmax activation;

[0080] Data partitioning and local window attention: The input image X is divided into non-overlapping patches, each patch being P×P pixels in size. For each patch, self-attention is performed using a local window attention mechanism.

[0081]

[0082] in These represent the query, key, and value, respectively, through patching the graph. Obtained by linear transformation;

[0083] Global attention: Perform global self-attention operation on the entire image X to capture global information.

[0084]

[0085] The shift-window-based module includes local window attention and global attention, as well as a feedforward network:

[0086]

[0087] Classification head: The final feature representation is obtained by stacking shift-window based modules, and then fed into the classifier (usually a linear layer) through global pooling.

[0088]

[0089] In the intermediate layers of the Swin-Transformer network, the feature representations undergo a series of transformations and attention operations to obtain the high-level features learned by the network. These features are stored in a tensor with the shape [B,C,H,W], where B is the batch size, C is the number of channels, and H and W are the height and width of the feature map, respectively.

[0090] Global pooling maps the feature map to a vector of fixed size; global average pooling or global max pooling averages or maxes the feature map in the spatial dimension, resulting in a tensor of size [B,C,1,1].

[0091]

[0092] The tensor obtained from global pooling is flattened into a vector for input into the fully connected layer:

[0093]

[0094] The classification head is a fully connected layer that maps the flattened feature vectors to a space containing the number of classes. The weight matrix W of the fully connected layer has a size of [C, num_classes], where num_classes is the number of classes in the classification task.

[0095]

[0096] Output Defect Categories

[0097] The output of the fully connected layer is transformed into a class probability distribution using the softmax activation function.

[0098]

[0099] The predicted category with the highest probability is output. If the prediction result is "no defect", the process terminates; if it is "defective", the image is passed to the next stage object detection subtask for defect localization.

[0100] The target detection subtask described in this invention includes:

[0101] Input: Data preparation. Prepare the dataset required for the object detection task, including images and corresponding bounding box annotations; each bounding box annotation contains category information. , center coordinates ( , ),width and height In this task, the category information accurately outlines the location of defects. The target detection task dataset consists of all defective images in the dataset after subtask classification.

[0102] Processing: Bounding box regression and construction of the DETR object detection model. DETR consists of a Swin-Base backbone network, an object embedder, and a Transformer decoder component, expressed by the formula:

[0103]

[0104] Where X represents the image feature information after preprocessing;

[0105] Data preprocessing: Image preprocessing includes scaling and normalization operations, implemented using the image processing module of the transform framework; the formula is expressed as:

[0106]

[0107] Object detection: The preprocessed image is input into the DETR object detection model to obtain the predicted object detection results; the output of the DETR object detection model includes the object category probability distribution. Bounding box coordinate prediction Information such as; the formula is expressed as:

[0108]

[0109] Loss function: DETR uses the Hungarian algorithm to establish a one-to-one correspondence between the predicted set of bounding boxes and the actual set of bounding boxes, thereby minimizing the total loss. The set loss consists of two parts:

[0110] 1. Classification Loss:

[0111] For each predicted bounding box, DETR uses the cross-entropy loss function to measure the accuracy of the object detection model in predicting the object category:

[0112]

[0113] Where N is the number of predicted object boxes, It is the probability distribution of the object detection model's category prediction for the i-th object box;

[0114] 2. Box Regression Loss:

[0115] DETR uses a smooth L1 loss function to measure the accuracy of the object detection model in regressing the location of the bounding box;

[0116]

[0117] in, It is the actual position of the i-th target box. It is the predicted location of the i-th bounding box by the object detection model.

[0118] Total Loss: The total loss of DETR is the sum of the classification loss and the bounding box regression loss, and the optimization problem of the Hungarian algorithm is considered in the loss calculation process.

[0119]

[0120] in It is a hyperparameter used to balance the two losses;

[0121] This ensemble loss design enables DETR to perform target classification and location regression simultaneously during end-to-end training, and effectively handles the matching problem between target boxes using the Hungarian algorithm.

[0122] Output defect location and category: The output is to outline the defect location in the image with the crimping quality defects of the transmission line hardware and give the probability of the predicted box.

[0123] The counting of subtasks described in this invention includes:

[0124] Data preparation: The counting dataset includes data on defect location boxes in images with crimping quality defects of transmission line fittings, images without defects, and images with location boxes that accurately define the defect locations; and each image contains the number of defects at the defect locations in the defect images. For example, if an image has 15 defect locations, then the image label has nums=15.

[0125] Processing: The fully connected layer and ReLU activation object detection model is prepared, using a Swin-Base backbone network for subtask counting. When setting the classification head, `num_classes` is set to 20.

[0126] The classification head is a fully connected layer that maps the flattened feature vectors to the space of the number of classes. The weight matrix W of the fully connected layer has a size of [C, num_classes], where num_classes is the number of classes in the classification task.

[0127]

[0128] The output of the fully connected layer is transformed into a class probability distribution using the softmax activation function.

[0129]

[0130] What we get is a counter based on an improved classification network.

[0131] Loss function and optimizer settings: In subtask counting, use the cross-entropy loss function:

[0132]

[0133] in It refers to the actual number of defects, which is the actual number of labels. It is the number of defects predicted and output by the target detection model;

[0134] Output: The number of defects in the output image.

[0135] Compared with the prior art, the technical problem solved and the beneficial effects of the present invention are:

[0136] 1. Expand the dataset. Some images may have a relatively small number of images. During training, common data augmentation methods can be used, such as scaling, cropping, flipping, and rotating. Each data augmentation method can increase the number of images by more than two times, so the final dataset can be at least twice the size of the original dataset, reaching 1000-2000 images.

[0137] 2. From the perspective of deep neural network training, this data is still insufficient. However, manually labeled data is too costly, and the occurrence rate of crimping defects in transmission line fittings is low in actual operation, resulting in a natural scarcity of real defect samples. Continuing to use ordinary data augmentation methods cannot generate realistic defect instances with new morphologies, spatial locations, or combination patterns, making it difficult to meet the basic requirements of deep neural networks for the scale and diversity of training data. This invention introduces a diffusion model as a generative data augmentation method, which can learn the inherent distribution patterns and physical imaging characteristics of defects from a small number of real X-ray images, synthesizing a large number of new defect images with reasonable structure, realistic details, and consistent semantics. This method overcomes the dual limitations of physical acquisition conditions and manual labeling resources, significantly expanding the coverage and complexity of training data. It not only avoids dependence on large-scale labeled data but also effectively improves the convergence stability, recognition accuracy, and generalization ability of multi-task deep networks under varying operating conditions such as different fitting models, X-ray equipment parameter differences, and complex background interference.

[0138] 3. A deep neural network was constructed capable of simultaneously performing three tasks: identification of four types of transmission line fitting leakage samples, detection of leakage-groove defects, and determination of the number of grooves. The performance of the deep neural network was quantitatively evaluated. This invention constructs defect identification as a three-level cascaded task of "classification-detection-counting," simultaneously outputting the existence, spatial location, type, and quantity information of defects in a single model. This design fully aligns with the actual judgment logic of operation and maintenance personnel (first determining presence or absence, then locating and characterizing, and finally assessing the severity), and the generated structured results can be directly integrated with various systems.

[0139] 4. A system for identifying and detecting typical defects in the crimping quality of four types of transmission line fittings was built based on the Python platform. Employing a modular design, it supports fully automated processing from X-ray image input, automatic preprocessing, defect identification to structured report generation. Its code implementation fully leverages Python's rich toolchain in scientific computing and AI, ensuring high development efficiency, good maintainability, and cross-platform compatibility, facilitating rapid deployment and integration into existing IT environments. Attached Figure Description

[0140] Figure 1 A flowchart for expanding the dataset by generating images of compression defects for the diffusion model;

[0141] Figure 2 Here is a flowchart for classifying crimping defects;

[0142] Figure 3 Flowchart for detecting crimp defects;

[0143] Figure 4 Here is a flowchart for counting crimp defects;

[0144] Figure 5 This is a structural diagram of a multi-task cascaded defect identification system. Detailed Implementation

[0145] The steps of the embodiments of the present invention are as follows:

[0146] Step 1. Data Acquisition:

[0147] a. Obtain raw imaging data of power transmission line fittings using X-ray inspection equipment;

[0148] b. The data is transmitted to a computer system via sensors for processing.

[0149] Step 2. Data Preprocessing:

[0150] a. Denoise the original image data using Gaussian filtering or median filtering methods;

[0151] b. Standardize the image to ensure consistency in brightness and contrast;

[0152] c. Diffusion model training: Use the diffusion model to augment data and generate new datasets to expand the dataset.

[0153] Step 3. Feature Extraction:

[0154] a. Use a convolutional neural network (CNN) or an enhanced version of the convolutional neural network, the swin-transformer, to extract features from the preprocessed image;

[0155] b. Select appropriate convolutional and pooling layers to ensure that the extracted features are representative.

[0156] Step 4: Defect Identification

[0157] a. Use a pre-trained defect classification model with a swin-transformer backbone network to perform preliminary identification of fitting crimping quality defects.

[0158] b. The defective images are further processed using a pre-trained defect detection model with a swin-base backbone network, and the defective regions are bounded out using object detection. The final step is to count the total number of bounding boxes, which represents the number of defects.

[0159] Step 5. Output Results:

[0160] a. Visualize the recognition results and generate labeled images;

[0161] bb outputs a detailed report of the defect type, location, and severity (number).

[0162] Description of input, processing, and output for each step:

[0163] (1). Data acquisition and preprocessing: Acquire and clean the raw X-ray images to provide high-quality input for subsequent analysis;

[0164] Input: Raw X-ray imaging data of transmission line fittings;

[0165] - Processing: Denoising, standardization, segmentation;

[0166] - Output: Preprocessed image data;

[0167] (2). Feature extraction: Discriminative feature representations are automatically learned from preprocessed images using deep neural networks;

[0168] - Input: Preprocessed image data;

[0169] - Processing: Feature extraction is performed using a convolutional neural network;

[0170] - Output: Feature vector;

[0171] (3) Defect identification: A multi-task joint model is used to comprehensively analyze features, achieving integrated identification of defect classification, location, and counting;

[0172] - Input: Feature vector;

[0173] - Processing: Pre-trained models perform recognition and post-processing;

[0174] - Output: Identification results, including defect type, location, and severity (number);

[0175] (4) Output Results: The model prediction results are converted into visual images and structured reports for easy manual review and system integration;

[0176] Input: Recognition result;

[0177] - Processing: Generate labeled images and detailed reports;

[0178] - Output: Annotated images, detailed reports.

[0179] Step c, the data preprocessing step in step two of this invention, includes:

[0180] Diffusion Models: Diffusion models are a type of generative model, typically used to generate high-quality image data. The main idea behind these models is to gradually generate the distribution of real data through a multi-step diffusion process.

[0181] Choose a pre-trained diffusion model, such as the pre-trained DALL-E model. (No further training is required. The steps involved in training and using the diffusion model are as follows:)

[0182] (1). Forward diffusion:

[0183] Forward diffusion process:

[0184] Input: The original, defective image;

[0185] - Processing: Gradually add noise to generate a sequence of noisy images;

[0186] - Output: Noisy image sequence.

[0187] Detailed processing steps:

[0188] The key to generating new data is to use a reverse diffusion process to gradually generate samples similar to real data from noisy data. The forward diffusion process (i.e., the noise-adding process simulated during training) starts from real data and gradually adds Gaussian noise. The formula for calculating step t is as follows:

[0189]

[0190] in, It is a real, original image. This represents the data generated at time t. The activation function (usually tanh or sigmoid) ensures that the output value range is reasonable. It is Gaussian noise. It is the diffusion coefficient that increases with increasing t. This indicates that the original image retention factor decreases as t increases;

[0191] (2). Backdiffusion:

[0192] Reverse diffusion process:

[0193] Input: A sequence of noisy images;

[0194] - Processing: Denoising is performed step-by-step multiple times to reconstruct a high-quality image;

[0195] Output: A new image with defects;

[0196] Detailed processing steps:

[0197] When generating new images with defects, from noisy data Initially, the inverse process of diffusion is applied step by step, i.e., in reverse order from t to 0; this inverse process usually requires Monte Carlo sampling.

[0198]

[0199] in, It is the inverse function of the activation function, so from the generated data The reverse generation of noise data .

[0200] (3). Sampling process:

[0201] Input: Noisy image;

[0202] - Processing: Sample from noise to generate a new image;

[0203] - Output: The newly generated image.

[0204] Detailed steps:

[0205] To generate new data with diversity, the above reverse process can be performed multiple times, each time from different noise samples. Initially, this process of repeatedly outputting new images from different noise samples is called the sampling process;

[0206]

[0207] Where k is the number of samples, each These are all noise samples generated through a reverse diffusion process;

[0208] Save the newly generated sample data.

[0209] In step c of the data preprocessing in step two, the technical solution for generating new defective images using a diffusion model pre-trained with defective images to expand the dataset is as follows:

[0210] (1) Module loading: Load the denoising generation module of the diffusion model (e.g., DALL-E) trained on the defective image. This module has the ability to reconstruct the mapping of the defective image from the noise.

[0211] (2) Noise sampling: Randomly sample the initial noise image from the standard Gaussian distribution as input data;

[0212] (3) Reverse denoising generation: The initial noisy image is input into the denoising generation module, and a multi-step reverse denoising iterative process is performed to gradually remove noise and reconstruct a clear new defective image;

[0213] (4) Dataset construction: Repeat steps (2) to (3) multiple times, each time using different random initial noise to obtain a diverse set of synthetic defect samples, and merge them with the original real images to form an expanded training dataset.

[0214] The above method can generate some new high-quality image datasets for training the following three tasks.

[0215] Step c of feature extraction in step three of this invention is as follows:

[0216] - Input: Preprocessed image data;

[0217] Processing: Feature extraction is performed using the Swin Transformer convolutional neural network; the Swin Transformer convolutional neural network adopts a hierarchical window attention mechanism on the basis of the Swin-base network, which segments the image into non-overlapping patches and performs local attention operations on these patches to achieve global perception of the entire image.

[0218] Swin Transformer is a deep learning model based on visual attention mechanisms, suitable for computer vision tasks. It employs a hierarchical window attention mechanism to segment the image into non-overlapping patches and perform local attention operations on these patches, thereby achieving global perception of the entire image.

[0219] The core architecture of the Swin Transformer network consists of several Swin Transformer Blocks. Each Block contains two sub-modules:

[0220] Local Window Attention:

[0221] This part captures local information by performing a self-attention operation on local tiles. The formula can be expressed as:

[0222]

[0223] Where Q, K, and V represent query, key, and value, respectively. It is the dimension of attention.

[0224] Global Attention: Captures global information by performing global self-attention on the entire image. Its formula can be expressed as:

[0225]

[0226] The Swin-Base model is used as the baseline for the backbone network.

[0227] - Output: Feature vector.

[0228] Step b of the defect identification step four in this invention includes three sub-tasks: classification, target detection, and counting.

[0229] The method for classifying subtasks described in this invention is as follows, see Figure 2 The crimping defect classification process:

[0230] Data Preparation and Splitting: Prepare a dataset containing training images and corresponding labels. Ensure the dataset is split into training, validation, and test sets for model training, validation, and evaluation. Divide the dataset into training, validation, and test sets in an 8:1:1 ratio. The dataset includes both defective and defect-free images.

[0231] Input image feature vector: The dataset is preprocessed, assuming the input image to be judged is... Where H is the image height, W is the image width, and C is the number of channels. In this embodiment, Swin-Base is selected as the core backbone network. To adapt to the pre-trained weights and architecture specifications of this network, the input image to be judged is uniformly scaled and cropped to a fixed size (224, 224, 3). Processing: fully connected layer, softmax activation.

[0232] Patch segmentation and local window attention: After entering the Swin-Base network, a linear embedding operation is first performed. The image X to be judged is divided into non-overlapping patches, each patch being P×P pixels in size. The Swin-Transformer performs deep feature extraction through multiple Swin Transformer Blocks, such as Swin-Transformer Block1, Swin-Transformer Block2, ..., Swin-TransformerBlockN. In the local window attention stage of each Block, the system re-divides the patch sequence into several non-overlapping local windows. For the patch set within each window, a self-attention operation is performed independently.

[0233]

[0234] in They represent query, key, and value, respectively, and are used to patch graphs. The linear transformation is obtained as follows: This is the scaling factor.

[0235] Global attention: Perform global self-attention operation on the entire image X to capture global information.

[0236]

[0237] Swin-transformer-Block includes local window attention and global attention, as well as a feedforward network:

[0238]

[0239] Classification Head: The final feature representation is obtained by stacking Swing Transformer Block modules, and then input into the classifier (usually a linear layer) through operations such as global pooling.

[0240]

[0241] Represented as:

[0242] In the intermediate layers of the model (such as the intermediate layers of the Swin Transformer network), the feature representations undergo a series of transformations and attention operations to obtain the high-level features learned by the network. These features are usually stored in a tensor with the shape [B,C,H,W], where B is the batch size, C is the number of channels, and H and W are the height and width of the feature map, respectively.

[0243] To map a feature map to a vector of fixed size, global pooling is typically applied. Global average pooling or global max pooling averages or max-pools the feature map in spatial dimensions, resulting in a tensor of size [B, C, 1, 1].

[0244]

[0245] The tensor obtained from global pooling is flattened into a vector for input into the fully connected layer:

[0246]

[0247] The classification head is typically a fully connected layer that maps the flattened feature vectors to a space representing the number of classes. The weight matrix W of the fully connected layer has a size of [C, num_classes], where num_classes is the number of classes in the classification task, set to 3 in this task, corresponding to class 1, class 2, and class 3.

[0248]

[0249] Output: Image category recognition. Finally, the output of the fully connected layer is transformed into three category probability distributions using the softmax activation function.

[0250]

[0251] Then, the predicted category with the highest probability is output, and it is determined whether the image prediction result is a defect image. If the prediction result is False (corresponding to "no defect"), the process terminates; if it is True (corresponding to "defect"), the image is passed to the next stage target detection subtask for defect localization, and finally the image with the required frame of the transmission line hardware crimping quality defect is obtained.

[0252] The sub-task target detection method described in this invention is as follows, see Figure 3 Flowchart for detecting crimp defects:

[0253] Input: Feature vectors, data preparation. Prepare the dataset required for the object detection task, including images and corresponding bounding box annotations. Each bounding box annotation contains category information. , center coordinates ( , ),width and height Category information is responsible for semantic discrimination in this task, identifying the region as a defect. The object detection task dataset is theoretically a collection of defective images from well-classified photos after the first task's classification.

[0254] Processing: Bounding box regression and construction of the DETR object detection model. DETR consists of a Swin-Base backbone network, an object embedder, and a Transformer decoder component, expressed by the formula:

[0255]

[0256] Here, X is the input image, and Swin_base(X) is the preprocessed image feature set, which refers to the multi-scale semantic feature set extracted by the Swin-Base backbone network. Swin-Base, through its hierarchical structure, downsamples at each level and fuses local and global attention to finally output one or more layers of feature maps, such as {F1, F2, F3, F4}. Each feature map corresponds to semantic features at different resolutions, and these multi-scale feature maps together constitute the "image feature set".

[0257] Data preprocessing: First, the image is preprocessed, including scaling, normalization, etc. This can be achieved using the image processing module of the transform framework. The formula is expressed as:

[0258]

[0259] Object detection: The preprocessed image is input into the DETR object detection model. A multi-scale feature set is extracted via the Swin-Base backbone network, and then fed into the Transformer encoder-decoder for global context modeling. The decoder outputs a set of learnable object query vectors and their corresponding hidden layer representations. These hidden layer representations are fed into the detection head, which is responsible for mapping abstract semantic features to specific detection results. Each detection head processes one object query in parallel, and the outputs of all detection heads together constitute a "set of predicted bounding boxes," including the object category probability distribution. Bounding box coordinate prediction Information such as... The formula is expressed as:

[0260]

[0261] Predicted bounding box classification:

[0262] 1. Correct predicted bounding boxes (with target defects): These are those boxes that are successfully matched to a real defect using the Hungarian algorithm, and whose Intersection over Union (IoU) ≥ τ (usually τ = 0.5) and class prediction probability. The highest-scoring predicted bounding box matches the true category. These boxes represent defects accurately identified by the model and will be retained as final output during the inference phase.

[0263] 2. Incorrect predicted boxes (no target defect) include the following two situations: unmatched boxes that were not assigned to any ground truth boxes by the Hungarian algorithm (i.e., false background detection); and low-quality matched boxes that were matched but had IoU < τ or were classified incorrectly (i.e., inaccurate localization or misclassification).

[0264] Loss Function: DETR introduces the concept of ensemble loss to handle the issue of an irregular set of bounding boxes in object detection tasks. The core idea of ​​ensemble loss is to model the matching problem between the set of predicted bounding boxes and the set of actual bounding boxes as a minimization problem.

[0265] Specifically, DETR uses the Hungarian algorithm to establish a one-to-one correspondence between the predicted set of bounding boxes and the actual set of bounding boxes, thereby minimizing the total loss. The set loss consists of two parts:

[0266] 1. Classification Loss:

[0267] For each predicted target box, DETR uses the cross-entropy loss function to measure the model's accuracy in predicting the target category.

[0268]

[0269] Where N is the number of predicted object boxes, It is the probability distribution of the model's class prediction for the i-th target box.

[0270] 2. Box Regression Loss:

[0271] DETR uses a smooth L1 loss function to measure the model's regression accuracy for the bounding box location.

[0272]

[0273] in, It is the actual position of the i-th target box. It is the model's predicted location of the i-th bounding box.

[0274] Total DETR loss:

[0275] The total loss of DETR is the sum of the classification loss and the bounding box regression loss, and the optimization problem of the Hungarian algorithm is considered in the loss calculation process;

[0276]

[0277] in It is a hyperparameter used to balance the two losses.

[0278] This ensemble loss design enables DETR to perform both object classification and location regression during end-to-end training, and effectively handles the matching problem between object boxes using the Hungarian algorithm.

[0279] Output: Defect location and category: The final output is to outline the defect location in the image with the quality defects of the crimping of the transmission line hardware and give the probability of the predicted box.

[0280] The method for counting subtasks described in this invention is as follows: see Figure 4 Flowchart for counting crimp defects.

[0281] Subtask counting and subtask classification are somewhat similar. By replacing the number of counts with the number of categories, we can understand the basic model of subtask classification by improving the final classification header and changing num_classes to the maximum value of the count. In subtask counting, the maximum value of the count is set to 20.

[0282] Input: Feature vector data preparation. The dataset for the subtask counting consists of defect images containing only targets (with a minimum of 1 defect target). These defect images serve as model input, helping to focus on learning the features of the defect region and improve counting accuracy. Furthermore, each image contains the number of defects at the defect locations in the defect images. For example, if an image has 15 defect locations, then the image label has nums=15.

[0283] Processing: Preparation of the fully connected layer, ReLU activation object detection model. As described in the subtask classification, a counting task using a Swin-Base backbone network is used. After the backbone network, a lightweight counter module is added. This module is modified from a traditional classification head, with num_classes in the classification head set to 20.

[0284] The classification head is typically a fully connected layer that maps the flattened feature vectors to a space representing the number of classes. The weight matrix W of the fully connected layer has a size of [C, num_classes], where num_classes is the number of classes in the classification task, set to 20 in this case.

[0285]

[0286] Finally, the output of the fully connected layer is transformed into a class probability distribution using the softmax activation function:

[0287]

[0288] The final result is a counter based on an improved classification network.

[0289] Loss function and optimizer settings: In subtask counting, use the cross-entropy loss function:

[0290]

[0291] in It refers to the actual number of defects, which is the actual number of labels. It is the number of defects predicted by the target detection model.

[0292] Output: An image showing the number of defects. If the model outputs that category 15 has the highest probability, then the image is determined to contain 15 defects.

[0293] Each subtask can be trained individually or together, such as Figure 5 As shown in the structure diagram of the multi-task cascaded defect recognition system, after individual training, the image obtained from the previous subtask still needs to be used as the input for the next subtask in order to finally obtain the image to be recognized (1. There is leakage pressure, 2. Mark the location of leakage pressure, 3. Calculate the number of leakage pressure locations).

[0294] Detailed explanation of each task:

[0295] Task 1: Use the sorting head to determine if there is any leakage pressure.

[0296] A lightweight Transformer encoder is used to aggregate global features, and a classification head (fully connected layer + Softmax) is connected at the end to output a binary probability distribution P (with leakage pressure). If it is determined to be "with leakage pressure", subsequent tasks are triggered.

[0297] Task 2: Detect the location of the pressure leak marked on the head.

[0298] The DETR structure (i.e., Swin-Base + Transformer Decoder + Detection Head) is adopted to directly output N candidate boxes and their class probabilities. Valid leaky boxes are selected by Hungarian matching, and their coordinates and confidence scores are labeled to achieve "marking the location of leaky boxes".

[0299] Task 3: Use a counter to count the number of leakage locations.

[0300] Reuse the classification header structure from Task 1, but change the output dimension from 2 to the maximum number of defects (e.g., 20) to form a Count counter. Its output is a discrete probability distribution, and the total number of predicted defects is obtained by taking argmax, which is "calculating the number of leakage locations".

[0301] Each subtask is trained separately, and testing after training is complete requires a step-by-step approach:

[0302] For example, in the first subtask: the input to the test is an image (the image to be judged to determine if there is a leak). After inputting it into Task 1 for testing, if the output shows a leak, then this image (with leak) is input into Task 2 for testing. If the final output from Task 1 is an image without leaks, then there is no need to continue with Task 2. Task 2 performs object detection on the image (with leaks), marks the locations with leaks, outlines the locations of leaks, and gives the predicted probability. After obtaining this image (with leaks and locations), it is input into the model of Task 3 for testing, and finally an image is obtained that has leaks, marked leak locations, and the number of leak locations.

[0303] When training multiple sub-tasks together, the loss functions of the three sub-task models need to be adjusted in terms of weights. The specific adjustment strategy needs to be adjusted according to the training effect.

[0304] The training strategy is as follows: (The rest of the text appears to be incomplete and requires further context.) Figure 5 As shown, the image to be judged is input into the Backbone Swin-Base network. Task 1 involves adding a transitional Transformer architecture after the Swin-Base network, followed by a classification head that outputs the classification of the image from Task 1. Images identified as having leakage pressure defects in Task 1 then become the subsequent input for Task 2, inputting after the DETR structure and before the object detection head. After passing through the detection head, the locations of leakage pressure defects are marked. The outputs of the two sub-tasks are then used as the input for Task 3, inputting before the count counter. Finally, the number of leakage pressure locations is calculated using the counter. By training three sub-tasks simultaneously, the functionality of all three sub-tasks can be achieved within a single network model.

[0305] The final total loss can be expressed by the following formula:

[0306]

[0307] Loss1: The loss for classification subtask 1 (such as cross-entropy loss), used to determine whether the image has defects;

[0308] TotalLoss2: The total loss of object detection subtask 2 (usually includes classification loss + bounding box regression loss, such as the Hungarian matching loss in DETR).

[0309] Loss3: Loss for the counting subtask 3 (e.g., if counting is treated as a 20-class classification problem, use cross-entropy loss);

[0310] α, β, γ: Adjustable hyperparameters (weighting coefficients) used to balance the contribution of each task to the total gradient and prevent one task from dominating the training.

[0311] The reasons for setting the total loss are as follows:

[0312] Unified optimization objective: The three subtasks share a portion of the network (such as the Swin-Base backbone), but each has its own independent output header. The total loss ensures that a single backpropagation process can update all parameters simultaneously.

[0313] Task collaboration and regularization: Classification results guide detection (e.g., only detecting "defective" images), and detection results assist in counting (e.g., the number of defective boxes ≈ the count value), forming semantic consistency constraints and improving overall performance.

[0314] The overall logic of this invention is as follows: a classification task determines whether there is a crimping defect in the X-ray image; if so, the classification result is used as prior information to input the target detection task to accurately locate the defect and identify its type; finally, based on the defect instances detected, a counting task counts the number of defects, thus realizing a complete defect cognition chain of "presence → location and type → quantity".

[0315] To address the problem of insufficient datasets in existing technologies, this invention uses a diffusion model to further enhance the data, generating new images so that the final dataset size reaches approximately 3000-4000 images. This allows for more significant improvements in model training, resulting in higher model accuracy and better generalization.

Claims

1. A method for identifying quality defects in the crimping of transmission line fittings based on X-ray inspection imaging, characterized in that... Includes the following steps: Step 1: Data Collection a. Obtain raw imaging data of power transmission line fittings using X-ray inspection equipment; b. The data is transmitted from the sensors to the computer system for processing; Step 2: Data Preprocessing a. Denoise the original image data using Gaussian filtering or median filtering methods; b. Standardize the image to ensure consistent brightness and contrast; c. Diffusion model training: Using a pre-trained diffusion model to generate synthetic defect images to expand the dataset; Step 3: Feature Extraction a. Use a convolutional neural network to extract features from the preprocessed image; b. Select convolutional layers and pooling layers; Step 4: Defect Identification a. Use a pre-trained defect classification model with a swin-transformer backbone network to perform preliminary identification of fitting crimping quality defects; b. The defective images identified are further processed using a pre-trained defect detection model with a swin-base backbone network (DERT), and the defective regions are bounded out using object detection. Step 5. Output Results: a. Visualize the recognition results and generate labeled images; b. Output a detailed report of the defect type, location, and severity.

2. The method for identifying quality defects in transmission line fittings based on X-ray inspection and imaging according to claim 1, characterized in that, The steps involved in using the diffusion model in step c of the data preprocessing in step two are as follows: (1). Forward diffusion: The forward diffusion process starts with the acquired original defect image, injects Gaussian noise sequentially through multiple time steps, and finally outputs a complete noise evolution sequence. Forward diffusion process: - Input: Raw image data; - Processing: Gradually add noise to generate a sequence of noisy images; Output: Noisy image sequence; Processing steps: The forward diffusion process, which is the noise addition process simulated during the training phase, starts from the real data and gradually adds Gaussian noise. The formula for calculating the noise at time t is as follows: ； in, It is a real, original image. This represents the data generated at time t. It is an activation function. It is Gaussian noise. It is the diffusion coefficient that increases with increasing t. This indicates that the original image retention factor decreases as t increases; (2). Backdiffusion: The reverse diffusion process uses a pre-trained denoising network as its core. After providing initial noise, it performs reverse time step operations to estimate and remove the noise components introduced at each stage, ultimately generating new samples that conform to the distribution of real data. Reverse diffusion process: Input: A sequence of noisy images; - Processing: Gradual denoising to reconstruct a high-quality image; - Output: The reconstructed image; Processing steps: When generating new data, start with an initial noisy image that follows a standard Gaussian distribution. Starting with a pre-trained denoising neural network, the reverse process of diffusion—that is, the reverse order from t to 0—is applied step by step to reconstruct a high-quality image. This reverse process requires Monte Carlo sampling. ； in, It is the inverse function of the activation function, derived from the generated data. The reverse generation of noise data ; (3). Sampling process: Input: Noisy image; - Processing: Sample from noise to generate a new image; Output: The newly generated image; Processing steps: To generate new data with diversity, the above reverse process is repeated multiple times, each time starting from different noise samples. To begin; this is achieved through multiple samplings; ； Where k is the number of samples, each These are all noise samples generated through a reverse diffusion process; Save the newly generated sample data.

3. The method for identifying quality defects in transmission line fittings based on X-ray inspection and imaging according to claim 1, characterized in that, In step c of the data preprocessing in step two, the technical solution for generating synthetic defect images using a pre-trained diffusion model to expand the dataset is as follows: (1) Module loading: Load the denoising generation module of the pre-trained diffusion model, which has the ability to reconstruct the mapping of defective images from noise; (2) Noise sampling: Randomly sample the initial noise image from the standard Gaussian distribution as input data; (3) Reverse denoising generation: The initial noisy image is input into the denoising generation module, and a multi-step reverse denoising iterative process is performed to gradually remove noise and reconstruct a clear synthetic defect image; (4) Dataset construction: Repeat steps (2) to (3) multiple times, each time using different random initial noise to obtain a diverse set of synthetic defect samples, and merge them with the original real images to form an expanded training dataset.

4. The method for identifying quality defects in transmission line fittings based on X-ray inspection and imaging according to claim 1, characterized in that: Step a in the feature extraction step three includes: Input: Preprocessed image data; Processing: Feature extraction is performed using the Swin Transformer convolutional neural network; the Swin Transformer convolutional neural network adopts a hierarchical window attention mechanism on the basis of the Swin-base network, which segments the image into non-overlapping patches and performs local attention operations on these patches to achieve global perception of the entire image. The core architecture of the Swin Transformer network consists of several shift-window-based modules; each module contains two sub-modules: Local Window Attention Module: Captures local information by performing self-attention operations on local tiles. Its formula is expressed as: ； Where Q, K, and V represent query, key, and value, respectively. It is the dimension of attention head; Global Attention Module: Captures global information by performing global self-attention operations on the entire image. Its formula is expressed as: ； Output: Feature vector.

5. The method for identifying quality defects in transmission line fittings based on X-ray inspection and imaging according to claim 1, characterized in that: Step b in the defect identification step four includes three sub-tasks: classification, target detection, and counting.

6. The method for identifying quality defects in transmission line fittings based on X-ray inspection and imaging according to claim 5, characterized in that... The method for classifying subtasks includes: Data preprocessing: Prepare a dataset containing training images and corresponding labels, and divide the dataset into training set, validation set and test set. The dataset contains images with defects and without defects. Input image feature vector: Preprocess the dataset, assuming the input image is... , where H is the image height, W is the image width, and C is the number of channels; Handling fully connected layers and Softmax activation; Data partitioning and local window attention: The input image X is divided into non-overlapping patches, each patch being P×P pixels in size. For each patch, self-attention is performed using a local window attention mechanism. ； in These represent the query, key, and value, respectively, through patching the graph. Obtained by linear transformation; Global attention: Perform global self-attention operation on the entire image X to capture global information. ； The shift-window-based module includes local window attention and global attention, as well as a feedforward neural network: ； Classification Head: The final feature representation is obtained by stacking modules based on shift windows, and then input into the classifier through global pooling. ； In the intermediate layers of the Swin-Transformer network, the feature representations undergo a series of transformations and attention operations to obtain the high-level features learned by the network. These features are stored in a tensor with the shape [B,C,H,W], where B is the batch size, C is the number of channels, and H and W are the height and width of the feature map, respectively. Global pooling maps the feature map to a vector of fixed size; global average pooling or global max pooling averages or maxes the feature map in the spatial dimension to obtain a tensor of size [B,C,1,1]. ； The tensor obtained from global pooling is flattened into a vector for input into the fully connected layer: ； The classification head is a fully connected layer that maps the flattened feature vectors to a space containing the number of classes. The weight matrix W of the fully connected layer has a size of [C, num_classes], where num_classes is the number of classes in the classification task. ； Output Defect Categories The output of the fully connected layer is transformed into a class probability distribution using the softmax activation function. ； The predicted category with the highest probability is output. If the prediction result is "no defect", the process terminates; if it is "defective", the image is passed to the next stage object detection subtask for defect localization.

7. The method for identifying transmission line fitting crimping quality defects based on X-ray detection imaging according to claim 5, characterized in that: The sub-task target detection includes: Input: Data preparation. Prepare the dataset required for the object detection task, including images and corresponding bounding box annotations; each bounding box annotation contains category information. , center coordinates ( , ),width and height In this task, the category information accurately outlines the location of defects. The target detection task dataset consists of all defective images in the dataset after subtask classification. Processing: Bounding box regression and construction of the DETR object detection model. DETR consists of a Swin-Base backbone network, an object embedder, and a Transformer decoder component, expressed by the formula: ； Where X represents the image feature information after preprocessing; Data preprocessing: Image preprocessing includes scaling and normalization operations, implemented using the image processing module of the transform framework; the formula is expressed as: ； Object detection: The preprocessed image is input into the DETR object detection model to obtain the predicted object detection results; the output of the DETR object detection model includes the object category probability distribution. Bounding box coordinate prediction Information such as; the formula is expressed as: 。 8. The method for identifying quality defects in transmission line fittings based on X-ray inspection and imaging according to claim 7, characterized in that: The sub-task object detection also includes: Loss Function: DETR introduces ensemble loss, modeling the matching problem between the set of predicted target boxes and the set of actual target boxes as a minimization problem. DETR uses the Hungarian algorithm to establish a one-to-one correspondence between the set of predicted target boxes and the set of actual target boxes, minimizing the total loss. The ensemble loss consists of two parts:

1. Classification Loss: For each predicted bounding box, DETR uses the cross-entropy loss function to measure the accuracy of the object detection model in predicting the object category; ； Where N is the number of predicted object boxes, It is the probability distribution of the object detection model's category prediction for the i-th object box; 2. Box Regression Loss: DETR uses a smooth L1 loss function to measure the accuracy of the object detection model in regressing the location of the bounding box; ； in, It is the actual position of the i-th target box. It is the predicted position of the i-th target box by the object detection model; Total loss: The total loss of DETR is the sum of the classification loss and the bounding box regression loss, and the optimization problem of the Hungarian algorithm is considered in the loss calculation process; ； in It is a hyperparameter used to balance the two losses; Output defect location and category: The output is to outline the defect location in the image with the crimping quality defects of the transmission line hardware and give the probability of the predicted box.

9. The method for identifying quality defects in transmission line fittings based on X-ray inspection and imaging according to claim 5, characterized in that: The subtask count includes: Data preparation: The counting dataset includes data on defect location boxes in images with images of transmission line fitting crimping quality defects, images without defects, and images with location boxes that accurately outline the defect locations; and for each image, there is the number of defects at the defect locations in the defect images. Processing: Fully connected layers, ReLU activation, object detection model preparation, using a Swin-Base backbone network for subtask counting: The classification head is a fully connected layer that maps the flattened feature vectors to the space of the number of classes. The weight matrix W of the fully connected layer has a size of [C, num_classes], where num_classes is the number of classes in the classification task. ； The output of the fully connected layer is transformed into a class probability distribution using the softmax activation function. ； We obtain a counter based on an improved classification network; Loss function and optimizer settings: In subtask counting, use the cross-entropy loss function: ； in It refers to the actual number of defects, i.e., the actual number of defects. It is the number of defects predicted and output by the target detection model; Output: The number of defects in the output image.

10. The method for identifying quality defects in transmission line fittings based on X-ray inspection and imaging according to claim 9, characterized in that: In the process, a swin-Base backbone network is used to count subtasks, and num_classes is set to 20 when setting the classification header.

Images