An equipment defect sample enhancement and identification system based on a generative adversarial network

Abstract

The application discloses an equipment defect sample enhancement and recognition system based on a generative adversarial network, and belongs to the technical field of equipment defect detection. The system comprises a data preprocessing module, a sample enhancement module, an enhanced data set construction module, a model training module and a defect detection module. The improved conditional generative adversarial network with a channel-space attention mechanism and a multi-scale feature fusion structure is introduced to realize batch generation of equipment defect images, and a composite loss function is used to complete the adversarial training of the improved conditional generative adversarial network. Meanwhile, a defect feature attention fusion module is embedded into a YOLOv8 model, and a self-adaptive weighted loss function is used to train the model based on an enhanced data image set, so that a defect recognition model is obtained. The application can effectively expand the equipment defect sample quantity, solve the data imbalance problem, improve the equipment defect recognition precision, and is suitable for the equipment defect detection scene.

Description

Technical Field

[0001] This invention relates to the field of equipment defect detection technology, and more specifically to an equipment defect sample enhancement and identification system based on generative adversarial networks. Background Technology

[0002] Equipment defect detection is a crucial step in ensuring product quality and equipment operational safety in manufacturing, aerospace, and other fields. Current mainstream detection solutions largely rely on deep learning-based target detection models for automated detection. However, the performance of these models is highly dependent on sufficient and balanced training samples. In real-world industrial scenarios, collecting equipment defect image samples is difficult and costly, often resulting in sample scarcity and imbalanced data distribution. In particular, the number of rare defect samples is extremely limited, severely restricting the training effectiveness and recognition accuracy of detection models.

[0003] In existing technologies, some solutions use conditional generative adversarial networks to enhance defect samples, but the generated samples suffer from low fidelity and ambiguous defect features, and the network structure is not optimized for equipment defect detection scenarios. When using general target detection models such as YOLOv8 directly for defect identification, the models have difficulty effectively capturing the features of minor and rare defects in equipment, which can easily lead to missed detections and false detections. At the same time, they lack adaptation design for imbalanced datasets and cannot meet the high accuracy and high stability requirements of industrial equipment defect detection. Summary of the Invention

[0004] This invention provides an equipment defect sample enhancement and identification system based on generative adversarial networks to solve the problems of scarce equipment defect samples, unbalanced data distribution, poor sample quality of existing solutions, and low defect identification accuracy of general detection models in industrial scenarios.

[0005] To address the aforementioned problems, according to one aspect of the present invention, an equipment defect sample enhancement and identification system based on generative adversarial networks is disclosed, comprising a data preprocessing module, a sample enhancement module, an enhanced dataset construction module, a model training module, and a defect detection module. The data preprocessing module is used to denoise, standardize, label defects, and extract features from the equipment image sample set, generating a standardized equipment image sample set and a conditional coding vector set. The sample augmentation module is used to generate a synthetic defect image sample set by combining a conditional coding vector set and a standardized equipment image sample set with a composite loss function and performing adversarial training through an improved conditional generative adversarial network. An enhanced dataset construction module is used to fuse a synthetic defect image sample set and a standardized equipment image sample set to obtain a fused data image set, and to perform data balancing correction on the fused data image set to obtain an enhanced data image set; The model training module is used to train the improved YOLOv8 model based on the enhanced data image set to obtain the defect recognition model. The defect detection module is used to denoise and standardize the image of the equipment to be inspected before inputting it into the defect recognition model to output the defect detection result.

[0006] Preferably, the equipment image sample subset A subset of normal images consisting of several normal equipment images Composed of, and satisfying and Among them, for the defective image subset Each equipment defect image in Adaptive Gaussian filtering for noise reduction, standardization, defect annotation, and feature extraction are performed sequentially to obtain standardized defect images with one-to-one correspondence. and conditional encoding vector The conditional coding vector This is a concatenated and fused vector of the one-hot encoded vector of the defect label and the defect feature vector; for a subset of normal images Normal image of each piece of equipment Adaptive Gaussian filtering for noise reduction and image normalization are performed sequentially to obtain a one-to-one normalized image. Ultimately, all standardized defect images and standardized normal images Standardized equipment image sample set A conditional encoding vector corresponding to each of all equipment defect images. Constructing a conditional encoding vector set .

[0007] Preferably, the denoising process employs an adaptive Gaussian filtering algorithm to design a dynamic filtering kernel for the subtle features of both defective and normal equipment images, avoiding the loss of details during noise removal. The filtering kernel function of the dynamic filtering kernel is as follows: In the formula, ( , () represents the coordinates of the filter kernel center. This is an adaptive variance dynamically adjusted based on the local gray-level variance of the image within a 3×3 neighborhood window, for regions where the local gray-level variance is ≥15. Regions with a local gray-level variance < 15, defined as 0.8-1.2. Take a value of 0.3-0.7. The filter kernel function is obtained through convolution operations. Achieve noise reduction To equip defective images or equip normal images, pixel values. The pixel values ​​are the denoised equipment defect images or normal equipment images; the standardization process includes image size unification and image pixel value normalization, adjusting all denoised equipment defect images or normal equipment images to a uniform size of 256×256×3, and normalizing the pixel values ​​of the denoised equipment defect images or normal equipment images to the [0,1] interval. The normalization calculation formula is: ,in, These are the normalized pixel values. , These are the minimum and maximum pixel values ​​of the denoised equipment defect image or the normal equipment image, respectively; the defect annotation is performed manually on the standardized equipment defect image using the LabelMe tool, indicating the defect type. Defect location and defect size To generate defect labels The feature extraction involves using the SIFT algorithm to extract defect features from the standardized equipment defect image, generating a 128-dimensional defect feature vector. .

[0008] Preferably, the improved conditional generative adversarial network is a conditional generative adversarial network that introduces a channel-space attention mechanism and a multi-scale feature fusion structure to solve the problems of low detail fidelity and insufficient defect feature representation in equipment defect images generated by the original conditional generative adversarial network. It includes a generator and a discriminator, wherein the generator includes an encoder, a channel-space attention fusion module and a decoder, and the discriminator includes an input layer, a multi-scale feature extraction branch, a feature fusion layer and an output layer. The encoder is used to process the input 128-dimensional random noise that conforms to a standard Gaussian distribution. Conditional encoding vector of corresponding equipment defect image Perform multi-level convolutional feature extraction and dimensionality compression to output encoded feature maps. The encoder consists of four cascaded convolutional layers, batch normalized (BN) layers, and Leaky ReLU activation functions. Each convolutional layer has a 3×3 kernel size, a stride of 2, and padding of 1. The negative slope of the Leaky ReLU activation function is fixed at 0.2. The mathematical expression for the multi-level convolutional feature extraction and dimensionality compression process is as follows: ; in, This indicates a vector channel dimension concatenation operation. These are intermediate feature maps output sequentially during the multi-level convolution feature extraction and dimensionality compression process, with dimensions of 128×128×64, 64×64×128, 32×32×256, and 16×16×512, respectively. The channel-spatial attention fusion module is used to sequentially apply channel attention weighting and spatial attention weighting to the encoded feature map, outputting an attention-enhanced feature map. The channel-spatial attention fusion module consists of a cascaded channel attention submodule (CAM) and a spatial attention submodule (SAM); wherein the channel attention submodule (CAM) applies channel attention weighting and spatial attention weighting to the input encoded feature map. After performing a global average pooling (GAP) operation, the channel attention weights are generated by passing through two fully connected (FC) layers and then through a sigmoid activation function. The mathematical formula for calculating channel attention weights is: Channel attention weights With encoding feature map Element-wise multiplication yields the channel-enhanced feature map. , For element-wise multiplication; the Spatial Attention Submodule (SAM) enhances the channel feature map. Global average pooling (GAP) and global max pooling (GMP) are performed separately along the channel dimension. The results of the two pooling operations are concatenated by channel and then passed through a 3×3 convolutional layer and a sigmoid activation function to generate spatial attention weights. The mathematical formula for calculating spatial attention weights is: In the formula, For channel-dimensional splicing operations; spatial attention weights With channel enhancement feature map Element-wise multiplication is performed to finally output the attention-enhanced feature map. ; The decoder is used to perform multi-level transposed convolution upsampling on the attention-enhanced feature map and then decode it to generate a synthetic defect image with the same image size specifications as the standardized defect image. The decoder consists of three cascaded transposed convolutional layers, batch normalized (BN) layers, and ReLU activation functions, as well as one output transposed convolutional layer and Tanh activation function. Each transposed convolutional layer has a 3×3 kernel size, a stride of 2, and padding of 1. The mathematical expression for the multi-level transposed convolutional upsampling and decoding process is as follows: ; in, These are the upsampled feature maps output sequentially during the multi-level transposed convolution upsampling process; The input layer is used to concatenate the input standardized defect image or synthetic defect image with the corresponding conditional coding vector along the channel dimension, and output the discriminator input feature map. Specifically, the conditional coding vector is processed through a 1×1 convolutional layer. Mapped to a conditional feature map of the same size as the input image. Then input the standardized defect image Or synthesized defect images With conditional feature map Perform channel-dimensional concatenation to output the discriminator's input feature map. The mathematical formula for calculating the input feature map of the discriminator is: ; The multi-scale feature extraction branch is used to extract fine-grained defect features, medium-scale defect features, and global defect features from the discriminator input feature map through three parallel convolutional branches of different scales, and output a multi-scale defect feature map. The kernel size of the three parallel convolutional branches of different scales is 3×3, the stride is 1, and the padding is 1. Each convolutional layer is followed by a batch normalization (BN) layer and a LeakyReLU activation function. The negative slope of the LeakyReLU activation function is fixed at 0.2. Its structure is as follows: the convolutional branch for extracting fine-grained features consists of two cascaded convolutional layers, used to capture pixel-level fine-grained defect features such as minor scratches and microcracks on the surface of industrial equipment, and output a fine-grained defect feature map. The convolutional branch for extracting medium-scale defect features consists of three cascaded convolutional layers, used to capture medium-scale defect features such as pitting, pitting, and localized corrosion on the surface of industrial equipment, and outputs a medium-scale defect feature map. The convolutional branch for extracting global defect features consists of four cascaded convolutional layers, used to capture global defect features such as large-area deformation and corrosion on the equipment surface, as well as background features, and outputs a global defect feature map. The final output multi-scale defect feature map is as follows: , and A set; The feature fusion layer is used to perform channel stitching and convolutional dimensionality reduction fusion on multi-scale defect feature maps, outputting a fused feature map. Specifically, the process involves fusing the fine-grained defect feature map... Medium-scale defect feature map Global Defect Feature Map Channel-dimensional concatenation is performed to obtain a high-dimensional feature map. Then, a 1×1 convolutional layer is used to perform dimensionality reduction and feature fusion on the high-dimensional feature map, eliminating channel redundancy and outputting a fused feature map with uniform dimensions. ; The output layer is used to perform dimensionality mapping on the fused feature map using a fully connected layer, followed by a Sigmoid activation function to output the image authenticity identification result. Specifically, the process involves mapping the fused feature map... After flattening, the dimension mapping is completed through two fully connected layers (FC), and then the final true / false identification result is output through the Sigmoid activation function. The mathematical formula for calculating the authenticity verification result is as follows: ,in, The input discriminator is either a standardized defect image or a synthetic defect image. The closer the output value is to 1, the higher the probability that the input image is a standardized defect image; the closer the output value is to 0, the higher the probability that the input image is a synthetic defect image.

[0009] Preferably, the composite loss function is a weighted composite loss function that integrates adversarial loss, perceptual loss, and defect feature loss. This function is used to constrain the adversarial training process of the improved conditional generative adversarial network (GAN), addressing the problems that easily arise when the original GAN ​​uses only a single adversarial loss, such as distorted defect details in the generated synthetic defect images, lack of prominence of core defect features, and poor consistency with real defect image features. The mathematical formula for the composite loss function is: ; in, This is the total loss value. To combat the losses, In order to perceive loss, For defect feature loss; , , The optimal values ​​for the weighting coefficients corresponding to each loss were determined through cross-validation experiments. , and It can be manually adjusted within ±0.2 depending on the type of defect; The adversarial loss is the cross-entropy loss constructed during adversarial training of the improved conditional generative adversarial network. It is used to measure the difference in the distribution of synthetic defect images and real defect images on the discriminator output, and is divided into discriminator loss. With generator loss The formula for calculating the counter-loss is: ; ; ; in, To standardize the joint distribution of defect images and corresponding conditional coding vectors of real samples, Let z be the probability distribution of the random noise input to the generator, and z follows a standard Gaussian distribution. , The sample distribution of the conditional coding vector; To improve the generator mapping function of conditional generative adversarial networks, For the discriminator mapping function, This is the output value of the discriminator for distinguishing between genuine and fake images of real standardized defects. The discriminator outputs a value to distinguish the authenticity of the synthesized defect image output by the generator. The perceptual loss is based on the Euclidean distance loss between the high-level visual features of the real defect image and the synthetic defect image extracted by the pre-trained VGG16 network. It is used to constrain the visual semantic consistency between the synthetic defect image and the real defect image, and improve the overall visual fidelity of the synthetic defect image. The formula for calculating the perceptual loss is as follows: ; in, The feature extraction function for the i-th convolutional layer of the pre-trained VGG16 network is L, which is the number of convolutional layers selected, with a preferred value of 3. The high-level visual features output from the 2nd, 4th, and 6th convolutional layers of the VGG16 network are selected respectively, and C, H, and W are the number of channels, height, and width, respectively. The L2 norm is the Euclidean distance. The defect feature loss is based on the cosine similarity loss between the defect feature vectors of the real defect image and the synthetic defect image extracted by the SIFT algorithm. This loss is used to constrain the consistency between the synthetic and real defect images, ensuring that the defect type and morphological features of the synthetic defect image match those of the real defect image. The formula for calculating the defect feature loss is as follows: ; in, This is the defect feature vector of the real defect image extracted by the SIFT algorithm. These are the defect feature vectors of the synthetic defect images extracted using the same SIFT algorithm. This is a vector dot product operation.

[0010] Preferably, the adversarial training method is as follows: S1. Initialize the network parameters of the generator and discriminator using the He normal initialization method, and set the training hyperparameters, including batch size, initial learning rate and maximum number of training iterations; S2. Randomly select real defect images of a preset batch size from the standardized equipment image sample set and the conditional coding vector set. With the corresponding conditional encoding vector The real defect image is the standardized defect image. Simultaneously, random noise z conforming to a standard Gaussian distribution N(0,1) is generated. This random noise z is then combined with the conditional coding vector. After channel-dimensional stitching, the images are input into generator G to obtain batch-synthesized defect images. ; S3. Fix the network parameters of the generator, and input batches of real defect images and their corresponding conditional coding vectors, as well as batches of synthetic defect images and their corresponding conditional coding vectors, into the discriminator. Calculate the discriminator loss value based on adversarial loss. The network parameters of the discriminator are updated through backpropagation using the Ada optimizer; S4. Fix the network parameters of the discriminator, input the batch-synthesized defect images and corresponding conditional coding vectors into the discriminator, and calculate the total loss value based on the composite loss function. Based on this, the network parameters of the generator are updated through backpropagation using the Adam optimizer; S5. Repeat S2-S4 for multiple iterations of training. After each iteration, calculate the mean SSIM (structural similarity score) of the generated batch of synthetic defect images. and average PSNR (Peak Signal-to-Noise Ratio) When the mean SSIM value is ≥0.85 and the mean PSNR value is ≥30dB, or when the iterative training reaches the preset maximum number of iterations, the iterative training is terminated to obtain the optimal generator. Based on this, the standardized defect images are input into the optimal generator to generate synthetic defect images in batches, so as to form a synthetic defect image sample set. The formulas for calculating the mean SSIM and mean PSNR are as follows: ; Where N is the total number of images in a single batch of synthesized defect images. Let be the structural similarity coefficient between the k-th synthetic defect image and its paired real defect image within a single batch of synthetic defect images. Let $\frac{k}{k}$ be the peak signal-to-noise ratio (PSNR) of the k-th synthetic defect image and its paired real defect image within a single batch of synthetic defect images. The The calculation formula is: ; in, , These are the pixel mean values ​​of the paired real defect image and the synthetic defect image, respectively. , These represent the pixel variances of the paired real defect images and the synthetic defect images, respectively. The pixel covariance between the paired real defect image and the synthetic defect image. and To avoid stability constants with a denominator of 0, This is the maximum pixel value, which is 1. The The calculation formula is: ; in, The mean square error between the paired real defect image and the synthetic defect image; The The calculation formula is: ; in, , These are the image height and image width, respectively. , , respectively, are the pixel values ​​at coordinates (i,j) of the paired real defect image and the synthetic defect image.

[0011] Preferably, the fused image data set includes standardized normal images, standardized defect images, and synthetic defect images; the data balancing correction is as follows: the proportion of each defect type in the fused image data set is statistically analyzed. If the proportion of each defect type is less than 5%, a synthetic defect image of the corresponding defect type is generated by improving the conditional generative adversarial network until the proportion of each defect type is greater than or equal to 5%; at the same time, the ratio of the total number of defect images to the total number of standardized normal images in the fused image data set is adjusted to (1-3):1, and the total number of defect images is the sum of the total number of standardized defect images and synthetic defect images.

[0012] Preferably, the improved YOLOv8 model is a YOLOv8 target detection model that incorporates a defect feature attention fusion module, comprising a backbone network, a feature fusion network, and a detection head connected in sequence; the feature fusion network adopts the native PAN-FPN multi-scale feature fusion structure of YOLOv8, and the defect feature attention fusion module is embedded in the multi-scale feature input link of the feature fusion network, and is connected in series between the multi-scale feature output end of the backbone network and the feature fusion layer input end of the feature fusion network, used to perform defect feature weighting enhancement on the multi-scale basic feature map output by the backbone network to suppress background feature interference; the weight calculation and feature enhancement formula of the defect feature attention fusion module are as follows: ; ; in, This is the single-scale basic feature map output by the backbone network. This is a standard defect feature map extracted by the SIFT algorithm. For the defect feature attention weight map, This is an element-wise multiplication operation. The output feature map is a weighted and enhanced version of the defect features.

[0013] Preferably, the model training is as follows: the augmented data image set is divided into a training image set and a validation image set in a 7:3 ratio, and each image in the training image set and the validation image set is labeled with the defect type and defect location, or no defect is used as the training label. The improved YOLOv8 model is used as the training object, and the training image set is input for end-to-end iterative training. During the iterative training, an adaptive weighted loss function is used to calculate the loss value, and the model parameters are updated by backpropagation through the Adam optimizer until a preset training termination condition is reached. The preset training termination condition is: the average accuracy of the improved YOLOv8 model on the validation image set has no improvement for 10 consecutive iterations of training, or the number of iterations of training reaches the preset maximum number of iterations of training. The model parameters of the improved YOLOv8 model at this time are saved to obtain the defect recognition model. The adaptive weighted loss function is: ; in, This represents the total number of defect types. The number of image samples in the training image set. Defect type The number of image samples, , These are the corresponding defect types. The classification loss and CIoU bounding box regression loss.

[0014] The technological advancement of this invention lies in addressing the industry pain points of scarce samples and imbalanced data distribution in industrial equipment defect detection. It designs an improved conditional generative adversarial network (GAN) that incorporates a channel-space attention mechanism and a multi-scale feature fusion structure. Adversarial training is completed using a composite loss function combining fusion adversarial loss, perceptual loss, and defect feature loss. The generated synthetic defect images exhibit high fidelity and complete defect feature representation. Combined with a data balancing correction strategy, it effectively supplements the number of rare defect samples, achieving balanced expansion of the dataset. Simultaneously, a defect feature attention fusion module is embedded into the YOLOv8 model, enhancing its ability to capture minute and rare defect features. Model training is conducted using an adaptive weighted loss function adapted to imbalanced datasets, solving the problem of missed and false detections in general detection models. This improves the accuracy and stability of equipment defect identification, forming an integrated detection solution combining sample enhancement and defect identification, suitable for the practical needs of defect detection in most industrial equipment categories. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a schematic diagram of the system functional modules of the present invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0018] In one embodiment, such as Figure 1 The system described above is an equipment defect sample enhancement and identification system based on generative adversarial networks. It includes a data preprocessing module, a sample enhancement module, an enhanced dataset construction module, a model training module, and a defect detection module, which work together to enhance equipment defect samples and accurately identify defects.

[0019] The data preprocessing module is used to denoise, standardize, label defects, and extract features from the equipment image sample set, generating a standardized equipment image sample set and a conditional coding vector set. The sample augmentation module is used to generate a synthetic defect image sample set by combining a conditional coding vector set and a standardized equipment image sample set with a composite loss function and performing adversarial training through an improved conditional generative adversarial network. An enhanced dataset construction module is used to fuse a synthetic defect image sample set and a standardized equipment image sample set to obtain a fused data image set, and to perform data balancing correction on the fused data image set to obtain an enhanced data image set; The model training module is used to train the improved YOLOv8 model based on the enhanced data image set to obtain the defect recognition model. The defect detection module is used to denoise and standardize the image of the equipment to be inspected before inputting it into the defect recognition model to output the defect detection result.

[0020] Furthermore, the working process of this system will be fully described below using the detection of surface defects in metal parts of industrial machinery and equipment as an example: The detection object in this embodiment is the surface image of metal parts of mechanical equipment, and the defect types to be identified include four categories: scratches, cracks, dents and corrosion.

[0021] The data preprocessing module acquires an equipment image sample set S, which contains 1200 images of the surface of metal parts. The defect image subset, consisting of equipment defect images, contains 400 defect images, and the normal image subset, consisting of normal equipment images, contains 800 normal images. For each equipment defect image in the defect image subset, adaptive Gaussian filtering for denoising, standardization, defect annotation, and feature extraction are sequentially performed to obtain a standardized defect image and a conditional coding vector. Similarly, for each equipment normal image in the normal image subset, adaptive Gaussian filtering for denoising and standardization are sequentially performed to obtain a standardized normal image. Finally, all standardized defect images and normal images constitute a standardized equipment image sample set, and all conditional coding vectors constitute a conditional coding vector set.

[0022] The sample augmentation module employs an improved conditional generative adversarial network (including a generator and a discriminator) that incorporates a channel-space attention mechanism and a multi-scale feature fusion structure. Combined with a composite loss function, adversarial training is completed to generate synthetic defective images in batches. First, the network parameters of the generator and discriminator are initialized using a He normal distribution. Training hyperparameters are set with a batch size of 32, an initial learning rate of 0.0002, and a maximum of 2000 iterations. The weight coefficients of the composite loss function are set to... , and The generator employs an "encoder-channel-spatial attention fusion module-decoder" structure (the encoder consists of four cascaded 3×3 convolutional layers, BN layers, and Leaky ReLU activation functions with a negative slope of 0.2; the channel-spatial attention fusion module comprises cascaded CAM and SAM layers; the decoder consists of three cascaded 3×3 transposed convolutional layers, BN layers, and ReLU activation functions, plus one output transposed convolutional layer with Tanh activation). The discriminator employs an "input layer-multi-scale feature extraction branch" structure. The structure is "Feature Fusion Layer - Output Layer" (the input layer maps the conditional coding vector to a conditional feature map of the same size through 1×1 convolution and concatenates it with the input image channels; the multi-scale feature extraction branch contains three parallel convolutional branches, namely fine-grained, medium-scale, and global feature branches with 2, 3, and 4 layers of cascaded convolutions, respectively, with each branch having a 3×3 convolutional kernel and a BN layer and LeakyReLU activation function; the feature fusion layer uses 1×1 convolution to reduce dimensionality and fuse features; the output layer outputs discriminant data after a fully connected layer and a Sigmoid function). (Results); then iterative adversarial training is performed: 32 sets of real standardized defect images and corresponding conditional coding vectors are randomly selected from the standardized equipment image sample set and conditional coding vector set, 128-dimensional standard Gaussian distributed random noise z is generated and concatenated with the conditional coding vector channels and input into the generator to obtain batch synthetic defect images; the network parameters of the generator are fixed, the real defect images and synthetic defect images and corresponding conditional coding vectors are input into the discriminator, the discriminator loss value is calculated based on adversarial loss and the discriminator network parameters are updated through the Ada optimizer; the network parameters of the discriminator are fixed, the synthetic defect images and corresponding conditional coding vectors are input into the discriminator, the total loss value is calculated based on the composite loss function and the generator network parameters are updated through the Adam optimizer; after each iteration, the mean SSIM and mean PSNR of the batch synthetic defect images are calculated, and the training is terminated when the mean SSIM is ≥0.85 and the mean PSNR is ≥30dB or the iteration reaches 2000 times, the optimal generator is saved and batch synthetic defect images are generated to form a synthetic defect image sample set.

[0023] The augmented dataset construction module merges the synthetic defect image sample set with the standardized equipment image sample set to obtain a fused data image set containing standardized normal images, standardized defect images, and synthetic defect images. Data balancing correction is then performed on this fused data image set: the proportion of images of the four defect types is statistically analyzed, and for the crack defect type with an original proportion of 3.2% (<5%), synthetic defect images of this defect type are generated by the optimal generator until the proportion of all defect types is ≥5%. At the same time, the ratio of the total number of defect images to the total number of standardized normal images is adjusted to 1:1. After the correction is completed, the augmented data image set is obtained.

[0024] The model training module uses an improved YOLOv8 model with an introduced defect feature attention fusion module (containing a backbone network, a PAN-FPN structured feature fusion network, and a detection head connected in sequence; the defect feature attention fusion module is embedded in the multi-scale feature input link of the feature fusion network and connected in series between the multi-scale feature output end of the backbone network and the input end of the feature fusion layer). The augmented data image set is divided into a training image set and a validation image set in a 7:3 ratio, and training labels are matched and labeled. The improved YOLOv8 model is initialized with ImageNet pre-trained weights, and then the training image set is input for end-to-end iterative training. During the iterative training, an adaptive weighted loss function is used to calculate the total loss value, and the model parameters of the improved YOLOv8 model are updated through backpropagation using the Adam optimizer. The training termination condition is set as follows: the average accuracy on the validation image set does not improve for 10 consecutive iterations or the number of iterations reaches 200 rounds. After training terminates, the model parameters of the fixed improved YOLOv8 model at this time are saved to obtain the defect recognition model.

[0025] The defect detection module acquires the surface image of the metal part to be inspected as the equipment image to be inspected. It first performs adaptive Gaussian filtering denoising and standardization processing on the image to obtain a standardized image to be inspected. The standardized image to be inspected is then input into the defect recognition model, which outputs the detection results containing the defect type and location bounding box, thus completing the automated identification of equipment defects.

[0026] The above description is merely a specific embodiment of the present invention and is not intended to limit the present invention in any way. Any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention shall still fall within the scope of the technical solution of the present invention.

Claims

1. A system for enhancing and identifying equipment defect samples based on generative adversarial networks, characterized in that, include: The data preprocessing module is used to denoise, standardize, label defects, and extract features from the equipment image sample set, generating a standardized equipment image sample set and a conditional coding vector set. The sample augmentation module is used to generate a synthetic defect image sample set by combining a conditional coding vector set and a standardized equipment image sample set with a composite loss function and performing adversarial training through an improved conditional generative adversarial network. An enhanced dataset construction module is used to fuse a synthetic defect image sample set and a standardized equipment image sample set to obtain a fused data image set, and to perform data balancing correction on the fused data image set to obtain an enhanced data image set; The model training module is used to train the improved YOLOv8 model based on the enhanced data image set to obtain the defect recognition model. The defect detection module is used to denoise and standardize the image of the equipment to be inspected before inputting it into the defect recognition model to output the defect detection result.

2. The equipment defect sample enhancement and identification system based on generative adversarial networks according to claim 1, characterized in that, The equipment image sample set consists of several equipment defect images and several equipment normal images. Each equipment defect image is sequentially subjected to denoising, standardization, defect labeling and feature extraction to obtain the corresponding standardized defect image and conditional coding vector. Each equipment normal image is sequentially subjected to denoising and standardization to obtain the corresponding standardized normal image.

3. The equipment defect sample enhancement and identification system based on generative adversarial networks according to claim 2, characterized in that, The denoising process employs an adaptive Gaussian filtering algorithm. The standardization process includes image size unification and image pixel value normalization. The defect labeling process involves labeling the defect type, defect location, and defect size of the standardized equipment defect image to generate defect labels. The feature extraction process uses the SIFT algorithm to extract the defect feature vector of the standardized equipment defect image.

4. The equipment defect sample enhancement and identification system based on generative adversarial networks according to claim 1, characterized in that, The improved conditional generative adversarial network is a conditional generative adversarial network that introduces a channel-space attention mechanism and a multi-scale feature fusion structure, including a generator and a discriminator. The generator includes an encoder, a channel-space attention fusion module and a decoder, and the discriminator includes an input layer, a multi-scale feature extraction branch, a feature fusion layer and an output layer. The encoder is used to perform multi-level convolutional feature extraction and dimensionality compression on the input random noise and conditional coding vector, and output a coded feature map. The channel-spatial attention fusion module is used to sequentially perform channel attention weighting and spatial attention weighting on the encoded feature map, and output an attention-enhanced feature map. The decoder is used to perform multi-level transposed convolution upsampling on the attention-enhanced feature map and then decode it to generate a synthetic defect image with the same image size specifications as the standardized defect image. The input layer is used to concatenate the input standardized defect image or synthetic defect image with the corresponding conditional coding vector along the channel dimension, and output the discriminator input feature map. The multi-scale feature extraction branch is used to extract fine-grained defect features, medium-scale defect features and global defect features from the discriminator input feature map through three parallel convolutional branches of different scales, and output a multi-scale defect feature map. The feature fusion layer is used to perform channel splicing and convolutional dimensionality reduction fusion on multi-scale defect feature maps and output a fused feature map. The output layer is used to perform dimensionality mapping on the fused feature map using a fully connected layer and then output the image authenticity identification result through the Sigmoid activation function.

5. The equipment defect sample enhancement and identification system based on generative adversarial networks according to claim 1, characterized in that, The composite loss function is a weighted composite loss function that integrates adversarial loss, perceptual loss, and defect feature loss. The adversarial loss is the cross-entropy loss constructed during adversarial training of the improved conditional generative adversarial network. The perceptual loss is based on the Euclidean distance loss between the high-level visual features of the real defect image and the synthetic defect image extracted by the pre-trained VGG16 network. The defect feature loss is based on the cosine similarity loss between the defect feature vectors of the real defect image and the synthetic defect image extracted by the SIFT algorithm.

6. The equipment defect sample enhancement and identification system based on generative adversarial networks according to claim 4 or 5, characterized in that, The method of adversarial training is as follows: S1. Initialize the network parameters of the generator and discriminator using the He normal initialization method, and set the training hyperparameters; S2. Randomly extract real defect images and corresponding conditional coding vectors of a preset batch size from the standardized equipment image sample set and the conditional coding vector set. At the same time, generate random noise that conforms to the standard Gaussian distribution. Then, concatenate the random noise and the conditional coding vector and input them into the generator to obtain the batch synthetic defect images. S3. Fix the network parameters of the generator, input the batch of real defect images and corresponding conditional coding vectors and the batch of synthetic defect images and corresponding conditional coding vectors into the discriminator, calculate the discriminator loss value based on the adversarial loss, and update the network parameters of the discriminator through backpropagation of the Ada optimizer. S4. Fix the network parameters of the discriminator, input the batch synthesized defect images and corresponding conditional coding vectors into the discriminator, calculate the total loss value based on the composite loss function, and update the network parameters of the generator through backpropagation of the Adam optimizer accordingly. S5. Repeat S2-S4 for multiple iterations of training. After each iteration, calculate the mean SSIM and mean PSNR of the generated batch of synthetic defect images. When the mean SSIM is ≥0.85 and the mean PSNR is ≥30dB, or when the iteration training reaches the preset maximum number of iterations, terminate the iteration training to obtain the optimal generator. Based on this, input the standardized defect images into the optimal generator to generate synthetic defect images in batches to form a synthetic defect image sample set.

7. The equipment defect sample enhancement and identification system based on generative adversarial networks according to claim 1, characterized in that, The fused image data set includes standardized normal images, standardized defect images, and synthetic defect images. The data balancing correction is as follows: the proportion of each defect type in the fused image data set is statistically analyzed. If the proportion of each defect type is less than 5%, synthetic defect images of the corresponding defect type are generated by improving the conditional generative adversarial network until the proportion of each defect type is greater than or equal to 5%. At the same time, the ratio of the total number of defect images to the total number of standardized normal images in the fused image data set is adjusted to (1-3):

1.

8. The equipment defect sample enhancement and identification system based on generative adversarial networks according to claim 1, characterized in that, The improved YOLOv8 model is a YOLOv8 model for target detection that incorporates a defect feature attention fusion module. It includes a backbone network, a feature fusion network, and a detection head. The defect feature attention fusion module is embedded in the multi-scale feature input link of the feature fusion network and is connected in series between the multi-scale feature output end of the backbone network and the feature fusion layer input end of the feature fusion network.

9. The equipment defect sample enhancement and identification system based on generative adversarial networks according to claim 8, characterized in that, The model training process involves dividing the augmented data image set into a training image set and a validation image set in a 7:3 ratio and labeling them with training labels. The improved YOLOv8 model is used as the training object. The training image set is input for training, and an adaptive weighted loss function is used to calculate the loss value during the training process. Based on this, the model parameters of the improved YOLOv8 model are updated through backpropagation using the Adam optimizer until the model training termination condition is met, thus obtaining the defect recognition model.

Images