A method and device for generating defective images of equipment components based on improved DCGAN

Abstract

The present application provides a method and device for generating defective images of equipment components based on an improved DCGAN, the method comprising: obtaining image data of components; dividing the image data into a background set and a defect set based on the defect type classification label of the image data; performing deduplication, normalization and data enhancement processing on the background set and the defect set to obtain the processed background set and defect set; constructing an improved DCGAN for model training, the improved DCGAN algorithm comprising: using the SiLU activation function in the discriminator of the improved DCGAN, and introducing the PPA attention module; based on the processed background set and the defect set, using the generator of the improved DCGAN, generating equipment component defective image data. The improved DCGAN combined with the PPA module and the SiLU activation function can generate high-quality virtual defect samples, significantly improving the performance of the defect detection system.

Description

Technical Field

[0001] The present application belongs to the field of equipment component defect detection, and in particular, to a method and device for generating equipment component defect images based on an improved DCGAN. Background Art

[0002] Smart equipment components are mainly produced in a semi-automatic manner during the production process. Considering the limitations of raw materials, production processes and production environment, surface stains, cracks, irregular shapes and defects are inevitable in the finished products. Limiting the entry of defective products into the market is an important way to reduce customer complaints, ensure equipment performance and improve product quality. Therefore, all smart equipment manufacturers are investing heavily to improve the efficiency and accuracy of testing and reduce testing costs. However, due to the complex and changeable shapes of components, traditional automatic detection methods are difficult to implement. With the development of deep learning technology, the use of neural network models for online product defect detection can replace the current manual detection and improve detection efficiency and accuracy. This method has been widely used in various detection links of industrial production.

[0003] However, due to the lack of a sufficient number of defective component images, it is difficult to train an ideal defect detection model. Traditional data enhancement expands the training samples by performing geometric affine transformation, color space transformation, rotation distortion transformation, polar coordinate transformation and other methods on existing image data samples. In addition, the traditional DCGAN generator is prone to mode collapse and low quality of generated samples in a small sample environment, making it difficult to capture the details of component images. These methods are simple to operate, but cannot enrich the distribution of samples in high-dimensional feature space. Summary of the invention

[0004] The purpose of this invention is to provide a component defect image generation method based on improved DCGAN, which expands the small sample library of components in the intelligent equipment industry through the deep convolutional generative adversarial network (DCGAN), aiming to improve the fault diagnosis accuracy under small sample conditions by enriching fault information.

[0005] In a first aspect, a method for generating defective images of equipment components based on an improved DCGAN is provided, the method comprising: S1, acquiring image data of the components through a high-definition camera; S2, dividing the image data into a background set and a defect set based on defect type classification labels of the image data; S3, performing deduplication, normalization and data enhancement processing on the background set and the defect set to obtain processed background set and defect set; S4, constructing an improved DCGAN algorithm model for model training, the improved DCGAN algorithm comprising: in the discriminator of the improved DCGAN algorithm model, using a SiLU activation function and introducing a PPA (Parallelized Patch-Aware Attention) module; S5: based on the processed background set and the defect set, using a generator of the trained improved DCGAN algorithm model to generate defective image data of equipment components.

[0006] Preferably, the SiLU activation function is:

[0007] Among them, x is the real sample data distribution, is the activation function.

[0008] Preferably, the improved DCGAN algorithm model includes: a generator G and a discriminator D, wherein the generator G includes a 5-layer convolutional neural network structure of a fully connected layer, a deconvolution layer, a batch normalization layer, an upsampling layer and an activation layer. In the network of the generator G, the last activation layer uses Tanh as the activation function, and the other layers use LeakyReLU as the activation function, and a PPA attention module is placed before the upsampling layer; The discriminator D includes a five-layer network of an input layer, a convolution layer, a batch normalization layer, an activation layer and a fully connected layer, wherein an attention module is embedded before the input layer, the fully connected layer uses a Sigmoid activation function, and the other layers use a SiLU activation function.

[0009] Preferably, the step of generating equipment component defect image data comprises: The inception network is used to extract feature vectors, and the difference in feature vector space distribution between the generated sample data and the real sample data is compared to obtain the FID value; based on the FID value, the generated equipment component defect image data is evaluated, and the FID value calculation formula is:

[0010] in, For real samples Data distribution, To generate samples Data distribution, , are the mean and covariance of the feature vector after feature extraction, is the trace of the matrix.

[0011] In a second aspect, a device for generating defective images of equipment components based on an improved DCGAN is provided, characterized in that the device comprises: an acquisition module, which acquires image data of the components through a high-definition camera; a division module, which divides the image data into a background set and a defect set based on defect type classification labels of the image data; a processing module, which performs deduplication, normalization and data enhancement processing on the background set and the defect set to obtain the processed background set and defect set; a model construction module, which constructs an improved DCGAN algorithm model for model training, and the improved DCGAN algorithm comprises: in the discriminator of the improved DCGAN algorithm model, a SiLU activation function is adopted, and a Parallelized Patch-Aware Attention (PPA) module is introduced; a generation module, which generates equipment component defective image data based on the processed background set and the defect set using a generator of the trained improved DCGAN algorithm model.

[0012] In a third aspect, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a method for generating defective images of equipment components based on an improved DCGAN as provided in the first aspect is implemented.

[0013] In a fourth aspect, a non-transitory computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, a method for generating defect images of equipment components based on an improved DCGAN as provided in the first aspect is implemented.

[0014] With respect to the prior art, the beneficial effects of the present invention are: The improved DCGAN algorithm combined with the PPA module and SiLU activation function can generate high-quality virtual defect samples. These generated defect samples have higher detail accuracy and diversity, which can effectively make up for the lack of sufficient defect samples in actual production. By using these generated defect samples to expand the training data set, the model can be exposed to a richer range of defect types during training, thereby improving its defect detection capabilities in actual production environments. The generated virtual samples not only enhance the diversity of the data set, but also effectively avoid the overfitting problem in training, helping the model to improve its adaptability to unseen data, especially in the case of scarce samples, which can greatly improve the performance of the detection system. In practical applications, the improved DCGAN can significantly improve the performance of the defect detection system, especially in scenarios where defect samples are difficult to obtain. It can provide more high-quality samples for training, solve the problem of data scarcity, and ensure the accuracy and stability of the system in actual production environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in describing the embodiments of the present application are briefly introduced below.

[0016] Figure 1 A flowchart of a method for generating defect images of equipment components based on an improved DCGAN provided in one embodiment of the present invention; Figure 2 A schematic diagram of an improved DCGAN algorithm model according to an embodiment of the present invention; Figure 3 A schematic diagram of the physical structure of an electronic device provided by the present invention. DETAILED DESCRIPTION

[0017] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar modules or modules with the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be interpreted as limiting the present invention.

[0018] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "a", "an", "said" and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, modules and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, modules, components and / or groups thereof. It should be understood that when we refer to a module as being "connected" or "coupled" to another module, it may be directly connected or coupled to the other modules, or there may be intermediate modules. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any modules and all combinations of one or more associated listed items.

[0019] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation method of the present application will be further described in detail below in conjunction with the accompanying drawings.

[0020] The technical solution of the present application and how to solve the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0021] like Figure 1 FIG. 1 is a flowchart of a method for generating defect images of equipment components based on an improved DCGAN according to an embodiment of the present invention. The method includes: S1, obtaining image data of the component through a high-definition camera; In the embodiment of the present invention, specifically, a suitable high-definition camera is selected, and a high-definition camera with appropriate resolution, frame rate and interface type is selected according to specific needs. For example, if you need to capture fast-moving objects, you should choose a camera with a high frame rate; if you have high requirements for details, you need to choose a camera with high resolution. Use professional image acquisition software to control the camera operation, set acquisition parameters (such as exposure time, gain value, etc.), and save the acquired image data. The software preferably supports automatic recognition and marking functions, which can further simplify the subsequent processing flow.

[0022] S2, based on the image data and according to the defect type classification label, dividing the image data into a background set and a defect set; Specifically, the image data needs to be annotated to identify the defect type in each image. Using a pre-trained model, the model can be used to automatically annotate new images, but regular manual review is required to ensure accuracy. The image is divided into multiple small blocks for easy processing and analysis: According to the typical size of the defect and the image resolution, the appropriate block size is selected. Generally, the block size should be large enough to contain the complete defect features, but not too large to avoid increasing the computational burden. Use the sliding window technique to traverse the entire image and generate multiple overlapping small blocks. This ensures that no defect area is missed. Assign a label to each block and divide it into a background set and a defect set: the background set does not contain any defective image blocks. Defect set: image blocks containing defects, and further subdivided according to the defect type (such as cracks, scratches, stains, etc.).

[0023] S3, performing deduplication, normalization and data enhancement processing on the background set and the defect set to obtain a processed background set and defect set; Specifically, the original image is preprocessed to improve the efficiency and accuracy of subsequent processing: Cropping: remove irrelevant areas in the image and keep only the parts containing zero components. Normalization: normalize the pixel values ​​of the image to a fixed range (such as 0-1) to reduce the impact of lighting changes. Image enhancement: use image enhancement techniques (such as histogram equalization, sharpening, etc.) to improve image quality.

[0024] S4. constructing an improved DCGAN algorithm model for model training, wherein the improved DCGAN algorithm includes: in the discriminator of the improved DCGAN algorithm model, using a SiLU activation function and introducing a Parallelized Patch-Aware Attention (PPA) module; Specifically, see Figure 2 The improved DCGAN algorithm model includes: a generator G and a discriminator D. The generator G includes a 5-layer convolutional neural network structure of a fully connected layer, a deconvolution layer, a batch normalization layer, an upsampling layer and an activation layer. In the network of the generator G, the last activation layer uses Tanh as the activation function, and the other layers use LeakyReLU as the activation function, and a PPA attention module is placed before the upsampling layer; The discriminator D includes a five-layer network of an input layer, a convolution layer, a batch normalization layer, an activation layer and a fully connected layer, wherein an attention module is embedded before the input layer, the fully connected layer uses a Sigmoid activation function, and the other layers use a SiLU activation function.

[0025] The improved PASDCGAN has made significant improvements on the basis of the traditional generative adversarial network (DCGAN), especially by introducing the PPA module and SiLU activation function, making the generated virtual defect samples more realistic and detailed, and better adapted to the needs of component defect detection. The PPA module introduces a local self-attention module, which can pay close attention to the details in the image during the image generation process, especially the tiny defects in the component image. Traditional generators may not capture defect areas accurately enough. The PPA module performs self-attention calculations in local areas, allowing the generator to capture defect features more finely within each patch. In particular, when component defects are usually manifested as slight changes in the image, the PPA module can help the generator capture these subtle differences, generate images that conform to the real defect distribution, and enhance the diversity and detail expression of the data set.

[0026] In the discriminator of PASDCGAN, we use the SiLU (Swish) activation function, which is:

[0027] Where x is the real sample data distribution, is the activation function.

[0028] The SiLU activation function is more suitable for processing complex details in images than traditional ReLU or LeakyReLU. SiLU can provide smooth nonlinear transformations, effectively reduce the problem of gradient vanishing, and compared to ReLU, SiLU can better promote the flow of information in the network and avoid the "dead neuron" phenomenon. Especially when processing complex structures and details of images, SiLU can show better performance, making the images generated by the network more delicate and realistic. The SiLU activation function has the characteristics of no upper bound, lower bound, smoothness, and non-monotonicity. SiLU works better than ReLU on deep models. This improvement, which can be regarded as a smooth ReLU activation function, enables PASDCGAN to not only expand the data set in quantity when generating defective samples, but also greatly improve the image quality and detail performance.

[0029] Combined with the PPA module and the SiLU activation function, the improved DCGAN algorithm can generate high-quality virtual defect samples. These generated defect samples have higher detail accuracy and diversity, which can effectively make up for the lack of sufficient defect samples in actual production. By using these generated defect samples to expand the training data set, the model can be exposed to a richer range of defect types during the training process, thereby improving its defect detection capabilities in the actual production environment. The generated virtual samples not only enhance the diversity of the data set, but also effectively avoid the overfitting problem in training, helping the model to improve its adaptability to unseen data, especially in the case of scarce samples, which can greatly improve the performance of the detection system. In practical applications, the improved DCGAN can significantly improve the performance of the defect detection system, especially in scenarios where defect samples are difficult to obtain. It can provide more high-quality samples for training, solve the problem of data scarcity, and ensure the accuracy and stability of the system in the actual production environment.

[0030] S5: Based on the processed background set and the defect set, using the generator of the trained improved DCGAN algorithm model, generate equipment component defect image data.

[0031] Specifically, based on the processed background set and the defect set, using the generator of the trained improved DCGAN algorithm model, generating equipment component defect image data; After generating equipment component defect image data, the method includes: The inception network is used to extract feature vectors, and the difference in feature vector space distribution between the generated sample data and the real sample data is compared to obtain the FID value; based on the FID value, the generated equipment component defect image data is evaluated, and the FID value calculation formula is:

[0032] in, For real samples Data distribution, To generate samples Data distribution, , are the mean and covariance of the feature vector after feature extraction, is the trace of the matrix.

[0033] Compared with the DCGAN model, the FID value of the images generated by the improved DCGAN model is reduced from 20.02 to 10.07, which can more effectively complete the task of generating defective images of equipment components.

[0034] In summary, the improved DCGAN algorithm of the embodiment of the present invention can generate high-quality virtual defect samples by combining the PPA module and the SiLU activation function. These generated defect samples have higher detail accuracy and diversity, and can effectively make up for the lack of sufficient defect samples in actual production. The improved DCGAN algorithm makes full use of the attention module in the PPA module to efficiently extract features, strengthen the interaction of key information, and improve the adaptive activation ability of the network in combination with the improved SiLU activation function. Compared with DCGAN, it is more stable and the quality of the generated defect samples is significantly enhanced. The trained model can identify more types and more subtle defects in the equipment component defect detection task, thereby improving the overall detection accuracy.

[0035] Figure 3 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 3 As shown, the electronic device may include: a processor (processor) 601, a communication interface (Communications Interface) 602, a memory (memory) 603 and a communication bus 604, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus. The processor may call the logic instructions in the memory to execute a method for generating defect images of equipment components based on an improved DCGAN.

[0036] In addition, the logic instructions in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0037] On the other hand, an embodiment of the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute a method for generating equipment component defect images based on an improved DCGAN provided in the above-mentioned method embodiments.

[0038] On the other hand, an embodiment of the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to execute a method for generating defect images of equipment components based on an improved DCGAN provided in the above embodiments.

[0039] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0040] The above is only a partial implementation of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for generating defective images of equipment components based on improved DCGAN, characterized in that: The method comprises: S1, obtaining image data of the component through a high-definition camera; S2, based on the image data and according to the defect type classification label, dividing the image data into a background set and a defect set; S3, performing deduplication, normalization and data enhancement processing on the background set and the defect set to obtain a processed background set and defect set; S4. constructing an improved DCGAN algorithm model for model training, wherein the improved DCGAN algorithm includes: in the improved DCGAN algorithm model, adopting a SiLU activation function and introducing a PPA attention module; S5: Based on the processed background set and the defect set, using the generator of the trained improved DCGAN algorithm model, generate equipment component defect image data.

2. The method according to claim 1, characterized in that The SiLU activation function is: ； Among them, x is the real sample data distribution, is the activation function.

3. The method according to claim 1, characterized in that The improved DCGAN algorithm model includes: a generator G and a discriminator D, wherein the generator G includes a 5-layer convolutional neural network structure of a fully connected layer, a deconvolution layer, a batch normalization layer, an upsampling layer and an activation layer. In the network of the generator G, the last activation layer uses Tanh as the activation function, and the other layers use LeakyReLU as the activation function, and a PPA attention module is placed before the upsampling layer; The discriminator D includes a five-layer network of an input layer, a convolution layer, a batch normalization layer, an activation layer and a fully connected layer, wherein the PPA attention module is embedded before the input layer, the fully connected layer uses a Sigmoid activation function, and the other layers use a SiLU activation function.

4. The method according to claim 1, characterized in that After generating equipment component defect image data, the method includes: The inception network is used to extract feature vectors, and the difference between the generated sample data and the real sample data in the feature vector space distribution is compared to obtain the FID value; based on the FID value, the generated equipment component defect image data is evaluated, and the FID value calculation formula is: ； in, For real samples Data distribution, To generate samples Data distribution, , are the mean and covariance of the feature vector after feature extraction, is the trace of the matrix.

5. A device for generating defective images of equipment components based on improved DCGAN, characterized in that: The device comprises: An acquisition module, which acquires image data of the components through a high-definition camera; A partitioning module, based on the image data and classified by defect type, divides the image data into a background set and a defect set; A processing module, performing deduplication, normalization and data enhancement processing on the background set and the defect set to obtain a processed background set and defect set; A model building module, constructing an improved DCGAN algorithm model for model training, wherein the improved DCGAN algorithm includes: in the improved DCGAN algorithm model, adopting a SiLU activation function and introducing an attention module; A generation module generates equipment component defect image data based on the processed background set and the defect set using a generator of a trained improved DCGAN algorithm model.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method for generating equipment component defect images based on the improved DCGAN is implemented as described in any one of claims 1-4.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for generating defect images of equipment components based on an improved DCGAN as described in any one of claims 1 to 4 is implemented.

Images