Multi-class defect detection method and device based on variational auto-encoder and denoising network

Abstract

The invention discloses a multi-class defect detection method and device based on a variational auto-encoder and a denoising network, and the method comprises the steps: obtaining an industrial product surface defect image, carrying out the preprocessing, and constructing a training set and a verification set; the training process of the method is divided into two stages: in the first stage, a preprocessed training image is input into a variational auto-encoder for training, feature distribution of the image is learned, and features of a normal image are captured by reconstructing the image; in the second stage, a denoising network and a feature guiding module are used for conducting secondary reconstruction on image features in the feature space of the variational auto-encoder, and therefore the reconstruction capacity and the multi-class reconstruction capacity of the model are enhanced; in the reasoning stage, multi-scale features of an original image and a reconstructed image are extracted through a feature extractor, feature comparison is carried out, and an abnormal scoring graph is generated. According to the method, normal information is reserved while the reconstruction capability of the variational auto-encoder is enhanced, and defect missing detection caused by reconstruction failure is effectively reduced.

Description

Technical Field

[0001] The present invention relates to the fields of computer vision and industrial inspection, and in particular to a multi-category defect detection method and device based on a variational autoencoder and a denoising network, which are used for automatic detection and positioning of surface defects of multi-category objects such as industrial parts. Background Art

[0002] In the industrial production process, defect detection on the surface of parts is an important part of ensuring product quality. Most existing automated defect detection methods are based on the detection of single-category products. For multi-category products, it is often necessary to train a separate detection model for each category of products, which not only increases the complexity and computational cost of the model, but also has obvious limitations when facing cross-category product detection. In addition, when dealing with defects with small or complex textures, the existing methods have poor reconstruction effects and are prone to false positives or missed positives.

[0003] In recent years, variational autoencoders (VAE) and denoising diffusion models have received extensive attention in anomaly detection tasks due to their powerful image reconstruction capabilities. VAE can learn the feature distribution of input data and use its latent space to model normal samples. However, existing VAE-based detection methods are prone to category confusion problems when dealing with multi-category anomaly detection and cannot effectively distinguish normal from abnormal samples. In addition, although the diffusion model can generate high-quality images, it usually requires multiple diffusion steps, resulting in high computational costs.

[0004] In view of the above problems, how to uniformly detect different types of parts and achieve accurate positioning of abnormal areas is one of the problems that technical personnel in this field need to solve urgently. Summary of the invention

[0005] The main purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art and to provide a multi-category defect detection method and device based on variational autoencoder and denoising network. By combining the advantages of VAE and denoising network, the problems of insufficient detection accuracy, high computational cost and category confusion in multi-category defect detection of the prior art can be solved, and more reliable detection, identification and positioning of product surface defects can be achieved.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] In a first aspect, the present invention provides a multi-category defect detection method based on a variational autoencoder and a denoising network, comprising the following steps:

[0008] Acquire and preprocess images of surface defects of industrial products to construct training sets and validation sets;

[0009] An image reconstruction model is constructed, wherein the image reconstruction model includes a variational encoder, a feature guidance network, a denoising network and a variational decoder connected in sequence; the variational encoder is used to extract compact features of an image and learn image feature distribution; the feature guidance network interacts with the denoising network by extracting features of the variational encoder to guide the denoising network to perform multi-category correct reconstruction; the denoising network further reconstructs image features in a feature space to enhance the reconstruction capability of the image reconstruction model; the variational decoder is used to perform preliminary reconstruction of the image according to features of the latent space;

[0010] The image reconstruction model is trained in stages using the training set, specifically including: in the first stage, a variational encoder and a variational decoder are trained to learn the feature distribution and reconstruction of normal images; in the second stage, a denoising network is trained to further optimize the reconstruction effect of image features;

[0011] Design a multi-scale feature extraction and comparison network to extract multi-scale features of the input image and the reconstructed image for comparison, and output an abnormality score map, based on which defect location is achieved;

[0012] The verification set is input into the defect detection model composed of an image reconstruction model and a multi-scale feature extraction and comparison network for verification testing to evaluate the detection performance of the defect detection model. Based on the evaluated defect detection model, multi-category defect detection in actual industrial scenarios is realized.

[0013] As a preferred technical solution, the surface defect images of industrial products are obtained and preprocessed to construct training sets and verification sets, specifically:

[0014] Use industrial cameras to complete the image data collection of industrial product surfaces. Fix the area array camera in a bird's-eye view position, illuminate the sample with uniform light, and collect bright and uniformly illuminated industrial product surface images. Classify the images; perform data preprocessing on industrial product surface defect images according to different categories; normalize and evenly brighten the image acquisition module, and convert it into a format that the model can recognize.

[0015] Clean, enhance and normalize the collected image data and convert it into a format that can be recognized by the model;

[0016] Among the preprocessed images, images without abnormalities are selected as training sets, and images with abnormalities and their corresponding abnormal mask images are selected as verification sets.

[0017] As a preferred technical solution, the image reconstruction model is constructed as follows:

[0018] Use the pre-trained variational encoder to compress the input image x into the latent space representation Z and extract the intermediate layer features f i, where z is generated by a partial forward diffusion process with added noise to generate the perturbed latent variable z t ;f i Input feature guided network to guide denoising network;

[0019] z t Input denoising network, output reconstructed latent representation through one-step inverse diffusion process And use the pre-trained variational decoder to Restore to the original image level and get the reconstructed image

[0020] As a preferred technical solution, the image reconstruction model is trained in stages using the training set, specifically:

[0021] In the first stage of training, the variational encoder and variational decoder are trained to compress the normal image into the latent space and restore the pixel-level representation, and the mean square error loss and the perceptual image similarity loss are used as the reconstruction loss metrics, and the KL divergence and the maximum mean difference are used to constrain the latent space features; the mean square error loss is used to reconstruct the pixel-level error, and the perceptual image similarity loss is used to enhance the local realism of the reconstructed image;

[0022] In the second stage of training, the denoising network is trained to perform secondary reconstruction of the latent space variables, reconstruct large-scale anomalies and texture anomalies, and a feature-guided network is introduced to guide the reconstruction of the correct category; the loss function of the second stage guides the image reconstruction model to gradually restore the normal data distribution by minimizing the difference between the predicted noise and the actual noise, thereby enhancing the reconstruction capability of complex defects and anomalies.

[0023] As a preferred technical solution, the feature-guided network extracts features from the middle layer of the variational encoder block And through the cross-attention mechanism, it interacts with the output features of the denoising network to guide the denoising network to reconstruct image details more accurately.

[0024] As a preferred technical solution, the multi-scale feature extraction and comparison network, in the inference stage, compares the input original image x with the reconstructed image Through the pre-trained feature extractor Extract features and generate anomaly score maps by calculating the cosine similarity of feature maps at different scales Anomaly score It is calculated by taking the weighted sum of the scores of all feature layers.

[0025] As a preferred technical solution, the abnormal score map The calculation formula is as follows:

[0026]

[0027] Where n represents the feature extraction layer, The nth layer of features extracted by the pre-trained feature extractor. Anomaly score The weighted sum of the scores of all feature layers is calculated as:

[0028]

[0029] Among them, σ n is the upsampling factor used to keep the pixel space dimension consistent, and N is the set of feature layers used in the inference process.

[0030] In a second aspect, the present invention provides a multi-category defect detection system based on a variational autoencoder and a denoising network, which is applied to the multi-category defect detection method based on a variational autoencoder and a denoising network, including an image acquisition module, an image reconstruction module, an information guidance module, a multi-scale feature comparison module and a model deployment module;

[0031] The image acquisition module is used to acquire and preprocess images of surface defects of industrial products to construct training sets and verification sets;

[0032] The image reconstruction module is used to construct an image reconstruction model, which includes a variational encoder, a feature guidance network, a denoising network and a variational decoder connected in sequence; the variational encoder is used to extract compact features of the image and learn the image feature distribution; the feature guidance network interacts with the denoising network by extracting variational encoder features, and guides the denoising network to perform multi-category correct reconstruction; the denoising network further reconstructs the image features in the feature space to enhance the reconstruction capability of the image reconstruction model; the variational decoder is used to perform preliminary reconstruction of the image according to the features of the latent space;

[0033] The information guidance module is used to train the image reconstruction model in stages using the training set, specifically including: training the variational encoder and the variational decoder in the first stage to learn the feature distribution and reconstruction of normal images; training the denoising network in the second stage to further optimize the reconstruction effect of image features;

[0034] The multi-scale feature comparison module is used to design a multi-scale feature extraction comparison network to extract multi-scale features of the input image and the reconstructed image for comparison, and output an abnormality score map, based on which defect location is achieved;

[0035] The model deployment module is used to input the verification set into the defect detection model composed of the image reconstruction model multi-scale feature extraction and comparison network for verification testing, evaluate the detection performance of the defect detection model, and realize multi-category defect detection in actual industrial scenarios based on the evaluated defect detection model.

[0036] In a third aspect, the present invention provides an electronic device, the electronic device comprising:

[0037] at least one processor; and,

[0038] a memory communicatively connected to the at least one processor; wherein,

[0039] The memory stores computer program instructions that can be executed by the at least one processor, and the computer program instructions are executed by the at least one processor so that the at least one processor can perform the multi-category defect detection method based on variational autoencoder and denoising network.

[0040] In a fourth aspect, the present invention provides a computer-readable storage medium storing a program, which, when executed by a processor, implements the multi-category defect detection method based on a variational autoencoder and a denoising network.

[0041] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0042] 1. The present invention proposes a multi-category defect detection method based on variational autoencoders and denoising networks, which can simultaneously process multiple categories of industrial parts in one model without the need to train a separate model for each category. Compared with the existing method that requires training independent models for different categories, the present invention greatly reduces the complexity of training and deployment, saves a lot of computing resources, and improves the generalization ability of the model.

[0043] 2. By combining the variational autoencoder (VAE) and the denoising network, the present invention can more effectively handle large-scale anomalies and complex texture defects during image reconstruction. VAE is responsible for preliminary feature learning and reconstruction, while the denoising network performs secondary reconstruction in the latent space, so that the model can better capture the potential defect information in the image. Compared with traditional reconstruction methods, the present invention shows significant advantages in detail recovery and reconstruction accuracy, reducing the possibility of false positives and false negatives.

[0044] 3. The present invention uses the information interaction branch (IIB) to guide the denoising U-Net to correctly reconstruct the categories, avoiding the category confusion problem in multi-category scenarios. This mechanism enhances the robustness of the model when processing multi-category data, ensuring that parts of different categories can be accurately reconstructed and detected. Compared with traditional methods, this mechanism significantly improves the adaptability and detection accuracy of the model in multi-category scenarios.

[0045] 4. The present invention uses a multi-scale feature comparison module to compare the multi-layer features of the input image and the reconstructed image, and uses cosine similarity calculation to generate an anomaly score map, which can quickly and accurately locate the defective area on industrial parts. Compared with the existing detection method that relies on pixel-level errors, the feature comparison method of the present invention is more in line with human visual perception, can detect and locate anomalies in a shorter time, and significantly improves the detection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0047] Figure 1 A flowchart of a multi-category defect detection method based on a variational autoencoder and a denoising network according to an embodiment of the present invention;

[0048] Figure 2 This is a flow chart of an image reconstruction module according to an embodiment of the present invention;

[0049] Figure 3 This is a flow chart of a multi-scale feature comparison module according to an embodiment of the present invention;

[0050] Figure 4 A block diagram of a multi-category defect detection method based on a variational autoencoder and a denoising network according to an embodiment of the present invention;

[0051] Figure 5 It is a schematic structural diagram of the electronic device of the present invention. DETAILED DESCRIPTION

[0052] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application.

[0053] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0054] The autoencoder in the name of this application consists of two parts: a variational encoder and a variational decoder.

[0055] See also Figure 1 In one embodiment of the present application, a multi-category defect detection method based on a variational autoencoder and a denoising network is provided, comprising the following steps:

[0056] S1. Collect the surface image data of industrial products through industrial cameras, process the images through the image acquisition module, and build training sets and verification sets; clean, enhance and normalize the collected image data, and convert them into a format that can be recognized by the model;

[0057] Furthermore, the industrial camera is used to complete the data collection of industrial product surface images. The array camera is fixed in a bird's-eye view position, and the sample is illuminated with uniform light to collect bright and uniformly illuminated industrial product surface images. The images are classified; the industrial product surface defect images are preprocessed according to different categories; the images are normalized and the brightness is uniformed through the image acquisition module, and converted into a format that can be recognized by the model. Among the processed images, the images without abnormalities are selected as the training set, and the images with abnormalities and their corresponding abnormal mask images are selected as the verification set.

[0058] S2. Please refer to Figure 2 , establish an image reconstruction model which is connected in sequence by a variational encoder, a feature guidance network, a denoising network and a variational decoder, wherein: the variational encoder is used to extract compact features of the image and learn the image feature distribution; the feature guidance network interacts with the denoising network by extracting the variational encoder features, and guides the denoising network to perform correct multi-category reconstruction; the denoising network further reconstructs the image features in the feature space to enhance the reconstruction capability of the model; the variational decoder is used to perform preliminary reconstruction of the image according to the features of the latent space; it can be understood that the variational autoencoder in this embodiment includes two parts, a variational encoder and a variational decoder, and the denoising network is added between the variational encoder and the variational decoder.

[0059] Furthermore, the multi-category defect detection method based on variational autoencoder and denoising network is characterized in that the step of establishing the image reconstruction model includes:

[0060] S21. Use the pre-trained variational encoder to train the input image Encode and generate the latent space representation Z as follows:

[0061]

[0062] where μ and σ are the mean and variance of the encoder output, respectively, and ∈ is the noise sampled from a standard normal distribution.

[0063] S22, perform partial forward diffusion processing on the latent space variable z to generate the disturbed latent variable z t , as follows:

[0064]

[0065] where α t is the noise scheduling parameter, represents the cumulative noise scheduling, ∈ is the added Gaussian noise.

[0066] S23, the disturbed latent variable z t Input into the denoising network, and output the reconstructed potential representation through a one-step inverse diffusion process Then, the pre-trained variational decoder is used to Decode and restore to the original image level to obtain the reconstructed image The specific process is:

[0067]

[0068] S3, using the training set to train the defect detection model in stages, specifically including: in the first stage, training the variational encoder and the variational decoder to learn the feature distribution and reconstruction of normal images; in the second stage, training the denoising network to further optimize the reconstruction effect of image features;

[0069] Furthermore, the first stage trains a variational encoder and a variational decoder to compress the normal image into the latent space and restore the pixel-level representation. The following loss function is used during training:

[0070] The mean square error (MSE) loss is used to reconstruct the pixel-level error:

[0071]

[0072] Perceptual image similarity (LPIPS) loss is used to enhance the local realism of the reconstructed image and avoid blurring:

[0073]

[0074] Among them, φ l is the feature layer extracted from the pre-trained network.

[0075] The Kullback-Leibler divergence (KL divergence) is used to align the latent space distribution to the prior Gaussian distribution:

[0076]

[0077] Maximum mean difference (MMD) is used to maintain the diversity of latent variables and prevent the latent space from collapsing:

[0078]

[0079] The second stage trains the denoising U-Net to reconstruct the latent space variables, reconstruct large-scale anomalies and texture anomalies, and introduces a feature-guided network to guide the reconstruction of the correct category. The loss function is:

[0080]

[0081] Among them, ∈ θ is the noise predicted by the denoising network, and r is the original data.

[0082] S4. The feature-guided network extracts variational encoder features to interact with the denoising network, guiding the denoising network to perform correct multi-category reconstruction.

[0083] Furthermore, the feature-guided network extracts features from the middle layer of the variational encoder block And through the cross attention mechanism, it interacts with the output features of the denoising network to guide the denoising network to reconstruct image details more accurately. The formula of the cross attention mechanism is:

[0084]

[0085] Where Q is the query vector in the denoising network, K and V are the key and value in the guidance network, respectively. k is the dimension of the key vector.

[0086] S5. Please refer to Figure 3 , the multi-scale features of the input image and the reconstructed image are extracted and compared through the multi-scale feature extraction and comparison network, and an abnormality score map is output, based on which defect location is achieved;

[0087] Furthermore, the multi-scale feature extraction and comparison network is constructed. In the inference stage, the input original image x is compared with the reconstructed image Through the pre-trained feature extractor Extract features and generate anomaly score maps by calculating the cosine similarity of feature maps of different scales The specific formula is as follows:

[0088]

[0089] Where n represents the feature extraction layer, The nth layer of features extracted by the pre-trained feature extractor. Anomaly score The weighted sum of the scores of all feature layers is calculated as:

[0090]

[0091] Among them, σ n is the upsampling factor used to keep the pixel space dimension consistent, and N is the set of feature layers used in the inference process.

[0092] S6. Use the validation set to verify the trained defect detection model, evaluate its detection performance, and deploy the model to the detection system to achieve multi-category defect detection in actual industrial scenarios.

[0093] Furthermore, the defect detection model includes the aforementioned image reconstruction model and a multi-scale feature extraction and comparison network.

[0094] It should be noted that, for the sake of convenience, the aforementioned method embodiments are all expressed as a series of action combinations, but those skilled in the art should know that the present invention is not limited to the described order of actions, because according to the present invention, certain steps can be performed in other orders or simultaneously.

[0095] Based on the same idea as the multi-category defect detection method based on variational autoencoder and denoising network in the above-mentioned embodiment, the present invention also provides a multi-category defect detection system based on variational autoencoder and denoising network, which can be used to execute the above-mentioned multi-category defect detection method based on variational autoencoder and denoising network. For ease of explanation, the structural diagram of the embodiment of the multi-category defect detection system based on variational autoencoder and denoising network only shows the parts related to the embodiment of the present invention. Those skilled in the art can understand that the illustrated structure does not constitute a limitation on the device, and may include more or fewer components than shown in the diagram, or combine certain components, or arrange the components differently.

[0096] See also Figure 4 In another embodiment of the present application, a multi-category defect detection system 100 based on a variational autoencoder and a denoising network is provided, the system comprising an image acquisition module 101, an image reconstruction module 102, an information guidance module 103, a multi-scale feature comparison module 104 and a model deployment module 105;

[0097] The image acquisition module 101 is used to acquire and pre-process images of surface defects of industrial products to construct training sets and verification sets;

[0098] The image reconstruction module 102 is used to construct an image reconstruction model, which includes a variational encoder, a feature guidance network, a denoising network and a variational decoder connected in sequence; the variational encoder is used to extract compact features of the image and learn the image feature distribution; the feature guidance network interacts with the denoising network by extracting the variational encoder features, and guides the denoising network to perform multi-category correct reconstruction; the denoising network further reconstructs the image features in the feature space to enhance the reconstruction capability of the image reconstruction model; the variational decoder is used to perform preliminary reconstruction of the image according to the features of the latent space;

[0099] The information guidance module 103 is used to train the image reconstruction model in stages using the training set, specifically including: training the variational encoder and the variational decoder in the first stage to learn the feature distribution and reconstruction of normal images; training the denoising network in the second stage to further optimize the reconstruction effect of image features;

[0100] The multi-scale feature comparison module 104 is used to design a multi-scale feature extraction comparison network to extract multi-scale features of the input image and the reconstructed image for comparison, and output an abnormality score map, based on which defect location is achieved;

[0101] The model deployment module 105 is used to input the verification set into the defect detection model composed of the image reconstruction model multi-scale feature extraction and comparison network for verification testing, evaluate the detection performance of the defect detection model, and realize multi-category defect detection in actual industrial scenarios based on the evaluated defect detection model.

[0102] It should be noted that the multi-category defect detection system based on variational autoencoder and denoising network of the present invention corresponds one-to-one to the multi-category defect detection method based on variational autoencoder and denoising network of the present invention. The technical features and beneficial effects described in the above-mentioned embodiment of the multi-category defect detection method based on variational autoencoder and denoising network are applicable to the embodiment of multi-category defect detection based on variational autoencoder and denoising network. For specific contents, please refer to the description in the embodiment of the method of the present invention, which will not be repeated here. This is hereby declared.

[0103] In addition, in the implementation of the multi-category defect detection system based on variational autoencoder and denoising network in the above-mentioned embodiment, the logical division of each program module is only an example. In actual applications, the above-mentioned functions can be assigned to different program modules as needed, for example, for the convenience of corresponding hardware configuration requirements or software implementation. That is, the internal structure of the multi-category defect detection system based on variational autoencoder and denoising network is divided into different program modules to complete all or part of the functions described above.

[0104] See also Figure 5In one embodiment, an electronic device for implementing a multi-category defect detection method based on a variational autoencoder and a denoising network is provided. The electronic device 200 may include a first processor 201, a first memory 202 and a bus, and may also include a computer program stored in the first memory 202 and executable on the first processor 201, such as a multi-category defect detection program 203 based on a variational autoencoder and a denoising network.

[0105] The first memory 202 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (such as SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. In some embodiments, the first memory 202 can be an internal storage unit of the electronic device 200, such as a mobile hard disk of the electronic device 200. In other embodiments, the first memory 202 can also be an external storage device of the electronic device 200, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 200. Further, the first memory 202 can also include both an internal storage unit of the electronic device 200 and an external storage device. The first memory 202 can not only be used to store application software and various types of data installed in the electronic device 200, such as the code of the multi-category defect detection program 203 based on the variational autoencoder and the denoising network, but also can be used to temporarily store data that has been output or is to be output.

[0106] In some embodiments, the first processor 201 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips, etc. The first processor 201 is the control core (Control Unit) of the electronic device, and uses various interfaces and lines to connect various components of the entire electronic device, and executes various functions and processes data of the electronic device 200 by running or executing programs or modules stored in the first memory 202, and calling data stored in the first memory 202.

[0107] Figure 5 Only an electronic device with components is shown, and those skilled in the art will understand that Figure 5The structure shown does not constitute a limitation on the electronic device 200 , and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0108] The multi-category defect detection program 203 based on variational autoencoder and denoising network stored in the first memory 202 of the electronic device 200 is a combination of multiple instructions. When running in the first processor 201, it can achieve:

[0109] Acquire and preprocess images of surface defects of industrial products to construct training sets and validation sets;

[0110] An image reconstruction model is constructed, wherein the image reconstruction model includes a variational encoder, a feature guidance network, a denoising network and a variational decoder connected in sequence; the variational encoder is used to extract compact features of an image and learn image feature distribution; the feature guidance network interacts with the denoising network by extracting features of the variational encoder to guide the denoising network to perform multi-category correct reconstruction; the denoising network further reconstructs image features in a feature space to enhance the reconstruction capability of the image reconstruction model; the variational decoder is used to perform preliminary reconstruction of the image according to features of the latent space;

[0111] The image reconstruction model is trained in stages using the training set, specifically including: in the first stage, a variational encoder and a variational decoder are trained to learn the feature distribution and reconstruction of normal images; in the second stage, a denoising network is trained to further optimize the reconstruction effect of image features;

[0112] Design a multi-scale feature extraction and comparison network to extract multi-scale features of the input image and the reconstructed image for comparison, and output an abnormality score map, based on which defect location is achieved;

[0113] The verification set is input into the defect detection model composed of the image reconstruction model and multi-scale feature extraction and comparison network for verification testing to evaluate the detection performance of the defect detection model. Based on the evaluated defect detection model, multi-category defect detection in actual industrial scenarios is realized.

[0114] Furthermore, if the module / unit integrated in the electronic device 200 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a non-volatile computer-readable storage medium. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, and a read-only memory (ROM).

[0115] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0116] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0117] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A multi-category defect detection method based on variational autoencoder and denoising network, characterized in that: The steps include: Acquire and preprocess images of surface defects of industrial products to construct training sets and validation sets; An image reconstruction model is constructed, wherein the image reconstruction model includes a variational encoder, a feature guidance network, a denoising network and a variational decoder connected in sequence; the variational encoder is used to extract compact features of an image and learn image feature distribution; the feature guidance network interacts with the denoising network by extracting features of the variational encoder to guide the denoising network to perform multi-category correct reconstruction; the denoising network further reconstructs image features in a feature space to enhance the reconstruction capability of the image reconstruction model; the variational decoder is used to perform preliminary reconstruction of the image according to features of the latent space; The image reconstruction model is trained in stages using the training set, specifically including: in the first stage, a variational encoder and a variational decoder are trained to learn the feature distribution and reconstruction of normal images; in the second stage, a denoising network is trained to further optimize the reconstruction effect of image features; Design a multi-scale feature extraction and comparison network to extract multi-scale features of the input image and the reconstructed image for comparison, and output an abnormality score map, based on which defect location is achieved; The verification set is input into the defect detection model composed of an image reconstruction model and a multi-scale feature extraction and comparison network for verification testing to evaluate the detection performance of the defect detection model. Based on the evaluated defect detection model, multi-category defect detection in actual industrial scenarios is realized.

2. According to claim 1, the multi-category defect detection method based on variational autoencoder and denoising network is characterized in that: The acquisition of industrial product surface defect images and preprocessing to construct a training set and a verification set is specifically as follows: Use industrial cameras to complete the image data collection of industrial product surfaces. Fix the area array camera in a bird's-eye view position, illuminate the sample with uniform light, and collect bright and uniformly illuminated industrial product surface images; classify the images; perform data preprocessing on industrial product surface defect images according to different categories; After passing through the image acquisition module, it is normalized, the brightness is evened out, and converted into a format that can be recognized by the model; Clean, enhance and normalize the collected image data and convert it into a format that can be recognized by the model; Among the preprocessed images, images without abnormalities are selected as training sets, and images with abnormalities and their corresponding abnormal mask images are selected as verification sets.

3. According to claim 1, the multi-category defect detection method based on variational autoencoder and denoising network is characterized in that: The constructing of the image reconstruction model is specifically as follows: Use the pre-trained variational encoder to compress the input image x into the latent space representation z and extract the intermediate layer features f i , where Z generates the perturbed latent variable z through a partial forward diffusion process with added noise t ;f i Input feature guided network to guide denoising network; z t Input denoising network, output reconstructed latent representation through one-step inverse diffusion process And use the pre-trained variational decoder to Restore to the original image level and get the reconstructed image 4. According to claim 1, the multi-category defect detection method based on variational autoencoder and denoising network is characterized in that: The image reconstruction model is trained in stages using the training set, specifically: In the first stage of training, the variational encoder and variational decoder are trained to compress the normal image into the latent space and restore the pixel-level representation, and the mean square error loss and the perceptual image similarity loss are used as the reconstruction loss metrics, and the KL divergence and the maximum mean difference are used to constrain the latent space features; the mean square error loss is used to reconstruct the pixel-level error, and the perceptual image similarity loss is used to enhance the local realism of the reconstructed image; In the second stage of training, the denoising network is trained to perform secondary reconstruction of the latent space variables, reconstruct large-scale anomalies and texture anomalies, and a feature-guided network is introduced to guide the reconstruction of the correct category; the loss function of the second stage guides the image reconstruction model to gradually restore the normal data distribution by minimizing the difference between the predicted noise and the actual noise, thereby enhancing the reconstruction capability of complex defects and anomalies.

5. According to claim 1, the multi-category defect detection method based on variational autoencoder and denoising network is characterized in that: The feature-guided network extracts features from the intermediate layers of the variational encoder block And through the cross-attention mechanism, it interacts with the output features of the denoising network to guide the denoising network to reconstruct image details more accurately.

6. The multi-category defect detection method based on variational autoencoder and denoising network according to claim 1, characterized in that: The multi-scale feature extraction and comparison network, in the inference stage, compares the input original image x with the reconstructed image Through the pre-trained feature extractor Extract features and generate anomaly score maps by calculating the cosine similarity of feature maps at different scales Anomaly score It is calculated by taking the weighted sum of the scores of all feature layers.

7. The multi-category defect detection method based on variational autoencoder and denoising network according to claim 6 is characterized in that: The anomaly score graph The calculation formula is as follows: Among them, n represents the feature extraction layer, The nth layer features extracted by the pre-trained feature extractor, anomaly score The weighted sum of the scores of all feature layers is calculated as: Among them, σ n is the upsampling factor used to keep the pixel space dimension consistent, and N is the set of feature layers used in the inference process.

8. A multi-category defect detection system based on variational autoencoder and denoising network, characterized in that: A multi-category defect detection method based on a variational autoencoder and a denoising network applied to any one of claims 1 to 7, comprising an image acquisition module, an image reconstruction module, an information guidance module, a multi-scale feature comparison module, and a model deployment module; The image acquisition module is used to acquire and preprocess images of surface defects of industrial products to construct training sets and verification sets; The image reconstruction module is used to construct an image reconstruction model, which includes a variational encoder, a feature guidance network, a denoising network and a variational decoder connected in sequence; the variational encoder is used to extract compact features of the image and learn the image feature distribution; the feature guidance network interacts with the denoising network by extracting variational encoder features, and guides the denoising network to perform multi-category correct reconstruction; the denoising network further reconstructs the image features in the feature space to enhance the reconstruction capability of the image reconstruction model; the variational decoder is used to perform preliminary reconstruction of the image according to the features of the latent space; The information guidance module is used to train the image reconstruction model in stages using the training set, specifically including: training the variational encoder and the variational decoder in the first stage to learn the feature distribution and reconstruction of normal images; The second stage trains the denoising network to further optimize the reconstruction effect of image features; The multi-scale feature comparison module is used to design a multi-scale feature extraction comparison network to extract multi-scale features of the input image and the reconstructed image for comparison, and output an abnormality score map, based on which defect location is achieved; The model deployment module is used to input the verification set into the defect detection model composed of the image reconstruction model multi-scale feature extraction and comparison network for verification testing, evaluate the detection performance of the defect detection model, and realize multi-category defect detection in actual industrial scenarios based on the evaluated defect detection model.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores computer program instructions that can be executed by the at least one processor, and the computer program instructions are executed by the at least one processor so that the at least one processor can execute the multi-category defect detection method based on variational autoencoder and denoising network as described in any one of claims 1-7.

10. A computer-readable storage medium storing a program, characterized in that: When the program is executed by a processor, the multi-category defect detection method based on a variational autoencoder and a denoising network as described in any one of claims 1 to 7 is implemented.

Images