Defect identification method and device fusing discriminant model and generative model

Abstract

The embodiment of the invention discloses a defect identification method fusing a discriminant model and a generative model, and the method comprises the steps: obtaining a first training data set, the first training data set comprises at least one training sample, and each training sample comprises a sample image and a defect marking result corresponding to the sample image; obtaining a production parameter set, wherein the production parameter set is a set of externally input element parameters related to a production link; generating the first training data set through a preset generative model and fusing the production parameter set to obtain a second training data set; based on the second training data set, training a preset discriminant model to obtain a trained discriminant model; and obtaining a to-be-detected image, inputting the to-be-detected image into the discriminant model, and obtaining an output result of the discriminant model as a defect classification result of the to-be-detected image. According to the invention, the defect identification accuracy and identification efficiency can be improved.

Description

Technical Field

[0001] The present invention relates to the field of industrial visual automated detection technology, and in particular to a defect recognition method, device, computer equipment and computer-readable storage medium that integrate a discriminant model and a generative model. Background Art

[0002] In the scenario of industrial production, defect detection is required during the production process to determine whether the products are good. In the relevant industrial yield management model, full use can be made of the industrial production process, that is, the centralized analysis of various massive data in the production line and the new data set of the status of unknown defects should be considered, while the fragmented application of industrial scenarios and the timeliness and accuracy of problem judgment in the rapid production process should be considered to improve the production yield. However, the accuracy of defect identification in the existing defect detection is still insufficient. For example, when new types of defects appear in the generation process, there may be unrecognizable phenomena, resulting in missed detection or over-inspection, which leads to insufficient accuracy of defect detection, thus affecting the production yield. Summary of the invention

[0003] Based on this, it is necessary to propose a defect recognition method, device, computer equipment and computer-readable storage medium that integrates discriminant model and generative model to address the above problems.

[0004] In the first part of the present invention, a defect recognition method integrating a discriminant model and a generative model is provided, the method comprising: Acquire a first training data set, where the first training data set includes at least one training sample, and each training sample includes a sample image and a defect annotation result corresponding to the sample image; Acquire a production parameter set, which is a set of factor parameters related to the externally input production process; generate and process the first training data set by integrating the production parameter set through a preset generative model to obtain a second training data set; Based on the second training data set, the preset discriminant model is trained to obtain a trained discriminant model; An image to be detected is obtained, the image to be detected is input into the discriminant model, and an output result of the discriminant model is obtained as a defect classification result of the image to be detected.

[0005] Optionally, the defect annotation result includes a defect recognition result corresponding to the sample image and defect feature data when the defect recognition result indicates that a defect exists, and the defect feature includes a defect picture and a defect description text.

[0006] Optionally, after the step of obtaining the first training data set, it also includes: based on preset defect categories, classifying the first training data set according to a preset classification algorithm to determine a training subset under each defect category, wherein the preset classification algorithm is a deep learning model, and the deep learning model includes a combination of one or more of ThunderNet, PeleeNet, Yolo, EfficientDet, RFBNet, Corner-Lite, CBnet, and DetectoRS.

[0007] Optionally, before the step of generating and processing the first training data set by using a preset generative model and fusing the production parameter set to obtain a second training data set, the step also includes: based on a preset cross-modal large model, mapping the defect images and defect description texts to a preset latent space, wherein in the latent space, the defect images and the defect description texts under the same defect category are associated; based on a preset loss function, pre-training the first training data set mapped to the latent space to obtain a first training data set after pre-training.

[0008] Optionally, the step of generating and processing the first training data set through a preset generative model and fusing the production parameter set to obtain a second training data set also includes: inputting the generation parameter set into the generative model, and generating and processing the defective training samples in the first training data set through the network architecture contained in the generative model to generate new training samples, wherein the new training samples constitute the second training data set.

[0009] Optionally, the step of training the preset discriminant model based on the second training data set to obtain a trained discriminant model further includes: inputting the second training data set into the preset discriminant model, and training the preset discriminant model based on training samples included in the second training data set and defect labeling results corresponding to the training samples; The steps of obtaining an image to be detected, inputting the image to be detected into the discriminant model, and obtaining the output result of the discriminant model as the defect classification result of the image to be detected, also include: obtaining an image to be detected, inputting the image to be detected into the discriminant model; calculating the probability value corresponding to the image to be detected and each defect category through the discriminant model, and determining the defect classification result of the image to be detected according to the probability value corresponding to each defect category.

[0010] Optionally, the step of determining the defect classification result of the image to be detected according to the probability value corresponding to each defect category also includes: taking the defect category with the largest probability value as the target defect category of the image to be detected, and determining the defect classification result of the image to be detected according to the target defect category; wherein, if the probability values ​​corresponding to all defect categories are less than a preset probability threshold, it is determined that there is no defect in the image to be detected.

[0011] In the second part of the present invention, a defect recognition device integrating a discriminant model and a generative model is provided, the device comprising: A first training data set acquisition module, used to acquire a first training data set, wherein the first training data set includes at least one training sample, each training sample includes a sample image and a defect annotation result corresponding to the sample image; A generative model processing module is used to obtain a production parameter set, which is a set of factor parameters related to the production process input externally; through a preset generative model and integrating the production parameter set, the first training data set is generated and processed to obtain a second training data set; A discriminant model processing module, used to train the preset discriminant model based on the second training data set to obtain a trained discriminant model; The defect classification module is used to obtain the image to be detected, input the image to be detected into the discriminant model, and obtain the output result of the discriminant model as the defect classification result of the image to be detected.

[0012] In the third part of the present invention, a computer device is provided, which includes a memory and a processor, the memory has executable code, and when the executable code is run on the processor, the defect recognition method of fusing the discriminant model and the generative model as described in the first part is implemented.

[0013] In the fourth part of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium is used to store a computer program, wherein the computer program is used to execute the defect recognition method of fusing the discriminant model and the generative model as described in the first part.

[0014] The embodiments of the present invention have the following beneficial effects: After adopting the above-mentioned defect recognition method, device and corresponding computer equipment and readable storage medium that integrate the discriminant model and the generative model, in the process of generating the defect classification model, firstly, a first training data set is established based on the training samples, wherein each training sample includes a sample image and a defect annotation result corresponding to the sample image, and then a production parameter set is obtained, wherein the production parameter set is a set of element parameters related to the production link input externally; through a preset generative model and integrating the production parameter set, the first training data set is generated and processed to obtain a second training data set, so as to obtain a new training data set; secondly, based on the second training data set, the preset discriminant model is trained to obtain a trained discriminant model, so that the discriminant model can calculate the probability of the image that needs to be defectively identified under each defect category; and in the defect recognition scenario, the image to be detected is obtained, the image to be detected is input into the discriminant model, and the output result of the discriminant model is obtained as the defect classification result of the image to be detected. Among them, the construction concept of the generative model and the discriminative model is integrated to solve the problem of insufficient scalability of defect detection in the existing technology. The generative model is used to extend the defect category to unknown or weakly known new defect forms, thereby solving the phenomenon of being unable to recognize the emergence of new defect categories during the generation process and improving the accuracy of defect identification. In addition, the discriminative model is used to learn the characteristics of new defect categories and then calculate the probability under each defect category, further improving the accuracy and speed of defect identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0016] in: Figure 1 A schematic diagram of a flow chart of a defect recognition method integrating a discriminant model and a generative model in one embodiment; Figure 2 is a schematic diagram of mapping a first training data set to a latent space in one embodiment; Figure 3 A schematic diagram of a defect recognition process integrating a discriminative model and a generative model in one embodiment; Figure 4 It is a structural schematic diagram of a defect recognition device integrating a discriminant model and a generative model in one embodiment; Figure 5The figure is a schematic diagram of the structure of a computer device for running the defect recognition method integrating the discriminant model and the generative model in one embodiment. DETAILED DESCRIPTION

[0017] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] In this embodiment, a defect recognition method that integrates a discriminant model and a generative model is provided to improve the accuracy of defect recognition in the production process and improve the yield rate in the production process. Specifically, the defect recognition method that integrates a discriminant model and a generative model is applied to defect detection in the industrial production process. During the production process, images corresponding to the products produced are collected, and then analyzed based on the images to determine whether the product has defects. Among them, the process of determining whether a product has defects based on an image is based on recognition based on a large model.

[0019] See also Figure 1 , a flow chart of the defect recognition method integrating the discriminant model and the generative model is given, wherein the defect recognition method integrating the discriminant model and the generative model includes: Figure 1 Steps S101-S104 shown: Step S101: Acquire a first training data set, where the first training data set includes at least one training sample, and each training sample includes a sample image and a defect annotation result corresponding to the sample image.

[0020] In this step, it is necessary to manually collect images corresponding to various defects that occur in products during the industrial production process in advance, mark them, and establish a training data set (first training data set). For each training sample in the first training data set, determine whether it has a defect (OK or NG), and in the case where the image corresponding to the product has a defect, it is also necessary to determine the defect image and defect description text corresponding to the existing defect. That is to say, in this embodiment, the first training data set includes multiple training samples, and these multiple training samples include multiple training samples without defects, and the defect recognition result is that there is no defect. Further, these multiple training samples also include multiple training samples with defects, and the defect recognition result is that there is a defect, and further include defect images corresponding to the existing defects and corresponding defect description text.

[0021] Among them, corresponding to each training sample, it includes a sample image corresponding to the training sample, and a defect annotation result corresponding to the sample image, wherein the defect annotation result includes a defect recognition result corresponding to the sample image of whether a defect exists, and defect feature data describing the corresponding existing defect when a defect exists, wherein the defect feature includes a defect picture and a corresponding defect description text.

[0022] In this embodiment, the defect feature data includes a defect image in a graphical form and a defect description text in a textual form.

[0023] It should be noted that, in this embodiment, the acquisition and labeling of training samples can be obtained by using the standard samples provided in the production process as a reference, which are the initial samples and need to be obtained through manual labeling or other means in the early stage. Among them, for training samples with defects, the data set corresponding to the training samples with defects in defective morphology, parameter indicators, etc. can be obtained.

[0024] Furthermore, in this embodiment, in order to facilitate the subsequent processing of defects of different defect categories, in this embodiment, after step S101, it also includes: based on the preset defect category, the first training data set is classified according to the preset classification algorithm to determine the training subset under each defect category, that is, to establish a data set corresponding to each defect type. Wherein, the preset classification algorithm is a deep learning model, and the deep learning model includes a combination of one or more of ThunderNet, PeleeNet, Yolo, EfficientDet, RFBNet, Corner-Lite, CBnet, and DetectoRS.

[0025] Step S102: Obtain a production parameter set, which is a collection of factor parameters related to the externally input production process; generate and process the first training data set through a preset generative model and integrating the production parameter set to obtain a second training data set.

[0026] In this step, it is necessary to establish a new second training data set based on the preset production model and the first training data set.

[0027] Specifically, before establishing a new second training data set, the first training data set needs to be preprocessed.

[0028] In the specific implementation, based on the preset cross-modal big model, the defect pictures and defect description texts are mapped to a preset latent space, wherein in the latent space, the defect pictures and defect description texts under the same defect category are associated with each other; based on the preset loss function, the first training data set mapped to the latent space is pre-trained to obtain the first training data set after pre-training.

[0029] Among them, based on the preset cross-modal large model, the (training subset) in the first training data set under each defect category is mapped, and the defect pictures and defect description texts corresponding to the corresponding training samples are mapped to the preset latent space. In this mapping process, the BLIP model in the large model is used to map the image-shaped defect pictures and the text-shaped defect description texts to the same latent space through the image encoder and the text encoder to achieve the matching of image-text correlation. That is to say, in this latent space, the defect pictures and defect description texts under the same defect category are associated. Here, the BLIP model is an upgrade of the CLIP model. On the basis of the CLIP model, a cross-modal encoder and decoder are added to realize cross-modal information flow. For details, please refer to Figure 2 shown.

[0030] Furthermore, the cross-modal large model can be pre-trained on the first training data set mapped to the same factor space by using multiple different loss functions. The loss functions are the following three loss functions.

[0031] (1) Image-Text Contrastive Loss (ITC), for image encoder and text encoder, aligns the latent feature space of image and text by comparing positive and negative image-text pairs; (2) Image-Text Matching Loss (ITM), for image-based text encoders, models the correlation of multimodal information between images and text by performing binary classification on the matching of images and texts; (3) Language Modeling Loss (LM), which is optimized for the image-based text decoder through cross entropy loss, trains the model to generate target text in an autoregressive manner.

[0032] After the pre-training of the first training data set is completed, it can be further processed based on the generative model to generate a new second training data set based on the original first training data set. Specifically, the first training data set is generated and processed by a preset generative model and the production parameter set is integrated to obtain the second training data set. Specifically, the generation parameter set is input into the generative model, and the training samples with defects in the first training data set are generated and processed by the network architecture included in the generative model to generate new training samples, wherein the new training samples constitute the second training data set.

[0033] The generative model is the most important basic model in AIGC. The generative model is based on statistics and Bayes. It mainly includes four categories: GAN, VAE, Flow-based Model, Diffusion Model, and has achieved great success in generating high-quality samples. Among them, GAN generates samples through adversarial training, VAE relies on variational processes, Flow-based Model must use a special network to build reversible transformations, and Diffusion Model is inspired by thermodynamics. It slowly adds random noise to the data through the diffusion process, and then learns to reverse the diffusion process to build the required samples from the noise. The main core algorithm models: Naive Bayes, Hidden Markov Model, etc. In this step, through the generative model, external parameters are added to the original first training data set, and through GAN adversarial network modeling, a new second training data set of the probability distribution of the previously unknown or weakly known defect categories is generated. After the second training data set is generated, the defect recognition model can expand the depth of physical defects and improve the accuracy of defect recognition detection. Among them, through the original training samples with defects, the external production parameter set is integrated as the external conditions, and various network architectures such as Silkworm Pupa CAN are used to update and extend the original defect categories, forming a multilateral defect cognition. However, the overall defects still revolve around the target setting defects, with more diversified and refined forms and higher accuracy classification.

[0034] In the generative model, the basic training data set is the first training data set, in which each training sample is a data point, that is, an observation. Each observation consists of multiple defect features. For image generation problems, features are usually pixel values ​​of each pixel, and for text generation, features are usually words or letter combinations. In the processing of the generative model, the purpose is to generate a new feature set (second training data set) created using the same rules as the original data (first training data set). For image generation, the generated images need to have the characteristics and details of the real world, including texture, color, shape, etc. At the same time, the generative model requires a large amount of training data to learn the characteristics and distribution of the image. The generative model is also probabilistic rather than deterministic. It is hoped that outputs with different changes can be sampled instead of the same output every time. For image data, the generated images should have different styles, angles, and changes, rather than just copying the images in the training data.

[0035] In the second training data set, a training data set of previously unknown or similar defects is formed, and the defect categories in the new second training data set are closer to the actual defects of the production products that need to be tested. Among them, the accuracy of controlling product defects or good product problems in related production links can be improved, and the accuracy and efficiency of defect identification can be improved.

[0036] Furthermore, the first training data set can be expressed as y , , in, y Represents different defects, each defect has multiple characteristic values These defects Constitute training subsets under different defect categories, where , Among them, defects By multiple features composition.

[0037] Step S103: Based on the second training data set, the preset discriminant model is trained to obtain a trained discriminant model.

[0038] In this step, based on the generative model, the discriminant model is further integrated to process the training data set. The discriminant model is the conditional probability distribution obtained in the learning process. , that is, in the feature x Marking when it occurs yThe probability of occurrence includes a combination of one or more algorithms such as perceptron (linear classification model), k-nearest neighbor method, decision tree, logistic regression model, maximum entropy model, support vector machine (SVM), boosting method (AdaBoost, etc.), conditional random field (CRF), CNN, etc. In this step, the discriminant model directly classifies or regresses the second training data set, where the classification algorithm or regression algorithm can be a combination of one or more of logistic regression, k-nearest neighbor, perceptron, decision tree, SVM, which can provide higher performance in classification decision-making, complete data classification, establish boundaries of different categories, and expand the fault tolerance of defective products.

[0039] For example, a dataset of defective images of textile fabrics contains images of holes, wrinkles, oil stains, broken tendons, folds, fish scales, dirt, and other defects. By using enough data, a discriminant model can be trained to predict whether a given defect image is a hole defect image. The discriminant model can learn features such as color, shape, and texture in the image to determine whether the defect image is a hole class. For images with holes, the discriminant model will increase its prediction weight accordingly.

[0040] The second training data set is input into the preset discriminant model, and the preset discriminant model is trained based on the training samples contained in the second training data set and the defect labeling results corresponding to the training samples. Each training sample contained in the second training data set is an observation value (i.e., defect y), which corresponds to a label, and this label is the defect labeling result. For a binary classification problem, for example, a classifier for whether it is a hole defect, the picture of the hole defect is marked as 0, and the picture of the non-hole defect is marked as 1. Then the discriminant model learns how to distinguish the pictures corresponding to multiple groups of training samples, and predicts the probability that the sample in the new second training data belongs to label 0, that is, the probability that the sample contains a hole defect.

[0041] In this embodiment, the features of defect images and defect description texts of different defect categories are learned through a discriminant model, and the boundaries corresponding to each defect category are obtained through training, so that the trained discriminant model can calculate the probability of the image to be classified in each defect category.

[0042] Step S104: obtaining an image to be detected, inputting the image to be detected into the discriminant model, and obtaining an output result of the discriminant model as a defect classification result of the image to be detected.

[0043] After the training of the discriminant model is completed, defect recognition can be performed based on the trained discriminant model. Specifically, in a scenario where defect detection is required, an image corresponding to the production product that needs to be defect-detected is collected, that is, the image to be detected, and then the image to be detected is input into the discriminant model, and the image to be detected is subjected to defect recognition through the discriminant model to output the defect classification result of the image to be detected. Among them, the image to be detected is obtained, and the image to be detected is input into the discriminant model; the probability value corresponding to the image to be detected and each defect category is calculated by the discriminant model, and the defect classification result of the image to be detected is determined according to the probability value corresponding to each defect category. In other words, the defect category with the largest probability value is used as the target defect category of the image to be detected, and the defect classification result of the image to be detected is determined according to the target defect category; wherein, if the probability values ​​corresponding to all defect categories are less than the preset probability threshold, it is determined that the image to be detected does not have defects.

[0044] For further information, see Figure 3 , pre-label the training samples, obtain a training data set containing defect labeling results, and then train the generative model based on the external parameters and the training data set to generate a new training data set, wherein the new training data set contains training samples of previously unknown or similar product defects; then the new training data set is input into the discriminant model for model training, so that the discriminant model can perform defect recognition on new defect images that need to be recognized, and output the corresponding defect classification results.

[0045] In the defect recognition method that integrates the discriminant model and the generative model, the defect recognition method integrates the generative model and the discriminant model, and combines the core algorithm of the discriminant model with the core algorithm of the generative model, so as to quickly generate targeted new training data sets, expand the defect classification model's deep understanding of defects, and have better performance in classification decisions, thereby expanding the fault tolerance capability of defect recognition.

[0046] For further information, see Figure 4 , provides a schematic diagram of the structure of a defect recognition device that integrates a discriminant model and a generative model according to another embodiment, wherein the defect recognition device that integrates a discriminant model and a generative model includes: Figure 4 The following modules are shown: A first training data set acquisition module 101 is used to acquire a first training data set, where the first training data set includes at least one training sample, each training sample includes a sample image and a defect annotation result corresponding to the sample image; The generative model processing module 102 is used to obtain a production parameter set, which is a set of factor parameters related to the production process input externally; generate and process the first training data set through a preset generative model and integrating the production parameter set to obtain a second training data set; The discriminant model processing module 103 is used to train the preset discriminant model based on the second training data set to obtain a trained discriminant model; The defect classification module 104 is used to obtain the image to be detected, input the image to be detected into the discriminant model, and obtain the output result of the discriminant model as the defect classification result of the image to be detected.

[0047] Optionally, in another embodiment, the defect annotation result includes a defect recognition result corresponding to the sample image and defect feature data when the defect recognition result is a defect, and the defect feature includes a defect picture and a defect description text.

[0048] Optionally, in another embodiment, the first training data set acquisition module is used to classify the first training data set based on preset defect categories and according to a preset classification algorithm to determine a training subset under each defect category, wherein the preset classification algorithm is a deep learning model, and the deep learning model includes a combination of one or more of ThunderNet, PeleeNet, Yolo, EfficientDet, RFBNet, Corner-Lite, CBnet, and DetectoRS.

[0049] Optionally, in another embodiment, the generative model processing module 102 is also used to map the defect images and defect description texts to a preset latent space based on a preset cross-modal large model, wherein in the latent space, the defect images and the defect description texts under the same defect category are associated with each other; and based on a preset loss function, pre-train the first training data set mapped to the latent space to obtain a first training data set after pre-training.

[0050] Optionally, in another embodiment, the generative model processing module 102 is further used to input the generation parameter set into the generative model, and to perform generation processing on defective training samples in the first training data set through the network architecture contained in the generative model to generate new training samples, wherein the new training samples constitute the second training data set.

[0051] Optionally, in another embodiment, the discriminant model processing module 103 is also used to input the second training data set into the preset discriminant model, train the preset discriminant model based on the training samples contained in the second training data set and the defect labeling results corresponding to the training samples; obtain the image to be detected, and input the image to be detected into the discriminant model; calculate the probability value corresponding to the image to be detected and each defect category through the discriminant model, and determine the defect classification result of the image to be detected according to the probability value corresponding to each defect category.

[0052] Optionally, in another embodiment, the defect classification module 104 is also used to take the defect category with the largest probability value as the target defect category of the image to be detected, and determine the defect classification result of the image to be detected according to the target defect category; wherein, if the probability values ​​corresponding to all defect categories are less than a preset probability threshold, it is determined that there is no defect in the image to be detected.

[0053] Figure 5 The internal structure diagram of a computer device that implements the defect recognition method of integrating the discriminant model and the generative model in one embodiment is shown. The computer device can be a terminal or a server. Figure 5 As shown, the computer device includes a processor, a memory and a network interface connected via a system bus. The memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and may also store a computer program. When the computer program is executed by the processor, the processor can implement the above method. The internal memory may also store a computer program. When the computer program is executed by the processor, the processor can implement the above method. Those skilled in the art can understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0054] After adopting the above-mentioned defect recognition method, device and corresponding computer equipment and readable storage medium that integrate the discriminant model and the generative model, in the process of generating the defect classification model, firstly, a first training data set is established based on the training samples, wherein each training sample includes a sample image and a defect annotation result corresponding to the sample image, and then a production parameter set is obtained, wherein the production parameter set is a set of element parameters related to the production link input externally; through a preset generative model and integrating the production parameter set, the first training data set is generated and processed to obtain a second training data set, so as to obtain a new training data set; secondly, based on the second training data set, the preset discriminant model is trained to obtain a trained discriminant model, so that the discriminant model can calculate the probability of the image that needs to be defectively identified under each defect category; and in the defect recognition scenario, the image to be detected is obtained, the image to be detected is input into the discriminant model, and the output result of the discriminant model is obtained as the defect classification result of the image to be detected. Among them, the construction concept of the generative model and the discriminative model is integrated to solve the problem of insufficient scalability of defect detection in the existing technology. The generative model is used to extend the defect category to unknown or weakly known new defect forms, thereby solving the phenomenon of being unable to recognize the emergence of new defect categories during the generation process and improving the accuracy of defect identification. In addition, the discriminative model is used to learn the characteristics of new defect categories and then calculate the probability under each defect category, further improving the accuracy and speed of defect identification.

[0055] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0056] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0057] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A defect recognition method integrating a discriminant model and a generative model, characterized in that: The method comprises: Acquire a first training data set, where the first training data set includes at least one training sample, and each training sample includes a sample image and a defect annotation result corresponding to the sample image; Acquire a production parameter set, which is a set of factor parameters related to the externally input production process; generate and process the first training data set by integrating the production parameter set through a preset generative model to obtain a second training data set; Based on the second training data set, the preset discriminant model is trained to obtain a trained discriminant model; An image to be detected is obtained, the image to be detected is input into the discriminant model, and an output result of the discriminant model is obtained as a defect classification result of the image to be detected.

2. The defect recognition method integrating the discriminant model and the generative model according to claim 1 is characterized in that: The defect annotation result includes a defect recognition result corresponding to the sample image and defect feature data when the defect recognition result indicates that a defect exists, and the defect feature includes a defect picture and a defect description text.

3. The defect recognition method integrating the discriminant model and the generative model according to claim 1 is characterized in that: After the step of obtaining the first training data set, the method further includes: Based on preset defect categories, the first training data set is classified and processed according to a preset classification algorithm to determine a training subset under each defect category, wherein the preset classification algorithm is a deep learning model, and the deep learning model includes a combination of one or more of ThunderNet, PeleeNet, Yolo, EfficientDet, RFBNet, Corner-Lite, CBnet, and DetectoRS.

4. The defect recognition method integrating the discriminant model and the generative model according to claim 2 is characterized in that: Before the step of generating and processing the first training data set by using a preset generative model and fusing the production parameter set to obtain a second training data set, the method further includes: Based on a preset cross-modal large model, the defect images and defect description texts are mapped to a preset latent space, wherein in the latent space, defect images and defect description texts under the same defect category are associated; Based on a preset loss function, the first training data set mapped to the latent space is pre-trained to obtain a first training data set after pre-training.

5. The defect recognition method integrating the discriminant model and the generative model according to claim 4 is characterized in that: The step of generating and processing the first training data set by using a preset generative model and fusing the production parameter set to obtain a second training data set also includes: The generation parameter set is input into the generative model, and the training samples with defects in the first training data set are generated through the network architecture included in the generative model to generate new training samples, wherein the new training samples constitute a second training data set.

6. The defect recognition method integrating the discriminant model and the generative model according to claim 5 is characterized in that: The step of training the preset discriminant model based on the second training data set to obtain a trained discriminant model further includes: Inputting the second training data set into the preset discriminant model, and training the preset discriminant model based on the training samples included in the second training data set and the defect labeling results corresponding to the training samples; The step of acquiring the image to be detected, inputting the image to be detected into the discriminant model, and acquiring the output result of the discriminant model as the defect classification result of the image to be detected further includes: Acquire an image to be detected, and input the image to be detected into the discriminant model; The probability value corresponding to the image to be detected and each defect category is calculated by the discriminant model, and the defect classification result of the image to be detected is determined according to the probability value corresponding to each defect category.

7. The defect recognition method integrating the discriminant model and the generative model according to claim 6 is characterized in that: The step of determining the defect classification result of the image to be detected according to the probability value corresponding to each defect category also includes: Taking the defect category with the largest probability value as the target defect category of the image to be detected, and determining the defect classification result of the image to be detected according to the target defect category; If the probability values ​​corresponding to all defect categories are smaller than a preset probability threshold, it is determined that the image to be detected has no defects.

8. A defect recognition device integrating a discriminant model and a generative model, characterized in that: The device comprises: A first training data set acquisition module, used to acquire a first training data set, wherein the first training data set includes at least one training sample, each training sample includes a sample image and a defect annotation result corresponding to the sample image; A generative model processing module is used to obtain a production parameter set, which is a set of factor parameters related to the production process input externally; through a preset generative model and integrating the production parameter set, the first training data set is generated and processed to obtain a second training data set; A discriminant model processing module, used to train the preset discriminant model based on the second training data set to obtain a trained discriminant model; The defect classification module is used to obtain the image to be detected, input the image to be detected into the discriminant model, and obtain the output result of the discriminant model as the defect classification result of the image to be detected.

9. A computer device, characterized in that: The computer device includes a memory and a processor, the memory has executable code, and when the executable code is run on the processor, the defect recognition method of fusing the discriminant model and the generative model as described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program, wherein the computer program is used to execute the defect recognition method of fusing a discriminant model and a generative model as described in any one of claims 1 to 7.

Images