Small sample surface defect image generation method and system based on feedback reinforcement learning
By using feedback reinforcement learning method in small sample data scenarios, combining conditional diffusion model and reward model, surface defect image generation is optimized, and the problem of detection of small and medium sample data in the existing technology is solved, and high-quality and diverse image generation is achieved, improving The accuracy and industrial application capabilities of surface defect detection.
Patent Information
- Application Number
- CN202311781415.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-12-21
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2043-12-21
AI Technical Summary
Existing surface defect image generation methods based on deep learning and GAN are difficult to effectively detect in small sample data scenarios, and the generated images are limited in diversity and quality, which cannot meet industrial needs.
Using a small sample surface defect image generation method based on feedback reinforcement learning, the conditional diffusion model and reward model are introduced through manual feedback, and the generative model is optimized to improve the quality and diversity of surface defect image samples, and secondary optimization is performed using semantic labels and manual evaluation. , generate the optimal surface defect image.
In small sample data scenarios, the performance and detection accuracy of surface defect image generation are significantly improved, and high-quality diversified surface defect images can be generated, which improves the ability to detect surface defects and meets industrial needs.
Smart Images

Figure CN117710349B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing technology, and in particular to a method and system for generating small sample surface defect images based on feedback reinforcement learning. Background Art
[0002] At present, in industrial operations, surface defect detection is a key link to ensure the quality of industrial products. With the development of deep learning, surface defect detection methods based on deep learning have become the mainstream method to solve this problem. Through visual means such as image classification, target detection, and image segmentation, various needs of surface defect detection at different granularities in industrial scenes can be solved. However, methods based on deep learning often need to rely on a large number of surface defect image samples, but in actual industrial production, surface defect image samples are often difficult to obtain, which undoubtedly greatly limits the performance of surface defect recognition tasks.
[0003] To solve this problem of missing data, a variety of data augmentation methods have been proposed. Among them, the classic data augmentation method is to process the existing surface defect images by means of image rotation, translation, flipping, cropping, etc. to obtain more defect data. However, this method can often only obtain image data with the same distribution, and new distributions and features cannot be introduced.
[0004] With the development of generative adversarial network (GAN) methods, a variety of GAN-based surface defect image generation methods have begun to emerge. The basic idea is to improve the model's ability to generate surface defect images by modifying the network structure, loss design, and latent space control. However, most of the above methods complete the defect generation task with relatively more training data, and often need to rely on defect-free images. Very few generation methods for limited defect data can only be used for coarse-grained recognition tasks such as defect classification. At the same time, under limited training data, GAN itself is often difficult to train and prone to overfitting, which undoubtedly leads to the diversity and quality of the images it generates. Based on the above problems, it is difficult for the GAN-based surface defect generation method to meet actual industrial needs. The existing defect generation schemes are difficult to effectively enhance and detect small sample defect data. Summary of the invention
[0005] In order to address the deficiencies of the above-mentioned prior art, the present invention provides a small sample surface defect image generation method and system based on feedback reinforcement learning, which solves the problem that the existing methods cannot effectively detect surface defect images in limited data scenarios (i.e., small sample data scenarios). By introducing artificial feedback into the generation of small sample surface defect images, the performance of surface defect image sample generation and the ability of data enhancement are improved, and the generated surface defect image samples are used for training to improve the accuracy of small sample surface defect detection.
[0006] In a first aspect, the present invention provides a method for generating small sample surface defect images based on feedback reinforcement learning.
[0007] A small sample surface defect image generation method based on feedback reinforcement learning, comprising:
[0008] Obtain a small number of surface defect image samples, annotate each surface defect image sample with a semantic label through manual annotation, and construct a training set;
[0009] Construct a conditional diffusion model, use the training set to train the conditional diffusion model, and complete the preliminary fine-tuning of the conditional diffusion model;
[0010] Expand the semantic labels and use the fine-tuned conditional diffusion model to generate multiple initial surface defect images under each semantic label;
[0011] Manually score multiple initial defect images under each semantic label, and construct an evaluation dataset with multiple initial defect images generated corresponding to each semantic label and their scores;
[0012] Build a reward model and use the evaluation dataset to train the reward model;
[0013] Expand the semantic tags, build a semantic tag library, and generate the optimal surface defect image corresponding to each semantic tag in the semantic tag library by quadratically optimizing the conditional diffusion model based on the reward model.
[0014] A further technical solution, using a conditional diffusion model after secondary optimization based on a reward model, generates the optimal surface defect image corresponding to each semantic label, which means:
[0015] In the process of using the conditional diffusion model to generate the surface defect image corresponding to each semantic label, the trained reward model is used to evaluate the generated surface defect image, so that the conditional diffusion model generates the surface defect image that maximizes the reward, thereby completing the secondary optimization conditional diffusion model.
[0016] A further technical solution is to construct a conditional diffusion model as a basic generation model, and use the training set to train the conditional diffusion model. During the training process, the conditional diffusion model learns and establishes a mapping between image defects and Gaussian distribution by means of forward diffusion with noise and reverse diffusion with denoising, and introduces semantic labels in the denoising reverse diffusion process to complete the training of the conditional diffusion model and obtain the conditional diffusion model after preliminary fine-tuning.
[0017] A further technical solution is to generate the optimal surface defect image corresponding to each semantic label based on the normal surface image by using the conditional diffusion model after secondary optimization based on the reward model, and to construct a training data set with a large number of semantic labels and their corresponding optimal surface defect images;
[0018] Construct a surface defect recognition and classification network model, and use the training data set to train the surface defect recognition and classification network model;
[0019] The surface image of the product to be inspected is input into the trained surface defect recognition and classification network model, and the surface defect recognition result and defect classification result are output.
[0020] According to a further technical solution, the backbone network of the reward model adopts the VGG16 network, the semantic labels in the evaluation data set and their corresponding initial defect images are input into the reward model, and the reward model outputs the score of the initial defect image, so as to train the reward model using the evaluation data set.
[0021] In a further technical solution, the loss function of the reward model is:
[0022]
[0023] Among them, r w (y,x0) is the output of the reward model, and They represent the initial surface defect images generated under the same semantic label y, D represents the evaluation dataset, σ is the reward strategy, w represents the model weight, and K represents K initial samples.
[0024] In a second aspect, the present invention provides a small sample surface defect image generation system based on feedback reinforcement learning.
[0025] A small sample surface defect image generation system based on feedback reinforcement learning, comprising:
[0026] The training set construction module is used to obtain a small number of surface defect image samples, annotate each surface defect image sample with a semantic label through manual annotation, and construct a training set;
[0027] The conditional diffusion model preliminary fine-tuning module is used to build the conditional diffusion model, train the conditional diffusion model using the training set, and complete the preliminary fine-tuning of the conditional diffusion model;
[0028] The initial surface defect image generation module is used to expand the semantic labels and generate multiple initial surface defect images under each semantic label using the fine-tuned conditional diffusion model;
[0029] An evaluation data set construction module is used to manually score multiple initial defect images under each semantic label, and construct an evaluation data set with multiple initial defect images generated corresponding to each semantic label and their scores;
[0030] The reward model training module is used to build a reward model and train the reward model using the evaluation data set;
[0031] The surface defect image generation module is used to expand the semantic labels, build a semantic label library, and generate the optimal surface defect image corresponding to each semantic label in the semantic label library based on the reward model secondary optimization conditional diffusion model.
[0032] In a third aspect, the present invention further provides an electronic device comprising a memory and a processor, and computer instructions stored in the memory and executed on the processor, wherein when the computer instructions are executed by the processor, the steps of the method described in the first aspect are completed.
[0033] In a fourth aspect, the present invention further provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, complete the steps of the method described in the first aspect.
[0034] One or more of the above technical solutions have the following beneficial effects:
[0035] 1. The present invention provides a method and system for generating small sample surface defect images based on feedback reinforcement learning, which solves the problem that existing methods cannot effectively detect surface defect images in limited data scenarios (i.e., small sample data scenarios). By introducing artificial feedback into the generation of small sample surface defect images, the performance of surface defect image sample generation and the ability of data enhancement are improved, and the generated surface defect image samples are used for training to improve the accuracy of small sample surface defect detection.
[0036] 2. The present invention introduces artificial feedback into the optimization of the conditional diffusion model by means of reinforcement learning, and realizes effective enhancement of the performance of surface defect generation for small samples, thereby improving the ability of surface defect detection. In the present invention, a conditional diffusion model is designed to use semantic labels as conditions to guide the model to generate controllable defect images. At the same time, on the basis of the conditional diffusion model, an artificial feedback reinforcement learning mechanism for surface defect generation is designed to further improve the generation effect of the generation model under restricted data, that is, a small number of defect samples are used to fine-tune the conditional diffusion model, and based on artificial feedback, a reward model is designed to evaluate the generation quality of the diffusion model. Based on the reward model, the diffusion model is further optimized through the PPO strategy, and finally the generated defect data is used to enhance the existing data set to perform surface defect recognition tasks, so as to further improve the performance of surface defect detection.
[0037] 3. In the present invention, artificial prior knowledge of defect categories and locations is introduced into the defect sample generation model through reinforcement learning to further improve the ability of surface defect detection. This method can achieve high-quality defect generation with a small number of samples. In the actual working environment, this method can provide an effective solution for the defect recognition task of limited samples. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.
[0039] Figure 1 This is an overall flow chart of the method for generating a small sample surface defect image based on feedback reinforcement learning according to an embodiment of the present invention;
[0040] Figure 2 A flowchart of the overall method of the embodiment of the present invention;
[0041] Figure 3 A schematic diagram of the basic structure of a conditional diffusion model in the method according to an embodiment of the present invention;
[0042] Figure 4 A schematic diagram of generating an initial surface defect image using a conditional diffusion model guided by pixel-level semantic labels in the method according to an embodiment of the present invention;
[0043] Figure 5 It is a schematic diagram of using manual evaluation software to perform evaluation in the method described in the embodiment of the present invention. DETAILED DESCRIPTION
[0044] It should be noted that the following detailed descriptions are exemplary only, are intended to describe specific embodiments, are intended to provide further explanation of the present invention, and are not intended to limit exemplary embodiments according to the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those of ordinary skill in the art to which the present invention belongs. In addition, it should also be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.
[0045] Embodiment 1
[0046] In view of the problem of low precision of small sample defect detection, this embodiment provides a method for generating small sample surface defect images based on feedback reinforcement learning, and uses a large number of generated surface defect images to train the defect detection network model, so as to improve the precision of small sample defect detection. Considering that diffusion training is simpler and more stable than GAN, and has better performance in methods such as generation quality and diversity, based on this characteristic, this embodiment designs a conditional diffusion model, using semantic labels as conditions to guide the model to generate controllable surface defect images; in addition, in view of the amazing performance of chatGPT in NLP with the help of artificial feedback reinforcement learning, this embodiment also designs an artificial feedback reinforcement learning mechanism for surface defect generation on the basis of the conditional diffusion model, further improving the generation effect of the generation model under restricted data, that is, a small number of surface defect image samples are used to fine-tune the pre-trained diffusion model, and then based on artificial feedback, a reward model is designed to evaluate the generation of the diffusion model, and then based on the reward model, the diffusion model is further optimized through the PPO strategy, and finally the generated defect data (i.e., surface defect images) are used to enhance the existing data set, so as to perform surface defect recognition tasks and further improve the performance of surface defect detection.
[0047] This embodiment provides a surface defect image generation method based on feedback reinforcement learning, such as Figure 1 As shown, the following steps are included:
[0048] Obtain a small number of surface defect image samples, annotate each surface defect image sample with a semantic label through manual annotation, and construct a training set;
[0049] Construct a conditional diffusion model, use the training set to train the conditional diffusion model, and complete the preliminary fine-tuning of the conditional diffusion model;
[0050] Expand the semantic labels and use the fine-tuned conditional diffusion model to generate multiple initial surface defect images under each semantic label;
[0051] Manually score multiple initial defect images under each semantic label, and construct an evaluation dataset with multiple initial defect images generated corresponding to each semantic label and their scores;
[0052] Build a reward model and use the evaluation dataset to train the reward model;
[0053] Expand the semantic tags, build a semantic tag library, and generate the optimal surface defect image corresponding to each semantic tag in the semantic tag library by quadratically optimizing the conditional diffusion model based on the reward model.
[0054] In the method proposed in this embodiment, Figure 2As shown, firstly, a small number of defect samples are manually labeled, and the conditional diffusion model is fine-tuned using these limited labeled samples, and the initial defects are generated using the fine-tuned conditional diffusion model; then these initial defects are manually scored to establish an evaluation data set, and the reward model is optimized using the evaluation data set; using the optimized reward model, the conditional diffusion model is optimized twice to further improve its accuracy; finally, the diffusion model after the second optimization is used to generate defect samples, and the generated defect samples are used to assist the discriminant model to improve its defect detection capability. The following content introduces the surface defect image generation method based on feedback reinforcement learning proposed in this embodiment in more detail.
[0055] Step S1, obtaining a small number of surface defect image samples, annotating each surface defect image sample with a semantic label through manual annotation, and constructing a training set; constructing a conditional diffusion model, using the training set to train the conditional diffusion model, and completing the preliminary fine-tuning of the conditional diffusion model.
[0056] In step S1, the conditional diffusion model is mainly used as the basic generation model for training and subsequent optimization fine-tuning. During the training process, the conditional diffusion model learns and establishes the mapping between image defects and Gaussian distribution by means of noise forward diffusion and denoising reverse diffusion, and introduces semantic labels in the denoising reverse diffusion process to complete the training of the conditional diffusion model and obtain the conditional diffusion model after preliminary fine-tuning. Based on the conditional diffusion model after preliminary fine-tuning, the task of generating the initial surface defect image is achieved. The specific steps of the training process of the conditional diffusion model are as follows:
[0057] (1) Establish the noise-added forward diffusion process. Given a clear image x0, the forward process q(x T |x0) can be expressed as a Markov chain with noise added in each step t, which can be specifically expressed as:
[0058]
[0059]
[0060] in, is the variance sequence of the added noise, and I represents the identity matrix. Using the forward process, when the noise sample When is given, the forward diffusion sample under the step length t state can be obtained by the following relationship:
[0061]
[0062] Among them, α t =1-β t ,
[0063] (2) Construct the denoising reverse diffusion process. The main purpose of this process is to obtain the reverse operation q(x t-1 |x t ), this operation can be used to convert the noise image x t The clear image x0 is restored through iteration. t-1 |x t ) is often difficult to obtain directly, so the neural network p θ (x t-1 |x t ) to estimate the operation. In order to make the reverse process controllable, the manually annotated semantic label y is introduced into p θ (x t-1 |x t ) to achieve pixel-level control over its generation process. The basic structure of this process is as follows Figure 3 shown.
[0064] Based on the structure constructed above, p θ (x t-1 |x t ) can be expressed in its basic form as:
[0065]
[0066] in,
[0067]
[0068] In the above formula, ∑ θ (x t ,t) represents the distribution variance.
[0069] In the above representation, ∈ θ (y,x t ,t) is the estimated noise, which can be obtained by training using the following loss function:
[0070] min θ L(θ)=min θ E t,∈ [||∈-∈ θ (y,x t ,t)|| 2 ]
[0071] After training, through iterative denoising, x0 can be obtained through the following operation cycle:
[0072]
[0073] The above uses pixel-level annotation labels as conditions to guide the conditional diffusion model to generate the initial surface defect image, where the semantic label is the pixel-level category label, such as Figure 4 As shown in (a), the image generated based on the pixel-level label is as follows Figure 4 As shown in (b).
[0074] Step S2, expand the semantic labels, and use the fine-tuned conditional diffusion model to generate multiple initial surface defect images under each semantic label; manually score the multiple initial defect images under each semantic label, and construct an evaluation data set with the multiple initial defect images generated corresponding to each semantic label and their scores; construct a reward model, and use the evaluation data set to train the reward model.
[0075] Through the above step S1, the basic operation steps of the conditional diffusion model are clarified. In step S2, the main focus is on how to use artificial feedback reinforcement learning based on the conditional diffusion model to achieve a comprehensive improvement in the generation performance of the diffusion model, so as to further improve the surface defect detection capability. Specifically, the diffusion model is first fine-tuned. In actual industrial scenarios, the amount of defect sample information is often small, so the defect samples are labeled by manual labeling. In order to generate images more finely and accurately, pixel-level labeling is used to label the existing image samples to obtain the label labels. Based on the existing samples and label labels, the conditional diffusion model is trained to achieve preliminary fine-tuning; then the reward model is trained. After fine-tuning the diffusion model, the conditional diffusion model can generate preliminary defect image samples, but the image quality and accuracy of these samples are often difficult to guarantee. In order to solve this problem, this embodiment uses artificial feedback to design a reward model for quantitative evaluation of the results of sample generation.
[0076] Before building a reward model, it is necessary to obtain enough semantic labels. In this embodiment, the semantic sample labels are first expanded by data enhancement, manual drawing, and random generation of existing semantic labels; then manual feedback is given to the generated samples by manual sorting, that is, for a single semantic label, the fine-tuned conditional diffusion model generates K initial samples, which are scored by multiple people and manually sorted according to the generation quality and accuracy of the samples, and the most recognized sorting sequence is used as the manual evaluation sequence of the sample. By performing the above operations on multiple semantic labels, a manual evaluation data set is constructed. In this embodiment, the following is used: Figure 5 The software shown assists the evaluator in completing the evaluation operation.
[0077] The backbone network of the above reward model uses the VGG16 network. The semantic labels in the evaluation data set and their corresponding initial defect images are input into the reward model. The reward model outputs the score of the initial defect image, so as to train the reward model using the evaluation data set. Based on the evaluation data set, the loss function of the reward model is designed as:
[0078]
[0079] Among them, r w (y,x0) is the output of the reward model; and They represent defect samples generated under the same semantic label y; D represents the evaluation dataset; σ is the reward strategy, which can be understood as a function here; w represents the model weight; K represents K initial samples, that is, the K initial samples generated by the above-mentioned fine-tuned conditional diffusion model.
[0080] Step S3: Expand the semantic tags, build a semantic tag library, and generate the optimal surface defect image corresponding to each semantic tag in the semantic tag library based on the reward model secondary optimization conditional diffusion model.
[0081] In this step, the conditional diffusion model is further optimized by reinforcement learning. Specifically, a large number of semantic labels are obtained through the above-mentioned expanded semantic label strategy, and it is expected to obtain corresponding defect samples, so as to establish a basic environment for reinforcement learning. It should be noted that the semantic labels obtained here and the semantic labels expanded in the above step S2 are not the same in reference and expansion purpose. The semantic labels in the above step S2 are expanded for training the reward model, while the semantic labels are expanded and the semantic label library is constructed in this step S3 for secondary optimization of the conditional diffusion model; given the semantic labels and the corresponding generated samples, the corresponding rewards are obtained through the reward model in this environment. Based on this environment, the PPO strategy is used to maximize the reward of the conditional diffusion model. Specifically, in the process of training the diffusion model, clear images It can be estimated by the following relationship:
[0082]
[0083] Using the above clear image, the value r of the reward model is w can be maximized while initially fine-tuning the conditional diffusion model distribution and the current reinforcement learning distribution The KL divergence is used to limit the diffusion model distribution from being over-optimized, and also to estimate the clear image Restricted to follow the existing sample distribution D SFT , based on the above operations, the reward function of reinforcement learning is set to:
[0084]
[0085] Among them, φ is the learned parameter and γ is the confidence of KL divergence.
[0086] That is to say, in the process of using the conditional diffusion model to generate the surface defect image corresponding to each semantic label, the above-mentioned reward model is used to evaluate the generated surface defect image, so that the conditional diffusion model generates the surface defect image that maximizes the reward, thereby completing the secondary optimization conditional diffusion model.
[0087] Step S4: surface defect identification.
[0088] A semantic label generation strategy is used to generate diversified semantic labels. Based on the conditional diffusion model after the above-mentioned secondary optimization, a large number of final defect images are generated. This large number of synthetic defect images are used to enhance the existing defect data. By using the enhanced data set, the accuracy of the recognition network constructed and trained can be significantly enhanced. Moreover, since the above-mentioned generation method uses pixel-level annotation labels as conditions to guide defect generation, this method can be used to enhance classification tasks of different granularities such as defect classification and defect segmentation. Therefore, based on the surface normal image, the optimal surface defect image corresponding to each semantic label is generated using the conditional diffusion model after secondary optimization based on the reward model. A training data set is constructed with a large number of semantic labels and their corresponding optimal surface defect images; a surface defect recognition and classification network model is constructed, and the surface defect recognition and classification network model is trained using the training data set; the surface image of the product to be inspected is input into the trained surface defect recognition and classification network model, and the surface defect recognition result and defect classification result are output.
[0089] Embodiment 2
[0090] This embodiment provides a small sample surface defect image generation system based on feedback reinforcement learning, including:
[0091] The training set construction module is used to obtain a small number of surface defect image samples, annotate each surface defect image sample with a semantic label through manual annotation, and construct a training set;
[0092] The conditional diffusion model preliminary fine-tuning module is used to build the conditional diffusion model, train the conditional diffusion model using the training set, and complete the preliminary fine-tuning of the conditional diffusion model;
[0093] The initial surface defect image generation module is used to expand the semantic labels and generate multiple initial surface defect images under each semantic label using the fine-tuned conditional diffusion model;
[0094] An evaluation data set construction module is used to manually score multiple initial defect images under each semantic label, and construct an evaluation data set with multiple initial defect images generated corresponding to each semantic label and their scores;
[0095] The reward model training module is used to build a reward model and train the reward model using the evaluation data set;
[0096] The surface defect image generation module is used to expand the semantic labels, build a semantic label library, and generate the optimal surface defect image corresponding to each semantic label in the semantic label library based on the reward model secondary optimization conditional diffusion model.
[0097] Embodiment 3
[0098] This embodiment provides an electronic device, including a memory and a processor, and computer instructions stored in the memory and running on the processor. When the computer instructions are run by the processor, the steps in the small sample surface defect image generation method based on feedback reinforcement learning as described above are completed.
[0099] Embodiment 4
[0100] This embodiment also provides a computer-readable storage medium for storing computer instructions. When the computer instructions are executed by a processor, the steps in the method for generating a small sample surface defect image based on feedback reinforcement learning as described above are completed.
[0101] The steps involved in the above embodiments 2 to 4 correspond to the method embodiment 1, and the specific implementation methods can refer to the relevant description part of embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood to include any medium that can store, encode or carry an instruction set for execution by a processor and enable the processor to execute any method in the present invention.
[0102] Those skilled in the art should understand that the modules or steps of the present invention described above can be implemented by a general-purpose computer device, or alternatively, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.
[0103] The above description is only a preferred embodiment of the present invention. Although the specific implementation mode of the present invention is described in conjunction with the accompanying drawings, it is not a limitation of the protection scope of the present invention. Those skilled in the art should understand that on the basis of the technical solution of the present invention, various modifications or variations that can be made by those skilled in the art without creative work are still within the protection scope of the present invention.
Claims
1. A method for generating small sample surface defect images based on feedback reinforcement learning, characterized in that: include: Obtain a small number of surface defect image samples, annotate each surface defect image sample with a semantic label through manual annotation, and construct a training set; Construct a conditional diffusion model, use the training set to train the conditional diffusion model, and complete the preliminary fine-tuning of the conditional diffusion model; Expand the semantic labels and use the fine-tuned conditional diffusion model to generate multiple initial surface defect images under each semantic label; Manually score multiple initial defect images under each semantic label, and construct an evaluation dataset with multiple initial defect images generated corresponding to each semantic label and their scores; Construct a reward model and train the reward model using the evaluation data set; the backbone network of the reward model adopts the VGG16 network, the semantic labels in the evaluation data set and their corresponding initial defect images are input into the reward model, and the reward model outputs the score of the initial defect image, so as to train the reward model using the evaluation data set; Expand the semantic tags, build a semantic tag library, and generate the optimal surface defect image corresponding to each semantic tag in the semantic tag library based on the secondary optimization conditional diffusion model of the reward model; Expand semantic labels, build a semantic label library, obtain a large number of semantic labels, and expect to obtain corresponding defect samples, so as to establish the basic environment of reinforcement learning; given semantic labels and corresponding generated samples, obtain corresponding rewards through the reward model in the basic environment; based on the basic environment, the PPO strategy is used to maximize the reward of the conditional diffusion model; specifically, during the training of the diffusion model, clear images It is estimated by the following relationship: in represents the estimated clear image; x t represents a noisy image; α t =1-β t β t , t=[0,...T] is the variance sequence of the added noise; y represents the semantic label; t represents the step size; Using clear images, the value of the reward model is maximized while initially fine-tuning the conditional diffusion model distribution and the current reinforcement learning distribution The KL divergence is used to limit the diffusion model distribution from being over-optimized and to estimate a clear image Restricted to follow the existing sample distribution D SFT , therefore, the reward function for reinforcement learning is set to: Among them, φ is the learned parameter, γ is the confidence of KL divergence, and D represents the evaluation data set; in the process of using the conditional diffusion model to generate the surface defect image corresponding to each semantic label, the reward model is used to evaluate the generated surface defect image, so that the conditional diffusion model generates the surface defect image that maximizes the reward, thereby completing the secondary optimization conditional diffusion model.
2. The method for generating small sample surface defect images based on feedback reinforcement learning as claimed in claim 1, characterized in that: A conditional diffusion model is constructed as the basic generation model, and the training set is used to train the conditional diffusion model. During the training process, the conditional diffusion model learns and establishes the mapping between image defects and Gaussian distribution by means of forward diffusion with noise and reverse diffusion with denoising. Semantic labels are introduced in the denoising reverse diffusion process to complete the training of the conditional diffusion model and obtain the conditional diffusion model after preliminary fine-tuning.
3. The method for generating small sample surface defect images based on feedback reinforcement learning as claimed in claim 1, characterized in that: Based on the normal surface image, the optimal surface defect image corresponding to each semantic label is generated by using the conditional diffusion model after secondary optimization based on the reward model. A training data set is constructed with a large number of semantic labels and their corresponding optimal surface defect images. Construct a surface defect recognition and classification network model, and use the training data set to train the surface defect recognition and classification network model; The surface image of the product to be inspected is input into the trained surface defect recognition and classification network model, and the surface defect recognition result and defect classification result are output.
4. The method for generating small sample surface defect images based on feedback reinforcement learning as claimed in claim 1, characterized in that: The loss function of the reward model is: in, and They represent the initial surface defect images generated under the same semantic label y, σ is the reward strategy, w represents the model weight, and K represents K initial samples.
5. A small sample surface defect image generation system based on feedback reinforcement learning, characterized in that: include: The training set construction module is used to obtain a small number of surface defect image samples, annotate each surface defect image sample with a semantic label through manual annotation, and construct a training set; The conditional diffusion model preliminary fine-tuning module is used to build the conditional diffusion model, train the conditional diffusion model using the training set, and complete the preliminary fine-tuning of the conditional diffusion model; The initial surface defect image generation module is used to expand the semantic labels and generate multiple initial surface defect images under each semantic label using the fine-tuned conditional diffusion model; An evaluation data set construction module is used to manually score multiple initial defect images under each semantic label, and construct an evaluation data set with multiple initial defect images generated corresponding to each semantic label and their scores; A reward model training module is used to build a reward model and train the reward model using the evaluation data set; the backbone network of the reward model uses a VGG16 network, the semantic labels in the evaluation data set and their corresponding initial defect images are input into the reward model, and the reward model outputs the score of the initial defect image, so as to train the reward model using the evaluation data set; The surface defect image generation module is used to expand the semantic labels, build a semantic label library, and generate the optimal surface defect image corresponding to each semantic label in the semantic label library based on the reward model secondary optimization conditional diffusion model; Expand semantic labels, build a semantic label library, obtain a large number of semantic labels, and expect to obtain corresponding defect samples, so as to establish the basic environment of reinforcement learning; given semantic labels and corresponding generated samples, obtain corresponding rewards through the reward model in the basic environment; based on the basic environment, the PPO strategy is used to maximize the reward of the conditional diffusion model; specifically, during the training of the diffusion model, clear images It is estimated by the following relationship: in represents the estimated clear image; x t represents a noisy image; α t =1-β t β t , t=[0,...T] is the variance sequence of the added noise; y represents the semantic label; t represents the step size; Using clear images, the value of the reward model is maximized while initially fine-tuning the conditional diffusion model distribution and the current reinforcement learning distribution The KL divergence is used to limit the diffusion model distribution from being over-optimized and to estimate a clear image Restricted to follow the existing sample distribution D SFT , therefore, the reward function for reinforcement learning is set to: Among them, φ is the learned parameter, γ is the confidence of KL divergence, and D represents the evaluation data set; in the process of using the conditional diffusion model to generate the surface defect image corresponding to each semantic label, the reward model is used to evaluate the generated surface defect image, so that the conditional diffusion model generates the surface defect image that maximizes the reward, thereby completing the secondary optimization conditional diffusion model.
6. An electronic device, characterized in that: The invention comprises a memory and a processor and computer instructions stored in the memory and executed on the processor. When the computer instructions are executed by the processor, the steps of a method for generating a small sample surface defect image based on feedback reinforcement learning as described in any one of claims 1 to 4 are completed.
7. A computer-readable storage medium, characterized in that: Used to store computer instructions, which, when executed by a processor, complete the steps of a method for generating a small sample surface defect image based on feedback reinforcement learning as described in any one of claims 1-4.
