Construction method of diffusion data enhancement model based on prototype prompt, and data generation method and system
By using a prototype-based propagation-based diffusion data augmentation model, realistic training data is generated by combining CLIP image and text encoders with a U-Net network. This solves the accuracy and generalization problems of industrial vision inspection models in small sample scenarios, and achieves efficient data expansion and model improvement.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HUAZHONG UNIV OF SCI & TECH
- Filing Date
- 2026-01-23
- Publication Date
- 2026-05-08
AI Technical Summary
Existing industrial vision inspection models lack accuracy and generalization ability in small sample scenarios, and existing data augmentation techniques are unable to generate realistic training data, especially when facing complex industrial environments and new defect morphologies.
A prototype-based propagation-based diffusion data augmentation model is adopted. Visual and textual features are extracted through CLIP image encoder and text encoder, and image generation is performed by combining U-Net network. The model is trained using inverse estimation and perceptual hybrid loss to generate realistic training data.
It improves the accuracy and generalization ability of detection models in small sample scenarios, generates data that is highly consistent with the real data domain, solves the problems of data scarcity and diversity, and reduces training costs and time.
Smart Images
Figure CN121998937A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the technical field of industrial visual quality inspection, and more specifically, relates to a method for constructing a diffusion data enhancement model based on prototype prompts, a data generation method, and a system. Background Technology
[0002] Deep learning-based visual inspection technology has become a core means to improve the automation and intelligence of quality control in the manufacturing industry. Especially in scenarios such as electronic component assembly (e.g., PCBA), precision parts manufacturing, and textile inspection, target detection and defect detection models can efficiently and accurately identify various defects on the product surface, significantly outperforming traditional manual visual inspection.
[0003] However, these data-driven deep learning models are essentially complex function fitting processes, and their performance heavily relies on training with large-scale, high-quality, and diverse labeled datasets. In practical industrial applications, constructing such ideal datasets faces significant challenges, primarily in the following aspects:
[0004] (1) Sample scarcity problem: Industrial production processes are highly optimized, and the yield rate is usually maintained at an extremely high level, resulting in a very low frequency of defective samples, forming a typical "small sample" learning scenario. For example, in some precision packaging processes, the occurrence rate of a certain type of defect may be less than one in a thousand. Collecting a sufficient number of defective samples often requires a long period of time and high costs, which seriously restricts the timely deployment and iterative updates of the detection model.
[0005] (2) Data diversity and domain adaptation issues: The industrial environment is complex and ever-changing. Factors such as lighting conditions, camera parameters, workpiece posture, and background interference can all lead to significant differences in the distribution of image data (i.e., differences in the "domain"). Even on the same production line, data collected from different batches and different machines may have distribution shifts. Although existing data augmentation techniques (such as rotation, scaling, cropping, color dithering, etc.) can easily expand the amount of data, the generated samples are mostly limited to linear transformations of the original data distribution, making it difficult to simulate complex nonlinear changes and new defect patterns in the real world. This results in insufficient generalization ability of the model when facing new environments and new defect morphologies.
[0006] (3) Limitations of generative models: In recent years, deep generative models such as generative adversarial networks and diffusion models have made breakthrough progress in the field of image synthesis. However, the application of these networks in the field of data augmentation has problems such as unstable training, weak semantic control, and insufficient domain fidelity, which may lead to excessive deviation between the feature domain of the generated data and the source data, and have a negative impact on the performance of the detection model.
[0007] Therefore, there is an urgent need in this field for a data augmentation solution that can overcome the aforementioned limitations. An ideal method should possess the following characteristics: stable training with a limited number of real samples; precise control over generated content, particularly the attributes of foreground objects, based on semantic cues (such as text descriptions); and the ability to ensure that the generated images maintain a high degree of consistency with the real data domain at both the pixel and feature levels, thereby effectively improving the accuracy and robustness of downstream detection models in data-scarce scenarios. Summary of the Invention
[0008] In view of the above-mentioned defects or improvement needs of the existing technology, the present invention provides a method for constructing a diffusion data enhancement model based on prototype prompts, a data generation method and system, which aims to solve the problems of insufficient accuracy and poor generalization of existing industrial detection models in small sample scenarios.
[0009] To achieve the above objectives, according to one aspect of the present invention, a method for constructing a diffusion data augmentation model based on prototype prompts is provided, comprising the following steps: (1) The CLIP image encoder is used to extract the image features of the image to be redrawn. The average features of the foreground region are obtained based on the image features and the foreground mask as the prototype embedding. The CLIP text encoder is used to encode the prompt text to obtain the prompt text features. The prototype embedding and the prompt text features are combined to form the prompt features. (2) After occluding the foreground of the image, the original image and the image after occluding the foreground are input into the VAE encoder to obtain latent features. The latent features of the original image are denoised to obtain noisy latent features. The noisy latent features, the foreground mask and the latent features after occluding the foreground are concatenated and input into the U-Net network to obtain prediction noise. (3) The denoising latent features are estimated by back-estimating the predicted noise, and the denoising latent features are input into the VAE decoder to obtain the estimated image; the mean square error loss between the predicted noise and the real noise is calculated, the perceptual mixing loss between the estimated image and the original image is calculated, and the U-Net network parameters are adjusted based on the mean square error loss and the perceptual mixing loss to obtain the diffusion data augmentation model; wherein, the diffusion data augmentation model includes CLIP image encoder, CLIP text encoder, VAE encoder, U-Net network and VAE decoder.
[0010] Furthermore, the prototype embedding is obtained by calculating the region-of-interest average pooling features of the 4th, 8th, and 12th layer features output by the CLIP image encoder, specifically using the following formula:
[0011] In the formula, For the input image, For CLIP image encoder, For the target truth box, is the prototype embedding set; i is the layer number corresponding to the output of the CLIP image encoder.
[0012] Furthermore, the cue features are obtained by concatenating the prototype embedding with a redrawn foreground, cue text features, and trainable labels, and then inputting them into a projection network; the projection network consists of linear layers and self-attention layers, and the corresponding formula for the projection network is:
[0013]
[0014]
[0015]
[0016]
[0017]
[0018] In the formula, , , , The linear projection weights and biases used for the features of the i-th layer are... This is the output of the projected prototype. Features are formed by splicing together prototypes of multiple layers of features. The predicted gating strength coefficient, These are visual cue features transformed by the self-attention mechanism. and For trainable location markers, To indicate text features, Features for the spliced visual text prompts.
[0019] Furthermore, the inverse estimation process reverses the noisy image based on the predicted noise, and the corresponding formula is:
[0020] In the formula, These are scalar coefficients in the diffusion process, representing the values from time step 0 to time step 1. The sum of all noise scaling factors at each time step. This represents the noise tensor predicted by the U-Net network. Indicates VAE decoder, This represents the latent characteristics estimated by the predictive noise. This represents the estimated image.
[0021] Furthermore, the perceptual fusion loss includes pixel-level mean square error loss and feature-level mean square error loss, with the corresponding formulas as follows:
[0022]
[0023] In the formula, Indicates the input image. Represents the estimated image, This represents the foreground mask of the redrawn area. This indicates the CLIP image encoder. This represents the foreground mask for downsampling. This represents the pixel-level mean square error loss. This represents the characteristic-level mean square error loss.
[0024] Furthermore, a low-rank adapter is used to fine-tune the parameters of the U-Net network.
[0025] Furthermore, apart from its own low-rank weights and the projection network, all other parameters of the diffusion data augmentation model are frozen during training.
[0026] The present invention also provides a data generation method, wherein the data generation method uses the diffusion data enhancement model constructed by the prototype-based prompting diffusion data enhancement model construction method described above to generate simulated data.
[0027] The present invention also provides a system for constructing a diffusion data augmentation model based on prototype prompts. The system includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to perform the method for constructing a diffusion data augmentation model based on prototype prompts as described above.
[0028] The present invention also provides a computer-readable storage medium storing machine-executable instructions, which, when invoked and executed by a processor, cause the processor to implement the method for constructing a diffusion data augmentation model based on prototype hints as described above, or the data generation method as described above.
[0029] In summary, compared with the prior art, the method for constructing a diffusion data enhancement model based on prototype hints, the data generation method, and the system provided by this invention have the following advantages: 1. This invention employs a visual prototype cue embedding method, which combines learnable structured templates and CLIP visual prototypes in text cue embedding to guide the model to generate features closer to the original data domain, improve the model's understanding and expression of image and text data, and thus generate realistic training samples, expand training data, and improve the accuracy of detection models in small sample scenarios.
[0030] 2. This invention employs an image inverse estimation method and a perceptual hybrid loss training method. By performing the denoising process in reverse, the estimated image is obtained during training and aligned with the original input image at the pixel level and feature level, respectively. This improves the model's ability to generate image details and ensures the similarity between the generated data domain and the original data domain.
[0031] 3. This invention employs a simple and efficient simulated data generation framework, which can quickly generate a large amount of simulated data for downstream task training in small sample scenarios. The model has high running efficiency, is easy to train, and generates realistic data. It solves the problem of difficulty in collecting a large number of training samples for detection models in real industrial scenarios, thereby improving the accuracy of detection models in small sample scenarios. Attached Figure Description
[0032] Figure 1 This is a flowchart of a method for constructing a diffusion data enhancement model based on prototype prompts, provided in an embodiment of the present invention. Figure 2 This is a schematic diagram illustrating the construction of a prompt feature provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the data generation method provided in an embodiment of the present invention; Figure 4 This is a schematic diagram illustrating the application effect of the algorithm involved in the embodiments of the present invention on real PCBA production line data; Figure 5 This is a schematic diagram illustrating the application effect of the algorithm involved in the embodiments of the present invention on the MVTec dataset. Detailed Implementation
[0033] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.
[0034] This invention provides a method for constructing a diffusion data augmentation model based on prototype prompts. This method is used for data simulation in downstream detection tasks (such as target detection and defect detection), generating realistic training samples, expanding training data, and improving the accuracy of small-sample, small-detection models. This invention can improve target detection accuracy and generalization under small-sample conditions, while reducing the hardware resource consumption and time cost of fine-tuning.
[0035] Please see Figure 1 The construction method mainly includes the following steps: Step 1: Use CLIP image encoder to extract image features of the image to be redrawn, and obtain the average features of the foreground region based on the image features and foreground mask as prototype embedding; use CLIP text encoder to encode the prompt text to obtain prompt text features, and combine the prototype embedding with the prompt text features to form prompt features.
[0036] The prototype embedding is obtained by calculating the average pooling features of the Region of Interest (RoI) in the 4th, 8th, and 12th layers of the CLIP image encoder output, using the following formula:
[0037] In the formula, For the input image, For CLIP image encoder, For the target truth box, is the prototype embedding set; i is the layer number corresponding to the output of the CLIP image encoder.
[0038] The cue features are obtained by concatenating prototype embeddings with redrawn foreground, cue text features, and trainable labels, and then inputting them into a projection network. The projection network consists of linear layers and self-attention layers, and the corresponding formula for the projection network is:
[0039]
[0040]
[0041]
[0042]
[0043]
[0044] In the formula, , , , The linear projection weights and biases used for the features of the i-th layer are... This is the output of the projected prototype. Features are formed by splicing together prototypes of multiple layers of features. The predicted gating strength coefficient, These are visual cue features transformed by the self-attention mechanism. and For trainable location markers, To indicate text features, Features for the spliced visual text prompts.
[0045] Step 2: After occluding the foreground of the image, input the original image and the image after occluding the foreground into the VAE encoder to obtain latent features. Add noise to the latent features of the original image to obtain noisy latent features. Concatenate the noisy latent features, the foreground mask and the latent features after occluding the foreground and input them into the U-Net network to obtain prediction noise.
[0046] Step 3: The denoising latent features are estimated from the predicted noise and input into the VAE decoder to obtain the estimated image; the mean square error loss between the predicted noise and the real noise is calculated, and the perceptual mixing loss between the estimated image and the original image is calculated. The U-Net network parameters are adjusted based on the mean square error loss and the perceptual mixing loss to obtain the diffusion data augmentation model; wherein, the diffusion data augmentation model includes CLIP image encoder, CLIP text encoder, VAE encoder, U-Net network and VAE decoder.
[0047] The inverse estimation process reverses the noisy image by inverting the predicted noise. The specific formula is as follows:
[0048] In the formula, These are scalar coefficients in the diffusion process, representing the values from time step 0 to time step 1. The sum of all noise scaling factors at each time step. This represents the noise tensor predicted by the U-Net network. Indicates VAE decoder, This represents the latent characteristics estimated by the predictive noise. This represents the estimated image.
[0049] The perceptual fusion loss includes pixel-level mean square error loss and feature-level mean square error loss, and the specific formula is as follows:
[0050]
[0051] In the formula, This indicates the input original image. Represents the estimated image, This represents the foreground mask of the redrawn area. This indicates the CLIP image encoder. This represents the foreground mask for downsampling. This represents the pixel-level mean square error loss. This represents the characteristic-level mean square error loss.
[0052] The parameters of the U-Net network are fine-tuned using a low-rank adapter (LoRA); except for its own low-rank weights and the projection network, the parameters of the diffusion data augmentation model are frozen during training.
[0053] The present invention also provides a data generation method, wherein the data generation method uses the diffusion data enhancement model constructed by the prototype-based prompting diffusion data enhancement model construction method described above to generate simulated data.
[0054] The data generation method uses a sliding window of constant size to generate large-size simulated image data, with a small overlap between adjacent windows.
[0055] The present invention will be further described in detail below with reference to specific embodiments.
[0056] This invention describes the application scenario of component inspection in printed circuit board assemblies (PCBAs), but the method described in this invention is also applicable to other industrial visual inspection tasks, such as metal surface scratch detection and textile stain detection.
[0057] PCBA component target detection data enhancement S101: Data Preparation and Preprocessing Raw data collection: Collect a small number of high-definition images of completed PCBAs from the SMT production line. The resolution of each image should be no less than 1024x1024 pixels.
[0058] Data annotation: Experienced quality control personnel perform detailed annotations on the images. Annotation information includes: (1) Bounding box: precisely outlines the area where each defect is located.
[0059] (2) Category label: Each defect is given a corresponding text description, usually represented by the English name or code of the component, such as “Resistance”, “Capacitance” or “R”, “C”.
[0060] (3) Foreground mask: Set the area inside the bounding box as the foreground (pixel value of 1) and the area outside the box as the background (pixel value of 0) to generate a binary mask image. To improve the robustness of the model, the mask boundary can be slightly Gaussian blurred to simulate the transition effect of real defects.
[0061] Data preprocessing: All images and their corresponding masks are scaled to a standard size for model processing, such as 512x512 pixels. Image pixel values are then normalized.
[0062] S102: Model Building and Detailed Training Process Step 1: Integrating Visual Prototypes and Textual Hints like Figure 2 As shown, the core of this step is to effectively integrate the visual information of defects in the image to be redrawn with the semantic information of the text.
[0063] (1) Visual prototype extraction: extracting the input image Image encoder for input CLIP model The feature maps of the intermediate layers of the input image (e.g., layers 4, 8, and 12) are extracted. For each layer's feature map, the labeled defect bounding boxes are used... The RoIAlign operation is applied to accurately extract features from the defect region, and global average pooling is performed to obtain the prototype embedding at this level. The combination of multiple feature layers helps the model capture complete information from local details to global context.
[0064] (2) Text feature encoding: Input the text description of the defect category into CLIP's text encoder to obtain the text feature vector. .
[0065] (3) Feature projection and fusion: The visual prototypes of three levels are embedded into the input projection network. The projection network first uses independent linear projection layers (using the GELU activation function) to transform and concatenate the visual prototype embeddings to obtain projection features. Then based on projection features Weight coefficients are predicted using a gating mechanism (consisting of a linear layer and a sigmoid function). The projected features are then weighted. The weighted features are then passed sequentially through layer normalization, a multi-head self-attention layer, and a feedforward network to finally obtain the enhanced visual cue features. .
[0066] (4) Cue feature concatenation: Concatenate the trainable start markers Visual cue features Trainable end marker and text features The features are then combined to form the final conditional cue feature. This condition suggests a feature. It will serve as a strong semantic guide for the diffusion model generation process.
[0067] Step Two: Conditioned Diffusion Training Process (1) Latent feature extraction and noise addition: First, the original image is processed using a VAE encoder. Encoding as latent features Then, using the foreground mask. The defective areas in the original image are zeroed out (or filled with the mean of the background) to obtain the occluded image, which is then encoded as a latent feature. Next, at the time step of the diffusion process... ,right Add Gaussian noise To obtain the latent features of noise addition .
[0068] (2) Noise prediction: Including potential features of noise Foreground mask and background potential features The channels are concatenated to form the input to the U-Net. Simultaneously, the conditional cue features obtained in step one are... The noise is injected into U-Net through a cross-attention mechanism. The goal of the U-Net network is to predict the noise added to z. .
[0069] (3) Efficient parameter fine-tuning: In order to significantly reduce the number of training parameters and computational cost while ensuring performance, this embodiment of the invention uses the low-rank adapter (LoRA) technique to fine-tune the attention layer in U-Net. During training, only the parameters of the LoRA adapter and the projection network parameters mentioned in step one are updated, while the pre-trained parameters of CLIP and VAE remain frozen.
[0070] Step 3: Loss Function Calculation and Model Optimization The overall loss function of the model consists of three parts: (1) Noise prediction loss : Calculate the noise predicted by U-Net With real noise The mean square error between them.
[0071] (2) Perceptual hybrid loss: including pixel-level loss and feature-level loss: Pixel-level loss : Calculate the image estimation within the foreground mask region With the original image In foreground mask The mean square error within the range. This pixel-level loss ensures pixel-level fidelity in the generation of defects.
[0072] Feature-level loss : Estimate the image With the original image The image is re-entered into the CLIP image encoder to extract its high-level features. Then, the downsampled foreground mask is used. As weights, the root mean square error of the two high-level features in the foreground region is calculated. This feature-level loss ensures the consistency between the generated defects and the real defects in the semantic feature space.
[0073] The total loss function can be expressed as: ,in These are the weighting coefficients for each loss.
[0074] S103: Simulation Data Generation and Application Once the model is trained, it can be used to generate high-quality simulation data in batches.
[0075] Generation process: Input a reference image and a series of masks, including masks for each component and a background mask. Process the image as described in steps S101 and S102 above and input it into the model to obtain an image sequence after redrawing the content of the specified mask area. These images can be used directly for model training or clipped from the masks to form a single image to improve data diversity.
[0076] Large image generation: For scenarios requiring the processing of high-resolution large images (such as entire PCBA boards), a sliding window mechanism is used, such as... Figure 3 As shown, a fixed-size window (e.g., 512x512) is slid across the original large image, and each window region is generated separately. A certain percentage of overlap (e.g., 64 pixels) is retained between adjacent windows. Finally, the generated results of each window are stitched together to form a complete large image. The overlapping areas can be smoothly blended using a weighted average.
[0077] Downstream task training: The generated large amount of simulated data is mixed with a limited amount of real defect data to train downstream object detection models (such as YOLOv7, DETR, etc.). This significantly increases the diversity and quantity of training data, especially improving the model's ability to identify rare components (such as various custom components, plug-ins, etc.).
[0078] Implementation effect verification To verify the effectiveness of this invention, experiments were conducted on real PCBA production line datasets and the publicly available MVTec dataset. Figure 4 This demonstrates the generation effect on real PCBA data, with the generated components highlighted in the boxes. The generated components are similar to the original components in terms of features, while retaining subtle differences. Figure 5 The product defect generation results are demonstrated on the public dataset MVTec, and the generated defects are accurately located and realistically shaped.
[0079] The present invention also provides a system for constructing a diffusion data augmentation model based on prototype prompts. The system includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to perform the method for constructing a diffusion data augmentation model based on prototype prompts as described above.
[0080] The present invention also provides a computer-readable storage medium storing machine-executable instructions, which, when invoked and executed by a processor, cause the processor to implement the method for constructing or generating a diffusion data augmentation model based on prototype hints as described above.
[0081] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A method for constructing a diffusion data augmentation model based on prototype prompts, characterized in that, The steps are as follows: (1) The CLIP image encoder is used to extract the image features of the image to be redrawn. The average features of the foreground region are obtained based on the image features and the foreground mask as the prototype embedding. The CLIP text encoder is used to encode the prompt text to obtain the prompt text features. The prototype embedding and the prompt text features are combined to form the prompt features. (2) After occluding the foreground of the image, input the original image and the image after occluding the foreground into the VAE encoder to obtain latent features, and add noise to the latent features of the original image to obtain noisy latent features; The concatenated latent features with added noise, the foreground mask, and the latent features after occluding the foreground are input into the U-Net network to obtain the predicted noise. (3) The denoising latent features are estimated by back-estimating the predicted noise, and the denoising latent features are input into the VAE decoder to obtain the estimated image; the mean square error loss between the predicted noise and the real noise is calculated, the perceptual mixing loss between the estimated image and the original image is calculated, and the U-Net network parameters are adjusted based on the mean square error loss and the perceptual mixing loss to obtain the diffusion data augmentation model; wherein, the diffusion data augmentation model includes CLIP image encoder, CLIP text encoder, VAE encoder, U-Net network and VAE decoder.
2. The method for constructing a diffusion data augmentation model based on prototype prompts as described in claim 1, characterized in that: The prototype embedding is obtained by calculating the region-of-interest average pooling features of the 4th, 8th, and 12th layers of features output by the CLIP image encoder, using the following formula: In the formula, For the input image, For CLIP image encoder, For the target truth box, is the prototype embedding set; i is the layer number corresponding to the output of the CLIP image encoder.
3. The method for constructing a diffusion data augmentation model based on prototype prompts as described in claim 1, characterized in that: The cue features are obtained by concatenating prototype embeddings with redrawn foreground, cue text features, and trainable labels, and then inputting them into a projection network. The projection network consists of linear layers and self-attention layers, and the corresponding formula for the projection network is: In the formula, , , , The linear projection weights and biases used for the features of the i-th layer are... This is the output of the projected prototype. Features are formed by splicing together prototypes of multiple layers of features. The predicted gating strength coefficient, These are visual cue features transformed by the self-attention mechanism. and For trainable location markers, To indicate text features, Features for the spliced visual text prompts.
4. The method for constructing a diffusion data augmentation model based on prototype prompts as described in claim 1, characterized in that: The inverse estimation process reverses the noisy image by inverting the predicted noise; the corresponding formula is: In the formula, These are scalar coefficients in the diffusion process, representing the values from time step 0 to time step 1. The sum of all noise scaling factors at each time step. This represents the noise tensor predicted by the U-Net network. Indicates VAE decoder, This represents the latent characteristics estimated by the predictive noise. This represents the estimated image.
5. The method for constructing a diffusion data augmentation model based on prototype prompts as described in any one of claims 1-4, characterized in that: The perceptual fusion loss includes pixel-level mean square error loss and feature-level mean square error loss, and the corresponding formula is: In the formula, Indicates the input image. Represents the estimated image, This represents the foreground mask of the redrawn area. This indicates the CLIP image encoder. This represents the foreground mask for downsampling. This represents the pixel-level mean square error loss. This represents the characteristic-level mean square error loss.
6. The method for constructing a diffusion data augmentation model based on prototype prompts as described in any one of claims 1-4, characterized in that: The parameters of the U-Net network are fine-tuned using a low-rank adapter.
7. The method for constructing a diffusion data augmentation model based on prototype prompts as described in claim 3, characterized in that: Except for its own low-rank weights and the projection network, all other parameters of the diffusion data augmentation model are frozen during training.
8. A data generation method, characterized in that: The data generation method uses the diffusion data enhancement model constructed by the method of constructing a diffusion data enhancement model based on prototype prompts as described in any one of claims 1-7 to generate simulated data.
9. A system for constructing a diffusion data augmentation model based on prototype prompts, characterized in that: The system includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it performs the method for constructing a diffusion data augmentation model based on prototype prompts as described in any one of claims 1-7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores machine-executable instructions that, when invoked and executed by a processor, cause the processor to implement the method for constructing a diffusion data augmentation model based on prototype hints as described in any one of claims 1-7 or the data generation method as described in claim 8.