A method and system for generative data augmentation of earth-rock dam surface defects

Abstract

This invention discloses a generative data augmentation method and system for apparent defects in earth-rock dams, belonging to the fields of water conservancy engineering safety inspection and computer vision technology. The method includes: acquiring original defect images; using a Florence-2 model to infer textual prompts; using a Redux model to extract visual features and uniformly encode them with the textual prompts to form joint guidance conditions; loading LoRA weights pre-trained based on a small number of defect images to fine-tune the FLUX generative model, and using Nunchaku quantization technology to accelerate inference; inputting the joint guidance conditions into the fine-tuned model, and using flow matching to solve ordinary differential equations for denoising sampling to generate high-fidelity defect images. This invention also provides two generative frameworks: local redrawing and visual guidance, and fuses the generated images with the original images to construct a dataset for training downstream detection models. This invention can quickly and controllably generate realistic and diverse defect images with very few samples, significantly improving detection accuracy and reducing data acquisition costs.

Description

Technical Field

[0001] This invention relates to the fields of safety inspection of water conservancy projects and computer vision technology, and in particular to a generative data augmentation method and system for apparent defects in earth-rock dams. Background Technology

[0002] Earth-rock dams, as crucial water conservancy infrastructure, directly impact the safety of life and property and the stability of social production activities downstream. Under the long-term combined effects of complex environmental factors such as earthquakes, rainfall, hydrostatic pressure, temperature changes, and material aging, surface defects such as cracks and calcification are common on the surface of earth-rock dams. This inevitably leads to damage to the dam's structural integrity, posing a serious threat to its safe operation and maintenance. Traditional defect detection in earth-rock dams relies primarily on manual inspections, which are not only inefficient and have limited coverage, but also present significant inconvenience and safety risks in critical areas such as spillways due to their complex structures. With the rapid development of computer vision theory, deep learning-based target detection algorithms are gradually becoming the mainstream method for intelligent detection of surface defects in earth-rock dams. Deploying these algorithms on drones can effectively improve detection efficiency and reduce operational risks.

[0003] However, the detection performance of deep learning models is highly dependent on the size and quality of the training dataset. In practical applications of earth-rock dam defect detection, there are generally severe challenges such as data scarcity and imbalanced sample distribution. Due to the hazardous nature of the on-site collection environment, the complexity of the structure, and the variability of lighting conditions, it is extremely difficult to construct a large-scale dataset covering various defect morphologies and texture features. Data shortage and class imbalance directly lead to insufficient training of detection models, insufficient sensitivity to rare defects, and poor generalization ability when faced with real-world defects of diverse shapes, making it difficult to meet the high-precision detection requirements at the engineering level.

[0004] To alleviate the aforementioned data bottleneck, existing data augmentation methods based on geometric or color space transformations, such as random rotation, cropping, and flipping, are essentially recombinations of existing data and cannot add new defect morphologies or textures. While Generative Adversarial Networks (GANs) can synthesize new samples, they suffer from inherent drawbacks such as training instability, pattern collapse, and image artifacts, and their training performance is poor with small sample sizes. Diffusion models, which have emerged in recent years, surpass GANs in generation quality, still have limitations in specific applications of earth-rock dam defect detection: on the one hand, existing methods mainly rely on text prompts, lacking fine-grained visual semantic control over defect morphology, size, and background texture; on the other hand, the generated images often bear "AI traces," lacking physical realism, and their generation speed is slow, making it difficult to meet the real-time requirements of large-scale data augmentation. Therefore, the industry urgently needs a method for generating high-quality, controllable, and efficient earth-rock dam defect images under small sample conditions. Summary of the Invention

[0005] The purpose of this invention is to provide a generative data augmentation method and system for apparent defects in earth-rock dams, so as to solve the technical problems in the prior art where insufficient training of deep learning detection models, weak generalization ability, and low detection accuracy are caused by data scarcity.

[0006] To achieve the above objectives, the present invention provides a generative data augmentation method for apparent defects in earth-rock dams, comprising the following steps: S1: Acquire raw defect images of the surface of the earth-rock dam; S2: Use a multimodal visual model to reverse-engineer the prompt words from the original defect image and generate text prompt words; S3: Use a reference model to encode the visual features of the original defect image to generate a visual feature vector containing image texture and style information, and encode the visual feature vector and the text prompt words in a unified manner to form a joint guidance condition; S4: Load the weights of a low-rank adaptive fine-tuning model pre-trained based on a small amount of earth-rock dam defect image data, fine-tune the generative model based on Transformer architecture and flow matching optimization, and use quantization technology to accelerate the inference of the fine-tuned generative model. S5: Input the joint guiding conditions into the generator model after fine-tuning and quantization acceleration, solve the ordinary differential equations through the flow matching algorithm to perform denoising sampling, and generate an enhanced image with target defect features.

[0007] Preferably, the generative model is the FLUX generative model, whose backbone architecture is the Diffusion Transformer, and it adopts a modular design that combines dual-stream and single-stream. The first few layers are designed as dual-stream for bidirectional cross-attention fusion of text and image tokens, and the subsequent layers switch to single-stream mode for self-attention and feature transformation of image sequences. The text encoder of the generative model adopts a dual text encoder architecture that includes both CLIP and T5. The pooled text vector output by the CLIP text encoder is fused to the time step embedding of the Diffusion Transformer after being projected by a linear layer. The text sequence features output by the T5 encoder are linearly transformed and used as the text token sequence for cross-attention of the Diffusion Transformer.

[0008] Preferably, the generative model uses two-dimensional rotational position encoding (2D RoPE) instead of the sinusoidal position encoding in the traditional Transformer, and performs rotational encoding based on the position of each row vector in the query matrix and key matrix when calculating the self-attention score.

[0009] Preferably, the flow matching algorithm in step S5 employs the Rectified Flow technique, and its ordinary differential equation form is defined as follows: , , (1); in, Z t Indicates time t The state; v(z,t) It is the velocity field to be learned (usually parameterized by a neural network), representing the velocity field in time. t and location z The velocity field at that location aims to make Z t The trajectory should be as close as possible X 1- X The direction of 0; π 0 represents the initial distribution (such as noise). π 1 represents the target data distribution; The velocity field is learned by minimizing the following loss function: , (2); in, X t yes( X 0, X 1) linear interpolation represents the ideal straight-line path; v ( X t , t ) is the model's prediction speed, and the goal is to make it as close as possible to the straight line direction ( X 0- X 1).

[0010] Preferably, in step S4, low-rank adaptive fine-tuning is achieved by freezing the pre-trained model weights and injecting a trainable rank decomposition matrix into each layer of the Transformer architecture, assuming the weight matrix in the pre-trained model is... LoRA will update the weights. The constraints are in low-rank decomposition form, i.e. ,in , , and rank During forward propagation, for the input... x fine-tuned output h The calculation formula is: (2); During the training process, W 0. Keep frozen, optimize only the matrix. A and B The parameters. Due to the rankr The method requires only a small number of field defect samples to converge quickly, enabling the transfer and feature injection of specific texture styles for earth-rock dams.

[0011] Preferably, the quantization technique in step S4 is the Nunchaku quantization technique, which uses the SVDQuant4-bit quantization method to quantize the model weights and activation values ​​and optimize the memory access of the inference engine.

[0012] Preferably, in step S5, a local redrawing image generation framework is adopted to address the requirements for generating apparent defects in earth-rock dams, specifically including: S511: Select the original image to be generated, and paint a mask on the target redraw area; S512: Select another original defect image as a reference image, apply a mask to the defect location, and use the Florence-2 model to infer the text prompt words; S513: Use the Redux reference model to uniformly encode text prompts, reference images, and their masks; S514: Load a pre-trained low-rank adaptive fine-tuning model to fine-tune the FLUX.1-Fill generation model, use Nunchaku quantization technology to accelerate inference, and use the fine-tuned model to redraw the masked region to generate similar defects while maintaining the integrity of the image background.

[0013] Preferably, in step S5, a visually guided image generation framework is used to address the requirements for generating apparent defects in earth-rock dams, specifically including: S521: Use one or more original defect images as reference images and use the Florence-2 model to infer the text prompt words; S522: Use the Redux reference model to uniformly encode text prompts and reference images; S523: Load a pre-trained low-rank adaptive fine-tuning model to fine-tune the FLUX.1 generative model, use Nunchaku quantization to accelerate inference, and use the fine-tuned model to generate a new defect image with similar features to the reference image.

[0014] Preferably, the multimodal visual model is the Florence-2 model, the reference model is the Redux model, and the text prompts include descriptions of the global scene, object attributes, and spatial relationships.

[0015] This invention also provides a generative data augmentation system for apparent defects in earth-rock dams, which applies the above-described method and includes: The data acquisition module is configured to collect raw defect images of the surface of the earth-rock dam; The prompt word reverse inference module is connected to the data acquisition module and configured to run the Florence-2 model to generate text prompt words based on the original defect image; The feature encoding module is configured to run the Redux model to extract visual features from the original defect image and to uniformly encode text prompts and visual features. The model fine-tuning and quantization module is configured to store and load low-rank adaptive fine-tuning model weights pre-trained based on a small amount of earth-rock dam defect image data, fine-tune the FLUX generative model based on Transformer architecture and flow matching optimization, and accelerate model inference using Nunchaku quantization technology. The image generation module connects the prompt word inference module, the feature encoding module, and the model fine-tuning and quantization module. It is configured to run the fine-tuned and quantized FLUX generation model and generate appearance defect enhancement images with similar features based on the joint guidance conditions formed after unified encoding. The system combines the generated enhanced images with the original images to construct a fusion dataset, which is used to train the downstream object detection model.

[0016] Therefore, the generative data augmentation method and system for apparent defects in earth-rock dams using the above-described structure of the present invention have the following beneficial effects: (1) The present invention adopts the FLUX generation model based on Transformer architecture and flow matching optimization. Compared with the traditional GAN ​​model, the generated defect images such as cracks and calcification are closer to the real scene in terms of edge clarity, illumination consistency and rock particle texture, effectively avoiding the artifacts and mode collapse problems common in GAN. Compared with the existing diffusion model, the number of sampling steps required for generation is significantly reduced by the flow matching algorithm (formula (1)-(2)). Combined with Nunchaku quantization technology, the generation time of a single image is greatly shortened, solving the problems of slow sampling speed and high computing power threshold of the traditional diffusion model, and is feasible for deployment on consumer-grade hardware.

[0017] (2) This invention enables fine-grained and controllable generation of defect images of earth-rock dams. By integrating Florence-2 cue word back-inference, Redux visual feature encoding, and LoRA fine-tuning technology (Formula (3)), it can not only understand the semantics of the text, but also deeply analyze and replicate the visual style and spatial structure in the reference image. Users can flexibly choose between local redrawing or visual guidance as two generation frameworks using a small number of on-site reference images, accurately guiding the model to generate defect images that meet expectations, greatly improving the pertinence and effectiveness of data augmentation.

[0018] (3) This invention significantly reduces the reliance on a large number of on-site defect images for earth-rock dam defect detection. Thanks to the combination of LoRA fine-tuning technology and the pre-trained master model, only a very small number of original defect samples are needed to complete model adaptation and generate a diverse dataset sufficient to support the training of a high-precision detection model. Experiments show that training the detection model using the fusion dataset constructed in this invention can significantly improve the mean accuracy (mAP). With only one-tenth of the original samples, the detection effect can surpass the traditional training effect using the full amount of original data, significantly reducing the data collection cost and risk at the engineering site.

[0019] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating a generative data augmentation method for apparent defects in earth-rock dams according to the present invention. Figure 2 This is a schematic diagram of the local redrawing image generation framework of a generative data augmentation method and system for apparent defects in earth-rock dams according to the present invention. Figure 3 This is a schematic diagram of the visually guided image generation framework of a generative data augmentation method and system for apparent defects in earth-rock dams according to the present invention. Figure 4 This is an example of a local redrawing image generation method and system for generative data augmentation of apparent defects in earth-rock dams according to the present invention. Figure 5 This is an example of visually guided image generation for a generative data augmentation method and system for apparent defects in earth-rock dams according to the present invention. Figure 6 This is an overall flowchart of a generative data augmentation system for apparent defects in earth-rock dams according to the present invention. Detailed Implementation

[0021] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0022] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0023] Example like Figure 1 As shown in the figure, this embodiment provides a generative data augmentation method and system for apparent defects in earth-rock dams. The specific implementation steps of the method are as follows: Step S1: Collect raw defect image data: Use a drone (UAV) equipped with a high-resolution visual sensor to take multi-angle pictures of the surface of the earth-rock dam to obtain raw image data containing typical defects such as cracks and calcification.

[0024] Step S2, Prompt Word Reverse Derivation: Load the Florence-2 multimodal vision model. For each type of defect, select one of the original defect images and input it into the Florence-2 model to generate detailed text prompt words. For example, for a crack image, the generated description is: "A close-up photograph of a weathered, textured dam surface. The concrete surface, predominantly gray, shows signs of erosion with two deep, dark cracks running horizontally across the image. The cracks are filled with a mixture of water and dirt, creating a rough, uneven texture. The background is a gray hues, indicating an outdoor, possibly damp environment."

[0025] Step S3, Feature Encoding: Load the Redux reference model. Input an original defect image as a reference image. Crop the reference image into a square and scale it to 384×384 pixels, then divide it into 27×27 patches. Project these image patches into the CLIP latent space for analysis, generating a visual feature vector consisting of 729 tokens. This vector encodes the image's texture, lighting, and style information, and will subsequently be used as a "visual cue" to be uniformly encoded with text cue words, forming a joint guiding condition for image generation.

[0026] Step S4, LoRA Fine-tuning and Nunchaku Quantization Acceleration: To adapt the general FLUX model to the specific style characteristics of the target earth-rock dam, LoRA fine-tuning technology is used to fine-tune the FLUX master model. The main intervention training weights of the FLUX model are maintained. W Freeze the layer to 0 and inject a trainable low-rank matrix into the Transformer layer. A and B ( By optimizing the matrix A andB This process injects the stylistic features of earth-rock dams into the model. For different types of defects, 10 images from each of the original defect images obtained in step S1 are selected for LoRA fine-tuning model training. The training parameters are set as shown in Table 1. After training, the trained LoRA model is loaded to fine-tune the FLUX.1 or FLUX.1-Fill model. Subsequently, the model weights and activation values ​​are quantized using the SVDQuant 4-bit quantization method with Nunchaku quantization technology, and the memory access of the inference engine is optimized to accelerate inference.

[0027] Table 1. Main parameters for training the LoRA fine-tuning model.

[0028] Step S5: Execute controllable defect image generation: Based on actual engineering needs, this embodiment provides two specific image generation frameworks, as shown in the schematic diagram. Figure 2 , Figure 3 As shown.

[0029] Framework 1: Local Redrawing Image Generation Framework (1) Select an original defect image and apply a mask to the area that needs to be redrawn. (2) Select another original defect image as a reference image, and apply a mask to the defect location for reference; (3) Input the text prompts, reference images and masks derived from Florence-2 into the Redux reference model for unified encoding; (4) Load the pre-trained LoRA model to fine-tune the FLUX.1-Fill model, use Nunchaku quantization technology to accelerate the inference of the model, and use the model to redraw the mask area.

[0030] Example of local redrawing image generation Figure 4 As shown. From Figure 4 As can be seen, this method generates new defects with different shapes, directions, and widths but highly consistent defect features and texture details in the mask area while perfectly preserving the original image background (such as surface texture and lighting environment), effectively expanding the diversity of defects under the same background.

[0031] Framework Two: Visually Guided Image Generation Framework (1) Select one or more representative original defect images as reference images; (2) Input the text prompts and reference images derived from Florence-2 into the Redux reference model for unified encoding; (3) Load the pre-trained LoRA model to fine-tune the FLUX.1 model, use Nunchaku quantization technology to accelerate inference, and use the model to generate a new defect image with similar features to the reference image.

[0032] Examples of visually guided image generation Figure 5 As shown. From Figure 5 It can be seen that the model successfully learned and transferred the style and defect detail features of the reference image, generating a new image that is highly similar to the reference image in terms of global structure, defect morphology and background texture, and maintaining good diversity among the generated images.

[0033] Step S6: Constructing the fused dataset and training the detection model: (1) Construction of fused dataset: According to Table 2, the defect images generated in step S5 are combined with the original defect images collected in step S1 to construct a fused dataset. Fusion strategies with different ratios of original images to generated images were explored.

[0034] Table 2 Construction of the fused dataset

[0035] (2) Model Training: The fused datasets were divided into training and validation sets in an 8:2 ratio. The YOLOv5 object detection model was then trained for 200 epochs. The training parameters were set as follows: input image resolution was uniformly adjusted to 512×512, learning rate was 0.001, momentum was 0.937, batch size was 32, the Adam optimizer was selected, and a lightweight YOLOv5 model was chosen as the pre-trained model.

[0036] Experimental Results: The Fréchet Inception Distance (FID) metric was used to evaluate the quality of the generated defect images. A lower FID value indicates better overall diversity, clarity, and realism of the generated image. The evaluation results are summarized in Table 3. It can be seen that both the local redrawing and visual guidance frameworks exhibit stable generation performance for various types of defects, and the generated images have similar distribution characteristics to the original images.

[0037] Table 3 Image generation quality assessment results

[0038] The detection accuracy of the YOLOv5 model trained on the fused dataset was evaluated using a unified test set, and the results are shown in Table 4. It can be seen that the defect detection performance is significantly improved after enhancement of the images generated using the method in this embodiment. Among them, Dataset-V (original:generated = 1:20) showed the best performance, with an mAP of 74.64%, which is 6.72 percentage points higher than Dataset-I (mAP 67.92%), which uses only 300 original images. Particularly noteworthy is that Dataset-VI achieved an mAP of 68.19% using only 10% of the original data (30 images), surpassing Dataset-I which uses 100% of the original data (300 images). This fully demonstrates that the data enhancement effect of the method of this invention is extremely significant with small samples, and can greatly reduce the dependence on on-site defect images.

[0039] Table 4 Detection Accuracy Evaluation Results

[0040] like Figure 6 As shown, the present invention also provides a generative data augmentation system for apparent defects in earth-rock dams, which is used to implement the steps in the above method embodiments. The system specifically includes the following modules: The data acquisition module, configured as a high-resolution visual sensor mounted on a drone, is used to acquire raw defect images of the earth-rock dam surface. This module sends the acquired raw image data to the prompt word reverse deduction module and the feature encoding module, respectively.

[0041] The prompt word reverse inference module, whose input is connected to the output of the data acquisition module, is configured to run the Florence-2 multimodal vision model. This module receives the original defect image, automatically generates detailed text prompt words containing descriptions of the global scene, object attributes, and spatial relationships, and sends the generated text prompt words to the feature encoding module.

[0042] The feature encoding module, whose inputs are connected to the outputs of both the data acquisition module and the prompt word inference module, is configured to run the Redux reference model. This module receives the original defect image as a reference image, crops and segments it, projects it onto the CLIP latent space, and extracts visual feature vectors containing image texture, lighting, and style information. Simultaneously, it receives the text prompt words. The feature encoding module uniformly encodes the visual feature vectors and text prompt words to form joint guidance conditions, which are then sent to the image generation module.

[0043] The model fine-tuning and quantization module stores LoRA (Low-Rank Adaptive) fine-tuning model weights pre-trained based on a small amount of image data of earth-rock dam defects. This module is configured to: load the LoRA weights to fine-tune the FLUX generative model (including the FLUX.1 model or the FLUX.1-Fill model) based on the Transformer architecture and stream matching optimization; and utilize Nunchaku quantization technology, employing the SVDQuant 4-bit quantization method to quantize the weights and activation values ​​of the fine-tuned model, while simultaneously optimizing memory access in the inference engine to accelerate model inference. The fine-tuned and quantized generative model is then deployed to the image generation module.

[0044] The image generation module has its inputs connected to the outputs of the feature encoding module and the model fine-tuning and quantization module, respectively. Internally, this module runs a FLUX generation model that has been accelerated through fine-tuning and quantization. Depending on actual engineering requirements, the image generation module can operate in two modes: Local redraw mode: Receives the user-specified original image to be generated and its mask, another reference defect image and its mask, and joint guiding conditions provided by the feature encoding module. Calls the FLUX.1-Fill model to redraw the mask area and outputs a similar defect image while maintaining the integrity of the original image background.

[0045] Visual guidance mode: Receives one or more reference images specified by the user and joint guidance conditions, calls the FLUX.1 model, and generates a new defect image with similar features to the reference image.

[0046] The fusion dataset construction module (optional, or as a post-processing step of the image generation module) combines the enhanced image output by the image generation module with the original defect image acquired by the data acquisition module at a preset ratio (e.g., 1:1, 1:5, 1:10, etc.) to construct a fusion dataset. This fusion dataset can be used to train downstream target detection models such as YOLOv5, thereby improving the detection accuracy of surface defects in earth-rock dams.

[0047] Figure 6 The data flow and control relationship between the above modules are illustrated: After the data acquisition module acquires the original image, one path is sent to the prompt word backpropagation module to generate text prompt words, and the other path is sent to the feature encoding module to extract visual features; the text prompt words and visual features are uniformly encoded into joint guidance conditions in the feature encoding module; the model fine-tuning and quantization module deploys the optimized generation model to the image generation module; the image generation module receives the joint guidance conditions and outputs enhanced images according to the generation framework selected by the user (local redrawing or visual guidance); finally, the enhanced images and the original images jointly construct a fusion dataset for training the downstream detection model.

[0048] The working principle of this invention is as follows: First, the Florence-2 multimodal model is used to extract rich semantic descriptions from a small number of real defect samples, while the Redux model is used to extract its ineffable visual style features, forming a joint control condition of "text + image". Then, through LoRA fine-tuning technology (see formula (3)), the weights of the general FLUX generation model are shifted to the specific earth-rock dam defect data distribution, enabling it to learn to generate domain-specific features. Next, the Nunchaku quantization technology is used to compress and accelerate the model, solving the bottleneck of slow inference in the diffusion model. Finally, through the more efficient generation path planning method of flow matching (Rectified Flow, see formulas (1)-(2)), the model can quickly and stably generate high-fidelity and diverse defect images from noise. Users can choose the local redrawing framework to generate defects in specific areas or choose the visual guidance framework to generate entirely new defect images as needed, thereby systematically expanding the training dataset and ultimately improving the performance of downstream detection models.

[0049] Therefore, the present invention employs the above-mentioned generative data augmentation method and system for apparent defects of earth-rock dams, which can achieve high-quality, rapid, and controllable generation of apparent defect images of earth-rock dams under conditions of very few samples. This effectively reduces the dependence of actual detection tasks on on-site defect images and significantly enhances the generalization ability and accuracy of defect detection models.

[0050] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A generative data augmentation method for apparent defects in earth-rock dams, characterized in that, Includes the following steps: S1: Acquire raw defect images of the surface of the earth-rock dam; S2: Use a multimodal visual model to reverse-engineer the prompt words from the original defect image and generate text prompt words; S3: Use the reference model to encode the visual features of the original defect image, generate a visual feature vector containing image texture and style information, and encode the visual feature vector and text prompt words in a unified manner to form a joint guidance condition; S4: Load the weights of a low-rank adaptive fine-tuning model pre-trained based on a small amount of earth-rock dam defect image data, fine-tune the generative model based on Transformer architecture and flow matching optimization, and use quantization technology to accelerate the inference of the fine-tuned generative model. S5: Input the joint guiding conditions into the generative model after fine-tuning and quantization acceleration, solve the ordinary differential equations through the flow matching algorithm to perform denoising sampling, and generate an enhanced image with target defect features.

2. The generative data augmentation method for apparent defects in earth-rock dams according to claim 1, characterized in that: The generative model is the FLUX generative model, whose backbone architecture is the Diffusion Transformer. It adopts a modular design that combines dual-stream and single-stream approaches. The first few layers are designed as dual-stream for bidirectional cross-attention fusion of text and image tokens, while the subsequent layers switch to single-stream mode for self-attention and feature transformation of image sequences. The text encoder of the generative model adopts a dual text encoder architecture that includes both CLIP and T5. The pooled text vectors output by the CLIP text encoder are projected through a linear layer and then fused into the time-step embedding of the Diffusion Transformer. The text sequence features output by the T5 encoder are linearly transformed and used as the text token sequence for cross-attention of the Diffusion Transformer.

3. The generative data augmentation method for apparent defects in earth-rock dams according to claim 1, characterized in that: The generative model uses two-dimensional rotational positional encoding instead of the sinusoidal positional encoding in the traditional Transformer. This rotational encoding is used to calculate the self-attention score based on the position of each row vector in the query matrix and the key matrix.

4. The generative data augmentation method for apparent defects in earth-rock dams according to claim 1, characterized in that: The flow matching algorithm in step S5 employs the Rectified Flow technique, and its ordinary differential equation form is defined as follows: ， ， ； in, Z t Indicates time t The state; v(z,t) It is the velocity field to be learned, representing the velocity field in time. t and location z The velocity field at that location aims to make Z t The trajectory should be as close as possible X 1- X The direction of 0; π 0 is the initial distribution. π 1 represents the target data distribution; The velocity field is learned by minimizing the following loss function: ， （2）； in, X t yes( X 0, X 1) linear interpolation represents the ideal straight-line path; v ( X t , t ) is the model's prediction speed, and the goal is to make it as close as possible to the straight line direction ( X 0- X 1).

5. The generative data augmentation method for apparent defects in earth-rock dams according to claim 1, characterized in that: In step S4, low-rank adaptive fine-tuning is achieved by freezing the pre-trained model weights and injecting a trainable rank decomposition matrix into each layer of the Transformer architecture, assuming the weight matrix in the pre-trained model is... LoRA will update the weights. The constraints are in low-rank decomposition form, i.e. ,in , , and rank During forward propagation, for the input x fine-tuned output h The calculation formula is: （3）； During the training process, W 0. Keep frozen, optimize only the matrix. A and B The parameters.

6. The generative data augmentation method for apparent defects in earth-rock dams according to claim 1, characterized in that: In step S4, the quantization technique is the Nunchaku quantization technique, which uses the SVDQuant4-bit quantization method to quantize the model weights and activation values ​​and optimize the memory access of the inference engine.

7. The generative data augmentation method for apparent defects in earth-rock dams according to claim 1, characterized in that: In step S5, to address the requirement of generating apparent defects in earth-rock dams, a local redrawing image generation framework is adopted, specifically including: S511: Select the original image to be generated, and paint a mask on the target redraw area; S512: Select another original defect image as a reference image, apply a mask to the defect location, and use the Florence-2 model to infer the text prompt words; S513: Use the Redux reference model to uniformly encode text prompts, reference images, and their masks; S514: Load a pre-trained low-rank adaptive fine-tuning model to fine-tune the FLUX.1-Fill generation model, use Nunchaku quantization technology to accelerate inference, and use the fine-tuned model to redraw the masked region to generate similar defects while maintaining the integrity of the image background.

8. The generative data augmentation method for apparent defects in earth-rock dams according to claim 1, characterized in that: In step S5, a visually guided image generation framework is adopted to address the requirement of generating apparent defects in earth-rock dams, specifically including: S521: Use one or more original defect images as reference images and use the Florence-2 model to infer the text prompt words; S522: Use the Redux reference model to uniformly encode text prompts and reference images; S523: Load a pre-trained low-rank adaptive fine-tuning model to fine-tune the FLUX.1 generative model, use Nunchaku quantization to accelerate inference, and use the fine-tuned model to generate a new defect image with similar features to the reference image.

9. The generative data augmentation method for apparent defects in earth-rock dams according to claim 1, characterized in that: The multimodal visual model is the Florence-2 model, the reference model is the Redux model, and the text prompts include descriptions of the global scene, object attributes, and spatial relationships.

10. A generative data augmentation system for apparent defects in earth-rock dams, characterized in that, The method described by any one of claims 1 to 9 comprises: The data acquisition module is configured to collect raw defect images of the surface of the earth-rock dam; The prompt word reverse inference module is connected to the data acquisition module and configured to run the Florence-2 model to generate text prompt words based on the original defect image; The feature encoding module is configured to run the Redux model to extract visual features from the original defect image and to uniformly encode text prompts and visual features. The model fine-tuning and quantization module is configured to store and load low-rank adaptive fine-tuning model weights pre-trained based on a small amount of earth-rock dam defect image data, fine-tune the FLUX generative model based on Transformer architecture and flow matching optimization, and accelerate model inference using Nunchaku quantization technology. The image generation module connects the prompt word inference module, the feature encoding module, and the model fine-tuning and quantization module. It is configured to run the fine-tuned and quantized FLUX generation model and generate appearance defect enhancement images with similar features based on the joint guidance conditions formed after unified encoding. The system combines the generated enhanced images with the original images to construct a fusion dataset, which is used to train the downstream object detection model.

Images