A diffusion model weed image generation method fusing structure geometric guidance and multi-mechanism optimization

Abstract

This invention discloses a diffusion model for generating weed images that integrates structural geometry guidance and multi-mechanism optimization. The method includes: collecting and preprocessing weed images; proposing a diffusion model that integrates structural geometry guidance and multi-mechanism optimization, including introducing a structural geometry guidance method to reduce the risk of structural collapse and maintain stable guidance in the later stages of sampling; introducing a U-Net skip connection scaling method to improve the model's expressive power and generation quality in the sampling scenario; employing a signal-to-noise ratio-aware weighted loss function to correct system prediction bias and prioritize the learning of key visual information; using the optimal model to generate candidate weed images for the target category, and filtering the candidate images by realism score to retain high-quality weed generation samples. This invention can improve the structural consistency, detail realism, and sample diversity of generated weed images, and enhance the generation quality and stability of weed images in complex field scenarios, showing promising application prospects.

Description

Technical Field

[0001] This invention belongs to the field of natural image generation and relates to a method for generating images of weeds. Background Technology

[0002] Weeds compete with crops for resources such as light, water, and nutrients, causing significant yield and economic losses to global agriculture. For a long time, chemical control, centered on herbicide-resistant genetically modified crops, has been the mainstream approach. However, over-reliance on chemical agents has led to the proliferation of resistant weeds, causing environmental pollution and ecological damage. Simultaneously, the development of new herbicides has stalled, diminishing the effectiveness of traditional control methods. This has prompted agriculture to shift towards precision intelligent weed control technologies with real-time sensing and decision-making capabilities. These technologies achieve targeted removal through visual recognition, enabling physical removal while reducing the use of chemical agents, and have already been commercially applied to some crops. However, the similarity in morphology between seedling weeds and crops, the diversity of weed species, and their variable phenotypes, coupled with field disturbances such as changes in light intensity and plant shading, lead to problems such as missed detections, misjudgments, and crop damage in visual recognition, severely hindering its widespread application.

[0003] Deep learning, with its powerful representation capabilities, has become the core method for visual weeding; however, its performance is highly dependent on the scale, quality, and diversity of the training data. Although several publicly available crop and weed datasets have emerged in recent years, the sample size, species, and environmental diversity are still insufficient to support robust generalization of models in complex scenarios. Traditional data collection and annotation methods are costly, and acquiring data across seasons and multiple locations is difficult, further exacerbating the data bottleneck. Therefore, constructing a weed image dataset with both scale and diversity is crucial for improving the model's generalization ability.

[0004] To alleviate data scarcity, researchers employ various data augmentation techniques. While traditional methods such as geometric and color transformations can expand the sample size, the generated data is highly correlated with the original samples, making it difficult to provide truly new visual features. Generative models, as more powerful data augmentation tools, have gradually gained attention. Among them, Generative Adversarial Networks (GANs) have made significant progress in image synthesis and have been used to augment weed and disease images to improve downstream recognition performance. However, inherent problems such as unstable GAN training, pattern collapse, and limited diversity restrict their widespread applicability in agricultural scenarios.

[0005] In recent years, denoising diffusion probabilistic models have achieved breakthroughs in image realism and diversity, demonstrating superior performance compared to mainstream GANs in fields such as medical imaging and video generation. However, directly transferring diffusion models to agricultural weed image generation still faces challenges, such as controlling the accuracy of generated sample categories and structures, preserving leaf edge and texture details, and ensuring consistency between the generated distribution and the real data distribution. To address these issues, developing conditional generation and structure-guided methods for agricultural scenarios, combined with transfer learning and domain adaptation techniques, has become a key research focus for improving the quality of generated data and promoting the practical application of precise visual weeding systems. Summary of the Invention

[0006] This invention addresses the technical problems in existing weed image generation processes, such as easily distorted generated sample structure, insufficient detail representation, accumulated noise prediction bias, and unstable image quality. It proposes a diffusion model-based weed image generation method that integrates structural geometry guidance and multi-mechanism optimization. By introducing a structural geometry guidance mechanism, a skip connection scaling mechanism, and a signal-to-noise ratio (SNR)-aware optimization strategy into the classifier-guided diffusion model, the method improves the structural consistency, detail realism, and generation stability of the generated weed images, thereby enhancing the application value of the generated samples in downstream weed identification tasks. The implementation steps of this method are as follows:

[0007] (1) First, collect the original dataset consisting of weed image datasets, and preprocess the weed dataset to output the preprocessed dataset;

[0008] (2) To address the challenges of weed generation and the lack of research in this task area, a diffusion model GSS-ADM that integrates structural geometry guidance and multi-mechanism optimization was constructed. This includes introducing a structural geometry guidance method (SGG) to correct structural biases and maintain guidance strength in the later stages of sampling; introducing a U-Net skip connection scaling method (Skip-Scaling) to improve global transformation capability under few sampling steps; and using signal-to-noise ratio perceptual weighted loss (SPL) to correct noise prediction biases and guide the model to prioritize learning perceptual important information.

[0009] (3) Input the preprocessed dataset obtained in (1) into the GSS-ADM model for training; the total number of training iterations is 150,000; the batch size is 4; the optimizer is AdamW, the initial learning rate is 1e-4 and cosine annealing strategy is used; after training, multiple evaluation metrics are used on the generated image set to evaluate the generation quality and diversity of the model, and the model weight with the best generation quality is saved as the "best model weight".

[0010] (4) After loading the optimal model weights, the GSS-ADM model is used to perform conditional sampling based on the target category label and the preset number of generated images to generate a set of candidate weed images for the corresponding category. Then, the set of candidate images is evaluated for realism score, and the generated samples are screened for quality based on the preset screening threshold to retain high-quality weed generated image data that meet the realism requirements.

[0011] Furthermore, the preprocessing method described in step 1 specifically involves: before the experiment, assigning corresponding category labels to the weeds in each image; and processing the images input to the network... Perform size normalization and scale the image to 256*256 pixels.

[0012] Furthermore, the diffusion model that integrates structural geometry guidance and multi-mechanism optimization described in step 2 is specifically as follows:

[0013] (2.1) A structural geometry guidance method is proposed to simultaneously correct structural biases and maintain the scale consistency of the classifier guidance amplitude during the sampling process of the diffusion model, thereby improving the global structural rationality and local detail fidelity of the generated image. The core of this method is: applying a structural perturbation to the self-attention layer of the diffusion model U-Net to generate a structural degradation reference; using the difference between this reference and the unperturbed noise prediction as the structural correction signal; thereby eliminating structural biases in the noise prediction space; and calculating the corrected Gaussian mean. Subsequently, the geometric distance from the sample to the scaled manifold is approximated based on the scale of the forward diffusion, and this is used as the scale reference for the later classifier guidance amplitude, thus maintaining a stable correction intensity throughout the sampling process.

[0014] Specifically, in the selected self-attention layer, the original self-attention distribution is replaced with an identity matrix, severing the attention interaction between different image patches. This causes the network output to retain only local appearance information while weakening or losing cross-patch global structural information. The forward propagation after this processing yields a noise prediction under structural degradation, denoted as... The standard noise prediction calculated simultaneously This is used to construct the structure-guided signal. The structure-guided signal is defined as the vector difference between the two:

[0015]

[0016] According to the structural guiding strength coefficient After scaling, the data is added back to the standard prediction to obtain the structure-corrected noise prediction. :

[0017]

[0018] Based on this corrected noise prediction, the Gaussian mean value during the reverse diffusion process is calculated using the reverse calculation formula of the diffusion model. :

[0019]

[0020] in, This represents the proportion of signal retained during the forward diffusion at step t. This represents the variance coefficient of the Gaussian noise injected in this step. This represents the cumulative percentage of the original image signal retained from step 1 to step t of the forward diffusion. This represents the proportion of cumulative noise variance.

[0021] Next, the classifier gradient is reshaped based on the geometric magnitude reference; the standard classifier's gradient for noisy samples is then reshaped. gradient First, L2 normalization is performed to extract orientation information (where y is the true image label of the sample). Then, the geometric magnitude is determined using the typical distance between the noisy sample and its corresponding scaled manifold as a scaling factor, and the gradient norm is reshaped, thus forming the geometric guiding term. .

[0022]

[0023] Where D represents the dimension of the data space. The geometric amplitude in this embodiment is T, where T is the total number of denoising steps; the geometric guidance term is scaled according to the classifier guidance scaling factor. After scaling and incorporating the sampled data into the update, the sampling formula is obtained:

[0024]

[0025] in Let be the noise covariance term at step t. The noise is random noise that follows a standard normal distribution. By incorporating structural correction at the mean estimation and applying geometric amplitude constraints at the gradient magnitude, this method effectively reduces the risk of structural collapse in the early stages of sampling and maintains stable guidance in the later stages of sampling. Thus, it significantly improves the global structural rationality and detail representation of the generated image without sacrificing the fidelity of local details.

[0026] (2.2) The U-Net skip connection scaling method is introduced to actively adjust the information flow of skip paths during the inference stage, thereby enhancing the network's ability to perform semantic transformations through deep nonlinear paths and improving the generation quality of the diffusion model. This method applies element-wise scaling to the skip features from the encoder before concatenation into the decoder, reducing the information weight of the skip paths and prompting the decoder to rely more on deep features passing through the bottom bottleneck. This improves the overall nonlinear mapping and semantic reconstruction capabilities while preserving necessary low-level details. The specific implementation of the U-Net skip connection scaling method is as follows:

[0027] In traditional skip fusion, the i-th layer of the decoder receives upsampled features. Jump features of the encoder Through splicing and fusion:

[0028]

[0029] The U-Net skip connection scaling method proposed in this invention introduces an element-wise scaling factor on the skip features before splicing. (0< ≤1) First, define the jump feature With scaling factor Perform element-wise multiplication and then concatenate:

[0030]

[0031] This operation only takes effect during the inference phase. By reducing the information weight of skip paths, it forces the decoder to rely more on deep semantic features that pass through the bottom bottleneck, thereby improving the overall nonlinear mapping capability.

[0032] Considering that different layers in U-Net have different responsibilities—deeper layers focus on global semantics and structure, while shallower layers focus on local details and textures—directly applying the same scaling factor to all layers is not optimal. To balance global and detail considerations, this invention proposes a hierarchical linear interpolation strategy requiring only two parameters: setting the scaling factor for the lower layers. With top-level scaling factor The remaining layers transition smoothly using linear interpolation at each level.

[0033]

[0034] Where k is the total number of layers with skip connections, and i is the layer index. This strategy allows necessary details to be preserved at lower layers, while actively reducing skip weights in mid-to-deep layers to promote semantically dominant reconstruction, thereby achieving a balance between semantically dominant reconstruction and detail preservation at different scales.

[0035] (2.3) Next, a diffusion model training method based on signal-to-noise ratio sensing loss is introduced. The method is characterized by applying non-constant weights to the noise prediction error according to the sampling time step t in the standard noise prediction training paradigm, constructing training targets with different penalty intensities for different noise scales, thereby prioritizing the reduction of errors in time periods that significantly affect the final reconstruction, and improving the reconstruction performance of the model in the low signal-to-noise ratio stage.

[0036] To illustrate the theoretical basis and derivation process of this weighted loss, let's start with a clean image. The forward diffusion at step n takes the standard form as follows:

[0037]

[0038] in This represents the cumulative signal retention percentage at step n of the forward diffusion. When using the noise prediction paradigm, the model performs better on clean images. The estimate can be written as:

[0039]

[0040] in For a model with parameter θ, under the noise prediction training paradigm, given noisy samples... The noise prediction output at time step t. Substituting the forward expression into the above estimation and simplifying, we can obtain the closed-form expression for the estimation error as:

[0041]

[0042] Introducing the signal-to-noise ratio formula Then the above equation is equivalent to:

[0043]

[0044] Therefore, the noise prediction error can be seen. The amplification factor for the final image reconstruction error is: Based on this conclusion, this invention proposes using the amplification factor as a weighting coefficient for time step t to measure the relative impact of prediction error on the final reconstruction at different time steps. The weighting function is defined as follows:

[0045]

[0046] Based on the above weights, the weighted mean square error loss for noise prediction is defined as:

[0047]

[0048] in This represents the noise sampled during forward diffusion. This represents the model's prediction of noise, with an expected value for time step t and sample... With noise Sampling. This weighting method imposes a higher training penalty on the prediction error of low signal-to-noise ratio time steps, thereby prioritizing the reduction of errors in time periods that significantly affect the final reconstruction during training.

[0049] Beyond this theoretical derivation, this invention also provides an intuitive explanation and practical supplement to the weighting strategy by incorporating experimental observations of how perceptual information changes with noise levels. By analyzing perceptual information at different noise levels during the forward diffusion process, the denoising process is roughly divided into three stages: a "coarse stage" with low signal-to-noise ratio (SNR), where the model learns global features such as overall hue and general outlines; a "content stage" with medium SNR, where the model learns structural and semantic information, representing the most discriminative stage for human perception; and a "cleaning stage" with high SNR, where the model removes subtle noise, contributing little to advanced visual semantics. Based on this observation at the perceptual level, an efficient training allocation strategy should significantly reduce training weights in the "cleaning stage" to avoid wasting limited model capacity on redundant details; simultaneously, it should give greater attention to the "content stage" and retain sufficient weights for the "coarse stage" to ensure the correct construction of the global structure.

[0050] The present invention proposes The impact of noise prediction error on the final reconstruction is indicated by... Scaling is used so that the weights increase as the SNR decreases, thus prioritizing the constraint of errors in low and medium SNR periods during training, and consistent with the importance ranking based on perceptual information in stages: the weights will give higher relative penalties in low SNR (coarse stage) and medium SNR (content stage), while the weights will be smaller in high SNR (cleaning stage), thereby achieving perceptual-driven allocation of training resources.

[0051] Furthermore, the specific method for evaluating the generation quality and diversity of the model described in step 3 is as follows: multiple images are generated using the trained model weights to form a generation dataset. The performance of the generation model is evaluated by verifying the passing of the generation dataset, Frechet Inception distance (FID), Inception score (IS), improved accuracy and recall. The optimal model weight file is saved by comparison to verify the effectiveness and sophistication of the model.

[0052] Furthermore, the specific method for generating weed images of the target category and performing sample screening in step 4 is as follows:

[0053] (4.1) Call the optimal model trained in step 3 to perform conditional sampling on the target category y to construct a candidate weed image set; the sampling process is executed by a reproducible sampling script, which takes the target category label, preset generation quantity, random seed, sampler type, sampling steps and guiding hyperparameters as input parameters and outputs the candidate generated images of the corresponding category.

[0054] (4.2) Calculate the realism score R for each sample in the candidate generated image set, and use R>1 as the screening condition for qualified samples. Retain the generated images that meet the condition as qualified samples; further sort and screen the qualified samples by category to obtain the final high-quality weed generated image data.

[0055] To clarify the definition and calculation process of the realism score R, the following explanation is provided. The realism score is mathematically defined as:

[0056]

[0057] Where φ represents the feature extractor. A set of feature vectors representing a real image. For a single generated image, Indicates the true feature set In, vector The k-th nearest neighbor. (Numerator) Calculated real samples The Euclidean distance between it and its k-th nearest neighbor in the real data manifold. This distance can be considered as a defined... The radius of the surrounding "neighborhood hypersphere" reflects the local density of real data in that region. (Denominator) The generated samples were then calculated. Compared to the real sample The distance between them. The realism score seeks the real samples that maximize this ratio. .

[0058] This invention has the following characteristics:

[0059] 1. This invention proposes a diffusion model for weed image generation that integrates structural geometry guidance and multi-mechanism optimization. This method enhances the model's ability to represent global structure and local texture by co-introducing structural geometry guidance, U-Net skip connection scaling, and signal-to-noise ratio perceptual weighted loss into the diffusion generation process, thereby improving the stability, realism, and diversity of weed image generation.

[0060] 2. The structural geometry guidance method proposed in this invention constructs a structural degradation reference in parallel in the self-attention layer of U-Net and uses the difference between the two as a structural correction signal to perform directional correction on noise prediction. Combined with the guidance of a classifier that reshapes the geometric scale, it suppresses structural collapse in the early stage of sampling and maintains a stable correction intensity in the later stage of sampling, thereby improving the structural rationality and semantic consistency of the generated samples.

[0061] 3. The U-Net skip connection scaling proposed in this invention applies hierarchical scaling to the features of skip connections during the inference stage, forcing more information to be processed through the bottom bottleneck and deep nonlinear paths. This enhances the model's expressive and global structure recovery capabilities in sampling or high nonlinear mapping scenarios, while preserving necessary high-frequency details in the shallow layers.

[0062] 4. The signal-to-noise ratio (SNR) perceptual weighted loss proposed in this invention is trained by weighting the noise prediction error based on the SNR at each step. This loss effectively corrects training bias during training and prioritizes reducing errors in the low-to-medium SNR time period, which has a greater impact on the final reconstruction. This improves the model's learning efficiency of structural and semantic information, thereby enhancing the structural consistency of the generated image and the perceptual performance of downstream tasks. Attached Figure Description

[0063] Figure 1 This is a flowchart illustrating the process of generating weed image data as proposed in this invention.

[0064] Figure 2 This is a schematic diagram of the structural geometry guidance method proposed in this invention.

[0065] Figure 3 This is a schematic diagram of the U-Net skip connection scaling method proposed in this invention.

[0066] Figure 4 This is a comparison chart of the signal-to-noise ratio sensing loss proposed in this invention and the standard loss.

[0067] Figure 5 This is a diagram showing the analysis of perceived information at different noise levels during the forward diffusion process.

[0068] Figure 6 This is a qualitative ablation comparison image of the GSS-ADM proposed in this invention on DeepWeeds (a dataset of nine types of weeds) and CottonWeedID10 (a dataset of ten types of weeds).

[0069] Figure 7 This is a visualization analysis of the GSS-ADM proposed in this invention on DeepWeeds (a dataset of nine types of weeds) and CottonWeedID10 (a dataset of ten types of weeds) compared with mainstream generative models. Detailed Implementation

[0070] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0071] This invention proposes a diffusion model-based weed image generation method that integrates structural geometry guidance and multi-mechanism optimization. Figures 1 to 7 Detailed explanation is as follows:

[0072] like Figure 1 The diagram shows a flowchart of a weed image generation method based on a diffusion model that integrates structural geometry guidance and multi-mechanism optimization, as proposed in this invention. In this process, weed image data is first acquired through internet search and collection, and the original dataset is preprocessed to construct a standardized weed image dataset. Subsequently, this invention constructs a diffusion model that integrates structural geometry guidance and multi-mechanism optimization to improve the structural realism and detail representation of the generated weed images. The improvements include: introducing a structural geometry guidance method to suppress structural collapse during the early stages of sampling and maintain stable structural correction capabilities in the later stages; introducing a signal-to-noise ratio (SNR) perceptual weighted loss to alleviate training bias caused by noise prediction errors at different time steps and guide the model to prioritize learning image information more important to human visual perception; and employing a U-Net skip connection scaling method to optimize the information fusion process between features at different levels, enhancing the model's nonlinear expression capabilities and global structural recovery capabilities. Afterward, the preprocessed weed images are input into the diffusion model at a fixed size of 256×256 for training and validation. After training, multiple generation quality evaluation metrics are used to assess the realism and diversity of the model's generated results, and the weights of the best-performing model are saved. Finally, the optimal model is used to generate weed images of the target category, and the generated samples are quality-screened by realism score to obtain high-quality weed image data.

[0073] like Figure 2 The diagram illustrates the structural geometry guidance method proposed in this invention. This method constructs two parallel pathways—original and degraded—during the forward propagation of the diffusion model. A degraded reference is generated through targeted self-attention intervention, and the difference between the original and degraded predictions is used as a structural correction signal to directionally correct noise predictions. Simultaneously, the amplitude of the classifier guidance is adjusted based on the geometric expectation of the samples relative to the scaled manifold, maintaining a stable correction intensity consistent with the data manifold during sampling. Specifically, this significantly reduces the probability of global structural collapse in the early sampling stage and continuously enhances the depiction of micro-textures and details in the later sampling stage, thereby reducing structural defects such as leaf deformation, missing veins, and fusion of multiple plant outlines, and improving the layout rationality and semantic consistency of the generated image.

[0074] like Figure 3The diagram illustrates the U-Net skip connection scaling method proposed in this invention. This method applies adjustable scaling coefficients to the skip paths from the encoder to the decoder in the U-Net during the inference phase to dynamically adjust the transmission intensity of skip features, thereby changing the weight distribution of information between shallow detail paths and deep semantic paths. In practice, a hierarchical interpolation scaling strategy can be set for each layer, weakening the impact of skips at the bottom bottleneck to strengthen deep nonlinear transformations, while maintaining high skip weights at the top layer to preserve necessary high-frequency details. This method does not require modification of the original network structure; it only adds a lightweight scaling operation at feature fusion, thus improving the model's expressive power and structural recovery ability in scenarios with few sampling steps or high nonlinear mappings without increasing inference complexity.

[0075] like Figure 4 The graph compares the signal-to-noise ratio (SNR) perceptual loss with the standard loss, illustrating their weight allocation strategies and differences. The horizontal axis represents the SNR, and the vertical axis represents the weight values ​​used in the training loss at the corresponding time step. The blue curve represents the SNR perceptual weights, and the orange dashed line represents the standard constant weights. This comparison visually illustrates the design intent of this method: concentrating more training resources on the low and medium SNR stages, while significantly reducing weights in the high SNR "cleaning" stages, thereby avoiding wasting model capacity on minor details and improving overall reconstruction and perception quality.

[0076] Figure 5 This diagram illustrates the perceptual information analysis under different noise levels during the forward diffusion process. A quantitative analysis of perceptual information at different noise levels is presented. The experiment uses LPIPS as the perceptual similarity measure, with the horizontal axis representing the signal-to-noise ratio (SNR). Three stages of behavior can be observed in the diagram: In the low SNR range, noise dominates, image semantic information is almost completely obscured, and the distances between images are close and change steadily; in the medium SNR range, the structural and semantic information of the images begins to emerge, and the differences between different images rapidly increase, indicating that this range is a critical period for the model to learn discriminative visual concepts; in the high SNR range, the noise level is extremely low, and the model mainly performs micro-detail purification, with intra-image distances significantly decreasing while inter-image differences remain large. Based on these observations, the denoising process is divided into three stages: "coarsening—content purification," and accordingly, non-uniform resources are allocated to different time steps during training; the weights for low and medium SNR are increased to prioritize learning structural and semantic information, while the weights for high SNR are decreased to avoid consuming limited model capacity on redundant details.

[0077] like Figure 6This is a qualitative ablation comparison of the proposed GSS-ADM model on the CottonWeedID10 and DeepWeeds image datasets. The figures show different configurations by row and different class samples by column, facilitating observation of the marginal contributions and synergistic effects of each core component. Under the same generation settings, as structural geometry guidance, skip connection scaling, and signal-to-noise ratio perceptual loss are introduced sequentially, non-target textures are gradually suppressed, the consistency between detailed morphology and overall structure becomes more stable, color distribution tends to be more balanced, and visual quality shows a step-like improvement. This figure intuitively confirms the trend of quantitative evaluation, indicating that each module has a complementary relationship in improving fidelity, clarity, and diversity, and the complete framework has better overall performance in complex backgrounds and multi-class scenes.

[0078] like Figure 7 This is a visual comparison of the GSS-ADM model of this invention with several mainstream generation methods. The rows correspond to different generation frameworks, and the columns display representative samples of weeds, facilitating a direct comparison of the differences in appearance consistency and detail rendering between the methods. Under uniform sampling and resolution settings, some of the compared methods are prone to edge breakage, texture repetition, or background color cast. In contrast, this invention excels in the coherence of detailed structures, overall morphological stability, and background harmony, effectively suppressing non-target textures and artifacts.

[0079] To further verify the independent contributions and synergistic effects of each innovative module on model performance, a series of ablation experiments were designed and implemented. The three core improvements were systematically compared by integrating them individually and in combination into the baseline model using the controlled variable method. The experimental data are shown in Table 1. Experiments were conducted on the CottonWeedID10 and DeepWeeds weed image datasets. From the perspectives of generation quality and discriminative ability, multiple metrics, including FID, IS, and improved precision and recall, were used to comprehensively evaluate each approach. Individual comparison results show that structural geometry guidance contributes the most to improving the rationality of the global structure, reducing FID to 18.92, a decrease of 12.6%. Skip connection scaling helps enhance global consistency under few sampling steps and has a positive impact on fine texture recovery. Signal-to-noise ratio perceptual weighted loss effectively corrects training bias and improves perceptual quality. After combining the three improvements, the overall performance of the complete model reaches the best.

[0080] Table 1 Ablation Experiment Tables for CottonWeedID10 and DeepWeeds Weed Datasets

[0081] IS FID Accuracy Recall rate IS FID Accuracy Recall rate ADM-G (baseline) 7.5305 21.6584 0.7825 0.5284 5.6386 25.3177 0.7710 0.4350 ADM-G + SGG 7.5659 18.9241 0.7960 0.5427 6.1252 17.9196 0.7742 0.4770 ADM-G + Skip-Scaling 7.5522 21.2912 0.7855 0.5168 5.7864 24.5045 0.7697 0.4385 ADM-G + SPL 7.5303 21.4079 0.7907 0.5127 5.7408 24.3632 0.7780 0.4295 ADM-G + SGG + Skip-Scaling 7.4460 18.8298 0.8045 0.5301 6.2187 17.6764 0.7747 0.4831 ADM-G +SGG+ Skip-Scaling + SPL 7.4108 18.7152 0.8132 0.5352 6.4346 16.0973 0.7720 0.5063

[0082] To verify the superiority of this invention over other advanced classification algorithms, comparative experiments were added, and the results are shown in Table 2. The comparative models cover different types of diffusion models and generative adversarial networks (GANs), including ADM, ADM-DDIM, StyleGAN2-ada, StyleGAN3, and LDM.

[0083] Table 2 details the generation quality evaluation results of all models on the CottonWeedID10 and DeepWeeds datasets. Our model achieved an FID score of 18.72 on the CottonWeedID10 dataset, the best performance among all comparison methods. This result surpasses unguided ADM, LDM, and two state-of-the-art GAN architectures, StyleGAN2-ada and StyleGAN3. Notably, ADM-DDIM exhibits a significant decrease in generation quality due to its fast sampling strategy. Furthermore, our model also achieves the best overall performance in Precision and Recall metrics, demonstrating its ability to effectively cover the diversity of real-world data while generating high-fidelity samples.

[0084] Comparison results on the more complex DeepWeeds dataset further validate the robustness of our method. On this dataset, our model achieves state-of-the-art performance with an FID score of 16.09. In comparison, our model not only leads in FID but also significantly outperforms StyleGAN2-ada in key fidelity metrics such as IS and Precision, while maintaining high Recall, demonstrating the strongest overall performance and stability.

[0085] Table 2. Performance comparison of each model on the DeepWeeds and CottonWeedID10 weed datasets.

[0086] IS FID Accuracy Recall rate IS FID Accuracy Recall rate ADM-G (baseline) 7.5305 21.6584 0.7825 0.5284 5.6386 25.3177 0.7710 0.4350 ADM 6.8592 19.5764 0.7517 0.5328 5.9263 21.5878 0.7737 0.4861 ADM-DDIM 7.0598 28.3342 0.6890 0.4647 2.6397 136.6983 0.156 0.0676 StyleGAN2-ada 7.2378 24.0717 0.6482 0.3979 5.1040 16.4872 0.4682 0.5556 StyleGAN3 6.0645 59.0579 0.3045 0.3097 1.3852 338.4835 0 0 LDM 3.5663 97.5187 0.393 0.075 6.2661 39.1689 0.5975 0.3109 This invention 7.4108 18.7152 0.8132 0.5352 6.4346 16.0973 0.7720 0.5063

[0087] To verify the effectiveness of the high-quality weed images generated by this invention in downstream classification tasks, comparative experiments were added to the downstream classification tasks. The experiments were conducted on two weed image datasets: CottonWeedID10 and DeepWeeds. Specifically, weed image samples were generated based on the improved diffusion model of this invention, and the quality of the generated samples was screened using a realism score. After screening, 400 high-quality generated samples were retained for each weed category and added to the training set for data augmentation. This invention selected four classification networks—Inception-v3, DenseNet-121, ResNet-50, and EfficientNet-B3—as baseline classifiers. To ensure the comparability of experimental results, all models used the same data partitioning, training hyperparameters, and optimization procedures, and each experimental setup was repeated five times using Monte Carlo simulations. Experimental results show that after using the samples generated by this invention to expand the training set, most classifiers achieved performance improvements on both datasets. The improvement was more significant on the CottonWeedID10 dataset. For example, Inception-v3 improved its Top-1 accuracy from 97.36% to 98.56%, with precision and recall increasing by approximately 1.75% and 2.10%, respectively. On the DeepWeeds dataset, due to the large size of the original sample, the improvement was relatively smaller, but still showed a stable positive gain. For instance, Inception-v3 improved its Top-1 accuracy from 94.45% to 94.73%; when generative augmentation was combined with traditional augmentation, the Top-1 accuracy further improved to 95.00%.

[0088] The above results show that the weed images generated by the present invention can effectively expand the training samples and improve the sample coverage within the category, thereby improving the recognition accuracy and robustness of the downstream classification model, and are especially suitable for weed recognition scenarios with limited sample numbers or large differences within the category.

[0089] Table 3 shows the test performance (mean ± standard deviation) of the four classifiers on the CottonWeedID10 and DeepWeeds datasets, based on five Monte Carlo replicates. The performance of the original training data, traditional data augmentation (With Trad. Aug), the proposed generative data augmentation (With Diff Aug), and a combination of both (With Both Aug) on ​​Top-1 accuracy, precision, and recall is compared. Values ​​in parentheses represent performance changes (percentage points) relative to the original data.

[0090] Top-1 accuracy (%) Accuracy (%) Recall rate (%) Top-1 accuracy (%) Accuracy (%) Recall rate (%) Original 97.36 ± 0.45 96.55 ± 0.54 94.56 ± 1.14 94.45 ± 0.37 93.12 ± 0.49 92.35 ± 0.41 Inception-v3 WithTrad. Aug 97.12(↓0.24) ±0.38 96.29(↓0.26) ±0.46 94.02(↓0.54)± 0.76 94.63(↑0.18) ±0.20 93.14(↑0.02) ±0.52 92.75(↑0.40) ±0.51 With DiffAug 98.56(↑1.20) ± 0 98.3(↑1.75) ±0.38 96.66(↑2.10)± 0.4 94.73(↑0.28) ±0.36 92.98(↓0.14) ±0.86 93.10(↑0.75) ±0.43 With BothAug 98.37(↑1.01) ±0.31 97.96(↑1.41) ±0.45 96.5(↑1.94)± 0.59 95.00(↑0.55) ±0.28 93.51(↑0.39) ±0.47 93.29(↑0.94) ±0.44 Original 97.79 ± 0.26 97.57 ± 0.86 95.19 ± 0.83 96.47 ± 0.19 95.37 ± 0.21 95.32 ± 0.43 DenseNet121 WithTrad. Aug 97.79(↑0) ± 0.20 97.78(↑0.21) ±0.45 95.01(↓0.18)± 0.66 96.39(↓0.08) ±0.27 95.41(↑0.04) ±0.34 95.16(↓0.16) ±0.56 With DiffAug 98.32(↑0.53) ±0.18 97.93(↑0.36) ±0.57 96.16(↑0.97)± 0.55 96.70(↑0.23) ±0.22 95.51(↑0.14) ±0.33 95.74(↑0.42) ±0.47 With BothAug 98.32(↑0.53) ±0.18 98.12(↑0.55) ±0.5 96.16(↑0.97)± 0.5 96.64(↑0.17) ±0.32 95.52(↑0.15) ±0.37 95.81(↑0.49) ±0.52 Original 98.51 ± 0.31 98.46 ± 0.25 96.93 ± 0.61 96.96 ± 0.19 95.99 ± 0.32 96.08 ± 0.34 ResNet50 WithTrad. Aug 98.32(↓0.19) ±0.24 98.31(↓0.15) ±0.21 96.46(↓0.47)± 0.35 96.93(↓0.03) ±0.28 96.14(↑0.15) ±0.39 96.14(↑0.06) ±0.39 With DiffAug 98.61(↑0.10) ±0.52 98.59(↑0.13) ±0.35 96.97(↑0.04)± 0.64 96.99(↑0.03) ±0.24 96.05(↑0.06) ±0.34 96.39(↑0.31) ±0.18 With BothAug 98.56(↑0.05) ±0.34 98.44(↓0.02) ±0.44 97.01(↑0.08)± 0.73 97.06(↑0.10) ±0.40 96.15(↑0.16) ±0.57 96.44(↑0.36) ±0.59 Original 94.72 ± 0.61 92.2 ± 1.08 89.07 ± 0.72 93.74 ± 0.38 91.44 ± 0.42 92.38 ± 0.62 efficientnet-b3 WithTrad. Aug 94.67(↓0.05) ±0.71 91.98(↓0.22) ±1.39 88.97(↓0.10)± 0.60 93.77(↑0.03) ±0.33 91.42(↓0.02) ±0.29 92.42(↑0.04) ±0.43 With DiffAug 97.12(↑2.40) ±0.38 96.02(↑3.82) ±0.88 95.30(↑6.23)± 0.41 94.29(↑0.55) ±0.43 91.95(↑0.51) ±0.68 93.75(↑1.37) ±0.53 With BothAug 97.22(↑2.50) ±0.24 96.23(↑4.03) ±0.63 95.42(↑6.35)± 0.37 94.33(↑0.60) ±0.33 92.00(↑0.56) ±0.41 93.80(↑1.42) ±0.39

Claims

1. A method for generating weed images using a diffusion model that integrates structural geometry guidance and multi-mechanism optimization, characterized in that, Includes the following steps: a. First, collect a dataset of weed images. The dataset contains several weed images I. Image sizes of images I in the weed image dataset are normalized to obtain the processed image dataset. ; b. Construct a classifier-guided diffusion model (GSS-ADM) that integrates structural geometry guidance and multi-mechanism optimization. The diffusion model incorporates a structural geometry guidance method, a U-Net skip connection scaling method, and a signal-to-noise ratio (SNR)-aware loss. Specifically, the structural geometry guidance method is used to structurally perturb the self-attention layer during the diffusion model sampling process to obtain structural degradation noise prediction results. A structural correction signal is generated based on the difference between the standard noise prediction results and the structural degradation noise prediction results, and the geometric guidance term is used to correct the inverse denoising process in the sample space. The U-Net skip connection scaling method is used to dynamically scale and adjust the skip connection features between the U-Net encoder and decoder during the inference phase. The SNR-aware loss weights the noise prediction error according to the SNR at different time steps. c. Input the preprocessed weed image data into the GSS-ADM model above for training and validation, and obtain the validated optimal model weights; after loading the optimal model weights, perform conditional sampling based on the target category label and the preset generation quantity to generate a set of candidate weed images for the corresponding category; d. Calculate the realism score of the candidate weed images of the corresponding category generated in step c, and retain the generated images with a realism score greater than a preset threshold as qualified samples to obtain high-quality weed generated image data.

2. The method as described in claim 1, characterized in that, The structural geometry guidance method in step b specifically includes: At time t during the denoising process, firstly, in one or more selected self-attention layers of the diffusion model backbone network U-Net, the original self-attention map of that self-attention layer is replaced with a predetermined degradation operator identity matrix to perform structural perturbation, thereby constructing a structural degradation reference; then, intermediate samples from the same denoising time step are... The input models, both unperturbed and perturbed, are forward propagated to obtain standard noise predictions. Predicting structural degradation noise The difference between the two is defined as the differential structure correction signal. ; Subsequently, the differential structure correction signal is applied in the noise prediction space according to the set structure guidance strength coefficient. The noise prediction is scaled and added back to the standard noise prediction to obtain the structure-corrected noise prediction. ; Noise prediction using the structure correction Based on the reverse calculation formula of the diffusion model, the corrected mean of the Gaussian distribution of the reverse process is calculated. ; Next, the gradient of the standard classifier with respect to noisy samples is reshaped in the sample space based on the geometric magnitude reference; firstly, the gradient of the standard classifier with respect to noisy samples is reshaped. gradient L2 normalization is performed to extract orientation information, where y is the true image label of the sample; then, based on noisy samples... The typical distance between the data manifolds at their corresponding time steps is used as a scaling factor to determine the geometric amplitude, reshaping the gradient norm; finally, the overall guiding strength is evenly distributed over the total reverse steps; thus, the geometric guiding term is obtained. ; Where D represents the dimension of the data space. The set geometric amplitude, T being the total number of denoising steps; the geometric guidance term is scaled according to the classifier guidance scaling factor. After scaling and incorporating the samples, a new sampling update formula is obtained; The three terms in the sampling update formula represent the reverse mean after structural correction. Random noise term injected according to the noise standard deviation in this step , For the corresponding standard deviation, and the scaled geometric guide term .

3. The method as described in claim 1, characterized in that, The U-Net skip connection scaling method in step b specifically includes: An adjustable skip connection scaling mechanism is introduced in the inference stage. By dynamically controlling the intensity of skip features, the network adjusts the fusion ratio of feature information at different levels during the generation process. Specifically, based on the existing skip feature fusion mechanism, the skip features at layer n... Corresponding decoder features During fusion, the original direct stitching fusion method is replaced with a fusion method that scales the jump features before stitching, as shown below: in Let be the jump scaling factor for layer 'x', used to adjust the relative contribution of jump information from that layer in the fusion process. This method employs a hierarchical linear interpolation strategy to set the scaling coefficients for each layer, defining two endpoint parameters. and The coefficients of the remaining layers are smoothly transitioned by hierarchical linear interpolation, specifically as follows: Where k is the total number of layers with skip connections and i is the layer index; the hierarchical linear interpolation strategy can achieve smooth control of the distribution of skip information across the entire network by adjusting these two parameters.

4. The method as described in claim 1, characterized in that, The signal-to-noise ratio sensing loss in step b specifically includes: In the training process of noise prediction for the diffusion model, a time-step-dependent weight is applied to the noise prediction error according to the signal-to-noise ratio at time step t to construct the loss function; let the signal preservation coefficient at step t be... The signal-to-noise ratio formula is: Weighting coefficients based on noise prediction error As a weight: Based on this weight, the signal-to-noise ratio (SNR) perceived weighted noise prediction loss is defined as: in This represents the noise sampled during forward diffusion. This represents the model's prediction of noise, with an expected value for time step t and sample... With noise Sampling; the weighted form imposes a higher training penalty on the prediction error at low signal-to-noise ratio time steps.

5. The method as described in claim 1, characterized in that, The method for generating weed images of the corresponding category using the optimal model weights in step c specifically includes: Load the trained and validated optimal GSS-ADM weight file, set sampling parameters including total denoising steps, classifier guidance strength coefficient, and structure guidance strength coefficient; for each target weed category, input the corresponding category label as conditional information, and sample random noise samples from a standard Gaussian distribution as initial samples; perform a backdiffusion sampling process within the set denoising time steps, and in each time step, combine the structure geometry guidance method and the U-Net skip connection scaling method to correct the sampling process, and calculate the image samples corresponding to the current time step according to the updated sampling formula; after completing all denoising steps, output the generated weed image, and repeat the sampling process for each weed category until the preset number of generated images is reached.

6. The method as described in claim 1, characterized in that, The realism score screening method in step d specifically includes: The feature extractor is used to extract features from a dataset of real weed images to obtain a set of real image feature vectors. The same feature extractor is used to extract features from the weed image generated in step c, resulting in the feature vector of the generated image. ; In the set of feature vectors of real images In the context of any real sample feature vector Search its k-th nearest neighbor sample And calculate the Euclidean distance between the real sample and its k-th nearest neighbor: To characterize the local feature distance in the real data distribution; further calculate the generated sample feature vector. With the feature vector of the real sample Euclidean distance between them: The realism score R of the generated image is calculated using the following formula: The realism score R is used to measure the similarity between the generated image and the feature distribution of the real image. The realism score R is compared with a preset screening threshold. When R > the preset screening threshold, the generated image is determined to be a high-quality sample that conforms to the feature distribution of the real data and is retained. When R ≤ the preset screening threshold, the generated image is determined to be a low-quality sample and is discarded.

Images