Plant fiber electron microscope image generation method based on diffusion model and related equipment

Abstract

The embodiment of the invention provides a plant fiber electron microscope image generation method based on a diffusion model and related equipment, and belongs to the field of computer vision and image generation. The method comprises the following steps: acquiring and marking an image data set containing a fiber category and an amplification factor; preprocessing and enhancing the data; a conditional noise prediction network is constructed, the network adopts a UNet architecture, continuous amplification factor conditions are processed through a Fourier coding layer based on logarithmic coordinates, and multi-modal condition information is deeply fused through a feature map affine modulation mechanism; training a network based on a modified flow framework to learn a vector field from noise to a clean image; in the reasoning stage, a classifier-free multi-condition guide technology is utilized to carry out iterative denoising, and a high-quality fiber electron microscope image conforming to a specified category and amplification factor is generated. The method has the advantages of being high in generation efficiency, accurate in continuous condition control, high in multi-condition fusion depth and the like, and an efficient data enhancement tool can be provided for fiber scientific research and industrial application.

Description

Technical Field

[0001] This application relates to the fields of computer vision and image generation, and in particular to a method and related equipment for generating electron microscopy images of plant fibers based on a diffusion model. Background Technology

[0002] The microscopic morphology of plant fibers, such as length, diameter, and degree of fibrillation, is a key factor determining their physicochemical properties and the performance of their final products (such as paper, textiles, and composite materials). Scanning electron microscopy (SEM) is the primary tool for observing these microstructures and is indispensable in both basic research and industrial applications. However, obtaining SEM images through physical experiments suffers from inherent bottlenecks such as high cost, cumbersome procedures, and long cycles, resulting in a severe scarcity and uneven distribution of image data for specific species or experimental conditions. This "data shortage" significantly restricts the application and development of data-driven modern research methods, particularly deep learning technology, in the field of plant fiber science.

[0003] To alleviate data shortages, traditional methods employ techniques such as affine transformations and color dithering for data augmentation. However, the resulting images exhibit limited diversity, making it difficult to create new samples with essential feature variations. While deep generative models such as generative adversarial networks have made progress, they often face problems such as training instability, pattern collapse, and insufficient detail fidelity when processing high-detail, structurally complex scientific images like fiber electron microscope images.

[0004] As a representative of the new generation of generative models, the diffusion model has achieved significant breakthroughs in image quality. However, its direct application to the generation of electron microscopy images of plant fibers still faces severe challenges: (1) Traditional diffusion models have many sampling iteration steps and slow generation speed, making it difficult to meet the practical needs of rapid data amplification; (2) For continuous numerical conditions such as "magnification," there is a lack of effective coding and control mechanisms, making it difficult to accurately generate images at a specified scale and level of detail; (3) It is difficult to perform in-depth collaborative control of multimodal conditions such as discrete categories and continuous parameters, and there is a lack of flexible adjustment mechanisms for the influence of each condition during the inference stage, making it impossible to achieve a good balance between the fidelity and diversity of the generated results. Summary of the Invention

[0005] The main objective of this application is to propose a method, electronic device, storage medium, and program product for generating SEM images of plant fibers based on a diffusion model. This aims to solve the problems of difficulty in acquiring SEM images of plant fibers and data scarcity. It can quickly and controllably generate high-fidelity synthetic images that meet the specified fiber type and magnification conditions, providing an efficient data support tool for fiber science research, industrial quality inspection, and deep learning model training.

[0006] To achieve the above objectives, one aspect of this application proposes a method for generating electron microscopy images of plant fibers based on a diffusion model, the method comprising: Data preparation: Obtain a dataset containing electron micrographs of plant fibers and their annotations, wherein the annotations include at least fiber type and magnification; Data preprocessing: The dataset is divided into training and testing sets, and the images are preprocessed. Model Construction: A noise prediction network based on the UNet network architecture is constructed. The input of the network includes the noisy image, diffusion time step, fiber category condition and magnification condition. Model training: Based on the modified flow framework, the noise prediction network is trained with the goal of predicting the vector field from the noisy image to the original clean image, and the network parameters are saved. Image generation: Load the trained network, start with random noise, and perform multiple rounds of iterative denoising with classifier-free guidance to generate electron micrographs of plant fibers that meet the specified fiber category and magnification conditions.

[0007] In some embodiments, the data preparation steps include: Preparation of plant fiber samples includes pretreatment, dissociation, washing and freeze-drying steps; The sample was observed using a scanning electron microscope (SEM), and images were acquired in multiple regions at multiple different magnifications to construct an original image set.

[0008] In some embodiments, the data preprocessing steps include: The image is randomly cropped, and the magnification value of the corresponding image is updated according to the ratio between the cropped area and the original image. The cropped image is scaled to a uniform size and then randomly flipped horizontally with a preset probability. Normalize the image pixel values.

[0009] In some embodiments, the constructed noise prediction network is specifically implemented as follows: The noise prediction network accepts four inputs: a noisy image. Fiber category label Magnification value diffusion time step ; A category embedding module is provided to embed discrete fiber category labels. Mapped to categorical feature vectors ; A magnification encoding module is provided, which uses continuous Fourier encoding based on logarithmic coordinates to encode the magnification value. Mapped to magnification feature vector ; A time step mapping module is provided to diffuse time steps. Mapped to time step feature vectors ; The category feature vector Magnification feature vector and time step feature vector By splicing the data, a unified conditional representation is obtained. ; The UNet network includes an encoder, intermediate blocks, and a decoder; each layer of the encoder and decoder is provided with a corresponding feature modulation module; The feature modulation module is characterized by the aforementioned conditions. As input, scaling parameters for performing affine transformations on the output feature map of the current layer are predicted using at least one multilayer perceptron. and bias parameters This allows for the deep modulation of network feature maps by conditional information.

[0010] In some embodiments, the specific calculation method of the continuous Fourier encoding based on logarithmic coordinates is as follows:

[0011]

[0012] in, For feature vectors Dimensions The frequency weighting parameter is either preset or randomly sampled.

[0013] In some embodiments, the specific steps of model training include: Forward noise addition process: for a clean image Noise is added through linear interpolation. To obtain a noisy image ,in , It follows a standard Gaussian distribution; Loss function calculation: The training objective of the noise prediction network is the prediction vector field. This makes it approximate the real vector field. The loss function is defined as the L2 norm between the two: ; Random conditional discarding: During training, the input fiber category conditions are discarded independently with a certain probability. Or magnification conditions Set it to zero so that the network can learn both conditional and unconditional generation simultaneously.

[0014] In some embodiments, the specific steps of image generation include: A1: The initial noise image is obtained by sampling from a standard Gaussian distribution. ; A2: Set the target fiber category and target magnification And set category-guided weights and magnification-guided weights ; A3: For each step in the denoising process, calculate the following output: Unconditional output: ; Category conditional output: ; Magnification conditional output: ; A4: Use a classifier-free approach to correct the model output, resulting in the final denoising direction: ; A5: Update the image based on the corrected denoising direction: ,in This is the noise reduction step size; A6: Iterate through steps A3 to A5 until the final generated image is obtained. .

[0015] To achieve the above objectives, another aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described above.

[0016] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described above.

[0017] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer program product, including a computer program that, when executed by a processor, implements the method described above.

[0018] Compared with the prior art, this application has the following beneficial effects: 1) High generation efficiency and good image quality: This application adopts the modified flow framework, whose linear noise path and concise vector field learning objective enable the model to generate high-quality images with fewer sampling steps during inference, which significantly improves generation efficiency and meets the actual needs of rapid data augmentation in scientific computing.

[0019] 2) Precise control of continuous conditions: An innovative approach employs continuous Fourier coding based on logarithmic coordinates to handle the continuous condition of magnification. This coding method effectively matches the exponential characteristics of SEM image scale changes, achieving precise, non-linear control of the generated image magnification (i.e., scale and detail levels), thus solving the problem of inaccurate control in existing methods.

[0020] 3) Multi-condition deep fusion and flexible guidance: By encoding multiple conditions into a unified representation and utilizing a feature modulation mechanism for deep fusion at each layer of UNet, the conditional information is fully integrated into the generation process. Combined with a specially designed multi-condition classifier-free guided inference strategy, users can independently adjust two guiding weights ( and It can flexibly balance the relationship between fiber type fidelity, magnification accuracy and overall diversity in the generated image, and has strong controllability.

[0021] 4) Effectively solves the problem of data scarcity: This application can generate a large number of high-fidelity electron micrographs of plant fibers with specified categories and magnifications according to needs, which greatly expands the scarce small sample dataset and provides rich and high-quality training data for subsequent tasks such as fiber morphology analysis, automatic classification, and process inference based on deep learning, thereby reducing data acquisition costs. Attached Figure Description

[0022] Figure 1 This is a flowchart of a method for generating electron microscopy images of plant fibers based on a diffusion model, as described in an embodiment of this application.

[0023] Figure 2 This is a schematic diagram of scanning electron microscope image data acquisition in an embodiment of this application.

[0024] Figure 3 This is a schematic diagram illustrating data augmentation of training images in an embodiment of this application.

[0025] Figure 4 This is a structural diagram of the conditional modulation UNet network model constructed in the embodiments of this application.

[0026] Figure 5 This is a flowchart illustrating the steps of a method for generating electron microscopy images of plant fibers based on a diffusion model, as described in an embodiment of this application.

[0027] Figure 6 This is a schematic diagram of the hardware structure of the electronic device in the embodiments of this application. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0029] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0030] Before providing a detailed description of the embodiments of this application, some of the nouns and terms involved in the embodiments of this application will be explained first. The nouns and terms involved in the embodiments of this application are subject to the following interpretations.

[0031] 1) Scanning electron microscopy (SEM) is an electron optical instrument that obtains information about the microstructure and composition of materials by scanning the surface of a sample with a focused electron beam. It is widely used in the fields of chemistry and materials science, and can be used for observing the microstructure of materials, analyzing the morphology of inclusions, detecting fracture morphology, and analyzing the composition of micro-areas.

[0032] The microscopic morphology of plant fibers (such as fiber length, diameter, and cell wall thickness) is a decisive factor in their physicochemical properties and final application performance (such as paper strength and the mechanical properties of composite materials). Scanning electron microscopy (SEM), as the primary tool for observing these microstructures, plays an irreplaceable role in plant fiber research, pulp and paper making, textiles, and the development of biocomposite materials. However, the traditional paradigm of relying on physical experiments to obtain electron microscopic images of fibers has a series of inherent bottlenecks, severely restricting the efficiency of related scientific research and industrial innovation.

[0033] From a data acquisition perspective, acquiring SEM images of plant fibers is a costly, complex, and highly specialized task. The process typically involves multiple complex steps, including sample preparation (such as pretreatment, dissociation, cleaning, dehydration, freeze-drying, and gold sputtering), electron microscope parameter adjustment, and image capture. This not only results in significant time and economic costs but also makes accumulating large-scale image data for specific rare fiber species or under specific experimental conditions (such as ultra-high magnification) extremely difficult. This problem of "data scarcity" and "uneven data distribution" has become a core obstacle restricting the application of data-driven modern research methods (especially deep learning technology) in this field. The lack of sufficient, high-quality training data significantly reduces the performance and generalization ability of advanced fiber automatic classification, segmentation, and morphological parameter measurement models.

[0034] To overcome the data shortage problem, creating synthetic data using generative artificial intelligence techniques has become a promising solution. Early research attempted to use traditional image processing techniques (such as affine transformations and color dithering) for data augmentation, but these methods produced images with limited diversity and struggled to create new samples with essential feature variations, resulting in unsatisfactory results. Subsequently, deep generative models such as generative adversarial networks showed greater potential, but when applied to detailed and structurally complex scientific images such as fiber electron microscope images, their inherent shortcomings became apparent: for example, the training process was difficult to stabilize, often resulting in "mode collapse" leading to poor diversity of generated images; and the fidelity in restoring image details and textures was insufficient, failing to meet the stringent requirements of image realism in scientific analysis.

[0035] In recent years, diffusion models, as a benchmark for the new generation of generative artificial intelligence, have achieved breakthroughs in image generation quality, with the clarity and diversity of generated images generally superior to previous models. However, applying diffusion models directly to the generation of electron microscopy images of plant fibers still faces the following key challenges: (1) The diffusion model has many iterative denoising steps and slow generation speed, which cannot meet the actual needs of rapid data amplification in scientific computing; (2) For continuous numerical conditions such as magnification, existing methods lack effective coding mechanisms, making it difficult to accurately control their nonlinear changes, resulting in inaccurate scale and detail of the generated images; (3) It is difficult to deeply coordinate and control multimodal conditions such as discrete categories and continuous parameters, and there is a lack of a flexible mechanism to adjust the weights of each condition during inference, making it impossible to balance the fidelity and diversity of the generated results.

[0036] In summary, existing technologies lack a comprehensive solution that simultaneously addresses generation efficiency, precise control of continuous conditions, and flexible guidance of multiple conditions. Therefore, this application proposes a method, electronic device, storage medium, and program product for generating electron microscopy images of plant fibers based on a diffusion model. This solution significantly improves generation efficiency by linearizing the diffusion path; it uses Fourier encoding to map continuous magnification to high-dimensional features, ensuring precise control of scale changes; and it designs a conditional fusion mechanism based on feature map modulation, combined with classifier-free multi-condition guidance technology, enabling flexible intervention of multi-condition weights during the generation process. This solution effectively overcomes the limitations of existing technologies, providing a high-quality, highly controllable image generation tool for plant fiber research.

[0037] This application provides a method for generating electron microscopy images of plant fibers based on a diffusion model, relating to the fields of computer vision and artificial intelligence-generated content technology. This method can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, or in-vehicle terminal, but is not limited to these. The server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network. The software can be an application implementing a diffusion model-based method for generating electron microscopy images of plant fibers, but is not limited to the above forms.

[0038] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0039] like Figure 5 As shown, this embodiment provides a method for generating electron microscopy images of plant fibers based on a diffusion model, including the following steps: S1. Data preparation: Obtain a dataset containing electron micrographs of plant fibers and their annotations, wherein the annotations include at least fiber type and magnification.

[0040] For example, in the data preparation and annotation stage, plant fiber samples are first prepared according to the standard procedure; then, the samples are observed using a scanning electron microscope, and images at different magnifications are collected in different regions to construct the original image dataset; finally, each image in the dataset is annotated, and the annotation content mainly includes the fiber type and the actual magnification of the image.

[0041] S2. Data preprocessing: Divide the dataset into training and testing sets, and preprocess the images.

[0042] Specifically, the labeled electron microscopy image dataset of plant fibers was divided into a training set for model training and a test set for evaluating model performance. The divided images were preprocessed, including random cropping (and corresponding correction of their magnification labels), scaling the images to a uniform size, randomly flipping them horizontally with a certain probability, and normalizing the pixel values ​​to improve the model's generalization ability and training stability.

[0043] S3. Model Construction: Construct a noise prediction network based on the UNet network architecture. The input of the network includes the noisy image, diffusion time step, fiber category condition, and magnification condition.

[0044] In some embodiments, a noise prediction network with UNet as its backbone is constructed. The inputs of this model include: a noisy image, a diffusion time step, fiber category, and magnification. First, the model encodes discrete categories, continuous magnification, and time steps into feature vectors through different embedding layers, and concatenates them into a unified conditional representation. Subsequently, this conditional representation is input into a series of dedicated multilayer perceptrons to generate scaling (γ) and bias (β) parameters for dynamically modulating network features. At each layer of the U-shaped network, the feature maps output by the encoder and decoder are affine transformed with these parameters, thereby achieving deep control of the generation process by the conditional information. Finally, the model outputs a predicted vector field, which is the difference between the real noise and the original clean image.

[0045] S4. Model Training: Based on the modified flow framework, the noise prediction network is trained with the goal of predicting the vector field from the noisy image to the original clean image, and the network parameters are saved.

[0046] In some embodiments, the model is trained based on a modified flow framework, which mixes the original image with standard Gaussian noise through linear interpolation to obtain a noisy image. The training objective is to enable the UNet model to predict the vector field from the noise to the real image. A random conditional discarding strategy, which sets the category or magnification condition to zero with a certain probability, allows the model to simultaneously master both conditional and unconditional generation capabilities, laying the foundation for subsequent precise and controllable image generation.

[0047] S5. Image Generation: Load the trained network, start with random noise, and perform multiple rounds of iterative denoising with classifier-free guidance to generate electron micrographs of plant fibers that meet the specified fiber category and magnification conditions.

[0048] During the inference phase, a completely noisy image is first sampled from a standard Gaussian distribution, followed by multiple rounds of iterative denoising. In each denoising step, the current noisy image, time step, class condition, and magnification condition are input into the trained UNet model to predict a denoising direction vector. To improve the quality of the generated image and its adherence to the specified conditions, a classifier-free multi-condition guidance technique is used to correct the model's original output; that is, the model's output with specified conditions and its output without conditions (conditions set to zero) are calculated simultaneously. Then, the unconditional output is subtracted from the conditional output to obtain a difference representing the "conditional signal." This difference is multiplied by a guidance weight coefficient and added back to the unconditional output, resulting in a corrected final output that more strongly adheres to the generation instructions.

[0049] The solutions of the embodiments of this application will be described in detail below with reference to the accompanying drawings and specific application examples.

[0050] like Figure 1 As shown, this embodiment provides a method for generating electron microscopy images of plant fibers based on a diffusion model. The main steps of this method are as follows: Step 1: Data acquisition and annotation, including preparing fiber samples, taking images using a scanning electron microscope, and labeling each image with its category and magnification.

[0051] As an optional implementation, step 1 specifically includes the following steps: Step 1.1: Prepare fiber samples. The sample preparation process includes raw material pretreatment, dissociation, cleaning, freeze drying, gold sputtering, etc. Step 1.2: Place the processed sample into a scanning electron microscope and observe it. The observation method is as follows: Figure 2As shown, the operator systematically selects multiple representative areas to ensure the capture of various fiber forms (intact, broken, intertwined), and takes pictures of each selected area at different magnifications, including ×50, ×100, ×250, ×500, ×1500, ×3500, and ×5000.

[0052] Step 1.3: Create a corresponding annotation file for each electron microscope image, including information such as fiber type and magnification.

[0053] Step 2: Data partitioning and preprocessing. Divide the dataset into training and test sets, and perform necessary preprocessing and data augmentation.

[0054] As an optional implementation, step 2 specifically includes the following steps: Step 2.1: Divide the collected data into two parts: a training set and a test set, which are used to train the model and evaluate the model's generation capability, respectively. Step 2.2: Perform data augmentation on the images in the training set, such as... Figure 3 As shown, the image is first randomly cropped by selecting a small patch from the high-resolution image. The actual field of view of the image changes after cropping. Based on the ratio of the cropped area to the original image, the corresponding magnification label is recalculated and updated. Then, all cropped images are scaled to a uniform fixed size of 192×256. Next, the image is randomly horizontally flipped with a probability of 0.5. Finally, the image is normalized by scaling the image value range to [-1, 1] to ensure the stability of the model training process.

[0055] Step 3: Construct the UNet network model, whose core feature is that it can accept multiple conditional inputs to control image generation.

[0056] In some embodiments, the overall model structure of the UNet network is as follows: Figure 4 As shown, the specific implementation method is as follows: (1) The model input includes noisy images. Category tags Magnification diffusion time step ; (2) Category Labels Category representations are obtained through discrete learnable embedding layers (each category corresponds to a learnable vector). To achieve category-based condition control; (3) Magnification Encoding is performed using a continuous Fourier coding layer to obtain the magnification representation. To achieve magnification control, the specific calculation is as follows:

[0057]

[0058] in, For the first Frequency weights randomly sampled from a Gaussian distribution The dimension representing the magnification.

[0059] (4) Diffusion time step Time step representations are obtained through mapping using a multilayer perceptron (MLP). .

[0060] (5) The UNet network consists of three parts: Encoder, MidBlock, and Decoder, where the Encoder contains The MidBlock consists of a convolutional block (EncoderBlock) and a downsampling layer (Downsample). The Decoder contains multiple convolutional layers. Each convolutional block (DecoderBlock) and upsampling layer (Upsample) are used. For ease of explanation, we will use... .

[0061] (6) Representing categories Magnification characterization Time step representation The characterization is obtained by splicing. Using representation The feature maps output from each layer of UNet are modulated. The overall computation of UNet is shown below: In EncoderBlock, the specific calculation is as follows:

[0062]

[0063]

[0064]

[0065] in, It is the first Encoder Each block, It is the first Input of each block ( ), For the first Feature maps output by each block and Two MLPs are used to predict the scaling and bias coefficients of the feature maps. This is the modulated feature map.

[0066] In MidBlock, the specific calculation is as follows:

[0067] In DecoderBlock, the specific calculation is as follows:

[0068]

[0069]

[0070]

[0071] in, It is the Decoder's first Each block, after upsampling and splicing as the first The input of each block, and Two MLPs are used to predict the scaling and bias coefficients of the feature maps. The modulated feature map, the final output .

[0072] Step 4: Train the network model based on the modified flow. Linearly add noise to a real image and use it as input to the UNet network. The target of the model output is the vector difference between the noise and the real image.

[0073] As an optional implementation, the specific steps for training a UNet model based on the modified flow framework are as follows: (1) The noisy image at any given moment is derived from the initial image. The result obtained after linear noise addition is shown in the following equation:

[0074] in, The noise sampled from a standard Gaussian distribution is the same size as the image.

[0075] (2) The loss function for training the UNet model is shown in the following formula:

[0076] (3) During training, the category representation and magnification representation are set to 0 with a probability of 20% to achieve unconditional generation. The output of the model is denoted as follows: and .

[0077] Step 5: Using the trained UNet model, perform multiple rounds of iterative denoising starting from a completely noisy image to finally obtain a clear fiber image.

[0078] As an optional implementation, step 5 specifically includes the following steps: Step 5.1: Sample a completely noisy image from a standard Gaussian distribution. ; Step 5.2: Correct the model output using a classifier-free approach, as shown in the following formula:

[0079] in, , The corrected model output. and These are the weights for category-based condition control and magnification-based condition control, respectively, in practical applications. and It can be selected according to actual needs.

[0080] Step 5.3: For completely noisy images Perform multiple rounds of noise reduction, as shown in the following formula:

[0081] in, This is the step size for each noise reduction step. In practical applications... It can be selected according to actual needs. Generally speaking, The smaller the value, the higher the quality of the generated image, but the slower the generation speed.

[0082] In summary, the method of this embodiment has the following advantages compared with the existing technology: 1) This embodiment addresses the problem of scarce plant fiber electron microscopy image data by proposing a high-quality image generation method based on a diffusion model. By introducing category and magnification as conditional information, it can synthesize detailed and highly realistic plant fiber electron microscopy images, allowing users to precisely control the attributes of the generated images and meet the needs of image sample diversity and controllability in different application scenarios.

[0083] 2) The method proposed in this embodiment can effectively achieve data augmentation, significantly expand the small-scale and scarce plant fiber electron microscopy image dataset, provide a large number of high-quality training samples for deep learning-based fiber morphology analysis, classification and recognition tasks, reduce dependence on real samples and data acquisition costs.

[0084] 3) This embodiment adopts the modified flow model, which has the advantages of simple training objectives and high sampling efficiency. Combined with classifier-free guidance technology, it can flexibly adjust the balance between the quality and diversity of generated images during the inference stage, thereby ensuring that the generated results have high stability and reliability.

[0085] For example, the method of this embodiment can be applied to the following fields: 1) Industrial quality control and inspection: In industrial fields such as pulp and paper making, textile manufacturing, and biocomposite materials, it can be used to generate fiber images of different quality grades, expand the defect sample library, and thus improve the robustness and accuracy of vision-based automatic quality inspection and classification models.

[0086] 2) Pulping Process Optimization and Fault Diagnosis: By generating fiber morphology images under different cooking degrees and beating degrees, a mapping library of process parameters and fiber morphology is constructed. When problems such as insufficient paper strength occur during production, the abnormality in the process can be quickly located by comparing the real fiber images with the generated standard image library representing different process states, thus assisting in process optimization and fault diagnosis.

[0087] 3) Digital prediction of beating energy consumption: Fiber morphology (such as the degree of fibrillation) directly affects beating energy consumption. By generating fiber images with different degrees of fibrillation, a correlation model between fiber micromorphology and beating energy consumption can be established, providing data support for optimizing the beating process and reducing energy consumption.

[0088] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0089] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0090] Please see Figure 6 , Figure 6 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes: The processor 601 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 602 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 602 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 602 and is called and executed by the processor 601 using the methods described in the embodiments of this application. The input / output interface 603 is used to implement information input and output; The communication interface 604 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 605 transmits information between various components of the device (e.g., processor 601, memory 602, input / output interface 603, and communication interface 604); The processor 601, memory 602, input / output interface 603, and communication interface 604 are connected to each other within the device via bus 605.

[0091] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0092] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0093] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0094] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0095] It is understood that the content of the above method embodiments is applicable to the embodiments of this program product. The specific functions implemented in the embodiments of this program product are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments. The executable computer program code or "code" used to perform the various embodiments can be written in high-level programming languages ​​such as C, C++, Python, Smalltalk, Java, JavaScript, Visual Basic, Structured Query Language (e.g., Transact-SQL), Perl, or in various other programming languages.

[0096] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0097] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0098] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0099] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0100] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0101] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0102] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0103] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0104] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0105] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0106] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A method for generating electron microscopy images of plant fibers based on a diffusion model, characterized in that, The method includes the following steps: Data preparation: Obtain a dataset containing electron micrographs of plant fibers and their annotations, wherein the annotations include at least fiber type and magnification; Data preprocessing: The dataset is divided into training and testing sets, and the images are preprocessed. Model Construction: A noise prediction network based on the UNet network architecture is constructed. The input of the network includes the noisy image, diffusion time step, fiber category condition and magnification condition. Model training: Based on the modified flow framework, the noise prediction network is trained with the goal of predicting the vector field from the noisy image to the original clean image, and the network parameters are saved. Image generation: Load the trained network, start with random noise, and perform multiple rounds of iterative denoising with classifier-free guidance to generate electron micrographs of plant fibers that meet the specified fiber category and magnification conditions.

2. The method according to claim 1, characterized in that, The data preparation steps include: Preparation of plant fiber samples includes pretreatment, dissociation, washing and freeze-drying steps; The sample was observed using a scanning electron microscope, and images were acquired in multiple regions at multiple different magnifications to construct an original image set.

3. The method according to claim 1, characterized in that, The data preprocessing steps include: The image is randomly cropped, and the magnification value of the corresponding image is updated according to the ratio between the cropped area and the original image. The cropped image is scaled to a uniform size and then randomly flipped horizontally with a preset probability. Normalize the image pixel values.

4. The method according to claim 1, characterized in that, The noise prediction network is specifically implemented as follows: The noise prediction network accepts four inputs: a noisy image. Fiber category label Magnification value diffusion time step ; A category embedding module is provided to embed discrete fiber category labels. Mapped to categorical feature vectors ; A magnification encoding module is provided, which uses continuous Fourier encoding based on logarithmic coordinates to encode the magnification value. Mapped to magnification feature vector ; A time step mapping module is provided to diffuse time steps. Mapped to time step feature vectors ; The category feature vector Magnification feature vector and time step feature vector By splicing the data, a unified conditional representation is obtained. ; The UNet network consists of an encoder, intermediate blocks, and a decoder; each layer of the encoder and decoder has a corresponding feature modulation module. The feature modulation module is characterized by the aforementioned conditions. As input, scaling parameters for performing affine transformations on the output feature map of the current layer are predicted using at least one multilayer perceptron. and bias parameters This allows for the deep modulation of network feature maps by conditional information.

5. The method according to claim 4, characterized in that, The specific calculation method for the continuous Fourier encoding based on logarithmic coordinates is as follows: in, For feature vectors Dimensions The frequency weighting parameter is either preset or randomly sampled.

6. The method according to claim 1, characterized in that, The specific steps for training the model include: Forward noise addition process: for a clean image Noise is added through linear interpolation. To obtain a noisy image ,in , It follows a standard Gaussian distribution; Loss function calculation: The training objective of the noise prediction network is the prediction vector field. This makes it approximate the real vector field. The loss function is defined as the L2 norm between the two: ; Random conditional discarding: During training, the input fiber category conditions are discarded independently with a certain probability. Or magnification conditions Set it to zero so that the network can learn both conditional and unconditional generation simultaneously.

7. The method according to claim 1, characterized in that, The specific steps for generating the image include: A1: The initial noise image is obtained by sampling from a standard Gaussian distribution. ; A2: Set the target fiber category and target magnification And set category-guided weights and magnification-guided weights ; A3: For each step in the denoising process, calculate the following output: Unconditional output: ; Category-based conditional output: ; Magnification conditional output: ; A4: Use a classifier-free approach to correct the model output, resulting in the final denoising direction: ; A5: Update the image based on the corrected denoising direction: ,in This is the noise reduction step size; A6: Iterate through steps A3 to A5 until the final generated image is obtained. .

8. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method according to any one of claims 1 to 7.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

Images