Food image generation method based on double-flow diffusion control model

Abstract

The invention discloses a food image generation method based on a double-flow diffusion control model. The method comprises the following steps: constructing a food image generation architecture based on a diffusion model; a self-adaptive scaling mechanism and a global and local attention module are introduced, and semantic information and detail textures in the food image are optimized; based on a fusion network of a gating mechanism, dynamically adjusting a fusion proportion of a control signal and a generation signal, and adaptively optimizing a generation process; and based on a double-flow diffusion generation strategy, the generation model de-noising a noise image step by step through a diffusion model to generate a high-quality image. According to the food image generation method based on the double-flow diffusion control model, a double-flow interaction mechanism, a gating fusion mechanism and a self-adaptive scaling mechanism are introduced, so that the quality and detail capturing capability of the generated food image are remarkably improved; by adding bidirectional information flow between the generation network and the control network, the problem of balance between a global structure and local details in food image generation is effectively solved.

Description

Technical Field

[0001] The present invention relates to the technical field of food image generation, and in particular to a food image generation method based on a dual-flow diffusion control model. Background Art

[0002] The technical development in the field of food image generation mainly relies on generative adversarial networks (GAN) and diffusion models. GAN-type models (such as CookGAN) perform well in generating the overall structure of the image, but have significant defects in detail presentation (such as the surface texture and gloss reflection of food), especially in the generation of complex food images, where the lack of details is particularly prominent. In addition, when generating high-quality food images, although the diffusion model can restore the global structure well, it still lacks in capturing local details.

[0003] For example, when processing food images with complex details, existing diffusion models cannot accurately capture local details such as texture and gloss, resulting in a lack of realism and delicacy in the generated food images. Control conditions (such as edge maps, depth maps, etc.) have a weak guiding effect on the generated images, and the fine control of details during the generation process is insufficient. Therefore, existing methods fail to fully utilize the differentiated characteristics of multimodal conditions, resulting in the inability to flexibly adjust the contributions of different conditions during the image generation process, thereby limiting the generation quality.

[0004] Therefore, based on the above-mentioned technical defects, the present invention proposes a food image generation method based on a dual-flow diffusion control model to effectively solve the problem of balancing global structure and local details in food image generation. Summary of the invention

[0005] The purpose of the present invention is to provide a food image generation method based on a two-stream diffusion control model, which significantly improves the quality and detail capture ability of generated food images by introducing a two-stream interaction mechanism, a gating mechanism, an adaptive scaling mechanism and an attention mechanism; by increasing the two-way information flow between the generation network and the control network on the basis of the diffusion model, and realizing feature transfer through a zero convolution layer, the problem of balancing the global structure and local details in food image generation is effectively solved.

[0006] To achieve the above object, the present invention provides a method for generating food images based on a dual-flow diffusion control model, comprising the following steps: Step S1, constructing a food image generation architecture based on a dual-flow diffusion control model; Step S2, introducing an adaptive scaling mechanism and global and local attention modules to optimize the semantic information and detail texture in food images; Step S3: Based on the fusion network of the gating mechanism, dynamically adjust the fusion ratio of the control signal and the generated signal, and adaptively optimize the generation process according to the characteristics of the food image and the control conditions; Step S4: Based on the dual-stream diffusion generation strategy, the noisy image is gradually denoised through the generation model of the food image generation architecture based on the dual-stream diffusion control model to generate a high-quality image that meets the text description and control conditions.

[0007] Preferably, in step S1, the food image generation architecture based on the dual-stream diffusion control model includes a CLIP encoder, a variational autoencoder, a temporal encoder, a control network based on a gating mechanism, an image generation network and a decoder; The food image generation architecture based on the two-stream diffusion control model combines the image generation network with the control network to generate food images in an end-to-end manner.

[0008] Preferably, the input of the food image generation architecture based on the two-stream diffusion control model includes text, time steps, conditional images and Gaussian noise; the time steps are processed by the temporal encoder, the text is processed by the CLIP encoder, and the generation process is further guided by the cross-attention mechanism; The control network based on the gating mechanism takes the conditional image as input and extracts features related to the shape and spatial information of food through the image annotation module; The image generation network works in parallel with the control network, and their interaction is dynamically controlled and fused by a gating mechanism that selectively determines the flow of features to subsequent stages.

[0009] Preferably, in the food image generation architecture based on the two-stream diffusion control model, an adaptive scaling mechanism is introduced to adjust the feature allocation of the backbone network and the skip connection; Global and local attention modules are introduced into each stage of the decoder to further focus on the semantic information and detailed textures in food images.

[0010] Preferably, in step S2, an adaptive scaling mechanism is introduced to adjust the feature allocation of the backbone network and the jump connection, and the specific process is as follows: Step S211: Input noise After being processed by the U-Net encoder, multi-scale features are extracted ;in, is the batch size, is the number of channels, and Respectively represent the spatial size of the feature map; Step S212, enhancing the main features; Main features After channel-level scaling, channel-level scaling is achieved through learnable parameters Adjustments are made to enhance the global consistency of the backbone network as follows: ; in, Learnable channel-level scaling parameters for backbone features; Acting on each channel, the features are scaled differently on different channels; Step S213: performing frequency domain adjustment on the skip connection feature to optimize the skip connection feature; Step S214: Concatenate the optimized backbone features and skip connection features in the channel dimension to obtain fusion features, as shown below: ; in, Indicates fusion features; and They represent the optimized backbone features and skip connection features respectively.

[0011] Preferably, in step S213, the skip connection feature Directly passed from the encoder to the decoder, frequency domain adjustment is performed to optimize information transmission. The specific process is as follows: (1) First, the skip connection features are converted to the frequency domain to obtain the frequency domain representation: ; in, represents Fourier transform, which converts the features of spatial domain into frequency domain so that different frequency components can be processed separately; (2) In order to distinguish high-frequency and low-frequency information, the spectral energy distribution of the skip connection feature is calculated , which measures the contribution of different frequency components, as follows: ; in, Indicates the radius The total energy of the frequency components within represents the proportion of information within this frequency range; In order to dynamically determine the dividing point between low frequency and high frequency , set the cumulative energy ratio threshold , as shown below: ; in, Indicates the energy proportion of controlling the low-frequency part; (3) In order to dynamically scale low-frequency and high-frequency features, a frequency mask is introduced , as shown below: ; in, is the learnable channel-level scaling parameter of the skip connection feature, and the skip connection feature after frequency mask adjustment , as shown below: ; (4) The adjusted frequency domain features are converted back to the spatial domain through inverse Fourier transform as shown below: .

[0012] Preferably, in step S2, global and local attention modules are added in each stage of the decoder to further focus on the semantic information and detail texture in the food image. The specific process is as follows: Step S221: local attention takes the input fusion features It is implemented by dividing into 3×3 fixed windows and calculating the query, key and value as follows: ; ; ; in, , and The query vector, key vector and value vector represent the local features respectively; , and The transformation matrices of the query vector, the key vector, and the value vector respectively represent the local features; Represents the local features obtained by dividing the fixed window of 3×3; The local attention is calculated as follows: ; in, Represents the feature output after weighting by the local attention mechanism; Represents the key vector The transpose of is used to calculate the similarity between the query vector and the key vector of the local feature; represents the scaling factor, which is equal to the dimensions of the query vector and the key vector; Step S222: Global attention is obtained by fusion of input features. This is done by performing global pooling and calculating the query, key, and value as follows: ; ; ; in, , and Represent the query vector, key vector and value vector of global features respectively; , and The transformation matrices of the query vector, the key vector, and the value vector respectively represent the global features; Represents the features obtained after global pooling; The global attention is calculated as follows: ; in, Represents the feature output after weighting by the global attention mechanism; Represents the key vector The transpose of is used to calculate the similarity between the query vector and the key vector of the global feature; Step S223: Fuse the features obtained by the local and global attention mechanisms to obtain an optimized feature representation: ; in, represents the optimized features; α is a learnable weight that balances the contribution of local and global features; the optimized features The input is sent to the decoder and combined with the upsampling operation to restore the image details and generate a denoised image.

[0013] Preferably, in step S3, based on the fusion network of the gating mechanism, the fusion ratio of the control signal to the generated signal is dynamically adjusted, and the generation process is adaptively optimized according to the characteristics of the food image and the control conditions. The specific process is as follows: Step S31, generate features by and control characteristics Add to get the initial fusion features , as shown below: ; Step S32: generating features and controlling features interact with each other through a multi-head attention mechanism; wherein the multi-head attention mechanism is used in each layer to allow the network to focus on different features in multiple subspaces in parallel; The update formula for generating features is as follows: ; in, It comes from the fusion feature Query; and are the keys and values ​​of the generated features; The update formula of the control feature is as follows: ; in, It comes from the fusion feature Query; and are the keys and values ​​of the control features; Step S33: Calculate the gating coefficient through the gating mechanism and ,These coefficients dynamically adjust the importance of generated features and control features in each layer; The gating coefficient determines the contribution of generated features and controlled features in different contexts as follows: ; ; in, Indicates The gating coefficients of the generated features of the layer; Indicates The weight matrix corresponding to the generated features in the layer; Indicates Generative characteristics of the layer; Indicates Control characteristics of the layer; Indicates The bias term of the generated features of the layer; Indicates The gating coefficients of the control features of the layer; Indicates The weight matrix corresponding to the control features in the layer; Indicates The bias term of the control feature of the layer; Step S34: Updated generated features and control characteristics Perform weighted summation and normalize through LayerNorm to obtain the final updated features as shown below: ; ; Step S35: concatenate the updated features to obtain the fused features , as shown below: ; Features after stitching The aggregation of features is further enhanced through the Softmax and tanh mechanisms, as shown below: ; in, represents the transpose of the learnable weight vector; Represents a weight matrix, which is used to fusion features Perform linear transformation; represents the bias term; Based on the above process, the generated features and control features interact and update effectively at each layer, and finally generate food images.

[0014] Preferably, in step S4, based on the dual-flow diffusion generation strategy, the noisy image is gradually denoised by a generation model of a food image generation architecture based on a dual-flow diffusion control model, and the objective function is trained. , as shown below: ; in, is the target image; is the image after noise processing; is the current time step; It is a text condition; is the control condition; is the noise term; is the noise in the model prediction.

[0015] Therefore, the present invention adopts the above-mentioned food image generation method based on the dual-flow diffusion control model, and the beneficial effects are as follows: (1) Fine control of the generation process: The present invention introduces a dual-stream interaction mechanism and a gating mechanism to optimize the features between the generation network and the control network, ensuring that the generation features and control features of the image are finely balanced, and the generated food images are more realistic and delicate. Compared with traditional methods, the method proposed in this aspect can capture the detailed features of food more accurately; (2) Distribution of backbone network and skip connection contributions: The adaptive scaling mechanism enables the Unet architecture to dynamically adjust the weights of backbone network features and skip connection features according to the specific needs of the generation process, and combines global and local attention mechanisms to further optimize the features in the network, so that it can dynamically focus on different important areas during the generation process, ensuring that the food images are overall coordinated and the details are realistic; (3) Meeting diverse food generation needs: The present invention supports multimodal input of text embedding and conditional images, which can guide the generation network to flexibly respond to different control requirements, so that the appearance, form and details of food images can be customized according to user-specified requirements; this multimodal guidance not only improves the flexibility of generation, but also ensures that the generation results are highly consistent with the user input conditions; (4) Excellent results verified by experiments: Through experimental verification on multiple food image datasets, the method proposed in this paper is significantly superior to the existing diffusion model and generative adversarial network method. In the generated food images, it performs well in capturing details, reproducing complex textures, and generating gloss, and the visual quality and control effect of the generated images are significantly improved.

[0016] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a schematic diagram of a dual-flow-based diffusion control generation framework of the present invention; Figure 2 yes Figure 1 Schematic diagram of the workflow of the crisscross attention-downsampling module, downsampling module, crisscross attention-intermediate sampling module, global-local attention-upsampling module, and crisscross-global-local attention-upsampling module; Figure 3 is a schematic diagram of the global and local attention modules of the present invention; Figure 4 It is a schematic diagram of the fusion module based on the dual-stream gating mechanism of the present invention. DETAILED DESCRIPTION

[0018] The technical solution of the present invention is further described below through the accompanying drawings and embodiments.

[0019] like Figure 1 As shown, a method for generating food images based on a dual-flow diffusion control model comprises the following steps: Step S1, constructing a food image generation architecture based on a dual-flow diffusion control model; Step S2, introducing an adaptive scaling mechanism and global and local attention modules to optimize the semantic information and detail texture in food images; Step S3: Based on the fusion network of the gating mechanism, dynamically adjust the fusion ratio of the control signal and the generated signal, and adaptively optimize the generation process according to the characteristics of the food image and the control conditions; Step S4: Based on the dual-stream diffusion generation strategy, the noisy image is gradually denoised through the generation model of the food image generation architecture based on the dual-stream diffusion control model to generate a high-quality image that meets the text description and control conditions.

[0020] Example Step S1, constructing a food image generation architecture based on a dual-flow diffusion control model.

[0021] like Figure 1 and Figure 2 As shown in the figure, the food image generation architecture based on the two-stream diffusion control model includes a CLIP encoder, a variational autoencoder, a temporal encoder, a control network based on a gating mechanism, an image generation network and a decoder.

[0022] The food image generation architecture based on the two-stream diffusion control model combines the image generation network with the control network to generate food images in an end-to-end manner, improving the quality and control accuracy of the generated images. The input includes text, time steps, conditional images, and Gaussian noise. The time steps are processed by the temporal encoder, and the text is processed by the CLIP encoder as an additional condition to further guide the generation process through the cross-attention mechanism.

[0023] The control network based on the gating mechanism takes the conditional image as input and extracts features related to food shape and spatial information through the image annotation module. The image generation network works in parallel with the control network, and their interaction is dynamically controlled by the gating mechanism and then fused, which selectively determines the flow of features to subsequent stages.

[0024] In addition, in the food image generation architecture based on the two-stream diffusion control model, an adaptive scaling mechanism is introduced to adjust the feature allocation of the backbone network and the jump connection, and the global and local attention modules are introduced into each stage of the decoder to further focus on the semantic information and detailed texture in the food image. As the features propagate through each network layer, the predicted noise is obtained. After t time steps, denoising is completed to obtain the generated food image.

[0025] Step S2: Introduce an adaptive scaling mechanism and global and local attention modules to optimize the semantic information and detail texture in food images.

[0026] Step S21: introducing an adaptive scaling mechanism to adjust the feature allocation of the backbone network and the skip connection.

[0027] Step S211: Input noise After being processed by the U-Net encoder, multi-scale features are extracted ;in, is the batch size, is the number of channels, and They represent the spatial size of the feature map respectively.

[0028] The task of generating food images requires both capturing overall information and retaining local detail textures. Therefore, the feature allocation of the backbone network and the skip connection needs to be optimized to ensure a balance between global consistency and detail expression. To optimize this allocation, the backbone features are first enhanced, and then the skip connection features are adjusted in the frequency domain to optimize information transfer.

[0029] Step S212: Enhance the main features.

[0030] Main features After channel-level scaling, the global information expression capability is improved. Channel-level scaling is achieved through learnable parameters. Adjustments are made to enhance the global consistency of the backbone network so that it has a stronger contribution in the final feature fusion, as shown below: ; in, Learnable channel-level scaling parameters for backbone features; Acting on each channel, the features are scaled differently on different channels.

[0031] This adjustment ensures that the backbone features are more expressive, thereby playing a stable global control role in the skip connection and decoding stages.

[0032] Step S213: perform frequency domain adjustment on the skip connection feature to optimize the skip connection feature.

[0033] Skip connection features Directly passed from the encoder to the decoder, the unprocessed skip connection features may carry too much high-frequency information, thereby introducing noise and affecting the generation quality. Therefore, the skip connection features are adjusted in the frequency domain to optimize information transfer.

[0034] (1) First, the skip connection features are converted to the frequency domain to obtain the frequency domain representation: ; in, Represents Fourier transform, which converts the features of the spatial domain into the frequency domain so that different frequency components can be processed separately.

[0035] (2) In order to distinguish high-frequency and low-frequency information, the spectral energy distribution of the skip connection feature is calculated , which measures the contribution of different frequency components, as follows: ; in, Indicates the radius The total energy of the frequency components within , indicating the proportion of information in this frequency range.

[0036] In order to dynamically determine the dividing point between low frequency and high frequency , set the cumulative energy ratio threshold , as shown below: ; in, It controls the energy proportion of the low-frequency part, ensuring that the model can adaptively adjust the division point between low-frequency and high-frequency parts according to different input images to meet the detail requirements of different food images.

[0037] (3) In order to dynamically scale low-frequency and high-frequency features, a frequency mask is introduced , as shown below: ; in, is the learnable channel-level scaling parameter of the jump connection feature, and the jump connection feature after frequency mask adjustment is as follows: ; (4) Subsequently, the adjusted frequency domain features are converted back to the spatial domain through inverse Fourier transform as follows: ; This step ensures that the skip connection features can be restored to the spatial domain after being adjusted in the frequency domain, and carry the optimized low-frequency and high-frequency information to enhance the quality of the generated image.

[0038] Step S214: combine the backbone features and the skip connection features and , and concatenate them in the channel dimension to obtain the fusion features, as shown below: ; This splicing operation ensures that the global consistency of the backbone features and the local detail information of the jump connection features can be fully integrated to improve the quality of the final generated image.

[0039] Step S22: adding global and local attention modules to each stage of the decoder to further focus on the semantic information and detail texture in the food image.

[0040] Since global features contain the semantic information of the whole food, and local features contain the surface texture and details of the food, in the decoder stage, global-local attention is used to further optimize the expression of global and local information, such as Figure 3 shown.

[0041] Step S221: local attention takes the input fusion features It is implemented by dividing into 3×3 fixed windows and calculating the query, key and value as follows: ; ; ; in, , and Represent the query vector, key vector, and value vector of local features respectively; , and The transformation matrices of the query vector, the key vector, and the value vector respectively represent the local features; Represents the local features obtained by dividing the fixed window of 3×3; the calculation method of local attention is as follows: ; in, Represents the feature output after weighting by the local attention mechanism; Represents the key vector The transpose of is used to calculate the similarity between the query vector and the key vector of the local feature; represents the scaling factor, which is equal to the dimensions of the query vector and the key vector; Step S222: Global attention is obtained by fusion of input features. This is done by performing global pooling and calculating the query, key, and value as follows: ; ; ; in, , and Represent the query vector, key vector and value vector of global features respectively; , and The transformation matrices of the query vector, the key vector, and the value vector respectively represent the global features; Represents the features obtained after global pooling; The global attention is calculated as follows: ; in, Represents the feature output after weighting by the global attention mechanism; Represents the key vector The transpose of is used to calculate the similarity between the query vector and the key vector of the global feature.

[0042] Step S223, finally, the features obtained by the local and global attention mechanisms are fused to obtain an optimized feature representation: ; in, represents the optimized features; α is a learnable weight that balances the contribution of local and global features.

[0043] Optimized features The image is further input into the decoder and combined with the upsampling operation to gradually restore the image details, and finally generate a denoised image, ensuring that the generated food image can retain details while maintaining global consistency.

[0044] Through the adaptive scaling mechanism, the contribution of the backbone network and the jump connection is balanced during the generation process, so that the generation network can process global information and local details at the same time. This mechanism ensures that the generated food images pay more attention to detail processing while maintaining global consistency, especially in terms of local features such as texture, gloss and layering of food. In addition, by combining the global and local attention mechanisms, the network can focus on global structure and local details according to the importance of different regions, further improving the realism and visual expression of food images. Compared with the standard U-Net, this method improves the generation quality and makes the processing details of food images more realistic, while keeping the overall color and morphology stable.

[0045] Step S3: Based on the fusion network of the gating mechanism, the fusion ratio of the control signal and the generated signal is dynamically adjusted, and the generation process is adaptively optimized according to the characteristics of the food image and the control conditions.

[0046] In the food image generation task based on the two-stream diffusion control model network, the interaction between the generated features and the controlled features is crucial. The generated features represent the feature representation of the existing food images, while the controlled features are the conditional information that guides the generation process. By interacting these two types of features, more meaningful controls can be added to the generation process to enhance the diversity and accuracy of the generated images. The purpose of the two-stream interaction is to use the relationship between the generated features and the controlled features to ensure that the generation process not only depends on the existing image features, but can also be accurately guided by the controlled features, so as to generate food images that meet various requirements, such as Figure 4 shown.

[0047] Step S31, first, by generating features and control characteristics Add to get the initial fusion features , as shown below: ; Step S32: Next, the generated features and control features will interact through the multi-head attention mechanism.

[0048] At each layer, the use of a multi-head attention mechanism allows the network to focus on different features in multiple subspaces in parallel, thereby enhancing the richness of the interaction. The update formula for generating features is as follows: ; in, It comes from the fusion feature Query; and are the keys and values ​​of the generated features. Through this formula, the generated features can interact with the control features in a fine-grained manner at each layer, enhancing the model generation capability.

[0049] The control features are also calculated through multi-head attention to update the control features, as shown below: ; in, It comes from the fusion feature Query; and are the keys and values ​​that control the characteristics.

[0050] The control features and generation features are updated at this layer, thus achieving cross-modal information exchange.

[0051] Step S33: Calculate the gating coefficient through the gating mechanism and , these coefficients dynamically adjust the importance of generated features and control features in each layer.

[0052] The gating coefficients help the network decide the contribution of generated features and control features in different contexts, as shown below: ; ; in, Indicates The gating coefficients of the generated features of the layer; Indicates The weight matrix corresponding to the generated features in the layer; Indicates Generative characteristics of the layer; Indicates Control characteristics of the layer; Indicates The bias term of the generated features of the layer; Indicates The gating coefficients of the control features of the layer; Indicates The weight matrix corresponding to the control features in the layer; Indicates The bias term for the control features of the layer.

[0053] These gating coefficients adjust the contribution ratio of the generated features and the control features according to the combination of the current layer. This step enables the generation process to flexibly weight each feature according to different conditions of the input at different levels, thereby improving the generated image quality.

[0054] Step S34: Updated generated features and control characteristics The weighted sum is then normalized by LayerNorm to obtain the final updated fusion features, as shown below: ; ; The purpose of this step is to ensure that the generated features and the control features are reasonably weighted in each layer so that the final fusion features can reflect the best combination of the two modal information.

[0055] Step S35: splice the updated features to obtain fused features , as shown below: ; Through splicing, the expression of generated features and control features can be processed uniformly, making the next step of feature update more effective. The update will continue, and the Softmax and tanh mechanisms will be used to further enhance the aggregation of features, as shown below: ; in, represents the transpose of a learnable weight vector; Represents a weight matrix, which is used to fusion features Perform linear transformation; Represents the bias term, a learnable parameter.

[0056] This step effectively integrates the generated features and the control features in a weighted manner, further improving the quality of the information flow and enabling the network to more accurately generate images that meet the requirements.

[0057] Through these steps, the generated features and control features can effectively interact and update between each layer, and finally generate food images with higher quality and diversity. When the detail area needs to be emphasized, the gating mechanism will enhance the effect of the control signal to ensure that the details such as texture are more finely adjusted, while in other areas, the control signal will be appropriately weakened to avoid excessively affecting the image generation and maintain the naturalness and realism of the generated results.

[0058] In addition, the gating mechanism further improves the quality of image generation by dynamically adjusting the feature flow between the generation network and the control network. Through the gating mechanism, the model can automatically and selectively pass and enhance specific features according to the different needs of each generation stage, avoiding the problem of information flow delay and ensuring the simultaneous optimization of image details and structure.

[0059] Step S4: Based on the dual-stream diffusion generation strategy, the noisy image is gradually denoised through the generation model of the food image generation architecture based on the dual-stream diffusion control model to generate a high-quality image that meets the text description and control conditions.

[0060] During the generation process, the control network receives not only textual conditions, but also additional image inputs to further guide the details of the generated image. In this way, the generated image not only conforms to the text description, but also reflects more control over the details, such as the appearance, color, and texture of the food.

[0061] Specific training objective function , as shown below: ; in, is the target image; is the image after noise processing; is the current time step; It is a text condition; is the control condition; is the noise term; is the noise in the model prediction.

[0062] During training, the model optimizes the generation process by minimizing the difference between predicted noise and actual noise, resulting in clear and detailed food images.

[0063] Therefore, the present invention adopts the above-mentioned food image generation method based on the two-stream diffusion control model, and significantly improves the quality and detail capture ability of generated food images by introducing two-stream interaction mechanism, gating mechanism, adaptive scaling mechanism and attention mechanism; by increasing the two-way information flow between the generation network and the control network on the basis of the diffusion model, and realizing feature transfer through the zero convolution layer, the problem of balancing global structure and local details in food image generation is effectively solved.

[0064] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A food image generation method based on a dual-flow diffusion control model, characterized in that: The following steps are involved: Step S1, constructing a food image generation architecture based on a dual-flow diffusion control model; Step S2, introducing an adaptive scaling mechanism and global and local attention modules to optimize the semantic information and detail texture in food images; Step S3: Based on the fusion network of the gating mechanism, dynamically adjust the fusion ratio of the control signal and the generated signal, and adaptively optimize the generation process according to the characteristics of the food image and the control conditions; Step S4: Based on the dual-stream diffusion generation strategy, the noisy image is gradually denoised through the generation model of the food image generation architecture based on the dual-stream diffusion control model to generate a high-quality image that meets the text description and control conditions.

2. The method for generating food images based on a dual-flow diffusion control model according to claim 1, characterized in that: In step S1, a food image generation architecture based on a two-stream diffusion control model includes a CLIP encoder, a variational autoencoder, a temporal encoder, a control network based on a gating mechanism, an image generation network, and a decoder; The food image generation architecture based on the two-stream diffusion control model combines the image generation network with the control network to generate food images in an end-to-end manner.

3. The method for generating food images based on a dual-flow diffusion control model according to claim 2, characterized in that: The input of the food image generation architecture based on the two-stream diffusion control model includes text, time steps, conditional images and Gaussian noise; the time steps are processed by the temporal encoder, and the text is processed by the CLIP encoder, and the generation process is further guided by the cross-attention mechanism; The control network based on the gating mechanism takes the conditional image as input and extracts features related to the shape and spatial information of food through the image annotation module; The image generation network works in parallel with the control network, and their interaction is dynamically controlled and fused by a gating mechanism that selectively determines the flow of features to subsequent stages.

4. The method for generating food images based on a dual-flow diffusion control model according to claim 3, characterized in that: In the food image generation architecture based on the two-stream diffusion control model, an adaptive scaling mechanism is introduced to adjust the feature allocation of the backbone network and the skip connection; Global and local attention modules are introduced into each stage of the decoder to further focus on the semantic information and detailed textures in food images.

5. The method for generating food images based on a dual-flow diffusion control model according to claim 1, characterized in that: In step S2, an adaptive scaling mechanism is introduced to adjust the feature allocation of the backbone network and the skip connection. The specific process is as follows: Step S211: Input noise After being processed by the U-Net encoder, multi-scale features are extracted ;in, is the batch size, is the number of channels, and Respectively represent the spatial size of the feature map; Step S212, enhancing the main features; Main features After channel-level scaling, channel-level scaling is achieved through learnable parameters Adjustments are made to enhance the global consistency of the backbone network as follows: ； in, Learnable channel-level scaling parameters for backbone features; Acting on each channel, the features are scaled differently on different channels; Step S213: performing frequency domain adjustment on the skip connection feature to optimize the skip connection feature; Step S214: Concatenate the optimized backbone features and skip connection features in the channel dimension to obtain fusion features, as shown below: ； in, Indicates fusion features; and They represent the optimized backbone features and skip connection features respectively.

6. The method for generating food images based on a dual-flow diffusion control model according to claim 5, characterized in that: In step S213, the skip connection feature Directly passed from the encoder to the decoder, frequency domain adjustment is performed to optimize information transmission. The specific process is as follows: (1) First, the skip connection features are converted to the frequency domain to obtain the frequency domain representation: ； in, represents Fourier transform, which converts the features of spatial domain into frequency domain so that different frequency components can be processed separately; (2) In order to distinguish high-frequency and low-frequency information, the spectral energy distribution of the skip connection feature is calculated , which measures the contribution of different frequency components, as follows: ； in, Indicates the radius The total energy of the frequency components within represents the proportion of information within this frequency range; In order to dynamically determine the dividing point between low frequency and high frequency , set the cumulative energy ratio threshold , as shown below: ； in, Indicates the energy proportion of controlling the low-frequency part; (3) In order to dynamically scale low-frequency and high-frequency features, a frequency mask is introduced , as shown below: ； in, is the learnable channel-level scaling parameter of the skip connection feature, and the skip connection feature after frequency mask adjustment , as shown below: ； (4) The adjusted frequency domain features are converted back to the spatial domain through inverse Fourier transform as shown below: 。 7. The method for generating food images based on a dual-flow diffusion control model according to claim 1, characterized in that: In step S2, global and local attention modules are added to each stage of the decoder to further focus on the semantic information and detail texture in the food image. The specific process is as follows: Step S221: local attention takes the input fusion features It is implemented by dividing into 3×3 fixed windows and calculating the query, key and value as follows: ； ； ； in, , and The query vector, key vector and value vector represent the local features respectively; , and The transformation matrices of the query vector, the key vector, and the value vector respectively represent the local features; Represents the local features obtained by dividing the fixed window of 3×3; The local attention is calculated as follows: ； in, Represents the feature output after weighting by the local attention mechanism; Represents the key vector The transpose of is used to calculate the similarity between the query vector and the key vector of the local feature; represents the scaling factor, which is equal to the dimensions of the query vector and the key vector; Step S222: Global attention is obtained by fusion of input features. This is done by performing global pooling and calculating the query, key, and value as follows: ； ； ； in, , and Represent the query vector, key vector and value vector of global features respectively; , and The transformation matrices of the query vector, the key vector, and the value vector respectively represent the global features; Represents the features obtained after global pooling; The global attention is calculated as follows: ； in, Represents the feature output after weighting by the global attention mechanism; Represents the key vector The transpose of is used to calculate the similarity between the query vector and the key vector of the global feature; Step S223: Fuse the features obtained by the local and global attention mechanisms to obtain an optimized feature representation: ； in, represents the optimized features; α is a learnable weight that balances the contribution of local and global features; the optimized features The input is sent to the decoder and combined with the upsampling operation to restore the image details and generate a denoised image.

8. The method for generating food images based on a dual-flow diffusion control model according to claim 1, characterized in that: In step S3, based on the fusion network of the gating mechanism, the fusion ratio of the control signal to the generated signal is dynamically adjusted, and the generation process is adaptively optimized according to the characteristics of the food image and the control conditions. The specific process is as follows: Step S31, generate features by and control characteristics Add to get the initial fusion features , as shown below: ； Step S32: generating features and controlling features interact with each other through a multi-head attention mechanism; wherein the multi-head attention mechanism is used in each layer to allow the network to focus on different features in multiple subspaces in parallel; The update formula for generating features is as follows: ； in, It comes from the fusion feature Query; and are the keys and values ​​of the generated features; The update formula of the control feature is as follows: ； in, It comes from the fusion feature Query; and are the keys and values ​​of the control features; Step S33: Calculate the gating coefficient through the gating mechanism and ,These coefficients dynamically adjust the importance of generated features and control features in each layer; The gating coefficient determines the contribution of generated features and controlled features in different contexts as follows: ； ； in, Indicates The gating coefficients of the generated features of the layer; Indicates The weight matrix corresponding to the generated features in the layer; Indicates Generative characteristics of layers; Indicates Control characteristics of the layer; Indicates The bias term of the generated features of the layer; Indicates The gating coefficients of the control features of the layer; Indicates The weight matrix corresponding to the control features in the layer; Indicates The bias term of the control feature of the layer; Step S34: Updated generated features and control characteristics Perform weighted summation and normalize through LayerNorm to obtain the final updated features as shown below: ； ； Step S35: concatenate the updated features to obtain the fused features , as shown below: ； Features after stitching The aggregation of features is further enhanced through the Softmax and tanh mechanisms, as shown below: ； in, represents the transpose of the learnable weight vector; Represents a weight matrix, which is used to fusion features Perform linear transformation; represents the bias term; Based on the above process, the generated features and control features interact and update effectively at each layer, and finally generate food images.

9. The method for generating food images based on a dual-flow diffusion control model according to claim 1, characterized in that: In step S4, based on the two-stream diffusion generation strategy, the noisy image is gradually denoised through the generation model of the food image generation architecture based on the two-stream diffusion control model, and the objective function is trained. , as shown below: ； in, is the target image; is the image after noise processing; is the current time step; It is a text condition; is the control condition; is the noise term; is the noise in the model prediction.

Images