A recipe generation method and system based on graph neural networks and adversarial training

Abstract

This application belongs to the field of computer technology, specifically disclosing a recipe generation method and system based on graph neural networks and adversarial training. The method includes: receiving target input, whereby the target input indicates the ingredients, flavor requirements, and nutritional requirements of the recipe; obtaining complete ingredients for the recipe based on the ingredients indicated by the target input through an ingredient completion model; obtaining ingredient description vectors, flavor description vectors, and nutritional description vectors through encoding processing based on the complete ingredients, flavor requirements, and nutritional requirements of the recipe; and obtaining a target recipe based on each description vector through a recipe generation model. The ingredient completion model is obtained by training a graph neural network using masked ingredient samples and corresponding labels, and the generator, after adversarial training, serves as the recipe generation model. This application enables the generation of recipes that meet user requirements based on incomplete ingredients provided by the user.

Description

Technical Field

[0001] This application belongs to the field of computer technology, and more specifically, relates to a recipe generation method and system based on graph neural networks and adversarial training. Background Technology

[0002] Existing recipe generation technologies typically rely on complete ingredient lists or fixed pairing patterns. However, in real-world applications, users often provide only limited and incomplete ingredients, while also having specific, personalized requirements regarding the recipe's flavor (e.g., saltiness, spiciness) and nutritional composition (e.g., calorie, protein content). Existing methods struggle to effectively handle this complex generation task, which combines partial ingredient input with multidimensional user constraints (flavor and nutrition), resulting in recipes that are deficient in terms of ingredient rationality, flavor compatibility, or nutritional balance. Therefore, how to generate recipes that meet user requirements based on incomplete user-provided ingredients is a pressing technical problem that needs to be solved in this field. Summary of the Invention

[0003] In view of the shortcomings of the existing technology, the purpose of this application is to realize the generation of recipes that meet the user's requirements based on incomplete ingredients provided by the user.

[0004] To achieve the above objectives, in a first aspect, this application provides a recipe generation method based on graph neural networks and adversarial training, comprising:

[0005] Receive target input, which indicates the ingredients, flavor requirements, and nutritional requirements of the recipe;

[0006] Based on the ingredients indicated by the target input, the complete ingredients of the recipe are obtained through the ingredient completion model;

[0007] Based on the complete ingredients of the recipe, the flavor requirements of the recipe, and the nutritional requirements of the recipe, the ingredient description vector, flavor description vector, and nutritional description vector are obtained through encoding processing.

[0008] Based on ingredient description vectors, flavor description vectors, and nutritional description vectors, the target recipe is obtained through a recipe generation model.

[0009] Among them, the ingredient completion model predicts the complete ingredient node set based on a partial ingredient node set, where each node in the ingredient node set corresponds to an ingredient; the ingredient completion model is obtained by training a graph neural network based on masked ingredient samples and corresponding labels. The masked ingredient samples are obtained by generating a complete ingredient node set based on the complete ingredients in the recipe sample and masking some of the ingredient nodes in the complete ingredient node set; the labels corresponding to the masked ingredient samples are obtained by generating a complete ingredient node set based on the complete ingredients in the recipe sample.

[0010] The recipe generation model is obtained by training an adversarial generative network based on recipe samples. The adversarial generative network includes a generator and a discriminator. The generator is used to generate recipes based on ingredient description vectors, flavor description vectors, and nutritional description vectors. The discriminator is used to distinguish the authenticity of recipes. The generator trained adversarially serves as the recipe generation model.

[0011] In one possible implementation, the input to the graph neural network is a vectorized representation of a partial set of food node values. , , express The Middle A vector of nodes;

[0012] The output of a graph neural network is a vectorized representation of the complete set of food node sets. , , express The Middle A vector of nodes;

[0013] Graph neural networks include: a dynamic edge learning module and a graph attention module with edge awareness;

[0014] The dynamic edge learning module includes: a KNN candidate edge construction unit and an edge probability prediction unit;

[0015] The KNN candidate edge building unit is used to determine the set of candidate edges using the following formula:

[0016] ;

[0017] in, Denotes the set of candidate edges. This represents the K-nearest neighbor algorithm. Represents the nearest neighbor number. for One of the candidate edges in the middle, This represents the first element in the set of partial food ingredient nodes. The node and the first Candidate edges between nodes;

[0018] The edge probability prediction unit is determined by the following formula. Corresponding edge probability :

[0019] ;

[0020] ;

[0021] ;

[0022] The edge probability prediction unit includes a vector concatenation layer, a multilayer perceptron, and an edge probability prediction layer. and This is the input to the vector concatenation layer. This is the output of the vector concatenation layer; Indicates to and Perform vector concatenation operation; This indicates the number of layers in the neural network of a multilayer perceptron; For the first multilayer perceptron Input to a layered neural network; and For the first multilayer perceptron Weights and biases of a multilayer neural network; For the first multilayer perceptron The output of a layered neural network; Indicates the activation function; This is the input to the edge probability prediction layer. This is the output of the edge probability prediction layer; and The weights and biases of the edge probability prediction layer, Represents the Sigmoid function;

[0023] The edge-aware graph attention module includes: a feature transformation unit, a weighted adjacency matrix construction unit, an attention coefficient calculation unit, an attention weight normalization unit, and a feature aggregation and residual connection unit;

[0024] The feature transformation unit is used to determine the initial node representation using the following formula:

[0025] ;

[0026] in, This represents the initial node representation. This indicates the data batch normalization layer. Indicates the activation function; and The weights and biases of the feature variation units;

[0027] The weighted adjacency matrix construction unit is used to determine the weighted adjacency matrix using the following formula. :

[0028] ;

[0029] in, express The Middle Line number Column elements;

[0030] The attention coefficient calculation unit is used to determine the first attention coefficient using the following formula. The node and the first Attention coefficient between nodes :

[0031] ;

[0032] in, To correct the linear activation function, The weight vector representing the attention mechanism. and For generating the weight matrix of queries and keys, for The Middle OK, for The Middle OK, This is the default value; In ;

[0033] Attention weight normalization unit is used for Perform a normalization operation to obtain the normalized attention coefficients. ;

[0034] Feature aggregation and residual connection units are determined by the following formula :

[0035] ;

[0036] ;

[0037] in, For the first The latent representation vector of each node The weight matrix used to generate the values. and These are the weights and biases used to adjust the output.

[0038] In one possible implementation, the generator includes: a multi-source feature fusion module and a text generation module;

[0039] The multi-source feature fusion module is used to output a fused conditional context vector based on the input ingredient description vector, flavor description vector, and nutritional description vector through vector concatenation, linear projection layer, and stacked Transformer encoder layers.

[0040] The text generation module is used to output the text sequence of the target recipe based on the conditional context vector and through the text generation model;

[0041] The discriminator includes: a text encoding module and a classification decision module;

[0042] The text encoding module is used to output a text aggregation representation vector based on the input recipe text sequence through the embedding layer of a pre-trained language model and stacked Transformer encoder layers;

[0043] The classification decision module is used to output the probability that a recipe text sequence is a real recipe based on the text aggregation representation vector, through the processing of the multilayer perceptron classification head and the sigmoid function.

[0044] In one possible implementation, the loss function used to train the graph neural network... as follows:

[0045] ;

[0046] in, This represents the adjacency matrix reconstruction loss. This represents the KL divergence loss. Indicates the balance coefficient;

[0047] ;

[0048] in, The total number of nodes represents the vectorized representation of the complete set of food node output by the graph neural network; Number the nodes; Represents the actual adjacency matrix elements; Represents the Sigmoid function; The vectorized representation of the complete set of food node output by the graph neural network. A vector of nodes, The vectorized representation of the complete set of food node output by the graph neural network. A vector of nodes;

[0049] ;

[0050] in, Indicates the first The latent representation vector of each node The distribution mean, Indicates the first The latent representation vector of each node The distribution variance.

[0051] In one possible implementation, prior to adversarial training, the generator and discriminator are pre-trained as follows:

[0052] Based on the generator pre-training loss function and the ingredient description vector, flavor description vector and nutritional description vector corresponding to the first batch of recipe samples, the generator is pre-trained to obtain the pre-trained generator.

[0053] Obtain a second batch of recipe samples to serve as the actual recipe samples;

[0054] The ingredient description vector, flavor description vector, and nutritional description vector corresponding to the second batch of recipe samples are input into the pre-trained generator, and the recipes generated by the pre-trained generator are used as pseudo recipe samples.

[0055] Based on the discriminator pre-training loss function, real recipe samples, and pseudo recipe samples, the discriminator is pre-trained to obtain the pre-trained discriminator;

[0056] Among them, the generator pre-training loss function It is determined by the following formula:

[0057] ;

[0058] This indicates the number of the first batch of recipe samples; The first batch of recipe samples A text sequence of recipe samples; yes The corresponding ingredient description vector, flavor description vector, and nutritional description vector; defined. For generators The generated text sequence of recipes; express The number of lexical units; express The Each word element, express Center front Each word element;

[0059] The parameter is The generator, in a given And it has already been generated Then, predict whether the next lexical unit is a real lexical unit. The probability of;

[0060] Discriminator pre-training loss function It is determined by the following formula:

[0061] ;

[0062] Indicates the number of recipe samples in the second batch; The first in the second batch of recipe samples A text sequence of recipe samples; definition yes The corresponding ingredient description vector, flavor description vector, and nutritional description vector; For the pre-trained generator The generated text sequence of recipes; This represents the probability estimate by which the discriminator identifies the input recipe text sequence as a genuine recipe.

[0063] In one possible implementation, adversarial training includes:

[0064] Some ingredients in the third batch of recipe samples are deleted. The remaining ingredients are used to construct a set of ingredient nodes and input into the ingredient completion model. The complete ingredients output by the ingredient completion model are used to replace the ingredients in the third batch of recipe samples to obtain the updated third batch of recipe samples.

[0065] Based on the adversarial training loss function and the updated third batch of recipe samples, the generator and discriminator are trained to obtain the recipe generation model;

[0066] The adversarial training loss function includes the loss function used to train the generator. and the loss function used to train the discriminator ;

[0067] It is determined by the following formula:

[0068] ;

[0069] For cross-entropy loss, To combat the losses, This is the balance coefficient;

[0070] ;

[0071] This indicates the number of recipe samples in the third batch; The third batch of recipe samples A text sequence of a recipe sample. Through the The text sequence of the updated recipe sample obtained by partially deleting ingredients and then completing them using an ingredient completion model; yes The corresponding ingredient description vector, flavor description vector, and nutritional description vector; defined. For generators The generated text sequence of recipes; express The number of lexical units; express The Each word element, express Center front Each word element;

[0072] The parameter is The generator, in a given And it has already been generated Then, predict whether the next lexical unit is a real lexical unit. The probability of;

[0073] ;

[0074] It is determined by the following formula:

[0075] ;

[0076] This represents the probability estimate by which the discriminator identifies the input recipe text sequence as a genuine recipe.

[0077] In one possible implementation, the target input is received, including:

[0078] Audio data is collected via the microphone of the smart glasses;

[0079] Based on the collected audio data, text data is obtained through speech-to-text conversion;

[0080] Semantic analysis of text data is performed using a large language model to obtain semantic analysis results;

[0081] If the semantic analysis results include flavor and nutritional requirements, and the semantic analysis results indicate that recipes should be generated for ingredients within the field of view of the smart glasses according to the flavor and nutritional requirements, the flavor and nutritional requirements included in the semantic analysis results are used as the target inputs of the indicated flavor and nutritional requirements, and images are captured by the camera of the smart glasses.

[0082] Based on the acquired images, image recognition is performed, and the ingredients identified in the images are used as the target input to indicate the ingredients.

[0083] Secondly, this application provides a recipe generation system based on graph neural networks and adversarial training, comprising:

[0084] The interaction module is used to receive target input, which indicates the ingredients, flavor requirements, and nutritional requirements of the recipe.

[0085] The ingredient completion module is used to obtain the complete ingredients of a recipe based on the ingredients indicated by the target input, through an ingredient completion model;

[0086] The vectorization module is used to obtain ingredient description vectors, flavor description vectors, and nutritional description vectors based on the complete ingredients, flavor requirements, and nutritional requirements of the recipe through encoding processing.

[0087] The recipe generation module is used to obtain target recipes based on ingredient description vectors, flavor description vectors, and nutritional description vectors through a recipe generation model.

[0088] Among them, the ingredient completion model predicts the complete ingredient node set based on a partial ingredient node set, where each node in the ingredient node set corresponds to an ingredient; the ingredient completion model is obtained by training a graph neural network based on masked ingredient samples and corresponding labels. The masked ingredient samples are obtained by generating a complete ingredient node set based on the complete ingredients in the recipe sample and masking some of the ingredient nodes in the complete ingredient node set; the labels corresponding to the masked ingredient samples are obtained by generating a complete ingredient node set based on the complete ingredients in the recipe sample.

[0089] The recipe generation model is obtained by training an adversarial generative network based on recipe samples. The adversarial generative network includes a generator and a discriminator. The generator is used to generate recipes based on ingredient description vectors, flavor description vectors, and nutritional description vectors. The discriminator is used to distinguish the authenticity of recipes. The generator trained adversarially serves as the recipe generation model.

[0090] In one possible implementation, the input to the graph neural network is a vectorized representation of a partial set of food node values. , , express The Middle A vector of nodes;

[0091] The output of a graph neural network is a vectorized representation of the complete set of food node sets. , , express The Middle A vector of nodes;

[0092] Graph neural networks include: a dynamic edge learning module and a graph attention module with edge awareness;

[0093] The dynamic edge learning module includes: a KNN candidate edge construction unit and an edge probability prediction unit;

[0094] The KNN candidate edge building unit is used to determine the set of candidate edges using the following formula:

[0095] ;

[0096] in, Denotes the set of candidate edges. This represents the K-nearest neighbor algorithm. Represents the nearest neighbor number. for One of the candidate edges in the middle, This represents the first element in the set of partial food ingredient nodes. The node and the first Candidate edges between nodes;

[0097] The edge probability prediction unit is determined by the following formula. Corresponding edge probability :

[0098] ;

[0099] ;

[0100] ;

[0101] The edge probability prediction unit includes a vector concatenation layer, a multilayer perceptron, and an edge probability prediction layer. and This is the input to the vector concatenation layer. This is the output of the vector concatenation layer; Indicates to and Perform vector concatenation operation; This indicates the number of layers in the neural network of a multilayer perceptron; For the first multilayer perceptron Input to a layered neural network; and For the first multilayer perceptron Weights and biases of a multilayer neural network; For the first multilayer perceptron The output of a layered neural network; Indicates the activation function; This is the input to the edge probability prediction layer. This is the output of the edge probability prediction layer; and The weights and biases of the edge probability prediction layer, Represents the Sigmoid function;

[0102] The edge-aware graph attention module includes: a feature transformation unit, a weighted adjacency matrix construction unit, an attention coefficient calculation unit, an attention weight normalization unit, and a feature aggregation and residual connection unit;

[0103] The feature transformation unit is used to determine the initial node representation using the following formula:

[0104] ;

[0105] in, This represents the initial node representation. This indicates the data batch normalization layer. Indicates the activation function; and The weights and biases of the feature variation units;

[0106] The weighted adjacency matrix construction unit is used to determine the weighted adjacency matrix using the following formula. :

[0107] ;

[0108] in, express The Middle Line number Column elements;

[0109] The attention coefficient calculation unit is used to determine the first attention coefficient using the following formula. The node and the first Attention coefficient between nodes :

[0110] ;

[0111] in, To correct the linear activation function, The weight vector representing the attention mechanism. and For generating the weight matrix of queries and keys, for The Middle OK, for The Middle OK, This is the default value; In ;

[0112] Attention weight normalization unit is used for Perform a normalization operation to obtain the normalized attention coefficients. ;

[0113] Feature aggregation and residual connection units are determined by the following formula :

[0114] ;

[0115] ;

[0116] in, For the first The latent representation vector of each node The weight matrix used to generate the values. and These are the weights and biases used to adjust the output.

[0117] In one possible implementation, the generator includes: a multi-source feature fusion module and a text generation module;

[0118] The multi-source feature fusion module is used to output a fused conditional context vector based on the input ingredient description vector, flavor description vector, and nutritional description vector through vector concatenation, linear projection layer, and stacked Transformer encoder layers.

[0119] The text generation module is used to output the text sequence of the target recipe based on the conditional context vector and through the text generation model;

[0120] The discriminator includes: a text encoding module and a classification decision module;

[0121] The text encoding module is used to output a text aggregation representation vector based on the input recipe text sequence through the embedding layer of a pre-trained language model and stacked Transformer encoder layers;

[0122] The classification decision module is used to output the probability that a recipe text sequence is a real recipe based on the text aggregation representation vector, through the processing of the multilayer perceptron classification head and the sigmoid function.

[0123] Overall, the technical solutions conceived in this application have the following beneficial effects compared with the prior art:

[0124] The system receives a target input containing partial ingredients, flavor requirements, and nutritional requirements. Then, using a graph neural network (GNN) model trained on masked ingredient samples, it completes the set of partial ingredient nodes corresponding to the target input into a complete set of ingredient nodes. Next, it encodes the complete ingredients, flavor requirements, and nutritional requirements to obtain ingredient description vectors, flavor description vectors, and nutritional description vectors, respectively. Finally, these description vectors are input into a recipe generation model (using a generator from an adversarially trained GAN, capable of generating a complete recipe under multi-source description vector constraints). The recipe generation model can then generate a complete recipe that meets the user's flavor and nutritional requirements. Attached Figure Description

[0125] Figure 1This is a flowchart illustrating the recipe generation method based on graph neural networks and adversarial training provided in an embodiment of this application.

[0126] Figure 2 This is a schematic diagram of the graph neural network structure provided in the embodiments of this application;

[0127] Figure 3 This is a schematic diagram of the generator provided in the embodiments of this application;

[0128] Figure 4 This is a schematic diagram of the discriminator provided in the embodiments of this application;

[0129] Figure 5 This is a schematic diagram of the structure of the recipe generation system based on graph neural network and adversarial training provided in the embodiments of this application;

[0130] Figure 6 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0131] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0132] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0133] In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more, for example, multiple processing units means two or more processing units, multiple elements means two or more elements, etc.

[0134] The embodiments of this application are described below with reference to the accompanying drawings.

[0135] Figure 1 This is a flowchart illustrating the recipe generation method based on graph neural networks and adversarial training provided in this application embodiment, as shown below. Figure 1 As shown, the method includes the following steps S101 to S104.

[0136] Step S101: Receive target input. The target input is used to indicate the ingredients of the recipe (generally some of the ingredients of the recipe), the flavor requirements of the recipe, and the nutritional requirements of the recipe (the flavor requirements consist of one or more flavor indicators, and the nutritional requirements consist of one or more nutritional component indicators).

[0137] Step S102: Based on the ingredients indicated by the target input (corresponding to a partial set of ingredient nodes), obtain the complete ingredients of the recipe (corresponding to a complete set of ingredient nodes) through the ingredient completion model.

[0138] Step S103: Based on the complete ingredients of the recipe, the flavor requirements of the recipe, and the nutritional requirements of the recipe, obtain the ingredient description vector, flavor description vector, and nutritional description vector through encoding processing.

[0139] Step S104: Based on the ingredient description vector, flavor description vector, and nutritional description vector, obtain the target recipe through the recipe generation model;

[0140] The ingredient completion model predicts the complete ingredient node set based on a partial ingredient node set, where one node in the (partial / complete) ingredient node set corresponds to one ingredient. The ingredient completion model is obtained by training a graph neural network (GNN) based on masked ingredient samples and their corresponding labels. The masked ingredient samples are generated from the complete ingredients in the recipe sample, creating a complete ingredient node set (one ingredient in the complete ingredient set corresponds to one node in the complete ingredient node set), and then masking some of the ingredient nodes in the complete ingredient node set (the set of unmasked ingredient nodes is defined as the partial ingredient node set, and this partial ingredient node set is used as the masked ingredient sample). The labels corresponding to the masked ingredient samples are obtained by generating the complete ingredient node set from the complete ingredients in the recipe sample (the complete ingredient node set is used as the label).

[0141] The recipe generation model is obtained by training a generative adversarial network (GAN) based on recipe samples. The GAN consists of a generator and a discriminator. The generator is used to generate recipes based on ingredient description vectors, flavor description vectors, and nutritional description vectors. The discriminator is used to determine the authenticity of recipes. The generator trained adversarially serves as the recipe generation model.

[0142] Understandably, the system receives a target input containing partial ingredients, flavor requirements, and nutritional requirements. Then, using a graph neural network (GNN) model trained on masked ingredient samples, it completes the set of partial ingredient nodes corresponding to the target input into a complete set of ingredient nodes. Next, it encodes the complete ingredients, flavor requirements, and nutritional requirements to obtain ingredient description vectors, flavor description vectors, and nutritional description vectors, respectively. Finally, these description vectors are input into a recipe generation model (using a generator from an adversarially trained GAN, capable of generating a complete recipe under multi-source description vector constraints). The recipe generation model can then generate a complete recipe that meets the user's flavor and nutritional requirements.

[0143] For example, a complete recipe generally includes ingredients and cooking process (process), where ingredients specifically include main ingredients, auxiliary ingredients, and seasonings. A recipe may also include flavor descriptions (including flavor indicators) and nutritional descriptions (including nutritional component indicators).

[0144] For example, during the process of receiving target input, the user can describe the ingredients of the recipe (usually some of the ingredients), flavor requirements, and nutritional requirements using natural language. Then, the user's natural language input (the user can input natural language) is processed by a large language model, which performs semantic analysis on the user's natural language and outputs structured data. This structured data includes the ingredients of the recipe (usually some of the ingredients), the flavor requirements of the recipe, and the nutritional requirements of the recipe.

[0145] This section provides an example of flavor metrics, which are parameters used to quantify specific sensory attributes (such as taste, smell, and texture) of a recipe. For example, saltiness, sweetness, acidity, spiciness, and umami can be considered typical flavor metrics.

[0146] This section provides an example of a nutrient content indicator, which is a parameter used to quantify the content or nutritional properties of a specific nutrient in a recipe. Examples include calories, protein content, fat content, carbohydrate content, and dietary fiber content.

[0147] For example, in order to generate flavor description vectors, a flavor-vector table can be pre-constructed, which is used to characterize the mapping relationship between flavor indices and flavor description vectors.

[0148] The complete flavor index of a recipe is expressed as follows: , Indicates the first Flavor indicators, In The range of values ​​is , Let be the total number of flavor indicators, and let the flavor description vector be represented as follows: , A fixed dimension can be used (e.g., dimension 1). ), through flavor coding function The flavor indicators of the recipe are encoded to obtain a flavor description vector:

[0149] ;

[0150] in, Specifically, it could be a function that queries a flavor-vector table, based on a given... Query the flavor-vector table to obtain the corresponding .

[0151] The flavor requirements indicated by the target input only involve The definition of some indicators in the text. = , , middle The range of values ​​is , express The total number of Chinese flavor indicators middle The range of values ​​is , express The total number of flavor indicators, and the flavor requirement indicated by the target input. The flavor requirements indicated by the target input do not involve In this case, the default value (indicating that the indicator is empty) can be used for the indicators. The indicators are assigned values.

[0152] For example, in order to generate nutrient description vectors, a nutrient-vector table can be pre-constructed, which is used to characterize the mapping relationship between nutrient component indicators and nutrient description vectors.

[0153] The complete nutritional information of the recipe is expressed as follows: , Indicates the first Nutritional component indicators, middle The range of values ​​is , The total number of nutritional component indicators is represented by the nutritional description vector as follows: , A fixed dimension can be used (e.g., dimension 1). ), through nutrient coding function The nutritional components of the recipe are encoded to obtain nutritional description vectors:

[0154] ;

[0155] in, Specifically, it could be a function that queries the nutrient-vector table, based on a given... Query the nutrition-vector table to obtain the corresponding... .

[0156] The nutritional requirements indicated by the target input only involve The definition of some indicators in the text. = , , middle The range of values ​​is , express The total number of Chinese flavor indicators middle The range of values ​​is , express The total number of flavor indicators, and the nutritional requirements indicated by the target input. Each indicator in the target input does not involve the nutritional requirements indicated by the target input. In this case, the default value (indicating that the indicator is empty) can be used for the indicators. The indicators are assigned values.

[0157] Here, we will use a masked food sample as an example. A masked food sample refers to a training sample that only contains unmasked food nodes (i.e., a partial set of food nodes) after masking (occluding some food nodes in the complete set of food nodes) during the training of the food completion model (GNN).

[0158] For example, the complete set of ingredient nodes output by the ingredient completion model can be used to determine the text of the complete ingredients in the recipe. Then, based on the text of the complete ingredients in the recipe, the text encoder can be used to encode the ingredients to obtain the ingredient description vector.

[0159] In one possible implementation, the input to the graph neural network (GNN) is a vectorized representation of a partial set of food node values. , , express The Middle A vector of nodes;

[0160] The output of a graph neural network (GNN) is a vectorized representation of the complete set of food node information. , , express The Middle Vectors of nodes (based on) It can determine the first element in the complete set of food ingredient nodes. (Ingredients corresponding to each node);

[0161] assumed and The number of elements is The number of nodes in a partial set of food ingredient nodes is , , middle It can be obtained by vectorizing the nodes in a partial set of food ingredient nodes. middle Use the default vector;

[0162] like Figure 2 As shown, a graph neural network (GNN) includes: a dynamic edge learning module and a graph attention module with edge awareness;

[0163] The dynamic edge learning module includes: a KNN (K-Nearest Neighbors) candidate edge construction unit and an edge probability prediction unit;

[0164] The KNN candidate edge building unit is used to determine the set of candidate edges using the following formula:

[0165] ;

[0166] in, Denotes the set of candidate edges. This represents the K-nearest neighbor algorithm. Represents the nearest neighbor number. for One of the candidate edges in the middle, This represents the first element in the set of partial food ingredient nodes. The node and the first Candidate edges between nodes (the first) The node and the first (each node is a candidate neighbor)

[0167] The edge probability prediction unit is determined by the following formula. Corresponding edge probability :

[0168] ;

[0169] ;

[0170] ;

[0171] The edge probability prediction unit includes a vector concatenation layer, a multilayer perceptron, and an edge probability prediction layer. and This is the input to the vector concatenation layer. This is the output of the vector concatenation layer; Indicates to and Perform vector concatenation operation; This indicates the number of layers in the neural network of a multilayer perceptron; For the first multilayer perceptron Input to a layered neural network; and For the first multilayer perceptron Weights and biases of a multilayer neural network; For the first multilayer perceptron The output of a layered neural network; This represents an activation function (such as GELU). This is the input to the edge probability prediction layer. This is the output of the edge probability prediction layer; and The weights and biases of the edge probability prediction layer, Represents the Sigmoid function;

[0172] The edge-aware graph attention module includes: a feature transformation unit, a weighted adjacency matrix construction unit, an attention coefficient calculation unit, an attention weight normalization unit, and a feature aggregation and residual connection unit;

[0173] The feature transformation unit is used to determine the initial node representation using the following formula:

[0174] ;

[0175] in, This represents the initial node representation. This indicates the data batch normalization layer. Indicates the activation function; and The weights and biases of the feature variation units;

[0176] The weighted adjacency matrix construction unit is used to determine the weighted adjacency matrix using the following formula. :

[0177] ;

[0178] in, express The Middle Line number Column elements;

[0179] The attention coefficient calculation unit is used to determine the first attention coefficient using the following formula. The node and the first Attention coefficient between nodes :

[0180] ;

[0181] in, To correct the linear activation function, The weight vector representing the attention mechanism. and For generating the weight matrix of queries and keys, for The Middle row (representing the first row) (initial representation of each node). for The Middle row (representing the first row) (initial representation of each node). This is a preset value (a very small value, used to prevent numerical overflow). In ;

[0182] Attention weight normalization unit is used for Perform a normalization operation to obtain the normalized attention coefficients. ;

[0183] Feature aggregation and residual connection units are determined by the following formula :

[0184] ;

[0185] ;

[0186] in, For the first The latent representation vector of each node The weight matrix used to generate the values. and These are the weights and biases used to adjust the output.

[0187] In one possible implementation, such as Figure 3 As shown, the generator includes a multi-source feature fusion module and a text generation module.

[0188] The multi-source feature fusion module is used to output a fused conditional context vector based on the input ingredient description vector, flavor description vector, and nutritional description vector through vector concatenation, linear projection layer, and stacked Transformer encoder layers.

[0189] The text generation module is used to output the text sequence of the target recipe based on the conditional context vector and through a text generation model (e.g., through autoregressive decoding processing of the T5 (Text-To-Text Transfer Transformer) model);

[0190] like Figure 4 As shown, the discriminator includes a text encoding module and a classification decision module;

[0191] The text encoding module is used to output a text aggregation representation vector based on the input recipe text sequence through the embedding layer of a pre-trained language model (such as BERT) and stacked Transformer encoder layers;

[0192] The classification decision module is used to output the probability that a recipe text sequence is a real recipe based on the text aggregation representation vector, through the processing of the multilayer perceptron classification head and activation function (such as the sigmoid function).

[0193] In one possible implementation, the loss function used to train the graphical neural network (GNN) is... as follows:

[0194] ;

[0195] in, This represents the adjacency matrix reconstruction loss (ensuring that the vectorized representation of the food node set output by the GNN can restore the true co-occurrence relationship between food items). This represents the KL divergence loss (used to constrain the distribution of the latent representation vectors of nodes to approximate a standard normal distribution, in order to enhance the robustness and generalization ability of the model). This represents the balance coefficient (used to adjust the weights between two losses);

[0196] ;

[0197] in, The vectorized representation of the complete set of food node output by the graph neural network is the total number of nodes (i.e., (number of elements); Number the node (node ​​index); Represents the actual adjacency matrix elements — It is determined based on the labels corresponding to the masked food samples. =1, indicating the first The ingredients corresponding to the node and the first The ingredients corresponding to each node have appeared in the same recipe; =0, indicating the first The ingredients corresponding to the node and the first The ingredients corresponding to each node have not appeared in the same recipe before; Represents the Sigmoid function; The vectorized representation of the complete set of food node output by the graph neural network. A vector of nodes, The vectorized representation of the complete set of food node output by the graph neural network. A vector of nodes;

[0198] ;

[0199] in, Indicates the first The latent representation vector of each node The distribution mean, Indicates the first The latent representation vector of each node The distribution variance.

[0200] Understandably, the adjacency matrix reconstruction loss, by comparing the reconstructed node relationships (based on the true adjacency matrix determined by the labels) with the vectorized representation of the complete set of ingredient nodes output by the GNN, drives the model to accurately predict which ingredient nodes should coexist in the same recipe. Meanwhile, the KL divergence loss, by forcing the latent representation vector of each node to approach a standard normal distribution, acts as a regularization mechanism, enhancing the model's generalization ability to combinations of missing ingredients. The synergistic effect of these two losses enables the trained GNN to reliably infer and complete the missing ingredient nodes based on the input set of partial ingredient nodes, forming a complete set of ingredient nodes.

[0201] In one possible implementation, prior to adversarial training, the generator and discriminator are pre-trained as follows:

[0202] Based on the generator pre-training loss function and the ingredient description vectors, flavor description vectors, and nutritional description vectors corresponding to the first batch of recipe samples (the ingredients in the recipe samples are complete), the generator is pre-trained to obtain the pre-trained generator.

[0203] Obtain a second batch of recipe samples (the ingredients in the recipe samples are complete) to serve as the real recipe samples;

[0204] The ingredient description vector, flavor description vector, and nutritional description vector corresponding to the second batch of recipe samples are input into the pre-trained generator, and the recipes generated by the pre-trained generator are used as pseudo recipe samples.

[0205] Based on the discriminator pre-training loss function, real recipe samples, and pseudo recipe samples, the discriminator is pre-trained to obtain the pre-trained discriminator;

[0206] Among them, the generator pre-training loss function It is determined by the following formula:

[0207] ;

[0208] This indicates the number of the first batch of recipe samples; The first batch of recipe samples A text sequence of recipe samples; yes The corresponding ingredient description vector, flavor description vector, and nutritional description vector; defined. For generators The generated text sequence of recipes; express The number of tokens; express The Each word element (representing a real word element) express Center front Each word element (historical word element);

[0209] The parameter is The generator, in a given And it has already been generated Then, predict whether the next lexical unit is a real lexical unit. The probability of;

[0210] Discriminator pre-training loss function It is determined by the following formula:

[0211] ;

[0212] Indicates the number of recipe samples in the second batch; The first in the second batch of recipe samples A text sequence of recipe samples (as real recipe samples); definition yes The corresponding ingredient description vector, flavor description vector, and nutritional description vector; For the pre-trained generator The generated text sequence of recipes (as pseudo-recipe samples). This indicates that the discriminator evaluates the input recipe text sequence ( , ) is a probability estimate of the actual recipe. The output value is between 0 and 1.

[0213] Understandably, using the first batch of complete recipe samples and their encoded ingredient description vectors, flavor description vectors, and nutritional description vectors, the generator is pre-trained using cross-entropy loss (the generator pre-training loss function). This allows the generator to initially learn to generate coherent recipe text sequences based on given multi-dimensional conditions. Subsequently, the second batch of complete recipe samples is used as real recipe samples, and their corresponding description vectors are input into the pre-trained generator to generate pseudo-recipe samples. Then, based on the real and generated samples, the discriminator is pre-trained using binary classification cross-entropy loss (the discriminator pre-training loss function), enabling it to initially distinguish between real and generated recipes. The purpose of this pre-training stage is to provide a generator and discriminator with certain basic capabilities as a starting point for subsequent adversarial training, avoiding mode collapse or training instability caused by random model initialization in the early stages of adversarial training, thereby ensuring that the adversarial training process can converge more efficiently and stably.

[0214] In one possible implementation, adversarial training includes: deleting some ingredients from the third batch of recipe samples (in which the ingredients are complete), constructing a set of ingredient nodes using the remaining ingredients (partial ingredients) and inputting it into the ingredient completion model, replacing the ingredients of the third batch of recipe samples with the complete ingredients output by the ingredient completion model, and obtaining the updated third batch of recipe samples.

[0215] Based on the adversarial training loss function and the updated third batch of recipe samples, the generator and discriminator are trained to obtain the recipe generation model;

[0216] The adversarial training loss function includes the loss function used to train the generator. and the loss function used to train the discriminator ;

[0217] It is determined by the following formula:

[0218] ;

[0219] For cross-entropy loss, To combat the losses, This is a balancing coefficient (adjusting the weights of cross-entropy loss and adversarial loss).

[0220] ;

[0221] This indicates the number of recipe samples in the third batch; The third batch of recipe samples A text sequence of a recipe sample. Through the The text sequence of the updated recipe sample obtained by partially deleting ingredients and then completing them using an ingredient completion model; yes The corresponding ingredient description vector, flavor description vector, and nutritional description vector; defined. For generators The generated text sequence of recipes; express The number of tokens; express The Each word element (representing a real word element) express Center front Each word element (historical word element);

[0222] The parameter is The generator, in a given And it has already been generated Then, predict whether the next lexical unit is a real lexical unit. The probability of;

[0223] ;

[0224] It is determined by the following formula:

[0225] ;

[0226] This indicates that the discriminator evaluates the input recipe text sequence ( , ) is a probability estimate of the actual recipe.

[0227] Understandably, after the generator and discriminator have completed pre-training, some ingredients are removed from the third batch of complete recipe samples. The remaining ingredients are used to predict the complete ingredients through an ingredient completion model, and this prediction is used to update the ingredient information of the third batch of recipe samples. Then, the weighted sum of cross-entropy loss and adversarial loss is used as the... The generator is trained using the updated samples.

[0228] On the one hand, through The provided cross-entropy loss further optimizes the accuracy and coherence of the generator's generated text; on the other hand, it introduces a discriminator to judge the "authenticity" of the generated recipes as an adversarial loss. The loss provided drives the generator to produce recipes that more closely resemble the real data distribution.

[0229] It is worth noting that during adversarial training, the input ingredient description vectors for the generator do not originate from the original complete recipe samples, but rather from the complete set of ingredient nodes generated by the ingredient completion model after partially completing the ingredients. This means that the generator's training process is constantly faced with ingredient combinations predicted by the completion model that differ from the original complete ingredients. Simultaneously, the discriminator continuously learns from the adversarial loss how to distinguish between real and generated recipes. This adversarial game forces the generator not only to generate coherent (fluent) text sequences (guaranteed by cross-entropy loss), but also to learn to adjust its generation logic so that the recipes generated based on the completed ingredients are highly adapted to user requirements in terms of flavor description and nutritional composition, thus appearing more "realistic" to the discriminator.

[0230] Therefore, in the iterative process of adversarial training, the generator gradually learns to understand and improve the output of the ingredient completion model, so that the recipes it generates can bridge the potential gap between the completed ingredients (the ingredients output by the ingredient completion model) and the user's requirements. This enables the generator to improve the ingredients output by the ingredient completion model. The improved ingredients are more compatible with the user's requirements (flavor requirements and nutritional requirements), so as to generate recipes that better meet the user's requirements and achieve a high degree of adaptability.

[0231] In one possible implementation, the target input is received, including:

[0232] Audio data is collected via the microphone of the smart glasses;

[0233] Based on the collected audio data, text data is obtained through speech-to-text conversion;

[0234] Semantic analysis of text data is performed using a large language model to obtain semantic analysis results;

[0235] If the semantic analysis results include flavor and nutritional requirements, and the semantic analysis results indicate that recipes should be generated for ingredients within the field of view of the smart glasses according to the flavor and nutritional requirements, the flavor and nutritional requirements included in the semantic analysis results are used as the target inputs of the indicated flavor and nutritional requirements, and images are captured by the camera of the smart glasses.

[0236] Based on the acquired images, image recognition is performed, and the ingredients identified in the images are used as the target input to indicate the ingredients.

[0237] Optionally, the text of the target recipe can be converted into corresponding voice data through text-to-speech (TTS), and the voice data can be broadcast through the speaker of the smart glasses.

[0238] Understandably, the system collects user audio data via the smart glasses' microphone and converts it into text data using speech-to-text technology. Subsequently, a large language model performs semantic analysis on the text data. When the analysis indicates that the user requests a recipe generated based on specific flavor and nutritional requirements within the smart glasses' field of view, the system extracts these requirements from the semantic analysis results as part of the target input. Simultaneously, the smart glasses' camera captures images within the field of view. Next, image recognition is performed on the captured images, and the identified ingredients, along with the extracted flavor and nutritional requirements, constitute the target input for the recipe generation system. This allows for quick input through the smart glasses.

[0239] The recipe generation device based on graph neural networks and adversarial training provided in this application is described below. The recipe generation device based on graph neural networks and adversarial training described below can be referred to in correspondence with the recipe generation method based on graph neural networks and adversarial training described above.

[0240] Figure 5 This is a schematic diagram of the structure of the recipe generation system based on graph neural networks and adversarial training provided in the embodiments of this application, as shown below. Figure 5 As shown, the system includes: an interaction module 10, an ingredient completion module 20, a vectorization module 30, and a recipe generation module 40. Among them:

[0241] The interaction module 10 is used to receive target input, which is used to indicate the ingredients of the recipe, the flavor requirements of the recipe, and the nutritional requirements of the recipe.

[0242] The ingredient completion module 20 is used to obtain the complete ingredients of the recipe based on the ingredients indicated by the target input through the ingredient completion model;

[0243] The vectorization module 30 is used to obtain ingredient description vectors, flavor description vectors, and nutritional description vectors through encoding processing based on the complete ingredients of the recipe, the flavor requirements of the recipe, and the nutritional requirements of the recipe.

[0244] The recipe generation module 40 is used to obtain the target recipe based on the ingredient description vector, flavor description vector, and nutritional description vector through the recipe generation model.

[0245] Among them, the ingredient completion model predicts the complete ingredient node set based on a partial ingredient node set, where each node in the ingredient node set corresponds to an ingredient; the ingredient completion model is obtained by training a graph neural network based on masked ingredient samples and corresponding labels. The masked ingredient samples are obtained by generating a complete ingredient node set based on the complete ingredients in the recipe sample and masking some of the ingredient nodes in the complete ingredient node set; the labels corresponding to the masked ingredient samples are obtained by generating a complete ingredient node set based on the complete ingredients in the recipe sample.

[0246] The recipe generation model is obtained by training an adversarial generative network based on recipe samples. The adversarial generative network includes a generator and a discriminator. The generator is used to generate recipes based on ingredient description vectors, flavor description vectors, and nutritional description vectors. The discriminator is used to distinguish the authenticity of recipes. The generator trained adversarially serves as the recipe generation model.

[0247] It is understood that the detailed functional implementation of each of the above units / modules can be found in the description in the aforementioned method embodiments, and will not be repeated here.

[0248] It should be understood that the above-described device is used to execute the methods in the above embodiments. The implementation principle and technical effect of the corresponding program modules in the device are similar to those described in the above methods. The working process of the device can be referred to the corresponding process in the above methods, and will not be repeated here.

[0249] Based on the methods in the above embodiments, this application provides an electronic device, such as... Figure 6 As shown, the electronic device may include a processor 810, a communications interface 820, a memory 830, and a communication bus 840, wherein the processor 810, the communications interface 820, and the memory 830 communicate with each other through the communication bus 840. The processor 810 can call logical instructions in the memory 830 to execute the methods in the above embodiments.

[0250] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0251] Based on the methods in the above embodiments, this application provides a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0252] Based on the methods in the above embodiments, this application provides a computer program product that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0253] It is understood that the processor in the embodiments of this application can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor can be a microprocessor or any conventional processor.

[0254] The method steps in this application embodiment can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.

[0255] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0256] It is understood that the various numerical designations used in the embodiments of this application are merely for the convenience of description and are not intended to limit the scope of the embodiments of this application.

[0257] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A recipe generation method based on graph neural networks and adversarial training, characterized in that, include: Receive target input, which indicates the ingredients, flavor requirements, and nutritional requirements of the recipe; Based on the ingredients indicated by the target input, the complete ingredients of the recipe are obtained through the ingredient completion model; Based on the complete ingredients of the recipe, the flavor requirements of the recipe, and the nutritional requirements of the recipe, the ingredient description vector, flavor description vector, and nutritional description vector are obtained through encoding processing. Based on ingredient description vectors, flavor description vectors, and nutritional description vectors, the target recipe is obtained through a recipe generation model. Among them, the ingredient completion model predicts the complete ingredient node set based on a partial ingredient node set, where each node in the ingredient node set corresponds to an ingredient; the ingredient completion model is obtained by training a graph neural network based on masked ingredient samples and corresponding labels. The masked ingredient samples are obtained by generating a complete ingredient node set based on the complete ingredients in the recipe sample and masking some of the ingredient nodes in the complete ingredient node set; the labels corresponding to the masked ingredient samples are obtained by generating a complete ingredient node set based on the complete ingredients in the recipe sample. The recipe generation model is obtained by training an adversarial generative network based on recipe samples. The adversarial generative network includes a generator and a discriminator. The generator is used to generate recipes based on ingredient description vectors, flavor description vectors, and nutritional description vectors. The discriminator is used to distinguish the authenticity of recipes. The generator trained adversarially serves as the recipe generation model. The input to the graph neural network is a vectorized representation of a partial set of food node values. , , express The Middle A vector of nodes; The output of a graph neural network is a vectorized representation of the complete set of food node sets. , , express The Middle A vector of nodes; Graph neural networks include: a dynamic edge learning module and a graph attention module with edge awareness; The dynamic edge learning module includes: a KNN candidate edge construction unit and an edge probability prediction unit; The KNN candidate edge building unit is used to determine the set of candidate edges using the following formula: ； in, Denotes the set of candidate edges. This represents the K-nearest neighbor algorithm. Represents the nearest neighbor number. for One of the candidate edges in the middle, This represents the first element in the set of partial food ingredient nodes. The node and the first Candidate edges between nodes; The edge probability prediction unit is determined by the following formula. Corresponding edge probability : ； ； ； The edge probability prediction unit includes a vector concatenation layer, a multilayer perceptron, and an edge probability prediction layer. and This is the input to the vector concatenation layer. This is the output of the vector concatenation layer; Indicates to and Perform vector concatenation operation; This indicates the number of layers in the neural network of a multilayer perceptron; For the first multilayer perceptron Input to a layered neural network; and For the first multilayer perceptron Weights and biases of a multilayer neural network; For the first multilayer perceptron The output of a layered neural network; Indicates the activation function; This is the input to the edge probability prediction layer. This is the output of the edge probability prediction layer; and The weights and biases of the edge probability prediction layer, Represents the Sigmoid function; The edge-aware graph attention module includes: a feature transformation unit, a weighted adjacency matrix construction unit, an attention coefficient calculation unit, an attention weight normalization unit, and a feature aggregation and residual connection unit; The feature transformation unit is used to determine the initial node representation using the following formula: ； in, This represents the initial node representation. This indicates the data batch normalization layer. Indicates the activation function; and The weights and biases of the feature variation units; The weighted adjacency matrix construction unit is used to determine the weighted adjacency matrix using the following formula. : ； in, express The Middle Line number Column elements; The attention coefficient calculation unit is used to determine the first attention coefficient using the following formula. The node and the first Attention coefficient between nodes : ； in, To correct the linear activation function, The weight vector representing the attention mechanism. and For generating the weight matrix of queries and keys, for The Middle OK, for The Middle OK, This is the default value; In ; Attention weight normalization unit is used for Perform a normalization operation to obtain the normalized attention coefficients. ; Feature aggregation and residual connection units are determined by the following formula : ； ； in, For the first The latent representation vector of each node The weight matrix used to generate the values. and For the weights and biases used to change the output; The generator includes: a multi-source feature fusion module and a text generation module; The multi-source feature fusion module is used to output a fused conditional context vector based on the input ingredient description vector, flavor description vector, and nutritional description vector through vector concatenation, linear projection layer, and stacked Transformer encoder layers. The text generation module is used to output the text sequence of the target recipe based on the conditional context vector and through the text generation model; The discriminator includes: a text encoding module and a classification decision module; The text encoding module is used to output a text aggregation representation vector based on the input recipe text sequence through the embedding layer of a pre-trained language model and stacked Transformer encoder layers; The classification decision module is used to output the probability that a recipe text sequence is a real recipe based on the text aggregation representation vector, through the processing of the multilayer perceptron classification head and the sigmoid function.

2. The recipe generation method based on graph neural networks and adversarial training according to claim 1, characterized in that, The loss function used in training a graph neural network as follows: ； in, This represents the adjacency matrix reconstruction loss. This represents the KL divergence loss. Indicates the balance coefficient; ； in, The total number of nodes represents the vectorized representation of the complete set of food node output by the graph neural network; Number the nodes; Represents the actual adjacency matrix elements; Represents the Sigmoid function; The vectorized representation of the complete set of food node output by the graph neural network. A vector of nodes, The vectorized representation of the complete set of food node output by the graph neural network. A vector of nodes; ； in, Indicates the first The latent representation vector of each node The distribution mean, Indicates the first The latent representation vector of each node The distribution variance.

3. The recipe generation method based on graph neural networks and adversarial training according to claim 1, characterized in that, Prior to adversarial training, the generator and discriminator are pre-trained as follows: Based on the generator pre-training loss function and the ingredient description vector, flavor description vector and nutritional description vector corresponding to the first batch of recipe samples, the generator is pre-trained to obtain the pre-trained generator. Obtain a second batch of recipe samples to serve as the actual recipe samples; The ingredient description vector, flavor description vector, and nutritional description vector corresponding to the second batch of recipe samples are input into the pre-trained generator, and the recipes generated by the pre-trained generator are used as pseudo recipe samples. Based on the discriminator pre-training loss function, real recipe samples, and pseudo recipe samples, the discriminator is pre-trained to obtain the pre-trained discriminator; Among them, the generator pre-training loss function It is determined by the following formula: ； This indicates the number of the first batch of recipe samples; The first batch of recipe samples A text sequence of recipe samples; yes The corresponding ingredient description vector, flavor description vector, and nutritional description vector; defined. For generators The generated text sequence of recipes; express The number of lexical units; express The Each word element, express Center front Each word element; The parameter is The generator, in a given And it has already been generated Then, predict whether the next lexical unit is a real lexical unit. The probability of; Discriminator pre-training loss function It is determined by the following formula: ； Indicates the number of recipe samples in the second batch; The first in the second batch of recipe samples A text sequence of recipe samples; definition yes The corresponding ingredient description vector, flavor description vector, and nutritional description vector; For the pre-trained generator The generated text sequence of recipes; This represents the probability estimate by which the discriminator identifies the input recipe text sequence as a genuine recipe.

4. The recipe generation method based on graph neural networks and adversarial training according to claim 3, characterized in that, Combat training includes: Some ingredients in the third batch of recipe samples are deleted. The remaining ingredients are used to construct a set of ingredient nodes and input into the ingredient completion model. The complete ingredients output by the ingredient completion model are used to replace the ingredients in the third batch of recipe samples to obtain the updated third batch of recipe samples. Based on the adversarial training loss function and the updated third batch of recipe samples, the generator and discriminator are trained to obtain the recipe generation model; The adversarial training loss function includes the loss function used to train the generator. and the loss function used to train the discriminator ; It is determined by the following formula: ； For cross-entropy loss, To combat the losses, This is the balance coefficient; ； This indicates the number of recipe samples in the third batch; The third batch of recipe samples A text sequence of a recipe sample. Through the The text sequence of the updated recipe sample obtained by partially deleting ingredients and then completing them using an ingredient completion model; yes The corresponding ingredient description vector, flavor description vector, and nutritional description vector; defined. For generators The generated text sequence of recipes; express The number of lexical units; express The Each word element, express Center front Each word element; The parameter is The generator, in a given And it has already been generated Then, predict whether the next lexical unit is a real lexical unit. The probability of; ； It is determined by the following formula: ； This represents the probability estimate by which the discriminator identifies the input recipe text sequence as a genuine recipe.

5. The recipe generation method based on graph neural networks and adversarial training according to claim 1, characterized in that, Receive target input, including: Audio data is collected via the microphone of the smart glasses; Based on the collected audio data, text data is obtained through speech-to-text conversion; Semantic analysis of text data is performed using a large language model to obtain semantic analysis results; If the semantic analysis results include flavor and nutritional requirements, and the semantic analysis results indicate that recipes should be generated for ingredients within the field of view of the smart glasses according to the flavor and nutritional requirements, the flavor and nutritional requirements included in the semantic analysis results are used as the target inputs of the indicated flavor and nutritional requirements, and images are captured by the camera of the smart glasses. Based on the acquired images, image recognition is performed, and the ingredients identified in the images are used as the target input to indicate the ingredients.

6. A recipe generation system based on graph neural networks and adversarial training, characterized in that, include: The interaction module is used to receive target input, which indicates the ingredients, flavor requirements, and nutritional requirements of the recipe. The ingredient completion module is used to obtain the complete ingredients of a recipe based on the ingredients indicated by the target input, through an ingredient completion model; The vectorization module is used to obtain ingredient description vectors, flavor description vectors, and nutritional description vectors based on the complete ingredients, flavor requirements, and nutritional requirements of the recipe through encoding processing. The recipe generation module is used to obtain target recipes based on ingredient description vectors, flavor description vectors, and nutritional description vectors through a recipe generation model. Among them, the ingredient completion model predicts the complete ingredient node set based on a partial ingredient node set, where each node in the ingredient node set corresponds to an ingredient; the ingredient completion model is obtained by training a graph neural network based on masked ingredient samples and corresponding labels. The masked ingredient samples are obtained by generating a complete ingredient node set based on the complete ingredients in the recipe sample and masking some of the ingredient nodes in the complete ingredient node set; the labels corresponding to the masked ingredient samples are obtained by generating a complete ingredient node set based on the complete ingredients in the recipe sample. The recipe generation model is obtained by training an adversarial generative network based on recipe samples. The adversarial generative network includes a generator and a discriminator. The generator is used to generate recipes based on ingredient description vectors, flavor description vectors, and nutritional description vectors. The discriminator is used to distinguish the authenticity of recipes. The generator trained adversarially serves as the recipe generation model. The input to the graph neural network is a vectorized representation of a partial set of food node values. , , express The Middle A vector of nodes; The output of a graph neural network is a vectorized representation of the complete set of food node sets. , , express The Middle A vector of nodes; Graph neural networks include: a dynamic edge learning module and a graph attention module with edge awareness; The dynamic edge learning module includes: a KNN candidate edge construction unit and an edge probability prediction unit; The KNN candidate edge building unit is used to determine the set of candidate edges using the following formula: ； in, Denotes the set of candidate edges. This represents the K-nearest neighbor algorithm. Represents the nearest neighbor number. for One of the candidate edges in the middle, This represents the first element in the set of partial food ingredient nodes. The node and the first Candidate edges between nodes; The edge probability prediction unit is determined by the following formula. Corresponding edge probability : ； ； ； The edge probability prediction unit includes a vector concatenation layer, a multilayer perceptron, and an edge probability prediction layer. and This is the input to the vector concatenation layer. This is the output of the vector concatenation layer; Indicates to and Perform vector concatenation operation; This indicates the number of layers in the neural network of a multilayer perceptron; For the first multilayer perceptron Input to a layered neural network; and For the first multilayer perceptron Weights and biases of a multilayer neural network; For the first multilayer perceptron The output of a layered neural network; Indicates the activation function; This is the input to the edge probability prediction layer. This is the output of the edge probability prediction layer; and The weights and biases of the edge probability prediction layer, Represents the Sigmoid function; The edge-aware graph attention module includes: a feature transformation unit, a weighted adjacency matrix construction unit, an attention coefficient calculation unit, an attention weight normalization unit, and a feature aggregation and residual connection unit; The feature transformation unit is used to determine the initial node representation using the following formula: ； in, This represents the initial node representation. This indicates the data batch normalization layer. Indicates the activation function; and The weights and biases of the feature variation units; The weighted adjacency matrix construction unit is used to determine the weighted adjacency matrix using the following formula. : ； in, express The Middle Line number Column elements; The attention coefficient calculation unit is used to determine the first attention coefficient using the following formula. The node and the first Attention coefficient between nodes : ； in, To correct the linear activation function, The weight vector representing the attention mechanism. and For generating the weight matrix of queries and keys, for The Middle OK, for The Middle OK, This is the default value; In ; Attention weight normalization unit is used for Perform a normalization operation to obtain the normalized attention coefficients. ; Feature aggregation and residual connection units are determined by the following formula : ； ； in, For the first The latent representation vector of each node The weight matrix used to generate the values. and For the weights and biases used to change the output; The generator includes: a multi-source feature fusion module and a text generation module; The multi-source feature fusion module is used to output a fused conditional context vector based on the input ingredient description vector, flavor description vector, and nutritional description vector through vector concatenation, linear projection layer, and stacked Transformer encoder layers. The text generation module is used to output the text sequence of the target recipe based on the conditional context vector and through the text generation model; The discriminator includes: a text encoding module and a classification decision module; The text encoding module is used to output a text aggregation representation vector based on the input recipe text sequence through the embedding layer of a pre-trained language model and stacked Transformer encoder layers; The classification decision module is used to output the probability that a recipe text sequence is a real recipe based on the text aggregation representation vector, through the processing of the multilayer perceptron classification head and the sigmoid function.

Images