Construction of allergen detection model, allergen detection method and cooking equipment

Abstract

The invention relates to the technical field of intelligent household appliances, and discloses an allergen detection model construction method, an allergen detection method and cooking equipment, and the method comprises the steps: obtaining a multi-dimensional time sequence data sample set which comprises multiple groups of training data, each group of training data comprises cooking conditions, gas detection data and allergen label information; cooking conditions and gas detection data of each group of training data in the multi-dimensional time sequence data sample set are analyzed based on a built deep learning model, and the allergen probability of a known allergen corresponding to an allergen label is obtained; and training the deep learning model based on the allergen probability and allergen label information of the known allergen to obtain an allergen detection model. According to the method, the space characteristics and the time characteristics of the gas generated in the cooking process are comprehensively considered, the gas generated by the food in the cooking process and the change of the gas characteristics along with the cooking process are accurately analyzed, and the potential allergens in the cooked food are accurately detected.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent household appliances, and particularly relates to a construction of an allergen detection model, an allergen detection method and a cooking device. BACKGROUND

[0002] With the wide application of various types of cooking devices such as steam ovens and the continuous development of their technology, their functions are increasingly rich, and users' demand for intelligentization is also significantly improved. In the face of various cooking scenes such as baking, steaming and grilling in the home and catering scenes, the ingredients of the cooked food are often complex and difficult to identify, and users often cannot accurately know whether there are allergens such as common allergens such as peanuts, nuts, seafood, milk and eggs in the food. The lack of information may cause the allergic population to ingest allergen ingredients without preparation, and thus trigger an allergic reaction, bringing potential risks to food safety.

[0003] In the prior art, allergen detection mainly adopts specific test paper detection method, Enzyme Linked Immunosorbent Assay (ELISA) or Polymerase Chain Reaction (PCR) and other technical means under laboratory conditions. Although the above methods can realize accurate detection of food allergens, they are difficult to be directly applied in the use scene of the steam oven due to the limitation of the detection environment and the operation condition, and thus cannot realize allergen detection in the cooking process. SUMMARY

[0004] Therefore, the present application provides a construction of an allergen detection model, an allergen detection method and a cooking device to solve the problem that allergen detection cannot be performed in the cooking process.

[0005] In a first aspect, the present application provides a construction method of an allergen detection model, which comprises: obtaining a multi-dimensional time series data sample set, wherein the multi-dimensional time series data sample set comprises: a plurality of groups of training data, wherein each group of training data comprises: cooking conditions, gas detection data and allergen label information; analyzing the cooking conditions and the gas detection data of each group of training data in the multi-dimensional time series data sample set based on a built deep learning model to obtain an allergen probability of a known allergen corresponding to the allergen label; and training the deep learning model based on the allergen probability of the known allergen corresponding to each group of training data and the allergen label information to obtain an allergen detection model.

[0006] The application provides a method for constructing an allergen detection model, which comprises the following steps: obtaining a multi-dimensional time sequence data sample set containing multiple groups of training data; analyzing the cooking conditions and gas detection data of each group of training data based on a deep learning model, obtaining the allergen probability of a known allergen corresponding to an allergen label, and training the deep learning model based on the allergen probability of the known allergen corresponding to each group of training data and the allergen label information, thereby obtaining an allergen detection model. The model is constructed based on the gas generated during the cooking process, the spatial features and the time features of the gas generated during the cooking process can be comprehensively considered, the change of the gas generated by food during the cooking process and the gas features can be accurately analyzed, the hidden allergen in the cooked food can be accurately detected, the model is suitable for various complex cooking scenes, and the cooking equipment is endowed with the function of detecting allergens, so that the user can be reminded in time to avoid allergic reactions and ensure the safety of the user's diet.

[0007] In an optional embodiment, before the deep learning model combining the convolutional neural network and the long short-term memory network is constructed, the multi-dimensional time sequence data sample set is further subjected to outlier rejection, sliding average filtering, standardization processing and dynamic window division processing.

[0008] The application can solve the problem that the data sequence is of different lengths due to the difference in cooking time, and ensure the uniformity of the model input.

[0009] In an optional embodiment, the deep learning model comprises an input layer, a convolutional layer, a long short-term memory network layer and a fully connected layer, and the convolutional layer comprises a first convolutional layer, a first maximum pooling layer, a dropout layer, a second convolutional layer and a second maximum pooling layer.

[0010] The application can mine the spatial features and the time features of the gas generated during the cooking process by constructing the deep learning model combining the convolutional neural network and the long short-term memory network, so that the model is suitable for various complex cooking scenes.

[0011] In an optional embodiment, the analysis of the cooking conditions and the gas detection data of each group of training data in the multi-dimensional time sequence data sample set based on the deep learning model to obtain the allergen probability of the known allergen corresponding to the allergen label comprises: inputting the cooking conditions and the gas detection data of each group of training data into the input layer, and processing them through the convolutional layer and the long short-term memory network layer, and outputting the confidence of each known allergen from the fully connected layer; and calculating the allergen probability of each known allergen according to the confidence of each known allergen and a mixed allergen decoupling formula, the mixed allergen decoupling formula being as follows:

[0012]

[0013] wherein, P i is the allergen probability, Z i is the confidence of the i-th known allergen, Z j is the confidence of the j-th known allergen, and γ is the competitive inhibition coefficient, and S represents a sigmoid function.

[0014] The present application can avoid cross interference of mixed food materials by decoupling the confidence of each known allergen obtained by analysis, and quickly detect the allergen contained in the cooked food.

[0015] In an alternative embodiment, the deep learning model is trained based on the allergen probability and allergen label information of the known allergen corresponding to each set of training data, and an allergen detection model is obtained, including: constructing a loss function according to the allergen probability and allergen label information of the known allergen, and optimizing and training the deep learning model based on the loss function and a preset optimization algorithm to obtain an evaluation index; during the optimization and training process, the cooking time corresponding to the multi-dimensional time series data sample set is segmented based on a segmented attention mechanism, and the feature weight coefficient corresponding to each segment is adjusted during the training process; the deep learning model is iteratively optimized based on the allergen probability and allergen label information of the known allergen corresponding to the single allergen experimental sample and the known allergen corresponding to the mixed allergen experimental sample, and the allergen detection model is obtained.

[0016] The present application can learn the time characteristics of the generated gas in the cooking process by segmenting and iteratively optimizing the deep learning model, so as to strengthen the feature weight of the key phase of gas release in the model, and gradually improve the recognition ability of the model in complex scenes from simple to complex.

[0017] In a second aspect, the present application provides an allergen detection method applied to a cooking device, the method comprising: after the cooking device cooks a target food, obtaining multi-dimensional time series data of the generated comprehensive gas, the multi-dimensional time series data comprising: cooking conditions and gas detection data; based on an allergen detection model, analyzing the spatial local features and time global features of the multi-dimensional time series data to determine the allergen probability of different preset allergens, wherein the allergen detection model is constructed based on the construction method of the allergen detection model of the first aspect or any one of the corresponding embodiments thereof; determining the actual allergen contained in the target food according to the allergen probability of different preset allergens.

[0018] The allergen detection method provided by the application can detect the gas generated during cooking, can endow the cooking equipment with the function of detecting allergens, can accurately detect the hidden allergens in the cooked food according to the gas generated by the food during cooking and the change of the gas characteristics with the cooking process, and can timely remind the user to avoid the occurrence of allergic reactions and ensure the safety of the user's diet.

[0019] In an alternative embodiment, determining the actual allergen contained in the target food according to the allergen probability of different preset allergens comprises: comparing the allergen probability of different preset allergens with a preset probability threshold to determine whether the allergen probability exceeds the preset probability threshold; if yes, determining that the target food contains the preset allergen corresponding to the allergen probability, and taking the preset allergen as the actual allergen.

[0020] In an alternative embodiment, determining the actual allergen contained in the target food according to the allergen probability of different preset allergens further comprises: generating an alert signal according to the actual allergen; or comparing the actual allergen with a pre-set specified allergen to determine whether the target food contains the specified allergen, and generating an alarm signal according to the specified allergen if yes.

[0021] The application can identify the actual allergen in the cooked food, provide a reminder or an alarm for the user, enable the user to accurately master the type of food material in the cooked food, and then determine whether the food can be eaten according to the demand, so as to avoid the occurrence of allergic events.

[0022] In a third aspect, the application provides a device for constructing an allergen detection model, which comprises: a sample acquisition module for acquiring a multi-dimensional time series data sample set, wherein the multi-dimensional time series data sample set comprises: a plurality of groups of training data, wherein each group of training data comprises: cooking conditions, gas detection data and allergen label information; a model analysis module for analyzing the cooking conditions and the gas detection data of each group of training data in the multi-dimensional time series data sample set based on a built deep learning model to obtain the allergen probability of the known allergen corresponding to the allergen label; and a model training module for training the deep learning model based on the allergen probability of the known allergen corresponding to each group of training data and the allergen label information to obtain the allergen detection model.

[0023] In a fourth aspect, the present application provides an allergen detection device applied to a cooking device, the device comprising: a data acquisition module configured to acquire multi-dimensional time series data of a generated comprehensive gas after the cooking device cooks a target food, the multi-dimensional time series data comprising: a cooking time, a real-time temperature, a real-time humidity and real-time gas detection data at a moment corresponding to the cooking time; a probability detection module configured to analyze spatial local features and temporal global features of the multi-dimensional time series data based on an allergen detection model to determine allergen probabilities of different preset allergens, wherein the allergen detection model is constructed based on the construction device of the allergen detection model of the third aspect; and an allergen determination module configured to determine actual allergens contained in the target food according to the allergen probabilities of the different preset allergens.

[0024] In a fifth aspect, the present application provides a cooking device comprising: a controller; the controller comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the allergen detection method of the second aspect or any of the corresponding embodiments thereof.

[0025] In an optional embodiment, the cooking device is a steam oven, and the steam oven further comprises: a gas sensor array configured to detect gases in the steam oven.

[0026] The cooking device provided by the present application can increase the allergen detection function based on the deployed sensor array. The gas sensor array reflects the spatial characteristics of the generated gas during cooking, and the gas sensor array continuously detects the generated gas to reflect the temporal characteristics of the generated gas, so as to accurately detect the hidden allergens in the cooked food according to the gas generated by the food during cooking and the change of the gas characteristics with the cooking process, timely remind the user, avoid the occurrence of allergic reactions, and ensure the safety of the user's diet. BRIEF DESCRIPTION OF DRAWINGS

[0027] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the following will briefly introduce the drawings needed to be used in the specific embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0028] Figure 1 is a flowchart of the construction method of the allergen detection model according to an embodiment of the present application;

[0029] Figure 2 is a flowchart of the allergen detection method according to an embodiment of the present application;

[0030] Figure 3 is a timing diagram of an allergen detection method according to an embodiment of the present application;

[0031] Figure 4 is a flowchart of another allergen detection method according to an embodiment of the present application;

[0032] Figure 5 is a first interaction process diagram of another allergen detection method according to an embodiment of the present application;

[0033] Figure 6 is a second interaction process diagram of another allergen detection method according to an embodiment of the present application;

[0034] Figure 7 is a structural block diagram of a construction device of an allergen detection model according to an embodiment of the present application;

[0035] Figure 8 is a structural block diagram of an allergen detection device according to an embodiment of the present application;

[0036] Figure 9 is a hardware structure diagram of a controller of a cooking device according to an embodiment of the present application. DETAILED DESCRIPTION

[0037] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0038] The embodiments of the present application are suitable for the scene of detecting whether common allergens exist in cooked food in a cooking process using a cooking device. The embodiments of the present application provide a construction method of an allergen detection model, which detects the gas generated in the cooking process by constructing the model to achieve the effect of judging whether the food contains allergens.

[0039] According to the embodiments of the present application, a construction method of an allergen detection model is provided. It should be noted that the steps shown in the flowchart of the drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that here.

[0040] In the present embodiment, a construction method of an allergen detection model is provided, which can be used in the above-mentioned cooking device, such as a steam oven, etc. Figure 1is a flowchart of a method for constructing an allergen detection model according to an embodiment of the present application, as shown in Figure 1 The flowchart includes the following steps:

[0041] In step S101, a multi-dimensional time series data sample set is obtained, where the multi-dimensional time series data sample set includes a plurality of groups of training data, and each group of training data includes cooking conditions, gas detection data, and allergen label information.

[0042] Specifically, in the present embodiment, different types of food will produce characteristic gases during cooking. Taking common allergens such as peanuts, nuts, seafood, milk, and eggs as examples, seafood mainly releases amine substances such as trimethylamine and dimethylamine; peanuts and nuts release aldehyde and ketone compounds; eggs mainly generate sulfur-containing compounds such as hydrogen sulfide; dairy products and milk release organic acid substances. In addition, the concentration of the gases generated by different food materials during cooking also exhibits unique time-domain characteristics, i.e., the gas release amount of different food materials will have significant differences at different stages of cooking. In addition, factors such as cooking method and set cooking temperature also have important influences on gas generation. By analyzing the above characteristic gases and their change rules, accurate classification and identification of different allergens can be achieved. Therefore, the present embodiment pre-constructs an allergen detection model through cooking experiments, so as to directly detect whether the cooked food contains allergens during cooking.

[0043] In some optional embodiments, high-quality sample data sets are an important key factor to ensure the accuracy of subsequent models, so the experimental design must comprehensively cover various variables in real scenarios. In terms of experimental sample design, the following four categories are mainly included: single allergen samples of different doses, mixed samples, control samples without allergens, and commonly used recipe samples. In terms of experimental condition design (mainly referring to cooking methods), in addition to the recipe samples with fixed cooking procedures (whose time, temperature and mode parameters are fixed values), all other experimental samples need to cover different experimental condition combinations, including: cooking mode (steaming, baking, steam baking), temperature range (60-230°C), cooking time (5-120 minutes) and humidity level (high, medium and low levels in steam baking mode). The cooking experiment of each group of experimental samples needs to be repeated at least 5 times or more to eliminate random errors, and various data in the entire experimental process are recorded in detail, including time, gas sensor array gas detection data (i.e. the response values of each sensor unit to different types of gases such as aldehydes, ketones, organic acids, amines and sulfides), and cooking conditions such as temperature and humidity. In this way, each group of experimental samples will generate a structured data table as a sample data, and the format is: {time}x{temperature, humidity, aldehyde sensor value, ketone sensor value, organic acid sensor value…sulfide sensor value, allergen label}. Different experimental samples correspond to different training data under different experimental conditions, and then a multi-dimensional time series data sample set containing multiple groups of training data is obtained which comprehensively covers real scenarios. The embodiment of the present application constructs multi-dimensional time series data based on all information in the cooking process, which can comprehensively consider the spatial and temporal characteristics of the generated gas in the cooking process when detecting allergens, avoiding the fragmentation of key information by traditional single-dimensional data (such as only recording gas concentration values), and providing accurate data reference for subsequent deep learning analysis.

[0044] In step S102, the cooking conditions and gas detection data of each group of training data in the multi-dimensional time series data sample set are analyzed based on the built deep learning model, and the allergen probability of the known allergen corresponding to the allergen label is obtained.

[0045] Specifically, in the embodiments of the present application, it can be known from experimental analysis that each set of sample data in the obtained sample data set belongs to typical multi-dimensional time series data. The specific representation is that the characteristic gas information (i.e. the signal of the specific gas concentration detected by each unit in the sensor array changing with time) released by different allergen foods in the heat processing process is embodied as a single-dimensional independent response (for example, chicken eggs mainly excite significant changes of sulfide sensor units) and also presents a multi-dimensional cooperative response mode (for example, peanuts can cause response changes of aldehyde and ketone sensor units at the same time). Among them, there is a significant spatial correlation feature between each sensor unit of the gas sensor array, which is derived from the fact that an allergen food usually releases multiple types of organic gases, causing multiple sensor units to respond at the same time or in a specific mode. At the same time, in the time dimension, the gas concentration presents a regular dynamic evolution, typically like a single-peak time sequence pattern of first increasing and then decreasing in the peanut protein decomposition process. Therefore, it can be known that the sample data has a spatial local feature (i.e. a specific allergen may mainly activate several adjacent or related sensor units in the array) in the sensor dimension and a global feature (i.e. the time sequence mode of the entire cooking process needs to be analyzed to identify the allergen) in the time dimension. Based on the multi-label classification characteristics (input is time sequence sensor data, output is allergen probability prediction) of the allergen detection task and the low real-time requirement, the embodiments of the present application select a light-weight CNN (Convolutional Neural Network, convolutional neural network)-LSTM (Long Short-Term Memory Network, long short-term memory network) hybrid architecture to construct a deep learning model, and realize spatial local feature and time global feature extraction through effective coupling of one-dimensional convolutional neural network (1D-CNN) and long short-term memory network (LSTM). Compared with traditional machine learning methods, the deep learning model has the following advantages: 1) automatic time sequence feature learning ability, which can effectively capture complex sensor response patterns and is suitable for the characteristics of the multi-dimensional time series data of the present scheme; 2) high-precision identification performance, which can extract microscopic time sequence patterns that are difficult for humans to design and find unique gas characteristics of allergen foods; 3) strong generalization ability, which has good adaptability to unknown cooking combinations; 4) robustness advantage, which has certain fault tolerance to sensor noise.

[0046] In some optional embodiments, the allergen detection model construction process covers three stages of data preprocessing, architecture design, and training optimization. Among them, the data preprocessing link implements outlier rejection (based on 3σ criterion), moving average filtering, Z-score standardization, and dynamic window division. To solve the problem of cooking time sequence length heterogeneity (i.e., different dishes have different cooking times, resulting in different lengths of collected data sequences, affecting the uniformity of model input), an adaptive dynamic window mechanism (adjusting the input window size according to the actual length of the sequence) is used in combination with an edge zero value padding strategy (filling zeros for the insufficient part of the short sequence), and a masking mechanism is introduced at the input layer to filter the padding values.

[0047] In some optional embodiments, the model architecture adopts a hierarchical design: input layer → Conv1D layer (64 3×1 convolution kernels, ReLU activation, extracting local time sequence features, i.e., the first convolution layer) → MaxPooling1D layer (pooling size 2, i.e., the first max-pooling layer) → Dropout layer (0.5 probability, preventing overfitting, i.e., the dropout layer) → Conv1D layer (32 3×1 convolution kernels, i.e., the second convolution layer) → MaxPooling1D layer (i.e., the second max-pooling layer) → LSTM layer (64 units, tanh activation, capturing long-range dependencies) → Dense layer (multilabel classification), which is only an example and is not limited thereto.

[0048] In some optional embodiments, the step S102 comprises:

[0049] Step S1021, input the cooking conditions and gas detection data of each set of training data into the input layer, and process them through the convolution layer and the long short-term memory network layer, and output the confidence of each known allergen from the fully connected layer.

[0050] Step S1022, calculate the allergen probability of each known allergen according to the confidence of each known allergen and the mixed allergen decoupling formula.

[0051] Specifically, in the embodiment of the present application, after the input layer of the hierarchical model is input with the relevant data, the Dense layer outputs the original logit value of each known allergen (i.e. the confidence of each allergen), and then the standard case uses the sigmoid to map the output logit value to the probability, obtaining the final probability of the existence of each allergen. However, due to the problem of cross response, such as cooking marine fish, some aldehydes (the characteristic product of peanut cooking) will be produced, which will cause the confidence score of the model on the allergen of peanut to increase. If only the sigmoid function is used to convert the probability of the model output, false positives will easily occur, and it will be reported as seafood + peanut. Therefore, the embodiment of the present application cannot use the sigmoid function to independently process each type of allergen, but uses an optimized mixed allergen decoupling formula to decouple the confidence of each allergen, and the formula is as follows:

[0052]

[0053] wherein P i represents the probability of a certain allergen, S represents the sigmoid function, Z i represents the original logit output of the i-th allergen, Z j represents the logit output of other allergens, and γ represents the competitive inhibition coefficient. The physical meaning of the entire formula is:

[0054]

[0055] Through the output optimization, the interference signal can be suppressed, and the misjudgment probability of the allergen can be reduced.

[0056] Step S103, training the deep learning model based on the allergen probability of the known allergen corresponding to each set of training data and the allergen label information, to obtain an allergen detection model.

[0057] Specifically, the above step S103 comprises:

[0058] Step S1031, constructing a loss function according to the allergen probability of the known allergen and the allergen label information, and optimizing and training the deep learning model based on the loss function and a preset optimization algorithm to obtain an evaluation index.

[0059] Specifically, in the embodiment of the present application, the training stage adopts a binary cross-entropy loss function, cooperates with an Adam optimizer (initial learning rate 0.001) and an AUC-PR evaluation index, and trains the deep learning model based on the allergen probability of the allergen and the label information of the known allergen, so that the deep model learns the spatial local features and the time global features in the sample data, thereby obtaining an allergen detection model capable of accurately judging the contained allergen. After the model training is completed, the test set needs to be used to test and evaluate the allergen detection model, and then the design parameters and training strategies of the model are fine-tuned according to the test results, including adjusting the number of neurons, modifying the learning rate, changing the optimizer, etc.

[0060] In step S1032, during the optimization training process, the cooking time corresponding to the multi-dimensional time sequence data sample set is segmented based on the segmented attention mechanism, and the feature weight coefficient corresponding to each segment is adjusted during the training process.

[0061] Specifically, in the embodiment of the present application, the model iterative optimization strategy during the training process includes: based on the characteristics that food gas does not start to produce until a certain temperature and cooking time, a segmented attention mechanism (Segment Attention Mechanism) is added after the LSTM layer, which divides the entire cooking time sequence into several segments (such as the heating segment, the constant temperature segment, and the cooling segment), calculates the attention weight of the features in each time window, and thereby strengthens the feature weight of the key phase of gas release (such as the constant temperature segment).

[0062] In step S1033, the deep learning model is iteratively optimized based on the allergen probability and the allergen label information of the known allergen corresponding to the single allergen experimental sample and the allergen probability and the allergen label information of the known allergen corresponding to the mixed allergen experimental sample in turn, and an allergen detection model is obtained.

[0063] Specifically, in the embodiment of the present application, the model iterative optimization strategy during the training process further includes: implementing a curriculum learning strategy (Curriculum Learning), that is, the model training is carried out in stages: in the first stage, only single allergen sample data is used for training, so that the model preliminarily masters the basic features; in the second stage, mixed allergen sample data is introduced for training, so as to improve the recognition ability of the model in complex scenes. In addition, feature engineering enhancement is also needed, that is, on the basis of the original sensor time sequence data, time domain statistics (such as mean, variance and zero-crossing rate in a sliding window), frequency domain features (such as wavelet coefficients extracted by discrete wavelet transform DWT) and cross-sensor fusion features (such as the ratio or difference between the response values of different sensors) are additionally calculated and fused, these derived features are spliced with the original time sequence data and input into the model together, to build a more rich composite feature space, so as to improve the recognition ability of the model to complex patterns.

[0064] In some optional embodiments, after the model training is completed, a high-performance forward calculation framework is used for deployment and operation on the host of the steaming oven. In the steaming oven pop-up interaction interface, the system will set a special user feedback area to collect user's labeling information of allergens. For example, the system will guide the user to perform feedback operations by voice broadcast: "Master, I am learning food safety knowledge, do you want to help me progress". The user can set the specified allergen that actually causes an allergic reaction, and the user feedback data will be transmitted to the cloud server as training data for model update iteration. When the model is updated, the user can choose to update the model on the device to obtain the latest function. The cloud server determines the feature weight coefficient according to the specified allergen, and adjusts the parameters of the convolutional neural network and the long short-term memory network in the allergen detection model according to the feature weight coefficient. The allergen detection model performs dynamic feature enhancement, thereby directly obtaining the judgment result of the specified allergen.

[0065] The method for constructing the allergen detection model provided by the present application comprises the following steps: obtaining a multi-dimensional time sequence data sample set comprising a plurality of training data sets; analyzing the cooking conditions and the gas detection data of each training data set based on a deep learning model to obtain the allergen probability of a known allergen corresponding to an allergen label; and training the deep learning model based on the allergen probability of the known allergen corresponding to each training data set and the allergen label information to obtain an allergen detection model. The present application can comprehensively consider the spatial features and the temporal features of the generated gas in the cooking process, accurately analyze the generated gas and the change of the gas features in the cooking process, accurately detect the hidden allergen in the cooked food, adapt to various complex cooking scenes, and thus endow the cooking equipment with the function of detecting allergens, timely remind the user, avoid allergic reactions, and ensure the safety of the user's diet.

[0066] In the present embodiment, an allergen detection method is provided, which can be used in the above-mentioned cooking equipment, such as a steaming oven, Figure 2 is a flowchart of the allergen detection method according to the embodiment of the present application, as shown in Figure 2 The flowchart comprises the following steps:

[0067] In step S201, after the cooking equipment cooks the target food, multi-dimensional time sequence data of the generated comprehensive gas is obtained, and the multi-dimensional time sequence data comprises cooking conditions and gas detection data.

[0068] Specifically, in the embodiments of the present application, a gas sensor array is deployed in advance in the cooking device, and the concentration of various volatile organic gases can be detected during the cooking process as real-time gas detection data, such as aldehyde, ketone, alcohol, amine, volatile acid, aromatic hydrocarbon, sulfide, nitrogen compound, etc. At the same time, the cooking device is also deployed with an environmental sensor array, which can detect the real-time temperature, real-time humidity and other environmental parameters in the cooking device in real time. In addition, the cooking time is recorded. After the cooking is completed, based on the cooking time collected during the whole cooking process, and the real-time temperature, real-time humidity and real-time gas detection data at the time corresponding to the cooking time, a multi-dimensional time series data is constructed, and the format of the multi-dimensional time series data is: {time}x{temperature, humidity, aldehyde sensor value, ketone sensor value, organic acid sensor value…sulfide sensor value}, as shown in Figure 3 Thus, when detecting allergens, the spatial and temporal characteristics of the gases generated during the cooking process are considered comprehensively, avoiding the fragmentation of key information by traditional single-dimensional data (such as only recording the gas concentration value), and providing accurate data reference for subsequent deep learning analysis.

[0069] Step S202, based on the allergen detection model, analyzing the spatial local features and temporal global features of the multi-dimensional time series data to determine the allergen probability of different preset allergens.

[0070] Specifically, in the embodiment of the present application, after cooking is completed, the constructed multi-dimensional time series data is taken as the input of the allergen detection model. First, based on the convolutional neural network in the allergen detection model, spatial local feature analysis is performed on the multi-dimensional time series data to determine the spatial pattern related to the preset gas type and gas concentration change in the multi-dimensional time series data, and the spatial pattern is the response combination pattern of the preset sensor unit. Second, based on the long short-term memory network in the allergen detection model, time global feature analysis is performed on the multi-dimensional time series data. The LSTM in the allergen detection model sequentially arranges the spatial features extracted by the CNN, thereby performing time global feature analysis on the multi-dimensional time series data, determining the time trend of the spatial pattern, and comparing the extracted time trend with the preset cooking stage template (such as “preheating → warming → constant temperature → cooling”), thereby enhancing the context awareness of the allergen judgment. Finally, the spatial pattern and the time trend of the spatial pattern are used to determine the allergen probability corresponding to different preset allergens. Taking the scenario of cooking peanut biscuit as an example, if it is detected that sensors 3-5 (aldehyde sensors) are continuously activated for 10-15 minutes, a spatial pattern of “3-5 number sensor cooperative response” is formed, which matches the peanut allergen characteristics. The spatial pattern presents a trend of “concentration rising from 0.2 ppm to 1.5 ppm” within 10-15 minutes, which conforms to the time law of aldehyde release during peanut heating. In combination with the spatial pattern and the time trend, the allergen detection model outputs a result of “peanut allergen existence probability 92%”.

[0071] In step S203, the actual allergen contained in the target food is determined according to the allergen probability of different preset allergens.

[0072] Specifically, in the embodiment of the present application, the allergen prediction model can obtain the allergen probability corresponding to different known allergens, such as the probability corresponding to common allergens such as peanuts, nuts, seafood, milk and eggs, and then compare the allergen probability with a preset probability threshold. If the preset probability threshold is exceeded, it is proved that the corresponding known allergen is contained in the food being cooked, which is used as the actual allergen. The cooking device reminds the actual allergen contained by means of a display interface and the like, and the user can judge whether the food being cooked contains food materials that will cause allergies according to the reminder. The embodiment of the present application can reduce the implementation cost of the scheme by using existing sensors and hardware, and at the same time, since the embodiment of the present application is a non-contact detection, it is perfectly suitable for the use scenario of the steamer oven.

[0073] The allergen detection method provided by the application can detect the allergen in the food cooked by the cooking equipment, and timely remind the user to avoid the occurrence of allergy and ensure the safety of the user's diet.

[0074] In the embodiment, an allergen detection method is provided, which can be used in the cooking equipment such as the steam oven. Figure 4 The flowchart of the allergen detection method according to the embodiment of the application is shown in FIG. 2, which includes the following steps: Figure 4

[0075] In step S401, after the cooking equipment cooks the target food, the multi-dimensional time sequence data of the generated comprehensive gas is obtained, and the multi-dimensional time sequence data includes the cooking condition and the gas detection data. For details, refer to step S201 of the embodiment shown in FIG. 1, which will not be repeated here. Figure 2

[0076] In step S402, based on the allergen detection model, the spatial local feature and the time global feature of the multi-dimensional time sequence data are analyzed to determine the allergen probability of different preset allergens. For details, refer to step S202 of the embodiment shown in FIG. 1, which will not be repeated here. Figure 2

[0077] In step S403, the actual allergen contained in the target food is determined according to the allergen probability of different preset allergens.

[0078] Specifically, the above step S403 includes:

[0079] In step S4031, the allergen probability of different preset allergens is compared with the preset probability threshold to determine whether the allergen probability exceeds the preset probability threshold.

[0080] Specifically, in the embodiment of the application, the preset probability threshold corresponding to different allergens can be preset according to the detection result of the allergen detection model, and then when the allergen is detected, the allergen probability of various known allergens obtained by the allergen detection model is compared with the corresponding preset probability threshold to determine whether various known allergens are ensured in the cooked food.

[0081] ​​​Step S4032: If the probability exceeds the threshold, determine the preset allergen corresponding to the probability of the target food containing the allergen, and use the preset allergen as the actual allergen.

[0082] Specifically, in this embodiment of the invention, if the probability of a known allergen exceeds a corresponding preset probability threshold, the cooked food is determined to contain this known allergen, and it is taken as the actual allergen contained in the cooked food. Multiple known allergens may exist in the same food; for example, a seafood pizza may contain both seafood and milk, thus the detection results may contain multiple actual allergens.

[0083] Step S404: Generate an alert signal based on the actual allergen; or, compare the actual allergen with a pre-set specified allergen to determine whether the target food contains the specified allergen, and if so, generate an alarm signal based on the specified allergen.

[0084] Specifically, in this embodiment of the invention, if the user has not pre-defined a specific allergen (i.e., a known allergen that can cause an allergic reaction), such as Figure 5 As shown, after placing the ingredients in the cooking device and closing the door, the user selects the corresponding cooking mode and sets the temperature and time. Once the cooking program is started, the device activates real-time monitoring, with the corresponding sensor array tracking temperature, humidity, and gas concentration throughout the cooking process. After cooking is complete, the device sends the completed time-series data to the allergen detection model. The model determines the probability of each known allergen and returns the detection results to the device's controller. The controller then displays the detected allergens directly on the device's panel and provides voice prompts. The user can then determine whether the food being cooked is safe to eat based on the detected allergens.

[0085] In some alternative implementations, if the user pre-sets a specified allergen (i.e., an allergen known to cause an allergic reaction), such as... Figure 6 As shown, the user selects peanuts and eggs as allergens on the cooking device's control panel, which then sends the detection configuration to the controller. After starting the cooking program, the device activates real-time monitoring, with the corresponding sensor array tracking temperature, humidity, and gas concentration throughout the cooking process. Upon completion, it sends the completed time-series data to the allergen detection model. The model determines the probability of each known allergen and returns the detection results to the device's controller. The controller then compares the results with the specified allergens to determine if an allergen that could trigger an allergic reaction is present. If so, a targeted alert is generated, such as "Peanuts detected (high content)." Alternatively, as shown... Figure 6As shown, the feature weight coefficient is determined according to the specified allergen, and the convolutional neural network and the long short-term memory network in the allergen detection model are adjusted in parameters according to the feature weight coefficient, the allergen detection model performs dynamic feature enhancement, and thus the judgment result of the specified allergen is directly obtained.

[0086] The allergen detection method provided by the application can detect the gas generated in the cooking process, can endow the cooking equipment with the allergen detection function, can accurately detect the hidden allergen in the cooked food according to the gas generated by the food in the cooking process and the change of the gas characteristics with the cooking process, can timely remind the user, can avoid the allergic reaction, and can ensure the food safety of the user.

[0087] In the embodiment, a construction device of the allergen detection model is also provided, which is used for implementing the above-mentioned embodiments and preferred embodiments, and will not be described herein again. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware or a combination of software and hardware is also possible and is conceived.

[0088] The embodiment provides a construction device of an allergen detection model, which comprises: Figure 7 As shown, the construction device comprises:

[0089] The sample acquisition module 701 is configured to acquire a multi-dimensional time series data sample set, wherein the multi-dimensional time series data sample set comprises a plurality of groups of training data, and each group of training data comprises cooking conditions, gas detection data and allergen label information.

[0090] The model analysis module 702 is configured to analyze the cooking conditions and the gas detection data of each group of training data in the multi-dimensional time series data sample set based on the built deep learning model, and obtain an allergen probability of a known allergen corresponding to the allergen label.

[0091] The model training module 703 is configured to train the deep learning model based on the allergen probability of the known allergen corresponding to each group of training data and the allergen label information, and obtain an allergen detection model.

[0092] In some optional embodiments, the device further comprises a data processing module configured to perform outlier rejection, sliding average filtering, standardization processing and dynamic window division processing on the multi-dimensional time series data sample set.

[0093] In some optional embodiments, the deep learning model comprises an input layer, a convolutional layer, a long short-term memory network layer, and a fully connected layer, and the convolutional layer comprises a first convolutional layer, a first max-pooling layer, a dropout layer, a second convolutional layer, and a second max-pooling layer.

[0094] In some optional embodiments, the model analysis module 702 comprises:

[0095] a data analysis unit configured to input the cooking conditions and the gas detection data of each set of training data to the input layer, and process the same through the convolutional layer and the long short-term memory network layer, and output the confidence of each known allergen by the fully connected layer.

[0096] an allergen decoupling unit configured to calculate the allergen probability of each known allergen according to the confidence of each known allergen and a mixed allergen decoupling formula, the mixed allergen decoupling formula being as follows:

[0097]

[0098] wherein P i is the allergen probability, Z i is the confidence of the i-th known allergen, Z j is the confidence of the j-th known allergen, and γ is a competitive inhibition coefficient, and S represents a sigmoid function.

[0099] In some optional embodiments, the model training module 703 comprises:

[0100] an optimization training unit configured to construct a loss function according to the allergen probability of each known allergen and the allergen label information, and perform optimization training on the deep learning model based on the loss function and a preset optimization algorithm to obtain an evaluation index.

[0101] a weight adjustment unit configured to, in the optimization training process, segment the cooking time corresponding to the multi-dimensional time series data sample set based on a segmented attention mechanism, and adjust the feature weight coefficient corresponding to each segment in the training process.

[0102] an iterative optimization unit configured to perform iterative optimization on the deep learning model based on the allergen probability of each known allergen and the allergen label information corresponding to the single allergen experimental sample and the allergen probability of each known allergen and the allergen label information corresponding to the mixed allergen experimental sample in sequence to obtain the allergen detection model.

[0103] Further function descriptions of each module and unit described above are the same as those of the corresponding embodiments described above, and will not be repeated here.

[0104] The construction device of the allergen detection model in the embodiment is presented in the form of functional units, where the units refer to ASIC (Application Specific Integrated Circuit) circuits, processors and memories that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0105] In the embodiment, an allergen detection device is also provided, which is used to implement the above embodiments and preferred embodiments, and will not be described again. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware, or a combination of software and hardware can also be implemented and conceived.

[0106] The embodiment provides an allergen detection device, as shown in the accompanying drawings, comprising: Figure 8

[0107] The data acquisition module 801 is configured to acquire multi-dimensional time series data of the generated comprehensive gas after the cooking device cooks the target food, and the multi-dimensional time series data comprises cooking conditions and gas detection data.

[0108] The probability detection module 802 is configured to analyze spatial local features and temporal global features of the multi-dimensional time series data based on the allergen detection model, and determine allergen probabilities of different preset allergens.

[0109] The allergen determination module 803 is configured to determine actual allergens contained in the target food according to the allergen probabilities of different preset allergens.

[0110] In some optional embodiments, the allergen determination module 803 comprises:

[0111] The probability comparison unit is configured to compare the allergen probabilities of different preset allergens with a preset probability threshold, and determine whether the allergen probability exceeds the preset probability threshold.

[0112] The result determination unit is configured to, if the allergen probability exceeds the preset probability threshold, determine that the target food contains the preset allergen corresponding to the allergen probability, and determine the preset allergen as the actual allergen.

[0113] In some optional embodiments, the device further comprises a reminding module configured to generate a reminding signal according to the actual allergen, or compare the actual allergen with a designated allergen set in advance, determine whether the target food contains the designated allergen, and generate an alarm signal according to the designated allergen if the target food contains the designated allergen.

[0114] ​Further function description of each module and unit above is the same as the corresponding embodiment above, and will not be repeated here.

[0115] The allergen detection device in the embodiment is in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and a memory executing one or more software or fixed programs, and / or other devices that can provide the above functions.

[0116] The embodiment of the present application also provides a cooking device, comprising a controller, wherein the controller has the above-mentioned Figure 8 allergen detection device. Wherein the cooking device is a steam oven, but not limited to, and the steam oven is deployed with a gas sensor array, and the gas sensor array communicates with the controller.

[0117] Please refer to Figure 9 , Figure 9 is a structure schematic diagram of a controller provided by an optional embodiment of the present application, as shown in Figure 9 , the controller comprises one or more processors 10, a memory 20, and an interface for connecting various components, including a high-speed interface and a low-speed interface. Various components are communicatively connected to each other by different buses, and can be installed on a common motherboard or in other ways as needed. The processor can process instructions executed within the computer device, including instructions stored in the memory or memory to display GUI on external input / output devices (such as a display device coupled to the interface). In some optional embodiments, multiple processors and / or multiple buses can be used with multiple memories and multiple memories, if necessary. Similarly, multiple computer devices can be connected, each providing part of the necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 9 In the figure, the processor 10 is taken as an example.

[0118] The processor 10 can be a central processor, a network processor, or a combination thereof. Wherein the processor 10 can further include a hardware chip. The above-mentioned hardware chip can be an application specific integrated circuit, a programmable logic device or a combination thereof. The above-mentioned programmable logic device can be a complex programmable logic device, a field programmable logic gate array, a general array logic or any combination thereof.

[0119] Wherein the memory 20 stores instructions executable by the at least one processor 10, so that the at least one processor 10 executes the method shown in the above-mentioned embodiment.

[0120] The memory 20 can include a program storage area and a data storage area, where the program storage area can store an operating system, application programs required for at least one function, and the data storage area can store data created according to the use of the computer device, etc. In addition, the memory 20 can include a high-speed random access memory, and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some alternative embodiments, the memory 20 can optionally include a memory disposed remotely from the processor 10, which can be connected to the computer device through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0121] The memory 20 can include a volatile memory, such as a random access memory, and can also include a non-volatile memory, such as a flash memory, a hard disk, or a solid state disk, and can also include a combination of the above-mentioned types of memories.

[0122] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30, and the output device 40 can be connected by a bus or other means, Figure 9 The connection by the bus is taken as an example.

[0123] The input device 30 can receive inputted digital or character information, and generate key signal inputs related to the user settings and function controls of the computer device, such as a touch screen, a keypad, a mouse, a trackpad, a touchpad, a pointing stick, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 can include a display device, an auxiliary lighting device (e.g., an LED), a tactile feedback device (e.g., a vibration motor), etc. The display device includes, but is not limited to, a liquid crystal display, a light-emitting diode, a display, and a plasma display. In some alternative embodiments, the display device can be a touch screen.

[0124] The embodiments of the present application further provide a computer readable storage medium, and the method according to the embodiments of the present application can be implemented in hardware, firmware, or recorded in a storage medium, or stored in a remote storage medium or a non-transitory machine readable storage medium and downloaded to a local storage medium through network, so that the method described herein can be processed by such software on a storage medium using a general purpose computer, a special purpose processor, or programmable or special hardware. The storage medium can be a magnetic disk, an optical disk, a read-only memory, a random access memory, a flash memory, a hard disk, or a solid state disk, etc. Further, the storage medium can also include a combination of the above-mentioned memories. It can be understood that the computer, the processor, the microprocessor controller, or the programmable hardware includes a storage component that can store or receive software or computer code, when the software or computer code is accessed and executed by the computer, the processor, or the hardware, the method shown in the above embodiments is implemented.

[0125] Part of the present application can be applied as a computer program product, for example, computer program instructions, when executed by a computer, through the operation of the computer, the method and / or technical solutions according to the present application can be called or provided. Those skilled in the art should understand that the form of computer program instructions in a computer readable medium includes but is not limited to source files, executable files, installation package files, etc. Correspondingly, the way of executing computer program instructions by computer includes but is not limited to: the computer directly executes the instructions, or the computer compiles the instructions and then executes the corresponding compiled program, or the computer reads and executes the instructions, or the computer reads and installs the instructions and then executes the corresponding installed program. Here, the computer readable medium can be any available computer readable storage medium or communication medium accessible to the computer.

[0126] Although the embodiments of the present application are described in conjunction with the accompanying drawings, various modifications and changes can be made by those skilled in the art without departing from the spirit and scope of the present application, and such modifications and changes fall within the scope defined by the appended claims.

Claims

1. A method of constructing an allergen detection model, characterized by, The method comprises: acquiring a multi-dimensional time series data sample set, wherein the multi-dimensional time series data sample set comprises: a plurality of sets of training data, wherein each set of training data comprises: cooking conditions, gas detection data, and allergen label information; based on the deep learning model built, analyzing the cooking conditions and the gas detection data of each set of training data in the multi-dimensional time series data sample set to obtain the allergen probability of the known allergen corresponding to the allergen label; based on the allergen probability of the known allergen corresponding to each set of training data and the allergen label information, training the deep learning model to obtain an allergen detection model.

2. The method of claim 1, wherein, Before constructing the deep learning model combining the convolutional neural network and the long short-term memory network, it further comprises: performing outlier rejection, moving average filtering, standardization processing, and dynamic window division processing on the multi-dimensional time series data sample set.

3. The method of claim 1, wherein, The deep learning model comprises an input layer, a convolutional layer, a long short-term memory network layer, and a fully connected layer, and the convolutional layer comprises: a first convolutional layer, a first maximum pooling layer, a dropout layer, a second convolutional layer, and a second maximum pooling layer.

4. The method of claim 3, wherein, The deep learning model built is used to analyze the cooking conditions and the gas detection data of each set of training data in the multi-dimensional time series data sample set to obtain the allergen probability of the known allergen corresponding to the allergen label, which comprises: inputting the cooking conditions and the gas detection data of each set of training data into the input layer and processing them through the convolutional layer and the long short-term memory network layer, and outputting the confidence of each known allergen from the fully connected layer; calculating the allergen probability of each known allergen according to the confidence of each known allergen and a mixed allergen decoupling formula, which is as follows: where P i is the probability of the allergen, Z i is the confidence of the ith known allergen, Z j is the confidence of the jth known allergen, γ is the competition inhibition coefficient, and S represents the sigmoid function.

5. The method of claim 1, wherein, Based on the allergen probability of the known allergen corresponding to each set of training data and the allergen label information, the deep learning model is trained to obtain an allergen detection model, which comprises: constructing a loss function according to the allergen probability of the known allergen and the allergen label information, and optimizing and training the deep learning model based on the loss function and a preset optimization algorithm to obtain an evaluation index; During the optimization and training process, the cooking time corresponding to the multi-dimensional time series data sample set is segmented based on a segmented attention mechanism, and the feature weight coefficient corresponding to each segment is adjusted during the training process; iteratively optimizing the deep learning model based on the allergen probability of the known allergen corresponding to a single allergen experimental sample and the allergen label information and the allergen probability of the known allergen corresponding to a mixed allergen experimental sample and the allergen label information to obtain the allergen detection model.

6. An allergen detection method characterized by, Applied to a cooking device, the method comprises: after the cooking device cooks a target food, acquiring multi-dimensional time series data of the generated comprehensive gas, wherein the multi-dimensional time series data comprises: cooking conditions and gas detection data; The allergen detection model is used to analyze the spatial local features and the time global features of the multi-dimensional time series data, and determine the allergen probabilities of different preset allergens, wherein the allergen detection model is constructed based on the construction method of the allergen detection model in any one of claims 1 to 5; The actual allergen contained in the target food is determined according to the allergen probabilities of different preset allergens.

7. The method of claim 6, wherein, The actual allergen contained in the target food is determined according to the allergen probabilities of different preset allergens, including: The allergen probability of different preset allergens is compared with a preset probability threshold to determine whether the allergen probability exceeds the preset probability threshold; If the allergen probability exceeds the preset probability threshold, it is determined that the target food contains the preset allergen corresponding to the allergen probability, and the preset allergen is taken as the actual allergen.

8. The method of claim 7, wherein, The actual allergen contained in the target food is determined according to the allergen probabilities of different preset allergens, further including: An alert signal is generated according to the actual allergen; Or, the actual allergen is compared with a pre-set specified allergen to determine whether the target food contains the specified allergen, and an alarm signal is generated according to the specified allergen if the target food contains the specified allergen.

9. A device for constructing an allergen detection model, characterized in that, The device includes: A sample acquisition module is configured to acquire a multi-dimensional time series data sample set, wherein the multi-dimensional time series data sample set includes a plurality of training data sets, and each training data set includes cooking conditions, gas detection data, and allergen label information; A model analysis module is configured to analyze the cooking conditions and the gas detection data of each training data set in the multi-dimensional time series data sample set based on a constructed deep learning model to obtain the allergen probabilities of known allergens corresponding to the allergen labels; A model training module is configured to train the deep learning model based on the allergen probabilities of known allergens corresponding to each training data set and the allergen label information to obtain an allergen detection model.

10. An allergen detection device, characterized in that The device is applied to a cooking device, and the device includes: A data acquisition module is configured to acquire multi-dimensional time series data of a generated comprehensive gas after the cooking device cooks a target food, wherein the multi-dimensional time series data includes cooking time, real-time temperature, real-time humidity, and real-time gas detection data at a time corresponding to the cooking time; A probability detection module is configured to analyze the spatial local features and the time global features of the multi-dimensional time series data based on an allergen detection model to determine the allergen probabilities of different preset allergens, wherein the allergen detection model is constructed based on the construction device of the allergen detection model in claim 9; An allergen determination module is configured to determine the actual allergen contained in the target food according to the allergen probabilities of different preset allergens.

11. A cooking apparatus, characterized by, It includes: A controller; The controller includes a memory and a processor, which are communicatively connected to each other, and the memory stores computer instructions, and the processor executes the computer instructions to implement the allergen detection method in any one of claims 6 to 8.

12. The cooking apparatus according to claim 11, wherein, The cooking apparatus is a steam oven, and the steam oven further comprises: a gas sensor array for detecting a gas in the steam oven.

Images