An operation control method and system for a heat preservation food warmer based on artificial intelligence

Abstract

The application provides an operation control method and system for a heat preservation food stove based on artificial intelligence, the method comprising: when a food tray of the heat preservation food stove is placed with food, controlling a preset camera module to shoot the food on the food tray to obtain a first food image; receiving the first food image sent by the camera module, pre-processing the first food image to obtain a second food image; calling a pre-trained food image analysis model to analyze the second food image, identifying a target dish type to which the food in the second food image belongs and determining an optimal heat preservation temperature of the target dish type, and setting the heat preservation temperature of the heat preservation food stove as the optimal heat preservation temperature. The application realizes automatic and intelligent control of the heat preservation food stove temperature in an artificial intelligence mode, the whole process does not need manual intervention, and the operation convenience and efficiency are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent control of heat preservation chafing dishes, and in particular to a heat preservation chafing dish operation control method and system based on artificial intelligence. BACKGROUND

[0002] A heat preservation chafing dish is a device used to maintain food at a set temperature, commonly used in the catering industry, such as schools, hospitals, company canteens, fast food restaurants, etc., to ensure that food remains at a suitable temperature for consumption over a long period of time.

[0003] A heat preservation chafing dish usually uses an electric heating element to provide heat and a temperature sensor to monitor the temperature inside the chafing dish. When the temperature is below the set temperature, the heating element starts to work to heat the food to the set temperature; when the temperature reaches or exceeds the set temperature, the heating element stops working to maintain the temperature of the food.

[0004] It is well known that different types of dishes have their optimal preservation temperature to maintain their flavor, texture and nutrition. For example, the optimal preservation temperature of hot dishes and cold dishes is obviously different. Currently, the method of setting the optimal preservation temperature for different types of dishes is to build a control table of different types of dishes and optimal preservation temperatures. After the food is placed in the chafing dish, the optimal preservation temperature of the dish type is determined based on the control table, and the optimal preservation temperature of the chafing dish is manually set to the optimal preservation temperature. However, the control table is difficult to cover all types of dishes, especially some special or uncommon dishes; at the same time, this method requires manual identification of dish types and searching for the control table, which is prone to errors, especially in a busy catering environment, and the manual setting of the optimal preservation temperature is inefficient and low in intelligence. Therefore, the current heat preservation chafing dish cannot accurately, efficiently and intelligently set the optimal preservation temperature of the dish type.

[0005] For example, the current patent application number 202311663276.3 discloses a temperature control heat preservation chafing dish control system based on artificial intelligence, which has a control table of dishes and target preservation temperature ranges, and compares the lower temperature of the dish with the lowest temperature of the target preservation temperature range to determine the preservation type of the dish, including cooling and heating, to ensure that the food is kept at the most suitable temperature. However, this technical solution uses a control table that is difficult to cover all types of dishes, and requires manual identification of dish types and searching for the control table, which is tedious and prone to errors. SUMMARY

[0006] The present application provides a heat preservation chafing dish operation control method and system based on artificial intelligence to accurately, efficiently and intelligently set the optimal preservation temperature of food in the chafing dish.

[0007] To solve the above problems, the application adopts the following technical solutions:

[0008] The application provides a heat preservation meal stove operation control method based on artificial intelligence, comprising:

[0009] When the food is placed on the meal plate of the heat preservation meal stove, a preset camera module is controlled to shoot the food on the meal plate to obtain a first food image;

[0010] The first food image sent by the camera module is received, and the first food image is preprocessed to obtain a second food image;

[0011] A pre-trained food image analysis model is called to analyze the second food image, and a target dish type to which the food in the second food image belongs and an optimal heat preservation temperature of the target dish type are identified, wherein the food image analysis model is a convolutional neural network, and is used to identify a dish type in a food image and determine an optimal heat preservation temperature of the dish type;

[0012] The heat preservation temperature of the heat preservation meal stove is set to the optimal heat preservation temperature.

[0013] Further, before the pre-trained food image analysis model is called to analyze the second food image and identify the target dish type to which the food in the second food image belongs and the optimal heat preservation temperature of the target dish type, the method further comprises:

[0014] Collecting an image data set of different dish types, wherein the image data set contains historical food images of multiple dish types and optimal heat preservation temperatures of each dish type;

[0015] Preprocessing the image data set to obtain a standard image data set;

[0016] Training a convolutional neural network using the standard image data set, and calculating a loss value of the trained convolutional neural network based on a preset loss function;

[0017] When the loss value is lower than a preset loss value, the trained convolutional neural network is used as a food image analysis model.

[0018] Preferably, the preprocessing of the image data set to obtain a standard image data set comprises:

[0019] Each historical food image of the image data set is subjected to format unification, size adjustment, color space conversion, standardization and data enhancement processing to obtain multiple standard food images;

[0020] The multiple standard food images are combined to form a standard image data set.

[0021] Further, after calculating the loss value of the trained convolutional neural network based on the preset loss function, the method further comprises:

[0022] When the loss value is not lower than a preset loss value, calculating a first gradient of the preset loss function with respect to a predicted output of the convolutional neural network and a second gradient of an output layer activation function with respect to an input of the convolutional neural network based on the loss value;

[0023] After multiplying the first gradient and the second gradient, an output gradient of the preset loss function with respect to an output layer of the convolutional neural network is calculated;

[0024] A third gradient of the output layer of the convolutional neural network with respect to a hidden layer output is calculated;

[0025] A fourth gradient of a hidden layer activation function of the convolutional neural network with respect to an input is calculated;

[0026] After multiplying the output gradient, the third gradient and the fourth gradient, a target gradient is obtained;

[0027] After multiplying each model parameter of the convolutional neural network with the target gradient, a plurality of target model parameters of the convolutional neural network are obtained;

[0028] The model parameters of the convolutional neural network are updated based on the plurality of target model parameters;

[0029] The convolutional neural network with the updated model parameters is trained again using the standard image dataset until the loss value is lower than the preset loss value, and a trained food image analysis model is obtained.

[0030] Further, after setting the holding temperature of the holding food warmer to the optimal holding temperature, the method further comprises:

[0031] Setting a stirring time of the food;

[0032] When the stirring time is reached, a preset stirring module is controlled to stir the food on the food tray.

[0033] Further, after setting the holding temperature of the holding food warmer to the optimal holding temperature, the method further comprises:

[0034] A preset camera module is controlled to capture the food on the food tray in real time, and a third food image is obtained;

[0035] The third food image sent by the camera module is received;

[0036] Call the food image analysis model to analyze the third food image and generate an analysis result;

[0037] When it is determined according to the analysis result that there is no food on the tray, a food adding reminder is sent to a terminal of a staff, and the heat preservation function of the heat preservation oven is closed.

[0038] Further, after setting the heat preservation temperature of the heat preservation oven to the optimal heat preservation temperature, the method further comprises:

[0039] Obtaining temperature data of food inside the heat preservation oven;

[0040] Extracting feature information of the temperature data to obtain temperature features, the temperature features including temperature deviation, deviation change rate and integral absolute error;

[0041] Inputting the temperature features and the optimal heat preservation temperature into a pre-trained neural network model to output optimal PID parameters;

[0042] Adjusting parameters of a PID controller of the heat preservation oven based on the optimal PID parameters, and recording the adjusted parameters of the PID controller of the heat preservation oven and system response.

[0043] Preferably, the calling of the pre-trained food image analysis model to analyze the second food image comprises:

[0044] Calling a pre-trained food image analysis model to extract a plurality of key features of the second food image;

[0045] Identifying a plurality of food material types and a target cooking method of the second food image based on the plurality of key features of the second food image;

[0046] Determining a target dish type of food in the second food image based on the plurality of food material types and the target cooking method;

[0047] When it is determined that the target dish type is a new dish type, calculating a pixel proportion of each food material type in the second food image to obtain a weight of each food material type;

[0048] Respectively querying an optimal heat preservation temperature corresponding to each food material type under the target cooking method;

[0049] Respectively weighting and summing the optimal heat preservation temperature of each food material type and the corresponding weight to obtain the optimal heat preservation temperature of the target dish type.

[0050] Further, after determining the target dish type to which the food in the second food image belongs based on the plurality of food material categories and the target cooking method, the method further includes:

[0051] When it is determined that the target dish type is a non-new dish type, an optimal holding temperature of the target dish type is queried from a database.

[0052] The application provides an intelligent holding oven operation control system, comprising:

[0053] A control module is configured to control a preset camera module to capture food on a dish of the holding oven to obtain a first food image after the dish is placed with food.

[0054] A preprocessing module is configured to receive the first food image sent by the camera module, pre-process the first food image, and obtain a second food image.

[0055] An analysis module is configured to call a pre-trained food image analysis model to analyze the second food image, identify a target dish type to which the food in the second food image belongs, and determine an optimal holding temperature of the target dish type.

[0056] A setting module is configured to set the holding temperature of the holding oven to the optimal holding temperature.

[0057] Compared with the prior art, the technical scheme of the application has at least the following advantages:

[0058] The intelligent holding oven operation control method and system provided by the application receive the first food image sent by the camera module, pre-process the first food image to obtain a second food image, call a pre-trained food image analysis model to analyze the second food image, identify a target dish type to which the food in the second food image belongs, and determine an optimal holding temperature of the target dish type, and set the holding temperature of the holding oven to the optimal holding temperature. The food image is analyzed by using a convolutional neural network, the dish type can be accurately identified, the optimal holding temperature of the dish type can be determined, and the holding temperature of the holding oven can be automatically set to the optimal holding temperature. Thus, the temperature of the holding oven is automatically and intelligently controlled by using artificial intelligence, the entire process does not require manual intervention, and the operation convenience and efficiency are improved. BRIEF DESCRIPTION OF DRAWINGS

[0059] Figure 1 An embodiment flowchart of the intelligent holding oven operation control method based on artificial intelligence is provided.

[0060] Figure 2 A flow chart of another embodiment of the operation control method of the heat preservation food warmer based on artificial intelligence;

[0061] Figure 3 A flow chart of another embodiment of the operation control method of the heat preservation food warmer based on artificial intelligence;

[0062] Figure 4 A structure block diagram of an embodiment of the operation control system of the heat preservation food warmer based on artificial intelligence. DETAILED DESCRIPTION

[0063] In order to make the person skilled in the art better understand the scheme of the present application, the technical scheme in the embodiments of the present application will be described clearly and completely below in combination with the drawings in the embodiments of the present application.

[0064] In some of the descriptions in the specification and claims of the present application and the above-mentioned drawings, a plurality of operations appearing in a specific order are included, but it should be clearly understood that these operations can be executed or in parallel without the order appearing in the text, and the serial numbers of the operations such as S11, S12, etc. are only used to distinguish different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes can include more or fewer operations, and the operations can be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second" and the like in the text are used to distinguish different messages, devices, modules, etc., and do not represent the order of precedence, nor do "first" and "second" represent different types.

[0065] Those skilled in the art can understand that, unless specifically stated, the singular forms "a", "an" and "the" used herein also include the plural forms. It should be further understood that the use of the phrase "comprising" in the specification of the present application means that the features, integers, steps, operations, elements and / or components exist, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we say that an element is "connected" or "coupled" to another element, it can be directly connected or coupled to the other element, or there can be intermediate elements. In addition, the "connection" or "coupling" used herein can include wireless connection or wireless coupling. The phrase "and / or" used herein includes all or any single unit and all combinations of the associated listed items.

[0066] As will be understood by one of ordinary skill in the art upon reading the present disclosure, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art in the field of the application, unless otherwise defined. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

[0067] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application, wherein the same or similar reference signs represent the same or similar elements or elements having the same or similar functions throughout. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0068] Please refer to Figure 1 The present application provides an operation control method of a thermal food warmer based on artificial intelligence, comprising the following steps:

[0069] S11, when the food tray of the thermal food warmer is placed with food, a preset camera module is controlled to shoot the food on the food tray to obtain a first food image;

[0070] S12, receiving the first food image sent by the camera module, pre-processing the first food image to obtain a second food image;

[0071] S13, calling a pre-trained food image analysis model to analyze the second food image, identifying the target dish type to which the food in the second food image belongs and determining the optimal holding temperature of the target dish type, wherein the food image analysis model is a convolutional neural network, which is used to identify the dish type in the food image and determine the optimal holding temperature of the dish type;

[0072] S14, setting the holding temperature of the thermal food warmer to the optimal holding temperature.

[0073] The embodiment can integrate a camera module beside the heat preservation food warmer. When the food is placed on the food tray of the heat preservation food warmer, the camera module is automatically controlled to capture the food on the food tray to obtain a first food image. Alternatively, when the food tray is placed in the heat preservation food warmer, the camera module is automatically triggered to capture the food on the food tray to obtain visual information of the food for subsequent image analysis. The position of the camera module needs to be designed to consider the influence of water vapor generated by the food in the heat preservation food warmer to avoid blurred images. For example, the camera module can be installed obliquely above the heat preservation food warmer to minimize the influence of water vapor in the food while ensuring that a complete image of the food is captured.

[0074] The food image captured by the camera module is received and preprocessed to form a second food image. The preprocessing methods can include adjusting the image size, cropping, rotation correction, color correction, normalization, etc. to ensure that the image is suitable for input into the convolutional neural network, thereby improving the recognition accuracy and robustness of the model.

[0075] Then, the system calls a pre-trained food image analysis model to analyze the second food image, identifies the target dish type to which the food in the second food image belongs, and determines the optimal heat preservation temperature of the target dish type. For example, a pre-trained convolutional neural network can be used to analyze the preprocessed second food image. The model can identify the dish type in the image and determine the optimal heat preservation temperature for each dish type based on the knowledge learned from the training data, thereby achieving intelligent recognition of food and determination of the optimal heat preservation temperature.

[0076] Finally, according to the optimal heat preservation temperature determined by the food image analysis model, the temperature setting of the heat preservation food warmer is automatically adjusted to ensure that the food can be kept at the best temperature, thereby achieving intelligent control of the heat preservation food warmer and improving the heat preservation effect of the food.

[0077] For example, assuming that a canteen uses the technical solution to manage food in a food warmer. When the staff puts a dish containing dishes into the warmer, the system controls the camera module to automatically take pictures of the food, obtains the image of the food, and ensures that the image quality is suitable for deep learning analysis. After the image quality is qualified, it is identified that the food in the dish is "stewed pork" or "cold cucumber salad" based on the food image, and it is determined that the optimal holding temperature of the stewed pork is 60 DEG C and the optimal holding temperature of the cold cucumber salad is 10 DEG C according to the model training data. The holding temperature of the warmer is automatically adjusted based on the identification result, the holding temperature of the stewed pork area is set to 60 DEG C, and the holding temperature of the cold cucumber area is set to 10 DEG C, so that the demand for manual temperature adjustment is reduced through the automatic temperature control mode, the work efficiency is improved, and the holding temperature of each dish type is accurately controlled to ensure that the food is supplied in the best state, improve the food quality and taste, and improve the dining experience of customers.

[0078] The method for controlling the operation of the food warmer based on artificial intelligence provided by the application comprises the following steps: receiving a first food image sent by a camera module, pre-processing the first food image to obtain a second food image, calling a pre-trained food image analysis model to analyze the second food image, identifying the target dish type to which the food in the second food image belongs and determining the optimal holding temperature of the target dish type, and setting the holding temperature of the food warmer to the optimal holding temperature. By using a convolutional neural network to analyze the food image, the dish type can be accurately identified and the optimal holding temperature of the dish type can be determined, and the holding temperature of the food warmer can be automatically set to the optimal holding temperature, so that the automatic and intelligent control of the holding temperature of the food warmer is realized through artificial intelligence, the entire process does not require manual intervention, and the operation convenience and efficiency are improved.

[0079] In one embodiment, before the calling of the pre-trained food image analysis model to analyze the second food image, identifying the target dish type to which the food in the second food image belongs and determining the optimal holding temperature of the target dish type, the method further comprises the following steps:

[0080] Collecting image data sets of different dish types, the image data sets containing historical food images of multiple dish types and the optimal holding temperature of each dish type;

[0081] Pre-processing the image data sets to obtain a standard image data set;

[0082] Training a convolutional neural network using the standard image data set, and calculating the loss value of the trained convolutional neural network based on a preset loss function;

[0083] When the loss value is lower than a preset loss value, the trained convolutional neural network is used as a food image analysis model.

[0084] The embodiment collects an image dataset containing historical food images of multiple dish types. The image dataset is the basis for training the convolutional neural network and can contain historical food images of various dish types and corresponding labels, i.e., the optimal holding temperature for each dish type, which can be determined by historical experience or artificial setting.

[0085] Next, the collected image dataset is preprocessed, such as adjusting the size of each historical food image in the image dataset, normalization, etc., to form a standard image dataset to meet the needs of model training. Then a convolutional neural network is constructed, which can include multiple convolutional layers, pooling layers, fully connected layers, etc. The loss function and optimizer of the convolutional neural network are defined, the preprocessed standard image dataset is used to train the convolutional neural network, and the model is iteratively trained through the forward propagation and back propagation algorithms until the model performance reaches a satisfactory level. During the training process, the model learns how to identify different dish types from the images and determine the optimal holding temperature corresponding to the dish types.

[0086] In the embodiment, during the training of the convolutional neural network, a preset loss function (such as cross-entropy loss function) can be used to calculate the loss value of the model to evaluate the performance of the model. When the loss value of the model is lower than the preset loss value, it means that the model has been trained well enough and can be used as a food image analysis model in actual application scenarios.

[0087] The embodiment can improve the quality and effectiveness of convolutional neural network training by constructing an image dataset containing historical food images of multiple dish types and the optimal holding temperature for each dish type, and preprocessing to obtain a standard image dataset, thereby improving the accuracy of food image recognition. At the same time, preprocessing and data augmentation of the image dataset, such as size unification, color space conversion, rotation, flipping, scaling, and cropping, can effectively improve the model's ability to recognize different food image perspectives and details, and enhance the model's generalization ability. Secondly, using a loss function and evaluation index suitable for multi-class classification, such as cross-entropy loss and accuracy, ensures the effectiveness of the classification task. At the same time, by calculating the loss value and comparing it with the preset loss value, the progress and effectiveness of the model training can be effectively monitored. In addition, the constructed model not only can recognize dishes in the training dataset, but also can generalize to new dish categories not included in the training dataset, providing users with dish ingredients and cooking method information for reference. In summary, the technical solution realizes efficient recognition and analysis of food images through deep learning and convolutional neural network technology, improves the accuracy of recognition and the generalization ability of the model, and optimizes the model training and evaluation process,

[0088] In one embodiment, the pre-processing of the image dataset to obtain a standard image dataset includes:

[0089] Each historical food image in the image dataset is subjected to format unification, size adjustment, color space conversion, standardization, and data enhancement processing to obtain multiple standard food images.

[0090] The multiple standard food images are combined into a standard image dataset.

[0091] In this embodiment, format unification ensures that all image files are in the same format, facilitating subsequent processing. For example, historical food images in different formats (such as JPEG, PNG, BMP, etc.) can be converted to a unified format, such as all being converted to JPEG format.

[0092] Size adjustment is to unify the size of the images to meet the input requirements of the model, i.e., to adjust all historical food images to the same size. For example, all historical food images are adjusted to 224x224 pixels.

[0093] Color space conversion is to convert the image from one color space to another to meet the needs of model training. For example, each historical food image in the RGB color space is converted to a grayscale image, or RGB is converted to the HSV color space to better capture color information.

[0094] Standardization is to eliminate the scale difference in the image data to make the model training more stable. For example, the pixel values of each historical food image can be scaled from [0, 255] to [0, 1], or more complex standardization methods such as subtracting the mean and dividing by the standard deviation can be used.

[0095] Data enhancement processing is to increase the diversity of the dataset by generating variant images to improve the generalization ability of the model. Data enhancement methods include rotation, flipping, scaling, cropping, adding noise, etc. to generate new versions of the images.

[0096] Finally, the multiple standard food images subjected to the above series of preprocessing methods are integrated into a new dataset to form a standard image dataset for model training. All processed images are stored in a folder or organized in the form of a database to ensure data accessibility and consistency.

[0097] In one embodiment, after calculating the loss value of the trained convolutional neural network based on the pre-set loss function, the method further includes:

[0098] when the loss value is not lower than a preset loss value, calculating a first gradient of a preset loss function with respect to a prediction output of the convolutional neural network and a second gradient of an output layer activation function with respect to an input of the convolutional neural network based on the loss value;

[0099] multiplying the first gradient and the second gradient to obtain an output gradient of the loss function with respect to an output layer of the convolutional neural network;

[0100] calculating a gradient of the output layer of the convolutional neural network with respect to a hidden layer output to obtain a third gradient;

[0101] calculating a gradient of the hidden layer activation function of the convolutional neural network with respect to the input to obtain a fourth gradient;

[0102] multiplying the output gradient, the third gradient and the fourth gradient to obtain a target gradient;

[0103] multiplying each model parameter of the convolutional neural network with the target gradient to obtain a plurality of target model parameters of the convolutional neural network;

[0104] updating the model parameters of the convolutional neural network based on the plurality of target model parameters;

[0105] retraining the convolutional neural network with the updated model parameters using the standard image dataset until the loss value is lower than the preset loss value to obtain a trained food image analysis model.

[0106] The embodiment can first determine the difference between the model prediction output and the actual output, and how this difference affects the input of the model. When the loss value is not lower than the preset loss value, the first gradient of the loss function with respect to the prediction output of the convolutional neural network and the second gradient of the output layer activation function with respect to the network input are calculated using the backpropagation algorithm.

[0107] The first gradient and the second gradient are combined to calculate the gradient of the loss function with respect to the output layer of the convolutional neural network. For example, the first gradient and the second gradient can be multiplied to obtain an output gradient, which represents the sensitivity of the loss function to the output layer of the convolutional neural network.

[0108] Then, the gradient of the output layer of the convolutional neural network with respect to the hidden layer output is calculated, and the gradient of the hidden layer activation function with respect to the input is calculated. For example, the third gradient can be obtained by calculating how the loss of the output layer affects the output of the hidden layer through the backpropagation algorithm. For the activation function of the hidden layer, the gradient is calculated to obtain the fourth gradient, which represents the sensitivity of the activation function to the input.

[0109] The output gradient, the third gradient, and the fourth gradient are combined to obtain a target gradient. For example, the output gradient, the third gradient, and the fourth gradient can be multiplied to obtain the target gradient, which will be used to update the model parameters of the convolutional neural network.

[0110] Preferably, after multiplying the output gradient, the third gradient, and the fourth gradient to obtain the target gradient, the output gradient, the third gradient, and the fourth gradient can be multiplied to obtain an initial gradient, and the initial gradient is clipped based on the value-based gradient clipping method to obtain the target gradient, including the following formula:

[0111]

[0112] Among them, the target gradient is g, the initial gradient is g, the preset positive number is c, and the maximum absolute value of gradient clipping is represented.

[0113] The value-based gradient clipping method can directly limit the size of the gradient value, ensuring that the gradient value of each parameter does not exceed a specified range. If the gradient value exceeds this range, it will be set to the boundary value of this range, thereby preventing gradient explosion.

[0114] When updating the model parameters according to the target gradient, each model parameter of the convolutional neural network can be multiplied by the target gradient to obtain a target model parameter corresponding to each model parameter, and the model parameters of the convolutional neural network are updated according to the target model parameter. For example, the weights and biases of the convolutional neural network can be updated according to the target model parameter using an optimization algorithm such as SGD, Adam, etc.

[0115] The updated convolutional neural network continues to train the model parameters until the loss value of the convolutional neural network is lower than the preset loss value, and a trained food image analysis model is obtained. For example, the updated convolutional neural network can be retrained using a standard image dataset, and iterated until the model performance reaches a satisfactory level.

[0116] Specifically, taking a two-layer convolutional neural network as an example, the forward propagation can be represented as:

[0117] z1=W1x+b1;

[0118] a1=f(z1);

[0119] z2=W2a1+b2;

[0120] a2=f(z2);

[0121] ​Wherein, W1, W2 are weights of the convolutional neural network, b1, b2 are biases of the convolutional neural network, f is an activation function of the convolutional neural network, x is an input of the convolutional neural network, and a2 is an output of the convolutional neural network.

[0122] First, the difference between the predicted output a2 and the true label y is calculated using the loss function L:

[0123] L = L(a2, y);

[0124] Then, the output gradient of the loss function with respect to the output layer of the convolutional neural network is calculated:

[0125]

[0126] Wherein, is the output gradient of the loss function with respect to the output layer of the convolutional neural network, is the first gradient of the loss function with respect to the predicted output of the convolutional neural network, is the second gradient of the output layer activation function with respect to the input of the convolutional neural network.

[0127] Next, the target gradient is calculated based on the output gradient, the third gradient and the fourth gradient:

[0128]

[0129] Wherein, is the target gradient, is the output gradient of the loss function with respect to the output layer of the convolutional neural network, is the third gradient of the output layer of the convolutional neural network with respect to the hidden layer output, is the fourth gradient of the hidden layer activation function of the convolutional neural network with respect to the input.

[0130] Finally, each model parameter of the convolutional neural network is multiplied by the target gradient to obtain multiple target model parameters of the convolutional neural network, and the model parameters of the convolutional neural network are updated based on the multiple target model parameters.

[0131] The embodiment can improve the accuracy of model recognition of food images through fine gradient calculation and parameter updating. The trained food image analysis model can adapt to different images and environments and has good generalization ability. At the same time, iterative training and parameter updating help optimize model performance and make it converge to the best state faster. In addition, precise control of parameter updating and training process can save computing resources and time.

[0132] In one embodiment, as shown in FIG. 1, after the heat preservation temperature of the heat preservation food stove is set to the optimal heat preservation temperature, the method further comprises: Figure 2 setting the heat preservation temperature of the heat preservation food stove to the optimal heat preservation temperature.

[0133] S15, set the stirring time of the food;

[0134] S16, when the stirring time is reached, control the preset stirring module to stir the food on the tray.

[0135] The embodiment can determine the optimal stirring time of the food to ensure that the food is evenly heated during heating and also ensure that the food is evenly mixed and maintains the ideal taste and temperature.

[0136] The stirring module is started at the precise stirring time to ensure that the food is stirred for the preset time, avoiding over-stirring or under-stirring. For example, the system can use a timer or program to control the stirring module (such as an electric stirring rod) to start working at the preset time to stir the food in the tray. The stirring module can be an automatic control module integrated in the heat preservation chafing dish.

[0137] For example, the system can set the stirring time of braised pork to 2 minutes to ensure that the food is evenly heated during heating. After the braised pork is prepared and placed in the tray of the heat preservation chafing dish, the intelligent timer of the heat preservation chafing dish is started, and when the intelligent timer reaches 2 minutes, the stirring module starts to work to evenly stir the braised pork. During the stirring process, the system can also monitor the stirring speed and intensity to ensure that the food is not over-stirred, thereby damaging the texture of the food. The stirring module can be an automatic stirring arm integrated in the heat preservation chafing dish or an electric stirring rod synchronized with the intelligent timer.

[0138] The automated stirring process of the embodiment reduces manual operation, saves time and labor, the preset stirring time ensures consistency of each stirring, improves the quality and taste of the food, and also ensures that the food is evenly heated.

[0139] In one embodiment, referring to Figure 3 the heat preservation temperature of the heat preservation chafing dish is set to the optimal heat preservation temperature, the method further includes:

[0140] S17, real-time control the preset camera module to shoot the food on the tray to obtain a third food image;

[0141] S18, receive the third food image sent by the camera module;

[0142] S19, call the food image analysis model to analyze the third food image to generate an analysis result;

[0143] S20, when it is determined according to the analysis result that there is no food on the tray, send an add food reminder to the terminal of the staff, and close the heat preservation function of the heat preservation chafing dish.

[0144] The embodiment can capture the state of food on the tray in real time through the camera module, so as to perform subsequent image analysis. For example, a camera connected to the system is started, and the food on the tray is photographed to obtain the current image of the food, and ensure that each time the clear and appropriate angle image is obtained.

[0145] Then, the food image photographed by the camera is transmitted to the system, and the system receives the third food image sent by the camera module for further processing. Among them, the image data can be transmitted to the system through wireless or wired network, or directly stored in the local storage device for subsequent analysis.

[0146] Next, the third food image is analyzed by using a deep learning model to identify the type, quantity and other information of the food. For example, the received third food image is input into a pre-trained food image analysis model, and the model analyzes the third food image according to the image features and outputs the analysis results such as the type and remaining amount of the food.

[0147] When it is determined based on the analysis result that the food on the tray is consumed, a food adding reminder is sent to the terminal of the staff to timely remind the staff to supplement the food, and the heat preservation function is stopped to save energy. For example, when the analysis result shows that there is no food on the tray, the system will automatically send a reminder message to the mobile device or workstation of the staff. At the same time, the system will send an instruction to the heat preservation food warmer to turn off the heat preservation function.

[0148] For example, assume that a self-service restaurant uses the technical solution to manage the food supply on the tray. When the food on the tray is taken by the customer, the camera installed above the tray automatically photographs the current state of the food. The camera sends the photographed image to the system, and the food image analysis model on the system analyzes the image to identify the type and quantity of the remaining food on the tray. If the analysis result shows that there is no food on the tray, the system will send a reminder to the mobile phone or workstation of the staff of the restaurant, prompting them to supplement the food. At the same time, the system will automatically turn off the heat preservation function under the tray to save energy.

[0149] The automatic food monitoring and supplement reminder of the embodiment reduces the staff's patrol work and improves the work efficiency. At the same time, by monitoring the food consumption in real time, the excessive preparation of food can be avoided, thereby reducing waste. In addition, when there is no food on the tray, the heat preservation function is turned off, which can save energy consumption and reduce operating costs.

[0150] In one embodiment, after setting the heat preservation temperature of the heat preservation food warmer to the optimal heat preservation temperature, the method further comprises:

[0151] acquire temperature data of food inside the heat preservation oven;

[0152] extract feature information of the temperature data to obtain temperature features, including temperature deviation, deviation change rate, and integral absolute error;

[0153] input the temperature features and the optimal holding temperature into a pre-trained neural network model to output optimal PID parameters;

[0154] adjust parameters of the PID controller of the heat preservation oven based on the optimal PID parameters, and record the adjusted parameters of the PID controller of the heat preservation oven and the system response.

[0155] In this embodiment, the system monitors the current temperature of the heat preservation oven in real time to control and adjust the temperature. For example, the temperature data of the food inside the heat preservation oven can be acquired in real time by a temperature sensor or other data acquisition device, and the temperature data is sent to the system for response.

[0156] Then, key features are extracted from the original temperature data for subsequent analysis and control. For example, the temperature data is analyzed to extract features such as temperature deviation (difference between actual temperature and set temperature), deviation change rate (change speed of temperature deviation over time), and integral absolute error (cumulative temperature deviation).

[0157] The neural network model is used to calculate the optimal PID control parameters according to the current temperature features and the optimal holding temperature, i.e., the extracted temperature features and the preset optimal holding temperature are input into the pre-trained neural network model to obtain the adjusted PID controller parameters, so as to achieve more accurate temperature control. For example, the neural network model can output the adjusted optimal PID parameters, including proportional (Kp), integral (Ki), and derivative (Kd) parameters, according to the input temperature features and the optimal holding temperature.

[0158] According to the optimal PID parameters provided by the neural network model, the PID controller of the heat preservation oven is adjusted to optimize the temperature control performance. For example, the optimal PID parameters output by the neural network model can be applied to the PID controller of the heat preservation oven to replace the original parameters or as fine tuning.

[0159] The adjusted PID parameters and the temperature response of the system are recorded for subsequent analysis and optimization, i.e., after adjusting the PID parameters, the temperature change of the heat preservation oven and the response of the PID controller are monitored and recorded to monitor the temperature response state of the heat preservation oven in real time.

[0160] For example, assume that a food warmer needs to be maintained at an optimal warming temperature, such as 60°C. The system monitors the actual temperature of the food warmer in real time through a temperature sensor and finds that the actual temperature is 58°C. The extracted temperature features include: temperature deviation (58°C-60°C=-2°C), rate of change of deviation (previous deviation is-1°C, rate of change is-1°C), and integrated absolute error (absolute value of cumulative temperature deviation). These temperature features and the optimal warming temperature 60°C are input into the neural network model, and the model outputs adjusted PID parameters, such as Kp=1.2, Ki=0.05, and Kd=0.1. According to the PID parameters output by the model, the PID controller of the food warmer is adjusted to more accurately control the temperature. After adjustment, the new PID parameters and the temperature change of the food warmer are recorded, such as the temperature gradually stabilizing at 60°C.

[0161] The embodiment can analyze temperature features and optimal warming temperatures in different situations through a neural network model, accurately determine optimal PID parameters of a PID controller, and adjust the parameters of the PID controller to the optimal PID parameters in real time, so that the temperature of the food warmer can be more accurately maintained and temperature fluctuations can be reduced. In addition, the optimized PID control can reduce energy waste and improve energy utilization efficiency. The entire temperature control process does not require manual intervention, and intelligent and automated temperature control is achieved.

[0162] In one embodiment, the calling of the pre-trained food image analysis model to analyze the second food image to identify the target dish type to which the food in the second food image belongs and determine the optimal warming temperature of the target dish type comprises:

[0163] The pre-trained food image analysis model is called to extract a plurality of key features of the second food image;

[0164] Based on the plurality of key features of the second food image, a plurality of food material types and a target cooking method of the second food image are identified;

[0165] Based on the plurality of food material types and the target cooking method, the target dish type to which the food in the second food image belongs is determined;

[0166] When it is determined that the target dish type is a new dish type, the weight of each food material type in the second food image is calculated based on the pixel proportion of each food material type in all food material types, and the weight of each food material type is obtained;

[0167] The optimal warming temperature corresponding to each food material type under the target cooking method is queried respectively;

[0168] The optimal warming temperature of each food material type is weighted and summed with the corresponding weight to obtain the optimal warming temperature of the target dish type.

[0169] In this embodiment, a plurality of key features are identified from the food image, which are helpful for subsequent identification of food material categories and cooking methods. For example, a deep learning model such as a convolutional neural network can be used to analyze the second food image and extract key features in the second food image, which can include the shape, color, texture, etc. of the food.

[0170] Then, by analyzing the extracted key features, the model can identify the food material categories (such as meat, vegetables, or specific food names, etc.) and cooking methods (such as stir-frying, boiling, roasting, etc.) in the image. Combining the identified food material categories and cooking methods, the model can determine the dish type of the food, such as Gongbao chicken, braised pork, etc.

[0171] When it is determined that the target dish type is a new dish type, the pixel number of each food material category and the total pixel number of all food material categories are calculated, and based on the pixel number of each food material category and the total pixel number of all food material categories, the pixel proportion of each food material category in all food material categories is calculated as the weight of each food material category, which reflects the proportion of each food material category in the second food image, and is used for subsequent calculation of the holding temperature.

[0172] Finally, the system calculates the optimal holding temperature for each food material category under a specific cooking method. For example, the optimal holding temperature of each food material category is weighted and summed with the corresponding weight to obtain the optimal holding temperature of the target dish type, i.e. the optimal holding temperature of each food material category is multiplied by its weight, and then the weighted temperatures are added to obtain the optimal holding temperature of the entire target dish type, which takes into account the holding requirements of all food material categories and can accurately determine the optimal holding temperature of the new dish type.

[0173] This embodiment can obtain the optimal holding temperature of the entire target dish type by weighting and summing the optimal holding temperature of each food material category with the corresponding weight, so as to comprehensively consider the holding requirements of each food material category, more accurately control the temperature of the entire dish, and thus enable customers to enjoy a more stable and delicious dining experience, with the temperature of the food always being at the best state.

[0174] In one embodiment, after determining the target dish type to which the food in the second food image belongs based on the plurality of food material categories and the target cooking method, the method further comprises:

[0175] When it is determined that the target dish type is a non-new dish type, the optimal holding temperature of the target dish type is queried from the database.

[0176] In this embodiment, for the dish type that has been recorded in the database, the system will quickly retrieve its optimal holding temperature, so as to directly apply to the temperature control of the holding food warmer. For example, through the dish recognition result, the system checks whether the dish type exists in the database. If it is a known dish type, the system searches the optimal holding temperature corresponding to the recognized dish type in the database. The optimal holding temperature can be obtained based on historical data and experience, which can ensure that the food maintains its best state during holding.

[0177] For example, when the staff puts a dish of "Kung Pao Chicken" into the holding food warmer, the system recognizes that it is a dish of "Kung Pao Chicken" through the food image analysis model. The system checks the database and finds that "Kung Pao Chicken" already exists in the database, so it is a non-new dish type. The system queries the optimal holding temperature of "Kung Pao Chicken" in the database, which is 60℃, and automatically sets the temperature of the holding food warmer to 60℃.

[0178] This embodiment can quickly retrieve the optimal holding temperature for known dish types, reducing manual intervention and setting time, and improving overall operation efficiency. At the same time, by accurately controlling the holding temperature, the best taste and nutrition of the food can be maintained, and the dining experience of customers can be improved. In addition, the system can automatically manage the holding temperature of different dishes, reducing the need for manual monitoring and achieving intelligent management.

[0179] For reference Figure 4 , the embodiment of the present application also provides a holding food warmer operation control system based on artificial intelligence, comprising:

[0180] The control module 41 is used for controlling the preset camera module to shoot the food on the dish after the dish of the holding food warmer is placed with food, to obtain a first food image;

[0181] The preprocessing module 42 is used for receiving the first food image sent by the camera module, and pre-processing the first food image to obtain a second food image;

[0182] The analysis module 43 is used for calling a pre-trained food image analysis model to analyze the second food image, to identify the target dish type to which the food in the second food image belongs and determine the optimal holding temperature of the target dish type. The food image analysis model is a convolutional neural network, which is used for identifying the dish type in the food image and determining the optimal holding temperature of the dish type;

[0183] The setting module 44 is used for setting the holding temperature of the holding food warmer to the optimal holding temperature.

[0184] The application provides an intelligent heat preservation food stove operation control system, which comprises a receiving module, a preprocessing module, a food image analysis model and a setting module.

[0185] As to the system in the above-mentioned embodiments, the specific manner in which each module performs operations has been described in detail in the embodiments related to the method, and will not be described in detail here.

[0186] In one embodiment, the application further provides a storage medium storing computer readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the above-mentioned intelligent heat preservation food stove operation control method. The storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0187] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing relevant hardware, and the computer program can be stored in a storage medium. When the program is executed, it can include the processes of the above-mentioned embodiments of the method. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), etc.

[0188] Each technical feature of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of each technical feature in the above-mentioned embodiments are not described, but as long as the combinations of the technical features do not exist, they should be considered as the scope of the present application.

[0189] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the patent scope of the present application. It should be noted that for ordinary skilled persons in the art, without departing from the concept of the present application, several modifications and improvements can be made, which are all within the protection scope of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

Claims

1. An operation control method of an artificial intelligence-based thermal food warmer, characterized by, The method comprises the steps of: When the food is placed on the plate of the heat preservation food warmer, a preset camera module is controlled to shoot the food on the plate to obtain a first food image; The first food image sent by the camera module is received, and the first food image is preprocessed to obtain a second food image; A pre-trained food image analysis model is called to analyze the second food image, and a target dish type to which the food in the second food image belongs is identified, and an optimal heat preservation temperature of the target dish type is determined. The food image analysis model is a convolutional neural network, which is used to identify the dish type in the food image and determine the optimal heat preservation temperature of the dish type; The heat preservation temperature of the heat preservation food warmer is set to the optimal heat preservation temperature; The method further comprises the steps of: The pre-trained food image analysis model extracts a plurality of key features of the second food image, identifies a plurality of food material types and a target cooking method of the second food image based on the plurality of key features of the second food image, determines the target dish type to which the food in the second food image belongs based on the plurality of food material types and the target cooking method, calculates the pixel proportion of each food material type in all food material types in the second food image when it is determined that the target dish type is a new dish type, obtains the weight of each food material type, queries the optimal heat preservation temperature corresponding to each food material type under the target cooking method respectively, and obtains the optimal heat preservation temperature of the target dish type by weighting and summing the optimal heat preservation temperature of each food material type and the corresponding weight respectively; The method further comprises the steps of: An image data set of different dish types is collected, the image data set contains historical food images of a plurality of dish types and the optimal heat preservation temperature of each dish type, the image data set is preprocessed to obtain a standard image data set, the convolutional neural network is trained using the standard image data set, and the loss value of the trained convolutional neural network is calculated based on a preset loss function. When the loss value is lower than a preset loss value, the trained convolutional neural network is used as a food image analysis model; The method further comprises the steps of: When the loss value is not lower than the preset loss value, a first gradient of the preset loss function with respect to a prediction output of the convolutional neural network is calculated based on the loss value, and a second gradient of an output layer activation function with respect to an input of the convolutional neural network is calculated, and the first gradient and the second gradient are multiplied to obtain an output gradient of the preset loss function with respect to an output layer of the convolutional neural network, a third gradient of the output layer of the convolutional neural network with respect to a hidden layer output is calculated, a fourth gradient of a hidden layer activation function of the convolutional neural network with respect to an input is calculated, the output gradient, the third gradient and the fourth gradient are multiplied to obtain a target gradient, each model parameter of the convolutional neural network is multiplied by the target gradient to obtain a plurality of target model parameters of the convolutional neural network, the model parameters of the convolutional neural network are updated based on the plurality of target model parameters, the convolutional neural network with the updated model parameters is retrained using the standard image data set, and the training of the food image analysis model is completed until the loss value is lower than the preset loss value.

2. The operation control method of the artificial intelligence-based holding warmer according to claim 1, characterized by, The preprocessing of the image data set to obtain the standard image data set comprises: performing format unification, size adjustment, color space conversion, standardization and data enhancement processing on each historical food image of the image data set to obtain a plurality of standard food images; the plurality of standard food images are combined to form the standard image data set.

3. The AI-based operation control method of the thermal food server according to claim 1, characterized by, After the heat preservation temperature of the heat preservation food warmer is set to the optimal heat preservation temperature, the method further comprises: setting a stirring time of the food; when the stirring time is reached, controlling a preset stirring module to stir the food on the food tray.

4. The operation control method of the artificial intelligence-based holding warmer according to claim 1, characterized by, After the heat preservation temperature of the heat preservation food warmer is set to the optimal heat preservation temperature, the method further comprises: controlling a preset camera module to capture the food on the food tray in real time to obtain a third food image; receiving the third food image sent by the camera module; calling the food image analysis model to analyze the third food image to generate an analysis result; when it is determined according to the analysis result that there is no food on the food tray, sending an addition food reminder to a terminal of a staff and closing the heat preservation function of the heat preservation food warmer.

5. The artificial intelligence-based operation control method of the thermal food server according to claim 1, characterized by, After the heat preservation temperature of the heat preservation food warmer is set to the optimal heat preservation temperature, the method further comprises: obtaining temperature data of the food inside the heat preservation food warmer; extracting feature information of the temperature data to obtain temperature features, the temperature features comprising a temperature deviation, a deviation change rate and an integral absolute error; inputting the temperature features and the optimal heat preservation temperature into a pre-trained neural network model to output optimal PID parameters; adjusting parameters of a PID controller of the heat preservation food warmer based on the optimal PID parameters, and recording the adjusted parameters of the PID controller of the heat preservation food warmer and a system response.

6. The artificial intelligence-based operation control method of the thermal food server according to claim 1, characterized by, After the target dish type to which the food in the second food image belongs is determined based on the plurality of food material categories and the target cooking method, the method further comprises: When it is determined that the target dish type is a non-new dish type, an optimal holding temperature of the target dish type is queried from a database.

7. A control system for an artificial intelligence-based insulated food oven, characterized in that, The method comprises the steps of: controlling a preset camera module to capture food on a dish placed in a holding food warmer to obtain a first food image; preprocessing the first food image to obtain a second food image; calling a pre-trained food image analysis model to analyze the second food image, identifying a target dish type to which food in the second food image belongs, and determining an optimal holding temperature of the target dish type, wherein the food image analysis model is a convolutional neural network used to identify a dish type in a food image and determine an optimal holding temperature of the dish type; setting a holding temperature of the holding food warmer to the optimal holding temperature; wherein the calling of the pre-trained food image analysis model to analyze the second food image, the identification of the target dish type to which the food in the second food image belongs, and the determination of the optimal holding temperature of the target dish type comprise: calling the pre-trained food image analysis model to extract a plurality of key features of the second food image, identifying a plurality of food material types and a target cooking method of the second food image based on the plurality of key features of the second food image, determining the target dish type to which the food in the second food image belongs based on the plurality of food material types and the target cooking method, calculating a pixel proportion of each food material type in all food material types in the second food image when it is determined that the target dish type is a new dish type to obtain a weight of each food material type, querying an optimal holding temperature corresponding to each food material type under the target cooking method, and performing weighted summation on the optimal holding temperature of each food material type and the corresponding weight to obtain the optimal holding temperature of the target dish type; wherein, before the calling of the pre-trained food image analysis model to analyze the second food image, the identification of the target dish type to which the food in the second food image belongs, and the determination of the optimal holding temperature of the target dish type, the method further comprises: collecting an image dataset of different dish types, the image dataset containing historical food images of a plurality of dish types and an optimal holding temperature of each dish type, preprocessing the image dataset to obtain a standard image dataset, training a convolutional neural network using the standard image dataset, and calculating a loss value of the trained convolutional neural network based on a preset loss function, and when the loss value is lower than a preset loss value, using the trained convolutional neural network as a food image analysis model; wherein, after the calculation of the loss value of the trained convolutional neural network based on the preset loss function, the method further comprises: When the loss value is not lower than the preset loss value, a first gradient of the preset loss function with respect to a prediction output of the convolutional neural network is calculated based on the loss value, and a second gradient of an output layer activation function with respect to an input of the convolutional neural network is calculated, and the first gradient and the second gradient are multiplied to obtain an output gradient of the preset loss function with respect to an output layer of the convolutional neural network, a gradient of the output layer of the convolutional neural network with respect to a hidden layer output is calculated to obtain a third gradient, a gradient of a hidden layer activation function of the convolutional neural network with respect to an input is calculated to obtain a fourth gradient, the output gradient, the third gradient and the fourth gradient are multiplied to obtain a target gradient, each model parameter of the convolutional neural network is multiplied by the target gradient to obtain a plurality of target model parameters of the convolutional neural network, the model parameters of the convolutional neural network are updated based on the plurality of target model parameters, the convolutional neural network with the updated model parameters is trained again using the standard image data set, and the training is performed until the loss value is lower than the preset loss value, and a trained food image analysis model is obtained.

Images