Temperature control method and system for Chinese cooking based on multi-modal representation recognition

Abstract

The application discloses a Chinese cuisine cooking temperature control method and system based on multi-modal representation recognition, relates to the technical field of Chinese cuisine cooking temperature control, and comprises the following steps: data acquisition and preprocessing are performed by using a sensor; food state changes are analyzed, food state changes are monitored in real time, and multi-dimensional feature fusion is performed; texture dynamic characteristics are captured, cooking images are monitored, feedback variables are introduced, and temperature control adjustment is performed; environmental variables are monitored, and preference settings are performed; big data and user historical data are analyzed; the method realizes comprehensive monitoring of multi-dimensional data such as temperature, humidity and color through fusion processing of multi-modal sensors, and ensures the accuracy and real-time performance of cooking data; through the combination of multi-modal sensor data fusion, big data analysis, time and space attenuation modeling, user preference setting and other technologies, the food state is dynamically analyzed, the cooking stage is accurately judged, the temperature control strategy is optimized, and the problem of extensive control in the traditional method is avoided.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of temperature control of Chinese cooking, and particularly to a temperature control method for Chinese cooking based on multi-modal representation recognition. BACKGROUND

[0002] With the rapid development of artificial intelligence technology, sensor technology and big data analysis technology, intelligent household appliances have been widely used in modern kitchens. Among them, the accurate control of temperature in the cooking process has become a research hotspot;

[0003] Temperature is a key parameter in Chinese cooking, which directly affects the physical and chemical changes of food materials, such as protein denaturation, enzyme activation and deactivation, and food flavor release;

[0004] At present, most of the existing intelligent cooking devices on the market use a single sensor to monitor temperature and control temperature through a preset program.

[0005] Although these methods have improved the convenience of cooking to some extent, the adaptability and accuracy of existing technologies still have great limitations when facing the diversified cooking methods and complex food types of Chinese cuisine.

[0006] In actual use, we found that the existing temperature control method for Chinese cooking mostly uses single sensor data acquisition, which cannot comprehensively capture multi-modal information such as temperature, humidity, optical characteristics and other multi-dimensional parameters in the cooking process, resulting in insufficient scientificity of temperature control strategy and easy over or under heating. Secondly, most intelligent cooking devices only control temperature based on simple threshold judgment, cannot combine dynamic changes of food state for stage control, and are difficult to adapt to complex and nonlinear temperature requirements in Chinese cooking process.

[0007] Therefore, the present application provides a temperature control method for Chinese cooking based on multi-modal representation recognition. SUMMARY

[0008] In view of the above existing problems, the present application is proposed.

[0009] Therefore, the present application provides a temperature control method for Chinese cooking based on multi-modal representation recognition to solve the problem that existing methods cannot comprehensively capture multi-modal information in the cooking process and also cannot combine dynamic changes of food state for stage control.

[0010] To solve the above technical problems, the present application provides the following technical solutions:

[0011] In a first aspect, the present application provides a temperature control method for Chinese cooking based on multi-modal representation recognition, which comprises:

[0012] Data collection and preprocessing using sensors;

[0013] Analyzing food state changes, monitoring food state changes in real time, and performing multi-dimensional feature fusion;

[0014] Capture texture dynamic characteristics, monitor cooking images, introduce feedback variables, and adjust temperature control;

[0015] Monitor environmental variables and make preference settings;

[0016] Analyze big data and user historical data to generate temperature control recommendations for visual display.

[0017] As a preferred scheme of the temperature control method for Chinese cooking based on multi-modal representation recognition, the temperature data is collected, the infrared temperature sensor is deployed, the food surface temperature measurement is monitored and completed, the invasive temperature sensor is deployed, and the food internal temperature measurement is monitored and completed;

[0018] Collecting gas concentration data, deploying steam concentration sensors and oil smoke sensors to monitor water vapor evaporation and oil smoke generation;

[0019] Collecting image data, deploying RGB cameras to capture food changes, installing infrared thermal imaging cameras to capture temperature distribution in different areas, and identifying uniform heating state;

[0020] Collecting environmental data, deploying air pressure sensors, air flow rate sensors and temperature and humidity sensors to monitor the temperature, humidity and air flow of the kitchen, and providing dynamic temperature compensation basis;

[0021] Sensor S i A set of measurement data D i (t), time The first temperature data, the data collected at different times is matched with the time, and the data points are matched with the collected time stamps.

[0022] As a preferred scheme of the temperature control method for Chinese cooking based on multi-modal representation recognition, the analysis of food state changes, real-time monitoring of food state changes specifically includes:

[0023] Cleaning the collected image data and normalizing the image data;

[0024] Based on the convolutional neural network structure, set the convolutional layer, and perform convolution operation on the input image through the mean filter;

[0025] The convolution operation is specifically a convolution calculation on the input image to generate a convolution feature.

[0026] ;

[0027] wherein, represents the feature map output after the convolution operation, is the pixel value of the offset position (m, n) of the input image pixel value (i, j), wherein (i+m, j+n) respectively represents the corresponding row and column positions of the convolution kernel on the input image, is a fixed filter matrix;

[0028] A nonlinear activation function is applied to the output pixel after convolution to introduce nonlinear features, and the activation function output of each time step is integrated, represented as:

[0029] ;

[0030] ;

[0031] wherein, represents the output value after applying the ReLU activation function, represents the cumulative value of the food state change from time 0 to time t.

[0032] As a preferred scheme of the temperature control method for Chinese cooking based on multi-modal representation recognition, the multi-dimensional feature fusion includes:

[0033] A pooling layer is set to perform a pooling operation on the cumulative value of the food state change after integration calculation, to reduce the calculation amount and enhance the abstractness of the features, and the output value after pooling is in the range [0, ∞);

[0034] The range divided by the image in the pooling operation is divided by a sliding window division method, wherein the gradient amplitude in the pooling window is calculated, and the adjustment factor is set based on the gradient amplitude and , represented as:

[0035] ;

[0036] ;

[0037] wherein is a small constant, is used to adjust the feedback intensity of the local region according to the change of the image in the local region, represents the gradient value of the local region, is used to adjust the spatial feedback;

[0038] The feature map regionization weighting processing after the convolution operation is performed to form a regionized feature map.

[0039] Specifically, the formula is represents the pixel value of the local region in the input feature map, and the feature map is a feature representation extracted from the input image, and is represented as:

[0040]

[0041] wherein, and represent the relative coordinates within the window, and the size of the pooling window is set as , an exponential decay factor is introduced, represents each pixel value within the pooling window in the input feature map , and are adjustment factors, the final value of the output feature map at position , wherein the calculation of the final value is to select a local region in the feature map, and calculate the pixel value of the region, represents the local region centered at , and

[0042] The multi-dimensional feature fusion is performed on the color channels of the image, the pooling is performed on each channel respectively, and the results are fused, and is represented as:

[0043]

[0044] wherein, represents the total number of channels of the input data, represents the number of a specific channel, specifically an index from 1 to C, is the pixel value of the i-th channel of the multi-channel input data , and C is the number of channels of the input data, is the pixel value in the input feature map, , is the multi-dimensional feature fusion after the pooling. As a preferred scheme of the temperature control method for Chinese cooking based on multi-modal representation recognition provided by the application, wherein:

[0045] based on the extracted multi-dimensional feature fusion feature

[0046] , the extracted features are classified and regression predicted through a full connection layer;

[0047] ​​​Fusing the extracted multi-dimensional features into a feature The image data in the fused multi-dimensional feature is converted into a single-channel grayscale value image, a gray level co-occurrence matrix is constructed to describe the spatial relationship between pixels, and the fused multi-dimensional feature is defined as a two-dimensional matrix with a size of , the number of rows of the image, and the number of columns of the image, where each pixel value belongs to a gray level set, where is the number of gray levels of the input grayscale image, the co-occurrence frequency of different gray levels in the input grayscale image at a specified direction and distance is calculated, and is represented as:

[0048] ;

[0049] where, is the gray level co-occurrence matrix, representing the co-occurrence frequency between gray levels and at direction and distance , is the Kronecker delta function, is the pixel value at position in the input grayscale image , is the distance between pixels, is the calculation direction;

[0050] The calculated gray level co-occurrence matrix is normalized, and is represented as:

[0051] ;

[0052] where, is the normalized gray level co-occurrence matrix, and the sum of the matrix is 1;

[0053] The texture feature of the image is captured, and the local gradient information of the image is introduced. The gradient amplitude of the image is calculated by the following formula, and is represented as:

[0054] ;

[0055] where, is the gradient of the image at position , representing the intensity of image change;

[0056] The normalized gray level co-occurrence matrix is time-integrated, and the texture change of the image at different time steps is accumulated, and is represented as:

[0057] ;

[0058] wherein, is the accumulated gray level co-occurrence matrix, representing the accumulated gray level co-occurrence frequency within time , is the pixel value of the gray image at time , is the time variable, representing different time steps in the cooking process;

[0059] wherein, has a value range of [0, ∞), representing the co-occurrence frequency between the gray level pairs, has a value range of [0, 1] representing the normalized co-occurrence matrix, and the value of each element reflects the relative frequency of the gray level pairs, has a value range of [0, ∞), representing the accumulated co-occurrence matrix.

[0060] As a preferred scheme of the temperature control method for Chinese cooking based on multi-modal representation recognition according to the present application, wherein: the introduction of the feedback variable, adjusting the temperature control, specifically includes:

[0061] introducing the feedback function required for temperature control strategy adjustment, constructing the feedback function represents the cooking state feedback at time , and is represented as:

[0062] ;

[0063] wherein, represents the i-th feedback dimension, and Z ;

[0064] introducing a time decay factor , and γ is the decay factor. It is assumed that the farther the feedback is from the current time , the smaller the impact of the feedback on the current temperature control strategy, and the feedback value at any time has an impact, and is represented as:

[0065] ;

[0066] wherein, is the feedback function after time decay adjustment, representing the impact of historical feedback after time decay factor;

[0067] introducing a spatial feedback decay factor to control the feedback impact of different spatial positions, and a Gaussian decay function is used to represent the spatial effect, wherein is the spatial decay factor, and x and y represent the spatial coordinates; ​

[0068] Considering the influence of feedback information in multiple time and space dimensions on the temperature control strategy, a high-order integral function is used to synthesize the feedback information, and the feedback information of all historical time and space regions is integrated to calculate the temperature control strategy adjustment amount:

[0069] ;

[0070] Wherein, represents the local feedback intensity of the spatial position at time , and the specific calculation process is as follows:

[0071] The actual temperature value at time and spatial position is obtained, ;

[0072] The actual humidity value at time and spatial position is obtained, ;

[0073] The color and texture feature data of the food surface are obtained, and the texture features are calculated by using an image analysis algorithm;

[0074] The target temperature of cooking is set as , the target humidity is set as , the target curing state and the expected food doneness are set;

[0075] The temperature feedback is calculated and represented as:

[0076] ;

[0077] The humidity feedback is calculated and represented as:

[0078] ;

[0079] A time decay factor is introduced to control the degree of weakening of historical feedback over time, and is represented as:

[0080] ;

[0081] Wherein, is the decay factor, which determines the influence of historical feedback on the current time;

[0082] A Gaussian decay function is introduced to control the feedback influence of different spatial positions:

[0083] ;

[0084] Wherein, is the sum of square of distance from the center position, used to calculate the distance of each pixel relative to the center position, is a factor used to control the decay speed of spatial feedback influence, adjust the value of a;

[0085] Combine time decay and spatial decay, get the final humidity feedback value, expressed as:

[0086] ;

[0087] where The feedback function is used to represent the error and deviation between the target value Exponential decay factor, indicating that the influence of historical feedback on the current temperature control strategy gradually weakens, the integral interval , represents all time feedback from the beginning of cooking to the current time ; is the system state value at time , obtained by accumulating and historical feedback;

[0088] Adjust the influence of feedback on temperature control strategy, and increase its nonlinearity, use Sigmoid function to compress feedback information into :

[0089] ;

[0090] where, x is the input value of feedback value , Sigmoid function maps it to the range of , which represents the input of feedback value , is the exponential function of natural logarithm base, which represents the nonlinear characteristics of Sigmoid function;

[0091] Introducing Sigmoid function to nonlinearly scale feedback value , control the strength of feedback on temperature control strategy, finally get the temperature control strategy adjustment model, calculation is expressed as:

[0092] ;

[0093] where is the target temperature expected to be reached at the current time ;

[0094] Based on the temperature difference between and the current time t, carry out temperature control adjustment.

[0095] As a preferred scheme of the temperature control method for Chinese cuisine cooking based on multi-modal representation recognition, the monitoring environmental variables and preference setting specifically include:

[0096] Set target temperature range through UI, monitor real-time data, and continuously track food surface and internal temperature;

[0097] Set humidity range and target humidity, and automatically adjust humidity according to food type requirements;

[0098] User sets heating power, which is determined by dynamically calculating the difference between food surface and internal temperature, humidity, and cooking state, and adjusts it according to different cooking stages;

[0099] Adjust heating power dynamically according to model and user-set preferences;

[0100] Calculate temperature and humidity change rate and color change signal to determine if food is close to completion, and adjust remaining cooking time and power output;

[0101] Adjust food settings through UI, system adjusts temperature control parameters according to user requirements, and monitors kitchen environment to adjust temperature control parameters;

[0102] Store and analyze feedback data based on historical data and user-set preferences.

[0103] As a preferred scheme of the temperature control method for Chinese cuisine cooking based on multi-modal representation recognition, the analysis of big data and user historical data specifically includes:

[0104] Preprocess real-time data and historical data, including data cleaning, removing outliers and noise, filling missing data, and detecting outliers;

[0105] Group users, record similar preferences of grouped users, and model them;

[0106] Compare user similarities and mine correlations to optimize cooking strategies;

[0107] Extract data features, including temperature change curve, food surface color change, and heating power, and use the above features as input parameters for prediction modeling;

[0108] Use cooking data as training data for reinforcement learning, adjust temperature control strategy, and keep it updated.

[0109] Temperature control system for Chinese cuisine cooking based on multi-modal representation recognition, the system includes:

[0110] A temperature sensor module for real-time monitoring of the temperature of the surface and interior of the food;

[0111] A humidity sensor module for monitoring the humidity changes of the surface of the food;

[0112] A gas concentration sensor module for monitoring the concentration of steam, oil fume and water vapor;

[0113] In a second aspect, the present application provides a temperature control system for Chinese cooking based on multi-modal representation recognition, comprising an image acquisition module including an RGB camera and an infrared thermal imaging camera, which captures image changes of the surface and interior of the food and identifies the cooking state of the food;

[0114] An environmental monitoring sensor module for real-time monitoring of the temperature and humidity, airflow environmental data in the kitchen;

[0115] A data acquisition module for real-time acquisition of temperature, humidity, gas concentration, image data and other information from various sensors, and time calibration of data according to time steps;

[0116] A data cleaning and preprocessing module for cleaning the collected data, including denoising, outlier detection, missing data filling, etc.;

[0117] A data fusion and analysis module for fusion analysis of sensor data, including image feature extraction using image processing technology, convolutional neural network, multi-dimensional data fusion, gray level co-occurrence matrix analysis and texture feature extraction;

[0118] The dynamic temperature control adjustment is realized by the following steps:

[0119] The cooking state at each moment is calculated by a feedback function, and a time decay factor and a space decay factor are used to adjust the influence of historical feedback;

[0120] The temperature control parameters are adjusted according to real-time data and user-set target temperature and humidity.

[0121] In a third aspect, the present application provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and wherein the computer program, when executed by the processor, implements any step of the temperature control method for Chinese cooking based on multi-modal representation recognition according to the first aspect of the present application.

[0122] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements any step of the temperature control method for Chinese cooking based on multi-modal representation recognition according to the first aspect of the present application.

[0123] The present application has the following advantages:

[0124] The fusion processing of the multi-modal sensor realizes comprehensive monitoring of multi-dimensional data such as temperature, humidity, color, etc., ensuring the accuracy and real-time of cooking data;

[0125] By combining multi-modal sensor data fusion, big data analysis, time and space decay modeling, user preference setting and other technologies, the food state is dynamically analyzed, the cooking stage is accurately judged, and the temperature control strategy is optimized, avoiding the problem of extensive control of traditional methods;

[0126] Through user preference setting and real-time feedback adjustment, seamless matching of intelligent temperature control and user demand is realized, significantly improving cooking quality and user satisfaction, thereby improving the intelligent operation demand of the method. BRIEF DESCRIPTION OF DRAWINGS

[0127] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0128] Figure 1 Flowchart of the temperature control method for Chinese cooking based on multi-modal representation recognition in embodiment 1.

[0129] Figure 2 Flowchart of the strengthening processing of the gray level co-occurrence matrix in embodiment 1. DETAILED DESCRIPTION

[0130] In order to make the above-mentioned purposes, features and advantages of the present application more apparent and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the drawings of the specification.

[0131] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application, but the present application can also be implemented in other ways different from those described herein, and those skilled in the art can make similar generalizations without departing from the connotation of the present application, therefore the present application is not limited to the specific embodiments disclosed below.

[0132] Secondly, the "one embodiment" or "embodiment" referred to herein means that the specific features, structures or characteristics can be included in at least one implementation of the present application. In this specification, "in one embodiment" does not mean the same embodiment, nor is it an independent or alternative embodiment that excludes other embodiments.

[0133] Embodiment 1, refer to Figure 1 and Figure 2For the first embodiment of the present application, the temperature control method for Chinese cooking based on multi-modal representation recognition is provided, including the following steps:

[0134] S1, using sensors to collect data and perform fusion processing;

[0135] Preferably, sensor data is collected and preprocessed, including:

[0136] The use of sensor data for collection and fusion processing includes:

[0137] Temperature data is collected, infrared temperature sensors are deployed on the heating surface of the cooking equipment and the surface of the food, the temperature distribution of the food surface is monitored in real time by non-contact means, the surface temperature measurement is completed, the invasive temperature sensor is inserted into the food material, the internal temperature of the food is monitored, and the internal temperature measurement is completed;

[0138] By monitoring the surface and internal temperature of the food material in real time, the system can timely adjust the heating power and change the heating method to ensure that the food is cooked at the best temperature, which is crucial for cooking methods that require precise temperature control. Real-time temperature data provides continuous feedback to the system, allowing it to dynamically adjust the fire to avoid large and small temperature fluctuations, ensuring uniform heating of the food, preventing overheating and undercooking, and thus achieving precise control of the temperature. In addition, by inserting the invasive temperature sensor into the interior of the food material, the internal temperature of the food can also be effectively monitored, especially for thicker food materials. The system can ensure uniform heating of the food by collecting temperature data from different parts of the food, avoiding the occurrence of overcooked and undercooked conditions, thereby effectively improving the uniformity of food heating;

[0139] Gas concentration data is collected, steam concentration sensors and oil smoke sensors are set up above the cooking equipment to monitor water vapor evaporation and oil smoke generation during cooking. Changes in steam concentration can reflect whether the food has reached the boiling point and whether the water evaporation is too fast. An increase in oil smoke concentration may indicate that the oil temperature is too high and the food is burning, requiring timely adjustment of the fire;

[0140] During cooking, especially during frying and stir-frying operations, oil smoke and steam are inevitable. By monitoring the steam concentration and oil smoke concentration in real time, the system can automatically adjust the cooking fire and activate the exhaust system to prevent oil smoke accumulation, which can damage the flavor of the food and also affect the normal use of the device. Furthermore, during actual use, especially during low-temperature cooking such as steaming and stewing, changes in steam concentration affect the moisture of the food. By monitoring the steam concentration, the system can adjust the heating power and adjust the steam release method based on the steam concentration data to ensure the ideal moisture of the food and ensure the use effect of the device;

[0141] Image data is collected, an RGB camera is installed above the cooking device to capture the surface color and shape changes of the food, which is used to determine whether the food is in the ideal cooking state; and an infrared thermal imaging camera is installed to capture the temperature distribution of different areas, which helps to identify whether the heating of the food is uniform, which is suitable for use in frying, baking and other use occasions that require observation of the surface temperature changes of the food;

[0142] The monitoring and transmission of image data can intuitively reflect the cooking state of the food. By identifying the surface color changes of the food, it can be inferred whether the food has reached the ideal degree of doneness. The system can intelligently analyze the image data, determine the cooking stage, and automatically adjust the temperature control strategy accordingly to avoid overcooking and undercooking.

[0143] Environmental data is collected, air pressure sensors, air flow sensors and temperature and humidity sensors are installed around the cooking device to monitor the temperature, humidity and air flow in the kitchen. These data have important influence on the evaporation and heat conduction of food, especially in different seasons and indoor environmental conditions, which can provide dynamic temperature compensation basis for the cooking device.

[0144] When the weather changes and the kitchen ventilation is unstable, the environmental temperature and humidity may cause the cooking temperature to be unstable. Specifically, when the humidity is high, the water on the surface of the food evaporates more easily, affecting the heating effect. When the temperature is too high, it may cause the food to burn too quickly and the outer layer to be too dry. By collecting real-time environmental data such as temperature and humidity in the kitchen, the system can compensate for external factors during cooking and dynamically adjust the temperature control parameters to ensure that the food is cooked in the best environmental conditions. In another extreme case, in a relatively humid kitchen, the system can adjust the fire according to the environmental humidity to prevent the water from evaporating too quickly. Conversely, in a relatively dry environment, the system may need to increase the heating power and extend the cooking time to maintain the moisture of the food to ensure that the food is not too dry and affects its own taste.

[0145] Through the above-mentioned multi-dimensional data collection, the system can not only accurately adjust the temperature, but also customize it based on user preferences and food types. Through real-time monitoring and analysis of temperature, humidity, gas concentration and images, the system can automatically adjust during cooking without manual intervention. The real-time collected data not only serves the current cooking control, but also can be stored and used for subsequent system learning and optimization. Through the analysis of multiple cooking data, the system can gradually learn the most suitable temperature control scheme for different food, different environment and different users, improving the cooking efficiency and accuracy.

[0146] S2, analyzing food state changes, real-time monitoring of food state changes, multi-dimensional feature fusion;

[0147] Preferably, analyzing food state changes, real-time monitoring of food state changes specifically includes:

[0148] The analysis of food state changes, real-time monitoring of food state changes specifically includes:

[0149] The collected image data is cleaned and normalized;

[0150] Based on the convolutional neural network structure, set the convolutional layer, and perform convolution operation with the input image through the mean filter;

[0151] Wherein the convolution operation is specifically a convolution calculation on the input image, generating convolution features:

[0152] ;

[0153] Wherein, The feature map output after convolution operation is represented as, Is the pixel value of the input image pixel value i, j, the offset position m, n, wherein i+m, j+n respectively represents the corresponding row and column position of the convolution kernel on the input image, Is a fixed filter matrix;

[0154] Based on the output pixel after convolution, a nonlinear activation function is applied to introduce nonlinear features, and the activation function output of each time step is integrated, wherein ReLU is used to truncate the negative part to zero, ensuring that the model can learn more complex features. Since convolutional neural networks are usually used to process dynamic data, in order to consider the change of time comprehensively, it is represented as:

[0155] ;

[0156] ;

[0157] Wherein, The output value after applying the ReLU activation function is represented as, The positive part will be retained, and the negative part will be converted to zero, The cumulative value of food state changes from time 0 to time t is represented as, which is used to judge the cooking stage of food and adjust the temperature control strategy, which is used to dynamically adjust the temperature control strategy, the pixel value of the input image or multi-dimensional data;

[0158] By introducing time and space decay factors, the model can effectively control the feedback influence of different time points and spatial regions. The time decay factor gradually reduces the influence of historical feedback and focuses only on the feedback within the current time period, thereby improving the model's response speed to real-time changes. The spatial decay factor makes the regions far from the central region in the image have less contribution to the final decision, ensuring that the model focuses more on the key areas of food cooking. By using high-order integration to integrate the feedback information of multiple time and space dimensions, the model can consider the changes at each time point in the cooking process from multiple dimensions, providing a more accurate temperature control strategy. The non-linear activation function further enhances the complexity of the model, enabling it to handle complex relationships between temperature, humidity, and food state, ensuring that the feedback value is suitable for subsequent temperature control adjustments. The model dynamically adjusts the feedback based on real-time collected sensor data such as temperature, humidity, color change, etc., thereby achieving precise control of heating power, cooking temperature, time, and other parameters. This dynamic feedback mechanism can optimize the temperature control strategy in real-time, improve the cooking effect of food, and avoid overcooking or undercooking caused by overheating or temperature fluctuations.

[0159] Further, the multi-dimensional feature fusion includes:

[0160] A pooling layer is set to perform a pooling operation on the accumulated food state change value after integration calculation, reducing the calculation amount and enhancing the abstractness of the features. The output value domain after pooling is [0, ∞), where indicates that the features in this region are not effectively extracted and ignored. The pooling operation does not find effective features in this region, where indicates that the pooling operation extracts the significant features of this region. Generally, the larger the output value, the more prominent the features in this region.

[0161] The range divided by the sliding window division method in the pooling operation process, where the gradient amplitude in the pooling window is calculated. Based on the gradient amplitude, the adjustment factor and are set as:

[0162] ;

[0163] ;

[0164] where is a small constant, and α is used to adjust the feedback intensity of the local region based on the changes in the image within the local region, represents the gradient value of the local region, is used to adjust the spatial feedback;

[0165] The regionized feature map is formed by regionized weighting processing of the feature map output after convolution operation.

[0166] Specifically, the formula in the above equation is represents the pixel value of the local region in the input feature map, which is a feature representation extracted from the input image, denoted as:

[0167] ;

[0168] wherein, and represent the relative coordinates within the window, and the size of the pooling window is set to , an exponential decay factor is introduced, represents each pixel value within the pooling window in the input feature map , and are adjustment factors, the final value of the output feature map at position , wherein the calculation of the final value is to select a local region in the feature map, and calculate the pixel value of the region, represents the local region in the input feature map centered at , is the size of the local region;

[0169] Multi-dimensional feature fusion is performed on the color channels of the image, and pooling is performed on each channel separately, and the results are fused, denoted as:

[0170] ;

[0171] wherein, represents the total number of channels of the input data, represents the number of a specific channel, specifically an index from 1 to C, and the summation operation in the above equation is performed on all channels from 1 to , so represents the current processing channel, and the specific calculation is performed separately for each channel, is the pixel value of the th channel of the multi-channel input data , and the number of channels of the input data is denoted as C, representing color channels and other multi-dimensional data, is the pixel value in the input feature map, , is the multi-dimensional feature fusion after pooling, which is to integrate the data features of multiple channels into an output with higher expression capacity after specific processing of the input data of different channels, wherein the specific processing includes weighted summation, activation function, exponential scaling, which retains the important features of the input data while reducing the dimension of the data, thereby improving the calculation efficiency and reducing the noise, so as to represent the features extracted from each local region, is a multi-dimensional feature fusion after pooling;

[0172] In image processing and convolutional neural networks, a local region refers to a small area in an image used to extract local features. This area can be obtained through window sliding. For example, in convolution operations, an image is divided into small matrix blocks, each of which is processed individually to extract image features through a convolution kernel. Local region processing can effectively reduce computational load and improve feature extraction efficiency. By processing images in local regions and combining spatial decay factors, the model can effectively focus on local details of the image while reducing interference from irrelevant information far from the center. This step effectively improves the computational efficiency and accuracy of the system, especially when dealing with large-scale data, significantly reducing computational load. During the cooking process, the state of the food usually changes in multiple dimensions and nonlinearly. The introduction of a feedback function can dynamically adjust the temperature control strategy based on real-time sensor data such as temperature error and humidity changes. This feedback mechanism ensures that each cooking process can adjust the heating power and cooking time in a timely manner based on the current food state, optimizing the final result. By introducing nonlinear mapping functions such as Sigmoid functions, the influence range of feedback data can be effectively compressed or expanded, avoiding overfitting problems in simple linear models. This not only increases the flexibility of the model but also improves the performance of the temperature control system in complex cooking tasks, ensuring the reasonable application of feedback signals. In traditional cooking feedback adjustment models, only single sensor data is considered, while multi-dimensional feature fusion can consider temperature, humidity, color change, and other multi-dimensional information, enabling the temperature control system to make decisions based on comprehensive data. This fusion method enables the system to efficiently and accurately adjust cooking parameters in complex environments.

[0173] S3, capture texture dynamic features, monitor cooking images, introduce feedback variables, and adjust temperature control;

[0174] Preferably, the capture texture dynamic features, monitor cooking images:

[0175] Based on the extracted multi-dimensional feature fusion features , the extracted features are classified and regressed through a fully connected layer;

[0176] Convert the image data in the extracted multi-dimensional feature fusion features into a single-channel grayscale value image. For the spatial distribution characteristics between pixel gray levels, construct a gray level co-occurrence matrix to describe the spatial relationship between pixels, where the gray level co-occurrence matrix represents the co-occurrence frequency of gray values and in a given direction and distance in the image , defined as a two-dimensional matrix of size , the number of rows of the image, the number of columns of the image, where each pixel value belongs to a set of gray level extremes, where is the number of gray levels of the input gray image, the co-occurrence frequency of different gray levels in the input gray image at a specified direction and distance is calculated, denoted as:

[0177] ;

[0178] where, is the gray co-occurrence matrix, representing the co-occurrence frequency of gray levels and between the direction and distance, is the Kronecker delta function, used to determine and whether equal, if equal, then 1, otherwise 0, is the pixel value of the input gray image at position , is the distance between pixels, usually set to 1, is the calculation direction, specifically including 0°, 45°, 90°, 135°, by high-order integration of the spatial relationship of the pixel points, the capture ability of the spatial structure is enhanced;

[0179] The calculated gray co-occurrence matrix is normalized to ensure that the sum of the matrix is 1 and meets the requirements of probability distribution, denoted as:

[0180] ;

[0181] where, is the normalized gray co-occurrence matrix, the sum of the matrix is 1, the normalization process replaces the simple summation with integration, ensuring correct measurement of gray level pairs;

[0182] Capture the texture features of the image, introduce the local gradient information of the image, the gradient amplitude of the image is calculated by the following formula, denoted as:

[0183] ;

[0184] where, is the gradient of the image at position , indicating the intensity of image change;

[0185] In the image changes during the cooking process, the time dimension is introduced to capture the dynamic characteristics of texture changes, and the calculation of gray-level co-occurrence matrix is ​​introduced into the time series to capture the texture characteristics of the image that change over time due to the temporal evolution of the image during the cooking process;

[0186] Gray-level co-occurrence matrix Perform time integration and accumulate the texture changes of the image at different time steps, which can be expressed as:

[0187] ;

[0188] in, is the cumulative gray-level co-occurrence matrix, which represents the The accumulated gray level co-occurrence frequency within is the grayscale image at time The pixel value at the moment, is the time variable, representing different time steps in the cooking process;

[0189] Among the above, The value range of is [0, ∞, which represents the co-occurrence frequency between grayscale pairs. A larger value indicates that the grayscale pair appears more frequently in the image. The range of is [0, 1], which means the normalized co-occurrence matrix. The value of each element reflects the relative frequency of the gray level pair. The range of is [0,∞), which represents the cumulative co-occurrence matrix, which changes over time and captures the changes in image texture;

[0190] Based on the accumulated co-occurrence matrix, the dynamic characteristics of texture changes are captured to monitor cooking images in real time.

[0191] By adopting an integral feedback mechanism and nonlinear adjustment, the temperature control strategy can not only rely on feedback at the current moment, but also consider feedback information from past moments. This consideration of historical data greatly improves the model's dynamic adjustment capabilities, especially in complex cooking tasks. It can maintain a balance between temperature and humidity to ensure optimal cooking results every time. This temperature control strategy adjustment model based on image processing and multidimensional feature fusion can achieve precise temperature and humidity control during the cooking process through precise feedback mechanisms, spatial attenuation and nonlinear mapping.

[0192] Preferably, the introduction of feedback variables to adjust the temperature control specifically includes:

[0193] Introducing the feedback function required for temperature control strategy adjustment, temperature control strategy Depends on the current time The feedback at the moment, in the cooking process, the adjustment of the temperature control strategy is not only affected by the feedback at the moment, but also affected by the feedback at the past moment, therefore, the adjustment of the temperature control strategy needs to introduce time accumulation effect, so that the feedback at the past moment has a certain influence on the current temperature control strategy, set the temperature control strategy By the feedback at the past moment The strategy at the past moment And the feedback information at each moment in the cooking process is weighted to obtain:

[0194] ;

[0195] Indicates the adjustment amount of the temperature control strategy at the moment , and the further derived expression ;

[0196] Introduce the feedback function required for the adjustment of the temperature control strategy, including temperature, humidity, food curing state, use the feedback function to represent the cooking state feedback at the moment , the feedback function includes the error between temperature and target temperature, humidity change, curing degree of food, which is expressed by the following formula:

[0197] ;

[0198] Among them, represents the zth feedback dimension, Z , introduce time decay factor , because the influence of past feedback on the current temperature control strategy gradually decreases, so by introducing the time decay factor, to reflect the gradual weakening effect of historical feedback on the current decision, γ is the decay factor, control the influence speed of past feedback, set the feedback The farther away from the current time, the smaller the influence on the current temperature control strategy, and the feedback The influence at a certain moment is expressed as:

[0199] ;

[0200] Among them, is the feedback function after time decay adjustment, which represents the influence of historical feedback after time decay factor, and is used to adjust the current temperature control strategy;

[0201] In the modeling, the spatial variation is considered, in the cooking process, the food at different positions, such as the surface and the inside, has different influences on the temperature control strategy, introduce the spatial feedback decay factor to control the feedback influence of different spatial positions, use the Gaussian decay function to represent the spatial effect, where α is the spatial decay factor, x and Spatial coordinates, is the square of the distance relative to the center position, after introducing spatial attenuation, local feedback Adjust according to the distance of the spatial position;

[0202] Consider the influence of feedback information in multiple time and space dimensions on the temperature control strategy, use high-order integral functions to synthesize feedback information, integrate feedback information of all historical time and space regions, and calculate the temperature control strategy adjustment amount:

[0203] ;

[0204] Where, represents the local feedback intensity of the spatial position at time , and the specific calculation process is:

[0205] Get the actual temperature value at time and spatial position ; ;

[0206] Get the actual humidity value at time and spatial position ; ;

[0207] Get the color, texture and other feature data of the food surface, usually use image analysis algorithm to calculate color change or texture features;

[0208] Set the target temperature of cooking , set the target humidity , set the target curing state or desired food doneness;

[0209] Combine the collected data and target state to calculate the feedback function , calculate the temperature feedback:

[0210] ;

[0211] Calculate the difference between the current temperature of the food at time and the target temperature;

[0212] Calculate the humidity feedback:

[0213] ;

[0214] Based on image processing algorithm, calculate the food curing degree feedback, extract the color change or texture features of the food surface, and generate the curing degree score;

[0215] Introduce time attenuation factor , controls the degree of weakening of historical feedback over time,

[0216] ;

[0217] in, is the attenuation factor, which determines the impact of historical feedback on the current moment;

[0218] A Gaussian decay function is introduced to control the feedback effect at different spatial positions:

[0219] ;

[0220] in, It is the spatial location of feedback information collection, specifically the horizontal and vertical coordinates, which are collected by sensors at different positions on the surface and inside of the food. is the sum of the squares of the distances from the center position, used to calculate the distance of each pixel relative to the center position. This is a factor used to control the decay rate of the spatial feedback effect. It is estimated experimentally by collecting feedback data at different spatial locations, observing the influence of different regions on the temperature control strategy, and adjusting the value of α.

[0221] Combining time attenuation and space attenuation, the final humidity feedback value is obtained, which is expressed as:

[0222] ;

[0223] The local feedback strength does not affect the entire system, The feedback function is used to express the error and deviation from the target value, indicating the spatial position In time Feedback information at all times, including temperature error and temperature deviation, Exponential decay factor, indicating that the impact of historical feedback on the current temperature control strategy gradually weakens, the integral interval , indicating the time from the start of cooking to the current moment All time feedback, The exponential decay coefficient describes the degree of decay of the feedback information over time. It is the time variable in the integral and represents the historical time point at which the feedback action occurs. Together, they are used to describe the cumulative impact of historical feedback on the current state. For time The initial state value at time , represents the initial condition or initial output value of the system, For time The system status value at the moment is obtained by Historical feedback is accumulated to describe the dynamic changes of the system;

[0224] The impact of the feedback on the temperature control strategy is adjusted so as not to be too large and too small, while increasing its nonlinearity, using a Sigmoid function to compress the feedback information into :

[0225] ;

[0226] wherein, is the input value of the feedback value , the Sigmoid function maps it to the range of , while representing the input of the feedback value , is the exponential function of the natural logarithm base, representing the nonlinearity of the Sigmoid function;

[0227] The Sigmoid function is introduced to nonlinearly scale the feedback value , controlling the strength of the feedback on the temperature control strategy, and finally obtaining the temperature control strategy adjustment model, the calculation is represented as:

[0228] ;

[0229] Starting from the initial setting , the feedback is gradually adjusted, and multiple feedback dimensions, including temperature, humidity, and food state, are used to adjust the temperature control strategy. An exponential decay function is used to model the problem that the impact of historical feedback will gradually decay over time, making it difficult to weigh. A Gaussian decay function is used to model the spatial position, actively absorbing feedback and considering the different impacts of different positions on the temperature control strategy. Then, through the integral function, the information of historical feedback and spatial feedback is integrated to form the adjustment of the current temperature control strategy. Finally, the Sigmoid function is used to nonlinearly compress the feedback, ensuring that the impact of the feedback on the temperature control strategy is within a reasonable range. In this way, the model can effectively dynamically adjust the temperature control during the cooking process, ensuring that each cooking process can accurately control the temperature, time, and other parameters to ensure the best cooking effect, is the target temperature that the system expects to achieve at the current time , combining the initial temperature and the result of adjustment under the influence of historical feedback information . This model describes the dynamic process of the temperature control strategy adjustment system, dynamically adjusting the temperature control strategy based on the feedback signals at each time in the cooking process to optimize the cooking effect, and adjusting the temperature control based on the temperature difference between and the current time t.

[0230] S4, monitoring environmental variables, setting preferences;

[0231] Preferably, the monitoring of environmental variables, setting preferences specifically includes:

[0232] Set target temperature range through UI, monitor real-time data, continuously track food surface and internal temperature;

[0233] Set humidity range and target humidity, and automatically adjust humidity according to food type requirements;

[0234] User sets heating power, which is determined by dynamically calculating the difference between food surface and internal temperature, humidity and cooking state, and adjusts it according to different cooking stages;

[0235] The system dynamically adjusts the heating power according to the model and user-set preferences;

[0236] Calculate the rate of change of temperature and humidity and the color change signal to determine if the food is close to completion, and adjust the remaining cooking time and power output;

[0237] Feedback during cooking, system optimizes temperature control strategy based on feedback, including automatic adjustment of temperature, humidity, pressure and time;

[0238] In different cooking stages, the system dynamically adjusts various parameters according to current environmental conditions and sensor data to ensure that the temperature control strategy matches user preferences and real-time environmental needs;

[0239] Adjust food settings through UI, system adjusts temperature control parameters according to user requirements, and monitors kitchen environment to adjust temperature control parameters;

[0240] According to the historical data of cooking feedback and user-set preferences, and store and analyze feedback data.

[0241] S5, analyze big data and user historical data, generate temperature control recommendations for visual display;

[0242] Preferably, the analysis of big data and user historical data specifically includes:

[0243] Preprocess real-time data and historical data, including data cleaning, removing outliers and noise, filling missing data, and detecting outliers;

[0244] Group users, record similar preferences of grouped users and model them;

[0245] Compare user similarities and mine correlations to optimize cooking strategies;

[0246] Extract data features, including temperature change curve, food surface color change, heating power, use the above features as input parameters for prediction modeling;

[0247] The cooking data is used as training data for reinforcement learning, the temperature control strategy is adjusted, and the update is maintained.

[0248] The embodiment also provides a temperature control system for Chinese cooking based on multi-modal representation recognition, which comprises:

[0249] A temperature sensor module is configured to monitor the temperature of the surface and interior of the food in real time.

[0250] A humidity sensor module is configured to monitor the humidity change of the surface of the food.

[0251] A gas concentration sensor module is configured to monitor the concentration of steam, oil fume and water vapor.

[0252] An image acquisition module comprising an RGB camera and an infrared thermal imaging camera is configured to capture the image change of the surface and interior of the food and identify the cooking state of the food.

[0253] An environment monitoring sensor module is configured to monitor the temperature and humidity, airflow environment data in the kitchen in real time.

[0254] A data acquisition module is configured to acquire temperature, humidity, gas concentration, image data and other information in real time from each sensor, and time mark the data according to time steps.

[0255] A data cleaning and preprocessing module is configured to clean the collected data, including denoising, outlier detection, missing data filling and the like.

[0256] A data fusion and analysis module is configured to fuse and analyze the sensor data, including using image processing technology, convolutional neural network for image feature extraction, multi-dimensional data fusion, gray level co-occurrence matrix analysis and texture feature extraction.

[0257] The dynamic temperature control adjustment is realized through the following steps:

[0258] The cooking state at each moment is calculated through a feedback function, and a time decay factor and a space decay factor are used to adjust the influence of historical feedback.

[0259] The temperature control parameters are adjusted according to the real-time data and the target temperature and humidity set by the user.

[0260] The embodiment also provides a computer device suitable for the temperature control method for Chinese cooking based on multi-modal representation recognition, which comprises a memory and a processor; the memory is configured to store computer executable instructions, and the processor is configured to execute the computer executable instructions to realize the temperature control method for Chinese cooking based on multi-modal representation recognition as proposed in the above embodiment.

[0261] The computer device can be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected by a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is configured to perform wired or wireless communication with an external terminal. The wireless communication can be achieved by WIFI, a carrier network, NFC (Near Field Communication) or other technologies. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the computer device, or an external keyboard, touchpad or mouse, etc.

[0262] The embodiment also provides a storage medium having a computer program stored thereon, the program being executed by a processor to implement the temperature control method for Chinese cooking based on multi-modal representation recognition as proposed in the above embodiment. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic storage, a flash memory, a magnetic disk or an optical disk.

[0263] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application but not limit the present application. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced equivalently without departing from the spirit and scope of the present application, and all modifications or equivalent replacements should be covered in the scope of the claims of the present application.

Claims

1. A temperature control method for Chinese food cooking based on multimodal characterization and recognition, characterized in that: include: Use sensors to collect data and perform pre-processing, including: Collect temperature data, deploy infrared temperature sensors to monitor and measure food surface temperature, and deploy invasive temperature sensors to monitor and measure food internal temperature; Collect gas concentration data, deploy steam concentration sensors and oil fume sensors to monitor water vapor evaporation and oil fume generation; Collect image data, deploy RGB cameras to capture changes in food ingredients, and install infrared thermal imaging cameras to capture temperature distribution in different areas and identify uniform heating conditions; Collect environmental data and deploy air pressure sensors, air velocity sensors, and temperature and humidity sensors to monitor the temperature, humidity, and airflow in the kitchen, and provide a basis for dynamic temperature compensation. Correspond the data collected by sensors at different moments to the time, and match the data points with the collected timestamps; Analyze food status changes, monitor food status changes in real time, and perform multi-dimensional feature fusion; Capture dynamic texture features, monitor cooking images, introduce feedback variables, and adjust temperature control; Monitor environment variables and set preferences, including: Set the target temperature range through the UI, monitor data in real time, and continuously track the surface and internal temperature of food; Set the humidity range and target humidity, and automatically adjust the humidity according to the needs of food types; The user sets the heating power, which is adjusted according to the different cooking stages. The heating power is determined by dynamically calculating the difference between the surface and internal temperature of the food, humidity, and cooking status. Dynamically adjust heating power based on the model and user settings preferences; Calculate the rate of change of temperature and humidity and color change signals to determine whether the food is nearly done and adjust the remaining cooking time and power output; Adjust food settings through the UI. The system adjusts temperature control parameters according to user requirements and monitors the kitchen environment to adjust temperature control parameters. Based on historical cooking feedback data and user-defined preferences, the feedback data is stored and analyzed; The analysis of big data and user historical data specifically includes: Preprocessing of real-time and historical data, including data cleaning, removal of outliers and noise, filling in missing data, and detecting outliers; Group users, record similar preferences of grouped users and build models; Compare user characteristics online, explore correlations, and optimize cooking strategies; Extract data features, including temperature change curves, food surface color changes, and heating power, and use these features as input parameters for predictive modeling; Use cooking data as training data for reinforcement learning to adjust temperature control strategies and keep them updated; Analyze big data and user historical data to generate temperature control suggestions for visual display.

2. The temperature control method for Chinese food cooking based on multimodal characterization and recognition according to claim 1, characterized in that: The analyzing of food status changes and real-time monitoring of food status changes specifically include: Clean the collected image data and normalize the image data; Based on the convolutional neural network structure, a convolution layer is set up to perform convolution operation with the input image through a mean filter; The convolution operation is to perform convolution calculation on the input image to generate convolution features: ； in, Represents the feature map output after the convolution operation, is the pixel value of the offset position (m, n) of the input image pixel (i, j), where (i+m, j+n) represent the row and column positions of the convolution kernel on the input image, respectively. is a fixed filter matrix; A nonlinear activation function is applied to the output pixels after convolution, nonlinear features are introduced, and the output of the activation function at each time step is integrated, which is expressed as: ； ； in, Represented as the output value after applying the ReLU activation function, Indicates the cumulative value of food status changes from time 0 to time t.

3. The temperature control method for Chinese food cooking based on multimodal characterization and recognition according to claim 2, characterized in that: The multi-dimensional feature fusion comprises: Set up a pooling layer to perform a pooling operation on the cumulative value of food state changes after integral calculation to reduce the amount of calculation and enhance the abstractness of the features. The output value range after pooling is [0, ∞); The image is divided into ranges during the pooling operation through the sliding window division method, the gradient amplitude within the pooling window is calculated, and the adjustment factor is set based on the gradient amplitude and , expressed as: ； ； in is a small constant, Represents the gradient value of the local area; The feature map output after the convolution operation is regionalized and weighted to form a regionalized feature map; Specifically, the formula Represents the pixel value of the local area in the input feature map. The feature map is a feature representation extracted from the input image and is expressed as: ； in, and Represents the relative coordinates within the window, and the pooling window size is , introducing an exponential decay factor , Represents the input feature map Each pixel value within the pooling window is and is the regulating factor, Represents the output feature map Middle position The final value of the feature map is calculated by selecting an area and calculating the pixel value of the area. Indicates that the input feature map is The central area, is the size of the region; Perform multi-dimensional feature fusion on the image color channels, perform pooling on each channel separately, and fuse the results, which can be expressed as: ； in, Represents the total number of channels of input data, Indicates the number of a specific channel, specifically an index from 1 to C, For multi-channel input data No. The pixel value of each channel, is the pixel value in the input feature map, Control the pixel value within the pooling window, It is the multi-dimensional feature fusion after pooling.

4. The temperature control method for Chinese food cooking based on multimodal characterization and recognition according to claim 3, characterized in that: The method captures dynamic texture features and monitors cooking images: Fusion features based on extracted multi-dimensional features , the extracted features are classified and regressed through the fully connected layer; Fusing the extracted multidimensional features The image data in the image is converted into a single-channel grayscale image, and a grayscale co-occurrence matrix is ​​constructed to describe the spatial relationship between pixels, and the multidimensional features are fused into features. , defined as the size of The two-dimensional matrix, is the number of rows in the image matrix, is the number of columns in the image matrix, where each pixel value Belongs to a grayscale set, ,in is the number of grayscale levels of the input grayscale image, and the co-occurrence frequency of different grayscale levels in the input grayscale image at the specified direction and distance is calculated, which is expressed as: ； in, is the gray-level co-occurrence matrix, representing the gray level and Between in the direction and Co-occurrence frequency under distance, is the Kronecker delta function, is the input grayscale image Middle position The pixel value of is the distance between pixels, is the calculation direction; The calculated gray-level co-occurrence matrix After normalization, it is expressed as: ； in, is the normalized gray-level co-occurrence matrix, the sum of the matrix is ​​1; Capturing the texture features of the image, introducing the local gradient information of the image, the gradient amplitude of the image is calculated by the following formula, expressed as: ； in, is an image In position The gradient of , which indicates the intensity of image change; The normalized gray-level co-occurrence matrix Perform time integration and accumulate the texture changes of the image at different time steps, which can be expressed as: ； in, is the cumulative gray-level co-occurrence matrix, which represents the The accumulated gray level co-occurrence frequency within is the grayscale image at time The pixel value at the moment, is the time variable, representing different time steps in the cooking process; in, The range of is [0,∞), which represents the co-occurrence frequency between gray level pairs. The range of is [0, 1], which means the normalized co-occurrence matrix. The value of each element reflects the relative frequency of the gray level pair. The range of is [0,∞), which represents the cumulative co-occurrence matrix; Based on the accumulated co-occurrence matrix, the dynamic characteristics of texture changes are captured to form a comprehensive description of the evolution of image content. Cooking images are monitored in real time, and time integration is performed on texture changes to obtain feedback signals, which provides a basis for the calculation of feedback functions.

5. The temperature control method for Chinese food cooking based on multimodal characterization and recognition according to claim 4, characterized in that: The introduction of feedback variables to adjust the temperature control specifically includes: Introduce the feedback function required for temperature control strategy adjustment and build the feedback function Indicates at time The cooking status feedback is expressed as: ； in, Representative Feedback variables, Z ; Introducing a time decay factor , γ is the attenuation factor, set the distance from the current time The farther the feedback is, the smaller the impact on the current temperature control strategy will be. The impact at any time is expressed as: ； in, It is the feedback function adjusted by time decay, which represents the impact of historical feedback after the time decay factor; Introduce the spatial feedback attenuation factor to control the feedback effect at different spatial positions, using the Gaussian attenuation function represents the spatial effect, where is the spatial attenuation factor, and Represents spatial coordinates; Use a high-order integral function to synthesize the feedback information, integrate the feedback information of all historical time and space regions, and calculate the adjustment amount of the temperature control strategy: ； in, Indicates at time When, spatial position The local feedback strength, Adjust the impact of feedback on the temperature control strategy and increase its nonlinearity. Use the Sigmoid function to compress the feedback information into : ； in, Is the feedback value The Sigmoid function maps the input value to In the range of Input, It is an exponential function of the natural logarithm base, which represents the nonlinear characteristics of the Sigmoid function; Introducing Sigmoid function to feedback value Perform nonlinear scaling to control the effect of feedback on the temperature control strategy, and finally obtain the temperature control strategy adjustment model. The calculation is expressed as: ； The specific calculation process is: Get in time and spatial location The actual temperature value at ; Get in time and spatial location The actual humidity value at ; Obtain color and texture feature data of food surface and calculate texture features using image analysis algorithm; Set cooking target temperature , set the target humidity , set target state of doneness and desired food doneness; Calculate the temperature feedback, expressed as: ； Calculate the humidity feedback, expressed as: ； Introducing a time decay factor , controls the degree of weakening of historical feedback over time, expressed as: ； in, is the attenuation factor, which determines the impact of historical feedback on the current moment; A Gaussian decay function is introduced to control the feedback effect at different spatial positions: ； in, is the sum of the squares of the distances from the center position, used to calculate the distance of each pixel relative to the center position. It is a factor used to control the attenuation speed of spatial feedback effect and adjust the value of α; Combining time attenuation and space attenuation, the final humidity feedback value is obtained, which is expressed as: ； is the feedback function, which is used to express the error and deviation from the target value. Exponential decay factor, indicating that the impact of historical feedback on the current temperature control strategy gradually weakens, the integral interval , indicating the time from the start of cooking to the current moment All time feedback, For the time The system state value at time t is obtained by accumulating historical feedback with T0; in For the current moment The target temperature you want to achieve; based on The temperature difference with the current time t is used to adjust the temperature control.

6. A temperature control system according to any one of claims 1 to 5, wherein: The system includes: Temperature sensor module, used to monitor the surface and internal temperature of food in real time; Humidity sensor module, used to monitor humidity changes on the food surface; Gas concentration sensor module, used to monitor the concentration of steam, oil smoke and water vapor; The image acquisition module, including an RGB camera and an infrared thermal imaging camera, captures image changes on the surface and inside of food and identifies the state of food cooking; Environmental monitoring sensor module, used to monitor temperature, humidity, and airflow environment data in the kitchen in real time; The data acquisition module is used to collect temperature, humidity, gas concentration, image data and other information from each sensor in real time, and time-calibrate the data according to time steps; Data cleaning and preprocessing module, used to clean the collected data, including denoising, outlier detection, missing data filling, etc.; Data fusion and analysis module, used to perform fusion analysis on sensor data, including image feature extraction using image processing technology, convolutional neural network, multi-dimensional data fusion, gray-level co-occurrence matrix analysis and texture feature extraction; The dynamic temperature control adjustment is achieved through the following steps: The cooking state at each moment is calculated through the feedback function, and the time decay factor and space decay factor are used to adjust the impact of historical feedback; Adjust temperature control parameters based on real-time data and the target temperature and humidity set by the user.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the temperature control method for Chinese food cooking based on multimodal representation and recognition are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the temperature control method for Chinese food cooking based on multimodal representation and recognition are implemented.

Images