Kitchen monitoring, detecting and monitoring system based on cloud platform

Abstract

The invention relates to the technical field of intelligent monitoring, and discloses a kitchen monitoring, detecting and monitoring system based on a cloud platform, which comprises a sensor acquisition module, a data preprocessing module, a data fusion and optimization module, a cloud platform reasoning and analysis module, a data feedback and alarm module and a reinforcement learning optimization module. The invention also provides a kitchen monitoring, detecting and monitoring method based on the cloud platform. The method comprises the following steps: collecting multi-modal data in a kitchen environment; performing noise removal, format standardization and abnormal value elimination on the collected data; optimizing a sensor data fusion process by adopting a weighted fusion method; and optimizing a data fusion strategy by adopting a mutual information analysis method. The multi-modal data fusion and optimization strategy is adopted, the weighted fusion method and the mutual information analysis technology are combined, the precision and efficiency of data processing are improved, and the system can reflect the real situation in the kitchen environment more accurately by conducting weighted processing and optimization fusion on various sensor data.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent monitoring, and specifically provides a kitchen monitoring, detection and guardianship system based on a cloud platform. Background Art

[0002] With the rapid development of smart home and Internet of Things technologies, the safety issues of kitchen environments have gradually attracted the attention of more and more families and commercial kitchens. Most traditional kitchen monitoring systems use a single type of sensor to monitor potential hazards such as fires and gas leaks in the kitchen. However, these single monitoring methods often fail to meet the changing safety requirements in complex kitchen environments. When using multiple sensors for data collection, the output results of different sensors may vary due to problems such as sensor quality, environmental interference, and faults, which in turn makes it difficult for the system to effectively integrate this multi-source data.

[0003] Kitchen monitoring systems often rely on fixed rules and thresholds to trigger alarms. For example, when the temperature or gas concentration exceeds a certain preset threshold, the system will trigger an alarm. However, this method does not fully consider the dynamic changes and complexity of the kitchen environment. For example, factors such as oil fume and steam may affect the sensors, resulting in false alarms or missed alarms. In addition, the reliance on fixed rules also makes the system lack flexibility, difficult to adapt to the needs of different kitchen environments, and fails to make full use of the correlation between multi-source data, resulting in the accuracy and response speed of the system being affected when dealing with complex events; moreover, existing monitoring systems often rely on manual intervention for safety management, resulting in a problem of delayed response. For example, the method of manual inspection and monitoring is easily affected by human factors. When an emergency such as a fire or gas leak occurs in the kitchen, the response of manual intervention may not be completed in the first time, thus delaying the opportunity for emergency response. Although some systems in the prior art support remote monitoring and alarm, these functions usually rely on traditional alarm methods and lack an intelligent processing mechanism; furthermore, existing kitchen monitoring systems also have bottlenecks in data processing capabilities. With the increase in the number of devices, the sharp increase in sensor data volume puts great data processing pressure on the cloud platform. When traditional systems process massive data, they often lack efficient optimization strategies such as data compression, transmission optimization, and parallel processing of algorithms, resulting in system response delays and data transmission instability. Therefore, those skilled in the art have proposed a kitchen monitoring, detection and guardianship system based on a cloud platform to solve the above problems. Summary of the Invention

[0004] Aiming at the deficiencies of the prior art, the present invention provides a kitchen monitoring, detection and guardianship system based on a cloud platform, which solves many deficiencies of the existing kitchen monitoring systems in aspects such as data fusion, intelligent processing, alarm accuracy, and response speed.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solutions: A kitchen monitoring, detection, and guardianship system based on a cloud platform, comprising:

[0006] A sensor acquisition module for acquiring multi-modal sensor data in the kitchen environment, where the data includes temperature and humidity, smoke concentration, gas concentration, image data, and audio data;

[0007] A data preprocessing module located on an edge computing device for removing noise, performing data smoothing processing on the data acquired by the sensors, and performing preliminary fusion;

[0008] A data fusion and optimization module connected to the data preprocessing module, including an edge computing unit and a cloud computing unit. The edge computing unit is used for standardizing the data format and removing outliers, and the cloud computing unit is used for further optimizing the sensor data based on a weighted fusion method and a mutual information analysis method to improve the data fusion quality;

[0009] A cloud platform inference and analysis module for receiving the data transmitted by the data fusion and optimization module and performing inference and analysis on the data based on a deep learning model to identify risk factors in the kitchen environment and generate an alarm signal;

[0010] A data feedback and alarm module for receiving the alarm signal from the cloud platform inference and analysis module and sending the alarm information to a user device through network communication. The user device includes a smartphone, a tablet computer, or a smart speaker;

[0011] A reinforcement learning optimization module for adjusting the fusion parameters in the data fusion and optimization module based on the historical records and real-time feedback of sensor data fusion to optimize the data processing strategy.

[0012] Preferably, the sensor acquisition module includes:

[0013] A temperature and humidity sensor for detecting the temperature and humidity of the kitchen environment;

[0014] A smoke sensor for detecting changes in the smoke concentration in the kitchen;

[0015] A gas sensor for detecting the concentration of combustible gas in the kitchen environment;

[0016] A high-definition camera for acquiring image information inside the kitchen;

[0017] A microphone for acquiring audio information in the kitchen environment.

[0018] Preferably, the data preprocessing module includes:

[0019] A filtering processing unit for removing noise and smoothing the signal from the sensor data;

[0020] A data standardization unit for uniformly processing the data formats of different sensors for subsequent fusion;

[0021] An abnormal data detection unit for identifying and removing abnormal or incorrect data.

[0022] Preferably, the cloud computing unit of the data fusion and optimization module includes:

[0023] A weighted fusion unit for calculating a weighted value based on the confidence of the sensor data and performing multi-source data fusion;

[0024] A mutual information analysis unit for calculating the correlation between different sensor data and optimizing the data fusion strategy.

[0025] Preferably, the deep learning models used in the cloud platform inference and analysis module include:

[0026] A convolutional neural network CNN for processing the image data collected by a high-definition camera;

[0027] A long short-term memory network LSTM for processing the audio data collected by a microphone;

[0028] A target classification model for analyzing the result of sensor data fusion and outputting the dangerous category information.

[0029] Preferably, the data feedback and alarm module includes:

[0030] An acoustic-optic alarm unit for triggering an acoustic-optic alarm signal inside the kitchen;

[0031] A network communication unit for sending the alarm information to a remote user device via Wi-Fi or a cellular network;

[0032] A remote control unit for allowing the user to perform emergency control operations through a mobile device, including closing the gas valve or activating the fire extinguishing device.

[0033] Preferably, the reinforcement learning optimization module includes:

[0034] A data feedback unit for receiving the result of historical data fusion and storing it in the database;

[0035] A policy adjustment unit for updating the parameters of the data fusion algorithm based on the reinforcement learning method;

[0036] A model training unit for continuously training the data fusion algorithm in the cloud to optimize the data processing strategy.

[0037] Preferably, the weighted fusion unit optimizes the data fusion process through the following steps:

[0038] Calculate the confidence levels of different sensor data and set initial fusion weights based on the confidence levels;

[0039] Adjust the weight allocation of each sensor data according to the historical fusion results, so that the data with high credibility accounts for a higher proportion in the fusion;

[0040] Calculate the error of the fused data and adjust the parameters of the fusion algorithm according to the error to improve the fusion accuracy;

[0041] Continuously monitor the reliability of the fused data and dynamically adjust the fusion strategy.

[0042] Preferably, the steps for the mutual information analysis unit to optimize the data fusion strategy include:

[0043] Calculate the independent information entropy of each sensor data to evaluate the effectiveness of the data;

[0044] Calculate the joint information entropy of the sensor data to determine the cross-correlation degree of different sensor data;

[0045] Adjust the data fusion algorithm based on the mutual information value of the sensor data to minimize data redundancy and improve information utilization;

[0046] Dynamically update the mutual information calculation results during the data fusion process to adapt to environmental changes.

[0047] There is also provided a kitchen monitoring, detection and guardianship method based on a cloud platform, including the following steps:

[0048] Collect multi-modal data in the kitchen environment;

[0049] Perform noise removal, format standardization and outlier rejection on the collected data;

[0050] Adopt a weighted fusion method to optimize the sensor data fusion process;

[0051] Adopt a mutual information analysis method to optimize the data fusion strategy and improve data utilization;

[0052] Perform inference analysis on the fused data through a deep learning model and generate an alarm message;

[0053] Send the alarm message to the user device through an audible and visual alarm device and network communication;

[0054] Dynamically adjust the data fusion parameters through a reinforcement learning optimization algorithm to improve the system's adaptive ability.

[0055] The present invention provides a kitchen monitoring, detection and guardianship system based on a cloud platform. It has the following beneficial effects:

[0056] 1. The present invention adopts a multi-modal data fusion and optimization strategy, combines a weighted fusion method and mutual information analysis technology, improves the accuracy and efficiency of data processing. By performing weighted processing and optimized fusion on various sensor data, the system can more accurately reflect the real situation in the kitchen environment. Compared with the traditional single-sensor monitoring scheme, the present invention solves the monitoring errors caused by data redundancy or inconsistency, and improves the reliability and response speed of the monitoring system.

[0057] 2. The present invention uses a deep learning model to perform inference and analysis on data, and combines a reinforcement learning optimization algorithm to dynamically adjust data fusion parameters. This technical solution effectively improves the intelligent level of the system, makes the monitoring of the kitchen environment more adaptable. Compared with the existing systems that rely on fixed thresholds and rules, the present invention improves the recognition ability of sudden dangerous events through real-time adjustment, and avoids false alarms and missed alarms.

[0058] 3. The present invention realizes real-time data transmission and intelligent alarm through the cloud platform, and provides an efficient remote monitoring and management system. When an abnormality occurs in the kitchen environment, the system can timely send alarm information through an acoustic-optic alarm device and a remote device. Compared with the traditional manual inspection and response mechanism in the prior art, the present invention can greatly reduce the response time and manual intervention, and improve the kitchen safety and management efficiency.

[0059] 4. The present invention adopts a systematic data preprocessing method, including noise removal, format standardization and outlier rejection, which improves the quality and accuracy of data. This solution solves the problem of judgment errors caused by noise or data inconsistency in traditional monitoring systems, ensures the accurate transmission and fusion of data, and provides a reliable basis for subsequent inference analysis and alarm decision-making. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 is a schematic diagram of the system architecture of the present invention;

[0061] Figure 2 is a schematic diagram of the architecture of the sensor acquisition module of the present invention;

[0062] Figure 3 is a schematic diagram of the architecture of the data preprocessing module of the present invention;

[0063] Figure 4 is a schematic diagram of the architecture of the data fusion and optimization module of the present invention;

[0064] Figure 5 is a schematic diagram of the architecture of the cloud platform inference and analysis module of the present invention;

[0065] Figure 6 Schematic diagram of the data feedback and alarm module framework of the present invention;

[0066] Figure 7 Schematic diagram of the reinforcement learning optimization module framework of the present invention;

[0067] Figure 8 Schematic diagram of the method flow of the present invention. Specific embodiments

[0068] Next, in combination with the accompanying drawings of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0069] Please refer to the attached Figure 1 - attached Figure 7 , the embodiment of the present invention provides a kitchen monitoring, detection and guardianship system based on a cloud platform, including:

[0070] A sensor acquisition module for acquiring multi-modal sensor data in the kitchen environment, where the data includes temperature and humidity, smoke concentration, gas concentration, image data and audio data;

[0071] Specifically, in the kitchen monitoring, detection and guardianship system of the present invention, the sensor acquisition module is the basic module of the entire system, and this module directly determines the accuracy and reliability of subsequent data processing, fusion optimization, and intelligent analysis. Therefore, in terms of sensor selection, data acquisition, communication methods, etc., this module has high design requirements. Generally, the kitchen environment has large temperature and humidity fluctuations, and there are also various complex working conditions such as smoke, lampblack, and gas leakage. The stability and real-time performance of the sensor are particularly crucial. As an option, this module adopts a multi-modal sensor configuration scheme to comprehensively obtain information in different dimensions to ensure the integrity and accuracy of the data. Specifically, the sensor acquisition module mainly includes temperature and humidity sensors, smoke sensors, gas sensors, high-definition cameras, and microphones, and is optimized by combining data synchronization technology.

[0072] In this embodiment, the temperature and humidity sensor is used to monitor the temperature and humidity changes in the kitchen environment, and its data can be used for the judgment of the precursor of a fire. For example, when the temperature rises abnormally and at the same time the smoke concentration is detected to increase, the system can give an early warning. Among them, the temperature and humidity sensor adopts a digital output method and performs data interaction with the edge computing device through the I 2 C or SPI bus to reduce analog signal interference and improve measurement accuracy.

[0073] The smoke sensor is used to detect the particulate matter concentration in the kitchen air, especially the particulate changes caused by combustion or cooking fumes. Specifically, the smoke sensor adopts a photoelectric or ion detection method, and identifies the concentration of smoke in the air through infrared scattering or ionization changes. The smoke concentration data can be calculated by the formula:

[0074] C s = K·V;

[0075] Where: C s represents the smoke concentration (ppm); K is the sensor calibration coefficient; V is the voltage signal (V) output by the sensor.

[0076] Generally, the measurement range of the sensor can reach 0 - 1000 ppm, and the error is controlled within ±5 ppm. As an option, the sensor can communicate with the edge computing device through a PWM signal or an analog-to-digital conversion interface to achieve efficient data transmission.

[0077] In this embodiment, the gas sensor is used to detect harmful gases in the kitchen environment, including but not limited to methane, carbon monoxide, propane, etc. Specifically, the gas sensor is based on semiconductor sensitive materials or electrochemical detection principles, and generates measurable electrical signals through the adsorption and oxidation reactions of gas molecules. The relationship between the output signal of the sensor and the gas concentration can be expressed as:

[0078] C g = A·e( B·V );

[0079] Where: C g is the gas concentration (ppm); A and B are the sensor calibration parameters; V is the output voltage (V) of the sensor; e is a mathematical constant.

[0080] In some embodiments, the gas sensor supports multi-gas detection, can selectively detect specific gases through different sensitive material layers, and combines a temperature and humidity compensation algorithm to improve the measurement accuracy.

[0081] The high-definition camera is used to capture the visual information inside the kitchen. Generally, the camera has an infrared night vision function to adapt to different lighting conditions. The camera resolution can be 1080P or higher, and the field of view angle is not less than 120°. It supports automatic white balance and wide dynamic range adjustment. In some embodiments, the camera data is transmitted to the edge computing device through the RTSP stream protocol or MJPEG encoding to reduce bandwidth occupancy.

[0082] In this embodiment, the microphone is used to collect sound information in the kitchen environment, such as abnormal noises, explosions, gas leakage sounds, etc. As an option, the microphone adopts MEMS technology and has the characteristics of high sensitivity and low power consumption. The audio signal can be processed by Fourier transform to extract frequency features. The formula is as follows:

[0083]

[0084] Where: X(f) is the frequency-domain representation of the audio signal; x(t) is the time-domain signal; f is the frequency (Hz); e -j2πft is the basis function of the Fourier transform, representing sinusoidal wave components of different frequencies; dt is the integration variable, representing the summation of the signal over the entire time range.

[0085] Generally, the audio sampling rate is set at 16 kHz to ensure sufficient spectral resolution. In a possible implementation, the audio signal is preprocessed by an edge computing device, including operations such as noise suppression and feature extraction.

[0086] In some embodiments, to ensure the time synchronization of multi-sensor data, the sensor acquisition module adopts a hardware clock synchronization or software timestamp mechanism. Generally, the software synchronization method corrects the sensor timestamp through the NTP protocol, while the hardware synchronization can achieve higher-precision time alignment through PTP (Precision Time Protocol), and the data synchronization deviation is controlled within 1 ms to ensure the accuracy of the fusion calculation.

[0087] In this embodiment, the data transmission method of the sensor acquisition module is selected according to the specific application environment. In some embodiments, short-distance communication can use I 2 C, SPI or UART for data transmission, while for remote transmission, Wi-Fi, Zigbee or LoRa protocols can be used. In low-power application scenarios, the Zigbee protocol can be preferentially selected to reduce the energy consumption of the device.

[0088] The data preprocessing module, located on the edge computing device, is used to remove noise, smooth the data, and perform preliminary fusion on the data collected by the sensors;

[0089] Specifically, in the kitchen monitoring, detection, and guardianship system of the present invention, the data preprocessing module plays a crucial role, especially in the process of processing multi-modal sensor data. The raw data generated by the sensor acquisition module usually contains noise, outliers, or missing data. If this data is not effectively processed, it may affect the subsequent data fusion and analysis effects. Therefore, the core task of the data preprocessing module is to ensure that the data collected from each sensor can be accurately and stably input into the subsequent processing links. This module mainly consists of a filtering processing unit, a data standardization unit, and an abnormal data detection unit. Data preprocessing not only includes basic noise removal and outlier elimination but also requires unified and standardized processing of different types of sensor data to ensure data quality and fusion accuracy.

[0090] In this embodiment, the main task of the filtering processing unit is to remove noise and smooth the signal from the sensor data. In the kitchen environment, sensor data is interfered by various factors. For example, the temperature and humidity sensor may introduce slight noise due to air fluctuations, and the gas sensor may be affected by other environmental gases to generate interference. To remove these unwanted interferences, the filtering processing unit uses the Kalman filtering algorithm for processing. Kalman filtering is an algorithm widely used in the state estimation of dynamic systems and can provide relatively accurate estimated values in a noisy environment.

[0091] Generally, the working principle of Kalman filtering is through a recursive algorithm that continuously estimates the current state based on the measurement data of the sensor and the system model, updates, and predicts the next state. The basic formula of Kalman filtering is as follows:

[0092]

[0093] Where: is the estimate of the current state; z k is the observed value of the sensor; H is the observation matrix; K k is the Kalman gain, representing the weighted relationship between the new observed value and the predicted value; is the state estimate value predicted at the previous moment k - 1. The Kalman gain is calculated by the following formula:

[0094] K k = P k-1 H T (HP k-1 H T + R) -1 ;

[0095] Where: P k-1 is the previous estimation error covariance matrix; R is the measurement noise covariance matrix of the sensor; H T is the transpose of the observation matrix H.

[0096] The filtering process updates and optimizes the estimated value of the sensor data through the above formula to reduce the impact of noise on the final data.

[0097] In some embodiments, the data normalization unit is responsible for normalizing multi-modal data from different sensors. Since the measurement ranges and units of each sensor are different, directly fusing this data may lead to information distortion or error accumulation. Therefore, the purpose of data normalization is to convert the outputs of all sensors into a unified scale for subsequent fusion analysis.

[0098] Specifically, for the output data of temperature and humidity sensors, smoke sensors, gas sensors, etc., it is usually first normalized to scale its value to the same range. Assume that the data x of a certain sensor is in the range [a, b], and the normalized data x ′ is calculated by the following formula:

[0099]

[0100] In this normalization method, the data range is transformed to be between [a, b], ensuring the comparability and consistency of data from different sensors. In some embodiments, Z-score normalization can also be used, that is:

[0101]

[0102] where: μ and σ are the mean and standard deviation of the sample respectively; a is the minimum value of the data; b is the maximum value of the data; x ′ is the normalized data; x is the data of the sensor.

[0103] Z-score normalization can effectively eliminate the dimensional differences of different data, enabling the data to be processed on the same scale and reducing the fusion error caused by data dimensional differences.

[0104] The abnormal data detection unit is used to identify and eliminate unreasonable, abnormal or missing sensor data. In the actual environment, sensors sometimes produce incorrect readings due to faults, external interference or environmental factors. For example, a temperature and humidity sensor may produce extremely high readings in a short period of time, while a gas sensor may produce distorted data due to sensor aging or contamination. To avoid the impact of these abnormal data on the system's judgment, the abnormal data detection unit adopts a threshold-based detection method and combines historical data for judgment.

[0105] In general, if the sensor output data exceeds the predetermined physical quantity range or is significantly inconsistent with the historical data trend, the system will determine it as abnormal data. For example, the output range of a temperature and humidity sensor is generally from 0°C to 100°C. If the reading is lower than 0°C or higher than 100°C, the system will automatically exclude this data. In addition, the anomaly detection unit can also use statistical methods, such as the mean-standard deviation method or the IQR method (interquartile range method), to identify outliers. For example, when using the mean-standard deviation method, if the sensor output exceeds twice the standard deviation range of the mean, the data will be considered an outlier and thus excluded.

[0106] In some embodiments, data standardization and outlier exclusion can be dynamically adjusted in combination with specific application scenarios. For example, in an environment with high-precision requirements, the reliability and accuracy of data can be ensured by increasing the sampling frequency of sensor data, adjusting the parameters of the Kalman filter, or optimizing the anomaly detection algorithm.

[0107] The data fusion and optimization module is connected to the data preprocessing module and is used to standardize the data format and exclude outliers, and further optimize the sensor data based on the weighted fusion method and the mutual information analysis method;

[0108] Specifically, in the kitchen monitoring, detection and guardianship system of the present invention, the data fusion and optimization module is a key bridge connecting the aforementioned sensor acquisition module, data preprocessing module and cloud platform inference and analysis module. The main task of this module is to effectively fuse and optimize multi-modal data from different types of sensors, improve the accuracy and reliability of the data, and provide high-quality input data for subsequent intelligent analysis and decision-making. Since the sensor data comes from multiple heterogeneous sensors, there may be differences in data format, dimension, accuracy, etc. Therefore, how to achieve efficient data fusion is the core task of this module. This module includes a weighted fusion unit and a mutual information analysis unit, which are responsible for formulating and optimizing the data fusion strategy respectively. The weighted fusion unit performs weighted fusion on the data according to the credibility of the sensors, while the mutual information analysis unit is used to evaluate the correlation between sensor data, so as to optimize the data fusion process.

[0109] In this embodiment, the weighted fusion unit is used to perform weighted fusion on the data from different sensors according to the confidence of the sensor data. The purpose of weighted fusion is to reasonably allocate the weight of each sensor in the final fusion result according to the data quality of each sensor, so as to improve the overall accuracy of the system. Generally, the accuracy and reliability of different types of sensors are different. For example, the error of a temperature and humidity sensor is relatively small, while a smoke sensor may be affected by environmental factors, resulting in a large measurement error. Therefore, the weighted fusion unit dynamically adjusts its weight according to the reliability and measurement error of each sensor.

[0110] Specifically, assume that the sensors S1, S2, …, S n provide data D1, D2, …, D n , and the corresponding confidence levels are w1, w2, …, w n , then the fused data D f can be expressed as a weighted average:

[0111]

[0112] where: w1, w2, …, w n represent the weight values of sensors S1, S2, …, S n , and are usually calculated based on the error range or accuracy of the sensors.

[0113] As an option, the weight w i can be calculated by the following formula:

[0114]

[0115] where: σ i is the standard deviation of sensor S i . The smaller the standard deviation, the more reliable the output value of the sensor, and the greater its weight w i should be.

[0116] The weighted fusion unit adjusts the weights according to the measurement uncertainties of different sensors to ensure that higher-precision sensors account for a larger proportion in the fusion.

[0117] In some embodiments, the mutual information analysis unit is used to evaluate the correlation between the data of each sensor and optimize the data fusion process according to the calculated mutual information. Mutual information is an index to measure the degree of dependence between two variables and is usually used for feature selection and data fusion. In the process of fusing sensor data, there may be a high correlation between the data of certain sensors, and the existence of redundant data may affect the fusion effect. Therefore, through mutual information analysis, it is possible to identify which sensor data are strongly correlated, so as to reasonably select the fusion strategy, reduce redundant information, and improve the fusion effect.

[0118] The calculation of mutual information is usually carried out by the following formula:

[0119] I(X,Y) = H(X) + H(Y) - H(X,Y);

[0120] where: I(X,Y) represents the mutual information between the random variables X and Y; H(X) and H(Y) are the entropies of X and Y respectively, representing their respective uncertainties; H(X,Y) is the joint entropy of X and Y, representing the uncertainty of their joint distribution.

[0121] Specifically, if there is strong mutual information between the data of two sensors, that is, I(X,Y) is large, it indicates that the data of these two sensors have a high correlation. When fusing, it may be necessary to reduce the weight of one of the sensors to avoid information redundancy.

[0122] For multi-modal sensor data, the mutual information analysis unit evaluates which sensors have a high redundancy and which sensor data can provide independent information by calculating the mutual information between different sensors, so as to dynamically optimize the fusion strategy. For example, in a kitchen environment, a smoke sensor and a gas sensor may generate highly correlated data, but the data of a temperature and humidity sensor may be independent of other sensors. Therefore, during the data fusion process, the system can automatically adjust the data weights of these sensors based on the calculation results of mutual information.

[0123] In some embodiments, the calculation of mutual information analysis can also be further optimized by combining the temporal correlation and spatial distribution of the data. In a kitchen environment, some sensors may be affected by temporary disturbances (such as lampblack, steam, etc.). At this time, the mutual information analysis based on time series data can help identify irrelevant interference information and improve the accuracy of data fusion.

[0124] The combination of the weighted fusion unit and the mutual information analysis unit can effectively reduce the fusion error caused by sensor measurement errors, redundant data, and irrelevant data. Through the collaborative work of these two units, this module can provide more accurate data support for subsequent intelligent reasoning and hazard detection.

[0125] The cloud platform reasoning and analysis module is used to receive the data transmitted by the data fusion and optimization module, and based on the deep learning model, it conducts reasoning and analysis on the data, identifies the risk factors in the kitchen environment, and generates an alarm signal.

[0126] Specifically, in the kitchen monitoring, detection, and guardianship system of the present invention, the cloud platform reasoning and analysis module is the core decision-making unit of the entire system. By deeply analyzing the fused multi-modal sensor data, this module can monitor and predict the abnormal states and potential hazards in the kitchen environment in real time. The cloud platform is not only responsible for receiving the data from each sensor and data processing module, but also uses deep learning and machine learning algorithms for intelligent reasoning, making judgments and outputting corresponding feedback. The key technologies of this module include convolutional neural network (CNN), long short-term memory network (LSTM), and target classification models. These technologies are comprehensively applied to classify and predict the kitchen monitoring data to achieve intelligent risk warning.

[0127] In this embodiment, a convolutional neural network (CNN) is used for feature extraction and deep learning of sensor data. In a kitchen environment, the data collected by sensors often has complex spatial and temporal characteristics, and CNN has good image processing capabilities and can perform multi-level feature extraction on the input data through convolutional operations. In some embodiments, the sensor data may be converted into a data format similar to an image. For example, the time series data of gas sensors, temperature and humidity sensors, etc. is converted into a two-dimensional matrix and then input into CNN for processing. CNN processes the input data layer by layer through multiple convolutional layers, pooling layers, and fully connected layers, thereby automatically extracting useful features.

[0128] Specifically, assuming that the input data is a multi-dimensional matrix X, after convolutional operation, a feature map F is obtained, and the formula for the convolutional operation is:

[0129] F i,j =(X * W) i,j + b1;

[0130] where: W is the convolutional kernel; b1 is the bias term; * represents the convolutional operation; F i,j is the feature map after convolution, representing the response of a specific area.

[0131] Through multiple convolutional and pooling layers, CNN can effectively extract the spatial features and local patterns in the sensor data, thereby providing strong feature support for subsequent classification and prediction.

[0132] The long short-term memory network (LSTM) is used in this embodiment to process time series data, especially for time series prediction of sensor data in a kitchen environment. LSTM is a special recurrent neural network (RNN) that can effectively solve the problem of gradient disappearance in traditional RNNs in long-term dependence problems. Since many abnormal events in the kitchen environment, such as fires, gas leaks, etc., usually show sudden changes in time, LSTM performs well in capturing these time series patterns.

[0133] In some embodiments, LSTM can capture the time series features in the data by learning historical sensor data and predict possible abnormal events in the future. Specifically, the working principle of LSTM is based on three main gating mechanisms: the forget gate, the input gate, and the output gate, which control the forgetting, updating, and output of information respectively. Assuming that the current input is x t , and the hidden state is h t , then the state update formula of LSTM is as follows:

[0134] f t = σ(W f [h t-1 , x t1+b f );

[0135] i t =σ(W i [h t-1 ,x t1 +b i );

[0136] o t =σ(W o [h t-1 ,x t1 +b o );

[0137] c t =f t *c t-1 +i t *tanh(W c [h t-1 ,x t1 +b c );

[0138] h t =o t *tanh(c t );

[0139] Where: f t is the forget gate; i t is the input gate; o t is the output gate; c t is the cell state; h t is the hidden state; x t1 is the current input; h t-1 is the hidden state at the previous time step; c t-1 is the cell state at the previous time step; W f , W i , W o , W c are the weight matrices corresponding to the gating mechanisms; b f , b i , b o , b c are the bias terms corresponding to the gating mechanisms; σ(·) is the Sigmoid activation function; tanh(·) is the hyperbolic tangent activation function.

[0140] Through these gating mechanisms, the LSTM can selectively remember and forget information in the time series, thus accurately capturing the long-term dependencies in the time series.

[0141] The target classification model is used to classify the data and determine whether there are abnormalities or potential dangers in the kitchen environment. In some embodiments, the target classification model adopts a combination of a convolutional neural network and a long short-term memory network, and determines whether there are dangerous situations such as fires and gas leaks by performing deep learning and time series analysis on the characteristics of kitchen sensor data.

[0142] In this embodiment, the target classification model combines the advantages of CNN and LSTM. First, CNN is used to extract features from the sensor data, then LSTM is used to analyze the time series information, and finally the data is classified and output through a fully connected layer. The output of the classification model can be normalized by the Softmax function to obtain the probabilities of each category:

[0143]

[0144] where: z k is the activation value of the output layer; K1 is the number of categories; P(y = k|X) is the probability that the input data X belongs to category k; is the sum of all category exponential values; the exponential operation result of the output layer neuron.

[0145] Through such a classification model, the cloud platform can accurately determine the possible dangerous situations in the sensor data.

[0146] The data feedback and alarm module is used to receive the alarm signal from the inference and analysis module of the cloud platform and send the alarm information to the user device through network communication. The user device includes a smart phone, a tablet computer or a smart speaker;

[0147] Specifically, in the kitchen monitoring, detection and guardianship system of the present invention, the alarm and remote control module, as the last execution unit, is responsible for performing operations such as alarm, warning and remote control after the inference and analysis module of the cloud platform makes an intelligent judgment on the data, so as to ensure that the system can respond to potential dangers in the kitchen environment in a timely and effective manner. This module includes an audible and visual alarm unit, a network communication unit and a remote control unit. These units work together to ensure the safety of the kitchen through a real-time alarm mechanism and remote intervention means. Through the audible and visual alarm unit, the system can emit sound and visual signals to remind the user when an abnormality or danger is detected. At the same time, the alarm information is transmitted to the remote device through the network communication unit, and finally the control measures are enabled through the remote control unit to ensure the effective management of the kitchen equipment and environment.

[0148] In this embodiment, the main task of the audible and visual alarm unit is to remind kitchen staff or other personnel of potential dangers through audible and visual signals. This unit usually includes an alarm sound emitting device (such as a buzzer, speaker) and a visual warning device (such as a flashing warning light, LED screen, etc.). When the cloud platform reasoning and analysis module detects an abnormality (such as a fire, gas leakage, high temperature, etc.), the audible and visual alarm unit immediately operates and emits a highly recognizable warning signal to attract the attention of on-site personnel. Generally, the audible and visual alarm unit will select a combination of a high-frequency sound and a bright flashing light to ensure the effectiveness of the warning. Specifically, the control module of the audible and visual alarm unit can receive a trigger signal from the cloud platform reasoning module and drive the buzzer and LED lights through a control circuit for alarm output.

[0149] In some embodiments, the sound frequency of the alarm can be adjusted according to the ambient noise to ensure that the alarm sound is clearly audible. For example, when the noise in the kitchen is relatively high, the system may increase the loudness or adjust the frequency of the alarm sound to make the alarm sound more prominent. The color and flashing frequency of the visual warning light can also be adjusted according to the type of different dangers. For example, for a fire alarm, a flashing red warning light can more effectively attract people's attention.

[0150] The network communication unit is used to transmit the status and alarm information of the kitchen monitoring system to remote devices in real time, such as terminals like mobile phones, tablets, PCs, etc. Through this unit, the system can realize remote monitoring and alarm functions. Users can view the status of the kitchen anytime and anywhere and respond when an abnormality occurs. Generally, the network communication unit adopts wireless communication technologies such as Wi-Fi, Bluetooth, Zigbee, etc. to ensure the efficient transmission of data.

[0151] Specifically, the network communication unit sends real-time alarm information to users by establishing a communication channel with the cloud platform, including detailed information such as the type of alarm, the specific location where it occurred, and the time when it occurred. To ensure the timeliness and stability of communication, the communication unit may adopt data compression and encryption technologies to improve the data transmission speed and security. For example, when a gas leakage alarm occurs in the system, the network communication unit will trigger a preset push notification, quickly transmit the alarm information to the user, and attach detailed alarm data to ensure that the user can obtain on-site information in a timely manner.

[0152] After receiving the alarm, the remote control unit can execute remote control instructions to perform real-time adjustment or intervention on the kitchen environment. For example, when the system detects a fire, the remote control unit can automatically cut off the power supply or gas supply in the kitchen to prevent the fire from spreading. The remote control unit can achieve these operations by controlling relays, electric valves or other intelligent devices.

[0153] In some embodiments, the functions of the remote control unit are not limited to emergency response, but can also be used for daily device control and maintenance. For example, users can adjust the ventilation system, lights, temperature, etc. in the kitchen through the remote control unit to improve the comfort and safety of the kitchen environment. The interface of the remote control unit can be connected to the user's mobile application, smart home platform or cloud platform to ensure that users can operate conveniently.

[0154] The reinforcement learning optimization module is used to adjust the fusion parameters in the data fusion and optimization module based on the historical records and real-time feedback of sensor data fusion, so as to optimize the data processing strategy.

[0155] Specifically, in the kitchen monitoring, detection and guardianship system of the present invention, the data feedback and strategy adjustment module is the core part for dynamically optimizing the system operation state. This module closely cooperates with the aforementioned cloud platform inference and analysis module and alarm and remote control module. According to the real-time monitoring data, user feedback and system state, it automatically adjusts the monitoring strategy and device control strategy to ensure the continuous and stable operation of the system and the maximization of efficiency. Through continuous optimization and adjustment, this module can improve the system's response ability, reduce false alarms, and enhance the safety of the kitchen environment.

[0156] In this embodiment, the data feedback unit is mainly responsible for collecting data from each module, including the output of the cloud platform inference and analysis module, the response data of the sound and light alarm unit, the feedback information of the network communication unit, etc. The task of the data feedback unit is to summarize, analyze and feedback this information into the system to support the decision-making process of strategy adjustment. Generally, the data feedback unit will process multi-modal sensor data, such as temperature, humidity, gas concentration, etc., and generate a comprehensive evaluation report in combination with the system operation state and alarm records.

[0157] Specifically, the data feedback unit establishes communication connections with each sensor and control unit to receive and transmit data in real time. As an option, the data feedback unit can regularly upload the statistical data of the current state to the cloud platform for subsequent analysis and model update. For the processing of sensor data, the data feedback unit may adopt certain data compression technologies to ensure the efficiency and accuracy during the transmission process. At the same time, the data feedback unit can also record and count alarm events for subsequent evaluation and improvement of the system performance.

[0158] The policy adjustment unit is responsible for optimizing and adjusting the policy according to the feedback information provided by the data feedback unit. Based on real-time monitoring data and alarm records, this unit makes timely adjustments to the inference model, alarm thresholds, device control policies, etc. in the system. Specifically, the policy adjustment unit first evaluates the current monitoring effect and risk level according to the feedback data, and then adjusts the alarm policy and device control. For example, in some embodiments, when the system detects a large number of false alarms, the policy adjustment unit can adjust the alarm threshold according to the new data analysis results to avoid frequent false alarms. In another possible implementation, if the environment in the kitchen changes significantly (such as a large fluctuation in gas concentration), the policy adjustment unit will timely adjust the alarm threshold or intervention strategy for gas leakage, thereby improving the response ability and accuracy of the system.

[0159] In some cases, the policy adjustment unit can also adjust the model parameters of the cloud platform inference and analysis module through real-time calculation. For example, when certain environmental factors (such as temperature, humidity, etc.) have a great impact on the monitoring results, the policy adjustment unit can optimize the inference results and reduce the occurrence of misjudgments by controlling the weight parameters in the cloud platform inference model. Specifically, the operation of the policy adjustment unit can adjust the parameters according to the following formula:

[0160]

[0161] Where: ΔW is the weight update amount; η is the learning rate; is the gradient of the loss function with respect to the weight W; L(W) represents the loss function of the model.

[0162] Under the control of the policy adjustment unit, the system can adjust the inference model according to new data to better adapt to the changes in the current kitchen environment.

[0163] The model training unit is responsible for training and optimizing the inference model according to the accumulated data during the operation of the system. This unit closely cooperates with the data feedback unit and the policy adjustment unit, and uses the feedback data to continuously update and train the prediction model of the system. Specifically, the model training unit can adopt deep learning algorithms, such as convolutional neural network (CNN), long short-term memory network (LSTM), etc., to perform online learning based on historical data, thereby continuously improving the intelligence level of the system.

[0164] In some embodiments, the model training unit adopts an incremental learning method to continuously absorb new data for training to avoid the obsolescence or failure of the model. For example, when dealing with sudden fire incidents in the kitchen, the model training unit can optimize through historical data to improve the system's fire prediction ability. The model training unit updates the model parameters through the following training process:

[0165]

[0166] where: θ( t ) is the current model parameter; α is the learning rate; θ( t+1 ) is the updated model parameter (the parameter at time step t+1); is the loss function; is the gradient of the loss function.

[0167] This process can be carried out in the way of batch update or incremental update, so as to ensure that the system can continuously learn and gradually improve its performance.

[0168] A kitchen monitoring, detection and guardianship method based on a cloud platform described below can be correspondingly referred to with a kitchen monitoring, detection and guardianship system based on a cloud platform described above.

[0169] Please refer to Appendix Figure 8 , a kitchen monitoring, detection and guardianship method based on a cloud platform, including the following steps:

[0170] S1. Collect multi-modal data in the kitchen environment;

[0171] S2. Remove noise, standardize the format and remove outliers from the collected data;

[0172] S3. Optimize the sensor data fusion process by using a weighted fusion method;

[0173] S4. Optimize the data fusion strategy by using a mutual information analysis method to improve data utilization;

[0174] S5. Perform inference and analysis on the fused data through a deep learning model and generate alarm information;

[0175] S6. Send the alarm information to the user device through an audible and visual alarm device and network communication;

[0176] S7. Dynamically adjust the data fusion parameters through a reinforcement learning optimization algorithm to improve the system's adaptability.

[0177] Specifically, for S1. Collect multi-modal data in the kitchen environment: In this step, the system collects various environmental data in the kitchen through a variety of sensors (such as temperature and humidity sensors, gas sensors, fire detectors, video monitoring devices, etc.). These data not only include traditional environmental monitoring data such as temperature, humidity, and gas concentration, but may also include image or thermal imaging data from video monitoring or infrared sensors for detecting abnormal situations such as fires and gas leaks.

[0178] S2. Remove noise, standardize the format, and eliminate outliers from the collected data: The raw data collected is usually interfered by noise, so preprocessing is required. In this step, first remove the noise, using filtering algorithms (such as low-pass filtering, Gaussian filtering, etc.) to remove the high-frequency noise in the data. Then, standardize the formats of data from different sources and types to ensure that they can be uniformly processed and fused. Finally, eliminate outliers through outlier elimination methods (such as statistics-based Z-score detection or IQR detection) to ensure the accuracy and reliability of the data.

[0179] S3. Optimize the sensor data fusion process using the weighted fusion method: In this step, optimize the data fusion of different sensors through the weighted fusion method. Since the measurement accuracies, response speeds, and data characteristics of different sensors may vary, directly fusing them may lead to information loss or deviation. The weighted fusion method assigns different weights to each sensor according to factors such as the credibility and accuracy of the sensor, thus ensuring that important data receives more attention.

[0180] S4. Optimize the data fusion strategy using the mutual information analysis method to improve data utilization: In this step, further optimize the data fusion strategy through the mutual information analysis method, aiming to improve data utilization. During this process, the system calculates the mutual information between different sensor data to evaluate the correlation and redundancy between each pair of data. For data with a high degree of redundancy, its weight in the fusion process can be appropriately reduced to avoid information duplication, thereby improving the data utilization efficiency.

[0181] S5. Perform inference analysis on the fused data through a deep learning model and generate alarm information: The preprocessed and optimized data is fed into a deep learning model for inference analysis. The deep learning model can be a convolutional neural network (CNN), a long short-term memory network (LSTM), etc. The model extracts features and recognizes patterns from the fused data to discover potential danger signals (such as fires, gas leaks, abnormal temperatures, etc.). Based on the prediction results of the data, the model generates corresponding alarm information and evaluates the severity of the event.

[0182] S6. Send the alarm information to the user device through an audible and visual alarm device and network communication: When the deep learning model generates alarm information, the system emits a loud sound or a flashing visual signal through the audible and visual alarm device to promptly alert kitchen staff or other personnel. At the same time, the system sends the alarm information to the user's device (such as a smartphone, tablet, PC, etc.) through the network communication unit, enabling the user to remotely obtain the alarm information and take corresponding measures.

[0183] S7. Dynamically adjust the data fusion parameters through the reinforcement learning optimization algorithm to improve the system's adaptability: In this step, the system uses the reinforcement learning algorithm to dynamically adjust the data fusion parameters according to the feedback information to improve the system's adaptability. Reinforcement learning learns from the system's feedback and continuously optimizes the weight and parameter settings in the data fusion process. Through this mechanism, the system can automatically adjust the data fusion strategy according to the changes in the kitchen environment, the fluctuations in sensor performance, and the user's feedback. For example, when the sensitivity of the gas sensor decreases, the system can adjust its weight in the data fusion process according to the feedback, thereby maintaining the stability and accuracy of the overall system performance.

[0184] The method of this embodiment can be used to implement the above system embodiment, and its principle and technical effect are similar, so it will not be elaborated here.

[0185] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principle and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A kitchen monitoring and detection system based on a cloud platform, characterized in that: include: A sensor acquisition module, used to collect multimodal sensor data in the kitchen environment, the data including temperature and humidity, smoke concentration, gas concentration, image data and audio data; A data preprocessing module, located on the edge computing device, is used to remove noise, smooth the data, and perform preliminary fusion on the data collected by the sensor; A data fusion and optimization module, connected to the data preprocessing module, is used to standardize the format of the data and remove outliers, and further optimize the sensor data based on a weighted fusion method and a mutual information analysis method; A cloud platform reasoning and analysis module, which is used to receive the data transmitted by the data fusion and optimization module, and perform reasoning and analysis on the data based on a deep learning model, identify dangerous factors in the kitchen environment, and generate an alarm signal; A data feedback and alarm module, which is used to receive the alarm signal from the cloud platform reasoning and analysis module and send the alarm information to the user device through network communication. The user device includes a smart phone, a tablet computer or a smart speaker; The reinforcement learning optimization module is used to adjust the fusion parameters in the data fusion and optimization module based on the historical records and real-time feedback of the sensor data fusion to optimize the data processing strategy.

2. According to the cloud platform-based kitchen monitoring and detection system of claim 1, it is characterized in that: The sensor acquisition module comprises: Temperature and humidity sensor, used to detect the temperature and humidity of the kitchen environment; Smoke sensor, used to detect changes in smoke concentration in the kitchen; Gas sensors, used to detect the concentration of combustible gases in the kitchen environment; High-definition camera, used to collect image information inside the kitchen; Microphone, used to collect audio information in the kitchen environment.

3. The kitchen monitoring, detection and supervision system based on a cloud platform according to claim 1 is characterized in that: The data preprocessing module comprises: A filtering processing unit, used for performing noise removal and signal smoothing on sensor data; Data standardization unit, used to unify the data formats of different sensors for subsequent fusion; The abnormal data detection unit is used to identify and remove abnormal or erroneous data.

4. The kitchen monitoring, detection and supervision system based on a cloud platform according to claim 1 is characterized in that: The cloud computing unit of the data fusion and optimization module includes: A weighted fusion unit, used to calculate a weighted value according to the confidence of sensor data and perform multi-source data fusion; The mutual information analysis unit is used to calculate the correlation between different sensor data and optimize the data fusion strategy.

5. The kitchen monitoring, detection and supervision system based on a cloud platform according to claim 1 is characterized in that: The deep learning models used by the cloud platform reasoning and analysis module include: Convolutional neural network (CNN), used to process image data collected by high-definition cameras; Long short-term memory network LSTM, used to process audio data collected by microphone; The target classification model is used to analyze the sensor data fusion results and output hazard category information.

6. The kitchen monitoring, detection and supervision system based on a cloud platform according to claim 1 is characterized in that: The data feedback and alarm module includes: An audible and visual alarm unit, used to trigger audible and visual alarm signals inside the kitchen; A network communication unit for sending alarm information to a remote user device via Wi-Fi or cellular network; Remote control unit, used to allow the user to perform emergency control operations via a mobile device, including closing a gas valve or activating a fire extinguishing device.

7. The kitchen monitoring, detection and supervision system based on a cloud platform according to claim 1 is characterized in that: The reinforcement learning optimization module includes: A data feedback unit is used to receive historical data fusion results and store them in a database; A strategy adjustment unit, used to update the parameters of the data fusion algorithm based on the reinforcement learning method; The model training unit is used to continuously train the data fusion algorithm in the cloud to optimize the data processing strategy.

8. The kitchen monitoring, detection and supervision system based on a cloud platform according to claim 4 is characterized in that: The weighted fusion unit optimizes the data fusion process by the following steps: Calculate the confidence of different sensor data and set the initial fusion weight based on the confidence; Adjust the weight distribution of each sensor data based on historical fusion results, so that high-reliability data accounts for a higher proportion in the fusion; Calculate the error of the fused data and adjust the parameters of the fusion algorithm based on the error to improve the fusion accuracy; Continuously monitor the reliability of fused data and dynamically adjust the fusion strategy.

9. The kitchen monitoring, detection and supervision system based on a cloud platform according to claim 4 is characterized in that: The step of optimizing the data fusion strategy by the mutual information analysis unit comprises: Calculate the independent information entropy of each sensor data to evaluate the validity of the data; Calculate the joint information entropy of sensor data to determine the degree of mutual correlation of different sensor data; Adjust the data fusion algorithm based on the mutual information value of sensor data to minimize data redundancy and improve information utilization; The mutual information calculation results are dynamically updated during the data fusion process to adapt to environmental changes.

10. A kitchen monitoring, detection and monitoring method based on a cloud platform, applied to a kitchen monitoring, detection and monitoring system based on a cloud platform according to any one of claims 1 to 9, characterized in that: The following steps are involved: Collect multimodal data in the kitchen environment; Remove noise, standardize the format and eliminate outliers from the collected data; The weighted fusion method is used to optimize the sensor data fusion process; Use mutual information analysis method to optimize data fusion strategy and improve data utilization; Perform reasoning and analysis on fused data through deep learning models and generate alarm information; Send alarm information to user equipment through sound and light alarm devices and network communication; The data fusion parameters are dynamically adjusted through reinforcement learning optimization algorithm to improve the system's adaptability.

Images