Smart cookware with monitoring system
By incorporating image quality detection and visual restoration mechanisms into smart cookers, combined with infrared temperature sensors, the problem of image quality degradation in cameras under high temperature and humidity conditions is solved. This enables accurate identification and closed-loop control of food status, improving the reliability of the cooking process.
Patent Information
- Application Number
- CN202611245629.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-17
- Publication Date
- 2026-11-17
AI Technical Summary
In high-temperature, high-humidity, and high-smoke environments, the image quality of existing smart cookware cameras is easily affected by oil film, water vapor, and smoke, resulting in inaccurate identification of food status and an inability to form a stable closed-loop control.
An image quality detection unit is used to detect the sharpness, color shift, and light attenuation parameters of the camera image. The image quality is restored through a visual enhancement algorithm, and the monitoring capability is restored when necessary by cleaning the transparent isolation component. Combined with an infrared temperature sensor to obtain the temperature data inside the pot, a closed-loop control is formed.
It improves the continuity of cooking status perception and the reliability of execution control, reduces misjudgments caused by image quality degradation, and ensures the accuracy of heating, stir-frying, and seasoning addition.
Smart Images

Figure CN123113453A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of intelligent cookware and electrical automatic control technology, and more specifically, to an intelligent cookware with a monitoring system that uses images and temperature information inside the pot for state perception and has a closed loop for visual degradation detection, compensation and recovery. Background Technology
[0002] Smart cookware typically controls heating, stir-frying, and seasoning addition through pre-set cooking programs to reduce manual operation. For the same dish, the initial temperature, moisture content, size, freshness, and actual amount of ingredients may vary between different batches. When actions are executed only according to fixed time points, deviations can easily occur between the actual cooking process and the preset program. Therefore, it is necessary to obtain feedback information that reflects the actual state during the cooking process.
[0003] Existing automated cooking equipment can be equipped with cameras to observe the state inside the pot. However, the camera's working position is near areas with high temperature, high humidity, and high oil fumes. The transparent observation area is prone to oil film or condensation, and fumes reduce image transmittance. If the control system directly sends images that are obscured or affected by color distortion to the recognition module, the recognized color, texture, and cooking information may not reflect the true state of the ingredients, thus further affecting the control of heat, stirring, and seasoning addition.
[0004] Chinese invention patent application CN108445793A discloses an intelligent cooking machine based on image monitoring and its control method. It uses an infrared imaging device to collect infrared thermal images of the ingredients in the pot, and an image data processing module obtains the distribution of ingredients and temperature based on this data, thereby controlling the stirring device and array heater. This scheme demonstrates that using images and temperature distribution to control the cooking process is existing technology. However, its main focus is on the distribution of ingredients and temperature reflected in the infrared thermal image, as well as the corresponding heating and stirring adjustments. From its disclosure, it is clear that when image quality deteriorates due to oil mist, water vapor, or contamination of transparent observation areas, how to first determine whether the image is still usable for identification, and how to perform graded compensation and recovery confirmation before identification, lacks a corresponding self-recovery closed loop for the monitoring channel.
[0005] Chinese invention patent CN108309021B discloses an intelligent control automatic cooking machine and its intelligent control method. The machine includes a camera, LED light source, infrared transmitter, infrared receiver, and gas sensor. A computer controls the motor, heating device, and fume extraction device based on image, infrared, and odor information to determine the external, internal, and aroma cooking states of the ingredients. This solution further illustrates that multi-source sensing information can be used to determine the cooking state, but its control premise remains that the information from each sensor can be effectively acquired. However, based on its disclosure, it does not establish a self-recovery link for the monitoring channel, sequentially connecting image quality detection, algorithm enhancement, physical cleaning, re-acquisition, and recovery judgment, to address the degradation of the visual channel itself in high-temperature, high-humidity, and high-fume environments.
[0006] Furthermore, simply adding cameras cannot create a stable closed-loop control. Monitoring data needs to establish defined data flows and control relationships with cooking temperature, cooking stage, and various execution components. When image quality deteriorates, it's crucial to first confirm the image's recognizability, and if necessary, restore the monitoring channel before continuing status assessment. Therefore, establishing a closed loop in the electrical control system of smart cookware—from monitoring data acquisition, image quality assessment, visual compensation, food status recognition to execution control—and ensuring this loop can self-recover under interference from oil mist, moisture, etc., is a key technical challenge that needs to be addressed.
[0007] Furthermore, the degradation of the cooking visual channel has obvious physical sources: oil film alters the light transmission and scattering characteristics of the transparent barrier, condensation causes local refraction and transmission attenuation, and smoke increases scattering and absorption in the imaging optical path. If image quality indicators are treated merely as isolated numerical quantities without establishing a calibration relationship with the optical state of the transparent barrier, different causes of contamination may produce similar changes in brightness or sharpness; if the cleaning device only moves according to a fixed stroke without matching the actual field of view boundary of the camera, there may be insufficient cleaning at the edge of the monitored area; simultaneously, the camera and control board are adjacent to the high-temperature zone inside the pot, and their heat load varies with the cooking stage and heating output. Therefore, it is necessary to simultaneously establish a quantitative correlation between image indicators and the optical state of contamination, a geometric correlation between the cleaning trajectory and the monitoring field of view, and a matching relationship between the water supply heat exchange capacity and the cooking heat load, so that the detection, recovery, and thermal reliability of the monitoring system can be quantitatively controlled. Summary of the Invention
[0008] The purpose of this invention is to provide an intelligent cooker with a monitoring system, which enables the image and temperature information output by the monitoring components to be directly used in the cooking stage judgment. When the image degrades due to oil mist, water vapor, or contamination of the observation area, the monitoring capability is restored through algorithm compensation and controlled cleaning, thereby improving the continuity of cooking status perception and execution control.
[0009] To achieve the above objectives, this invention provides an intelligent cooker with a monitoring system, including a cooking appliance, a stove assembly for heating the cooking appliance, a pot lid assembly, a seasoning assembly, a spatula assembly, a monitoring assembly, and a control module. The stove assembly, seasoning assembly, spatula assembly, and monitoring assembly are all electrically connected to the control module. The monitoring assembly includes an image acquisition unit and a pot temperature acquisition unit; the control module includes an image quality detection unit, a food ingredient status recognition unit, a cooking stage determination unit, a visual restoration control unit, and an execution control unit. The image quality detection unit generates an image quality status based on the clarity parameters, color shift parameters, and illumination attenuation parameters of the food ingredient image. When image degradation occurs, the visual restoration control unit first performs visual enhancement, and if the recognition conditions are still not met after enhancement, it controls the spatula assembly to clean the transparent isolation area of the monitoring assembly, and then re-acquires the image. The food ingredient status recognition unit forms food ingredient status information based on images that meet the recognition conditions. The cooking stage determination unit combines the food ingredient status information and pot temperature data to determine the current cooking stage, and the execution control unit adjusts the heating output, stir-fry parameters, and seasoning addition accordingly.
[0010] Preferably, the monitoring component includes a camera, an infrared temperature sensor, a control board, and a transparent insulating element. The camera and infrared temperature sensor are positioned at the monitoring location of the pot lid assembly and are electrically connected to the control board, which is electrically connected to the control module. The camera acquires images of the food through the transparent insulating element, and the infrared temperature sensor acquires the temperature data inside the pot in a non-contact manner.
[0011] Preferably, the image quality detection unit performs quality quantization on the region of interest image acquired by the camera through the transparent isolator: the composite gradient formed by the horizontal and vertical grayscale differences of adjacent pixels is averaged within the region of interest to obtain the average image gradient value G; the current image is converted to the Lab color space, and the color shift C is obtained based on the distance between the average values of channels a and b and the corresponding average values of the reference image in the clean state of the transparent isolator; the brightness attenuation value L is obtained based on the relative attenuation of the average brightness value of the current image relative to the average brightness value of the reference image. The first threshold, the second threshold, and the third threshold are determined by statistical results of multiple frames of reference images continuously acquired by the same monitoring component through the clean state transparent isolator. When any parameter reaches the corresponding degradation condition, the control module determines the image quality state as a degraded state.
[0012] More preferably, based on the calculation of image quality parameters, the brightness of the reference image of the clean state of the transparent isolation component is used as the incident / uncontaminated reference, and the Beer-Lambert type equivalent contamination optical thickness D is established according to the transmittance ratio of the current brightness to the reference brightness; by calibrating the equivalent attenuation coefficients caused by oil film, condensate and smoke, the sharpness parameter G, color shift parameter C, light attenuation parameter L and D together form a characterization of the degree of contamination.
[0013] More preferably, the movement trajectory of the cleaning component is geometrically matched according to the curvature of the transparent isolator and the actual field of view boundary of the camera, and the sweeping area formed by the center trajectory of the cleaning component and the effective cleaning width is used as the effective coverage area, and the coverage rate of the effective coverage area and the monitoring area is not less than 95%; the cleaning contact pressure is limited according to the elastic modulus, allowable stress, thickness of the transparent isolator and the contact area of the cleaning component.
[0014] More preferably, the food visual feature model adopts a directly deployable convolutional neural network or visual Transformer structure. During training, cooking scene enhancements such as fogging, brightness decay, and color shift are added to the samples of fresh, early heating, semi-cooked, cooked, and overcooked food. A weighted combination of classification loss and adjacent time-state consistency loss is used. The model output is fused with the pot temperature, action history, and target parameters of the dish stage through a stage scoring model.
[0015] More preferably, the water supply pipeline dynamically matches the heat exchange based on the predicted heat load of the current cooking stage; the real-time heat exchange capacity is calculated based on the water density, specific heat capacity, flow rate and the temperature difference between the inlet and outlet water, and the flow rate or duration is adjusted under the condition of meeting the constraints of the total amount of drinking water supplied for the current dishes and the timing of the supply.
[0016] Preferably, the control module switches between a normal visual recognition mode, an enhanced visual recognition mode, and a self-cleaning compensation mode. The enhanced visual recognition mode includes at least one of the following processing methods: defogging, brightness compensation, color correction, edge enhancement, and local contrast enhancement; the self-cleaning compensation mode is used to drive the cleaning component to clean the transparent isolation component, and uses the re-acquired image after cleaning as the basis for recovery judgment.
[0017] Preferably, the food ingredient state recognition unit includes a convolutional neural network model or a visual Transformer model, and the training samples cover at least the states of fresh, early heating, semi-cooked, mature and overcooked. The recognition results include at least one of the following: food ingredient category, color change, texture change, morphological change and degree of ripeness.
[0018] Preferably, the cooking stage determination unit matches the food status information with the pot temperature data, the stage target temperature conditions, and the actions already performed to determine the current stage in preheating, adding ingredients, heating, stir-frying, adding seasonings, reducing sauce, and cooking completion, and the execution control unit forms the next control action.
[0019] Preferably, the spatula assembly is equipped with a cleaning component driven by a drive device, the movement range of which covers the monitoring area of the transparent isolation component. The visual recovery judgment unit compares the image quality parameters before and after cleaning. If the recovery conditions are met, it returns to the normal visual recognition mode; if the recovery conditions are not met, it adjusts the number of cleaning cycles, cleaning speed, and / or cleaning movement range before making another judgment.
[0020] Preferably, the water supply line for supplying drinking water is coiled and / or attached to at least one of the control panel and camera within the pot lid assembly, so that the water supply process also serves to cool and heat the monitoring electronic components.
[0021] Compared to cookware that operates solely according to a fixed program, this invention uses image quality as a prerequisite for allowing food status recognition. Furthermore, it correlates G / C / L parameters with the optical state of contamination on the transparent isolator, linking visual enhancement, physical cleaning via field-of-view geometric matching, and post-cleaning re-detection into a closed-loop monitoring channel recovery mechanism. This reduces misjudgments caused by contaminated images continuing to participate in control. A cleaning coverage design of at least 95% of the monitoring area and contact pressure limitations based on material elasticity parameters improve the cleaning integrity and long-term reliability of the transparent isolator. By fusing visual status, pot temperature, action history, and stage target parameters, heating, stirring, and seasoning addition can be dynamically adjusted based on the actual cooking state. Dynamic matching of water supply heat exchange capacity with cooking heat load further reduces the heat load on the camera and control board in high-temperature environments. All these elements together form a quantifiable, recoverable, and verifiable monitoring-judgment-execution-feedback control chain. Attached Figure Description
[0022] Figure 1 This is an electrical function connection block diagram of an intelligent cooker with a monitoring system according to one embodiment of the present invention; Figure 2 This is a block diagram showing the signal processing relationship between the monitoring component and the control module in one embodiment of the present invention; Figure 3 This is a flowchart of cooking status monitoring and closed-loop control in one embodiment of the present invention. Detailed Implementation
[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Embodiments 1 to 10 described below proceed in the order of overall monitoring closed loop, monitoring hardware, image quality detection, visual enhancement, food status recognition, cooking stage judgment, execution control, monitoring area cleaning, visual restoration, and water supply heat exchange. The embodiments can be technically complementary. The described device models, interface types, and specific parameters can all be adapted to the overall machine specifications.
[0024] Example 1: Refer to Figures 1 to 3 As shown, this embodiment provides an intelligent cooker with a monitoring system, focusing on illustrating the overall closed-loop relationship between monitoring input, status judgment, and execution control.
[0025] Reference Figure 1As shown, the intelligent cooker with a monitoring system in this embodiment includes a cooking appliance, a cooktop assembly, a lid assembly, a seasoning assembly, a spatula assembly, a monitoring assembly, and a control module. The cooking appliance is used to hold the food, the cooktop assembly heats the cooking appliance, and the lid assembly provides a mounting location for the monitoring assembly and other execution components related to the cooking process. These mechanical components are only used to form the basic cooking space and execution interface. The control focus of this embodiment is on the electrical connection and data loop between the monitoring assembly, the control module, the cooktop assembly, the spatula assembly, and the seasoning assembly.
[0026] The control module can be configured with a main processing unit, an image processing unit, a storage unit, input interfaces, and drive output interfaces. The main processing unit performs cooking stage judgment, mode switching, and execution control; the image processing unit performs image preprocessing, image quality calculation, and data processing required for ingredient status recognition; the storage unit saves the stage control conditions corresponding to the dish, the visual feature model of the ingredients, image quality benchmarks, and cooking process records; the input interfaces receive status feedback from the camera, infrared temperature sensor, and various execution components; and the drive output interfaces output control signals to the stove assembly, spatula assembly, and seasoning assembly, respectively. Each function can be performed by a processor on a single control board, or distributed across the interconnecting main control board and image control board.
[0027] The monitoring component forms at least two detection channels directly involved in cooking control. The first channel is an image detection channel, where a camera continuously or according to a preset sampling period acquires images of the food inside the cooking appliance. The second channel is a temperature detection channel, where an infrared temperature sensor performs non-contact temperature measurement of the area inside the pot and outputs the pot's temperature data. Both data streams are transmitted to the control module via a control board, enabling the control module to simultaneously obtain changes in the food's appearance and temperature during the same cooking stage. The purpose of this setup is not simply to increase the number of sensors, but to allow image data, which only reflects appearance, and temperature data, which reflects thermal state, to converge at the cooking stage judgment point.
[0028] like Figure 2 As shown, the raw image captured by the camera first enters the image preprocessing chain. Preprocessing may include bad pixel removal, size normalization, region of interest (ROI) cropping, and pixel normalization adapted to the specifications of subsequent model inputs. The ROI preferably covers the area in the cooking appliance that actually contains the food, excluding background areas such as the edge of the lid and fasteners that do not change with the state of the food over a long period of time. The preprocessed image is sent to the image quality detection unit on one hand, and to the food state recognition unit on the other hand, after being determined to meet the recognition conditions. Thus, the image quality detection unit is located before the food state recognition, rather than passively judging it after the recognition result has been formed.
[0029] The electrical connection between the control module and each component can be via a wired signal interface or a bus communication system within the entire machine that meets real-time control requirements. Camera image data and temperature data can be aggregated on the same control board and sent to the main processing unit, or they can be input separately to the main processing unit. Regardless of the specific interface used, the image acquisition unit and the pot temperature acquisition unit should be used as input terminals, image quality detection, food status recognition, and stage judgment as intermediate processing terminals, and the stove assembly, spatula assembly, and seasoning assembly as execution terminals, maintaining the correct signal direction.
[0030] This invention retains the basic structure of the cooking appliance, stove assembly, pot lid assembly, seasoning assembly, and spatula assembly to provide monitoring and execution targets, without relying on specific complex mechanical transmission methods. The cooking appliance and stove assembly can be connected separately; for example, the cooking appliance can be removed from the stove assembly. Alternatively, they can be assembled into a single unit via transmission components to facilitate automatic food dispensing, automatic pot cleaning, and automatic resetting. The pot lid assembly only needs to provide a suitable coverage area for installing the monitoring components during cooking. For the electrical control scheme of this invention, the key lies in the monitoring components generating image and temperature inputs, the control module performing quality and stage judgments, and controlling each execution component via electrical connections.
[0031] Through the above implementation, this invention sets up image quality detection at the front end of the monitoring channel, enhancement and self-cleaning recovery in the middle, and food status recognition and temperature joint judgment at the back end, using the stove assembly, spatula assembly, and seasoning assembly as physical execution ends. Therefore, when the monitoring assembly is contaminated by oil mist, water vapor, or transparent insulating materials, it will not directly transmit low-quality images continuously to the cooking control, but will first restore the signal quality; the restored food status, together with temperature information, then determines the action to be executed, forming a complete data acquisition, status confirmation, control output, and re-acquisition process.
[0032] The above embodiments illustrate the main electrical control relationships of the present invention. For those skilled in the art, provided that image quality judgment precedes food condition recognition, that re-acquisition verification follows visual recovery, and that food condition and pot temperature jointly participate in the cooking stage control, adaptive selections can be made regarding the processor model, camera type, sensor interface, driver type, and specific parameters. These selections do not alter the basic working principle of the technical solution described in this invention.
[0033] This embodiment uses food images and pot temperature as common state inputs. Image quality assessment and visual restoration are performed before food state recognition. The restored monitoring information is further used for cooking stage judgments and control of heating, stirring, and seasoning addition, forming a continuous closed loop across detection, judgment, and execution stages. This solves the problems of fixed programs failing to adapt to actual changes in food state and low-quality monitoring signals potentially causing subsequent control inaccuracies, thus improving the reliability of automatic cooking state perception and execution control.
[0034] Example 2: Based on Example 1, this example further illustrates the camera, infrared temperature sensor, control board, and transparent isolation component of the monitoring component and their electrical connections.
[0035] Infrared temperature sensors can be placed near the camera or integrated into the same monitoring installation area, with their measurement field of view facing the cooking appliance. The temperature value output by the infrared temperature sensor is converted by the control board into pot temperature data that the control module can process. To reduce instantaneous anomalies caused by opening the lid, adding food, or vigorous stirring, the control module can perform validity checks on several consecutive sampled values. Samples that significantly exceed the device's allowable temperature range or are interrupted in data communication are marked as invalid. Under normal circumstances, the current valid temperature value and its trend over time are used in the stage judgment. The above validity processing is a signal quality processing step and does not change the basic measurement principle of infrared temperature measurement.
[0036] A basic installation method for the monitoring component includes a camera, an infrared temperature sensor, a control board, a mounting housing, and a transparent isolator. The transparent isolator is located on the side of the pot lid assembly facing the cooking appliance and can be made of a high-temperature resistant transparent material. This allows the camera to observe the inside of the pot through this area while simultaneously isolating the camera and control board from oil fumes and liquid droplets. The camera and infrared temperature sensor are respectively connected to the control board, which in turn is connected to the control module. The mounting housing is only used to maintain the relative position of the monitoring component and the transparent isolator; its specific fixing structure can be determined based on the spatial arrangement of the pot lid assembly and is not considered part of the control core of this embodiment.
[0037] The pot temperature data described in this embodiment can represent the non-contact temperature of the food surface area, the area near the bottom of the pot, or a pre-selected measurement area. The specific measurement area is determined by the installation direction of the infrared temperature sensor. The cooking stage uses temperature data consistent with the device's calibration method; therefore, it is not required to interpret the infrared temperature value as a single, unique physical location temperature. As long as it can stably reflect the thermal state changes during cooking and is used in conjunction with the target conditions for the corresponding dish, it can be used as a closed-loop control input.
[0038] When the monitoring component is arranged separately from the main control board, the control board can perform camera driving, infrared temperature acquisition, and basic image preprocessing, and send the food image, pot temperature data, and monitoring status to the main control board; the main control board then performs stage judgments and executes control. In a single-board structure, the above functions are performed by the same processor or different processing resources on the same board. Both implementations maintain the logic that monitoring input is first quality-verified before participating in control, therefore they are not limited to a particular hardware topology.
[0039] This embodiment configures image acquisition and non-contact temperature acquisition channels within the same monitoring area, and uses a transparent isolator to isolate oil fumes and droplets, enabling the control module to simultaneously obtain information on the appearance and thermal state of the food. This solution solves the problems that single visual information cannot directly reflect the thermal state inside the pot, and single temperature information is insufficient to reflect changes in the cooking appearance of the food, providing a complementary monitoring basis for subsequent joint stage judgments.
[0040] Example 3: Based on Example 2, this example quantifies the sharpness, color shift and light attenuation of food images and forms an image quality status accordingly.
[0041] The image quality detection unit generates sharpness parameters, color shift parameters, and illumination attenuation parameters for the preprocessed food image. The following presents a directly implementable calculation method, but it is not mandatory to use this mathematical expression. Let... For region of interest, for The number of effective pixels included in the calculation. For position The gray value at that point is the average gradient value of the image. It can be obtained by averaging the composite gradient formed by the horizontal and vertical gray-level differences of adjacent pixels within the region of interest; the clearer the image edges and textures, the better. Generally, the larger it is.
[0042] ; Color offset It can be calculated in the Lab color space. Let... , These are the mean values of channels a and b of the region of interest in the current image, respectively. , Given the mean values of the reference image when the transparent isolator is in a clean state, the following formula can be used to measure the overall color cast of the current image relative to the local clean reference: ; The illumination attenuation parameter can be the brightness attenuation value. Indicate. Let. The average brightness of the region of interest in the current image. To determine the average brightness of a clean baseline image under the same acquisition configuration, the relative brightness decay can be calculated using the following formula; when the current brightness is not lower than the baseline brightness, Set it to 0 to avoid misinterpreting normal brightness increases as brightness decreases: ; Where G is the sharpness parameter, C is the color shift parameter, and L is the illumination attenuation parameter; and These represent the differences between adjacent pixels in the horizontal and vertical directions of the grayscale image, respectively. All three parameters are referenced to a cleanliness baseline formed by the same camera, the same transparent isolator, and the same installation location. Therefore, the camera's fixed response, the transparent isolator's own light transmission characteristics, and the differences in fixed lighting are included in the local baseline. The detection results mainly reflect the relative changes caused by oil film, water vapor, and smoke during cooking, rather than relying on absolute image values common to all devices.
[0043] To further establish a physical correspondence between the aforementioned image quality parameters and the actual contamination state of the transparent isolator, this embodiment can also establish an equivalent contamination optical thickness model. Let the average brightness of the current image's region of interest be... The average brightness of the clean baseline image is Define the relative transmittance of the monitoring optical path. To avoid zero values in numerical calculations, a very small positive number can be set. As a lower limit, based on the Beer-Lambert attenuation relationship, the equivalent contamination optical thickness is... Defined as the negative logarithm of relative transmittance.
[0044] ; When calibrating the components of pollution, D can be further expressed as the linear attenuation superposition of the equivalent optical path lengths of oil film, condensate, and smoke, where... , These represent the equivalent thicknesses of the oil film and condensate, respectively. This represents the product of smoke concentration and equivalent optical path length. , , These are the equivalent attenuation coefficients calibrated under the spectral response and illumination conditions of the camera used. This model does not require direct measurement of the actual film thickness during runtime; instead, it utilizes equipment calibration samples to establish a traceable mapping between image attenuation and contamination levels.
[0045] ; Sharpness parameters can also be obtained by averaging the gradient under clean conditions. Normalization was performed, and the scattering attenuation coefficient associated with D was used. This describes the attenuation of edge information caused by contamination; color offset C is used to supplement and characterize the overall color shift caused by inconsistent attenuation across different bands / color channels. At runtime, it can... The combined input of C and D into the image quality judgment logic makes it easier to distinguish between normal exposure fluctuations caused by brightness changes alone and composite degradation caused by pollution.
[0046] ; To obtain the aforementioned equivalent attenuation coefficients, clean, transparent isolators and calibration samples with different equivalent contamination levels can be used during the equipment calibration phase. Camera exposure, lighting, and installation position should be kept consistent, and G, C, L, and D should be recorded respectively. Least squares fitting, table lookup interpolation, or piecewise linear fitting can be used to obtain the correspondence between each parameter and the contamination level. Therefore, the first, second, and third thresholds can be determined not only from the statistical distribution of clean states but also by combining the graded contamination calibration results of the transition from identifiable to unidentifiable states.
[0047] The first, second, and third thresholds can be determined during device factory calibration or initialization based on multiple consecutively acquired reference images under clean, transparent isolating conditions. Specifically, the average values of G, C, and L corresponding to the multiple reference images can be calculated respectively. , , and standard deviation , , A positive safety factor should be selected based on the permissible false trigger probability of the equipment and the monitoring stability requirements. , , , forming the first threshold Second threshold and the third threshold .
[0048] ; During runtime, when , or An image quality state can be defined as degraded if any one of the conditions is met; only when... , and Only then will the image meet the basic quality requirements for direct entry into food status recognition. The safety factor can be determined during equipment calibration based on camera noise, cooking lighting fluctuations, and desired mode switching sensitivity; it is not required that different devices or different dishes use the same value.
[0049] ; Initial calibration can be performed after the monitoring components are reinstalled or the transparent isolator is replaced. During calibration, a reference image is acquired under conditions where the transparent isolator is clean and the camera's field of view is unobstructed, establishing a reference for normal sharpness, color, and brightness; simultaneously, the infrared temperature sensor is checked for effective output. Automatic cooking mode is only permitted after calibration is complete. By placing the reference acquisition in a clean state, subsequent image quality parameters can be judged relative to the local monitoring channel, reducing fixed differences caused by different cameras and light-transmitting components.
[0050] When visual degradation occurs multiple times during the same cooking task, the control module can update the reference state for the current task using the image quality parameters from the most recently successfully recovered image. However, the updated reference state must not fall below the basic quality requirements for performing food recognition. This adapts to slow changes in lighting and overall brightness during cooking while retaining the quality thresholds comprised of the first, second, and third thresholds, as well as the recovery conditions, preventing the gradual misinterpretation of ongoing contamination as a new normal state.
[0051] This embodiment constructs a multi-dimensional image quality criterion using average gradient value, color shift, and brightness attenuation value, and determines the threshold based on a baseline image of the device in a clean state. This allows it to distinguish normal images from degraded images caused by oil film, moisture, smoke, etc. This solves the problem that the presence or absence of a camera alone cannot determine whether the current image is suitable for maturity recognition, reducing the probability of distorted images entering subsequent control links.
[0052] Example 4: Based on Example 3, this example further sets up a normal visual recognition mode, an enhanced visual recognition mode, and a self-cleaning compensation mode, and switches between different modes according to the image quality status.
[0053] In normal visual recognition mode, the image quality detection unit confirms that the food image meets the recognition conditions, and the control module directly inputs the pre-processed image into the food status recognition unit. Normal mode does not initiate additional cleaning actions to avoid frequently changing the current cooking action of the spatula assembly for visual maintenance. The control module continues or periodically calculates G, C, and L, updating the image quality status according to changes in oil fumes, moisture, and cooking stages.
[0054] When image degradation is detected but has not yet reached the point where physical cleaning is necessary, the algorithm switching unit enters the enhanced visual recognition mode. The visual enhancement unit selects the corresponding processing chain according to the image quality parameters that exceed the threshold: when L exceeds the third threshold and C is within the allowable range, brightness compensation and local contrast enhancement can be performed first; when C exceeds the second threshold, color correction can be performed in the Lab color space based on the a and b channel offsets of the cleaning state reference; when G is below the first threshold and accompanied by brightness decay, dehazing processing can be performed in combination with edge enhancement; when multiple parameters are abnormal at the same time, they can be processed in the order of dehazing, brightness / contrast compensation, color correction, and edge enhancement. The dehazing, local contrast enhancement, color correction, and edge enhancement can all be implemented using mature digital image processing algorithms, and the algorithm itself is not the focus of this invention. After enhancement, G, C, and L are recalculated, and only enhanced food images that meet the recognition conditions again are sent to the food state recognition unit.
[0055] For dehazing, an image restoration method based on an atmospheric scattering model can be used. Let... For degraded images, To reconstruct the image, A is the estimated global ambient light. Transmittance, To prevent excessive magnification of the lower limit of transmittance, pixels can be restored using the following formula; It can be obtained from dark channels, local contrast, or a lightweight transmittance estimation network trained on it.
[0056] ; For brightness compensation, the gamma coefficient γ can be selected for nonlinear correction based on the brightness attenuation value L; for color correction, the mean value of the Lab channel of the clean reference image can be directly used as the target, and mean shift compensation can be performed on the a and b channels of the current image; local contrast enhancement can be achieved using contrast-limited adaptive histogram equalization, and edge enhancement can be achieved using Laplacian or unsharpened masks. After each algorithm is executed, G, C, L, and D are recalculated, and enhancement only ends when the restored index enters the recognition range.
[0057] ; There is a clear switching relationship between the enhanced visual recognition mode and the self-cleaning compensation mode. If the enhanced image still does not meet the recognition conditions, or if the image quality parameters reach the preset self-cleaning trigger conditions, the control module enters the self-cleaning compensation mode. The core of this mode is to temporarily switch the spatula component from the regular stir-frying task to the monitoring area cleaning task, and then return to the cooking control process after completing the cleaning and restoration judgment. Therefore, the self-cleaning action is not an independent timed cleaning function, but is triggered by the real-time quality status of the monitoring channel.
[0058] The control module can also set recovery and retention conditions for image quality mode switching. For example, after an image recovers from a degraded state, it can revert to the normal visual recognition mode only after several consecutive detections meet the recognition conditions; when the parameters fluctuate only slightly around the threshold, the current enhancement mode can be maintained without frequent switching. Such retention conditions can be implemented by the control software, and their function is to improve the stability of mode switching without changing the basic relationship between the first, second, and third thresholds used to determine image quality.
[0059] When G, C, and L exhibit different combinations, the control module can select the recovery order accordingly. If G significantly decreases while L significantly increases, the current image is considered to have strong fogging or light transmission attenuation characteristics, and defogging and brightness compensation are prioritized. If C is abnormal while G remains within acceptable limits, color correction can be prioritized. If multiple parameters consistently exceed degradation conditions, the process of switching from enhancement mode to self-cleaning compensation mode can be shortened. The above processing utilizes only the already calculated image quality parameters and does not require additional mechanical detection devices.
[0060] This embodiment selects dehazing, brightness compensation, color correction, edge enhancement, or local contrast enhancement based on different degradation parameters, and sets recovery and retention conditions to form a processing sequence that gradually escalates from mild algorithmic compensation to necessary physical cleaning. This solution can solve the problem of frequent mechanical cleaning triggered by slight image fluctuations or the inability to handle persistent contamination with only digital enhancement, reducing unnecessary action switching while ensuring monitoring reliability.
[0061] Example 5: Based on Example 4, this example uses a pre-trained food visual feature model to perform state recognition on food images that meet the recognition conditions.
[0062] When the food ingredient state recognition unit uses a pre-trained food ingredient visual feature model, a convolutional neural network (CNN) or a visual Transformer can be used. As an executable structure of a CNN, the input image is uniformly 224×224 pixels, with four stages of feature extraction. Each stage uses 3×3 convolution, batch normalization, and SiLU activation, and downsampling via stride convolution or pooling. The number of output channels can be set to 64, 128, 256, and 512 respectively. The final stage uses global average pooling and a fully connected classification head to output the probabilities of each ripening state. As an executable structure of a visual Transformer, the 224×224 image can be divided into 16×16 image blocks, with an embedding dimension of 384. These blocks are passed through six Transformer encoding blocks, each with six attention heads and two layers of feedforward networks. The final stage uses classification labels to classify the corresponding features for state classification. The above parameters provide a directly reproducible implementation method, which can also be scaled down proportionally according to the computing power of the control board.
[0063] The training samples include at least four states: fresh, initial heating, semi-cooked, fully cooked, and overcooked, and are further subdivided according to food category. To adapt the model to the high temperature, high humidity, and high oil fume environment inside the pot, data augmentation can be applied to the original samples during the training phase, superimposed with data consistent with the actual degradation direction. This includes Gaussian / scattering fogging, brightness attenuation, local contrast reduction, Lab color shift, random droplet occlusion, and mild motion blur, so that the model remains stable to mild visual fluctuations within the limits allowed by image quality gating.
[0064] The model training objective can consist of classification loss and temporal consistency loss. Let y be the true state label. This is the state probability vector output by the model at the current moment. This represents the state probability vector of the adjacent previous valid time step. Cross-entropy classification loss, If the temporal consistency loss is caused by the probability difference between adjacent states, then the total loss is... It can be constructed in a weighted manner; in actual training, Used to control the strength of timing consistency terms.
[0065] ; As an example of training hyperparameters, the AdamW optimizer can be used, with an initial learning rate of... Weight decay The batch size is 32, and the training runs for 80 epochs, using cosine annealing to gradually reduce the learning rate. Training can be stopped early if there is no improvement on the validation set for several consecutive epochs. All of the above hyperparameters can be adjusted according to the data size and computing power. When deploying the model, the fixed parameters after training are used, and inference is only performed on images that have passed the image quality gating.
[0066] For different ingredients, the visual feature model can employ a multi-class model or call the corresponding sub-model according to the dish category. When the control module can obtain the dish to be cooked from the cooking task or user settings, it can first determine the corresponding model and then recognize the current image. This reduces irrelevant classification ranges caused by differences in the appearance of different ingredients. Model usage does not require retraining during the operation of the smart cooker; the trained model parameters can be stored in the control module's storage unit and retrieved when recognition is needed.
[0067] When the food ingredient status recognition unit uses continuous images, it can perform a temporal consistency check between the current recognition result and the previous recognition result. If the recognition state of a single frame is significantly inconsistent with the consecutive states before and after, the control module can request to re-acquire images and confirm the stage change after passing image quality inspection, rather than immediately changing the heat or adding ingredients based solely on the single frame result. This check is used to reduce the impact of momentary images such as food stirring or partial obstruction on the judgment of the cooking stage. The final stage is still determined by the food ingredient status, the temperature inside the pot, and the actions already performed.
[0068] This embodiment trains the food visual feature model by annotating images at different cooking stages, enabling it to output state information such as food category, color, texture, shape, and degree of cooking. It also utilizes consistency checks on consecutive images to suppress the effects of momentary occlusion. This solves the problem that fixed-time programs cannot reflect the actual cooking progress of different batches of food, providing actionable data for adjusting cooking parameters based on the actual state of the food.
[0069] Example 6: Based on Example 1, this example further illustrates how the cooking stage judgment unit matches the ingredient status information with temperature data, stage target conditions, and executed actions.
[0070] The results of food ingredient status recognition are not directly equivalent to control commands. For example... Figure 3 As shown, after the ingredient status information enters the cooking stage judgment unit, it is also matched with the pot temperature data obtained by the infrared temperature sensor, the target temperature conditions of the corresponding dish, and the actions already performed. For example, in the preheating stage, even if the camera can observe the pot area, the main basis for stage transition should be whether the pot temperature has reached the conditions for adding ingredients; after the ingredients are added, the color, texture, shape, degree of cooking, and temperature can be considered to determine whether it is still in the heating or stir-frying stage; when the visual status shows that the ingredients are close to the target degree of cooking and the temperature conditions are met, the seasoning or sauce reduction stage can be entered.
[0071] To integrate visual perception, temperature, motion history, and dish objectives into a unified set of stage-based judgment metrics, this embodiment can establish a stage score for each candidate stage k. .set up Output the visual matching probability of the current image belonging to stage k for the food visual feature model. The temperature matching degree between the current pot temperature and the target temperature range of stage k. To ensure historical consistency between executed control actions and the expected action sequence in stage k, The degree of matching between the current dish recipe / stage target parameters and the target of stage k. , , , If the weights are non-negative and the sum is 1, then the following fusion score can be formed.
[0072] ; The stage determination unit only compares the results from the set of candidate stages that are allowed to be reached in the current stage. The stage with the highest score is selected as the current cooking stage; for example, a transition from the heating stage can only proceed to the stir-frying stage or the heating-maintaining stage, and cannot directly jump to the cooking completion stage. For temperature matching, a piecewise function between 0 and 1 can be constructed based on the distance between the current temperature and the target range; the action history matching is determined based on whether the addition of ingredients, stir-frying, heating changes, and seasoning addition have occurred. This ensures that multi-source fusion is not merely a parallel arrangement of information, but rather the formation of executable numerical scores and constrained state transitions.
[0073] The cooking stage judgment unit can maintain states such as preheating stage, ingredient addition stage, heating stage, stir-frying stage, seasoning addition stage, sauce reduction stage, and cooking completion stage. Each state includes an entry condition, a hold condition, and an exit condition. The entry condition is used to determine whether sufficient feedback has been received in the previous stage, the hold condition is used to continuously adjust the execution parameters within the current stage, and the exit condition is used to determine whether to allow entry into the next stage. Through this state-based judgment, each execution component no longer operates independently based solely on a fixed timer, but instead accepts common stage variables updated by monitoring data.
[0074] Thresholds, target temperature ranges, stir-frying parameters, and seasoning addition parameters can be set up for each dish and stored in the storage unit. Image quality thresholds are preferably determined based on baseline data from the monitoring component itself in a clean state, while cooking target parameters are determined based on the dish's cooking procedure. During device operation, the device calls the parameter set corresponding to the current dish and writes the actual image, pot temperature, and stage status into the process record to maintain consistency in control before and after the cooking process. The preset here does not mean that all dishes use the same threshold or temperature, but rather that the corresponding parameters have been configured by the device before entering specific control judgment.
[0075] Image acquisition and temperature acquisition can be time-correlated within the same control cycle. When the control module receives a frame of food image, it simultaneously records the effective pot temperature corresponding to the image acquisition period, ensuring that the food state recognition result and temperature judgment target the same cooking stage. If the camera frame rate is higher than the temperature sampling frequency, the most recent effective temperature value can be correlated to the corresponding image cycle; if the temperature sampling frequency is even higher, the effective temperature values within the image acquisition period can be smoothed and used as the temperature input for that frame or group of images. This time correlation is used to avoid incorrectly combining the temperature of the previous stage with the image of the next stage.
[0076] The control module can also adjust the monitoring frequency according to the current cooking stage. During the preheating or stable heating stages where the state changes slowly, a standard sampling cycle that meets the control requirements can be used. At stages where ingredients are just added, nearing completion, or when visual recovery is just finished, the image and temperature update frequency can be increased to obtain new state information more quickly. Adjusting the sampling frequency only affects the data update time and does not change the order of image quality detection, ingredient state recognition, and stage judgment.
[0077] This embodiment sets entry, hold, and exit conditions for states such as preheating, adding ingredients, heating, stir-frying, adding seasonings, reducing sauce, and completion, and correlates images with temperature over time, enabling effective feedback on state transitions based on the same cooking stage. This solves the problem of lacking common stage variables due to each execution component operating on independent timers, and improves the consistency of cooking stage judgments under varying conditions of different batches of ingredients.
[0078] Example 7: Based on Example 6, referring to... Figure 3 As shown in the illustration, this embodiment further illustrates that the execution control unit adjusts the stove assembly, spatula assembly, and seasoning assembly according to the current cooking stage.
[0079] The control unit generates heating control signals for the cooktop components. The control module reads the current temperature inside the pot and compares it with the target temperature range corresponding to the current cooking stage. When the temperature is below the target range, the heating output is increased; when the temperature is within the target range, the heating output is maintained or slightly adjusted; when the temperature is above the target range, heating is reduced or stopped. For cooktop components that can be controlled in stages, the heating output corresponds to the stage; for cooktop components that can be continuously adjusted, the heating output corresponds to the power setpoint. The specific power adjustment method is determined by the drive interface of the cooktop component itself.
[0080] The control unit generates a stirring control signal for the spatula assembly. This control signal can include at least stirring speed and duration, and may also include steering or intermittent time depending on the drive unit's capabilities. Appropriate stirring parameters are used when ingredients are first added and need to be quickly dispersed; the stirring speed is adjusted based on the actual cooking state as the ingredients progress to later stages. Since the ingredient state recognition result comes from the current image, which has already undergone quality checks before recognition, the visual feedback used for stirring control has a clear validity threshold.
[0081] The control unit generates a dispensing control signal for the seasoning component. The control module determines the type of solid seasoning, liquid seasoning, or drinking water to be dispensed, and the appropriate dispensing timing, based on the current cooking stage and the preset recipe. The seasoning component can be dispensed using electrically driven actuators such as motors, pumps, or valves. The dispensing command is not triggered if the ingredients have not yet reached the corresponding stage; once the seasoning dispensing stage is reached, the corresponding action is executed, and then images and temperatures are re-acquired to determine the next stage, thus establishing a link between the seasoning action and the actual cooking state.
[0082] In one embodiment of a leafy vegetable dish, after the equipment is started, it first enters a preheating stage, where an infrared temperature sensor continuously monitors the temperature inside the pot. Once the temperature reaches the ingredient addition conditions corresponding to the dish's program, the ingredients are added and stir-frying begins. A camera captures leaf images at preset intervals. The image quality detection unit first confirms that the current image is not obscured by significant oil mist or moisture, and then the ingredient status recognition unit identifies changes in leaf color, texture, and shape. As the recognition results gradually transition from the initial heating state to near-cooked state, the cooking stage judgment unit, combined with the current temperature, confirms the entry into the next stage, and the control unit adjusts the stove output and stir-frying speed accordingly, triggering seasoning addition at a designated stage. The entire process does not rely on a fixed number of minutes as the sole criterion for ripeness.
[0083] Similarly, in normal cooking control, after the execution control unit issues instructions to reduce heating, change stirring, or add seasonings, the control module continues to acquire new images of the pot's temperature and the ingredients. The cooking stage judgment unit then confirms whether the expected change in state has occurred. If the state has not yet met the conditions for the next stage, adjustments continue within the current stage; only after the conditions are met does the stage switch. Therefore, Figure 3 The closed loop shown continues to run over time until the cooking completion conditions are confirmed.
[0084] This embodiment uses the temperature deviation inside the pot to adjust the heating output, uses the state of the ingredients to adjust the stir-frying parameters, and triggers the addition of seasonings or drinking water only at the corresponding stages. Furthermore, it re-acquires images and temperatures after each execution for re-evaluation. This solves the problem of independent control over heat, stir-frying, and ingredient addition, preventing the control actions from being continuously adjusted based on actual cooking results, thus forming a continuously updated closed-loop cooking execution system.
[0085] Example 8: Based on Example 2, this example further utilizes the driving device of the spatula assembly to drive the cleaning component to perform controlled wiping on the monitoring area of the transparent isolation component.
[0086] The spatula assembly may include a drive unit electrically connected to the control module and a spatula or connecting rod driven by the drive unit. When self-cleaning of the monitored area is required, at least one connecting rod moving with the drive unit is provided with a cleaning component facing the lid assembly. Preferably, the connecting rod closer to the lid assembly is selected as the cleaning component mounting rod. The cleaning component preferably uses a wear-resistant and high-temperature-resistant silicone brush head. The transparent isolator can be a plane, a cylinder, or an arc surface with a predetermined curvature. The control module or assembly calibration program determines the center trajectory of the cleaning component, the effective cleaning width, the start and end positions of the movement, and the required number of repeated sweeps based on the geometric parameters of the transparent isolator and the actual field of view boundary formed by the camera through the transparent isolator, so that the cleaning trajectory is geometrically matched with the monitoring field of view.
[0087] Let the projected area of the actual monitored area of the camera on the surface of the transparent insulating component be... The effective cleaning width when the cleaning part moves along the central trajectory s is After considering the overlap between adjacent trajectories, a correction factor is applied. If the area overlap is due to repeated sweeping, the effective cleaning area can be estimated using the trajectory arc length integral. and define cleaning coverage. ;in Designing The coverage rate should be no less than 95%. For edge areas, the coverage rate can be improved by adding local supplementary scanning trajectories.
[0088] ; The contact pressure between the cleaning component and the transparent isolator can also be designed in conjunction with the mechanical parameters of the transparent isolator material. Let the normal force applied by the drive device be... The effective contact area of the cleaning component is Then the average contact pressure According to the allowable stress of the transparent spacer and safety factor , can be limited For transparent spacers that can be approximated as supported thin plates, the deflection caused by cleaning loads can be estimated using their elastic modulus E, thickness h, characteristic span a, and support coefficient K. and demand Not exceeding the allowable deflection .
[0089] ; Through the aforementioned geometric coverage and contact force limitations, the cleaning action simultaneously meets the requirements of monitoring area coverage integrity and transparent isolator protection. For planar transparent isolators, a reciprocating grid trajectory can be used; for cylindrical or curved transparent isolators, an arc-shaped reciprocating trajectory along the direction of equal curvature can be used. The position command of the drive device is generated from the calibrated sequence of trajectory points.
[0090] After the self-cleaning compensation mode is activated, the control unit first determines whether the spatula assembly is ready to perform the cleaning action. When the spatula assembly is performing a continuous feeding action, cleaning can be performed immediately after that action is completed; during the normal stir-frying stage, the drive device can be directly switched to the preset cleaning motion. The drive device moves the cleaning component along the transparent partition to remove oil film or condensed water that affects the light transmission of the observation area. After cleaning, the camera re-captures images of the same area inside the pot and calculates new G, C, and L values.
[0091] The cleaning triggering of the transparent isolator can also be coordinated with cooking actions. For example, if slight image degradation occurs during a period of heavy oil fumes, and if enhancement processing has already restored recognition, physical cleaning can be delayed, reducing the switching of the spatula component during high-frequency stir-frying. Cleaning is then prioritized when image quality further deteriorates and affects recognition. By grading recovery measures through an algorithmic switching unit, a defined control order can be established between monitoring reliability and the continuity of cooking actions.
[0092] This embodiment further utilizes the controlled drive component originally used for stir-frying for cleaning transparent isolation areas, switching the cleaning action only when the image quality reaches the self-cleaning trigger condition, thus matching the cleaning trajectory with the camera's monitored area. This solves the problem that digital image processing alone cannot restore the true light transmission conditions after oil film or condensation adheres, while avoiding the need for an additional independent cleaning drive mechanism.
[0093] Example 9: Based on Example 8, this example further sets up a visual restoration judgment unit to re-detect the actual image quality after cleaning, and decides whether to end the cleaning based on the restoration result.
[0094] The visual restoration judgment unit compares the quality of the image after cleaning with the quality of the image before cleaning, as well as preset restoration conditions. The preset restoration conditions are preferably consistent with the recognition quality conditions in Embodiment 3, i.e., the G value of the re-acquired image after cleaning is not lower than the first threshold, the C value does not exceed the second threshold, and the L value does not exceed the third threshold. To reduce frequent switching caused by fluctuations near the thresholds, restoration can be confirmed only after several consecutive re-detections meet the above conditions. When the restoration conditions are met, the algorithm switching unit restores the normal visual recognition mode and sends the cleaned image to the food status recognition unit; when the restoration conditions are not met, the control module can increase the number of cleaning cycles, change the drive speed, or expand the range of motion covered by the cleaning component to perform cleaning again. Through the sequence of "degradation detection—enhancement processing—physical cleaning—re-acquisition—restoration judgment," the monitoring component acquires active restoration capabilities.
[0095] To prevent the self-cleaning process from looping indefinitely, the control module can be set with a maximum number of cleaning cycles or a maximum recovery time. If an image meeting the recognition criteria cannot be obtained even after reaching this limit, the control module will mark the monitored image as unavailable and stop using it to generate new food status information. In this state, the device can limit continued heating, maintain necessary smoke extraction, and output maintenance prompts according to a pre-set safety control strategy. The purpose of this process is to prevent already confirmed unreliable images from continuing to drive heat and seasoning control, without requiring a specific alarm interface.
[0096] If condensation occurs in the camera's observation area during the cooking process, causing a decrease in the average gradient and increased brightness attenuation, the control module first enters an enhanced visual recognition mode to compensate for image brightness and enhance local contrast. If the enhanced image recovers to a recognizable range, food status recognition continues without changing the current action of the spatula assembly; if it still cannot be recovered, a self-cleaning compensation mode is entered, temporarily adjusting the spatula drive to allow the cleaning component to wipe the transparent isolation component. After cleaning, the image is re-acquired and, after passing a recovery judgment, is allowed to participate in food status recognition again. This process demonstrates the constraint relationship between the effectiveness of the monitoring signal and subsequent control.
[0097] In another scenario, the image color shifts overall but the sharpness remains high. The image quality detection unit can identify this anomaly based on the color shift amount C. The enhanced visual recognition mode performs color correction and then recalculates the image quality parameters. If the correction result meets the recognition criteria, the corrected image is used for food status recognition; if the color shift persists and self-cleaning conditions are met, further cleaning with transparent separators is performed. Thus, different degradation types can trigger recovery measures of varying intensities without requiring mechanical cleaning for every image fluctuation.
[0098] When camera communication malfunctions, multiple consecutive frames of images are invalid, or infrared temperature data exceeds the allowable range, the control module can mark the corresponding detection channel as abnormal. For cooking stages that rely on this detection channel for further judgment, the stage transition can be temporarily suspended and the cooktop components can be limited to further increase heating output. Data quality judgment is re-executed after the signal is restored. This abnormality handling differs from the aforementioned image degradation handling: in image degradation, image data can still be obtained and attempts can be made to enhance or clean the image, while in the case of communication abnormality, the first step is to check whether the detection channel has data output.
[0099] For determining whether vision has been restored after the self-cleaning action is completed, the control module should use the newly acquired image, rather than solely relying on whether the drive unit has completed its predetermined stroke. The completion of the drive unit's stroke only indicates that the cleaning action has been performed; whether the transparent isolator has truly returned to a recognizable state still needs confirmation from the image quality detection unit. It is this separation of "execution feedback and functional feedback" that creates a closed loop between the cleaning mechanism and visual recognition, rather than equating a single mechanical action with guaranteed cleaning success.
[0100] During the self-cleaning compensation mode, the cooking stage judgment unit can temporarily maintain the stage state before entering cleaning. The execution control unit only suspends the spatula stirring action that conflicts with the cleaning trajectory, while continuing to control the stove components and necessary smoke extraction actions according to the safety requirements of the current stage. After the visual recovery judgment is completed, the food status is updated using the newly acquired image. This avoids the brief cleaning process being misinterpreted as a sudden change in the food status, and also allows the cooking and cleaning functions of the spatula component to be switched smoothly by the same control module.
[0101] The control module can establish a status record for each visual recovery, including the G, C, and L values at the time of triggering, the enhancement method used, whether physical cleaning was initiated, the parameters after cleaning, and the recovery result. This record is used to determine whether the monitoring channel repeatedly degrades during the cooking process and to provide traceable information for equipment maintenance. The record itself does not participate in the calculation of food maturity, so even if it is not uploaded to an external device, it does not affect the smart cooker from independently completing closed-loop cooking.
[0102] This embodiment does not use the completion of the predetermined cleaning cycle as the sole criterion for visual recovery. Instead, it re-acquires images and compares the image quality parameters before and after cleaning. If the recovery conditions are not met, it continues to adjust the number of cleaning cycles, speed, or range. If the allowable number of cycles is exceeded, it restricts the failed images from continuing to participate in control. This solves the problem that the observation area may not have recovered even after the mechanical cleaning action is completed, forming a complete closed loop of contamination detection, cleaning execution, functional verification, and identification recovery.
[0103] Example 10: Based on Example 2, this example further utilizes the drinking water supply pipeline of the seasoning component to perform heat exchange and cooling on the area where the camera and / or control board are located.
[0104] To reduce the heat load on the monitoring electronic components in high-temperature environments, the water supply pipes in the seasoning assembly for supplying drinking water can be coiled or attached to the control board and camera inside the pot lid assembly. The control module can adjust the temperature based on the current cooking stage and the temperature inside the pot. Current heating output And cooking duration t establishes a predicted heat load for monitoring electronic components. The prediction model can employ a lookup table model based on calibration data, a linear regression model, or a piecewise function. For control boards or cameras with onboard temperature detection capabilities, the temperature of electronic components can also be detected. As input for model correction.
[0105] Drinking water flowing through the water supply pipes exchanges heat with the area where the control panel and / or camera are located before entering the cooking appliances. Assume the density of water is... Specific heat capacity is Volumetric flow rate is The water temperatures entering and leaving the heat exchange section are respectively and The actual heat exchange power on the water side It can be calculated using the following formula.
[0106] ; During the phase where drinking water needs to be supplied to cooking appliances, the control module determines the water supply flow rate and duration based on the predicted heat load and drinking water formula constraints. Let the total allowable drinking water supply for the current phase be... The amount deployed during the phase is The water supply duration is Then it should satisfy ; at the same time, by selecting make As close as possible to or no less than the target heat dissipation capacity that the water system needs to handle. .
[0107] ; When there is no demand for drinking water in the current stage, no additional water is continuously added to the pot for heat dissipation. The system can release the heat load by reducing the instantaneous heating output, increasing the exhaust / air cooling, or waiting for the next water supply stage. Thus, dynamic heat dissipation matching utilizes the drinking water that would otherwise be supplied to remove heat without disrupting the water constraints of the food, and further transforms the fixed water cooling arrangement into a controlled heat exchange process that changes with the cooking stage.
[0108] Water cooling and visual restoration are different reliability measures. Water cooling primarily reduces the impact of high temperatures on the camera and control board, while visual restoration mainly addresses image degradation caused by transparent insulating components or the medium inside the container. The control module can utilize both simultaneously: heat exchange during normal water supply, and independent visual compensation when image quality parameters exceed thresholds. This prevents temperature issues, oil film issues, and smoke issues from being lumped together as a single fault, and facilitates separate assessments of the effectiveness of restoration.
[0109] This embodiment allows the piping originally used for cooking water supply to exchange heat with the monitoring electronic components simultaneously during the water supply process, eliminating the need for a separate continuous water supply to the pot for heat dissipation; at the same time, heat load control and image contamination recovery are processed separately. This solves the problem of high heat load on electronic components when the monitoring component is close to the high-temperature cooking space, and avoids confusing high-temperature faults with contamination of the transparent isolation component as the same anomaly.
Claims
1. A smart cooker with a monitoring system, comprising a cooking appliance, a cooktop assembly for heating the cooking appliance, a lid assembly, a seasoning assembly, a spatula assembly, a monitoring assembly, and a control module, wherein the cooktop assembly, seasoning assembly, spatula assembly, and monitoring assembly are all electrically connected to the control module, characterized in that: The monitoring component includes an image acquisition unit for acquiring images of food inside the cooking appliance, a pot temperature acquisition unit for acquiring pot temperature data, and a transparent isolator located on the imaging optical path of the image acquisition unit. The image acquisition unit acquires food images through the transparent isolator. The control module includes an image quality detection unit, a food ingredient status recognition unit, a cooking stage judgment unit, a visual recovery control unit, and an execution control unit. The image quality detection unit generates an image quality status based on the clarity parameters, color shift parameters, and light attenuation parameters of the food ingredient image, and forms a pollution degree characterization based on the reference optical characteristics of the cleanliness status of the transparent isolation component and the light transmission attenuation relationship caused by pollution. When the image quality status characterizes image degradation, the visual restoration control unit controls the visual enhancement of the food image, and when the visually enhanced image still does not meet the recognition conditions, it controls the cleaning component of the spatula assembly to move along a cleaning trajectory that matches the actual field of view boundary of the camera on the transparent isolation component to clean the monitoring area and reacquire the food image. The ingredient status recognition unit generates ingredient status information based on the ingredient image that meets the recognition conditions. The cooking stage judgment unit integrates the ingredient status information, pot temperature data, executed control actions, and the stage target parameters of the corresponding dish to determine the current cooking stage. The execution control unit controls the heating output of the stove component, the stir-frying parameters of the spatula component, and the dispensing action of the seasoning component according to the current cooking stage, and acquires the ingredient image and pot temperature data again after execution.
2. The intelligent cooking appliance with a monitoring system according to claim 1, characterized in that, The monitoring component includes a camera, an infrared temperature sensor, a control board, and a transparent isolator. The camera and infrared temperature sensor are positioned at the monitoring position of the pot lid assembly facing the cooking appliance. The camera and infrared temperature sensor are electrically connected to the control board, and the control board is electrically connected to the control module. The camera acquires images of the food through the transparent isolator, and the infrared temperature sensor acquires the internal temperature data of the pot in the cooking appliance in a non-contact manner.
3. The intelligent cooking appliance with a monitoring system according to claim 2, characterized in that, The image quality detection unit is configured to: for the region of interest image acquired by the camera through the transparent isolator, average the composite gradient formed by the horizontal gray level difference and the vertical gray level difference of adjacent pixels in the region of interest to obtain the average gradient value G of the image. Convert the current image to the Lab color space, calculate the distance between the mean values of channels a and b and the mean values of the reference image when the transparent isolator is in a clean state, and obtain the color offset C. Calculate the relative attenuation of the average brightness of the current image relative to the average brightness of the reference image to obtain the brightness attenuation value L; establish an equivalent contamination optical thickness D that conforms to the Beer-Lambert attenuation relationship based on the transmittance ratio of the current image and the reference image, and use G, C, L and D together to characterize the degree of contamination of the transparent insulating component affected by oil film, condensate and / or smoke. The first threshold, the second threshold, and the third threshold are determined based on statistical results from multiple reference images of the clean state and graded pollution calibration images.
4. The intelligent cooking appliance with a monitoring system according to claim 3, characterized in that, The visual restoration control unit includes an algorithm switching unit and a visual enhancement unit. The algorithm switching unit switches between normal visual recognition mode, enhanced visual recognition mode and self-cleaning compensation mode according to the image quality status. In the enhanced visual recognition mode, the visual enhancement unit performs at least one of the following on the original food image: dehazing, brightness compensation, color correction, edge enhancement, and local contrast enhancement, based on at least one of the sharpness parameter, color shift parameter, and illumination attenuation parameter, to obtain an enhanced food image.
5. The intelligent cooking appliance with a monitoring system according to claim 4, characterized in that, The food ingredient state recognition unit includes a pre-trained food ingredient visual feature model, which is a convolutional neural network model or a visual Transformer model. The training samples include at least food ingredient images labeled as fresh, early heating, semi-cooked, mature, and overcooked, and include cooking visual degradation enhancement samples constructed by means of fogging, brightness attenuation, and color shift. The model training uses a weighted combination of classification loss and adjacent time-state consistency loss as the training objective. The model takes food images that meet the recognition conditions as input and outputs the food category and at least one of the following: food color change, texture change, morphological change and degree of ripeness.
6. The intelligent cooking appliance with a monitoring system according to claim 1, characterized in that, The cooking stage determination unit is configured to: calculate, for each candidate cooking stage, the visual matching items between the visual state of the ingredients and the candidate stage, the temperature matching items between the temperature inside the pot and the target temperature range of the candidate stage, the historical matching items between the executed control actions and the action sequence of the candidate stage, and the target matching items of the corresponding dish stage target parameters; weight and fuse each matching item to form a stage score; and select the candidate stage with the highest stage score from the candidate stages that meet the preset stage transition constraints, and determine it as the current cooking stage among the preheating stage, ingredient addition stage, heating stage, stir-frying stage, seasoning addition stage, sauce reduction stage, and cooking completion stage.
7. The intelligent cooking appliance with a monitoring system according to claim 6, characterized in that, The execution control unit is configured to: increase the heating output of the stove component when the temperature inside the pot is lower than the target temperature range corresponding to the current cooking stage, and decrease the heating output of the stove component when the temperature inside the pot is higher than the target temperature range; change the stirring speed and / or stirring duration of the spatula component according to the food status information, and trigger the seasoning component to add corresponding seasonings and / or drinking water according to the current cooking stage; and reacquire the food image and pot temperature data after each control execution to update the current cooking stage.
8. The intelligent cooking appliance with a monitoring system according to claim 2, characterized in that, The spatula assembly includes a drive device electrically connected to the control module and a cleaning component driven by the drive device. The cleaning component is a wear-resistant and high-temperature resistant silicone brush head mounted on a connecting rod. The movement trajectory, effective cleaning width, and start and end positions of the cleaning component are determined based on the curvature parameters of the transparent isolation component and the monitoring field of view boundary formed by the camera through the transparent isolation component, so that the ratio of the effective coverage area formed by the cleaning component to the monitoring area is not less than 95%. The cleaning contact pressure is limited by the elastic modulus, allowable stress, and contact area of the transparent isolator material to remove oil film and / or condensate in the monitored area and reduce the risk of scratching the transparent isolator in the self-cleaning compensation mode.
9. The intelligent cooking appliance with a monitoring system according to claim 8, characterized in that, The visual restoration control unit further includes a visual restoration judgment unit, which compares the image quality parameters before and after the cleaning action. When the average gradient value G of the cleaned image is not lower than the first threshold, the color offset C does not exceed the second threshold, and the brightness attenuation value L does not exceed the third threshold, it is determined that the preset restoration conditions are met and the system switches to normal visual recognition mode. When the image quality parameters after cleaning do not meet the preset restoration conditions, the control module adjusts the number of cleaning cycles, the cleaning speed, and / or the cleaning movement range and then performs cleaning and visual restoration judgment again.
10. The intelligent cooking appliance with a monitoring system according to claim 2, characterized in that, The seasoning assembly includes a water supply pipe for supplying drinking water, and at least a portion of the water supply pipe is located within the pot lid assembly, coiled around and / or attached to at least one of the control panel and the camera; The control module determines the predicted heat load of the monitoring electronic components based on the current cooking stage, pot temperature, current heating output, and cooking duration. It also determines the heat exchange capacity based on the density, specific heat capacity, volumetric flow rate, and inlet / outlet water temperature difference of the water flowing through the water supply pipeline. Under the condition of meeting the drinking water supply constraints of the current cooking stage, it adjusts the water supply flow rate and / or water supply duration to match the heat exchange capacity with the predicted heat load.
Citation Information
Patent Citations
A smart control automatic cooking machine and its intelligent control method
CN108309021B
Intelligent cooking machine based on image monitoring and adjustment method thereof
CN108445793A
