A perception device suitable for a kitchen

Abstract

The present disclosure relates to a perception device suitable for a kitchen, comprising: a processor and at least one thermal imaging module, the thermal imaging module is arranged in the kitchen, the detection end faces the ground to detect the infrared radiation of a corresponding detection area on the ground to obtain temperature distribution data of the detection area, the processor identifies a person and a heat source in the detection area based on the temperature distribution data, the processor calculates a first distance between the identified person and the heat source, and / or the processor determines a relative position of the person and / or a relative position of the heat source, the processor determines state information of the heat source based on the temperature distribution data, the processor performs at least one function of the following based on the state information of the heat source, the identification result of whether there is a person in the thermal imaging image, and the first distance and / or the relative position: determining a kitchen scene; determining a normal state or an abnormal state; determining a risk state; generating a decision; sending a control command.

Description

Technical Field

[0001] This disclosure relates to the field of indoor sensing, and in particular to a sensing device suitable for kitchens. Background Technology

[0002] In kitchen settings, users often need to handle multiple tasks simultaneously while cooking, such as temporarily leaving the kitchen to perform other tasks. This multitasking behavior is common in daily life, but it also brings potential safety risks.

[0003] Therefore, a sensing device suitable for the kitchen is needed. Summary of the Invention

[0004] One object of this disclosure is to provide a sensing device suitable for kitchens.

[0005] According to a first aspect of this disclosure, a sensing device is provided, comprising: a processor and at least one thermal imaging module, the thermal imaging module being arranged in a kitchen with its detection end facing the ground to perform infrared radiation detection on a corresponding detection area on the ground to obtain temperature distribution data of the detection area; the processor identifying a person and a heat source within the detection area based on the temperature distribution data; the processor calculating a first distance between the identified person and the heat source; and / or, the processor determining the relative position of the person and / or the relative position of the heat source; the processor determining state information of the heat source based on the temperature distribution data; and the processor performing at least one of the following functions based on the state information of the heat source, the identification result of whether there is a person in the thermal imaging image, and the first distance and / or the relative position: determining a kitchen scene; determining a normal state or an abnormal state; determining a risk state; generating a decision; and sending a control command.

[0006] Optionally, each temperature value in the temperature distribution data corresponds to a pixel position in the image plane. The processor calculates the horizontal distance between the person and the heat source based on the first size information of a standard human body and the mapping principle of the thermal imaging module. The first distance includes the horizontal distance. Alternatively, the sensing device may also include a ranging module or a positioning module. The ranging module measures the second distance between the thermal imaging module and the person, and the positioning module determines the first position of the person relative to the thermal imaging module. The processor calculates the horizontal distance between the person and the heat source based on the second distance or the first position and the mapping principle of the thermal imaging module.

[0007] Optionally, the ranging module includes a thermal imaging module and a processor. The thermal imaging module measures the radiation temperature of the target object as it changes with the distance between the target object and the thermal imaging module. The processor determines a second distance between the thermal imaging module and the person based on the temperature value of at least one pixel location in a first pixel region that is different from the background temperature and is identified as a person, and pre-acquired calibration data. The calibration data characterizes the correspondence between the radiation temperature and the distance obtained by the thermal imaging module from measuring the human body at different distances.

[0008] Optionally, the processor calculates the third size information of the person based on the second distance and the first pixel region, and verifies whether the type of the imaged object in the first pixel region is a person based on the third size information.

[0009] Optionally, the ranging module includes a radar sensor. The radar sensor detects whether the type of the reflecting object in the detection area is a person, and detects a third distance between the identified person and the radar sensor. The second distance is the third distance, or the second distance is a distance corrected for the third distance based on the relative positional relationship between the radar sensor and the thermal imaging module. The processor detects whether there is a target pixel area in the pixel area corresponding to the second distance in the image plane. If there is a target pixel area, it determines that the type of the imaging object in the target pixel area is a person. The target pixel area is a pixel area where the type of the imaging object is likely to be a person, determined based on temperature distribution data.

[0010] Optionally, the radar sensor includes at least two receiving antennas. The radar sensor also measures a first offset angle of the person relative to the radar sensor. The second offset angle of the person relative to the thermal imaging module is the first offset angle, or the second offset angle is an offset angle corrected for the first offset angle based on the relative positional relationship between the radar sensor and the thermal imaging module. The processor detects whether there is a target pixel region in the image plane corresponding to the second distance and whose offset angle relative to the reference pixel corresponds to the second offset angle. If there is a target pixel region, the type of the imaging object in the target pixel region is determined to be a person. The optical axis of the reference pixel is substantially perpendicular to the horizontal plane.

[0011] Optionally, the processor calculates the person's height based on the second distance or first position, the image angle between the head position in the first pixel region and the reference pixel, and the mapping principle of the thermal imaging module, wherein the optical axis of the reference pixel is substantially perpendicular to the horizontal plane; and / or, the processor calculates the first physical size corresponding to the first pixel region, and the processor determines whether the person is an adult based on the height and / or the first physical size; the processor performs at least one of the following functions based on the state information of the heat source, the identification result of whether there is an adult in the thermal imaging image, and the first distance between the identified adult and the heat source: determining the kitchen scene; determining the risk state; generating a decision; sending a control command; or the processor performs at least one of the following functions based on the state information of the heat source, the first distance between the identified person and the heat source, and the determination result of whether the person is an adult: determining the kitchen scene; determining the normal state or abnormal state; determining the risk state; generating a decision; sending a control command.

[0012] Optionally, the processor determines the state information of the heat source based on the angle between the optical axis of the thermal imaging module and the horizontal plane and / or the installation height of the thermal imaging module, as well as temperature distribution data.

[0013] Optionally, the processor identifies the heat source type and / or cooking method based on the heat source status information, or the processor obtains the heat source type and / or cooking method from external input. Based on the heat source type and / or cooking method, the heat source status information, the identification result of whether there is a person in the thermal imaging image, and the first distance between the identified person and the heat source, the processor performs at least one of the following functions: determining the kitchen scene; determining the normal state or abnormal state; determining the risk state; generating a decision; and sending control commands.

[0014] Optionally, the processor identifies pets within the detection area based on temperature distribution data and calculates a fourth distance between the identified pet and the heat source. Based on the state information of the heat source, the identification result of whether there is a person in the thermal imaging image, the first distance between the identified person and the heat source, the identification result of whether there is a pet in the thermal imaging image, and the fourth distance between the identified pet and the heat source, the processor performs at least one of the following functions: determining the kitchen scene; determining the normal or abnormal state; determining the risk state; generating a decision; and sending control commands.

[0015] Optionally, the processor determines whether there are people in the operating range, the observation range, and the non-observation range based on the first distance and / or the relative position, obtains a first determination result, and performs at least one of the following functions based on the state information of the heat source and the first determination result: determining the kitchen scene; determining the normal state or the abnormal state; determining the risk state; generating a decision; and sending a control command, wherein the operating range is smaller than the observation range, and the observation range is smaller than the non-observation range.

[0016] Optionally, the processor sends control commands to the corresponding actuators based on the kitchen scenario and / or risk status and / or determined normal or abnormal status.

[0017] Optionally, the actuator includes a fire extinguishing module. When the processor determines that there is a fire and no person is detected within the operating range and observation range, or the first distance is increasing, it controls the fire extinguishing module to spray fire extinguishing agent toward the fire heat source based on the location information of the fire heat source.

[0018] Optionally, the actuator includes an output module, to which the processor sends control signals to control the output module to output alert information that matches the kitchen scene and / or risk status.

[0019] Optionally, the sensing device also includes a smoke sensor arranged in the kitchen. The processor determines whether the detection result of the smoke sensor is usable based on the execution result of at least one function, or the processor executes at least one function based on the detection result of the smoke sensor.

[0020] Optionally, the processor determines state node information based on the state information of the heat source, the identification result of whether there is a person in the thermal imaging image, and the first distance and / or the relative position. The state node information includes at least one of the following: current state node, cause of the current state node, future state node evolved from the current state node, and cause of the future state node. Based on the state node information, the processor performs at least one of the following functions: determining the kitchen scene; determining the normal state or abnormal state; determining the risk state; generating a decision; and sending control commands.

[0021] Optionally, the processor searches for a target basic change unit in the set of basic change units that matches the first key variable. The set of basic change units includes multiple basic change units, and the basic change unit includes at least one of the following: a first state node, a second key variable of the first state node and / or one or more second state nodes preceding the first state node, one or more third state nodes following the first state node, the cause of the evolution from the second state node to the first state node, and the cause of the evolution from the first state node to the third state node. The first key variable includes the state information of the heat source, the identification result of whether there is a person in the thermal imaging image, and the first distance and / or the relative position. Based on the found target basic change unit, the processor obtains the state node information.

[0022] According to a second aspect of this disclosure, a sensing device is provided, comprising: a processor and at least one thermal imaging module, the thermal imaging module being arranged in a kitchen with its detection end facing the ground to perform infrared radiation detection on a corresponding detection area on the ground to obtain temperature distribution data of the detection area; the processor identifying a person and a heat source within the detection area based on the temperature distribution data; the processor calculating a first distance between the identified person and the heat source; and / or, the processor determining the relative position of the person and / or the relative position of the heat source; the processor determining state information of the heat source based on the temperature distribution data; and the processor determining state node information based on the state information of the heat source, the identification result of whether a person is present in the thermal imaging image, and the first distance and / or the relative position, the state node information including at least one of the following: a current state node, the cause of the current state node, a future state node evolved from the current state node, and the cause of the future state node.

[0023] According to a third aspect of this disclosure, a sensing device is provided, comprising: a processor and at least one thermal imaging module, the thermal imaging module being arranged in a kitchen with its detection end facing the ground to perform infrared radiation detection on a corresponding detection area on the ground to obtain temperature distribution data of the detection area, the processor identifying heat sources within the detection area based on the temperature distribution data and determining the state information of the heat sources, and the processor determining the kitchen scene based on the state information of the heat sources.

[0024] Optionally, the processor identifies the heat source type and / or cooking method based on the heat source status information, or the processor obtains the heat source type and / or cooking method from external input, and determines the kitchen scene based on the heat source type and / or cooking method and the heat source status information.

[0025] Optionally, the processor identifies people within the detection area based on temperature distribution data, and determines the kitchen scene based on the status information of the heat source and the identification results of whether there are people in the thermal imaging image.

[0026] Optionally, the processor sends control commands to the corresponding actuators based on the kitchen scenario.

[0027] According to a fourth aspect of this disclosure, a sensing device is provided, comprising: a processor, at least one thermal imaging module, and a ranging module. The thermal imaging module is arranged indoors with its detection end facing the ground to perform infrared radiation detection on a corresponding detection area on the ground to obtain temperature distribution data of the detection area. The processor identifies a person within the detection area based on the temperature distribution data. The ranging module measures a second distance between the thermal imaging module and the person. Based on the second distance and the mapping principle of the thermal imaging module, the processor calculates a first relative position of the person within the detection area relative to the optical axis of a reference pixel projected onto the ground. The optical axis of the reference pixel is substantially perpendicular to the horizontal plane.

[0028] Optionally, each temperature value in the temperature distribution data corresponds to a pixel position in the image plane. The ranging module includes a thermal imaging module and a processor. The thermal imaging module measures the change in radiation temperature of the target object as the distance between the target object and the thermal imaging module changes. The processor determines a second distance between the thermal imaging module and the person based on the temperature value of at least one pixel position in the first pixel region identified as a person in the image plane and pre-acquired calibration data. The calibration data characterizes the correspondence between radiation temperature and distance obtained by the thermal imaging module from measuring the human body at different distances.

[0029] Optionally, the processor calculates the third size information of the person based on the second distance and the first pixel region, and verifies whether the type of the imaged object in the first pixel region is a person based on the third size information.

[0030] Optionally, each temperature value in the temperature distribution data corresponds to a pixel position in the image plane. The ranging module includes a radar sensor. The radar sensor detects whether the type of the reflecting object in the detection area is a person, and detects a third distance between the identified person and the radar sensor. The second distance is the third distance, or the second distance is the distance corrected for the third distance based on the relative positional relationship between the radar sensor and the thermal imaging module. The processor detects whether there is a target pixel area in the pixel area corresponding to the second distance in the image plane. If there is a target pixel area, it is determined that the type of the imaging object in the target pixel area is a person. The target pixel area is a pixel area where the type of the imaging object determined based on the temperature distribution data is likely to be a person.

[0031] Optionally, the radar sensor includes at least two receiving antennas. The radar sensor also measures a first offset angle of the person relative to the radar sensor. The second offset angle of the person relative to the thermal imaging module is the first offset angle, or the second offset angle is an offset angle corrected for the first offset angle based on the relative positional relationship between the radar sensor and the thermal imaging module. The processor detects whether there is a target pixel region in the image plane that corresponds to the second distance and whose offset angle relative to the reference pixel corresponds to the second offset angle. If a target pixel region exists, the type of the imaging object in the target pixel region is determined to be a person.

[0032] This disclosure, based on the state information of the heat source, the identification result of whether there is a person in the thermal imaging image, and the first distance and / or relative position between the identified person and the heat source, can more accurately perform at least one of the following functions: determine the kitchen scene; determine the normal or abnormal state; determine the risk state; generate a decision; and send a linkage control command. Attached Figure Description

[0033] The above and other objects, features and advantages of this disclosure will become more apparent from the more detailed description of exemplary embodiments thereof taken in conjunction with the accompanying drawings, wherein like reference numerals generally denote like parts.

[0034] Figure 1 This is a schematic block diagram of a sensing device according to some embodiments of the present disclosure.

[0035] Figure 2 This is a schematic diagram of the deployment of sensing devices in a kitchen according to some embodiments of the present disclosure.

[0036] Figure 3 This is a schematic diagram of a kitchen scene determined by understanding a kitchen scene based on some embodiments of this disclosure.

[0037] Figure 4 This is a schematic diagram illustrating the development trend of two typical kitchen scenarios over time according to some embodiments of this disclosure.

[0038] Figure 5 This is a diagram illustrating the differences between this disclosure and conventional solutions in terms of information processing.

[0039] Figure 6 This is a schematic diagram of calculating a first distance according to some embodiments of the present disclosure.

[0040] Figure 7 This is a schematic diagram of calculating a first distance according to other embodiments of this disclosure.

[0041] Figure 8 This is a schematic block diagram of a sensing device according to other embodiments of the present disclosure. Detailed Implementation

[0042] Preferred embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While preferred embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0043] Those skilled in the art will understand that the terms "first," "second," etc., in this disclosure are used to distinguish similar objects, rather than to describe a specific order or sequence, and do not have any additional limiting effect.

[0044] Figure 1 This is a schematic block diagram of a sensing device according to some embodiments of the present disclosure.

[0045] In some embodiments, the sensing device 100 includes a processor 110 and at least one thermal imaging module 120.

[0046] The processor 110 and the thermal imaging module 120 can be integrated together to form an "integrated" sensing device 100. Alternatively, the processor 110 can be deployed separately from the thermal imaging module 120 to form a "split" sensing device 100. In other words, the sensing device 100 supports both "integrated" and "split" structural forms, which can be flexibly configured according to actual needs.

[0047] A thermal imaging module 120 is installed in the kitchen with its detection end facing the ground to detect infrared radiation in a corresponding detection area on the ground, obtaining temperature distribution data for that area. The objects detected by the thermal imaging module 120 include people and heat sources (optionally including pets as described below). The height of a human body is typically greater than the height of a heat source (such as a stove). Therefore, the installation height of the thermal imaging module 120 (i.e., the height of the detection end of the thermal imaging module 120) should be higher than the height of a human body. For example, the installation height of the thermal imaging module 120 can be greater than 2 meters. Exemplarily, the thermal imaging module 120 can be ceiling-mounted, deployed in the ceiling of the kitchen. The kitchen can be an open kitchen or a closed kitchen.

[0048] The installation height of the thermal imaging module 120 can be a fixed height preset at the factory or a non-fixed height. That is, the installation height of the thermal imaging module 120 can be a known predetermined height or an unknown non-predetermined height. Typically, the indoor floor is horizontal or nearly horizontal, and the optical axis of the thermal imaging module 120 (the optical axis corresponding to the center pixel of the thermal imaging module 120) can be set to be substantially perpendicular to the horizontal plane (i.e., vertically installed), so that the optical axis of the thermal imaging module 120 is substantially perpendicular to the ground. The optical axis of the thermal imaging module 120 can also be set at an angle; when tilted, the angle between the optical axis of the thermal imaging module 120 and the horizontal plane can be determined, for example, by on-site measurement.

[0049] The number of thermal imaging modules 120 included in the sensing device 100 can be set according to actual needs. For example, if the detection range of a single thermal imaging module 120 is sufficient to cover the entire kitchen, the sensing device 100 may include only one thermal imaging module 120. If the detection range of a single thermal imaging module 120 is insufficient to cover the entire kitchen, the sensing device 100 may include multiple thermal imaging modules 120, so that the union of the detection areas of these multiple thermal imaging modules 120 can basically cover the entire kitchen, realizing the monitoring of the entire kitchen.

[0050] Figure 2 This is a schematic diagram of the deployment of sensing devices in a kitchen according to some embodiments of the present disclosure. Figure 2The sensing device 100 shown is an integrated sensing device. It should be understood that the sensing device 100 also supports modular deployment. That is, the processor 110 and the thermal imaging module 120 in the sensing device 100 can be deployed in different locations. See also... Figure 2 Taking the integrated sensing device 100 as an example, the sensing device 100 (that is, the thermal imaging module 120 in the sensing device 100) can be deployed on the top of the kitchen.

[0051] The thermal imaging module 120 may include at least one thermal imaging sensor and at least one thermal imaging lens. For example, the thermal imaging module 120 may consist of one thermal imaging sensor and one thermal imaging lens. Alternatively, the thermal imaging module 120 may be composed of multiple thermal imaging sensors and / or multiple thermal imaging lenses stitched together. When the thermal imaging module 120 is composed of multiple thermal imaging sensors and / or multiple thermal imaging lenses stitched together, the field of view angles of adjacent thermal imaging lenses may be adjacent at the boundary without overlapping, or they may overlap. The thermal imaging sensor may be, but is not limited to, a LWIR (long-wave infrared) wavelength range thermal imaging sensor, a MWIR (mid-wave infrared) wavelength range thermal imaging sensor, a SWIR (short-wave infrared) wavelength range thermal imaging sensor, a NIR (near-infrared) wavelength range thermal imaging sensor, or a FIR (far-infrared) wavelength range thermal imaging sensor.

[0052] In some embodiments, the thermal imaging module 120 may be a thermal imaging camera. The thermal imaging camera may be a thermal imaging camera with a wide field of view or greater. For example, the diagonal field of view (DFOV) of the thermal imaging camera may be greater than 80 degrees. The thermal imaging camera may be a thermal imaging camera designed based on hardware and / or software to identify the category of the imaged object without identifying the individual. For example, the resolution or sensitivity of the thermal imaging camera may be such that it can identify the category of the imaged object without identifying the individual. By using, for example, a thermal imaging camera with medium to low resolution (e.g., less than QVGA resolution, 320x240) or medium to low sensitivity, or by using a thermal imaging camera that undergoes post-processing such as software blurring or software encryption, privacy concerns can be reduced, thereby expanding the detection area. Additionally, for detection areas and / or objects with special requirements, a high-resolution thermal imaging camera may be considered, for example, when providing privacy alerts by informing the user of the presence of a thermal imaging camera.

[0053] The thermal imaging module 120 has a defined field of view. After the installation position and orientation of the thermal imaging module 120 are fixed, its image plane (i.e., the photosensitive plane where the pixel array of the thermal imaging sensor is located) and the range of incident directions in space (i.e., all incident directions within the solid angle covered by the field of view) form a defined projection mapping relationship. There is a one-to-one correspondence between the incident directions and the pixel positions on the image plane. Therefore, in the temperature distribution data obtained by the thermal imaging module 120 from infrared radiation detection of the detection area, each temperature value corresponds to a pixel position in the image plane, and this temperature value can be regarded as the pixel value at that pixel position. The temperature distribution data can also be called a thermal image.

[0054] The temperature distribution data obtained by the thermal imaging module 120 can be transmitted to the processor 110 via wired or wireless means.

[0055] The processor 110 identifies people and heat sources within the detection area based on temperature distribution data. The processor 110 can utilize an artificial intelligence (AI) model system or a machine vision processing system to process the temperature distribution data to identify people and heat sources within the detection area. The term "person" as used in this disclosure can include human life forms and robots with humanoid forms. That is, the "person" as used in this disclosure can be either a human life form or a robot.

[0056] The processor 110 can also calculate a first distance between the identified person and the heat source, and / or, the processor 110 can also determine (e.g., by calculation or acquisition from an external source) the relative position of the person and / or the relative position of the heat source. The relative position may refer to the position projected onto the ground relative to the optical axis of a reference pixel. See the description below for an example of a reference pixel.

[0057] In some embodiments, the processor 110 can identify hotspot regions (i.e., pixel regions) in temperature distribution data whose temperature values ​​differ from the background temperature. Based on information such as the shape, size, temperature, and movement of the hotspot regions, the processor determines the type of the hotspot region, thereby identifying people and heat sources within the detection area. Specifically, the processor 110 can obtain the positional change information of the same hotspot region over a period of time based on temperature distribution data from multiple different moments within that time period. Based on the positional change information, the processor determines the movement information of the hotspot region. The movement information may include a determination of whether the hotspot region is moving and / or specific movement parameters (such as movement speed) when the hotspot region is moving.

[0058] The processor 110 also determines the state information of the heat source based on temperature distribution data. When determining the state information of the heat source, the processor 110 can determine the state information of the heat source at the current moment based on the temperature distribution data at the current moment (optionally also based on temperature distribution data collected at one or more previous moments, or state information of the heat source determined at one or more previous moments). For example, the processor 110 can determine the temperature change information and / or spatial distribution information of the heat source based on multi-frame temperature distribution data, and determine the state of the heat source based on the temperature change information and / or spatial distribution information. In some embodiments, the processor 110 can utilize an AI model to determine the state information of the heat source.

[0059] The state information of a heat source can characterize its state under one or more feature dimensions. For example, the state information of a heat source can include, but is not limited to, any one or any combination of multiple feature dimensions such as on / off state, motion state, temperature state, expansion / contraction state, and heat source shape. On / off state characterizes whether the heat source (i.e., the stove heat source) is in an on or off state. Motion state characterizes whether the heat source is moving and / or its rate of movement. Temperature state characterizes the temperature changes of the heat source and the spatial distribution characteristics of the temperature field in the corresponding hot spot area. Temperature changes can refer to, for example, the rate of temperature change (e.g., linear rapid heating, linear temperature maintenance, linear cooling, abrupt heating, abrupt cooling, etc.). Spatial distribution characteristics can reflect, for example, the uniformity of the spatial distribution of the temperature field. Expansion / contraction state can reflect whether the heat source is in an expanding (e.g., abrupt diffusion) or contracting state.

[0060] The observation angle and / or observation distance of the thermal imaging module 120 can affect the determination of the state information of the heat source. Taking a stove heat source as an example, the temperature characteristics of the side and top of the stove are different. If the observation angle of the thermal imaging module 120 is vertically downward, it will mainly observe the temperature information above the stove. If the observation angle of the thermal imaging module 120 is at a certain angle to the vertical plane, it may observe the temperature information of the side and part of the top of the stove. In addition, the size and / or shape of the hot spot area corresponding to the stove heat source obtained by the thermal imaging module 120 at different distances and / or observation angles will also be different. Therefore, if the state information of the heat source (such as temperature state, expansion and contraction state, heat source shape, etc.) is determined solely based on temperature distribution data without considering the observation angle and observation distance, the state information of the heat source cannot be accurately determined, which will affect subsequent processes. For example, it may affect the accuracy of the processor 110 in performing specific functions (such as determining the kitchen scene, determining the risk state, generating decisions, and sending control commands, as described below).

[0061] In view of this, this disclosure proposes that the processor 110 can determine the state information of the heat source based on the angle between the optical axis of the thermal imaging module 120 and the horizontal plane and / or the installation height of the thermal imaging module 120, as well as temperature distribution data. The angle between the optical axis of the thermal imaging module 120 and the horizontal plane can, to some extent, characterize the observation angle of the thermal imaging module 120. The installation height of the thermal imaging module 120 can, to some extent, characterize the observation distance of the thermal imaging module 120. Thus, by combining the angle and the installation height, more accurate state information of the heat source can be obtained. For example, the determined state information of the heat source can characterize the state of the heat source under specific observation angle and / or distance conditions.

[0062] Based on the state information of the heat source, the identification result of whether there is a person in the thermal imaging image, and the first distance and / or relative position between the identified person and the heat source, the processor 110 performs at least one of the following functions: determining the kitchen scene; determining the normal state or the abnormal state; determining the risk state; generating a decision; and sending control commands.

[0063] The presence of a person in a thermal imaging image and the initial distance between the identified person and the heat source can reflect the relationship between the heat source and the person from dimensions such as operation and perception. For example, if there is no person in the thermal imaging image, it indicates that the heat source is not being manipulated, and the state information of the heat source may not be perceptible to the person. If there is a person in the thermal imaging image, the initial distance between the identified person and the heat source can be used to determine whether the heat source can be manipulated and / or whether the state information of the heat source is perceptible to the person. For example, if the initial distance is small, it can be assumed that the person is within the range where the heat source can be manipulated, or it can be assumed (for example, by combining other information, such as the person's hand posture and the distance between the person's hand and the heat source) that the heat source is being manipulated, and the state information of the heat source is a result of human operation. As another example, if the initial distance is large, it can be assumed that the person is located in the kitchen, far away from the heat source, and the heat source is not being manipulated, and the state of the heat source is unknown to the person.

[0064] In some cases, the relative position of a person and / or the relative position of a heat source can help determine the validity of the first distance assessment. For example, based on the relative position of a person and / or the relative position of a heat source, it can be determined whether there is obstruction between the person and the heat source (especially a cooking heat source). If a person is obstructed (e.g., by a non-cooking heat source such as a refrigerator) and cannot visually observe the heat source, even if the first distance between the person and the heat source is identified as close (e.g., within the observation range) through thermal imaging, the first distance cannot accurately reflect the relationship between the heat source and the person from operational and perceptual dimensions because the person cannot visually observe the heat source. For another example, if a person is preparing food within the observation range but not facing the cooking heat source, this is considered an invalid observation but allows for rapid reaction; therefore, the corresponding anomaly detection and alert strategies will differ.

[0065] In some cases, when multiple heat sources are present, their relative positions can provide a better understanding of their interrelationships. For example, the movement of a heat source in one location can derive other heat sources. This could be caused by a pot being moved (its location is usually far from the stove), a fire resulting from diffusion (its location is usually near the stove), or even a mirror reflection caused by a metal surface (which occurs simultaneously with the heat source, but is smaller, farther away, and similar in location). Information about the locations of multiple heat sources helps in understanding the derivation relationships between them.

[0066] In some situations, the relative positions of people and / or heat sources are particularly important. For example, in an automated fire suppression system after a fire has been confirmed, the location of the fire is a prerequisite for targeted fire suppression. If someone is present, whether their position obstructs the fire extinguishing nozzles is crucial information. A person's approaching or moving away can also serve as feedback for activating the fire suppression system. Furthermore, people generally cannot detect the location of non-combustible abnormal heat sources. Therefore, providing information about the location of these abnormal heat sources becomes essential. For instance, based on this information, LEDs can indicate the direction of the abnormal heat source (e.g., a non-cooking heat source), or the relative position of the abnormal heat source can be mapped onto a location map in a software app. Additionally, when a heat source moves, if the trajectory of a person's movement is close to that of the heat source, it can be interpreted as the person moving the heat source. Conversely, if the trajectories are different or opposite, it can be interpreted as an accident involving the heat source and the person trying to avoid it. Therefore, the relative positions of people and / or heat sources are extremely important for understanding the situation.

[0067] The processor 110, by combining the identification results of whether a person is present in the thermal imaging image with the first distance and / or relative position (relative position of the person and / or relative position of the heat source) between the identified person and the heat source, rather than solely based on the state information of the heat source, enables it to more accurately perform at least one of the following functions: determine the kitchen scene; determine the normal or abnormal state; determine the risk state; generate a decision; and send control commands.

[0068] In some embodiments, the distance between the heat source and a person can be divided into three non-overlapping ranges: the operating range, the observation range, and the non-observation range. The operating range is smaller than the observation range, and the observation range is smaller than the non-observation range. For example, the operating range can be 0-0.8m, the observation range can be 0.8m-3m, and the non-observation range can be greater than 3m. Optionally, the observation range can be further subdivided into a near observation range and a far observation range. For example, 0.8m-1.5m can be classified as the near observation range, and 1.5m-3m as the far observation range. If the first distance is within the operating range, it can be considered that the person is within a range where they can operate the heat source at any time, or it can be considered that the state information of the heat source is the result of human manipulation, and no reminder operation and / or control operation is required (or reminder and / or control operation is only applied when a fire risk state is determined). If the first distance is within the observation range (e.g., the near observation range), it can be considered that the state of the heat source can be directly perceived by the person in the kitchen, and no reminder (or only a weak reminder) or control operation is required (or control operation is only applied when a fire risk state is determined). If the first distance is outside the observation range, the heat source can be considered to be imperceptible to people in the kitchen, requiring alerts and / or control based on the heat source's status. For example, in the presence of the first distance, the processor 110 can determine whether there is a person in the operating range, observation range, or non-observation range based on the first distance and / or relative position, obtaining a first determination result. The first determination result indicates which of the operating range, observation range, or non-observation range the person is located in. The processor 110 can perform at least one of the following functions based on the heat source's status information, the first determination result (optionally also based on relative position and at least one of other information described below): determining the kitchen scene; determining a normal or abnormal state; determining a risk state; generating a decision; and sending a control command. For example, a second determination result can also be obtained based on indoor positioning technology or wireless network technology to determine whether a person might be beyond the reaction distance. For instance, whether a person is within the reaction distance can be determined based on whether the user's portable terminal device (such as a smartphone or smartwatch) is connected to the home network. The reaction distance can be understood as the distance from which a person can return to the kitchen scene in a short time, such as a location near the house or outside. The processor can perform at least one of the following functions based on the state information of the heat source, a first determination result, a second determination result (optionally also based on relative position and at least one of other information described below): determine the kitchen scene; determine the normal state or the abnormal state; determine the risk state; generate a decision; and send control commands.

[0069] The processor 110 can perform any one or more of the following five functions: "determining the kitchen scene," "determining a normal or abnormal state," "determining a risk state," "generating a decision," and "sending a control command." In some embodiments, these five functions can be sequentially ordered. For example, the kitchen scene can be determined first, then a normal or abnormal state can be determined based on the determined kitchen scene, then a risk state can be determined based on the determined normal or abnormal state, then a decision can be generated based on the determined risk state, and finally a control command can be issued based on the generated decision. In some embodiments, these five functions can also be executed independently. For example, the processor 110 can send control commands to the corresponding actuators based on any one or more of the kitchen scene, normal or abnormal state, risk state, and decision. For example, processor 110 can first determine the kitchen scene, and based on the determined kitchen scene, determine a normal or abnormal state, and / or determine a risk state, and / or generate a decision, and then send control commands based on the determined normal or abnormal state, and / or risk state, and / or decision. As another example, processor 110 can first determine the kitchen scene, and then send control commands based on the determined kitchen scene. As yet another example, processor 110 can first determine a normal or abnormal state, and / or risk state, and / or generate a decision, and then send control commands based on the determined normal or abnormal state, and / or risk state, and / or generated decision.

[0070] In this disclosure, the state information of the heat source, the identification result of whether there is a person in the thermal imaging image, and the first distance and / or relative position (optionally including but not limited to any one or more of the following: stove type, cooking method, time information, determination result of whether the person is an adult, identification result of whether there is a pet in the thermal imaging image, fourth distance between the identified pet and the heat source, size and / or position of the kitchen wall, size and / or position of the kitchen door, size and / or position of the kitchen stove, detection result of the smoke sensor, etc.) can be regarded as the input information when the processor 110 performs functions such as "determining the kitchen scene", "determining the normal state or abnormal state", "determining the risk state", "generating a decision", and "sending control commands". The processor 110 performs the above functions based on the input information, which can mean performing the corresponding functions by understanding the input information (i.e., based on understanding the input information). In other words, the input information can be regarded as a necessary condition for performing understanding. That is, only when the above input information is obtained can the conditions for understanding be met, thereby achieving the ability to understand. Understanding the input information can be regarded as a reasoning process. For example, input information, prior knowledge and models, and the accumulation of temporal context can be combined for joint reasoning to understand the input information.

[0071] Determining the kitchen scene can refer to the processor 110 understanding (e.g., modeling) the kitchen scene based on information such as the state of the heat source, the identification result of whether there are people in the thermal imaging image, and the first distance and / or relative position (optionally including but not limited to any one or more of the following: stove type, cooking method, time information, determination result of whether the person is an adult, identification result of whether there are pets in the thermal imaging image, fourth distance between the identified pet and the heat source, size and / or position of the kitchen walls, size and / or position of the kitchen door, size and / or position of the kitchen stove, detection result of the smoke sensor, etc.). The output result obtained through understanding is the determined kitchen scene. Time information can refer to the time node information and / or duration information of some events identified based on temperature distribution data (such as ultra-high temperature events of the heat source, events where no people are identified, events where no people are identified within the observation range, etc.). The time node information and / or duration information can be calculated based on the timestamp or the data frame corresponding to the event combined with the frame rate.

[0072] Understanding a kitchen scene can refer to determining the current scene type and stage, and making a judgment on its development trend, based on information prior to the state time slice (such as the input information at the current moment mentioned above, and optionally, input information from a past period, the kitchen scene determined over a past period, normal or abnormal state, risk state, generated decisions, sent control commands, etc.). In other words, the determined kitchen scene includes the current scene type, its current stage, and its development trend. The determined kitchen scene is not a single state, but rather a combination of the current state, historical state changes, spatial relationships between objects, and temporal persistence. That is, a kitchen scene can be viewed as a combination of "state + process + contextual relationships."

[0073] Figure 3 This is a schematic diagram of a kitchen scene determined by understanding a kitchen scene based on some embodiments of this disclosure.

[0074] See Figure 3 Information prior to the state time slice can include historical scene information, kitchen heat source information, distance or location of people, duration, and time point. Historical scene information can include the scene determined prior to the state time slice and its development trend before the state time slice. Different scenes can have different initial risk levels, and the risk level of different scenes will change accordingly under different development trends.

[0075] Non-cooking heat source information can include abnormal heat sources and equipment heat dissipation. Cooking heat source information can include cooking type, stove type, cookware type, and heat source power. Cooking heat source information can be obtained by analyzing temperature distribution data or through other methods. For example, the heat source power in cooking heat source information can be obtained by user input.

[0076] Understanding and defining the kitchen scene can refer to identifying the current scene type and stage based on information prior to the state time slice and making a judgment on the development trend.

[0077] Furthermore, it should be noted that this disclosure can utilize any method capable of transforming input information into meaningful scene judgments to understand kitchen scenes. This method can be AI, or it can be rule modeling, state machines, mathematical models, time-series judgments, sensor fusion, etc., that can form "scene understanding capabilities".

[0078] In some embodiments, the processor 110 can input the state information of the heat source, the identification result of whether there is a person in the thermal imaging image, and a first distance and / or relative position (optionally including but not limited to any one or more of the following: stove type, cooking method, time information, determination result of whether the person is an adult, identification result of whether there is a pet in the thermal imaging image, a fourth distance between the identified pet and the heat source, the size and / or location of the kitchen wall, the size and / or location of the kitchen door, the detection result of the smoke sensor, etc.) into a pre-trained AI model to determine the kitchen scene. The AI ​​model can have the ability to understand the input information by combining prior knowledge to determine the kitchen scene.

[0079] In some optional but not necessarily preferred embodiments, multiple kitchen scenarios can be preset. Different kitchen scenarios correspond to different combinations of feature parameter values. The processor 110 can compare the currently determined combination of feature parameter values ​​(including the status information of the heat source, the identification result of whether there is a person in the thermal imaging image, and the first distance and / or relative position, optionally including but not limited to the stove type, cooking method, time information, the determination result of whether the person is an adult, the identification result of whether there is a pet in the thermal imaging image, the fourth distance between the identified pet and the heat source, the size and / or position of the kitchen wall, the size and / or position of the kitchen door, the detection result of the smoke sensor, etc.) with the preset multiple kitchen scenarios to determine the kitchen scenario that matches the current combination of feature parameter values.

[0080] For example, multiple kitchen scenarios can be set according to feature dimensions such as "whether someone is present," "first distance," "stove type + cooking method," "cooking development stage," and "duration." This allows the currently determined kitchen scenario to reflect different development stages of the cooking process, thereby determining the appropriate time for the user to return to the kitchen for intervention, avoiding excessive interference with the user, and ensuring that the cooking task can be completed continuously and safely while allowing the user to handle other tasks normally in a multi-tasking kitchen scenario. Specifically, the characteristic that different stove types and cooking methods have different thermal behavior characteristics at different stages can be utilized. The cooking development stage can be determined based on stove type, cooking method, and heat source status information (optionally including the first distance between the person and the heat source). For example, when a user performs a cooking task according to a recipe that includes target temperature, time information, and operating instructions, the first distance between the user and the heat source can be used to determine whether the relevant guided steps have been completed, thereby determining whether the user has completed the normal cooking task. For instance, if the recipe indicates that the user needs to operate for a certain period of time at a certain point in time, and the distance between the user and the heat source (i.e., the first distance) remains within the operating range during that time period, it can be determined that this step has been completed normally, and the user can proceed to the next cooking stage. Otherwise, it can be determined that this step has not been completed normally, and the user is currently in an abnormal cooking state, and a reminder will be issued.

[0081] For example, the determined kitchen scene can characterize the state of the cooking task currently in progress in the kitchen (such as the development stage of the cooking task, whether the cooking state is abnormal), whether the current state of the cooking task is the result of human manipulation, and whether the current state of the cooking task is perceived by a person. Taking a water-based cooking task as an example, the processor 110 can determine a kitchen scene that can characterize whether the water-based cooking task has boiled dry and the possible time of boiling dry, and whether the water-based cooking task is manipulated and / or perceived by a person, based on the state information of the heat source (such as the heating rate), the identification result of whether a person is present, and the first distance between the identified person and the heat source.

[0082] Determining a normal or abnormal state refers to determining whether the current state of the kitchen scene is abnormal. Taking a cooking scene as an example, a normal or abnormal state could refer to whether the current development stage of the cooking task is normal, i.e., whether it is in a normal cooking stage. Taking a non-cooking scene as an example, a normal or abnormal state could refer to whether the current development stage of a non-cooking task is normal. For example, determining a normal or abnormal state can refer to the processor 110 determining a normal or abnormal state based on the state information of the heat source, the identification result of whether there is a person in the thermal imaging image, and the first distance and / or relative position (optionally including but not limited to stove type, cooking method, time information, the determination result of whether the person is an adult, the identification result of whether there is a pet in the thermal imaging image, the fourth distance between the identified pet and the heat source, the size and / or position of the kitchen wall, the size and / or position of the kitchen door, the detection result of the smoke sensor, etc.).

[0083] Risk status can characterize whether a risk exists in the kitchen, the type of risk, and the degree of risk. For example, determining the risk status can refer to the processor 110 determining whether a risk exists in the kitchen and the specific type of risk based on information such as the state of the heat source, the identification result of whether a person is present in the thermal imaging image, and the first distance and / or relative position between the identified person and the heat source (optionally including but not limited to any one or more of the following: stove type, cooking method, time information, determination of whether the person is an adult, identification result of whether a pet is present in the thermal imaging image, fourth distance between the identified pet and the heat source, size and / or location of the kitchen walls, size and / or location of the kitchen door, and detection results from the smoke sensor). For example, determining the risk status can also refer to the processor 110 determining the risk status based on a defined kitchen scenario. As mentioned above, the defined kitchen scenario can be considered as the result of understanding the input information; therefore, determining the risk status can also be considered as determining the risk status based on the output result of understanding the input information.

[0084] In some embodiments, the processor 110 can input the state information of the heat source, the identification result of whether there is a person in the thermal imaging image, and the first distance and / or relative position between the identified person and the heat source (optionally including but not limited to any one or more of the following: stove type, cooking method, time information, determination result of whether the person is an adult, identification result of whether there is a pet in the thermal imaging image, fourth distance between the identified pet and the heat source, size and / or location of the kitchen wall, size and / or location of the kitchen door, detection result of the smoke sensor, etc.) into a pre-trained AI model to determine the risk state. The AI ​​model can have the ability to understand the input information by combining prior knowledge and determine the risk state based on the understanding result.

[0085] In some optional, but not necessarily preferred, embodiments, multiple risk states can be preset. Different risk states correspond to different combinations of characteristic parameter values. The processor 110 can compare the currently determined combination of characteristic parameter values ​​(including the status information of the heat source, the identification result of whether there is a person in the thermal imaging image, and the first distance and / or relative position, optionally including but not limited to the stove type, cooking method, time information, the determination result of whether the person is an adult, the identification result of whether there is a pet in the thermal imaging image, the fourth distance between the identified pet and the heat source, the size and / or position of the kitchen wall, the size and / or position of the kitchen door, the detection result of the smoke sensor, etc.) with the preset multiple risk states to determine the risk state that matches the current combination of characteristic parameter values.

[0086] In some exemplary embodiments, the risk state can be divided into a risk state when someone is present and a risk state when no one is present. A risk state when someone is present refers to the risk state when someone is detected in the kitchen. A risk state when no one is present can include the risk state when no one is detected in the kitchen. A risk state when someone is present can be further divided into a risk state when someone is present within the operating range, a risk state when someone is present within the observation range, and a risk state when someone is present outside the observation range. A risk state when someone is present within the observation range can be further subdivided into a risk state when someone is present near the observation range and a risk state when someone is present far from the observation range. The processor 110 can first determine the current risk state category from the above-mentioned multiple risk states based on the identification results of whether there is a person in the thermal imaging image and the first distance between the identified person and the heat source. Then, based on the status information of the heat source (optionally including but not limited to any one or more of the following information: stove type, cooking method, time information, determination result of whether the person is an adult, identification result of whether there is a pet in the thermal imaging image, fourth distance between the identified pet and the heat source, relative position of the heat source, relative position of the person, size and / or position of the kitchen wall, size and / or position of the kitchen door, detection result of the smoke sensor, etc.), determine the specific risk state under the current risk state category.

[0087] For example, an abnormal risk state may include, but is not limited to, at least one of the following: fire state, abnormally high temperature (such as dry burning), abnormally high temperature movement, high temperature sudden spread without human intervention (indicating a potential fire risk), short-term unmanned sudden cooling (such as abnormal shutdown of a stove), short-term unmanned sudden heating, long-term unmanned sudden cooling, and long-term unmanned sudden heating.

[0088] Decision generation refers to determining the current course of action. This course of action can be a specific action (such as alarm activation, fire extinguishing, emergency lighting, etc.) or intentionally not taking any action. That is, not taking an action (such as not taking an action under normal circumstances) can also be considered a decision.

[0089] In some embodiments, the processor 110 can input the state information of the heat source, the identification result of whether there is a person in the thermal imaging image, and the first distance and / or relative position between the identified person and the heat source (optionally including but not limited to any one or more of the following: stove type, cooking method, time information, determination result of whether the person is an adult, identification result of whether there is a pet in the thermal imaging image, fourth distance between the identified pet and the heat source, size and / or location of the kitchen wall, size and / or location of the kitchen door, detection result of the smoke sensor, etc.) into a pre-trained AI model, and use the AI ​​model to generate a decision. The AI ​​model can have the ability to understand the input information by combining prior knowledge and generate a decision based on the understanding result.

[0090] In some optional but not necessarily preferred embodiments, multiple decisions can be preset. Different decisions correspond to different combinations of feature parameter values. The processor 110 can compare the currently determined combination of feature parameter values ​​(including the state information of the heat source, the identification result of whether there is a person in the thermal imaging image, and the first distance and / or relative position between the identified person and the heat source, optionally including but not limited to any one or more of the following: stove type, cooking method, time information, determination result of whether the person is an adult, identification result of whether there is a pet in the thermal imaging image, fourth distance between the identified pet and the heat source, size and / or position of the kitchen wall, size and / or position of the kitchen door, detection result of the smoke sensor, etc.) with the preset multiple decisions to determine the decision that matches the current combination of feature parameter values.

[0091] Sending control commands refers to sending control commands to the corresponding actuators. In some embodiments, the processor 110 sending control commands may refer to sending control commands to the corresponding actuators based on the kitchen scene, and / or the determined normal or abnormal state, and / or risk state, and / or decision. In other embodiments, the processor 110 sending control commands may refer to directly sending control commands to the corresponding actuators based on the state information of the heat source, the identification result of whether there is a person in the thermal imaging image, and the first distance and / or relative position between the identified person and the heat source (optionally including but not limited to any one or more of the following information: stove type, cooking method, time information, determination result of whether the person is an adult, identification result of whether there is a pet in the thermal imaging image, fourth distance between the identified pet and the heat source, size and / or position of the kitchen wall, size and / or position of the kitchen door, detection result of the smoke sensor, etc.), thus eliminating the intermediate analysis steps such as determining the kitchen scene, determining the normal or abnormal state, determining the risk state, and generating a decision. For example, a linkage table can be pre-designed, which records the trigger conditions corresponding to different control commands. Based on the status information of the heat source, the identification result of whether there is a person in the thermal imaging image, and the first distance and / or relative position between the identified person and the heat source (optionally including but not limited to any one or more of the following information: stove type, cooking method, time information, determination result of whether the person is an adult, identification result of whether there is a pet in the thermal imaging image, fourth distance between the identified pet and the heat source, size and / or position of the kitchen wall, size and / or position of the kitchen door, detection result of the smoke sensor, etc.), the processor 110 can directly determine the control command to be sent by looking up the table.

[0092] In some embodiments, the processor 110 may utilize an AI model to perform at least one of the following functions: determining a kitchen scene; determining a normal or abnormal state; determining a risk state; generating a decision; and sending control commands. Specifically, the AI ​​model may input information such as the state of the heat source, the identification result of whether a person is present in the thermal imaging image, and the first distance and / or relative position between the identified person and the heat source (optionally including but not limited to any one or more of the following: stove type, cooking method, time information, determination result of whether the person is an adult, identification result of whether a pet is present in the thermal imaging image, fourth distance between the identified pet and the heat source, size and / or location of the kitchen wall, size and / or location of the kitchen door, detection result of the smoke sensor, etc.). The AI ​​model then analyzes (i.e. understands) the input to determine the kitchen scene, and / or determine a normal or abnormal state, and / or determine a risk state, and / or generate a decision, and / or send control commands.

[0093] It should be understood that different types of kitchen scenarios may have different levels of risk in their initial stages. Even if the same type of kitchen scenario is in a normal state at different stages of development, these stages of development may have different levels of risk.

[0094] Figure 4 This is a schematic diagram illustrating the development trend of two typical kitchen scenarios over time according to some embodiments of this disclosure.

[0095] See Figure 4 Under normal circumstances, a cooking task scenario M can include stages such as "preparing ingredients," "heating over high heat," "heating over low heat," "observing the food," "turning off the heat," and "completing / removing the food." The risk level of the ingredient preparation stage is generally low, while the high-heat stage has a relatively high risk level. That is, even if both the ingredient preparation and high-heat stages are in a normal state, the risk level of the high-heat stage (represented by the vertical axis in the diagram) will be higher than that of the ingredient preparation stage. Different cooking stages can develop into different abnormal states. For example, the high-heat stage may develop into the "fire" abnormal state. During the low-heat stage, if someone leaves the stove, it may develop into the "short-term forgetting to check the fire" abnormal state. During the food observation stage, if the heat source is not turned off for an extended period, it may develop into the "long-term forgetting to turn off the heat" abnormal state. Both abnormal and normal states have corresponding risk levels.

[0096] Under normal circumstances, non-cooking task scenario K can include two task phases: "refrigerator heating normally" and "temperature fluctuating gently." Even if both phases are functioning normally, the risk level in the "temperature fluctuating gently" phase is slightly higher than in the "refrigerator heating normally" phase. In the "refrigerator heating normally" phase, the temperature may rise sharply, which is an abnormal situation.

[0097] The processor can obtain output information such as the scenario, whether the scenario state is normal, the risk level, decisions, and instructions based on the input information before the state time slice. For details on the input information before the state time slice, please refer to the relevant description above. For example, the processor can generate corresponding decisions or instructions with the goal of ensuring task completion or maintaining a normal state. Instructions are the control commands mentioned above.

[0098] Figure 5 This is a diagram illustrating the differences between this disclosure and conventional solutions in terms of information processing.

[0099] See Figure 5There are two main conventional solutions. One is to control the cooking equipment based on its own sensing results, such as triggering an alarm sound when the timer in the stove exceeds a threshold. This method can only achieve specific sensing and has zero understanding. The other is to control the cooking area based on the detection results of the cooking area (such as the temperature detection results of a specific stove type, or the detection results of the presence of a person). This method can only achieve area sensing.

[0100] This disclosure achieves advanced levels of understanding—scene-level, prediction-level, decision-level, and generalization-level—based on detection results across the entire kitchen space (including but not limited to cooking heat sources, non-cooking heat sources, distance between people and heat sources, relative positions of people and / or heat sources, and smoke detector detection results). Scene-level understanding refers to determining the current scene type and corresponding scene state based on perception and user input, and making judgments and / or decisions and / or commands based on the scene state. Prediction-level understanding refers to predicting short-term state changes and medium- to long-term task state machines based on known scene types and current scene states, and making judgments and / or decisions and / or commands based on these predictions. Decision-level understanding refers to making decisions based on normal / abnormal states and / or risk levels to achieve dynamic state control and risk control, ensuring task completion or maintenance of a normal state. Generalization-level understanding refers to being able to handle unknown scenes and / or states based on existing experience combined with learning capabilities.

[0101] The following provides further illustrative examples of the details of various aspects covered in this disclosure.

[0102] Heat source type and cooking method Heat sources in the kitchen can be categorized into cooking heat sources and non-cooking heat sources. Cooking heat sources can be further divided into stove heat sources and cookware heat sources. Non-cooking heat sources can be categorized into heat dissipation equipment heat sources (such as refrigerators) and abnormal heat sources (such as fire heat sources). Stove heat sources can be further subdivided into various types such as gas stoves, electric stoves / heating wires, induction cookers, ovens, and barbecue grills.

[0103] The following explanation uses common kitchen heat sources such as stoves and fire sources as examples.

[0104] The common thermal behavior characteristics of normal cooking (whether water-based or non-water-based) are that the heat source of the stove is spatially constrained by the geometry of the cookware, and exhibits a continuous and predictable evolution process in time. In thermal imaging, this manifests as a regular or semi-regular thermal pattern, with relatively clear boundaries of the thermal area, and gradual changes in temperature and area.

[0105] In contrast, fire heat sources exhibit significantly different thermophysical characteristics (also known as thermal characteristics or thermal behavior characteristics). During a fire, due to the combustion of combustibles and the involvement of high-temperature burning particles, thermal radiation is no longer confined to the original geometric structure. In thermal imaging, high-temperature regions diffuse irregularly into the surrounding space and grow rapidly over a short period. This process is typically accompanied by significantly higher peak temperatures, severe spatial inhomogeneity, and nonlinear temporal evolution characteristics. In a fire, the high-temperature region (i.e., the hot spot region corresponding to the heat source) is no longer confined to the original geometric structure, and its area grows rapidly and irregularly over time.

[0106] Due to the inherent limitation of traditional smoke / temperature sensors in distinguishing between cooking and fire, kitchens are often in a blind spot for fire early warning detection, especially in the early stages. This disclosure addresses this issue by differentiating between fire heat sources and stove heat sources based on their thermal behavior. For example, a stove heat source during normal cooking can be distinguished from an extremely abnormal fire heat source by comprehensively considering the spatial constraints of the heat source, the pattern of heat diffusion, and the rate of change over time.

[0107] Furthermore, there is a transitional phase between normal cooking and a fire (this transitional phase is an abnormal state), and the thermal behavior of this transitional phase differs from that of normal cooking and a fire. This disclosure can also identify the transitional phase between normal cooking and a fire based on this difference (e.g., based on scene understanding), enabling identification and risk escalation in the early stages of a fire. Therefore, this disclosure can achieve early fire detection without relying on the visibility of smoke or open flames.

[0108] Under normal cooking conditions, the heat source of the stove is spatially constrained by the geometry of the cookware, resulting in a regular or semi-regular thermal distribution. By detecting whether hot spots are consistently confined to a predetermined spatial area, it is possible to distinguish between normal cooking and abnormal heat diffusion. For example, it can effectively differentiate between confined heat sources inside the pot and abnormal external heat sources, suppressing interference from normal high temperatures such as those at the edge of the gas flame.

[0109] Different types of stoves have different thermal characteristics. For example, gas stoves have open flame radiation heat, heat up quickly, and have a heat halo around the stove; electric stoves / heating wires have no open flame, heat up slowly, and have concentrated heat zones; induction cookers heat the pot directly, and the cooktop itself does not get hot; ovens are enclosed heating systems with periodic temperature control; and barbecue grills have open flames and are characterized by high temperatures and uneven heating.

[0110] The thermal imaging module 120 observes differences depending on the type of stove heat source. For example, when a gas stove is the heat source, the edges of the heat source are blurred and the area is large; when an induction cooker is the heat source, only the hot spot area of ​​the pot on the induction cooker can be observed, without any heat halo on the cooktop. When an oven is the heat source, the top observation may only show heat from the exhaust vent; when a grill is the heat source, the heat source has a large area of ​​high temperature and an irregular shape.

[0111] Different cooking methods have different thermal characteristics. Cooking methods can include, but are not limited to, water-based cooking, frying / stir-frying, grilling / roasting, baking, keeping warm / slow cooking, and boiling water. Water-based cooking can include, for example, boiling, stewing, and steaming. Alternatively, boiling water can also be considered a special type of water-based cooking.

[0112] In water-based cooking, where water or water-containing media are the main components, thermal behavior is strongly constrained by the thermophysical properties of water. Water has a high specific heat capacity and latent heat of vaporization, and its heating process usually presents two stages: the first stage is the water temperature rise stage, where the water temperature gradually increases over time, and the rate of temperature rise is inversely proportional to the heating power and the amount of water; the second stage is the boiling and evaporation stage, in which the surface temperature of the water is limited to a range close to the boiling point, and the heat is mainly used for phase change rather than for further temperature increase. Therefore, in water-based cooking, the temperature field observed by thermal imaging usually has the following characteristics: (1) the surface temperature has a stable physical upper limit; (2) the spatial distribution is relatively uniform, and the temperature gradient is small; (3) the time evolution is gradual, with a characteristic time scale of minutes. When the amount of water gradually decreases until it is exhausted, this physical constraint will suddenly disappear, resulting in a significant increase in the rate of temperature rise and spatial non-uniformity, forming obvious state change characteristics. The cooking process of frying, grilling, and other dry cooking methods is not constrained by the phase change of water, and its thermal behavior exhibits characteristics that are significantly different from those of the water-based model. Specifically, during frying or oil-based cooking, the temperature of the heating medium can rise continuously and lacks a fixed temperature clamp, which is usually manifested in thermal imaging as: (1) higher peak temperature; (2) obvious spatial non-uniformity; and (3) rapid heating within a short time scale. In dry cooking scenarios such as grilling or direct grilling, the heat source itself is already at a high temperature, and the heat distribution is often concentrated in a limited area, but the temperature level and spatial gradient are significantly higher than those of water-based cooking.

[0113] To give further examples, the temperature range for water-based cooking is typically 90°C to 110°C, with a relatively long duration; the temperature range for frying / stir-frying is typically 160°C to 280°C, with a shorter or medium duration; the temperature range for grilling / roasting is typically 200°C to 350°C, with a medium or long duration; the temperature range for baking is typically 150°C to 250°C, with a relatively long duration; the temperature range for keeping warm / slow cooking is typically 60°C to 90°C, with an extremely long duration; and the temperature range for boiling water is typically from room temperature to 100°C, with a relatively short duration.

[0114] By analyzing the temperature upper limit, heating rate, spatial distribution characteristics and temporal consistency of the hot spot area corresponding to the heat source, water-based cooking and non-water-based cooking scenarios can be distinguished without relying on specific kitchen utensils or user operation information, and the corresponding risk assessment threshold can be selected accordingly, rather than using a uniform threshold for all cooking scenarios.

[0115] The thermal behavior varies greatly depending on the combination of "stove type + cooking method". For example, when stir-frying with a gas stove, the heating rate is extremely fast (e.g., greater than 5°C / s), and the steady-state temperature is generally between 200°C and 280°C. Due to the stirring motion, the temperature fluctuates greatly. When simmering soup with a gas stove, the heating rate is medium, and the steady-state temperature is generally between 95°C and 105°C, with very little temperature fluctuation. When frying with an electric stove / heating wire, the heating rate is slower (e.g., 1°C / s to 2°C / s), and the steady-state temperature is generally between 160°C and 220°C, with moderate temperature fluctuation.

[0116] Different types of stoves, different types of cooking methods, and different combinations of "stove type + cooking method" all have different thermal differences. Therefore, when the processor 110 determines the kitchen scene, the normal or abnormal state, the risk state, generates decisions, and sends control commands, if it does not distinguish between stove type and / or cooking method and instead uses a uniform threshold or threshold set, it will either produce false alarms (i.e., not being able to identify all risks) because the threshold setting is too tight (i.e., the threshold range is too small), or produce false alarms (such as identifying normal cooking state as risk) because the threshold setting is too lenient (i.e., the threshold range is too large).

[0117] Therefore, in some embodiments, the processor 110 may first determine the heat source type and / or cooking method, and then, based on the heat source type and / or cooking method, the state information of the heat source, the identification result of whether there is a person in the thermal imaging image, and the first distance and / or relative position between the identified person and the heat source (optionally including but not limited to time information, the determination result of whether the person is an adult, the identification result of whether there is a pet in the thermal imaging image, the fourth distance between the identified pet and the heat source, the size and / or position of the kitchen wall, the size and / or position of the kitchen door, the detection result of the smoke sensor, etc.), perform at least one of the following functions: determine the kitchen scene; determine the normal state or abnormal state; determine the risk state; generate a decision; and send a control command. It should be understood that the heat source type and / or cooking method can also be considered a special heat source state. That is, the heat source type and / or cooking method can be attributed to the state information of the heat source; that is, the state information of the heat source can include the heat source type and / or cooking method.

[0118] The processor 110 can identify the heat source type and / or cooking method based on the state information of the heat source and / or temperature distribution data of the hot spot area corresponding to the heat source. The processor 110 can also acquire the heat source type and / or cooking method from external input (such as user input). That is, the heat source type and cooking method can be determined through external input or through analysis. For example, first distance information can be used to assist in identifying the heat source type and / or cooking method. For instance, a person being within the operating range for an extended period means that the current cooking method requires frequent operation (such as frying). Therefore, when determining the heat source type and / or cooking method through analysis, the processor 110 can also incorporate the first distance information.

[0119] As mentioned above, different types of stoves, different types of cooking methods, and different combinations of "stove type + cooking method" all have different thermal characteristics. Therefore, in some optional but not necessarily preferred exemplary embodiments, the processor 110 can compare the first thermal characteristics (such as upper temperature limit, heating rate, spatial distribution characteristics, etc.) represented by the state information of the heat source with the second thermal characteristics of various types of stoves, various types of cooking methods, and various combinations of "stove type + cooking method," and identify the stove and cooking method corresponding to the second thermal characteristic that is closest to the first thermal characteristic as the identified heat source type and cooking method. Considering the complexity of the spatiotemporal relationships of these combinations, only the main combinations can be classified, and it is extremely difficult to exhaustively classify all related combinations (almost infinite). Therefore, it cannot be solved by mechanical rules, etc. So, in some exemplary embodiments, the processor 110 can also input the state information of the heat source into the AI ​​model and use the AI ​​model to identify the heat source type and cooking method. Thus, by utilizing the information understanding capability of the AI ​​model, the ability to identify all possible combinations can be achieved. The identification of stove type and / or cooking method does not rely on a single temperature threshold as a judgment condition, but is based on a comprehensive judgment on the inherent thermal behavior characteristics of different stove types and / or cooking methods, thus improving the accuracy of the identification results.

[0120] The first distance calculation method In a kitchen setting, the horizontal distance between a person and a heat source is more accurate than the straight-line distance (which is typically a non-horizontal, inclined distance) in reflecting the relationship between the heat source and the person in terms of operation and perception. For example, the horizontal distance better reflects whether the heat source can be operated by a person and / or whether the state information of the heat source is perceived by a person. Therefore, the first distance can include the horizontal distance between the person and the heat source. For example, the processor 110 can calculate the identified horizontal distance between the person and the heat source and use the calculated horizontal distance as the first distance. It should be understood that in some embodiments, the processor 110 can also calculate the identified straight-line distance between the person and the heat source and use the calculated straight-line distance as the first distance.

[0121] Heat sources mainly include stove heat sources and fire heat sources. Stove heat sources are typically around 1 meter high, and the center of a person's height is approximately at the same height as the stove. The source of a fire usually originates from the stove. That is, both stove heat sources and fire heat sources can be approximated as being at the same height as the center of a person's height, and this approximation error is acceptable in a kitchen setting. Therefore, the horizontal plane used to calculate horizontal distances can refer to either the horizontal plane where the heat source is located or the horizontal plane where the center of a person's height is located.

[0122] The following describes an exemplary calculation method for the first distance, taking the horizontal distance as an example. It should be understood that a similar calculation method can be used for non-horizontal distances (such as straight-line distances).

[0123] A reference pixel refers to a pixel whose optical axis is substantially perpendicular to the horizontal direction. Ideally (without considering factors such as adjustment tolerances, lens distortion, and installation deviations), when the optical axis of the thermal imaging module 120 is perpendicular to the horizontal direction, the reference pixel can refer to the center pixel. When the optical axis of the thermal imaging module 120 is not perpendicular to the horizontal direction, the reference pixel is no longer the center pixel. In this disclosure, the reference pixel is known by default. The angle between the optical axis of the thermal imaging module 120 and the horizontal plane can be determined based on the reference pixel. The reference pixel can also be determined based on the angle between the optical axis of the thermal imaging module 120 and the horizontal plane. Therefore, the reference pixel being known by default is equivalent to the angle between the optical axis of the thermal imaging module 120 and the horizontal plane being known by default. For example, the reference pixel can be determined based on the angle between the optical axis of the thermal imaging module 120 and the horizontal direction, or it can be determined by other means (such as a test chart after the thermal imaging module 120 is installed).

[0124] In this disclosure, the mapping principle of the thermal imaging module 120 is also assumed to be known. The mapping principle of the thermal imaging module 120 characterizes the mapping relationship between an object in the object plane and an image object in the image plane. The internal parameters (i.e., intrinsic parameters) of the thermal imaging module 120, such as the focal length, the numerical relationship between the object angle and the image angle, can be regarded as a parameterized expression of the mapping principle of the thermal imaging module 120. That is, the mapping principle of the thermal imaging module 120 "includes" the intrinsic parameters of the thermal imaging module 120. The intrinsic parameters of the thermal imaging module 120 are known parameters that are set at the factory.

[0125] In some embodiments, when the height of the thermal imaging module 120 above the ground (i.e., the installation height) and the height of the heat source are both known, the processor 110 can calculate the horizontal distance between the person and the heat source based on the installation height, reference pixels, the mapping principle of the thermal imaging module 120, and the height of the heat source. The height of the heat source can be determined through, but is not limited to, external input or estimation. For example, the user-inputted heat source height information can be obtained, or a preset heat source height (e.g., 80 cm to 85 cm) can be used. The height of the heat source can refer to the height of the stovetop (i.e., the height of the kitchen countertop).

[0126] Figure 6 This is a schematic diagram of calculating a first distance according to some embodiments of the present disclosure.

[0127] See Figure 6The first distance can refer to the distance between a person and the heat source on the horizontal plane where the heat source is located. Parameters required to calculate the first distance may include the installation height H, the heat source height h, the mapping principle of the thermal imaging module 120, and the angle between the optical axis of the thermal imaging module 120 and the horizontal plane. For example... Figure 6 As shown, the optical axis of the thermal imaging module 120 can be tilted. Based on the angle between the optical axis of the thermal imaging module 120 and the horizontal plane, the reference pixel A in the image plane of the thermal imaging module 120 can be determined.

[0128] After determining the reference pixel A, the first image angle α between pixel B in the first pixel region (whose temperature value differs from the background temperature and is identified as a person) and the reference pixel A is known. The second image angle β between pixel C in the second pixel region (whose temperature value differs from the background temperature and is identified as a heat source) and the reference pixel A is also known. Based on the mapping principle of the thermal imaging module 120 (e.g., the numerical relationship between image angle and object angle), the first object angle γ corresponding to the first image angle α and the second object angle θ corresponding to the second image angle β can be determined. If the thermal imaging module 120 is a linear camera, then the first object angle γ equals the first image angle α, and the second object angle θ equals the second image angle β; if the thermal imaging module 120 is a nonlinear camera, then the object angle corresponding to the image angle can be determined according to the nonlinear mapping relationship of the thermal imaging module 120 (i.e., the numerical relationship between image angle and object angle). Pixel B can be, for example, the center pixel of the first pixel region, and pixel C can be, for example, the center pixel of the second pixel region.

[0129] Based on the first object angle γ, the second object angle θ, the installation height H, and the heat source height h, the distance between the person and the heat source on the horizontal plane at the heat source height (i.e., the first distance) can be calculated using simple trigonometric functions. For example, based on the installation height H and the heat source height h, the vertical distance (Hh) between the person and the thermal imaging module on the horizontal plane at the heat source height can be determined. Based on the first object angle γ and the vertical distance (Hh) between the person and the thermal imaging module on the horizontal plane at the heat source height, the horizontal distance between the person and the thermal imaging module on the horizontal plane at the heat source height can be calculated using trigonometric functions. Similarly, based on the installation height H and the heat source height h, the vertical distance (Hh) between the heat source and the thermal imaging module can be determined. Based on the second object angle θ and the vertical distance (Hh) between the heat source and the thermal imaging module, the horizontal distance between the person and the heat source on the horizontal plane at the heat source height can be obtained.

[0130] In some embodiments, the installation height of the thermal imaging module 120 may be unknown, and the processor 110 may calculate the horizontal distance between the person and the heat source based on the first size information of a standard human body and the mapping principle of the thermal imaging module.

[0131] When calculating the horizontal distance between a person and a heat source, if the center of the person's height is assumed to be at the same height as the heat source, then the height of the heat source is not needed. Conversely, if the center of the person's height is assumed to be at a different height from the heat source, then the height of the heat source needs to be considered when calculating the horizontal distance between the person and the heat source to improve the accuracy of the calculation results.

[0132] A standard human body is a human model with known dimensions. The dimensions of a standard human body can be used as the dimensions of people identified within the kitchen. The initial dimensions of the standard human body can be set at the factory. Alternatively, the initial dimensions can be set by the user to more accurately represent the dimensions of a real user. These initial dimensions may include, but are not limited to, shoulder width, height, and arm length.

[0133] In some exemplary embodiments, the processor 110 can determine the proportional relationship between pixel size and physical size in real space based on first size information of a standard human body and second size information of a first pixel region whose temperature value in the image plane differs from the background temperature and is identified as a person. The proportional relationship between pixel size and physical size in real space can refer to the physical size of a single pixel in the pixel space corresponding to a horizontal plane in the real space. This proportional relationship reflects the physical size of a single pixel in the pixel space corresponding to a physical size in the real space, and the processor 110 can use this proportional relationship to calculate the horizontal distance between the person and the heat source.

[0134] Figure 7 This is a schematic diagram of calculating a first distance according to other embodiments of this disclosure.

[0135] See Figure 7When the optical axis of the thermal imaging module 120 is not perpendicular to the horizontal direction, the image plane has a certain angle with the horizontal plane (this angle is the angle between the optical axis of the thermal imaging module 120 and the horizontal direction). In this case, a virtual image plane parallel to the horizontal plane and passing through reference pixel A can be constructed. Pixel D is the pixel corresponding to the human head within the first pixel region, and pixel D' is the intersection point of the extension lines of pixel D in the virtual image plane. Based on the angle between the optical axis of the thermal imaging module 120 and the horizontal plane, the image angle between pixel D and reference pixel A, and the distance between pixel D and reference pixel A, the distance between pixel D' and reference pixel A can be determined. Pixel B is the pixel corresponding to the human foot within the first pixel region, and pixel B' is the intersection point of the extension lines of pixel B in the virtual image plane. The image angle between pixel B and reference pixel A is known. Based on the image angle between pixel B and reference pixel A and the mapping principle of the thermal imaging module 120, the object angle corresponding to this image angle can be determined. The distance between pixel D' and pixel B' can be calculated. Based on the object angle corresponding to the image angle between pixel B and reference pixel A, and the height of a standard human body, the physical dimensions of the human head on the horizontal plane (i.e., the horizontal distance between points E and F in the figure) can be obtained through trigonometric function calculations. This allows us to obtain the proportional relationship between the pixel dimensions of pixel D' and pixel B' in the virtual image plane and the physical dimensions of the human head on the horizontal plane. This proportional relationship characterizes the correspondence between the pixel dimensions in the virtual image plane parallel to the horizontal plane and the physical dimensions of the human head on the horizontal plane. Based on this proportional relationship and the pixel dimensions between pixel D' and reference pixel A, the horizontal distance between the human head and the thermal imaging module 120 can be determined. After obtaining this horizontal distance, the horizontal distance between the person and the heat source can be calculated. For example, based on the horizontal distance between the head and the thermal imaging module 120 and the object angle between the head and reference pixel A, the vertical distance between the head and the thermal imaging module 120 can be calculated. Adding this vertical distance to the height of a standard human body gives the installation height of the thermal imaging module 120. After obtaining the installation height of the thermal imaging module 120, the installation height can be calculated according to, for example... Figure 6 The method shown calculates the horizontal distance between a person and a heat source. Half the height of the person can be used as the height of the heat source. It should be understood that, given the first dimensions of a standard human body, other methods can also be used to calculate the horizontal distance between the person and the heat source, and this disclosure does not limit such methods.

[0136] In some embodiments, the sensing device 100 further includes a ranging module. The ranging module is used to measure a second distance between the thermal imaging module 120 and a person. The ranging module may employ, but is not limited to, a radar sensor. The ranging module may be positioned close to the thermal imaging module 120 such that the distance measured by the ranging module between the person and the ranging module can be used as the second distance between the person and the thermal imaging module 120. Exemplarily, the distance between the person and the ranging module may also be corrected based on the positional relationship between the ranging module and the thermal imaging module 120, and the corrected distance may be used as the second distance.

[0137] The processor 110 can calculate the horizontal distance between a person and a heat source based on the second distance and the mapping principle of the thermal imaging module 120. Therefore, even when the installation height of the thermal imaging module 120 is unknown, the horizontal distance between the person and the heat source can be calculated using the second distance measured by the ranging module 120.

[0138] Taking the second distance, which includes the distance between the person's head and the thermal imaging module 120, as an example, see [link to example]. Figure 7 Similarly, a virtual image plane can be constructed. Based on the image angle between pixel D corresponding to the human head and reference pixel A within the first pixel region, and the mapping principle of the thermal imaging module 120 (the numerical relationship between the image angle and the object angle), the object angle between the head and the thermal imaging module can be determined. Based on the distance between the head and the thermal imaging module 120 and the object angle between them, the vertical distance and the horizontal distance between the head and the thermal imaging module 120 can be calculated. Based on the internal parameters of the thermal imaging module 120, the distance between pixel D' and reference pixel A can be calculated. Based on the distance between pixel D' and reference pixel A, and the horizontal distance between the head and the thermal imaging module 120, the proportional relationship between the pixel size in the virtual image plane parallel to the horizontal plane and the physical size on the horizontal plane where the human head is located can be obtained. Based on this proportional relationship and the pixel dimensions between pixel D' and pixel B', the corresponding physical dimensions (i.e., the horizontal distance EF between points E and F) can be determined. Since the horizontal distance between the head and the thermal imaging module 120 (i.e., the horizontal distance between points O and F) is known, the horizontal distance OE between points O and E is also known. Following the proportional relationship of similar triangles, and based on OE, EF, and the vertical distance between the head and the thermal imaging module 120, the height can be calculated. The vertical distance between the head and the thermal imaging module plus the height equals the installation height of the thermal imaging module 120. After obtaining the installation height of the thermal imaging module 120, for example, the above combination can be used... Figure 6The described calculation method calculates the horizontal distance between a person and a heat source. It should be understood that, given that the second distance between the person and the thermal imaging module 120 is known, other methods can also be used to calculate the horizontal distance between the person and the heat source, and this disclosure does not limit this method.

[0139] Due to design, manufacturing, or application defects or limitations of the thermal imaging module 120, the radiation temperature of the target object measured by the thermal imaging module 120 may vary with the distance between the target object and the thermal imaging module. Reasons for this phenomenon may include, for example, that the object size is smaller than the pixel IFOV, resulting in insufficient heat collection, or that the thermal imaging module 120 is biased towards focusing at close range, failing to focus on distant objects, thus also resulting in insufficient heat collection.

[0140] Based on this, this disclosure proposes to utilize this temperature measurement deficiency of the thermal imaging module 120 to achieve distance measurement. In other words, distance measurement can be achieved using the existing processor 110 and thermal imaging module 120 without setting up a separate distance measurement module. That is, the distance measurement module includes the processor 110 and the thermal imaging module 120.

[0141] The processor 120 can determine a second distance between the thermal imaging module 120 and the person based on the temperature value of the first pixel area and pre-acquired calibration data. The calibration data characterizes the correspondence between the radiation temperature obtained by the thermal imaging module from measuring the human body at different distances and the distance itself. The human head (especially the face), due to its rich blood vessels and exposed skin, is usually warmer than the torso and limbs covered by clothing; therefore, the human head is often a prominent hotspot in thermal imaging. Thus, in an exemplary embodiment, the second distance between the thermal imaging module 120 and the human head can be determined based on the temperature value of the pixel portion corresponding to the human head within the first pixel area and pre-acquired calibration data. Optionally, the distance between other parts of the human body (e.g., the feet) and the thermal imaging module 120 can also be obtained.

[0142] In some exemplary embodiments, the processor 110 can also calculate a third size information of the person based on the second distance and the first pixel region, and verify whether the type of the imaged object in the first pixel region is a person based on the third size information. The third size information may include, but is not limited to, information that reflects the body size characteristics of a person, such as height and width (e.g., shoulder width). This disclosure does not limit the specific calculation method of the third size information. For example, the third size information can be calculated in conjunction with the mapping principle of the thermal imaging module 120. If the third size information differs significantly from the size characteristics of an ordinary person, it can be considered that the type identification of the imaged object in the first pixel region is incorrect, for example, it may be a non-human life form such as a pet.

[0143] In some embodiments, a radar sensor can be used for ranging. That is, the ranging module may include a radar sensor. For example, a high-frequency radar sensor that supports identifying whether a reflecting object is a person can be used (e.g., a millimeter-wave radar using a specific frequency band of millimeter waves). In other words, the radar sensor can detect whether the type of reflecting object in the detection area is a person and detect a third distance between the identified person and the radar sensor. The third distance can be used as a second distance between the person and the thermal imaging module 120, or the third distance can be corrected to obtain the second distance based on the relative positional relationship between the radar sensor and the thermal imaging module 120. The basic principle of radar detection of whether a target is a person is usually based on the detection of motion characteristics and micro-motion characteristics. Motion can be perceived through continuous changes in the target distance; while for a stationary human body, it is determined by the micro-motions on the body surface caused by physiological activities such as breathing or heartbeat.

[0144] The processor 110 can detect whether a target pixel region exists within the pixel region corresponding to the second distance in the image plane. When the distance data detected by the radar sensor lacks azimuth information, the pixel region corresponding to the second distance in the image plane can be a "ring-shaped region" centered on a reference pixel. The pixel distance between pixels in this ring-shaped region and the reference pixel can be considered as the image distance corresponding to the object distance represented by the second distance. The target pixel region is a pixel region whose type of imaging object, determined based on temperature distribution data, may be human (as mentioned above regarding the first pixel region). If a target pixel region exists within the pixel region corresponding to the second distance in the image plane, the type of imaging object in the target pixel region can be determined to be human. Conversely, if the target pixel region is not within the pixel region corresponding to the second distance in the image plane, the type of imaging object in the target pixel region can be determined to be non-human.

[0145] On the one hand, thermal patches generated by heat sources such as heat halos or thermal backgrounds from sunlight have certain similarities to thermal patches generated by relatively stationary people, which may form "dummy figures" or "ghost figures" at the thermal imaging end. By using the second distance obtained by the radar sensor that supports the identification of whether the reflecting object is a person, the target pixel area that is likely a person based on thermal imaging can be verified to be a person, which helps to verify and eliminate these "dummy figures" or "ghost figures". On the other hand, radar sensors are relatively sensitive to vibrating objects, which may cause the radar sensor to misidentify non-human vibrating objects (such as range hoods) in the kitchen as stationary people at a fixed distance. Thermal imaging can eliminate this misidentification based on its thermal characteristics combined with position and time information. In other words, the processor 110 can also filter the third distance between the person detected and identified by the radar sensor based on thermal imaging data to filter out the third distance that misidentifies vibrating objects as people.

[0146] In some further embodiments, the radar sensor also includes at least two receiving antennas, enabling the radar sensor to also support the measurement of a first offset angle of the person relative to the radar sensor. This first offset angle can be used as a second offset angle of the person relative to the thermal imaging module 120. Alternatively, the offset angle, corrected for the first offset angle based on the relative positional relationship between the radar sensor and the thermal imaging module, can also be used as the second offset angle of the person relative to the thermal imaging module 120. The second offset angle is also the object angle between the person and the optical axis of a reference pixel of the thermal imaging module 120.

[0147] The processor 110 can detect whether a target pixel region exists in the image plane within a pixel region corresponding to a second distance and whose offset angle relative to a reference pixel corresponds to a second offset angle. If a target pixel region exists, the type of the imaged object in the target pixel region is determined to be a human. The pixel region corresponding to the offset angle relative to the reference pixel and the second offset angle refers to the pixel region within the image angle corresponding to the second offset angle (i.e., the object angle). By employing a radar sensor capable of simultaneously acquiring distance and angle information of the target object, the accuracy and efficiency of verification can be improved.

[0148] In some embodiments, pixel regions corresponding to the temperature of a standard human body and radar distances (i.e., third distances) can be collected offline at different orientations and distances. These pre-collected offline data combinations (pixel regions + third distances) are used as a reference benchmark. When a data combination closely approximates the reference benchmark exists in the actual scene, the type of the imaged object in the pixel region of that data combination can be determined to be human. Optionally, a reference benchmark can also be used to correct the collected data; for example, the third distance can be corrected to correct deviations caused by the radar sensor collecting distance data.

[0149] When verifying whether the imaged object in the first pixel region is a person using the above method, it is also possible to make an auxiliary determination by considering whether the imaged object in the first pixel region has moved. That is, if it is moving, it can be determined to be a person with higher confidence.

[0150] In some embodiments, the sensing device 100 further includes a positioning module. The positioning module determines a first position of the person relative to the thermal imaging module. The positioning module may employ, but is not limited to, a depth camera (e.g., a TOF sensor) and a positioning structure based on other positioning technologies. The positioning module may be arranged close to the thermal imaging module 120 such that the position of the person relative to the positioning module determined by the positioning module can be considered as the position of the person relative to the thermal imaging module 120. The processor 110 can calculate the horizontal distance between the person and the heat source based on the first position and the mapping principle of the thermal imaging module 120. Thus, even when the installation height of the thermal imaging module 120 is unknown, the horizontal distance between the person and the heat source can be calculated using the first position of the person relative to the thermal imaging module 120 determined by the positioning module. The first position of the person relative to the thermal imaging module 120 may include the horizontal distance between the person and the thermal imaging module 120, for example, the horizontal distance between the person's head and the thermal imaging module 120. For example, as described above... Figure 7 Based on the distance between pixel D' and reference pixel A, and the horizontal distance between the head and the thermal imaging module 120, the proportional relationship between the pixel size in the virtual image plane parallel to the horizontal plane and the physical size on the horizontal plane where the human head is located can be obtained. Based on this proportional relationship, the height and the installation height of the thermal imaging module 120 can be further calculated. Then, the above combined... Figure 6 The described calculation method calculates the horizontal distance between a person and a heat source. It should be understood that, given that the person's first position relative to the thermal imaging module 120 is known, other methods can also be used to calculate the horizontal distance between the person and the heat source, and this disclosure does not limit this method.

[0151] Adult judgment The processor 110 can also determine whether the identified person is an adult.

[0152] The processor 110 can calculate a person's height based on a second distance or a first position, the image angle between the head position in the first pixel region and a reference pixel, and the mapping principle of the thermal imaging module. For details on the calculation method, please refer to the relevant description above.

[0153] Processor 110 can also calculate the first physical size corresponding to the first pixel region. The first physical size may include, but is not limited to, the area, outline size, or projected size of the first pixel region.

[0154] The processor 110 can determine whether the identified person is an adult based on height and / or first physical dimensions.

[0155] In some embodiments, the radar sensor can detect heart rate and / or respiratory rate. Children's heart rate and breathing are much faster than adults', so the heart rate and / or respiratory rate collected by the radar sensor can also be used as auxiliary features to determine whether the first pixel area belongs to a child or an adult.

[0156] In some embodiments, considering children's limited understanding of the kitchen, after determining whether the identified person is an adult, the processor 110 can filter out the identification results of non-adults (i.e., children), and perform at least one of the following functions based on the state information of the heat source, whether there is an adult identification result in the thermal imaging image, and the first distance between the identified adult and the heat source (optionally including any one or more of the following: relative position, stove type, cooking method, time information, determination result of whether the person is an adult, identification result of whether there is a pet in the thermal imaging image, fourth distance between the identified pet and the heat source, size and / or position of the kitchen wall, size and / or position of the kitchen door, size and / or position of the kitchen stove, detection result of the smoke sensor, etc.): determine the kitchen scene; determine the normal state or abnormal state; determine the risk state; generate a decision; send a control command.

[0157] In some embodiments, the event of a child entering the kitchen can itself be considered a risk, such as the possibility of accidental operation or burns. Therefore, after determining whether the identified person is an adult, the processor 110 can also perform at least one of the following functions based on the state information of the heat source, the first distance between the identified person and the heat source, and the determination result of whether the person is an adult (optionally including any one or more of the following: relative position, stove type, cooking method, time information, determination result of whether the person is an adult, identification result of whether there is a pet in the thermal imaging image, fourth distance between the identified pet and the heat source, size and / or position of the kitchen wall, size and / or position of the kitchen door, size and / or position of the kitchen stove, detection result of the smoke sensor, etc.): determine the kitchen scene; determine the normal state or abnormal state; determine the risk state; generate a decision; send a control command.

[0158] Pet recognition Pets typically roam the kitchen. A pet's presence in the kitchen usually requires extra attention. For example, a pet might touch or knock over a heat source, get burned by a hot surface, or interfere with cooking, leading to an accident.

[0159] Therefore, the processor 110 can also identify pets within the detection area based on temperature distribution data and calculate a fourth distance between the identified pet and the heat source. For details on how the fourth distance is calculated, please refer to the description of the first distance above.

[0160] The processor 110 can perform at least one of the following functions based on the heat source status information, the identification result of whether there is a person in the thermal imaging image and the first distance between the identified person and the heat source, the identification result of whether there is a pet in the thermal imaging image and the fourth distance between the identified pet and the heat source (optionally including any one or more of the following information: relative position, stove type, cooking method, time information, determination result of whether the person is an adult, identification result of whether there is a pet in the thermal imaging image, the fourth distance between the identified pet and the heat source, the size and / or position of the kitchen wall, the size and / or position of the kitchen door, the size and / or position of the kitchen stove, the detection result of the smoke sensor, etc.): determine the kitchen scene; determine the normal state or abnormal state; determine the risk state; generate a decision; send control commands.

[0161] This differs from the first distance between a person (especially an adult) and a heat source. The smaller the fourth distance between a pet and a heat source, the closer the pet is to the heat source, and the higher the risk. The smaller the first distance between a person (especially an adult) and a heat source, the easier the heat source is to manipulate and / or observe, and the lower the risk.

[0162] Determining relative position The following is an exemplary description of determining the relative position of a person and / or the relative position of a heat source through calculation. It should be understood that in some embodiments, the relative position of a person and / or the relative position of a heat source can also be obtained in other ways, for example, by user input.

[0163] For example, as described above Figure 6 Based on the installation height H and the heat source height h, the vertical distance (Hh) between a person and the thermal imaging module on the horizontal plane at the heat source height can be determined. Based on the first object angle γ and the vertical distance (Hh) between the person and the thermal imaging module on the horizontal plane at the heat source height, the horizontal distance between the person and the thermal imaging module on the horizontal plane at the heat source height can be calculated using trigonometric functions. This horizontal distance can be used as the person's relative position. Similarly, based on the second object angle θ and the vertical distance (Hh) between the heat source and the thermal imaging module, the horizontal distance between the heat source and the thermal imaging module can be calculated using trigonometric functions. This horizontal distance can be used as the relative position of the heat source's projection onto the ground relative to the optical axis of the reference pixel within the detection area.

[0164] For example, as described above Figure 7According to the relevant description, based on the proportional relationship and the pixel size between pixel D' and the reference pixel A, the horizontal distance between the human head and the thermal imaging module 120 can be determined, and this horizontal distance can be used as the relative position of the person. Based on the horizontal distance between the human head and the thermal imaging module 120, the installation height of the thermal imaging module 120 can be calculated, and then the horizontal distance between the heat source and the thermal imaging module can be calculated, which can be used as the relative position of the heat source.

[0165] In some embodiments, the size and / or location information of objects such as kitchen walls, doors, and stovetops can also be obtained. Based on the relative position of the person, the relative position of the heat source, and the size and / or location information of objects such as kitchen walls, doors, and stovetops, it can be determined whether the person is obstructed by a wall or door. The same first distance can lead to different interpretations of the scene and state depending on whether there is obstruction. Furthermore, whether the person moves towards or away from the door also implies different interpretations of the scene and state.

[0166] Smoke sensor The sensing device disclosed herein may further include a smoke sensor disposed in the kitchen. The detection area of ​​the smoke sensor may be comparable to the detection area of ​​the thermal imaging module. For example, the detection area of ​​the smoke sensor may be slightly larger than the detection area of ​​the thermal imaging module.

[0167] Based on principles and current regulations, kitchen areas are generally considered "unsuitable" for installing traditional smoke detectors.

[0168] In some embodiments, the processor can determine whether the smoke sensor's detection result is usable based on the result of performing at least one of the above functions, thereby enabling the smoke detector to be used in the kitchen area. The smoke sensor's detection result can refer to smoke detection data (such as smoke concentration), or whether the smoke detection data triggers an alarm, or it can include both of these information simultaneously. Determining whether the smoke sensor's detection result is usable means, based on the result of at least one of the above functions, determining whether the smoke sensor's detection result is erroneous, and if no error is found, determining that the smoke sensor's detection result is usable (e.g., determining whether to trigger an alarm based on whether the smoke detection data exceeds a threshold).

[0169] In some embodiments, considering that different stove types and cooking methods produce different smoke detection results under different conditions, the smoke sensor's detection results can also be used as one of the input information to assist in scene determination. That is, the processor can also perform at least one of the above functions based on the smoke sensor's detection results. For example, regardless of whether a heat source is detected based on thermal imaging, at least one of the above functions can be performed based on the smoke sensor's detection results.

[0170] Control commands The control commands sent by the processor to the actuator can be understood as commands adapted to the current kitchen scenario and / or risk status and / or decision-making. The actuator may include, but is not limited to, fire suppression modules (such as fire sprinklers), power switches (such as switching control of normal lighting and / or emergency lighting), gas switches, smoke exhaust fan switches, and output modules (such as indicator lights, buzzers, and mobile phones).

[0171] Therefore, control commands can include control commands for fire protection purposes (such as fire extinguishing instructions), as well as control commands for non-fire protection purposes, such as commands to turn off the power, gas switch, or output reminder information.

[0172] In some embodiments, the actuator includes a fire extinguishing module. Upon determining that a fire is currently occurring and that no person is detected within the operating and observation range, or the first distance is increasing, the processor 110 controls the fire extinguishing nozzles to spray extinguishing agent towards the fire heat source based on the location information of the fire heat source.

[0173] In some embodiments, the actuator includes an output module. The processor 110 sends a control signal to the output module based on the kitchen scene and / or risk state and / or decision, to control the output module to output reminder information that matches the kitchen scene and / or risk state and / or decision. For example, multiple reminder modes of different intensities can be set. A reminder mode of appropriate intensity can be selected to output reminder information based on the currently determined kitchen scene and / or risk state and / or decision.

[0174] For example, reminder information can include non-mandatory reminder information and mandatory reminder information. Non-mandatory reminder information refers to reminder information that can be automatically reset without user operation. Mandatory reminder information refers to reminder information that requires user intervention to reset. If the current cooking task is detected as normal or the risk status is low, non-mandatory reminder information can be output. For example, an indicator light corresponding to the direction of the heat source can be controlled to remain constantly lit or flash at a frequency varying according to a first distance (the flashing frequency can be positively correlated with the first distance, i.e., the larger the first distance, the faster the flashing frequency) to remind the person in the kitchen that a normal cooking task is currently underway. If the current cooking task is detected as abnormal (such as water about to boil dry or already boiled dry) or the risk status is high, mandatory reminder information can be output. For example, a buzzer can be controlled to emit an alarm sound at a specific frequency, or a notification message can be pushed to the user's app, or a phone call can be made to the user.

[0175] In some optional, but not necessarily preferred, embodiments, multiple reminder methods with different intensities can be set, each with corresponding trigger conditions. Trigger conditions may include, for example, combinations of the following characteristic parameters: the status information of the heat source, the identification result of whether a person is present in the thermal imaging image, and the first distance and / or relative position between the identified person and the heat source (optionally including but not limited to any one or more of the following: stove type, cooking method, time information, determination of whether the person is an adult, identification result of whether a pet is present in the thermal imaging image, fourth distance between the identified pet and the heat source, size and / or location of the kitchen wall, size and / or location of the kitchen door, size and / or location of the kitchen stove, and detection results from the smoke sensor). Different trigger conditions correspond to different combinations of characteristic parameter values. These reminder methods may include, but are not limited to: an indicator light in the direction corresponding to the heat source remaining constantly lit; an indicator light in the direction corresponding to the heat source flashing; a buzzer sounding intermittently at a first frequency; a buzzer sounding intermittently at a second frequency, where the second frequency is greater than the first frequency; a buzzer sounding intermittently at a third frequency, where the third frequency is greater than or equal to the second frequency; APP push notifications; and telephone reminders. In some exemplary embodiments, the triggering condition may also be an intelligent decision generated by the agent based on its understanding of the input. That is, the state information of the heat source, the identification result of whether there is a person in the thermal imaging image, and the first distance and / or relative position between the identified person and the heat source (optionally including but not limited to any one or more of the following: stove type, cooking method, time information, determination result of whether the person is an adult, identification result of whether there is a pet in the thermal imaging image, fourth distance between the identified pet and the heat source, size and / or position of the kitchen wall, size and / or position of the kitchen door, size and / or position of the kitchen stove, detection result of the smoke sensor, etc.) can be input into the agent (i.e., the AI ​​model), and the agent determines the reminder method that matches the current scene based on its understanding of the input.

[0176] Key variables The input information mentioned above can be considered as key variables.

[0177] That is, key variables include the state information of the heat source, the identification result of whether there is a person in the thermal imaging image, and the first distance and / or relative position. Key variables may also include, but are not limited to, any one or more of the following: stove type, cooking method, time information, the determination result of whether the person is an adult, the identification result of whether there is a pet in the thermal imaging image, the fourth distance between the identified pet and the heat source, the size and / or location of the kitchen wall, the size and / or location of the kitchen door, the size and / or location of the kitchen stove, and the detection result of the smoke sensor.

[0178] Key variables can be considered as the filtered non-key variables of the kitchen. Key variables can be divided into two categories: heat source data and human data. Heat source data refers to data related to heat sources. Heat source data can also be called natural variables. Heat source data generally follows natural laws; understanding the applicable natural laws for heat source data enables predictability of the heat source. Human data refers to data related to people. Human data can also be called agent variables. Besides following natural laws, human data generally has a certain purpose, especially in the kitchen setting where it often has a purpose related to heat sources. This allows for combining both natural laws and purposefulness to process key variables in order to perform corresponding functions.

[0179] For example, the core purpose of a person in a kitchen space is to complete the cooking task, optionally including concurrent non-cooking tasks. Under specific conditions such as stove power, cookware, and water volume, the heating characteristics of a heat source follow physical laws (obtainable through theoretical calculations or actual data collection). Therefore, it's possible to predict when boiling will occur, for instance, at the initial stage. Based on the person's cooking purpose, it's also possible to predict or expect that the person will be within operating distance when boiling to adjust power and handle the food; similar logic can be used to predict that the person will be within observation distance to check the status during intermediate cooking stages. The speed of movement and waiting time can be understood as whether an emergency or multitasking state is in effect. These can serve as statistical regularities, and based on this statistical regularity, it's also possible to predict, for example, when the person might appear or the timeliness of handling abnormal situations.

[0180] In summary, the input information disclosed herein for processor processing includes two key variables: heat source data and human data. The heat source data and human data respectively conceal natural laws and purpose. Therefore, the execution result obtained by the processor based on the aforementioned input information can be considered as an execution result combining natural laws and purpose.

[0181] In some embodiments, different kitchen scenarios may have different state machines. Each state machine may include at least one state node, and different state nodes may represent different state stages. A preceding state node can point to or connect to a subsequent state node via a one-way arrow. That is, there is a certain causal relationship or sequence between state nodes. The processor may determine state node information based on currently collected key variables (optionally including previously collected key variables). The state node information includes at least one of the following: the current state node, the cause of the current state node, the future state node derived from the current state node, and the cause of the future state node. Then, the processor may perform at least one of the following functions based on the state node information: determine the kitchen scenario; determine a normal state or an abnormal state; determine a risk state; generate a decision; and send control commands.

[0182] In some exemplary embodiments, a set of basic change units may be pre-constructed. The set of basic change units includes multiple basic change units. Each basic change unit corresponds to a first state node. Each basic change unit may include at least one of the following (preferably multiple of the following simultaneously): a first state node, a second key variable of the first state node and / or one or more second state nodes preceding the first state node, one or more third state nodes following the first state node, a cause of the evolution from the second state node to the first state node, and a cause of the evolution from the first state node to the third state node.

[0183] One or more second state nodes preceding the first state node include at least a preceding state node that is a one-hop neighbor of the first state node, and optionally also a preceding state node that is a multi-hop neighbor of the first state node. One or more third state nodes following the first state node include at least a following state node that is a one-hop neighbor of the first state node, and optionally also a following state node that is a multi-hop neighbor of the first state node. A one-hop neighbor relationship refers to a one-hop neighbor state node directly connected to the first state node; a multi-hop neighbor relationship refers to a multi-hop neighbor state node connected to the first state node through a one-hop neighbor state node of the first state node. Figure 4 Taking the state node NS3 in the M state machine of the cooking task scenario as an example, the one-hop neighbor state node in the state node before NS3 is NS2, the multi-hop neighbor state node in the state node before NS3 is NS1, the one-hop neighbor state nodes in the state node after NS3 include NS4 and ANS2, and the multi-hop neighbor state nodes in the state node after NS3 can include NS4, ANS2, NS5, and NS6.

[0184] The processor can locate target basic change units in the set of basic change units that match the first key variable. For example, it can locate target basic change units in the set of basic change units whose second key variable matches (i.e., is similar to) the first key variable. Based on the located target basic change units, state node information is obtained. The first key variable refers to the currently acquired key variable to be processed (optionally, it also includes previously acquired key variables). The second key variable refers to the key variable of the first state node, and / or the key variables of one or more second state nodes preceding the first state node. Both the first and second key variables can include heat source data and human data, such as including but not limited to the state information of the heat source, the identification result of whether there is a person in the thermal imaging image, and the first distance and / or relative position.

[0185] In some exemplary embodiments, the processor's algorithm for processing input information (i.e., key variables) may also be configured (e.g., trained) to identify state node information based on the input information.

[0186] For example, during the training data construction phase, key variable data can be collected. The collected information is the input information for the state time slice, including heat source classification information, heat source data, and human data. Optional auxiliary information such as smoke detection results, brightness, sound, time, and region can also be included. Next, the collected information is broken down into multiple basic change units. For example, a single basic change unit can contain at least one significant change node or turning point in the heat source data in time and / or space (often a node in the state machine, which generally includes human interaction). This change node or turning point is the first state node mentioned above. Another example is that a single basic change unit can contain at least one significant change node or turning point in the human data in time and / or space. This change node or turning point is the first state node mentioned above. In other words, even if the heat source does not change significantly, a significant change in the human's position or distance (such as moving into or away from the observation distance) can also be considered a significant change node or turning point (i.e., the first state node) and broken down into a basic change unit.

[0187] The dataset of the basic change units can include key variables and auxiliary information (optional) from the first state node to the next or previous node. Then, the large dataset is analyzed to determine the causes and consequences of these nodes, i.e., (optionally combined with human intent) what factors caused the change in this node (i.e., the cause of the first node), and what short-term and / or long-term consequences this change will produce (i.e., the third node after the first node, and the cause of the third node). After obtaining the dataset of the basic change units and the causes and consequences of the nodes represented by the basic change units, an algorithm capable of identifying the causes and consequences of nodes based on the dataset can be constructed. For example, a machine learning model can be trained using the dataset of the basic change units as sample features and the causes and consequences of the nodes represented by the basic change units as sample labels, enabling the trained machine learning model to predict the causes and consequences of nodes based on the dataset (i.e., key variables).

[0188] In some embodiments, the processor 110 can identify heat sources within the detection area based on temperature distribution data and determine the state information of the heat sources, and determine the kitchen scene based on the state information of the heat sources. That is, the determination of the kitchen scene can be based solely on the state information of the heat sources, without combining information on the presence or absence of people, or the first distance and / or relative position between people and heat sources.

[0189] For example, processor 110 can identify the heat source type and / or cooking method based on the heat source status information, or processor 110 can acquire the externally input heat source type and / or cooking method. Based on the heat source type and / or cooking method and the heat source status information, processor 110 can determine whether the current cooking state is normal. When it is determined that the current cooking state is abnormal, processor 110 can also determine the specific type of abnormality, such as burnt-out or sudden stove shutdown.

[0190] In some optional, but not necessarily preferred, exemplary embodiments, multiple kitchen scenarios can be defined according to feature dimensions such as "stove type + cooking method" and "cooking development stage." This allows the currently determined kitchen scenario to reflect different development stages of the cooking process (such as normal cooking state, high-risk cooking state such as dry burning or overheating of oil, extreme abnormal state such as fire). This helps determine the appropriate time for the user to return to the kitchen for intervention, avoiding excessive interference with the user. In multi-tasking scenarios in the kitchen, this ensures that the cooking task can be completed continuously and safely while allowing the user to handle other tasks normally. This approach can be used to determine mainstream, limited-number, discrete kitchen scenarios. For non-discrete, continuously changing, and nearly infinitely combinable kitchen scenarios, AI models with the ability to learn, understand, and predict scenarios can be used for determination.

[0191] For example, in water-based heating (such as boiling water) cooking, this disclosure can utilize the physical constraints formed by the specific heat and latent heat of vaporization of water to determine the current cooking stage. For instance, when the temperature of a hot spot area is consistently limited to a predetermined physical upper limit, it can be determined that the current cooking process is in a water-based heating state; when the aforementioned physical constraints disappear and are accompanied by significant changes in the rate of temperature increase and spatial non-uniformity, it can be determined that the water has been depleted or is about to enter a dry-burning state. Thus, by utilizing the "temperature clamping" brought about by phase change as a physical criterion, the cooking stage can be identified without relying on container identification or water volume, and risk assessment can be triggered by the "disappearance of physical constraints" rather than a fixed threshold.

[0192] It should be understood that this disclosure utilizes the state information of the heat source to determine the kitchen scenario, enabling early risk identification rather than reactive response, and supporting progressive early warning and risk escalation. Taking boiling water as an example, the rate of temperature increase (combining the size of the fire and the amount of water) can be used to determine the possible time it will take to boil dry before the water dries up, and reminders of varying degrees can be issued accordingly, such as gradually increasing the intensity of the reminder as the time to boil dry approaches.

[0193] For example, the processor 110 can also identify people in the detection area based on temperature distribution data, and determine the kitchen scene based on the status information of the heat source and the identification results of whether there are people in the thermal imaging image.

[0194] For example, the processor 110 can also send control commands to the corresponding actuators based on the kitchen scenario.

[0195] Figure 8 This is a schematic block diagram of a sensing device according to other embodiments of the present disclosure.

[0196] See Figure 8 The sensing device 100 may include a processor 110, at least one thermal imaging module 120, and a ranging module 130. In this embodiment, the thermal imaging module 120 may be arranged indoors, not limited to the kitchen, with its detection end facing the ground, to perform infrared radiation detection on the corresponding detection area on the ground and obtain temperature distribution data of the detection area.

[0197] The processor 110 identifies people within the detection area based on temperature distribution data. The ranging module 130 measures a second distance between the thermal imaging module and the person. Based on the second distance and the mapping principle of the thermal imaging module, the processor 110 calculates the first relative position of the person within the detection area, relative to the optical axis of a reference pixel projected onto the ground.

[0198] Taking the second distance, which includes the distance between a person's head and the thermal imaging module, as an example, and referring to the relevant description above, the horizontal distance between the head and the thermal imaging module can be calculated. This horizontal distance can be used as the first relative position of the person's projection position on the ground relative to the optical axis of the reference pixel within the detection area.

[0199] Therefore, the sensing device proposed in this embodiment can be used for indoor positioning.

[0200] In some embodiments, the location of the thermal imaging module 120 indoors is known, and the processor 110 can combine the location of the thermal imaging module 120 and the first relative position of the person with respect to the thermal imaging module 120 to achieve precise positioning of the person.

[0201] The location information of a person determined by sensing devices (including the first relative position and / or absolute position) can be sent to the relevant personnel / departments / units in real time or non-real time for fire-fighting purposes such as personnel evacuation, dispersal and search and rescue command in the event of a fire, as well as other social functions and / or commercial purposes in the event of a fire.

[0202] The ranging module can use a radar sensor, or it can utilize the thermal imaging module 120 and the processor 110 to perform ranging. See the relevant description above for more information on the ranging module.

[0203] The sensing device according to this disclosure has been described in detail above with reference to the accompanying drawings.

[0204] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A sensing device, comprising: Processor and at least one thermal imaging module, The thermal imaging module is installed in the kitchen with its detector facing the ground to detect infrared radiation in a corresponding area on the ground and obtain temperature distribution data for that area. The processor identifies people and heat sources within the detection area based on the temperature distribution data. The processor calculates a first distance between the identified person and the heat source, and / or the processor determines the relative position of the person and / or the relative position of the heat source. The processor determines the state information of the heat source based on the temperature distribution data. Based on the state information of the heat source, the identification result of whether there is a person in the thermal imaging image, and the first distance and / or the relative position, the processor performs at least one of the following functions: determining the kitchen scene; Determine whether the state is normal or abnormal; determine the risk state; generate a decision; send control commands.

2. The sensing device according to claim 1, wherein, Each temperature value in the temperature distribution data corresponds to a pixel position in the image plane. The processor calculates the horizontal distance between the person and the heat source based on the first dimension information of a standard human body and the mapping principle of the thermal imaging module, wherein the first distance includes the horizontal distance; or The sensing device further includes a ranging module or a positioning module. The ranging module measures a second distance between the thermal imaging module and the person. The positioning module determines a first position of the person relative to the thermal imaging module. The processor calculates the horizontal distance between the person and the heat source based on the second distance or the first position and the mapping principle of the thermal imaging module.

3. The sensing device according to claim 2, wherein, The ranging module includes the thermal imaging module and the processor. The thermal imaging module measures the change in the radiation temperature of the target object as the distance between the target object and the thermal imaging module changes. The processor determines a second distance between the thermal imaging module and the person based on the temperature value of at least one pixel location in the first pixel region of the image plane, which is different from the background temperature and is identified as a person, and pre-acquired calibration data. The calibration data characterizes the correspondence between the radiation temperature obtained by the thermal imaging module from measuring the human body at different distances and the distance.

4. The sensing device according to claim 3, wherein, The processor calculates the third size information of the person based on the second distance and the first pixel region, and verifies whether the type of the imaging object in the first pixel region is a person based on the third size information.

5. The sensing device according to claim 3, wherein, The ranging module includes a radar sensor, which detects whether the type of reflective object within the detection area is a person, and detects a third distance between the identified person and the radar sensor. This second distance is either the third distance itself, or a distance corrected for the third distance based on the relative positional relationship between the radar sensor and the thermal imaging module. The processor detects whether there is a target pixel region in the pixel region corresponding to the second distance in the image plane. If the target pixel region exists, it determines that the type of the imaging object in the target pixel region is a human. The target pixel region is a pixel region whose type of imaging object is likely to be a human, as determined based on the temperature distribution data.

6. The sensing device according to claim 5, wherein, The radar sensor includes at least two receiving antennas. The radar sensor also measures a first offset angle of the person relative to the radar sensor. A second offset angle of the person relative to the thermal imaging module is the first offset angle, or the second offset angle is an offset angle corrected for the first offset angle based on the relative positional relationship between the radar sensor and the thermal imaging module. The processor detects whether the target pixel region exists in the pixel region in the image plane that corresponds to the second distance and whose offset angle relative to the reference pixel corresponds to the second offset angle. If the target pixel region exists, it determines that the type of the imaging object in the target pixel region is human, and the optical axis of the reference pixel is substantially perpendicular to the horizontal plane.

7. The sensing device according to any one of claims 2 to 6, wherein, The processor calculates the person's height based on the second distance or the first position, the image angle between the head position in the first pixel region and the reference pixel, and the mapping principle of the thermal imaging module, wherein the optical axis of the reference pixel is substantially perpendicular to the horizontal plane; and / or, the processor calculates the first physical dimension corresponding to the first pixel region. The processor determines whether the person is an adult based on the height and / or the first physical dimensions. Based on the state information of the heat source, the identification result of whether there is an adult in the thermal imaging image, and the first distance between the identified adult and the heat source, the processor performs at least one of the following functions: determining the kitchen scene; determining the risk state; Generate decisions; Send control commands, or Based on the state information of the heat source, the first distance between the identified person and the heat source, and the determination result of whether the person is an adult, the processor performs at least one of the following functions: determining the kitchen scene; Determine whether the state is normal or abnormal; determine the risk state; generate a decision; send control commands.

8. The sensing device according to claim 1, wherein, The processor determines the state information of the heat source based on the angle between the optical axis of the thermal imaging module and the horizontal plane and / or the installation height of the thermal imaging module, as well as the temperature distribution data.

9. The sensing device according to claim 1, wherein, The processor identifies the heat source type and / or cooking method based on the state information of the heat source, or the processor obtains the heat source type and / or cooking method from external input. Based on the heat source type and / or cooking method, the state information of the heat source, the identification result of whether there is a person in the thermal imaging image, and the first distance between the identified person and the heat source, the processor performs at least one of the following functions: determining the kitchen scene; Determine whether the state is normal or abnormal; determine the risk state; generate a decision; send control commands.

10. The sensing device according to claim 1, wherein, Based on the first distance and / or the relative position, the processor determines whether there are people in the operating range, the observation range, and the non-observation range, obtains a first determination result, and performs at least one of the following functions based on the state information of the heat source and the first determination result: determine the kitchen scene; Determine whether the status is normal or abnormal; determine the risk status. Generate a decision; send a control command, wherein the operating range is smaller than the observation range, and the observation range is smaller than the non-observation range.

11. The sensing device according to claim 1, further comprising: The smoke sensor is located in the kitchen. The processor determines whether the detection result of the smoke sensor is usable based on the execution result of the at least one function, or the processor further executes the at least one function based on the detection result of the smoke sensor.

12. The sensing device according to claim 1, wherein, The processor determines state node information based on the state information of the heat source, the identification result of whether there is a person in the thermal imaging image, and the first distance and / or the relative position. The state node information includes at least one of the following: current state node, cause of the current state node, future state node evolved from the current state node, and cause of the future state node. The processor performs at least one of the following functions based on the state node information: determining the kitchen scene; determining the normal or abnormal state; determining the risk state; generating a decision; and sending control commands.

13. The sensing device according to claim 12, wherein, The processor searches for a target basic change unit in the set of basic change units that matches the first key variable. The set of basic change units includes multiple basic change units, and each basic change unit includes at least one of the following: a first state node, a second key variable of the first state node and / or one or more second state nodes preceding the first state node, one or more third state nodes following the first state node, a cause of the evolution from the second state node to the first state node, and a cause of the evolution from the first state node to the third state node. The first key variable includes the state information of the heat source, the identification result of whether there is a person in the thermal imaging image, and the first distance and / or the relative position. The processor obtains the state node information based on the found target basic change unit.

14. A sensing device, comprising: Processor and at least one thermal imaging module, The thermal imaging module is installed in the kitchen with its detector facing the ground to detect infrared radiation in a corresponding area on the ground and obtain temperature distribution data for that area. The processor identifies people and heat sources within the detection area based on the temperature distribution data. The processor calculates a first distance between the identified person and the heat source, and / or the processor determines the relative position of the person and / or the relative position of the heat source. The processor determines the state information of the heat source based on the temperature distribution data. The processor determines state node information based on the state information of the heat source, the identification result of whether there is a person in the thermal imaging image, and the first distance and / or the relative position. The state node information includes at least one of the following: current state node, cause of current state node, future state node evolved from current state node, and cause of future state node.

15. A sensing device, comprising: Processor and at least one thermal imaging module, The thermal imaging module is installed in the kitchen with its detector facing the ground to detect infrared radiation in a corresponding area on the ground and obtain temperature distribution data for that area. The processor identifies heat sources within the detection area based on the temperature distribution data and determines the status information of the heat sources. The processor determines the kitchen scene based on the state information of the heat source.

16. The sensing device according to claim 15, wherein, The processor identifies the heat source type and / or cooking method based on the state information of the heat source, or the processor obtains the heat source type and / or cooking method from external input. The processor determines the kitchen scenario based on the heat source type and / or cooking method, and the status information of the heat source.

17. A sensing device, comprising: The processor, at least one thermal imaging module, and a ranging module The thermal imaging module is installed indoors with its detector facing the ground to detect infrared radiation in a corresponding area on the ground, thereby obtaining temperature distribution data for that area. The processor identifies people within the detection area based on the temperature distribution data. The ranging module measures a second distance between the thermal imaging module and the person. Based on the second distance and the mapping principle of the thermal imaging module, the processor calculates the first relative position of the projection of the person's optical axis relative to the reference pixel on the ground within the detection area, wherein the optical axis of the reference pixel is substantially perpendicular to the horizontal plane.

Images