Iot-based kitchen monitoring management method and system

Abstract

This invention provides a kitchen monitoring and management method and system based on the Internet of Things (IoT). It performs real-time visual monitoring of the kitchen's internal environment using two different wavelengths: thermal infrared and visible light. This obtains real-time visual monitoring data of the kitchen's internal environment, comprehensively calibrates the heat sources and the presence of items in the kitchen, and then accurately and promptly determines whether a safety incident has occurred in the kitchen. This facilitates targeted alarm and fire extinguishing operations, effectively reducing the misjudgment rate of kitchen safety incidents and improving the reliability of kitchen internal environment monitoring.

Description

Technical Field

[0001] This invention relates to the technical field of smart home monitoring, and in particular to a method and system for kitchen monitoring and management based on the Internet of Things. Background Technology

[0002] As a key area in the home, the kitchen contains items such as gas stoves and knives. To ensure kitchen safety, modern residences typically install smoke sensors to collect smoke concentrations, thereby determining the possibility of a fire and triggering timely alarms. However, smoke sensors can only detect smoke within the kitchen and cannot provide visual monitoring of the kitchen environment. Furthermore, smoke sensors suffer from a high false alarm rate, reducing the reliability of kitchen environment monitoring. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this invention provides an IoT-based kitchen monitoring and management method and system. It captures and analyzes thermal infrared and visible light images of the kitchen's internal environment to obtain information on the current state of heat sources and items within the kitchen. This information is used to determine whether a fire safety incident or an item placement safety incident has occurred in the kitchen, and a notification message is sent back to the IoT platform. Based on the analysis results of the notification message, corresponding control commands are sent to alarm terminals or fire-fighting terminals connected to the IoT platform to control their operational status. This system performs real-time visual monitoring of the kitchen's internal environment using both thermal infrared and visible light bands, obtaining real-time visual monitoring data. It comprehensively calibrates the state of heat sources and items within the kitchen, thereby accurately and promptly determining whether a safety incident has occurred. This facilitates targeted alarm and fire-fighting operations, effectively reducing the false alarm rate and improving the reliability of kitchen environment monitoring.

[0004] This invention provides a kitchen monitoring and management method based on the Internet of Things, which includes the following steps:

[0005] Step S1: Take thermal infrared and visible light images of the kitchen interior environment to obtain thermal infrared and visible light images of the kitchen interior environment; analyze and process the thermal infrared and visible light images to determine the current heat source presence status information and item presence status information inside the kitchen.

[0006] Step S2: Based on the heat source presence status information and the item presence status information, determine whether a fire safety incident or an item placement safety incident has occurred inside the kitchen; and based on the determination result, return an event occurrence notification message to the IoT platform;

[0007] Step S3: Based on the parsing result of the event notification message, send corresponding control commands to the alarm terminal or fire terminal connected to the IoT platform to control the working status of the alarm terminal or the fire terminal.

[0008] Further, in step S1, thermal infrared and visible light imaging is performed on the kitchen interior environment to obtain thermal infrared and visible light images of the kitchen interior environment; the thermal infrared and visible light images are analyzed and processed to determine the current heat source presence status information and item presence status information inside the kitchen, specifically including:

[0009] Thermal infrared scanning and visible light scanning were performed on the kitchen interior environment to obtain thermal infrared panoramic images and visible light panoramic images of the kitchen interior environment.

[0010] The thermal infrared panoramic image is subjected to thermal infrared spectral analysis to obtain the distribution location information of all fire sources in the thermal infrared panoramic image; based on the distribution location information of each fire source, the outermost boundary contour of each fire source in the real environmental space is extracted from the thermal infrared panoramic image to determine the three-dimensional spatial occupancy information of each fire source in the real environmental space; based on the thermal infrared spectral distribution information of the thermal infrared panoramic image, the temperature distribution information of each fire source is determined; and the three-dimensional spatial occupancy information and the temperature distribution information are used as the fire source existence status information.

[0011] After performing pixel edge sharpening processing on the visible light panoramic image, the placement information of knives and the direction of their blades inside the kitchen, as well as the placement information of the fiber fabrics inside the kitchen, are identified from the visible light panoramic image, and these are used as the existence status information of the items.

[0012] Furthermore, in step S2, based on the heat source presence status information and the item presence status information, it is determined whether a fire safety incident or an item placement safety incident has occurred inside the kitchen; and based on the determination result, an event notification message is returned to the IoT platform, specifically including:

[0013] Based on the three-dimensional spatial occupancy information of the fire source and the placement information of the fiber fabric, determine whether the distance between the outermost boundary of the fiber fabric and the fire source is less than a preset distance threshold; if not, determine that no fire safety incident has occurred in the kitchen; if not, then based on the temperature distribution information, determine whether the temperature value of the outermost boundary of the fire source is greater than a preset temperature threshold; if not, determine that no fire safety incident has occurred in the kitchen; if yes, determine that a fire safety incident has occurred in the kitchen.

[0014] Based on the knife placement information, determine whether the knife is placed in a preset space area; if yes, determine that no item placement safety incident has occurred in the kitchen; if no, then based on the blade orientation information, determine whether the blade is currently exposed; if no, determine that no item placement safety incident has occurred in the kitchen; if yes, determine that an item placement safety incident has occurred in the kitchen.

[0015] If a fire safety incident or an item placement safety incident occurs, an event notification message containing the location information of the incident will be returned to the IoT platform.

[0016] Furthermore, in step S3, based on the parsing result of the event notification message, corresponding control commands are sent to the alarm terminal or fire terminal connected to the IoT platform to control the working status of the alarm terminal or the fire terminal, specifically including:

[0017] The location information corresponding to the fire safety event or the safe placement of items is extracted from the event notification message. This information is then used to send an audible alarm control command to the alarm terminal or a water spray control command to the fire terminal, thereby controlling the audible alarm status of the alarm terminal or the water spray extinguishing status of the fire terminal.

[0018] The present invention also provides an Internet of Things-based kitchen monitoring and management system, which includes:

[0019] The thermal infrared camera module is used to capture thermal infrared images of the kitchen interior environment.

[0020] Visible light camera module, which is used to capture visible light images of the kitchen interior environment to obtain visible light images of the kitchen interior environment;

[0021] The image analysis module is used to analyze and process the thermal infrared image and the visible light image to determine the current heat source presence status information and item presence status information inside the kitchen.

[0022] The event judgment and processing module is used to determine whether a fire safety incident or an item placement safety incident has occurred in the kitchen based on the heat source presence status information and the item presence status information; and to return an event occurrence notification message to the Internet of Things platform based on the judgment result.

[0023] The terminal control module is used to send corresponding control commands to the alarm terminal or fire terminal connected to the Internet of Things platform according to the parsing result of the event notification message, so as to control the working status of the alarm terminal or the fire terminal.

[0024] Furthermore, the thermal infrared camera module performs thermal infrared imaging of the kitchen interior environment to obtain thermal infrared images of the kitchen interior environment, specifically including:

[0025] Thermal infrared scanning was performed on the kitchen interior to obtain a panoramic thermal infrared image of the kitchen interior.

[0026] The visible light camera module captures visible light images of the kitchen's interior environment, specifically including:

[0027] The visible light scanning image of the kitchen interior environment is obtained by scanning the kitchen interior environment.

[0028] The image analysis module analyzes and processes the thermal infrared image and the visible light image to determine the current state of heat sources and the state of items inside the kitchen, specifically including:

[0029] The thermal infrared panoramic image is subjected to thermal infrared spectral analysis to obtain the distribution location information of all fire sources in the thermal infrared panoramic image; based on the distribution location information of each fire source, the outermost boundary contour of each fire source in the real environmental space is extracted from the thermal infrared panoramic image to determine the three-dimensional spatial occupancy information of each fire source in the real environmental space; based on the thermal infrared spectral distribution information of the thermal infrared panoramic image, the temperature distribution information of each fire source is determined; and the three-dimensional spatial occupancy information and the temperature distribution information are used as the fire source existence status information.

[0030] After performing pixel edge sharpening processing on the visible light panoramic image, the placement information of knives and the direction of their blades inside the kitchen, as well as the placement information of the fiber fabrics inside the kitchen, are identified from the visible light panoramic image, and these are used as the existence status information of the items.

[0031] Furthermore, the visible light camera module also adjusts the intensity and area of ​​visible light illumination based on the captured image of the previous frame.

[0032] Step A1: Using formula (1) below, obtain the brightness distribution of the captured previous frame image based on its RGB values.

[0033]

[0034] In the above formula (1), Y(i,j) represents the brightness value at the position of the i-th row and j-th column in the image matrix of the previous frame image; [R(i,j),G(i,j),B(i,j)] represents the RGB value at the position of the i-th row and j-th column in the image matrix of the previous frame image; int[] represents rounding the value in the parentheses to the nearest integer; Y(a,b) represents the brightness value at the position of the a-th row and b-th column in the image matrix of the previous frame image. The image matrix representing the previous frame image is the first... Line number The brightness value at the column position; m represents the total number of pixels in any row of the image matrix of the previous frame image; n represents the total number of pixels in any column of the image matrix of the previous frame image; This means substituting the values ​​of a from 1 to n and the values ​​of b from 1 to m into Y(a,b) to obtain the result that satisfies... In the case of Find the values ​​of a and b when the minimum value is obtained, and denote them as (a′, b′);

[0035] According to step A1 above, the brightness value at the i-th row and j-th column position in the image matrix of the previous frame image is Y(i,j), and from the image matrix of the previous frame image... Position is the center of the circle Draw a circle with the line connecting position (a′, b′) as the radius, and denote the radius of the circle as r(Q). Denote the set of position points contained within the circle as Q. The image part contained within the circle is the highlighted part of the image, and the image part outside the circle is the under-bright part of the image.

[0036] Step A2: Using formula (2) below, based on the brightness distribution of the previous frame image and the visible light intensity when the previous frame image was captured, control the visible light intensity when capturing the next frame.

[0037]

[0038] In the above formula (2), y next This indicates the control light intensity of visible light during the next frame capture; ∑ (i,j)∈Q Y(i,j) represents the sum of all Y(i,j) values ​​that satisfy the condition of belonging to set Q; ∑ (i,j)∈Q 1 represents the total number of (i,j) elements belonging to set Q;

[0039] Step A3: Using the formula (3) below, based on the brightness distribution of the previous frame image, control the radius of the visible light illumination range when capturing the next frame.

[0040]

[0041] In the above formula (3), R next R0 represents the radius of the visible light control range when the next frame is captured; R0 represents the radius of the visible light control range when the previous frame is captured.

[0042] Control the intensity of visible light during the next frame capture to y next The radius of the visible light irradiation range is controlled to be R. next .

[0043] Furthermore, the event judgment and processing module determines whether a fire safety incident or an item placement safety incident has occurred in the kitchen based on the heat source presence status information and the item presence status information; and returns an event notification message to the IoT platform based on the judgment result, specifically including:

[0044] Based on the three-dimensional spatial occupancy information of the fire source and the placement information of the fiber fabric, determine whether the distance between the outermost boundary of the fiber fabric and the fire source is less than a preset distance threshold; if not, determine that no fire safety incident has occurred in the kitchen; if not, then based on the temperature distribution information, determine whether the temperature value of the outermost boundary of the fire source is greater than a preset temperature threshold; if not, determine that no fire safety incident has occurred in the kitchen; if yes, determine that a fire safety incident has occurred in the kitchen.

[0045] Based on the knife placement information, determine whether the knife is placed in a preset space area; if yes, determine that no item placement safety incident has occurred in the kitchen; if no, then based on the blade orientation information, determine whether the blade is currently exposed; if no, determine that no item placement safety incident has occurred in the kitchen; if yes, determine that an item placement safety incident has occurred in the kitchen.

[0046] If a fire safety incident or an item placement safety incident occurs, an event notification message containing the location information of the incident will be returned to the IoT platform.

[0047] Furthermore, based on the parsing result of the event notification message, the terminal control module sends corresponding control commands to the alarm terminal or fire terminal connected to the IoT platform, thereby controlling the working status of the alarm terminal or the fire terminal. Specifically, this includes:

[0048] The location information corresponding to the fire safety event or the safe placement of items is extracted from the event notification message. This information is then used to send an audible alarm control command to the alarm terminal or a water spray control command to the fire terminal, thereby controlling the audible alarm status of the alarm terminal or the water spray extinguishing status of the fire terminal.

[0049] Compared to existing technologies, this IoT-based kitchen monitoring and management method and system captures and analyzes thermal infrared and visible light images of the kitchen's internal environment to obtain information on the current state of heat sources and items within the kitchen. This information is used to determine whether a fire safety incident or an item placement safety incident has occurred in the kitchen, and an event notification message is sent back to the IoT platform. Based on the analysis results of the event notification message, corresponding control commands are sent to alarm terminals or fire protection terminals connected to the IoT platform to control their operational status. This system performs real-time visual monitoring of the kitchen's internal environment using both thermal infrared and visible light bands, obtaining real-time visual monitoring data. It comprehensively calibrates the state of heat sources and items within the kitchen, thereby accurately and promptly determining whether a safety incident has occurred. This facilitates targeted alarm and fire suppression operations, effectively reducing the false alarm rate of kitchen safety incidents and improving the reliability of kitchen environment monitoring.

[0050] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings.

[0051] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 A flowchart illustrating the IoT-based kitchen monitoring and management method provided by this invention.

[0054] Figure 2 This is a schematic diagram of the structure of the Internet of Things-based kitchen monitoring and management system provided by the present invention. Detailed Implementation

[0055] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0056] See Figure 1 This is a flowchart illustrating the IoT-based kitchen monitoring and management method provided in an embodiment of the present invention. The IoT-based kitchen monitoring and management method includes the following steps:

[0057] Step S1: Take thermal infrared and visible light images of the kitchen interior environment to obtain thermal infrared and visible light images of the kitchen interior environment; analyze and process the thermal infrared and visible light images to determine the current heat source presence status information and item presence status information inside the kitchen.

[0058] Step S2: Based on the heat source presence status information and the item presence status information, determine whether a fire safety incident or an item placement safety incident has occurred inside the kitchen; and based on the determination result, return an event notification message to the IoT platform.

[0059] Step S3: Based on the parsing result of the event notification message, send corresponding control commands to the alarm terminal or fire terminal connected to the IoT platform to control the working status of the alarm terminal or fire terminal.

[0060] The beneficial effects of the above technical solution are as follows: This IoT-based kitchen monitoring and management method captures and analyzes thermal infrared and visible light images of the kitchen's internal environment to obtain information on the current state of heat sources and items within the kitchen. This information is used to determine whether a fire safety incident or an item placement safety incident has occurred in the kitchen, and a notification message is sent back to the IoT platform. Based on the analysis results of the notification message, corresponding control commands are sent to the alarm or fire-fighting terminals connected to the IoT platform to control their operation. The system performs real-time visual monitoring of the kitchen's internal environment using both thermal infrared and visible light wavelengths, obtaining real-time visual monitoring data. This allows for comprehensive calibration of the heat sources and items within the kitchen, enabling accurate and timely determination of whether a safety incident has occurred. This facilitates targeted alarm and fire-fighting operations, effectively reducing the misjudgment rate of kitchen safety incidents and improving the reliability of kitchen environment monitoring.

[0061] Preferably, in step S1, thermal infrared and visible light imaging are performed on the kitchen interior environment to obtain thermal infrared and visible light images of the kitchen interior environment; the thermal infrared and visible light images are analyzed and processed to determine the current heat source presence status information and item presence status information inside the kitchen, specifically including:

[0062] Thermal infrared scanning and visible light scanning were performed on the kitchen interior environment to obtain thermal infrared panoramic images and visible light panoramic images of the kitchen interior environment.

[0063] Thermal infrared spectral analysis was performed on the thermal infrared panoramic image to obtain the distribution location information of all fire sources in the image. Based on the distribution location information of each fire source, the outermost boundary contour of each fire source in the real environment space was extracted from the thermal infrared panoramic image to determine the three-dimensional spatial occupancy information of each fire source in the real environment space. According to the thermal infrared spectral distribution information of the thermal infrared panoramic image, the temperature distribution information of each fire source was determined. The three-dimensional spatial occupancy information and the temperature distribution information were used as the fire source existence status information.

[0064] After pixel edge sharpening processing of the visible light panoramic image, the placement information of knives and the direction of their blades inside the kitchen, as well as the placement information of the fiber fabrics inside the kitchen, are identified from the visible light panoramic image, and these are used as the existence status information of the items.

[0065] The beneficial effects of the above technical solution are as follows: Thermal infrared scanning of the kitchen interior environment yields a panoramic thermal infrared image. This panoramic image includes thermal infrared spectral distribution information for the entire kitchen environment. Based on this information, areas with the same spectral distribution characteristics as the fire source are identified, thus determining the fire source's location. Using this location information as a benchmark, the outermost boundary of the flame in the real-world environment is determined, thereby calibrating the spatial extent of the fire source. Furthermore, kitchen knives, their blades, and woven fabrics such as dishcloths possess unique surface gloss and texture. By identifying the pixel gloss and texture of all items in the visible light panoramic image, the placement and blade orientation of knives, as well as the placement of woven fabrics within the kitchen, can be obtained.

[0066] Preferably, in step S2, based on the heat source presence status information and the item presence status information, it is determined whether a fire safety incident or an item placement safety incident has occurred inside the kitchen; and based on the determination result, an event notification message is returned to the IoT platform, specifically including:

[0067] Based on the three-dimensional spatial occupancy information of the fire source and the placement information of the fiber fabric, determine whether the distance between the outermost boundary of the fiber fabric and the fire source is less than a preset distance threshold; if not, determine that no fire safety incident has occurred in the kitchen; if not, then based on the temperature distribution information, determine whether the temperature value of the outermost boundary of the fire source is greater than a preset temperature threshold; if not, determine that no fire safety incident has occurred in the kitchen; if so, determine that a fire safety incident has occurred in the kitchen.

[0068] Based on the knife's placement information, determine whether the knife is placed in a preset space area; if so, determine whether no item placement safety incident has occurred in the kitchen; if not, then based on the blade orientation information, determine whether the knife's blade is currently exposed; if not, determine whether no item placement safety incident has occurred in the kitchen; if so, determine whether an item placement safety incident has occurred in the kitchen.

[0069] If a fire safety incident or an item placement safety incident occurs, an event notification message containing the location information of the incident will be returned to the IoT platform.

[0070] The beneficial effects of the above technical solution are as follows: By using the three-dimensional spatial occupancy information and temperature distribution information of the fire source, as well as the placement information of the fabric, as a basis, it can be determined whether the fabric is too close to the fire source or whether the temperature of the fire source is too high, thus posing a potential fire safety incident. Furthermore, based on the placement information and blade orientation information of knives, it can be determined whether improper placement of knives poses a potential risk of injury. In the event of a fire safety incident or a placement safety incident, a notification message containing the corresponding location information of the incident is promptly returned to the IoT platform, facilitating subsequent targeted alarm or fire extinguishing operations based on the location of the incident within the kitchen environment.

[0071] Preferably, in step S3, based on the parsing result of the event notification message, corresponding control commands are sent to the alarm terminal or fire terminal connected to the IoT platform to control the working status of the alarm terminal or fire terminal, specifically including:

[0072] The location information corresponding to the fire safety incident or the safe placement of items is extracted from the incident notification message. This information is then used to send an audible alarm control command to the alarm terminal or a water spray control command to the fire terminal, thereby controlling the audible alarm status of the alarm terminal or the water spray extinguishing status of the fire terminal.

[0073] The beneficial effects of the above technical solution are as follows: when a safety incident occurs involving the placement of items, an audible alarm control command is sent to the alarm terminal, so that the alarm terminal can play a voice alarm message corresponding to the location where the safety incident occurred; when a fire safety incident occurs, a water spray control command is sent to the fire-fighting terminals such as sprinkler heads installed inside the kitchen, so that the sprinkler heads can aim and spray water to extinguish the fire at the location where the fire safety incident occurred.

[0074] See Figure 2 This is a schematic diagram of the structure of an IoT-based kitchen monitoring and management system provided in an embodiment of the present invention. The IoT-based kitchen monitoring and management system includes:

[0075] The thermal infrared camera module is used to capture thermal infrared images of the kitchen interior environment.

[0076] Visible light camera module, which is used to capture visible light images of the kitchen interior environment to obtain visible light images of the kitchen interior environment;

[0077] The image analysis module is used to analyze and process the thermal infrared image and the visible light image to determine the current state of heat sources and the state of items inside the kitchen.

[0078] The event judgment and processing module is used to determine whether a fire safety incident or an item placement safety incident has occurred in the kitchen based on the heat source presence status information and the item presence status information; and to return an event occurrence notification message to the IoT platform based on the judgment result.

[0079] The terminal control module is used to send corresponding control commands to the alarm terminal or fire terminal connected to the Internet of Things platform according to the parsing result of the notification message of the event occurrence, so as to control the working status of the alarm terminal or the fire terminal.

[0080] The beneficial effects of the above technical solution are as follows: The IoT-based kitchen monitoring and management system captures and analyzes thermal infrared and visible light images of the kitchen's internal environment to obtain information on the current state of heat sources and items within the kitchen. This information is used to determine whether a fire safety incident or an item placement safety incident has occurred in the kitchen, and a notification message is sent back to the IoT platform. Based on the analysis results of the notification message, corresponding control commands are sent to the alarm or fire-fighting terminals connected to the IoT platform to control their operation. The system performs real-time visual monitoring of the kitchen's internal environment using both thermal infrared and visible light wavelengths, obtaining real-time visual monitoring data. This allows for comprehensive calibration of the heat sources and item states within the kitchen, enabling accurate and timely determination of whether a safety incident has occurred. This facilitates targeted alarm and fire-fighting operations, effectively reducing the misjudgment rate of kitchen safety incidents and improving the reliability of kitchen environment monitoring.

[0081] Preferably, the thermal infrared camera module performs thermal infrared imaging of the kitchen interior environment to obtain thermal infrared images of the kitchen interior environment, specifically including:

[0082] Thermal infrared scanning was performed on the kitchen interior to obtain a panoramic thermal infrared image of the kitchen interior.

[0083] This visible light camera module captures images of the kitchen's interior environment using visible light, specifically including:

[0084] The visible light scanning image of the kitchen interior environment is obtained by scanning the kitchen interior environment.

[0085] The image analysis module analyzes and processes the thermal infrared image and the visible light image to determine the current state of heat sources and the state of items inside the kitchen, specifically including:

[0086] Thermal infrared spectral analysis was performed on the thermal infrared panoramic image to obtain the distribution location information of all fire sources in the image. Based on the distribution location information of each fire source, the outermost boundary contour of each fire source in the real environment space was extracted from the thermal infrared panoramic image to determine the three-dimensional spatial occupancy information of each fire source in the real environment space. According to the thermal infrared spectral distribution information of the thermal infrared panoramic image, the temperature distribution information of each fire source was determined. The three-dimensional spatial occupancy information and the temperature distribution information were used as the fire source existence status information.

[0087] After pixel edge sharpening processing of the visible light panoramic image, the placement information of knives and the direction of their blades inside the kitchen, as well as the placement information of the fiber fabrics inside the kitchen, are identified from the visible light panoramic image, and these are used as the existence status information of the items.

[0088] The beneficial effects of the above technical solution are as follows: Thermal infrared scanning of the kitchen interior environment yields a panoramic thermal infrared image. This panoramic image includes thermal infrared spectral distribution information for the entire kitchen environment. Based on this information, areas with the same spectral distribution characteristics as the fire source are identified, thus determining the fire source's location. Using this location information as a benchmark, the outermost boundary of the flame in the real-world environment is determined, thereby calibrating the spatial extent of the fire source. Furthermore, kitchen knives, their blades, and woven fabrics such as dishcloths possess unique surface gloss and texture. By identifying the pixel gloss and texture of all items in the visible light panoramic image, the placement and blade orientation of knives, as well as the placement of woven fabrics within the kitchen, can be obtained.

[0089] Preferably, the visible light camera module also adjusts the intensity and area of ​​visible light illumination based on the captured image of the previous frame.

[0090] Step A1: Using formula (1) below, obtain the brightness distribution of the captured previous frame image based on its RGB values.

[0091]

[0092] In the above formula (1), Y(i,j) represents the brightness value at the position of the i-th row and j-th column in the image matrix of the previous frame image; [R(i,j),G(i,j),B(i,j)] represents the RGB value at the position of the i-th row and j-th column in the image matrix of the previous frame image; int[] represents rounding the value in the parentheses to the nearest integer; Y(a,b) represents the brightness value at the position of the a-th row and b-th column in the image matrix of the previous frame image. This represents the first image in the image matrix of the previous frame. Line number The brightness value at the column position; m represents the total number of pixels in any row of the image matrix of the previous frame; n represents the total number of pixels in any column of the image matrix of the previous frame; This means substituting the values ​​of a from 1 to n and the values ​​of b from 1 to m into Y(a,b) to obtain the result that satisfies... In the case of Find the values ​​of a and b when the minimum value is obtained, and denote them as (a′, b′);

[0093] According to step A1 above, the brightness value at the i-th row and j-th column position in the image matrix of the previous frame is Y(i,j), and from the image matrix of the previous frame... Position is the center of the circle Draw a circle with the line connecting position (a′, b′) as the radius, and denote the radius of the circle as r(Q). Denote the set of position points contained within the circle as Q. The image part contained within the circle is the highlighted part of the image, and the image part outside the circle is the under-bright part of the image.

[0094] Step A2: Using formula (2) below, based on the brightness distribution of the previous frame image and the visible light intensity when the previous frame image was captured, control the visible light intensity when capturing the next frame.

[0095]

[0096] In the above formula (2), y next This indicates the control light intensity of visible light during the next frame capture; ∑ (i,j)∈Q Y(i,j) represents summing all T(i,j) corresponding to (i,j) belonging to set Q; ∑ (i,j)∈Q 1 represents the total number of (i,j) elements belonging to set Q;

[0097] Step A3: Using the formula (3) below, based on the brightness distribution of the previous frame image, control the radius of the visible light illumination range when capturing the next frame.

[0098]

[0099] In the above formula (3), R next R0 represents the radius of the visible light control range when the next frame is captured; R0 represents the radius of the visible light control range when the previous frame is captured.

[0100] Control the intensity of visible light during the next frame capture to y next The radius of the visible light irradiation range is controlled to be R. next .

[0101] The beneficial effects of the above technical solution are as follows: Using the above formula (1), the brightness distribution state of the previous frame image is obtained according to the RGB value of the previous frame image, so as to know the brightness state on the actual object and provide a basis for subsequent control; then using the above formula (2), the brightness distribution state of the previous frame image and the light intensity of visible light when the previous frame image is captured are used to control the light intensity of visible light when the next frame is captured, so as to ensure that the image of each frame will not be greatly affected by the lighting problem of visible light; finally using the above formula (3), the radius of the irradiation range of visible light when the next frame is captured is controlled according to the brightness distribution state of the previous frame image, so as to compare the controlled irradiation radius of visible light with the obtained high brightness distribution radius and intelligently adjust the irradiation radius of visible light when the next frame image is captured, so as to ensure that the image of each frame can be illuminated by visible light as much as possible and ensure the dynamic uniformity of the system.

[0102] Preferably, the event judgment and processing module determines whether a fire safety incident or an item placement safety incident has occurred in the kitchen based on the heat source presence status information and the item presence status information; and based on the judgment result, returns an event occurrence notification message to the IoT platform, specifically including:

[0103] Based on the three-dimensional spatial occupancy information of the fire source and the placement information of the fiber fabric, determine whether the distance between the outermost boundary of the fiber fabric and the fire source is less than a preset distance threshold; if not, determine that no fire safety incident has occurred in the kitchen; if not, then based on the temperature distribution information, determine whether the temperature value of the outermost boundary of the fire source is greater than a preset temperature threshold; if not, determine that no fire safety incident has occurred in the kitchen; if so, determine that a fire safety incident has occurred in the kitchen.

[0104] Based on the knife's placement information, determine whether the knife is placed in a preset space area; if so, determine whether no item placement safety incident has occurred in the kitchen; if not, then based on the blade orientation information, determine whether the knife's blade is currently exposed; if not, determine whether no item placement safety incident has occurred in the kitchen; if so, determine whether an item placement safety incident has occurred in the kitchen.

[0105] If a fire safety incident or an item placement safety incident occurs, an event notification message containing the location information of the incident will be returned to the IoT platform.

[0106] The beneficial effects of the above technical solution are as follows: By using the three-dimensional spatial occupancy information and temperature distribution information of the fire source, as well as the placement information of the fabric, as a basis, it can be determined whether the fabric is too close to the fire source or whether the temperature of the fire source is too high, thus posing a potential fire safety incident. Furthermore, based on the placement information and blade orientation information of knives, it can be determined whether improper placement of knives poses a potential risk of injury. In the event of a fire safety incident or a placement safety incident, a notification message containing the corresponding location information of the incident is promptly returned to the IoT platform, facilitating subsequent targeted alarm or fire extinguishing operations based on the location of the incident within the kitchen environment.

[0107] Preferably, the terminal control module sends corresponding control commands to the alarm terminal or fire protection terminal connected to the IoT platform based on the parsing result of the event notification message, thereby controlling the working status of the alarm terminal or fire protection terminal. Specifically, this includes:

[0108] The location information corresponding to the fire safety incident or the safe placement of items is extracted from the incident notification message. This information is then used to send an audible alarm control command to the alarm terminal or a water spray control command to the fire terminal, thereby controlling the audible alarm status of the alarm terminal or the water spray extinguishing status of the fire terminal.

[0109] The beneficial effects of the above technical solution are as follows: when a safety incident occurs involving the placement of items, an audible alarm control command is sent to the alarm terminal, so that the alarm terminal can play a voice alarm message corresponding to the location where the safety incident occurred; when a fire safety incident occurs, a water spray control command is sent to the fire-fighting terminals such as sprinkler heads installed inside the kitchen, so that the sprinkler heads can aim and spray water to extinguish the fire at the location where the fire safety incident occurred.

[0110] As can be seen from the above embodiments, the IoT-based kitchen monitoring and management method and system captures and analyzes thermal infrared and visible light images of the kitchen's internal environment to obtain information on the current state of heat sources and items within the kitchen. This information is used to determine whether a fire safety incident or an item placement safety incident has occurred in the kitchen, and an event notification message is returned to the IoT platform. Based on the parsing results of the event notification message, corresponding control commands are sent to the alarm terminal or fire terminal connected to the IoT platform to control the working status of the alarm terminal or fire terminal. The system performs real-time visual monitoring of the kitchen's internal environment using two different wavelengths: thermal infrared and visible light. This obtains real-time visual monitoring data of the kitchen's internal environment, comprehensively calibrates the state of heat sources and items within the kitchen, and accurately and promptly determines whether a safety incident has occurred in the kitchen. This facilitates targeted alarm and fire extinguishing operations, effectively reducing the misjudgment rate of kitchen safety incidents and improving the reliability of kitchen internal environment monitoring.

[0111] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A kitchen monitoring and management method based on the Internet of Things, characterized in that, It includes the following steps: Step S1: Take thermal infrared and visible light images of the kitchen interior environment to obtain thermal infrared and visible light images of the kitchen interior environment; analyze and process the thermal infrared and visible light images to determine the current heat source presence status information and item presence status information inside the kitchen. Step S2: Based on the heat source presence status information and the item presence status information, determine whether a fire safety incident or an item placement safety incident has occurred inside the kitchen; and based on the determination result, return an event occurrence notification message to the IoT platform; Step S3: Based on the parsing result of the event notification message, send corresponding control commands to the alarm terminal or fire terminal connected to the Internet of Things platform to control the working status of the alarm terminal or the fire terminal. The process of using a visible light camera module to capture images of the kitchen interior under visible light includes, in step S1: adjusting the visible light intensity and illuminated area of ​​the visible light from the camera module based on the previous frame image captured by the camera module, including: Step A1: Using formula (1) below, obtain the brightness distribution of the captured previous frame image based on its RGB values. In the above formula (1), Y(i,j) represents the brightness value at the position of the i-th row and j-th column in the image matrix of the previous frame image; [R(i,j),G(i,j),B(i,j)] represents the RGB value at the position of the i-th row and j-th column in the image matrix of the previous frame image; int[] represents rounding the value in the parentheses to the nearest integer; Y(a,b) represents the brightness value at the position of the a-th row and b-th column in the image matrix of the previous frame image. The image matrix representing the previous frame image is the first... Line number The brightness value at the column position; m represents the total number of pixels in any row of the image matrix of the previous frame image; n represents the total number of pixels in any column of the image matrix of the previous frame image; This means substituting the values ​​of a from 1 to n and the values ​​of b from 1 to m into Y(a,b) to obtain the result that satisfies... In the case of Find the values ​​of a and b when the minimum value is obtained, and denote them as (a′, b′); According to step A1 above, the brightness value at the i-th row and j-th column position in the image matrix of the previous frame image is Y(i,j), and from the image matrix of the previous frame image... Position is the center of the circle Draw a circle with the line connecting position (a′, b′) as the radius, and denote the radius of the circle as r(Q). Denote the set of position points contained within the circle as Q. The image part contained within the circle is the highlighted part of the image, and the image part outside the circle is the under-bright part of the image. Step A2: Using formula (2) below, based on the brightness distribution of the previous frame image and the visible light intensity when the previous frame image was captured, control the visible light intensity when capturing the next frame. In the above formula (2), y next This indicates the control light intensity of visible light during the next frame capture; ∑ (i,j)∈Q Y(i,j) represents the sum of all Y(i,j) values ​​that satisfy the condition of belonging to set Q; ∑ (i,j)∈Q 1 represents the total number of (i,j) elements belonging to set Q; Step A3: Using the formula (3) below, based on the brightness distribution of the previous frame image, control the radius of the visible light illumination range when capturing the next frame. In the above formula (3), R next R0 represents the radius of the visible light control range when the next frame is captured; R0 represents the radius of the visible light control range when the previous frame is captured. Control the intensity of visible light during the next frame capture to y next The radius of the visible light irradiation range is controlled to be R. next .

2. The kitchen monitoring and management method based on the Internet of Things as described in claim 1, characterized in that: In step S1, thermal infrared and visible light imaging is performed on the kitchen interior environment to obtain thermal infrared and visible light images of the kitchen interior environment; the thermal infrared and visible light images are analyzed and processed to determine the current heat source presence status information and item presence status information inside the kitchen, specifically including: Thermal infrared scanning and visible light scanning were performed on the kitchen interior environment to obtain thermal infrared panoramic images and visible light panoramic images of the kitchen interior environment. The thermal infrared panoramic image is subjected to thermal infrared spectral analysis to obtain the distribution location information of all fire sources in the thermal infrared panoramic image; based on the distribution location information of each fire source, the outermost boundary contour of each fire source in the real environmental space is extracted from the thermal infrared panoramic image to determine the three-dimensional spatial occupancy information of each fire source in the real environmental space; based on the thermal infrared spectral distribution information of the thermal infrared panoramic image, the temperature distribution information of each fire source is determined; and the three-dimensional spatial occupancy information and the temperature distribution information are used as the fire source existence status information. After performing pixel edge sharpening processing on the visible light panoramic image, the placement information of knives and the direction of their blades inside the kitchen, as well as the placement information of the fiber fabrics inside the kitchen, are identified from the visible light panoramic image, and these are used as the existence status information of the items.

3. The kitchen monitoring and management method based on the Internet of Things as described in claim 2, characterized in that: In step S2, based on the heat source presence status information and the item presence status information, it is determined whether a fire safety incident or an item placement safety incident has occurred inside the kitchen. Based on the judgment result, the event notification message returned to the IoT platform specifically includes: Based on the three-dimensional spatial occupancy information of the fire source and the placement information of the fiber fabric, determine whether the distance between the outermost boundary of the fiber fabric and the fire source is less than a preset distance threshold; if not, determine that no fire safety incident has occurred in the kitchen; if not, then based on the temperature distribution information, determine whether the temperature value of the outermost boundary of the fire source is greater than a preset temperature threshold; if not, determine that no fire safety incident has occurred in the kitchen; if yes, determine that a fire safety incident has occurred in the kitchen. Based on the knife placement information, determine whether the knife is placed in a preset space area; if yes, determine that no item placement safety incident has occurred in the kitchen; if no, then based on the blade orientation information, determine whether the blade is currently exposed; if no, determine that no item placement safety incident has occurred in the kitchen; if yes, determine that an item placement safety incident has occurred in the kitchen. If a fire safety incident or an item placement safety incident occurs, an event notification message containing the location information of the incident will be returned to the IoT platform.

4. The kitchen monitoring and management method based on the Internet of Things as described in claim 3, characterized in that: In step S3, based on the parsing result of the event notification message, corresponding control commands are sent to the alarm terminal or fire terminal connected to the IoT platform to control the working status of the alarm terminal or the fire terminal. Specifically, this includes: The location information corresponding to the fire safety event or the safe placement of items is extracted from the event notification message. This information is then used to send an audible alarm control command to the alarm terminal or a water spray control command to the fire terminal, thereby controlling the audible alarm status of the alarm terminal or the water spray extinguishing status of the fire terminal.

5. A kitchen monitoring and management system based on the Internet of Things, characterized in that: It includes: A thermal infrared camera module is used to capture thermal infrared images of the kitchen interior environment. Visible light camera module, which is used to capture visible light images of the kitchen interior environment to obtain visible light images of the kitchen interior environment; The image analysis module is used to analyze and process the thermal infrared image and the visible light image to determine the current heat source presence status information and item presence status information inside the kitchen. The event judgment and processing module is used to determine whether a fire safety incident or an item placement safety incident has occurred in the kitchen based on the heat source presence status information and the item presence status information; and to return an event occurrence notification message to the Internet of Things platform based on the judgment result. The terminal control module is used to send corresponding control commands to the alarm terminal or fire terminal connected to the Internet of Things platform according to the parsing result of the event occurrence notification message, so as to control the working status of the alarm terminal or the fire terminal. The visible light camera module also adjusts the intensity and area of ​​visible light illumination based on the captured image from the previous frame. Step A1: Using formula (1) below, obtain the brightness distribution of the captured previous frame image based on its RGB values. In the above formula (1), Y(i,j) represents the brightness value at the position of the i-th row and j-th column in the image matrix of the previous frame image; [R(i,j),G(i,j),B(i,j)] represents the RGB value at the position of the i-th row and j-th column in the image matrix of the previous frame image; int[] represents rounding the value in the parentheses to the nearest integer; Y(a,b) represents the brightness value at the position of the a-th row and b-th column in the image matrix of the previous frame image. The image matrix representing the previous frame image is the first... Line number The brightness value at the column position; m represents the total number of pixels in any row of the image matrix of the previous frame image; n represents the total number of pixels in any column of the image matrix of the previous frame image; This means substituting the values ​​of a from 1 to n and the values ​​of b from 1 to m into Y(a,b) to obtain the result that satisfies... In the case of Find the values ​​of a and b when the minimum value is obtained, and denote them as (a′, b′); According to step A1 above, the brightness value at the i-th row and j-th column position in the image matrix of the previous frame image is Y(i,j), and from the image matrix of the previous frame image... Position is the center of the circle Draw a circle with the line connecting position (a′, b′) as the radius, and denote the radius of the circle as r(Q). Denote the set of position points contained within the circle as Q. The image part contained within the circle is the highlighted part of the image, and the image part outside the circle is the under-bright part of the image. Step A2: Using formula (2) below, based on the brightness distribution of the previous frame image and the visible light intensity when the previous frame image was captured, control the visible light intensity when capturing the next frame. In the above formula (2), y next This indicates the control light intensity of visible light when the next frame is captured; ∑ (i,j)∈Q Y(i,j) means adding all Y(i,j) that satisfy the condition of (i,j) belonging to set Q; ∑ (i,j)∈Q 1 represents the total number of (i,j) elements belonging to set Q; Step A3: Using the formula (3) below, based on the brightness distribution of the previous frame image, control the radius of the visible light illumination range when capturing the next frame. In the above formula (3), R next R0 represents the radius of the visible light control range when the next frame is captured; R0 represents the radius of the visible light control range when the previous frame is captured. Control the intensity of visible light during the next frame capture to y next The radius of the visible light irradiation range is controlled to be R. next .

6. The IoT-based kitchen monitoring and management system as described in claim 5, characterized in that: The thermal infrared camera module captures thermal infrared images of the kitchen's interior environment to obtain thermal infrared images of the kitchen's interior environment, specifically including: Thermal infrared scanning was performed on the kitchen interior to obtain a panoramic thermal infrared image of the kitchen interior. The visible light camera module captures visible light images of the kitchen's interior environment, specifically including: The visible light scanning image of the kitchen interior environment is obtained by scanning the kitchen interior environment. The image analysis module analyzes and processes the thermal infrared image and the visible light image to determine the current state of heat sources and items inside the kitchen. Specifically, this includes: performing thermal infrared spectral analysis on the thermal infrared panoramic image to obtain the distribution location information of all fire sources in the thermal infrared panoramic image; using the distribution location information of each fire source as a reference, extracting the outermost boundary contour of each fire source in the real environmental space from the thermal infrared panoramic image to determine the three-dimensional spatial occupancy information of each fire source in the real environmental space; determining the temperature distribution information of each fire source based on the thermal infrared spectral distribution information of the thermal infrared panoramic image; and using the three-dimensional spatial occupancy information and the temperature distribution information as the fire source existence status information. After performing pixel edge sharpening processing on the visible light panoramic image, the placement information of knives and the direction of their blades inside the kitchen, as well as the placement information of the fiber fabrics inside the kitchen, are identified from the visible light panoramic image, and these are used as the existence status information of the items.

7. The IoT-based kitchen monitoring and management system as described in claim 6, characterized in that: The event judgment and processing module determines whether a fire safety incident or an item placement safety incident has occurred in the kitchen based on the heat source presence status information and the item presence status information. and Based on the judgment result, the event notification message returned to the IoT platform specifically includes: determining whether the distance between the outermost boundary of the fiber fabric and the fire source is less than a preset distance threshold based on the three-dimensional spatial occupancy information of the fire source and the placement information of the fiber fabric; if not, it is determined that no fire safety event has occurred in the kitchen; if not, it is further determined whether the temperature value of the outermost boundary of the fire source is greater than a preset temperature threshold based on the temperature distribution information; if not, it is determined that no fire safety event has occurred in the kitchen; if so, it is determined that a fire safety event has occurred in the kitchen. Based on the knife placement information, determine whether the knife is placed in a preset space area; if yes, determine that no item placement safety incident has occurred in the kitchen; if no, then based on the blade orientation information, determine whether the blade is currently exposed; if no, determine that no item placement safety incident has occurred in the kitchen; if yes, determine that an item placement safety incident has occurred in the kitchen. If a fire safety incident or an item placement safety incident occurs, an event notification message containing the location information of the incident will be returned to the IoT platform.

8. The IoT-based kitchen monitoring and management system as described in claim 7, characterized in that: The terminal control module, based on the parsing result of the event notification message, sends corresponding control commands to the alarm terminal or fire terminal connected to the IoT platform, thereby controlling the working status of the alarm terminal or the fire terminal. Specifically, this includes: The location information corresponding to the fire safety event or the safe placement of items is extracted from the event notification message. This information is then used to send an audible alarm control command to the alarm terminal or a water spray control command to the fire terminal, thereby controlling the audible alarm status of the alarm terminal or the water spray extinguishing status of the fire terminal.

Images