Home fire monitoring system based on internet of things cloud platform

Abstract

The application relates to a home fire monitoring system based on an Internet of Things cloud platform, and belongs to the field of fire monitoring. The system comprises smoke, temperature, illumination, gas and other sensors, a camera, a fire alarm, an NVS gateway and a client computer installed indoors, wherein the client computer is connected with a GPS module, a cloud platform and a terminal; the client computer comprises: an acquisition module for acquiring image information, smoke concentration information, temperature information, illumination information and gas concentration information in the room; a first alarm module for generating first alarm information when the smoke concentration information, the temperature information, the illumination information and the gas concentration information are abnormal; a flame suspected area determination module for determining a flame suspected area; a smoke suspected area determination module for determining a smoke suspected area; a correlation judgment module for judging whether the two suspected areas are correlated; and a second alarm module for generating second alarm information when the two suspected areas are correlated. The application improves the accuracy of home fire alarm.

Description

Technical Field

[0001] This application relates to the technical field of fire monitoring, and in particular to a home fire monitoring system based on an Internet of Things (IoT) cloud platform. Background Technology

[0002] It is essential to detect fires as early as possible and extinguish them in their initial stages.

[0003] With the popularization of the Internet of Things and cloud technology, fire management in residential communities is becoming increasingly intelligent. Fire detectors have entered every household. Generally, various sensors are installed in buildings and indoor spaces to determine whether a fire has occurred by judging whether smoke, temperature, or other factors exceed the limits.

[0004] However, the presence of smoke does not necessarily indicate a fire. For example, cooking fumes, dust rising from a room that has not been cleaned for a long time, and steam from a shower can all cause false alarms from smoke detectors, resulting in low accuracy of fire alarms. Summary of the Invention

[0005] To improve the accuracy of home fire alarms, this application provides a home fire monitoring system based on an Internet of Things (IoT) cloud platform.

[0006] Firstly, this application provides a home fire monitoring system based on an Internet of Things (IoT) cloud platform, employing the following technical solution:

[0007] It includes indoor smoke sensors, temperature sensors, light intensity sensors, gas sensors, cameras, and fire alarms, as well as an NVS gateway and a client computer. The smoke sensors, temperature sensors, light intensity sensors, gas sensors, cameras, and fire alarms are all connected to the NVS gateway. The NVS gateway is wirelessly connected to the client computer, which is connected to a GPS module and a cloud platform, and is also wirelessly connected to a terminal.

[0008] The client computer includes:

[0009] The acquisition module is used to acquire indoor image information, smoke concentration information, temperature information, illuminance information, and gas concentration information;

[0010] The first alarm module is used to generate first alarm information that causes the fire alarm to sound when at least one of the smoke concentration information, temperature information, illuminance information and gas concentration information is abnormal.

[0011] The suspected flame area determination module is used to analyze the color, brightness, and shape of the image information to determine the suspected flame area;

[0012] The smoke-suspected area determination module is used to compare the image information with a preset background image to determine the smoke-suspected area;

[0013] The correlation judgment module is used to determine whether the suspected flame area and the suspected smoke area are related;

[0014] The second alarm module is used to generate a second alarm message to be sent to the terminal when the relevant judgment module determines that it is true.

[0015] By adopting the above technical solution, the sensors installed indoors send the detected information to the client computer through the NVS gateway, and the cameras send the captured indoor image information to the client computer through the NVS gateway. After receiving the detection information and image information, the acquisition module triggers the fire alarm when at least one of the detection information is abnormal. Then, the flame suspected area determination module and the smoke suspected area determination module determine the flame suspected area and the smoke suspected area respectively based on the image information. The correlation judgment module determines whether the two areas are related, that is, whether the smoke is generated by the combustion of an object. If they are related, it can be determined that a fire has occurred indoors, and then the alarm information generated by the second alarm module is sent to the terminal to prompt the residents or community security to put out the fire. Through further analysis of the smoke, the accuracy of the fire alarm is improved.

[0016] Furthermore, the suspected flame area determination module includes:

[0017] The first pixel point determination submodule is used to determine the brightness information of each pixel point in the image information and filter out the first pixel point whose brightness reaches the brightness limit.

[0018] The pixel group determination submodule is used to aggregate and group the first pixel to obtain multiple pixel groups;

[0019] The first similarity determination submodule is used to compare the color of each group of pixels with the color of the flame to determine the first similarity;

[0020] The second similarity determination submodule is used to compare the shape of each group of pixels with the shape of a flame to determine the second similarity.

[0021] The second image information acquisition submodule is used to determine whether both the first similarity and the second similarity have reached a preset value; if so, it acquires second image information located at multiple times after the current time; otherwise, it determines that there is no suspected flame area.

[0022] The second image information pixel group determination submodule is used to determine the pixel group in the second image information;

[0023] The flame flickering judgment submodule is used to determine whether the image has flame flickering characteristics based on the characteristics of the pixel groups in the current image information and each second image information.

[0024] The flame suspected region determination submodule is used to determine the region corresponding to the pixel group as the flame suspected region when the first similarity and the second similarity reach a preset value and the region has flame flickering characteristics.

[0025] By adopting the above technical solution, the first pixel point determination submodule analyzes the brightness of each pixel in the image information and filters out the first pixel points whose brightness exceeds the limit. The pixel point group determination submodule aggregates the first pixel points to obtain multiple pixel point groups. The first similarity determination submodule determines the first similarity between the color of the pixel point group and the color of the flame. The second similarity determination submodule determines the second similarity between the shape of the pixel point group and the shape of the flame. When both the first similarity and the second similarity reach the preset value, the second image information submodule acquires multiple second image information. The flame flickering judgment submodule analyzes the pixel point groups in the current image information and the second image information to determine whether there are landscape features. If there are flickering features, the pixel point group can be basically identified as a suspected flame area. Therefore, by analyzing multiple factors such as the shape, color, and flame flickering features of the pixel point group, the suspected flame area is determined, improving the recognition accuracy.

[0026] Furthermore, the flame flickering detection submodule includes:

[0027] The top edge determination unit is used to determine the top edge of any group of pixels in each image information.

[0028] An approximate line determination unit is used to determine the approximate line of the top edge.

[0029] The ordinate determination unit is used to determine the ordinate of the intersection point of each of the approximate lines and the same vertical line; the vertical line is the vertical line where the x-coordinate of the first pixel in the image information is located;

[0030] The current image information determination unit is used to determine the first image information as the current image information;

[0031] The vertical coordinate difference calculation unit is used to calculate the vertical coordinate difference between the intersection point in the next image information and the intersection point in the current image information, and to use the next image as the current image;

[0032] The repeated execution unit is used to repeatedly calculate the difference in ordinate between the intersection point in the next image information and the intersection point in the current image information, and to obtain multiple ordinate differences by taking the next image as the current image.

[0033] A feature determination unit is used to determine whether the multiple ordinate differences have the characteristic of positive and negative intersection;

[0034] The first determining unit is used to determine that the flame flashing feature exists when the feature determining unit determines that it is true;

[0035] The second determining unit is used to determine that the flame flickering feature is not present when the feature determining unit determines that it is not present.

[0036] By employing the above technical solution, when determining whether flame flickering characteristics exist, the top edge determination unit determines the top edge of the same pixel group in each image, the approximate line determination unit determines the approximate line of the top edge, and the ordinate determination unit determines the ordinate of the intersection points of each approximate line and the same vertical line. The ordinates of each intersection point are used to measure the displacement of the top edge. The difference between the ordinate of the intersection point in the next image and the ordinate of the intersection point in the current image is calculated, and the change in the difference is used to assess whether flame flickering exists. Therefore, by combining the characteristics of flame flickering and analyzing the pixel groups in the image, a more accurate suspected flame region is obtained.

[0037] Furthermore, the approximate line determination unit is specifically used for:

[0038] Determine the length between the two endpoints of the top edge in the horizontal direction;

[0039] The number of segments is determined based on the length; the longer the length, the more segments are required.

[0040] Based on the number of divisions, the top edge is divided evenly along the horizontal direction to determine each division point;

[0041] Starting from one endpoint, connect it sequentially to each dividing point, and then connect it to the other endpoint to obtain an approximate line of the fixed edge.

[0042] By adopting the above technical solution, the approximate line determination unit first determines the length between the two ends of the top edge in the horizontal direction, then determines the number of segments based on the length, divides the top edge equally according to the number of segments, determines multiple segmentation points, and then connects the top edge and the segmentation points to obtain a rough approximate line. The roughness is determined based on the length of the approximate line to reduce the amount of subsequent calculations, while making the approximate line as similar as possible to the top edge.

[0043] Furthermore, the smoke-suspected area determination module includes:

[0044] The background image acquisition submodule is used to acquire the indoor image corresponding to a preset time before the anomaly occurs as the preset background image;

[0045] The abnormal region determination submodule is used to compare the image information with the preset background image to determine inconsistent abnormal regions;

[0046] The ambiguity determination submodule is used to determine the ambiguity of the abnormal area. When the ambiguity reaches a preset value, the abnormal area is determined to be a suspected smoke area.

[0047] By adopting the above technical solution, the background image acquisition submodule uses the indoor image before the anomaly as a preset background image as a reference standard. The anomaly area determination submodule compares the preset background image with the image information to identify inconsistent anomaly areas. Then, the ambiguity judgment submodule analyzes the ambiguity of the anomaly area and determines whether it is a suspected smoke area based on the magnitude of the ambiguity. Therefore, it can quickly identify suspected smoke areas.

[0048] Furthermore, the abnormal region determination submodule is specifically used for:

[0049] Increase the transparency of the preset background image;

[0050] The preset background image is overlaid with the image information to be compared to identify inconsistent abnormal areas.

[0051] By adopting the above technical solution, when identifying abnormal areas, the abnormal area identification submodule first increases the transparency of the preset background image, and then overlaps the two images to be compared, which can clearly identify inconsistent abnormal areas.

[0052] Furthermore, the relevant judgment module includes:

[0053] The center point determination submodule is used to determine the first center point of the suspected flame area and the second center point of the suspected smoke area;

[0054] The analysis direction determination submodule is used to connect the first center point and the second center point to obtain a line segment, and determine the analysis direction parallel to the line segment;

[0055] The associated region determination submodule is used to determine the associated region located between the suspected flame region and the suspected smoke region;

[0056] The overlapping edge determination submodule is used to determine the first overlapping edge in which the suspected flame area is located in the associated area, and the second overlapping edge in which the suspected smoke area is located in the associated area;

[0057] The analysis point determination submodule is used to determine multiple sets of analysis points along the analysis direction on the first and second overlapping edges.

[0058] The average value calculation submodule is used to determine the spacing between each group of analysis points and calculate the average value of the spacing between each group.

[0059] The relevant determination submodule is used to determine the correlation between the suspected flame area and the suspected smoke area when the average value is less than a preset value;

[0060] The irrelevance determination submodule is used to determine that the suspected flame area and the suspected smoke area are irrelevant when the average value is not less than a preset value.

[0061] By adopting the above technical solution, in order to determine the direction of smoke dispersion, the center point determination submodule determines the center points of the suspected flame area and the suspected smoke area. The analysis direction determination submodule connects the two center points to determine the analysis direction. The associated area determination submodule determines the associated area between the two suspected areas, which is used to determine the first and second overlapping edges of the suspected area and the associated area, and then determine multiple sets of analysis points. The average length of the analysis points can be used to determine the distance between the suspected smoke area and the suspected flame area. When the distance is less than a preset value, the correlation determination submodule can determine that the two suspected areas are related. Therefore, by combining the characteristics of flame and smoke when a fire occurs, it is possible to judge whether the two areas are related and obtain more accurate analysis results.

[0062] Furthermore, the associated region determination submodule is specifically used for:

[0063] Determine the center of the associated region on the line segment, and generate a circular associated region with the initial length as the radius;

[0064] Determine whether the suspected flame area and the suspected smoke area are located within the associated area; if not, increase the radius by one unit and update the associated area.

[0065] Repeat the step of determining whether the suspected flame area and the suspected smoke area are located within the associated area until the determination is yes, and then determine the associated area.

[0066] By adopting the above technical solution, when determining the associated region, the associated region determination submodule first determines the center of the circle, and then generates the associated region with the initial length as the radius. After determining that the suspected flame region and the suspected smoke region are not located in the associated region, the associated region is expanded until the two suspected regions are included, and then the associated region is determined.

[0067] Furthermore, the system also includes:

[0068] The temperature variance calculation module is used to calculate the average temperature based on multiple temperature information corresponding to consecutive time points, and to calculate the temperature variance based on each temperature information and the average temperature.

[0069] The illuminance variance calculation module is used to calculate the average illuminance based on multiple illuminance information corresponding to consecutive time moments, and to calculate the illuminance variance based on each illuminance information and the average illuminance.

[0070] The first ignition source detection module is used to determine that the ignition source is solid when the smoke concentration information is abnormal.

[0071] The second ignition source detection module is used to determine that the ignition source is gas when the illuminance information or gas concentration information is abnormal.

[0072] The third fire source detection module is used to determine that the fire source is a liquid when the illuminance variance or temperature variance is abnormal.

[0073] The fourth fire source detection module is used to determine that a fire source cannot be identified when the temperature difference alone reaches a preset time.

[0074] By adopting the above technical solution, the client computer calculates the temperature variance and illuminance variance, and then combines this with data such as smoke concentration information and gas concentration information to determine the fire source, providing more on-site information and reliable information for timely fire suppression.

[0075] In summary, this application includes at least one of the following beneficial technical effects:

[0076] 1. After receiving various detection information and image information, the acquisition module will trigger the fire alarm when at least one of the detection information is abnormal. Then, the flame suspected area determination module and the smoke suspected area determination module will determine the flame suspected area and the smoke suspected area respectively based on the image information. The correlation judgment module will determine whether the two areas are related, that is, whether the smoke is generated by the combustion of an object. If they are related, it can be determined that a fire has occurred indoors. Then, the alarm information generated by the second alarm module will be sent to the terminal to prompt the residents or community security to put out the fire. Through further analysis of the smoke, the accuracy of the fire alarm will be improved.

[0077] 2. The center point determination submodule determines the center point of the suspected flame area and the suspected smoke area. The analysis direction determination submodule connects the two center points to determine the analysis direction. The associated region determination submodule determines the associated region between the two suspected areas, which is used to determine the first and second overlapping edges of the suspected area and the associated region, and then determine multiple sets of analysis points. The average length of the analysis points can be used to determine the distance between the suspected smoke area and the suspected flame area. When the distance is less than a preset value, the correlation determination submodule can determine that the two suspected areas are related. Therefore, by combining the characteristics of flame and smoke at the time of fire generation, it is possible to judge whether the two areas are related and obtain more accurate analysis results. Attached Figure Description

[0078] Figure 1This is a schematic diagram of the structure of a home fire monitoring system based on an Internet of Things cloud platform in this application embodiment.

[0079] Figure 2 This is a structural block diagram of the client computer in an embodiment of this application.

[0080] Figure 3 This is a structural block diagram of the relevant area in the embodiments of this application.

[0081] Figure descriptions: 1. Smoke sensor; 2. Temperature sensor; 3. Illuminance sensor; 4. Gas sensor; 5. Camera; 6. Fire alarm; 7. NVS gateway; 8. GPS module; 9. Terminal; 10. Cloud platform; 100. Client computer; 101. Acquisition module; 102. First alarm module; 103. Flame suspected area determination module; 104. Smoke suspected area determination module; 105. Relevant judgment module; 106. Second alarm module. Detailed Implementation

[0082] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0083] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article, unless otherwise specified, generally indicates that the preceding and following related objects have an "or" relationship.

[0084] This application discloses a home fire monitoring system based on an Internet of Things (IoT) cloud platform. (Refer to...) Figure 1 It includes a smoke sensor 1, a temperature sensor 2, a light intensity sensor 3, a gas sensor 4, a camera 5, and a fire alarm 6 installed indoors, as well as an NVS gateway 7 and a client computer 100. The smoke sensor 1, temperature sensor 2, light intensity sensor 3, gas sensor 4, camera 5, and fire alarm 6 are all connected to the NVS gateway 7. The NVS gateway 7 is wirelessly connected to the client computer 100. The client computer 100 is connected to a GPS module 8 and a cloud platform 10, and is also wirelessly connected to a terminal 9.

[0085] Specifically, each sensor and fire alarm 6 is wirelessly connected to the NVS (Network Video System) gateway via the ZigBee wireless protocol, and the camera 5 is connected to the NVS gateway 7 via a BNC (Bayonet Nut Connector) interface. The NVS gateway 7 communicates with the client computer 100 via a local area network.

[0086] Wireless sensors at the IoT sensing layer collect data such as smoke concentration, temperature, illuminance, and gas concentration, and send it to the NVS gateway 7 at the network layer via a Zigbee network. The NVS gateway 7 then transmits the data to the client computer 100 at the network layer via local area network communication. The client computer 100 processes and analyzes the data, determines the fire situation, and uploads the data and analysis results to the cloud platform 10. This triggers the indoor fire alarm 6 to sound an alarm or sends an alarm message to the terminal 9. The terminal 9 can be a resident's terminal, a security room terminal, or a fire alarm terminal, allowing the security room and fire department to receive alarm information even when the resident is not home. In the event of a fire, the resident's, security room's, and fire department's terminals can also view the scene captured by cameras through the cloud platform.

[0087] Among them, reference Figure 2 Client computer 100 includes:

[0088] The acquisition module 101 is used to acquire indoor image information, smoke concentration information, temperature information, illuminance information and gas concentration information.

[0089] Specifically, camera 5 captures images of the interior to obtain indoor image information, smoke sensor 1 detects the indoor smoke concentration to obtain smoke concentration information, temperature sensor 2 detects the indoor temperature to obtain temperature information, illuminance sensor 3 detects the indoor illuminance to obtain illuminance information, and gas sensor 4 detects the indoor gas concentration to obtain gas concentration information; client computer 100 receives image information, smoke concentration information, temperature information, illuminance information, and gas concentration information sent by NVS gateway 7.

[0090] The first alarm module 102 is used to generate a first alarm message when at least one of the smoke concentration information, temperature information, illuminance information and gas concentration information is abnormal.

[0091] Specifically, the client computer 100 has preset limits for smoke concentration, temperature, illuminance, and gas concentration. When setting each limit, the client computer 100 obtains the limit values ​​through big data.

[0092] When the smoke concentration exceeds the smoke concentration limit, the temperature exceeds the temperature limit, the illuminance exceeds the illuminance limit, or the gas concentration exceeds the gas concentration limit, the corresponding abnormality is determined. When an abnormality exists, the client computer 100 generates a first alarm message. At this time, the client computer 100 activates the fire alarm 6, prompting residents inside the room to check for a fire and take appropriate action. To enhance safety, if the abnormality persists after a preset time, the client computer 100 sends an alarm message to the terminal 9, instructing residents to contact security and fire departments if no one is inside or if the fire has not been controlled in time.

[0093] The flame suspected area determination module 103 is used to analyze the color, brightness and shape of image information to determine the flame suspected area.

[0094] To further explain, the suspected flame area determination module 103 includes:

[0095] The first pixel determination submodule is used to determine the brightness information of each pixel in the image information and filter out the first pixel whose brightness reaches the brightness limit.

[0096] Specifically, the image information captured by camera 5 is a color image. For a color image, each pixel has three components (RGB), and the value of each component is between 0 and 255. To determine the brightness of the color image, a mapping function is used to convert the color image into a grayscale image, and then the brightness is determined based on the grayscale values. The RGB to grayscale conversion can be done using any of the following methods: selecting any channel as gray; using the maximum value in RGB as gray; using the average value in RGB as gray; or using the weighted average value in RGB as gray.

[0097] In a grayscale image, each pixel has only one component, ranging from 0 to 255, where 0 represents black (the darkest) and 255 represents white (the brightest). A larger component value indicates greater brightness. The client computer 100 then obtains the component value of each pixel, i.e., the brightness information, and compares each brightness value with a brightness limit. Pixels with a brightness limit greater than or equal to the limit are selected as the first pixel. The brightness limit is chosen based on the actual brightness of the flame in the image, aiming to filter out the first pixel corresponding to the flame in the image information as accurately as possible.

[0098] The pixel group determination submodule is used to group the first pixel into multiple pixel groups.

[0099] Specifically, the client computer 100 starts from any first pixel point, groups it with adjacent first pixels, and then groups it with the next adjacent pixel point, until there are no adjacent first pixels, thus obtaining a pixel group; then it starts from another first pixel point that does not belong to the previous pixel group and repeats the above steps to obtain another pixel group; until there are no ungrouped first pixels in the image information, thus obtaining at least one pixel group.

[0100] The first similarity determination submodule is used to compare the color of each pixel group with the color of the flame to determine the first similarity.

[0101] Specifically, the client computer 100 uses the RGB values ​​of each pixel to determine its color. Since the color of a pixel is represented by these three RGB values, the pixel matrix corresponds to three color vector matrices: the R matrix (500 * 338), the G matrix (500 * 338), and the B matrix (500 * 338). If the values ​​in the first row and first column of each matrix are R: 240, G: 223, B: 204, then the color of this pixel is (240, 223, 204).

[0102] The electronic device calculates the average RGB value of each pixel in each pixel group as the color of the pixel group. Simultaneously, it pre-determines the RGB value of the flame in the image information, and then calculates the distance between the two three-dimensional vectors. Dividing the distance by a unit length yields the first similarity score. The unit length can be 1. The longer the distance, the lower the first similarity score.

[0103] The second similarity determination submodule is used to compare the shape of each group of pixels with the shape of the flame to determine the second similarity.

[0104] Specifically, the client computer 100 obtains the edges of the pixel group, obtains the shape of the pixel group based on the closed edges, and then compares the shape with any preset flame shape, and can use the shape overlap as the second similarity.

[0105] The second image information acquisition submodule is used to determine whether both the first similarity and the second similarity have reached preset values; if so, it acquires the second image information located at multiple times after the current time; otherwise, it determines that there is no suspected flame area.

[0106] Specifically, when the first and second similarity scores reach preset values, multiple second image pieces are acquired for analysis to further verify whether it is a flame. To speed up the analysis, the number of second image pieces need not be excessive; two to three are sufficient. If the first and second similarity scores do not reach the preset values, the likelihood of it being a flame is low.

[0107] The second image information pixel group determination submodule is used to determine the pixel group in the second image information.

[0108] Specifically, the client computer 100 uses the same method to determine the pixel group in the second image information.

[0109] The flame flickering judgment submodule is used to determine whether the image has flame flickering characteristics based on the characteristics of the pixel groups in the current image information and each second image information.

[0110] Furthermore, the flame flickering detection submodule includes:

[0111] The top edge determination unit is used to determine the top edge of any group of pixels in each image information.

[0112] Specifically, the client computer 100 identifies groups of pixels in the same region within each image information as the same pixel group, and selects the top edge of the largest group of pixels in that same pixel group.

[0113] The approximate line determination unit is used to determine the approximate line of the top edge, specifically in the following steps:

[0114] Determine the length between the two endpoints of the top edge in the horizontal direction; determine the number of divisions based on the length; the longer the length, the more divisions are needed; divide the top edge evenly in the horizontal direction according to the number of divisions, and determine each division point; starting from one endpoint, connect it to each division point in sequence, and connect it to the other endpoint to obtain an approximate line of the fixed edge.

[0115] The client computer 100 has multiple preset length levels, each corresponding to a number of divisions. As shown in the figure, the client computer 100 first determines the length between the two endpoints A and B, compares this length with the preset length levels, and then determines the number of divisions n. Next, it divides the fixed edge between A and B into n equal parts, with the points on the top edge of the fixed edge on the dividing lines being the division points Mi (i=1 to n-1). Starting from point A, it connects each division point Mi sequentially until it reaches point B, thus obtaining a broken line, i.e., an approximate line. The approximate line roughly represents the outline of the top edge, reducing the amount of computation and improving the analysis speed.

[0116] The ordinate determination unit is used to determine the ordinate of the intersection point of each approximate line and the same vertical line; the vertical line is the vertical line where the x-coordinate of the first pixel in the image information is located.

[0117] Specifically, the client computer 100 determines the x-coordinate of the first pixel that appears in the image information, draws a vertical line along the x-coordinate, and then determines the coordinates of the intersection point of the approximate line and the vertical line. The first pixel that appears may be the center of the fire source, which can be used as a standard to measure changes in the approximate line.

[0118] The current image information determination unit is used to take the first image information as the current image information.

[0119] The vertical coordinate difference calculation unit is used to calculate the vertical coordinate difference between the intersection point in the next image information and the intersection point in the current image information, and to use the next image as the current image.

[0120] The repeated execution unit is used to repeatedly calculate the difference in ordinate between the intersection point in the next image information and the intersection point in the current image information, and to obtain multiple ordinate differences by using the next image as the current image.

[0121] The feature judgment unit is used to determine whether the differences between multiple vertical coordinates have the characteristic of positive and negative intersection.

[0122] Specifically, due to the influence of airflow, the flame will jump when burning. The most obvious surface is that the flame will jump up and down, and the vertical coordinates of the intersections of the approximate lines and the vertical line will move up and down.

[0123] Among them, positive and negative crossover means that positive and negative values ​​appear alternately in each difference. If there are 5 differences, as long as they include both positive and negative values, they can be considered to have crossover. However, if there are only positive values ​​or only negative values, they are not considered to have crossover.

[0124] The first determining unit is used to determine that the flame flashing feature exists when the feature determining unit determines that it is true.

[0125] The second determining unit is used to determine that the flame flickering feature is not present when the feature determining unit determines that it is not present.

[0126] Specifically, when positive and negative values ​​appear alternately in the difference, the flame will obviously jump up and down, and it can be determined that the flame is flickering; when there is no cross between positive and negative values, the flickering characteristic of the flame cannot be clearly identified.

[0127] The flame suspected region determination submodule is used to determine the region corresponding to the pixel group as the flame suspected region when the first similarity and the second similarity reach a preset value and the region has flame flickering characteristics.

[0128] Specifically, the client computer 100 analyzes the brightness, color, and flame flicker of the image information to determine if it is a flame, and then identifies the suspected flame area.

[0129] The smoke-suspected area determination module 104 is used to compare image information with a preset background image to determine the smoke-suspected area, including:

[0130] The background image acquisition submodule is used to acquire the indoor image corresponding to a preset time before the anomaly occurs as the preset background image.

[0131] The abnormal region identification submodule is used to compare image information with a preset background image to identify inconsistent abnormal regions.

[0132] Specifically, the client computer 100 increases the transparency of a preset background image, then overlays the preset background image with the image information to be compared to identify inconsistent and abnormal areas. The image information to be compared is the image information to be analyzed.

[0133] The ambiguity judgment submodule is used to determine the ambiguity of abnormal areas. When the ambiguity reaches a preset value, the abnormal area is determined to be a suspected smoke area.

[0134] Specifically, the client computer 100 can use a gray-scale variance algorithm to calculate the blurriness of the abnormal region. This involves using the average gray-scale value of all pixels in the abnormal region as a reference, calculating the sum of the squares of the differences in the gray-scale values ​​of each pixel in the abnormal region, and then standardizing the result using the total number of pixels in the abnormal region to obtain the blurriness. This represents the average degree of gray-scale variation in the image; the greater the average degree of gray-scale variation, the clearer the image; conversely, the smaller the average degree of gray-scale variation, the blurrier the image.

[0135] When the calculated blurriness reaches a preset value, the abnormal area is identified as a suspected smoke area. The preset value used to determine the blurriness is set based on the actual blurriness of the image when the smoke disperses.

[0136] The relevant judgment module 105 is used to determine whether the suspected flame area and the suspected smoke area are related.

[0137] Specifically, smoke from a fire is connected to the flame. If the smoke is produced by the combustion of the flame, then if the suspected flame area and the suspected smoke area are connected, a correlation is established, confirming that the smoke was produced by the combustion of the flame. However, in a kitchen, when gas is burning, the flame may also produce smoke. But during normal combustion, there are cooking utensils between the flame and the smoke, so the suspected flame area and the suspected smoke area are not connected. In a kitchen, when a fire gets out of control, such as when a stove catches fire, the suspected flame area and the suspected smoke area are connected due to the combustion of solids. Therefore, whether the two areas are related can be used to distinguish whether a fire has occurred.

[0138] To determine relevance, the relevance judgment module 105 includes:

[0139] The center point determination submodule is used to determine the first center point of the suspected flame area and the second center point of the suspected smoke area.

[0140] Specifically, the client computer 100 determines the centroid of the suspected flame area as the first center point and the centroid of the suspected smoke area as the second center point.

[0141] The analysis direction determination submodule is used to connect the first center point and the second center point to obtain a line segment, and determine the analysis direction parallel to the line segment.

[0142] Specifically, the direction of analysis can be seen as the direction in which the smoke disperses.

[0143] The associated region determination submodule is used to determine the associated region located between the suspected flame region and the suspected smoke region.

[0144] Specifically, the client computer 100 determines the center of the associated region on the line segment where the analysis direction is located, generates a circular associated region with the initial length as the radius, and determines whether the suspected flame region and the suspected smoke region are located within the associated region; if not, the radius is increased by a unit value and the associated region is updated; the step of determining whether the suspected flame region and the suspected smoke region are located within the associated region is repeated until the determination is yes, and the associated region is determined.

[0145] The overlapping edge determination submodule is used to determine the first overlapping edge in the associated region where the suspected flame area is located, and the second overlapping edge in the associated region where the suspected smoke area is located.

[0146] Reference Figure 3 In this context, A represents the suspected flame region, B represents the suspected smoke region, and C represents the associated region; MN represents the first overlapping edge, and OP represents the second overlapping edge.

[0147] The analysis point determination submodule is used to determine multiple sets of analysis points along the analysis direction on the first and second overlapping edges.

[0148] Specifically, the client computer 100 determines multiple straight lines parallel to the analysis direction, and takes the intersection points of the straight lines with the first and second coincident sides as a set of analysis points, thereby obtaining multiple sets of analysis points.

[0149] The average value calculation submodule is used to determine the spacing between each group of analysis points and calculate the average value of the spacing between each group.

[0150] The relevant determination submodule is used to determine the correlation between suspected flame areas and suspected smoke areas when the average value is less than a preset value;

[0151] The irrelevant determination submodule is used to determine that the suspected flame area and the suspected smoke area are irrelevant when the average value is not less than a preset value.

[0152] Specifically, the preset value is set based on the actual distance between the flame and the smoke. In the image, when the flame is closer to camera 5, the distance between the flame and the smoke is larger; while when the flame is farther from camera 5, the distance between the flame and the smoke is smaller. When the electronic device selects an appropriate preset value, regardless of the distance between the flame and camera 5, and the average value is less than the preset value, it can be determined that the two suspected areas are related.

[0153] Therefore, when the average value is less than the preset value, the suspected flame area and the suspected smoke area are relatively close, and it can be inferred that the smoke is produced by the combustion of flames; while when the average value is not less than the preset value, the suspected flame area and the suspected smoke area are relatively far apart, and the smoke is not necessarily caused by a fire.

[0154] The second alarm module 106 is used to generate a second alarm message when the relevant judgment module 105 determines that it is true.

[0155] Specifically, when the client computer 100 determines that the suspected flame area is related to the suspected smoke area, it confirms that a fire has indeed occurred and generates a second alarm message. After generating the second alarm message, the client computer 100 sends the alarm message to the user's terminal 9 via Ethernet, and also sends the GPS information and alarm message to the security room and fire station terminals 9. This facilitates security and fire personnel in knowing the location of the fire.

[0156] In another possible implementation, to provide more fire-related information, offer more effective reference information for firefighting, and improve firefighting efficiency, the system also includes:

[0157] The temperature variance calculation module is used to calculate the average temperature based on multiple temperature information corresponding to consecutive time points, and to calculate the temperature variance based on each temperature information and the average temperature.

[0158] Specifically, after detecting a temperature exceeding the limit, the client computer 100 acquires multiple temperature data points corresponding to consecutive moments following the moment the limit was exceeded, and then calculates the temperature variance. Temperature Variance

[0159] The illuminance variance calculation module is used to calculate the average illuminance based on multiple illuminance information corresponding to consecutive time points, and to calculate the illuminance variance based on each illuminance information and the average illuminance.

[0160] Specifically, after detecting that the illuminance exceeds the limit, the client computer 100 acquires multiple illuminance information corresponding to consecutive subsequent moments starting from the moment of exceeding the limit, and then calculates the illuminance variance.

[0161] The first ignition source detection module is used to determine that the ignition source is solid when the smoke concentration information is abnormal.

[0162] The second ignition source detection module is used to determine that the ignition source is gas when the illuminance information or gas concentration information is abnormal.

[0163] The third fire source detection module is used to determine that the fire source is a liquid when the variance of illuminance or temperature is abnormal.

[0164] The fourth fire source detection module is used to determine that a fire source cannot be identified when the temperature difference alone reaches a preset time.

[0165] The client computer 100 determines the source of the fire based on its characteristics and abnormal information. For example, when a solid burns, there is a large amount of smoke, so an abnormal smoke concentration is detected first; if a gas leak occurs and gas combustion occurs, the flame will form relatively quickly; liquid combustion is a comprehensive manifestation of combustible liquid turning into steam at high temperatures, and the steam then undergoing a combustion reaction, so the temperature may rise rapidly or flashover may occur, which is then monitored through the variance of illuminance and temperature; however, if only an abnormal temperature variance is detected and it persists for a preset time, without detecting other abnormal information, the source of the fire cannot be accurately determined.

[0166] The above are all preferred embodiments of this application, and are not intended to limit the scope of protection of this application. Therefore, all equivalent changes made in accordance with the structure, shape and principle of this application should be covered within the scope of protection of this application.

[0167] Additionally, it should be understood that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

Claims

1. A home fire monitoring system based on an Internet of Things (IoT) cloud platform, characterized in that: The system includes a smoke sensor (1), a temperature sensor (2), a light intensity sensor (3), a gas sensor, a camera (5), and a fire alarm (6) installed indoors. It also includes an NVS gateway (7) and a client computer (100). The smoke sensor (1), temperature sensor (2), light intensity sensor (3), gas sensor (4), camera (5), and fire alarm (6) are all connected to the NVS gateway (7). The NVS gateway (7) is wirelessly connected to the client computer (100). The client computer (100) is connected to a GPS module (8) and a cloud platform (10), and is wirelessly connected to a terminal (9). The client computer (100) includes: The acquisition module (101) is used to acquire indoor image information, smoke concentration information, temperature information, illuminance information and gas concentration information; The first alarm module (102) is used to generate first alarm information that causes the fire alarm (6) to sound an alarm when at least one of the smoke concentration information, temperature information, illuminance information and gas concentration information is abnormal. The suspected flame area determination module (103) is used to analyze the color, brightness and shape of the image information to determine the suspected flame area; The smoke-suspected area determination module (104) is used to compare the image information with a preset background image to determine the smoke-suspected area; The relevant judgment module (105) is used to determine whether the suspected flame area and the suspected smoke area are related; The second alarm module (106) is used to generate a second alarm message to be sent to the terminal (9) when the relevant judgment module (105) determines that it is true; The relevant judgment module (105) includes: The center point determination submodule is used to determine the first center point of the suspected flame area and the second center point of the suspected smoke area; The analysis direction determination submodule is used to connect the first center point and the second center point to obtain a line segment, and determine the analysis direction parallel to the line segment; The associated region determination submodule is used to determine the associated region located between the suspected flame region and the suspected smoke region; The overlapping edge determination submodule is used to determine the first overlapping edge in which the suspected flame area is located in the associated area, and the second overlapping edge in which the suspected smoke area is located in the associated area; The analysis point determination submodule is used to determine multiple sets of analysis points along the analysis direction on the first and second overlapping edges. The average value calculation submodule is used to determine the spacing between each group of analysis points and calculate the average value of the spacing between each group. The relevant determination submodule is used to determine the correlation between the suspected flame area and the suspected smoke area when the average value is less than a preset value; The irrelevance determination submodule is used to determine that the suspected flame area and the suspected smoke area are irrelevant when the average value is not less than a preset value.

2. The system according to claim 1, characterized in that, The suspected flame area determination module (103) includes: The first pixel point determination submodule is used to determine the brightness information of each pixel point in the image information and filter out the first pixel point whose brightness reaches the brightness limit. The pixel group determination submodule is used to aggregate and group the first pixel to obtain multiple pixel groups; The first similarity determination submodule is used to compare the color of each group of pixels with the color of the flame to determine the first similarity; The second similarity determination submodule is used to compare the shape of each group of pixels with the shape of a flame to determine the second similarity. The second image information acquisition submodule is used to determine whether both the first similarity and the second similarity have reached a preset value; if so, it acquires second image information located at multiple times after the current time; otherwise, it determines that there is no suspected flame area. The second image information pixel group determination submodule is used to determine the pixel group in the second image information; The flame flickering judgment submodule is used to determine whether the image has flame flickering characteristics based on the characteristics of the pixel groups in the current image information and each second image information. The flame suspected region determination submodule is used to determine the region corresponding to the pixel group as the flame suspected region when the first similarity and the second similarity reach a preset value and the region has flame flickering characteristics.

3. The system according to claim 2, characterized in that, The flame flickering detection submodule includes: The top edge determination unit is used to determine the top edge of any group of the same pixel points in the current image information and each of the second image information. An approximate line determination unit is used to determine the approximate line of the top edge. The ordinate determination unit is used to determine the ordinate of the intersection point of each of the approximate lines and the same vertical line; the vertical line is the vertical line where the x-coordinate of the first pixel in the image information is located; The current image information determination unit is used to determine the first image information as the current image information; The vertical coordinate difference calculation unit is used to calculate the vertical coordinate difference between the intersection point in the next image information and the intersection point in the current image information, and to use the next image as the current image; The repeated execution unit is used to repeatedly calculate the difference in ordinate between the intersection point in the next image information and the intersection point in the current image information, and to obtain multiple ordinate differences by taking the next image as the current image. A feature determination unit is used to determine whether the multiple ordinate differences have the characteristic of positive and negative intersection; The first determining unit is used to determine that the flame flashing feature exists when the feature determining unit determines that it is true; The second determining unit is used to determine that the flame flickering feature is not present when the feature determining unit determines that it is not present.

4. The system according to claim 3, characterized in that, The approximation line determination unit is specifically used for: Determine the length between the two endpoints of the top edge in the horizontal direction; The number of segments is determined based on the length; the longer the length, the more segments are required. Based on the number of divisions, the top edge is divided evenly along the horizontal direction to determine each division point; Starting from one endpoint, connect it sequentially to each dividing point, and then connect it to the other endpoint to obtain an approximate line of the fixed edge.

5. The system according to claim 1, characterized in that, The smoke-suspected area determination module (104) includes: The background image acquisition submodule is used to acquire the indoor image corresponding to a preset time before the anomaly occurs as the preset background image; The abnormal region determination submodule is used to compare the image information with the preset background image to determine inconsistent abnormal regions; The ambiguity determination submodule is used to determine the ambiguity of the abnormal area. When the ambiguity reaches a preset value, the abnormal area is determined to be a suspected smoke area.

6. The system according to claim 5, characterized in that, The abnormal region determination submodule is specifically used for: Increase the transparency of the preset background image; The preset background image is overlaid with the image information to be compared to identify inconsistent abnormal areas.

7. The system according to claim 1, characterized in that, The associated region determination submodule is specifically used for: Determine the center of the associated region on the line segment, and generate a circular associated region with the initial length as the radius; Determine whether the suspected flame area and the suspected smoke area are located within the associated area; if not, increase the radius by one unit and update the associated area. Repeat the step of determining whether the suspected flame area and the suspected smoke area are located within the associated area until the determination is yes, and then determine the associated area.

8. The system according to claim 1, characterized in that, The system also includes: The temperature variance calculation module is used to calculate the average temperature based on multiple temperature information corresponding to consecutive time points, and to calculate the temperature variance based on each temperature information and the average temperature. The illuminance variance calculation module is used to calculate the average illuminance based on multiple illuminance information corresponding to consecutive time moments, and to calculate the illuminance variance based on each illuminance information and the average illuminance. The first ignition source detection module is used to determine that the ignition source is solid when the smoke concentration information is abnormal. The second ignition source detection module is used to determine that the ignition source is gas when the illuminance information or gas concentration information is abnormal. The third fire source detection module is used to determine that the fire source is a liquid when the illuminance variance or temperature variance is abnormal. The fourth fire source detection module is used to determine that a fire source cannot be identified when the temperature difference alone reaches a preset time.

Images