A method for determining the working status of fire monitors based on the IoT cloud platform

Abstract

The present invention relates to the technical field of intelligent fire management, and discloses a method for distinguishing the working status of a fire cannon based on an IoT cloud platform, comprising: deploying a high-definition camera in the working area of ​​the fire cannon, and collecting on-site images in real time through the camera; preprocessing the on-site images collected in real time through an edge computing device; transmitting the preprocessed images to the IoT cloud platform, wherein: performing image recognition on the images through a set convolutional neural network to extract the state characteristics of the fire cannon; combining a time series analysis method, using a long short-term memory network to analyze the extracted fire cannon features and generate analysis results; comprehensively distinguishing the analysis results and outputting the distinction results. The present invention significantly improves the accuracy and real-time performance of distinguishing the working status of fire cannons through multi-source data fusion, advanced image processing technology and time series analysis methods.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent fire management, and in particular to a method for determining the working status of a fire monitor based on an Internet of Things cloud platform. Background Art

[0002] In traditional fire-fighting systems, fire monitors mainly rely on manual operation and monitoring, and have problems such as slow response speed and high misjudgment rate. In recent years, with the rapid development of technologies such as the Internet of Things (IoT), cloud computing, and artificial intelligence (AI), more and more intelligent fire-fighting systems have been realized. These systems improve the working efficiency and response speed of fire-fighting equipment through real-time monitoring and automated control; however, although these emerging technologies have alleviated the shortcomings of traditional systems to a certain extent, existing intelligent fire-fighting systems still face many technical challenges.

[0003] Existing intelligent fire protection systems mainly rely on a single data source for judgment. For example, they only rely on temperature sensors, smoke sensors or simple image recognition to judge the status of fire cannons. This makes it difficult to fully and accurately reflect the working status of fire cannons, especially in complex fire scenes. The ability to fuse and analyze multi-source data is insufficient, resulting in great limitations in the system when responding to sudden fire incidents.

[0004] Existing image processing technology performs poorly in low-light environments and severe weather conditions, making it difficult to ensure real-time monitoring and accurate identification of the working status of fire cannons. Most current systems lack in-depth analysis of time series data and are unable to effectively capture the dynamic changes and trends in the working status of fire cannons, making it easy for misjudgments and missed judgments to occur.

[0005] The existing technology has the following problems in judging the working status of fire monitors:

[0006] 1. The data source is single, and multi-dimensional and multi-level data fusion and analysis cannot be achieved;

[0007] Second, the image processing technology does not perform well in complex environments, which affects the accuracy and reliability of the system;

[0008] 3. The lack of in-depth analysis of time series data makes it difficult for the system to capture and determine the working status of fire monitors in real time and accurately. Summary of the invention

[0009] The purpose of this section is to summarize some aspects of the embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the abstract of the specification and the title of the invention of this application to avoid blurring the purpose of this section, the abstract of the specification and the title of the invention, and such simplifications or omissions cannot be used to limit the scope of the present invention.

[0010] In view of the above existing problems, the present invention is proposed.

[0011] Therefore, the technical problem solved by the present invention is: the problems of single data source, insufficient image processing capability and lack of time series analysis in the prior art.

[0012] In order to solve the above technical problems, the present invention provides the following technical solutions: deploying a high-definition camera in the working area of ​​the fire monitor, and collecting on-site images in real time through the camera;

[0013] Preprocessing the on-site images collected in real time by edge computing equipment;

[0014] The pre-processed image is transmitted to the IoT cloud platform, where:

[0015] Perform image recognition through the set convolutional neural network to extract the state features of the fire monitor;

[0016] In combination with the time series analysis method, a long short-term memory network is used to analyze the extracted fire monitor features to generate analysis results;

[0017] Perform comprehensive judgment on the analysis results and output the judgment results.

[0018] As a preferred solution of the method for determining the working status of a fire monitor based on the IoT cloud platform described in the present invention, the deployed working area of ​​the fire monitor at least includes a fire monitor body area, a water spray coverage area, and an operation and maintenance area.

[0019] As a preferred solution of the method for determining the working status of a fire monitor based on the IoT cloud platform described in the present invention, the collected on-site images include but are not limited to the fire monitor body, the spraying area, and the environmental background; wherein:

[0020] The fire monitor body at least includes a nozzle, a base and its peripheral structure of the fire monitor;

[0021] The spraying area at least includes an area within the spraying range of the fire monitor to monitor the direction and state of the water flow;

[0022] The environmental background at least includes buildings, obstacles and other environmental elements around the fire monitor to determine the impact of external factors on the operation of the fire monitor.

[0023] As a preferred solution of the method for determining the working status of a fire cannon based on the IoT cloud platform described in the present invention, the collected on-site images are preprocessed with denoising, image enhancement, image segmentation, and target detection, and the preprocessed images are then transmitted to the IoT cloud platform for image recognition.

[0024] As a preferred solution of the method for distinguishing the working status of a fire monitor based on the IoT cloud platform described in the present invention, the image recognition can be expressed by the following mathematical expression formula:

[0025]

[0026] in, is the convolution output of the jth convolution kernel of the i-th image block at time t, K is the depth of the convolution kernel, σ is the activation function, and W ijk is the kth weight of the jth convolution kernel of the ith image block, is the kth input image value of the jth convolution kernel of the ith image block, is the kth bias of the jth convolution kernel of the ith image block.

[0027] As a preferred solution of the method for distinguishing the working state of the fire monitor based on the IoT cloud platform described in the present invention, the extracted fire monitor features are analyzed by using a long short-term memory network in combination with a time series analysis method, which can be expressed by the following mathematical expression formula:

[0028]

[0029] Among them, L t is the long short-term memory result at time t, λ is the time attenuation coefficient, e is the natural constant, and τ is the time interval.

[0030] As a preferred solution of the method for distinguishing the working status of a fire monitor based on the IoT cloud platform described in the present invention, the mathematical expression formula for the comprehensive distinction is as follows:

[0031]

[0032] Among them, F s is the characteristic value of the fire monitor state, T is the total length of time, N is the number of image blocks, α i is the weight of the i-th image block, π is a constant, P is the number of feature points, ϵ is the minimum value to prevent the denominator from being zero, ω is the angular frequency, and L t is the long short-term memory result at time t, E k (t) is the edge detection result of the kth feature point at time t.

[0033] As a preferred solution of the method for distinguishing the working status of fire monitor based on the IoT cloud platform described in the present invention, F s The value range of is a non-negative number, indicating different fire monitor working status characteristics, and the discrimination results include:

[0034] F s =0, indicating that the fire monitor is in a static state with no obvious activity;

[0035] 0<F s ≤1, indicating that the fire monitor is in normal working condition with slight activity;

[0036] 1<F s ≤10, indicating that the fire monitor is in high-intensity working state and has obvious activities;

[0037] F s >10, it means the fire monitor is in an abnormal state and requires further inspection and maintenance.

[0038] Beneficial effects of the present invention:

[0039] 1. It can monitor the operating status of fire monitors in real time, detect and respond to any abnormal situation in time, thereby improving the efficiency and accuracy of fire warning and fire extinguishing;

[0040] 2. Edge computing can process data close to the data source, reducing data transmission delays and increasing processing speed;

[0041] 3. Upload the pre-processed image data to the IoT cloud platform, so that data processing and analysis can utilize the powerful computing power of the cloud platform, improving the processing efficiency and accuracy of the system;

[0042] 4. The system can identify the dynamic changes and trends of the operating status of fire monitors and generate more accurate and meaningful analysis results;

[0043] 5. Through comprehensive judgment, the final state characteristic value Fs of the fire monitor is obtained, which can comprehensively consider multiple factors and improve the accuracy and reliability of the judgment results. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them:

[0045] Figure 1 It is a flow chart of the method for determining the working status of a fire monitor based on the IoT cloud platform shown in the present invention. DETAILED DESCRIPTION

[0046] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings. It is obvious that the described embodiments are only part of the embodiments of the present invention, but not all of the embodiments.

[0047] Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without making any creative work should fall within the scope of protection of the present invention.

[0048] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0049] According to an embodiment of the present invention, Figure 1 The flowchart shown is a method for determining the working status of a fire monitor based on the IoT cloud platform, which specifically includes the following steps:

[0050] S1. Deploy high-definition cameras in the working area of ​​fire monitors and collect on-site images in real time through the cameras;

[0051] S2, pre-processing the real-time collected on-site images through edge computing devices;

[0052] S3, transmitting the pre-processed image to the IoT cloud platform, where:

[0053] S3.1, perform image recognition on it through the set convolutional neural network to extract the state characteristics of the fire monitor;

[0054] S3.2. Combined with the time series analysis method, the long short-term memory network is used to analyze the extracted fire monitor features and generate analysis results;

[0055] S3.3. Comprehensively judge the analysis results and output the judgment results.

[0056] Preferably, in the embodiment of the present invention, image quality is improved through image preprocessing, ensuring image clarity, facilitating the extraction of operating status characteristics of the fire cannon, providing higher quality data input for subsequent image recognition, and improving the reliability and accuracy of the overall system.

[0057] It should also be noted that the LSTM network can effectively capture long-term and short-term dependencies in time series and is suitable for processing data with a time dimension. This analysis method provides more abundant information for comprehensive judgment, making the judgment results more reliable.

[0058] The following describes in more detail the implementation process and / or effects of certain embodiments of the present invention in conjunction with some preferred or optional examples of the present invention.

[0059] In an optional embodiment, the deployed fire monitor working area at least includes the fire monitor body area, the water spray coverage area, and the operation and maintenance area, wherein:

[0060] Fire monitor body area: ensure that the camera can clearly capture the mechanical structure and movement of the fire monitor;

[0061] Water spray coverage area: The camera needs to cover the entire water spray range of the fire monitor in order to monitor the direction and status of the water flow;

[0062] Operation and Maintenance Area: Contains areas where firefighters may operate or maintain monitors to ensure full monitoring.

[0063] As an example, the parameters of the high-definition camera deployed in the embodiment of the present invention are as follows:

[0064] Resolution: Use 4K or higher resolution cameras to ensure image clarity;

[0065] Frame rate: at least 30 frames per second to ensure smooth dynamic images;

[0066] Infrared camera: Equipped with infrared function to ensure effective monitoring even in low-light environments.

[0067] In an optional embodiment, the collected on-site images include but are not limited to the fire monitor body, the spraying area, and the environmental background; wherein:

[0068] The fire monitor body at least includes the nozzle, base and surrounding structure of the fire monitor;

[0069] The spraying area at least includes the area within the spraying range of the fire monitor to monitor the direction and status of the water flow;

[0070] The environmental background shall at least include the buildings, obstacles and other environmental elements around the fire monitor to determine the impact of external factors on the operation of the fire monitor.

[0071] The collected on-site images are preprocessed by denoising, image enhancement, image segmentation, and target detection, and then the preprocessed images are transmitted to the IoT cloud platform for image recognition.

[0072] As an example, the Gaussian Blur method is used to remove noise from the image, the CLAHE method is used to enhance the image contrast, the edge detection (such as Canny edge detection) is used to segment the fire monitor and the water jet it sprays, and the pre-trained YOLO (You Only Look Once) model is used to detect the fire monitor and the water jet it sprays.

[0073] It should be further explained that the trained convolutional neural network (CNN), analysis model and comprehensive discrimination model are deployed in the IoT cloud platform to facilitate the recognition, analysis and discrimination of the uploaded images. In this embodiment of the present invention, the construction of a convolutional neural network (CNN) is taken as an example to illustrate the deployment and calculation of its network model:

[0074] The high-definition cameras deployed in the working area of ​​the fire monitors are used to collect on-site images in real time;

[0075] extracting previously recorded image data from a historical database of a fire protection system as historical data;

[0076] Simulate different working conditions of fire monitors in a controlled environment and collect corresponding image data to simulate experiments.

[0077] The data types include but are not limited to normal state images, water spray state images, fault state images and environmental images; among which:

[0078] Normal status image: the image of the fire monitor in normal standby status;

[0079] Water spraying status images: images of fire monitors spraying water, including images of different angles and water spraying intensities;

[0080] Fault status image: an image of a fire monitor when it fails, such as nozzle blockage or water flow interruption;

[0081] Environmental image: The surrounding environment image, including background noise and other non-fire monitor objects.

[0082] Each image needs to be accurately labeled, including: fire monitor location, water spray status (yes / no), water spray direction, and fault type.

[0083] In order to improve the generalization ability of CNN, data enhancement processing is performed, including image rotation, scaling, translation, and adding noise.

[0084] Construct a convolutional neural network model (CNN), whose model structure is:

[0085] Convolutional layer: extracts low-level features of the image, such as edges and textures;

[0086] Pooling layer: reduces data dimension and retains main features;

[0087] Fully connected layer: maps the extracted features to specific classification labels.

[0088] The constructed convolutional neural network model (CNN) is trained using the above labeled dataset, using the cross entropy loss function and Adam optimizer. The mathematical expression formula is as follows:

[0089]

[0090] Among them, L is the loss function, y i is the true label, N is the number of samples, is the predicted probability.

[0091] Optimize the model by adjusting hyperparameters, such as learning rate, batch size, and regularization parameters.

[0092] In an optional implementation, the trained CNN performs image recognition on an image, which can be expressed by the following mathematical expression formula:

[0093]

[0094] in, is the convolution output of the jth convolution kernel of the i-th image block at time t, K is the depth of the convolution kernel, σ is the activation function, and W ijk is the kth weight of the jth convolution kernel of the ith image block, is the kth input image value of the jth convolution kernel of the ith image block, is the kth bias of the jth convolution kernel of the ith image block.

[0095] Combined with the time series analysis method, the long short-term memory network is used to analyze the extracted fire monitor features, which can be expressed by the following mathematical expression formula:

[0096]

[0097] Among them, L t is the long short-term memory result at time t, λ is the time attenuation coefficient, e is the natural constant, and τ is the time interval.

[0098] The mathematical expression formula of comprehensive discrimination is as follows:

[0099]

[0100] Among them, F s is the characteristic value of the fire monitor state, T is the total length of time, N is the number of image blocks, α i is the weight of the i-th image block, π is a constant, P is the number of feature points, ϵ is the minimum value to prevent the denominator from being zero, ω is the angular frequency, and L t is the long short-term memory result at time t, E k (t) is the edge detection result of the kth feature point at time t.

[0101] Specifically, F s The value range of is a non-negative number, indicating different fire monitor working status characteristics, and the discrimination results include:

[0102] F s =0, indicating that the fire monitor is in a static state with no obvious activity;

[0103] 0<F s ≤1, indicating that the fire monitor is in normal working condition with slight activity;

[0104] 1<F s ≤10, indicating that the fire monitor is in high-intensity working state and has obvious activities;

[0105] F s >10, it means the fire monitor is in an abnormal state and requires further inspection and maintenance.

[0106] As an example, the extracted features include but are not limited to water spray status, water spray direction, water spray intensity, fault detection, and environmental features, where:

[0107] Water spraying status: Identify whether the fire monitor is spraying water and the specific status of the water spraying;

[0108] Water spray direction: detect the direction and angle of water spray;

[0109] Water spray intensity: Estimate the intensity and coverage of water spray;

[0110] Fault detection: Identify the type of fire monitor fault, such as nozzle blockage and water flow interruption;

[0111] Environmental characteristics: Analyze the impact of the surrounding environment on the operation of fire monitors, such as smoke, obstacles, etc.

[0112] The extracted features are then quantified, where:

[0113] Water spraying status: binary feature (0 means no water spraying, 1 means water spraying);

[0114] Water spray direction: angle characteristics, quantified as 0~360 degrees;

[0115] Water spray intensity: intensity characteristic, quantified from 0 to 100;

[0116] Fault type: multi-classification features, specific fault types such as nozzle blockage (1), water flow interruption (2);

[0117] Environmental characteristics: Quantify the intensity and type of environmental factors, such as smoke concentration and size of obstacles.

[0118] Preferably, in order to better verify and illustrate the technical effects used in the method of the present invention, this embodiment chooses to conduct comparative tests using the traditional method and the method of the present invention, and compares the test results by means of scientific demonstration to verify the real effect of the method.

[0119] In the traditional fire monitor working status judgment method, it usually relies on single sensor data or simple image processing technology for judgment. These methods have the following defects:

[0120] (1) Insufficient accuracy: Due to the single data source, traditional methods are difficult to fully and accurately reflect the working status of fire monitors, resulting in a high misjudgment rate;

[0121] (2) Slow response: Traditional methods mostly rely on manual monitoring and operation, which cannot achieve real-time monitoring and rapid response;

[0122] (3) Poor environmental adaptability: In low-light environments and complex weather conditions, the effectiveness of traditional image processing technology is greatly reduced, making it difficult to ensure the continuity and accuracy of monitoring.

[0123] In order to verify that the method of the present invention has higher accuracy and real-time performance than the traditional method, in this embodiment, the traditional single sensor method and the present method are used to perform real-time measurement and comparison on the recognition accuracy and response speed of the working status of the fire monitor.

[0124] The fire monitor system is run on a simulation platform to simulate the fire environment and different operation scenarios, and is tested using standard test samples.

[0125] The traditional method uses a single smoke sensor for condition monitoring and obtains test results through manual data analysis.

[0126] This method deploys high-definition cameras to collect images, uses edge computing devices to preprocess the images, and then analyzes the image data through the IoT cloud platform combined with convolutional neural networks (CNN) and long short-term memory networks (LSTM) to automatically determine the working status of fire monitors.

[0127] Experimental environment: In the simulated fire laboratory, a standard fire monitor system is configured. The laboratory area is 200 square meters and can simulate different fire scenes, such as high temperature, thick smoke and obstacles. The laboratory is equipped with a temperature controller, a smoke generator and movable obstacles.

[0128] Experimental equipment:

[0129] High-definition cameras: with a resolution of 4K and night vision function, they are placed at different angles of the fire monitor to ensure full coverage;

[0130] Edge computing device: high-performance computing module with real-time image processing capabilities;

[0131] IoT cloud platform: A cloud computing platform based on AWS or Azure, providing powerful computing and storage capabilities;

[0132] Traditional sensor: High-sensitivity smoke sensor, used for comparative experiments;

[0133] Simulation software: Use MATLAB for data processing and simulation analysis.

[0134] Each method tested 20 groups of data, and the recognition accuracy, response time and error of each group of data were calculated and compared with the actual working status of the simulated input.

[0135] Table 1. Test result data comparison table

[0136]

[0137] Referring to Table 1, it can be seen intuitively that the method of the present invention significantly improves the accuracy of recognition through multi-source data fusion and advanced image processing technology, with an average recognition accuracy of 95%, which is significantly improved compared with 78% of the traditional method; by adopting edge computing and automated processing, the response time of the method of the present invention is greatly shortened, with an average response time of only 1.2 seconds, which is nearly four times the response speed of 4.5 seconds of the traditional method; for example, through time series analysis and comprehensive discrimination, the average error of the method of the present invention is reduced to 3%, which is 75% less than 12% of the traditional method; for example, the method of the present invention performs well in reducing the false alarm rate and the missed alarm rate, which are 2% and 3% respectively, which are significantly reduced compared with 10% and 15% of the traditional method.

[0138] Preferably, the method of the present invention significantly improves the accuracy, real-time performance and reliability of fire monitor working status identification through multi-source data fusion, advanced image processing technology and time series analysis method.

[0139] The aforementioned image data preprocessing and model training methods can be performed using methods and means in the prior art and will not be described in detail in this example.

[0140] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for determining the working status of a fire monitor based on the IoT cloud platform, characterized in that: include: Deploy high-definition cameras in the working area of ​​the fire monitor to collect on-site images in real time through the cameras; Preprocessing the on-site images collected in real time by edge computing equipment; The pre-processed image is transmitted to the IoT cloud platform, where: The image is recognized by the set convolutional neural network to extract the state characteristics of the fire monitor; the image recognition can be expressed by the following mathematical expression formula: in, is the convolution output of the jth convolution kernel of the i-th image block at time t, K is the depth of the convolution kernel, σ is the activation function, and W ijk is the kth weight of the jth convolution kernel of the ith image block, is the kth input image value of the jth convolution kernel of the ith image block, is the kth bias of the jth convolution kernel of the ith image block; Combined with the time series analysis method, the long short-term memory network is used to analyze the extracted fire monitor features to generate analysis results; which can be expressed by the following mathematical expression formula: Among them, L t is the long-term and short-term memory result at time t, λ is the time decay coefficient, e is the natural constant, and τ is the time interval; Perform comprehensive judgment on the analysis results and output the judgment results; the mathematical expression formula of the comprehensive judgment is as follows: Among them, F s is the characteristic value of the fire monitor state, T is the total length of time, N is the number of image blocks, α i is the weight of the i-th image block, π is a constant, P is the number of feature points, ϵ is the minimum value to prevent the denominator from being zero, ω is the angular frequency, and L t is the long short-term memory result at time t, E k (t) is the edge detection result of the kth feature point at time t; F s The value range of is a non-negative number, indicating different fire monitor working status characteristics, and the discrimination results include: F s =0, indicating that the fire monitor is in a static state with no obvious activity; 0<F s ≤1, indicating that the fire monitor is in normal working condition with slight activity; 1<F s ≤10, indicating that the fire monitor is in high-intensity working state and has obvious activities; F s >10, it means the fire monitor is in an abnormal state and requires further inspection and maintenance.

2. The method for determining the working status of a fire monitor based on the IoT cloud platform according to claim 1 is characterized in that: The deployed fire monitor working area at least includes the fire monitor body area, the water spray coverage area, and the operation and maintenance area.

3. The method for determining the working status of a fire monitor based on the IoT cloud platform according to claim 1 is characterized in that: The collected on-site images include but are not limited to the fire monitor body, spraying area, and environmental background; among which: The fire monitor body at least includes a nozzle, a base and its peripheral structure of the fire monitor; The spraying area at least includes an area within the spraying range of the fire monitor to monitor the direction and state of the water flow; The environmental background at least includes buildings, obstacles and other environmental elements around the fire monitor to determine the impact of external factors on the operation of the fire monitor.

4. The method for determining the working status of a fire monitor based on the IoT cloud platform according to claim 1 or 3, characterized in that: The collected on-site images are preprocessed by denoising, image enhancement, image segmentation, and target detection, and then the preprocessed images are transmitted to the IoT cloud platform for image recognition.

Images