Remote internet-of-things control management system and method based on fire fighting

Abstract

The invention discloses a remote Internet of Things control management system and method based on fire fighting, and belongs to the technical field of Internet of Things. The system comprises an image recognition module which obtains a video stream of a fire scene through a monitoring camera, and carries out the real-time analysis and recognition of an image through image processing and a deep convolutional neural network; the gas recognition module is used for monitoring the gas concentration of a fire scene in real time and recognizing gas components by using a gas recognition algorithm; the data fusion analysis module is used for carrying out feature fusion on the data acquired by the image recognition module and the gas recognition module, and carrying out analysis and modeling on the fused data; the decision support module is used for performing fire risk assessment according to an analysis result of the data fusion analysis module; based on historical data and the scene model, performing simulation and prediction of a fire development trend; and the remote internet-of-things control module is used for providing data for checking fire scene conditions in real time, so that a user can remotely control all equipment and devices of the fire extinguishing system.

Description

Technical Field

[0001] The present invention relates to the technical field of Internet of Things, and in particular to a remote Internet of Things control management system and method based on fire fighting. Background Art

[0002] Traditional fire monitoring systems often rely on manual inspections and manual alarms, and have problems such as untimely monitoring and slow response. Traditional fire monitoring methods have limited accuracy and precision, especially in large buildings or confined spaces. The coverage and effectiveness of monitoring equipment are limited, and missed or false alarms are prone to occur. Existing technologies generally lack intelligence and automation levels, and are unable to accurately identify and evaluate the specific location of the fire, the size of the fire, and the location of the fire source, nor can they achieve remote monitoring and control of the fire scene. Existing technologies have weak processing capabilities for fire data, lack effective data fusion and analysis methods, are unable to achieve comprehensive analysis and modeling of fire scene conditions, and are unable to provide effective decision support in a timely manner. Summary of the invention

[0003] The purpose of the present invention is to provide a remote IoT control management system and method based on fire fighting to solve the problems raised in the above background technology.

[0004] In order to solve the above technical problems, the present invention provides the following technical solutions: A remote IoT control and management system based on fire fighting, comprising: Image recognition module: The video stream of the fire scene is obtained through the surveillance camera, and the image is analyzed and recognized in real time using image processing and deep convolutional neural network to determine the precise location, size and source of the fire; Gas identification module: deploys gas sensors to monitor the gas concentration at the fire scene in real time, and uses gas identification algorithms to identify the gas composition to determine the type and source of the gas causing the fire; Data fusion analysis module: It fuses the data obtained by the image recognition module and the gas recognition module, analyzes and models the fused data, and comprehensively analyzes the situation at the fire scene; Decision support module: Establish a risk assessment model, conduct fire risk assessment based on the analysis results of the data fusion analysis module, and issue alarm signals to relevant personnel; simulate and predict the development trend of fire based on historical data and scenario models, and generate emergency measures plans; Remote IoT control module: provides real-time viewing of fire scene data. Users can remotely control various equipment and devices of the fire protection system, flexibly configure the parameters of fire protection equipment and cameras, and support real-time communication and coordination with on-site staff.

[0005] The image recognition module obtains the video stream through the surveillance camera deployed at the fire scene. After receiving the video stream, the image is preprocessed. The preprocessing includes denoising, image enhancement and edge detection to remove the camera sensor noise and ambient light interference, improve the image clarity and visibility, and identify the boundaries and contours in the image; The preprocessed video stream is input into a deep convolutional neural network to obtain the precise location, size and source of the fire.

[0006] The deep convolutional neural network uses semantic segmentation technology to determine the precise location and size of the fire, and uses target detection technology to determine the location of the fire source; Collect image data of the scene after the fire in different scenarios, which comes from surveillance cameras, drone aerial photography and on-site photography by firefighters. The collected image data ensures that it can cover every corner and different perspectives of the fire scene; select the image annotation tool LabelImg to have the annotator perform pixel-level annotation. The annotator needs to draw the boundary of the flame area on the image and mark the location of the flame with pixel-level accuracy; use different colors or marks to distinguish different flames; also perform pixel-level annotation on the smoke area in the image. The annotator needs to mark the location and range of the smoke to ensure that the model can identify the smoke area; according to different scenarios, the annotator needs to annotate other items that affect fire fighting in different scenarios; Design a deep convolutional neural network suitable for semantic segmentation and target detection. The output of the network includes two parts: one is the semantic segmentation result of flames and smoke, which is used to determine the precise location and size of the fire; the other is the target detection result of the fire source location. Use U-Net for semantic segmentation and YOLO for target detection. The designed deep convolutional neural network is trained using the prepared image data of the fire scene. During the training process, an end-to-end training method is adopted to optimize the loss functions of semantic segmentation and target detection at the same time. Based on the results of semantic segmentation, the areas of flame and smoke can be obtained to determine the precise location and size of the fire, and the size of the fire can be evaluated by calculating the area and number of pixels of the flame and smoke areas. Based on the results of target detection, the detection results of the fire source location can be obtained, the coordinates of the fire source location can be obtained, and the coordinates can be marked on the floor plan of the fire scene.

[0007] The deep convolutional neural network uses U-Net for semantic segmentation. The U-Net network structure consists of an encoder and a decoder, wherein the encoder is used to extract high-level features of the image, and the decoder is used to map the extracted features back to the original image size. In order to perform the semantic segmentation task of flame and smoke, the final output of the network is set to two channels, which respectively represent the segmentation results of flame and smoke; The encoder inputs an image of the fire scene and outputs a feature map; the encoder is composed of several convolutional layers and pooling layers, wherein each convolutional layer is followed by an activation function ReLU for introducing nonlinearity, and the number of convolution kernels gradually increases to capture feature information; the decoder adopts an upsampling operation to gradually increase the resolution of the feature map through deconvolution to locate the position of the flame and smoke. After each upsampling operation, the decoder splices and fuses low-level and high-level features with the feature map of the corresponding layer of the encoder, thereby achieving semantic segmentation results; The loss function of U-Net is used to measure the difference between the model output and the true label. It uses a binary cross entropy loss function, where category one represents flame and category two represents smoke. Given an H*W image, let the output of U-Net be P and the true label be Y. Then the binary cross entropy loss function is The formula is as follows: Where N is the number of pixels, N = H*W; Indicates the true label of pixel i belonging to category c. If pixel i belongs to category c, then ,otherwise ; Represents the probability that the pixel i output by the model belongs to category c. For a binary classification task, each pixel output will have two probability values, representing the probability of category 1 flame and category 2 smoke respectively. The goal of the binary cross entropy loss function is to minimize value, so that the output probability distribution is close to the distribution of the true label.

[0008] The deep convolutional neural network uses YOLO for target detection. The YOLO network consists of a single convolutional neural network, which simultaneously predicts several bounding boxes and the class probabilities corresponding to the bounding boxes in one forward propagation; The loss function of YOLO consists of two parts: target classification loss and position regression loss. The cross entropy loss function is used to measure the classification accuracy of the target category. In the target detection task at the fire source location, the target classification loss function is It is expressed as: Where M is the number of fire source targets in the image; C is the number of categories; It is the indicator function of the target classification loss function, indicating whether the i'th bounding box contains the target of category c'; is the probability that the i'th bounding box belongs to category c' predicted by the model; The squared loss function is used to measure the regression accuracy of the model for the bounding box position, and the loss function is expressed as: in, is the weight of the position regression loss, It is The horizontal coordinate of the center coordinate of the bounding box, It is The ordinate of the center coordinate of the bounding box, It is The width of the bounding box, It is The height of the bounding box, It is the predicted The horizontal coordinate of the center coordinate of the bounding box, It is the predicted The ordinate of the center coordinate of the bounding box, It is the predicted The width of the bounding box, It is the predicted The height of the bounding box; is the indicator function of the square loss function, indicating that Whether the bounding box contains the object, if bounding boxes contain the object, then ,otherwise .

[0009] The gas sensor monitors the changes in gas concentration at the fire scene in real time and can detect several types of harmful gases. The gas concentration data collected by the gas sensor is transmitted to the gas identification module by wireless or wired means. The gas identification module uses the gas identification algorithm to analyze and process the gas concentration data collected by the sensor to identify the type and source of the gas caused by the fire; The gas identification algorithm adopts a support vector machine algorithm to classify known types of gases, and matches the identified gas patterns with known fire-caused gas types and their characteristics to determine whether the monitored gas is related to the fire; in the feature matching process, the similarity and distance between feature vectors are compared by using the Euclidean distance; The gas identification algorithm classifies and identifies the monitored gases according to the characteristics of the gas concentration data, and determines the type of the monitored gas by comparing it with a pre-established gas characteristic database. The gas identification algorithm analyzes the changing trends of the monitored gas concentration data, and infers the source of the gas caused by the fire based on the environmental factors at the fire scene.

[0010] In a support vector machine, given a set of training samples, each sample consists of a feature vector and a corresponding category label. SVM attempts to find a hyperplane that separates sample points of different categories and ensures that the distance from this hyperplane to the nearest sample point is maximized. In SVM, the original feature space is mapped to a high-dimensional space through a kernel function. The purpose of this is to linearly segment the sample points in the original feature space, so as to find a linear hyperplane in the high-dimensional space for classification.

[0011] The goal of SVM is to find a maximum margin hyperplane, that is, a hyperplane farthest from the sample points of each category. The advantage of doing so is that the generalization ability of the model can be improved, making the classification of unknown samples more accurate and reliable. The optimization problem of SVM is achieved by solving a convex quadratic programming problem. By introducing the Lagrange multiplier method, the original convex quadratic programming problem is converted into a dual form, and then the optimal hyperplane parameters are obtained by solving the dual problem. The kernel function is the key to SVM, which is used to map samples to a high-dimensional space. Commonly used kernel functions include linear kernel functions, polynomial kernel functions, Gaussian radial basis kernel functions, etc. The present invention uses a linear kernel function.

[0012] The final decision boundary is determined by the support vector, which is the sample point closest to the hyperplane. The decision boundary separates sample points of different categories and is used for classification during testing.

[0013] The data fusion analysis module extracts the features of the data in the image recognition module and the gas recognition module respectively, fuses the features extracted by the two modules, and forms a comprehensive feature representation; the fusion process is implemented through weighted summation to determine the weight of the features extracted by each module, and the weight is set by the user according to the actual situation; after the weighted summation, the fused features need to be normalized to ensure that the comprehensive features have a uniform scale and range, and the eigenvalues ​​are scaled to the range of [0, 1] through linear transformation.

[0014] The decision support module uses a decision tree to establish a risk assessment model, uses the fire level in the historical data as a label, and divides the fire events into different categories according to the fire level. The label will be used as the training target of the decision tree model; the decision tree algorithm is used to train the training data. The decision tree is a tree structure that makes decisions by recursively dividing the features; during the training of the decision tree, appropriate division nodes are selected according to the different data characteristics so that the samples represented by each node belong to the same category as much as possible. The trained decision tree model is evaluated using the cross-validation method to ensure the generalization ability and accuracy of the model. The evaluation indicators include accuracy, precision, recall rate and F1 score; when new data is input, the trained decision tree model is used for prediction. According to the new data features, the recursive division process of the decision tree model is used to predict the risk level of the fire and send an alarm signal to the relevant personnel; Establishing a fire scenario model, including building structure, environmental factors and fire-fighting facilities, is a model based on computer simulation; using historical data and scenario models, simulate the fire development process, predict the fire spread path, fire spread speed information, and the impact of the fire on the surrounding environment and buildings; based on the simulation results, predict the fire development trend and possible impact range according to historical data and scenario models; formulate corresponding emergency measures according to the fire prediction results, including personnel evacuation routes, fire extinguishing plans and material deployment; determine the specific implementation methods and priorities of various emergency measures according to the predicted fire spread path and impact range.

[0015] In the remote IoT control module, users can view the situation data of the fire scene in real time, remotely control various equipment and devices in the fire fighting system, and adjust the working status and parameters of the equipment in real time according to the situation at the fire scene; the remote IoT control module allows users to flexibly configure the parameters of fire fighting equipment and cameras, including: the direction of fire extinguishing agent injection, injection amount, camera viewing angle and camera resolution; the remote IoT control module supports users to communicate and coordinate with the staff at the fire scene in real time. Through the instant messaging function, users can exchange information with the on-site staff. Common fire fighting equipment that can be remotely operated includes but is not limited to: fire extinguishers, sprinkler systems, smoke exhaust systems, automatic sprinkler systems, gas fire extinguishing systems, fire doors and fire shutter doors, fire alarms, cameras, etc.

[0016] Firefighting equipment and control equipment need to be connected to the network through various communication technologies, such as Wi-Fi, Bluetooth, Zigbee, LoRa, etc. When the devices are connected to the network, they can transmit and communicate data through the network. In this way, the firefighting equipment can send the monitored data to the remote server, or receive remote instructions for control operations. In the Internet of Things, there will be a remote server for managing and processing the data sent by the device. This server is a cloud platform that is responsible for receiving the data sent by the device, storing, analyzing and processing it, and sending control instructions to the device. The remote server will provide an application programming interface API for third-party applications to interact with the Internet of Things system. Through the API, users can monitor the status of the device in real time, send control instructions, etc. through mobile applications or web applications.

[0017] When connecting devices to the Internet of Things, security and privacy protection are very important considerations. The present invention takes appropriate security measures, such as data encryption, identity authentication, permission control, etc., to protect the security and privacy of devices and data.

[0018] A remote IoT control management method based on fire fighting comprises the following steps: S100, obtaining a video stream of the fire scene through a surveillance camera; S200, using image processing and deep convolutional neural networks to analyze and identify images in real time to determine the precise location, size and source of the fire; S300, deploy gas sensors to monitor the gas concentration at the fire scene in real time; S400, identifying gas components using a gas identification algorithm to determine the type and source of the gas causing the fire; S500, performing feature fusion on the data acquired by the image recognition module and the gas recognition module; S600: Analyze and model the fused data and comprehensively analyze the situation at the fire scene; S700, establishing a risk assessment model, conducting fire risk assessment based on the analysis results of the data fusion analysis module, and issuing an alarm signal to relevant personnel; S800, based on historical data and scenario models, simulates and predicts fire development trends and generates emergency response plans; S900 provides real-time viewing of fire scene data. Users can remotely control various equipment and devices of the fire protection system, flexibly configure the parameters of fire protection equipment and cameras, and support real-time communication and coordination with on-site staff.

[0019] Compared with the prior art, the beneficial effects achieved by the present invention are: The present invention utilizes image recognition and gas recognition technology to monitor the situation at the fire scene in real time and accurately locate the location of the fire and the fire source, thereby improving the speed and accuracy of fire detection.

[0020] The data fusion analysis module can conduct a comprehensive analysis of the data obtained by the image recognition and gas recognition modules, establish a risk assessment model, realize fire risk assessment and early warning, and issue alarm signals in a timely manner, which helps to prevent fire accidents in advance.

[0021] The remote IoT control module allows users to view fire scene data in real time, remotely control fire-fighting equipment and cameras, flexibly configure equipment parameters, and support real-time communication and coordination with on-site staff, thereby improving the emergency response capabilities and fire-fighting efficiency at the fire scene. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings: Figure 1 This is a system structure diagram of a remote IoT control and management system based on fire fighting of the present invention; Figure 2 It is a schematic diagram of the steps of a remote Internet of Things control management method based on fire fighting of the present invention. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] See also Figure 1 and Figure 2 , the present invention provides a technical solution: According to one embodiment of the present invention, Figure 1 As shown in the system structure diagram of a remote IoT control and management system based on fire fighting, a remote IoT control and management system based on fire fighting includes: Image recognition module: The video stream of the fire scene is obtained through the surveillance camera, and the image is analyzed and recognized in real time using image processing and deep convolutional neural network to determine the precise location, size and source of the fire; Gas identification module: deploys gas sensors to monitor the gas concentration at the fire scene in real time, and uses gas identification algorithms to identify the gas composition to determine the type and source of the gas causing the fire; Data fusion analysis module: It fuses the data obtained by the image recognition module and the gas recognition module, analyzes and models the fused data, and comprehensively analyzes the situation at the fire scene; Decision support module: Establish a risk assessment model, conduct fire risk assessment based on the analysis results of the data fusion analysis module, and issue alarm signals to relevant personnel; simulate and predict the development trend of fire based on historical data and scenario models, and generate emergency measures plans; Remote IoT control module: provides real-time viewing of fire scene data. Users can remotely control various equipment and devices of the fire protection system, flexibly configure the parameters of fire protection equipment and cameras, and support real-time communication and coordination with on-site staff.

[0025] The image recognition module obtains the video stream through the surveillance camera deployed at the fire scene. After receiving the video stream, the image is preprocessed. The preprocessing includes denoising, image enhancement and edge detection to remove the camera sensor noise and ambient light interference, improve the image clarity and visibility, and identify the boundaries and contours in the image; The preprocessed video stream is input into a deep convolutional neural network to obtain the precise location, size and source of the fire.

[0026] The deep convolutional neural network uses semantic segmentation technology to determine the precise location and size of the fire, and uses target detection technology to determine the location of the fire source; Collect image data of the scene after the fire in different scenarios, which comes from surveillance cameras, drone aerial photography and on-site photography by firefighters. The collected image data ensures that it can cover every corner and different perspectives of the fire scene; select the image annotation tool LabelImg to have the annotator perform pixel-level annotation. The annotator needs to draw the boundary of the flame area on the image and mark the location of the flame with pixel-level accuracy; use different colors or marks to distinguish different flames; also perform pixel-level annotation on the smoke area in the image. The annotator needs to mark the location and range of the smoke to ensure that the model can identify the smoke area; according to different scenarios, the annotator needs to annotate other items that affect fire fighting in different scenarios; Design a deep convolutional neural network suitable for semantic segmentation and target detection. The output of the network includes two parts: one is the semantic segmentation result of flames and smoke, which is used to determine the precise location and size of the fire; the other is the target detection result of the fire source location. Use U-Net for semantic segmentation and YOLO for target detection. The designed deep convolutional neural network is trained using the prepared image data of the fire scene. During the training process, an end-to-end training method is adopted to optimize the loss functions of semantic segmentation and target detection at the same time. Based on the results of semantic segmentation, the areas of flame and smoke can be obtained to determine the precise location and size of the fire, and the size of the fire can be evaluated by calculating the area and number of pixels of the flame and smoke areas. Based on the results of target detection, the detection results of the fire source location can be obtained, the coordinates of the fire source location can be obtained, and the coordinates can be marked on the floor plan of the fire scene.

[0027] The deep convolutional neural network uses U-Net for semantic segmentation. The U-Net network structure consists of an encoder and a decoder, wherein the encoder is used to extract high-level features of the image, and the decoder is used to map the extracted features back to the original image size. In order to perform the semantic segmentation task of flame and smoke, the final output of the network is set to two channels, which respectively represent the segmentation results of flame and smoke; The encoder inputs an image of the fire scene and outputs a feature map; the encoder is composed of several convolutional layers and pooling layers, wherein each convolutional layer is followed by an activation function ReLU for introducing nonlinearity, and the number of convolution kernels gradually increases to capture feature information; the decoder adopts an upsampling operation to gradually increase the resolution of the feature map through deconvolution to locate the position of the flame and smoke. After each upsampling operation, the decoder splices and fuses low-level and high-level features with the feature map of the corresponding layer of the encoder, thereby achieving semantic segmentation results; The loss function of U-Net is used to measure the difference between the model output and the true label. It uses a binary cross entropy loss function, where category one represents flame and category two represents smoke. Given an H*W image, let the output of U-Net be P and the true label be Y. Then the binary cross entropy loss function is The formula is as follows: Where N is the number of pixels, N = H*W; Indicates the true label of pixel i belonging to category c. If pixel i belongs to category c, then ,otherwise ; Represents the probability that the pixel i output by the model belongs to category c. For a binary classification task, each pixel output will have two probability values, representing the probability of category 1 flame and category 2 smoke respectively. The goal of the binary cross entropy loss function is to minimize value, so that the output probability distribution is close to the distribution of the true label.

[0028] The deep convolutional neural network uses YOLO for target detection. The YOLO network consists of a single convolutional neural network, which simultaneously predicts several bounding boxes and the class probabilities corresponding to the bounding boxes in one forward propagation; The loss function of YOLO consists of two parts: target classification loss and position regression loss. The cross entropy loss function is used to measure the classification accuracy of the target category. In the target detection task at the fire source location, the target classification loss function is It is expressed as: Where M is the number of fire source targets in the image; C is the number of categories; It is the indicator function of the target classification loss function, indicating whether the i'th bounding box contains the target of category c'; is the probability that the i'th bounding box belongs to category c' predicted by the model; The squared loss function is used to measure the regression accuracy of the model for the bounding box position, and the loss function is expressed as: in, is the weight of the position regression loss, It is The horizontal coordinate of the center coordinate of the bounding box, It is The ordinate of the center coordinate of the bounding box, It is The width of the bounding box, It is The height of the bounding box, It is the predicted The horizontal coordinate of the center coordinate of the bounding box, It is the predicted The ordinate of the center coordinate of the bounding box, It is the predicted The width of the bounding box, It is the predicted The height of the bounding box; is the indicator function of the square loss function, indicating that Whether the bounding box contains the object, if bounding boxes contain the object, then ,otherwise .

[0029] The gas sensor monitors the changes in gas concentration at the fire scene in real time and can detect several types of harmful gases. The gas concentration data collected by the gas sensor is transmitted to the gas identification module by wireless or wired means. The gas identification module uses the gas identification algorithm to analyze and process the gas concentration data collected by the sensor to identify the type and source of the gas caused by the fire; The gas identification algorithm adopts a support vector machine algorithm to classify known types of gases, and matches the identified gas patterns with known fire-caused gas types and their characteristics to determine whether the monitored gas is related to the fire; in the feature matching process, the similarity and distance between feature vectors are compared by using the Euclidean distance; The gas identification algorithm classifies and identifies the monitored gases according to the characteristics of the gas concentration data, and determines the type of the monitored gas by comparing it with a pre-established gas characteristic database. The gas identification algorithm analyzes the changing trends of the monitored gas concentration data, and infers the source of the gas caused by the fire based on the environmental factors at the fire scene.

[0030] In a support vector machine, given a set of training samples, each sample consists of a feature vector and a corresponding category label. SVM attempts to find a hyperplane that separates sample points of different categories and ensures that the distance from this hyperplane to the nearest sample point is maximized. In SVM, the original feature space is mapped to a high-dimensional space through a kernel function. The purpose of this is to linearly segment the sample points in the original feature space, so as to find a linear hyperplane in the high-dimensional space for classification.

[0031] The goal of SVM is to find a maximum margin hyperplane, that is, a hyperplane farthest from the sample points of each category. The advantage of doing so is that the generalization ability of the model can be improved, making the classification of unknown samples more accurate and reliable. The optimization problem of SVM is achieved by solving a convex quadratic programming problem. By introducing the Lagrange multiplier method, the original convex quadratic programming problem is converted into a dual form, and then the optimal hyperplane parameters are obtained by solving the dual problem. The kernel function is the key to SVM, which is used to map samples to a high-dimensional space. Commonly used kernel functions include linear kernel functions, polynomial kernel functions, Gaussian radial basis kernel functions, etc. The present invention uses a linear kernel function.

[0032] The final decision boundary is determined by the support vector, which is the sample point closest to the hyperplane. The decision boundary separates sample points of different categories and is used for classification during testing.

[0033] The data fusion analysis module extracts the features of the data in the image recognition module and the gas recognition module respectively, fuses the features extracted by the two modules, and forms a comprehensive feature representation; the fusion process is implemented through weighted summation to determine the weight of the features extracted by each module, and the weight is set by the user according to the actual situation; after the weighted summation, the fused features need to be normalized to ensure that the comprehensive features have a uniform scale and range, and the eigenvalues ​​are scaled to the range of [0, 1] through linear transformation.

[0034] The decision support module uses a decision tree to establish a risk assessment model, uses the fire level in the historical data as a label, and divides the fire events into different categories according to the fire level. The label will be used as the training target of the decision tree model; the decision tree algorithm is used to train the training data. The decision tree is a tree structure that makes decisions by recursively dividing the features; during the training of the decision tree, appropriate division nodes are selected according to the different data characteristics so that the samples represented by each node belong to the same category as much as possible. The trained decision tree model is evaluated using the cross-validation method to ensure the generalization ability and accuracy of the model. The evaluation indicators include accuracy, precision, recall rate and F1 score; when new data is input, the trained decision tree model is used for prediction. According to the new data features, the recursive division process of the decision tree model is used to predict the risk level of the fire and send an alarm signal to the relevant personnel; Establishing a fire scenario model, including building structure, environmental factors and fire-fighting facilities, is a model based on computer simulation; using historical data and scenario models, simulate the fire development process, predict the fire spread path, fire spread speed information, and the impact of the fire on the surrounding environment and buildings; based on the simulation results, predict the fire development trend and possible impact range according to historical data and scenario models; formulate corresponding emergency measures according to the fire prediction results, including personnel evacuation routes, fire extinguishing plans and material deployment; determine the specific implementation methods and priorities of various emergency measures according to the predicted fire spread path and impact range.

[0035] In the remote IoT control module, users can view the situation data of the fire scene in real time, remotely control various equipment and devices in the fire fighting system, and adjust the working status and parameters of the equipment in real time according to the situation at the fire scene; the remote IoT control module allows users to flexibly configure the parameters of fire fighting equipment and cameras, including: the direction of fire extinguishing agent injection, injection amount, camera viewing angle and camera resolution; the remote IoT control module supports users to communicate and coordinate with the staff at the fire scene in real time. Through the instant messaging function, users can exchange information with the on-site staff. Common fire fighting equipment that can be remotely operated includes but is not limited to: fire extinguishers, sprinkler systems, smoke exhaust systems, automatic sprinkler systems, gas fire extinguishing systems, fire doors and fire shutter doors, fire alarms, cameras, etc.

[0036] Firefighting equipment and control equipment need to be connected to the network through various communication technologies, such as Wi-Fi, Bluetooth, Zigbee, LoRa, etc. When the devices are connected to the network, they can transmit and communicate data through the network. In this way, the firefighting equipment can send the monitored data to the remote server, or receive remote instructions for control operations. In the Internet of Things, there will be a remote server for managing and processing the data sent by the device. This server is a cloud platform that is responsible for receiving the data sent by the device, storing, analyzing and processing it, and sending control instructions to the device. The remote server will provide an application programming interface API for third-party applications to interact with the Internet of Things system. Through the API, users can monitor the status of the device in real time, send control instructions, etc. through mobile applications or web applications.

[0037] When connecting devices to the Internet of Things, security and privacy protection are very important considerations. The present invention takes appropriate security measures, such as data encryption, identity authentication, permission control, etc., to protect the security and privacy of devices and data.

[0038] According to another embodiment of the present invention, Figure 2 A remote IoT control management method based on fire fighting is shown in a schematic diagram of steps. A remote IoT control management method based on fire fighting includes the following steps: S100, obtaining a video stream of the fire scene through a surveillance camera; S200, using image processing and deep convolutional neural networks to analyze and identify images in real time to determine the precise location, size and source of the fire; S300, deploy gas sensors to monitor the gas concentration at the fire scene in real time; S400, identifying gas components using a gas identification algorithm to determine the type and source of the gas causing the fire; S500, performing feature fusion on the data acquired by the image recognition module and the gas recognition module; S600: Analyze and model the fused data and comprehensively analyze the situation at the fire scene; S700, establishing a risk assessment model, conducting fire risk assessment based on the analysis results of the data fusion analysis module, and issuing an alarm signal to relevant personnel; S800, based on historical data and scenario models, simulates and predicts fire development trends and generates emergency response plans; S900 provides real-time viewing of fire scene data. Users can remotely control various equipment and devices of the fire protection system, flexibly configure the parameters of fire protection equipment and cameras, and support real-time communication and coordination with on-site staff.

[0039] According to step S200, after the deep convolutional neural network analyzes the image of the fire scene, it determines that the precise location of the fire is (40.7128° N, 74.0060° W), that is, latitude 40.7128 degrees and longitude 74.0060 degrees. This is the geographic coordinate location of the fire. After image processing and deep convolutional neural network analysis, the size of the fire is estimated to be about 100 square meters, which is estimated by the number of pixels of the flame in the image and a certain conversion relationship. The fire source is located near the north wall of the building, at a height of about 2.5 meters and a horizontal distance of about 5 meters from the center of the building. This location is obtained based on the features in the fire image combined with the analysis results of the deep convolutional neural network.

[0040] According to steps S300 and S400, the results show that the gas sensor detected a toxic gas whose concentration exceeded the safety threshold. Based on the feedback from the sensor, we can determine that the type of gas monitored is carbon monoxide. The results show that the monitored carbon monoxide concentration reached 500ppm, which exceeds the safety standard, indicating that there may be a serious risk of carbon monoxide poisoning at the fire scene. Based on the gas concentration data and the situation at the fire scene, we speculate that the monitored carbon monoxide is likely to be produced by combustion caused by the fire. This conclusion is based on the fact that carbon monoxide is usually a product of fire combustion and the concentration exceeds the safety threshold.

[0041] According to step S500, in the experimental results, we fuse the image features and the gas features to form a comprehensive feature representation. The fusion is achieved by weighted summation to ensure that the fused features can fully describe the situation at the fire scene.

[0042] According to step S600, the fused data is analyzed, and combined with the image features and gas features, we can obtain a comprehensive situation analysis of the fire scene. This includes the location, scale, location of the fire source, and the type and concentration of the monitored gas. Based on the results of the comprehensive analysis, we can conduct a fire risk assessment. According to the scale of the fire, the location of the fire source, and the monitored gas conditions, the impact of the fire on the surrounding environment and personnel is assessed, and the corresponding risk level is determined. If the fire risk assessment results show that the safety threshold is exceeded, the system automatically sends an alarm signal to the relevant personnel to remind them to take appropriate emergency measures.

[0043] According to steps S700 and S800, based on the comprehensive analysis results provided by the data fusion analysis module, we established a risk assessment model. The model takes into account factors such as the location, scale, location of the fire source, and the type and concentration of the monitored gas, and comprehensively evaluates the impact of the fire on the surrounding environment and personnel. The simulation results show that according to the risk assessment model, the fire is assessed as a high-risk event. This is because the scale of the fire is large, the location of the fire source is relatively close to the densely populated area, and the monitored gas concentration exceeds the safety threshold, posing a serious risk of poisoning. According to the fire risk assessment results, the system automatically sends an alarm signal to the relevant personnel to remind them of the severity of the fire and recommends taking appropriate emergency measures, such as evacuating personnel and mobilizing rescue forces.

[0044] According to step S900, the user successfully connects to the monitoring system of the fire scene through the remote IoT control module and views the video stream and monitoring data of the fire scene in real time. They can clearly observe the fire situation, including the size of the fire, the location of the fire source, and the smoke situation. The user can remotely control the fire fighting equipment through the remote IoT control module, such as adjusting the injection direction and injection amount of the fire extinguishing agent. If necessary, they can also start other fire fighting equipment, such as a sprinkler system or an automatic fire extinguishing device, to control the spread of the fire.

[0045] Users can flexibly configure the parameters of firefighting equipment and cameras according to the situation at the fire scene. They can adjust the camera's viewing angle and resolution to obtain clearer images of the fire scene, and adjust the working parameters of the firefighting equipment to meet different firefighting needs. Through the remote IoT control module, users can communicate and coordinate with on-site staff in real time. They can send messages or instructions to on-site personnel to understand the latest situation at the fire scene and coordinate firefighting actions.

[0046] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0047] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A remote IoT control and management system based on fire fighting, characterized by: include: Image recognition module: The video stream of the fire scene is obtained through the surveillance camera, and the image is analyzed and recognized in real time using image processing and deep convolutional neural network to determine the precise location, size and source of the fire; Gas identification module: deploys gas sensors to monitor the gas concentration at the fire scene in real time, and uses gas identification algorithms to identify the gas composition to determine the type and source of the gas causing the fire; Data fusion analysis module: It fuses the data obtained by the image recognition module and the gas recognition module, analyzes and models the fused data, and comprehensively analyzes the situation at the fire scene; Decision support module: Establish a risk assessment model, conduct fire risk assessment based on the analysis results of the data fusion analysis module, and issue alarm signals to relevant personnel; simulate and predict the development trend of fire based on historical data and scenario models, and generate emergency measures plans; Remote IoT control module: provides real-time viewing of fire scene data. Users can remotely control various equipment and devices of the fire protection system, flexibly configure the parameters of fire protection equipment and cameras, and support real-time communication and coordination with on-site staff.

2. A remote IoT control and management system based on fire fighting according to claim 1, characterized in that: The image recognition module obtains the video stream through the surveillance camera deployed at the fire scene. After receiving the video stream, the image is preprocessed. The preprocessing includes denoising, image enhancement and edge detection to remove the camera sensor noise and ambient light interference, improve the image clarity and visibility, and identify the boundaries and contours in the image; The preprocessed video stream is input into a deep convolutional neural network to obtain the precise location, size and source of the fire.

3. A remote IoT control and management system based on fire fighting according to claim 2, characterized in that: The deep convolutional neural network uses semantic segmentation technology to determine the precise location and size of the fire, and uses target detection technology to determine the location of the fire source; Collect image data of the scene after the fire in different scenarios, which comes from surveillance cameras, drone aerial photography and on-site photography by firefighters. The collected image data ensures that it can cover every corner and different perspectives of the fire scene; select the image annotation tool LabelImg and have the annotator perform pixel-level annotation. The annotator needs to draw the boundary of the flame area on the image and mark the location of the flame with pixel-level accuracy; use different colors or marks to distinguish different flames; also perform pixel-level annotation on the smoke area in the image. The annotator needs to mark the location and range of the smoke to ensure that the model can identify the smoke area; according to different scenarios, the annotator needs to annotate other items that affect fire fighting in different scenarios; Design a deep convolutional neural network suitable for semantic segmentation and target detection. The output of the network includes two parts: one is the semantic segmentation result of flame and smoke, which is used to determine the precise location and size of the fire; the other is the target detection result of the fire source location; Use U-Net for semantic segmentation and YOLO for object detection; Use the prepared image data of the fire scene to train the designed deep convolutional neural network; During the training process, an end-to-end training method is adopted to optimize the loss functions of semantic segmentation and object detection at the same time; According to the results of semantic segmentation, the flame and smoke areas can be obtained to determine the precise location and size of the fire. The size of the fire can be evaluated by calculating the area and number of pixels of the flame and smoke areas. According to the result of the target detection, the detection result of the fire source position can be obtained, the coordinates of the fire source position can be obtained, and the coordinates can be marked on the plan view of the fire scene.

4. A remote IoT control and management system based on fire fighting according to claim 3, characterized in that: The deep convolutional neural network uses U-Net for semantic segmentation. The U-Net network structure consists of an encoder and a decoder, wherein the encoder is used to extract high-level features of the image, and the decoder is used to map the extracted features back to the original image size. In order to perform the semantic segmentation task of flame and smoke, the final output of the network is set to two channels, which respectively represent the segmentation results of flame and smoke; The encoder inputs an image of the fire scene and outputs a feature map; the encoder is composed of several convolutional layers and pooling layers, wherein each convolutional layer is followed by an activation function ReLU for introducing nonlinearity, and the number of convolution kernels gradually increases to capture feature information; the decoder adopts an upsampling operation to gradually increase the resolution of the feature map through deconvolution to locate the position of the flame and smoke. After each upsampling operation, the decoder splices and fuses low-level and high-level features with the feature map of the corresponding layer of the encoder, thereby achieving semantic segmentation results; The loss function of U-Net is used to measure the difference between the model output and the true label. It uses a binary cross entropy loss function, where category one represents flame and category two represents smoke. Given an H*W image, let the output of U-Net be P and the true label be Y. Then the binary cross entropy loss function is The formula is as follows: Where N is the number of pixels, N = H*W; Indicates the true label of pixel i belonging to category c. If pixel i belongs to category c, then ,otherwise ; Represents the probability that the pixel i output by the model belongs to category c. For a binary classification task, each pixel output will have two probability values, representing the probability of flame and smoke respectively. The goal of the binary cross entropy loss function is to minimize value, so that the output probability distribution is close to the distribution of the true label.

5. A remote IoT control and management system based on fire fighting according to claim 3, characterized in that: The deep convolutional neural network uses YOLO for target detection. The YOLO network consists of a single convolutional neural network, which simultaneously predicts several bounding boxes and the class probabilities corresponding to the bounding boxes in one forward propagation; The loss function of YOLO consists of two parts: target classification loss and position regression loss. The cross entropy loss function is used to measure the classification accuracy of the target category. In the target detection task at the fire source location, the target classification loss function is It is expressed as: Where M is the number of fire source targets in the image; C is the number of categories; It is the indicator function of the target classification loss function, indicating whether the i'th bounding box contains the target of category c'; is the probability that the i'th bounding box belongs to category c' predicted by the model; The squared loss function is used to measure the regression accuracy of the model for the bounding box position, and the loss function is expressed as: in, is the weight of the position regression loss, It is The horizontal coordinate of the center coordinate of the bounding box, It is The ordinate of the center coordinate of the bounding box, It is The width of the bounding box, It is The height of the bounding box, It is the predicted The horizontal coordinate of the center coordinate of the bounding box, It is the predicted The ordinate of the center coordinate of the bounding box, It is the predicted The width of the bounding box, It is the predicted The height of the bounding box; is the indicator function of the square loss function, indicating that Whether the bounding box contains the object, if bounding boxes contain the object, then ,otherwise .

6. A remote IoT control and management system based on fire fighting according to claim 1, characterized in that: The gas sensor monitors the changes in gas concentration at the fire scene in real time and can detect several types of harmful gases. The gas concentration data collected by the gas sensor is transmitted to the gas identification module by wireless or wired means. The gas identification module uses the gas identification algorithm to analyze and process the gas concentration data collected by the sensor to identify the type and source of the gas caused by the fire; The gas identification algorithm uses a support vector machine algorithm to classify known types of gas and match the identified gas pattern with known fire-causing gas types and their characteristics to determine whether the monitored gas is related to the fire; In the feature matching process, the similarity and distance between feature vectors are compared by using Euclidean distance; The gas identification algorithm classifies and identifies the monitored gases according to the characteristics of the gas concentration data, and determines the type of the monitored gas by comparing it with a pre-established gas characteristic database. The gas identification algorithm analyzes the changing trends of the monitored gas concentration data, and infers the source of the gas caused by the fire based on the environmental factors at the fire scene.

7. A remote IoT control and management system based on fire fighting according to claim 1, characterized in that: The data fusion analysis module extracts the features of the data in the image recognition module and the gas recognition module respectively, and fuses the features extracted by the two modules to form a comprehensive feature representation; the fusion process is implemented through weighted summation to determine the weight of the features extracted by each module, and the weight is set by the user according to the actual situation; after the weighted summation, the fused features need to be normalized to ensure that the comprehensive features have a uniform scale and range, and the eigenvalues ​​are scaled to the range of [0,1] through linear transformation.

8. The remote IoT control and management system based on fire fighting according to claim 1 is characterized by: The decision support module uses a decision tree to establish a risk assessment model, takes the fire level in the historical data as a label, and divides the fire events into different categories according to the fire level. The label will be used as the training target of the decision tree model; the training data is trained using a decision tree algorithm. The decision tree is a tree structure that makes decisions by recursively dividing the features; in the training process of the decision tree, according to the different data features, appropriate division nodes are selected so that the samples represented by each node belong to the same category as much as possible; the trained decision tree model is evaluated using a cross-validation method to ensure the generalization ability and accuracy of the model, and the evaluation indicators include accuracy, precision, recall rate and F1 score; when new data is input, the trained decision tree model is used for prediction, and according to the new data features, the risk level of the fire is predicted through the recursive division process of the decision tree model, and an alarm signal is issued to relevant personnel; Establishing a fire scenario model, including building structure, environmental factors and fire-fighting facilities, is a model based on computer simulation; using historical data and scenario models, simulate the fire development process, predict the fire spread path, fire spread speed information, and the impact of the fire on the surrounding environment and buildings; based on the simulation results, predict the fire development trend and possible impact range according to historical data and scenario models; formulate corresponding emergency measures according to the fire prediction results, including personnel evacuation routes, fire extinguishing plans and material deployment; determine the specific implementation methods and priorities of various emergency measures according to the predicted fire spread path and impact range.

9. A remote IoT control and management system based on fire fighting according to claim 1, characterized in that: In the remote IoT control module, users can view the situation data of the fire scene in real time, remotely control various equipment and devices in the fire fighting system, and adjust the working status and parameters of the equipment in real time according to the situation at the fire scene; The remote IoT control module allows users to flexibly configure the parameters of firefighting equipment and cameras, including: extinguishing agent injection direction, injection volume, camera viewing angle and camera resolution; The remote IoT control module supports real-time communication and coordination between users and fire scene staff. Through the instant messaging function, users can exchange information with on-site staff.

10. A remote IoT control management method based on fire fighting, applied to a remote IoT control management system based on fire fighting as claimed in any one of claims 1 to 9, characterized in that: The following steps are involved: S100, obtaining a video stream of the fire scene through a surveillance camera; S200, using image processing and deep convolutional neural networks to analyze and identify images in real time to determine the precise location, size and source of the fire; S300, deploy gas sensors to monitor the gas concentration at the fire scene in real time; S400, identifying gas components using a gas identification algorithm to determine the type and source of the gas causing the fire; S500, performing feature fusion on the data acquired by the image recognition module and the gas recognition module; S600: Analyze and model the fused data and comprehensively analyze the situation at the fire scene; S700, establishing a risk assessment model, conducting fire risk assessment based on the analysis results of the data fusion analysis module, and issuing an alarm signal to relevant personnel; S800, based on historical data and scenario models, simulates and predicts fire development trends and generates emergency response plans; S900 provides real-time viewing of fire scene data. Users can remotely control various equipment and devices of the fire protection system, flexibly configure the parameters of fire protection equipment and cameras, and support real-time communication and coordination with on-site staff.

Images