Smoke and fire detection method and system based on deep reinforcement learning and electronic equipment

Abstract

The invention provides a smoke and fire detection method and system based on deep reinforcement learning and electronic equipment, and relates to the technical field of fire fighting. The method comprises the following steps: acquiring thermal imaging data of a forest area through a thermal imager, obtaining temperature distribution conditions of different positions, and determining position coordinates, the number and specific temperature values of temperature abnormal points for an area with abnormal temperature rise; if the temperature value of the temperature abnormal point exceeds a preset temperature threshold value, judging that the temperature abnormal point is a suspected fire point, and collecting a visible light video image of the area through a visible light camera according to the position of the suspected fire point; smog detection processing is carried out on the collected visible light video image, whether smog exists in the image is judged by analyzing the color, texture, motion and other characteristics of the image, and the concentration, morphological characteristics and diffusion speed of the smog are obtained; the meteorological parameters such as the wind speed and the wind direction of a fire occurrence area are obtained through the meteorological sensor, the meteorological parameters are input into the fire spreading model, and the spreading direction and the spreading range of the fire under the current wind field condition are simulated.

Description

Technical Field

[0001] The present invention relates to the field of information technology, and in particular to a smoke and fire detection method, system and electronic equipment based on deep reinforcement learning. Background Art

[0002] In the forest fire early warning system, there is a technical contradiction in the integration of temperature anomaly detection and smoke recognition. Infrared thermal imaging can quickly capture temperature anomalies, but it is easily affected by sunlight reflection and other interferences to produce false alarms. Visible light video analysis has high reliability, but it is not sensitive enough to detect small flames in the early stage. How to reduce the false alarm rate while ensuring detection sensitivity is a key challenge. In addition, it is difficult to balance the accuracy and computational efficiency of the fire spread prediction model. High-precision models need to consider multi-dimensional parameters such as terrain, vegetation, and meteorology, which requires large amounts of calculation and is difficult to meet real-time requirements. Although the simplified model has a fast calculation speed, it is difficult to accurately describe the evolution of fire under complex terrain conditions. How to balance model accuracy and response speed with limited computing resources is another technical problem. At the same time, in the data assimilation process of on-site monitoring data and model prediction results, how to deal with the uncertainty of data of different scales and sources, and how to achieve rapid tuning of model parameters in a dynamically changing fire environment are also urgent problems to be solved. Solving these technical contradictions is crucial to improving the reliability and practicality of the forest fire early warning system. Summary of the invention

[0003] The present invention provides a smoke and fire detection method based on deep reinforcement learning, which mainly includes: The thermal imaging data of the forest area is collected by a thermal imager to obtain the temperature distribution at different locations. For areas with abnormally high temperatures, the location coordinates, number and specific temperature values ​​of the temperature anomaly points are determined. If the temperature value of the temperature anomaly point exceeds the preset temperature threshold, it is determined to be a suspected fire point. According to the location of the suspected fire point, the visible light video image of the area is collected by a visible light camera. The collected visible light video image is processed for smoke detection. By analyzing the color, texture, motion and other features of the image, it is determined whether there is smoke in the image, and the concentration, morphological characteristics and diffusion speed of the smoke are obtained. According to the temperature value and number of the suspected fire point and the smoke detection results, a data fusion algorithm is used to conduct a comprehensive analysis of multi-source data to determine whether there is a forest fire. If there is a fire, the fire warning level is determined and the possible cause of the fire is analyzed. If the monitoring In the case of forest fire, the main types of combustible materials and their accumulation are determined based on the tree species and density information around the fire location, and the fire propagation model is used to calculate the growth rate of the fire under different wind speed and direction conditions; the wind speed, wind direction and other meteorological parameters in the fire area are obtained through meteorological sensors, and the meteorological parameters are input into the fire spread model to simulate the spread direction and range of the fire under the current wind field conditions; the real-time temperature, humidity and other monitoring data of the fire monitoring point are assimilated with the prediction results of the fire spread model, and the parameters of the fire spread model are corrected in real time to continuously improve the accuracy of fire spread prediction; according to the fire warning level and the fire spread prediction results, fire warning information is automatically generated, and the warning information is sent to nearby forest rangers and fire departments in real time through the wireless communication network to provide a reference for fire fighting decisions.

[0004] The present invention further provides a smoke and fire detection system based on deep reinforcement learning, comprising The acquisition unit is used to collect thermal imaging data of the forest area through a thermal imager to obtain the temperature distribution at different locations, and determine the location coordinates, number and specific temperature values ​​of the temperature abnormality points in the area with abnormally high temperature; An abnormality unit is used to determine that a temperature abnormality point is a suspected fire point if the temperature value of the temperature abnormality point exceeds a preset temperature threshold, and collect visible light video images of the area through a visible light camera according to the location of the suspected fire point; The image unit is used to perform smoke detection processing on the collected visible light video images, and judge whether there is smoke in the image by analyzing the color, texture, motion and other characteristics of the image, and obtain the concentration, morphological characteristics and diffusion speed of the smoke; The multi-source unit is used to conduct a comprehensive analysis of multi-source data based on the temperature value, number and smoke detection results of suspected fire points, using a data fusion algorithm to determine whether there is a forest fire. If there is a fire, the fire warning level is determined and the possible cause of the fire is analyzed; The monitoring unit is used to determine the main burning material type and its accumulation amount based on the tree species and density information around the fire location if a forest fire is detected, and to calculate the growth rate of the fire under different wind speed and direction conditions using a fire propagation model; The fire spread unit is used to obtain meteorological parameters such as wind speed and wind direction in the fire area through meteorological sensors, input the meteorological parameters into the fire spread model, and simulate the spread direction and range of the fire under the current wind field conditions; The prediction unit is used to assimilate the real-time temperature, humidity and other monitoring data of the fire monitoring point with the prediction results of the fire spread model, and to perform real-time correction on the parameters of the fire spread model to continuously improve the accuracy of fire spread prediction; The early warning unit is used to automatically generate fire warning information according to the fire warning level and fire spread prediction results, and send the warning information to nearby forest rangers and fire departments in real time through the wireless communication network to provide reference for fire fighting decisions.

[0005] The present invention further provides an electronic device, comprising a memory and a processor, wherein the memory is used to store information including program instructions, and the processor is used to control the execution of the program instructions, and when the program instructions are loaded and executed by the processor, the aforementioned fireworks detection method based on deep reinforcement learning is implemented.

[0006] The technical solution provided by the embodiment of the present invention may have the following beneficial effects: The present invention discloses an intelligent monitoring and early warning method for forest fires. The method collects forest temperature distribution data through a thermal imager, identifies temperature anomalies, and comprehensively judges the fire conditions by combining visible light image smoke detection results. For confirmed fire conditions, a fire propagation model is used to predict the spread of the fire based on surrounding forest information and meteorological parameters. The model is calibrated through real-time monitoring data to improve prediction accuracy. Finally, fire warning information is generated and sent to relevant personnel. The present invention realizes intelligent monitoring, accurate early warning and fire spread prediction of forest fires, provides technical support for timely discovery and effective extinguishing of forest fires, is conducive to reducing losses caused by forest fires, and protects forest resources and the ecological environment. The method integrates multi-source data and multiple algorithms, and has strong practicality and reliability. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 This is a flow chart of a smoke and fire detection method based on deep reinforcement learning of the present invention.

[0008] Figure 2 Schematic diagram of a smoke and fire detection method based on deep reinforcement learning of the present invention.

[0009] Figure 3This is another schematic diagram of a smoke and fire detection method based on deep reinforcement learning of the present invention. DETAILED DESCRIPTION

[0010] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this specification.

[0011] like Figure 1-3 In this embodiment, a method for detecting fireworks based on deep reinforcement learning may specifically include: S101. Collect thermal imaging data of the forest area through a thermal imager to obtain temperature distribution at different locations, and determine the location coordinates, number and specific temperature values ​​of abnormal temperature points in areas with abnormally high temperatures.

[0012] Collect thermal imaging data of forest areas through thermal imagers to obtain temperature distribution at different locations Preprocess the collected thermal imaging data, including denoising and correction, to improve data quality Divide the preprocessed thermal imaging data into multiple small areas, each of which corresponds to a specific location in the forest Perform statistical analysis on the temperature data of each small area, and calculate the average temperature and temperature variance of the area Based on historical data and empirical thresholds, determine whether the temperature of each small area is abnormally high If the temperature of a small area is abnormally high, mark the area as a temperature anomaly point Summarize all temperature anomaly points to obtain the location coordinates, number and specific temperature values ​​of the temperature anomaly points Use spatial clustering algorithm to cluster the temperature anomaly points and identify abnormal temperature areas Visually display the location, range, temperature distribution and other information of the abnormal temperature area, and generate alarm information to send to relevant personnel for disposal Specifically, thermal imaging data of the forest area is collected by a thermal imager to obtain the temperature distribution at different locations. The infrared detector array of the thermal imager has a resolution of 640×480, a field of view of 30°×22°, a temperature measurement range of -20℃ to 150℃, and an accuracy of ±2℃. The collected thermal imaging data is preprocessed, including using the median filter algorithm to remove salt and pepper noise and using a blackbody radiation source for radiation correction to improve data quality. The preprocessed thermal imaging data is divided into 50×50 small areas, each of which corresponds to a specific location of 10 meters×10 meters in the forest. The temperature data of each small area is statistically analyzed to calculate the average temperature and temperature variance of the area. According to historical data and empirical thresholds, it is determined whether the average temperature of each small area exceeds 45℃ and whether the temperature variance exceeds 5℃. If so, the area is marked as a temperature anomaly point. All temperature anomaly points are summarized to obtain the location coordinates, number and specific temperature values ​​of the temperature anomaly points. The DBSCAN spatial clustering algorithm is used to cluster the temperature anomaly points, and the cluster radius is set to 20 meters and the minimum number of points is set to 5 to identify the abnormal temperature area. The location, range, temperature distribution and other information of the abnormal temperature area are visualized to generate heat maps and isotherm maps, and the alarm information is sent to forest managers in the form of text messages and emails to indicate the location and severity of possible fires so that timely measures can be taken to deal with them.

[0013] S102: If the temperature value of the temperature abnormal point exceeds a preset temperature threshold, it is determined to be a suspected fire point, and according to the location of the suspected fire point, a visible light video image of the area is collected by a visible light camera.

[0014] The thermal imaging camera is used to collect thermal imaging video images of the power plant area, and the temperature value of each pixel is obtained. According to the preset temperature threshold, the temperature value of each pixel is compared, and the pixel points with temperature values ​​exceeding the threshold are screened out to determine them as temperature abnormal points. The clustering algorithm is used to perform cluster analysis on the temperature abnormal points to obtain multiple temperature abnormal areas. The center position coordinates of each temperature abnormal area are calculated as the position coordinates of the suspected fire point. According to the position coordinates of the suspected fire point, the visible light camera is controlled to rotate and zoom and adjusted to the area of ​​the suspected fire point. The visible light camera is used to collect visible light video images of the suspected fire point area. The collected visible light video images are subjected to smoke detection and analysis, and the smoke concentration value in the image is calculated. If the smoke concentration value exceeds the preset smoke concentration threshold, the suspected fire point is judged to be a real fire point, and a fire alarm is triggered. The position coordinates of the real fire point, the visible light video image and other information are uploaded to the fire perception system for subsequent trend analysis and fire extinguishing decision-making. Specifically, the thermal imaging camera is used to collect thermal imaging video images of the power plant area and obtain the temperature value of each pixel. The system presets the temperature threshold to 80℃, and screens out abnormal points whose temperature exceeds the threshold by point-by-point comparison. The K-means clustering algorithm is used to cluster the abnormal points, and the cluster number K=5 is set. Five abnormal areas are obtained after 20 iterations, and the center coordinates of each area are calculated as the suspected fire point. According to the coordinates of the suspected fire point, the visible light camera is controlled to rotate 60° horizontally and 45° vertically through the two-axis servo and 3x optical zoom to collect the visible light video image of the area. The YOLOv3 smoke detection model is used to analyze the image frame by frame, and the smoke concentration threshold is set to 80%. When the smoke concentration of 30 consecutive frames exceeds the threshold, it is determined to be a real fire and trigger an alarm. At the same time, the fire point coordinates, visible light video and other information are uploaded to the fire perception system. The system uses a convolutional neural network model to predict and analyze the development trend of the fire, and automatically dispatches fire-fighting equipment and personnel to put out the fire according to the prediction results.

[0015] S103: Perform smoke detection processing on the collected visible light video image, and determine whether there is smoke in the image by analyzing the image's color, texture, motion and other features, and obtain the smoke's concentration, morphological characteristics and diffusion speed.

[0016] Collect visible light video images to obtain the image's color, texture, motion and other feature information Based on the acquired feature information, divide the image into different areas through the image segmentation algorithm Perform color analysis on the divided areas to determine whether the color distribution of the pixels in the area conforms to the color characteristics of smoke If the color distribution of the pixels in the area conforms to the smoke characteristics, perform texture analysis on the area to extract the texture features of the pixels in the area Based on the extracted texture features, use the texture classification algorithm to determine whether the area is a smoke area If it is determined to be a smoke area, obtain the motion trajectory of the area in continuous video frames through the target tracking algorithm Based on the acquired motion trajectory, calculate the diffusion speed and direction of the smoke area The number of pixels passing through the smoke area and the total number of image pixels calculates the concentration of the smoke The position, concentration, diffusion speed and other information of the detected smoke area are integrated to obtain the final smoke detection result, and trigger the corresponding alarm mechanism Specifically, when collecting visible light video images, a high-resolution camera is used to obtain a 640x480 pixel color image at a speed of 30 frames per second. The image is divided into several regions based on the color difference threshold ΔE=15 between pixels through the region growing-based image segmentation algorithm. The mean and variance of the pixel color distribution are calculated for each region. If the mean is within the grayscale range [180,220] and the variance is less than 30, the color of the region is considered to meet the smoke characteristics. The gray-level co-occurrence matrix is ​​then used to extract the texture features of the region, including contrast, correlation, entropy, etc., and the support vector machine classifier is used to determine whether it is a smoke region. For the detected smoke region, the Kalman filter algorithm is used to track its motion trajectory within 20 consecutive frames, and the smoke diffusion speed is estimated by the ratio of the trajectory length to the total time, and the diffusion direction is represented by the direction of the line connecting the end point and the starting point of the trajectory. Finally, the proportion of smoke pixels to the total pixels of the image is the smoke concentration, and an alarm is triggered when the concentration exceeds 5%. The smoke detection results are generated by integrating the location, concentration, diffusion speed and other parameters of the smoke region.

[0017] S104. Based on the temperature value, number and smoke detection results of the suspected fire points, a data fusion algorithm is used to comprehensively analyze the multi-source data to determine whether there is a forest fire. If there is a fire, the fire warning level is determined and the possible cause of the fire is analyzed.

[0018] The temperature value of the suspected fire point is obtained through the thermal imaging camera, and the presence of abnormal temperature points is determined based on the temperature threshold. If there are abnormal temperature points, the visible light camera is triggered to detect smoke in the area, and the smoke density is analyzed through the image processing algorithm. According to the number of abnormal temperature points and the smoke density, the multi-source data is comprehensively analyzed using the data fusion algorithm to obtain the comprehensive fire assessment results. If the comprehensive assessment results show that there is a forest fire, the fire warning level is determined based on the abnormal temperature value and smoke density. According to the spatial distribution characteristics of the abnormal temperature points, the possible causes of the fire are analyzed in combination with the geographic information system and meteorological data. If the comprehensive assessment results show that there is no forest fire, it is determined whether the abnormal temperature points are caused by other factors, such as direct sunlight, equipment heating, etc. If the abnormal temperature points are caused by other factors, the temperature threshold is updated, the fire judgment rules are optimized, and the accuracy of fire detection is improved. According to the fire warning level and the analysis results of the cause of the fire, the fire alarm information is automatically generated and sent to the command center and relevant departments through the communication network. The location, scope and development trend of the fire points are displayed in the GIS system of the command center to provide data support for fire fighting decisions. Specifically, the temperature value of the suspected fire point is obtained through the thermal imaging camera, and according to the preset temperature threshold, such as 300℃, it is judged whether there is an abnormal temperature point. If there is an abnormal temperature point, the visible light camera is triggered to detect smoke in the area, and the smoke concentration is analyzed by the image segmentation algorithm based on deep learning. If the proportion of smoke pixels exceeds 20%, it is considered that there is obvious smoke. According to the number of abnormal temperature points and smoke concentration, the fuzzy comprehensive evaluation algorithm is used to comprehensively analyze the multi-source data, set the membership function of temperature and smoke concentration, calculate the comprehensive membership, and obtain the comprehensive evaluation result of the fire. If the comprehensive evaluation result exceeds 6, it indicates that there is a forest fire. The fire warning level is determined according to the abnormal temperature value and smoke concentration. The highest warning is reached when the temperature exceeds 500℃ and the smoke concentration exceeds 50%. According to the spatial distribution characteristics of abnormal temperature points, combined with geographic information systems and meteorological data, machine learning algorithms such as support vector machines are used to classify the causes of fire, such as wild fire, lightning strikes, arson, etc. If the comprehensive evaluation result is lower than 3, it is determined whether the abnormal temperature point is caused by other factors. By analyzing the temperature change curve and image features, interference factors such as direct sunlight and equipment heating are identified. If the abnormal temperature point is caused by other factors, the temperature threshold is dynamically updated through the adaptive learning algorithm, the fire judgment rules are optimized, and the accuracy of fire detection is improved. According to the fire warning level and the analysis results of the cause of the fire, the fire alarm information is automatically generated and sent to the command center and relevant departments through the Internet of Things communication protocol. In the GIS system of the command center, the latitude and longitude position, impact range and spread trend of the fire point are displayed based on the electronic map, and the fire spread model and terrain data are combined to predict the development of the fire and provide data support for fire fighting decisions.

[0019] S105. If a forest fire is detected, the main burning material type and its accumulation amount shall be determined based on the tree species and density information around the fire location, and the fire propagation model shall be used to calculate the growth rate of the fire under different wind speed and direction conditions.

[0020] According to the coordinates of the fire location obtained by the fire awareness system, the forest species and density information around the location are queried in the GIS system. Using remote sensing image interpretation technology, satellite or drone images of the fire area are analyzed to further determine parameters such as forest species, density, and terrain characteristics. By consulting the forest combustible database, the combustion characteristic parameters such as the storage volume and moisture content of different forest species are obtained. According to the parameters such as the storage volume and moisture content of forest combustibles, the combustion calorific value and combustion rate of the main combustible materials in the fire area are calculated. Real-time meteorological data when the fire occurs are obtained, including parameters such as temperature, humidity, wind speed, and wind direction. Using a fire propagation model such as the Rothermel model or the Farsite model, combined with combustible material parameters and meteorological parameters, the spread speed and direction of the fire under different wind speed and direction conditions are simulated and calculated. If the fire spread trend calculated by the model is consistent with the actual monitoring data, the model is judged to be applicable and the model is continuously used for fire prediction; if the deviation is large, the model parameters are adjusted and the simulation calculation is re-performed. According to the fire prediction results, the fire spread range and risk level map are drawn in the GIS system to provide reference for fire fighting decisions. The fire scene dynamics are continuously monitored, the latest fire parameters are obtained, and the fire prediction results are updated in real time until the fire is extinguished.

[0021] Specifically, when the fire awareness system obtains the coordinates of the fire location, the system automatically queries the forest distribution information within a radius of 5 kilometers from the location in GIS, and obtains the forest density distribution map through spatial interpolation analysis. At the same time, the system calls high-resolution satellite image data and drone aerial images, and uses a semantic segmentation algorithm based on deep learning to extract the distribution and coverage information of different vegetation types in the image to form a regional vegetation classification map. The system compares the combustible accumulation and moisture content parameters of different vegetation types such as coniferous forests, broad-leaved forests, and shrubs in the forest information database, and combines the vegetation classification map to calculate the distribution of combustible loads in different plots in the fire area, and thus obtains the regional combustion calorific value distribution map and the combustion rate distribution map. The system obtains the temperature, humidity, wind speed and direction data of the meteorological stations around the fire site in real time, and generates the micro-meteorological distribution map of the fire site through spatial interpolation. In the Farsite fire simulation model, combustible parameters, terrain parameters and meteorological parameters are set to simulate and calculate the spread range and speed of the fire under different wind speed and direction scenarios. The simulation results are compared with the actual fire data fed back by the fire monitoring system. If the deviation between the two is less than 5%, the model is considered applicable. If the deviation exceeds 20%, the model parameters are automatically adjusted and the simulation is run again until the deviation is less than 5%. The system updates the fire parameters every 10 minutes, generates a forecast map of the fire spread trend for the next 1 hour and 2 hours, and marks the location of the fire head, the direction of spread and the risk level. The forecast results are automatically pushed to the fire scene command center and the local emergency management department to provide a reference for fire fighting decisions.

[0022] S106. Obtain meteorological parameters such as wind speed and wind direction in the fire occurrence area through meteorological sensors, input the meteorological parameters into the fire spread model, and simulate the spread direction and range of the fire under the current wind field conditions.

[0023] Meteorological parameters such as wind speed and wind direction in the fire area are obtained through meteorological sensors. A mathematical model of fire spread is established based on the obtained meteorological parameters, combined with fire environment factors, fire source characteristics and other factors. The convection-diffusion equation and atmospheric boundary layer theory are used to describe the spatial distribution of wind speed near the ground to obtain the wind field model. The Stefan-Boltzmann law is used to describe the intensity of fire thermal radiation to obtain the thermal radiation model. Meteorological parameters such as wind speed and wind direction are input into the established mathematical model of fire spread. If the wind speed exceeds the preset threshold, it is judged that the wind field has an important influence on the spread of the fire, and the relevant parameters in the wind field model are adjusted. The thermal radiation intensity within the fire spread range is calculated according to the thermal radiation model to determine the fire impact range. The calculation results of the wind field model and the thermal radiation model are combined to simulate the direction and range of fire spread under the current wind field conditions. The fire spread simulation results are compared and analyzed with the real-time monitoring data, and the model parameters are dynamically adjusted to improve the prediction accuracy. Specifically, meteorological parameters such as wind speed and wind direction of the fire scene can be obtained in real time by arranging meteorological sensors such as anemometers and wind vanes in the fire area. The anemometer can measure wind speed of 0-30m / s with an accuracy of 1m / s; the wind vane can measure wind direction of 0-360° with an accuracy of 1°. According to the acquired wind speed and wind direction data, combined with environmental factors such as the terrain, vegetation, and distribution of combustibles in the fire scene, as well as the location of the fire source, the burning area and other characteristics, a mathematical model of fire spread based on partial differential equations is established. The finite element method is used to solve the model to obtain the spread range and spread speed of the fire at different times. At the same time, the convection diffusion equation and the atmospheric boundary layer theory are used to establish a wind field model that describes the spatial distribution of wind speed within a height of 10 meters near the ground. The model takes into account the influence of factors such as surface roughness and atmospheric stability, and the wind speed distribution adopts the form of logarithmic rate or exponential rate function. Using the Stefan-Boltzmann law, the heat radiation flux of the fire is calculated according to the flame temperature, and then combined with the radiation propagation model, the heat radiation intensity at different positions is obtained, thereby determining the influence range of heat radiation. The measured wind speed and wind direction data are input into the fire spread model and wind field model. When the wind speed exceeds the threshold of 5m / s, the influence of the wind field on the spread of the fire cannot be ignored, and the wind speed distribution function and related parameters in the wind field model need to be adjusted. The calculation results of the wind field model and the thermal radiation model can be used to simulate and predict the spread direction, spread speed and influence range of the fire in the next hour under the current wind field conditions, providing a reference for fire fighting decisions. The fire spread simulation results are compared and analyzed with the real-time fire monitoring data obtained by drones, satellites, etc., the deviation between the simulation results and the actual situation is calculated, and the parameters in the model, such as combustible distribution, terrain factors, etc., are dynamically adjusted to continuously optimize the model and improve the prediction accuracy.

[0024] S107. Assimilate the real-time temperature, humidity and other monitoring data of the fire monitoring point with the prediction results of the fire spread model, perform real-time correction on the parameters of the fire spread model, and continuously improve the accuracy of fire spread prediction.

[0025] The real-time temperature, humidity and other monitoring data of the fire monitoring point are obtained through the perception layer equipment, and the data is transmitted to the back-end platform through the transmission network. After receiving the monitoring data, the fire perception system of the back-end platform preprocesses the data, including data cleaning, normalization and other operations, to obtain standardized monitoring data. The fire perception system calls the fire spread model and calculates the prediction results of the fire spread according to the current model parameters and standardized monitoring data. The prediction results of the fire spread model are compared with the actual fire monitoring data to calculate the prediction error. If the prediction error exceeds the preset threshold, the data assimilation process is triggered; otherwise, the current fire spread model parameters are continued to be used for prediction. In the data assimilation process, data assimilation algorithms such as Kalman filtering and ensemble Kalman filtering are used to fuse the real-time monitoring data with the prediction results of the fire spread model, and the model parameters are corrected in real time. The data assimilation algorithm dynamically adjusts the key parameters in the fire spread model, such as the burning rate, wind speed and direction, according to the prediction error and the credibility of the monitoring data, so that the model prediction results are more consistent with the actual fire situation. The corrected fire spread model parameters are updated to the fire situation perception system for the next fire situation prediction. The fire situation perception system, based on the corrected fire spread model and combined with real-time monitoring data, cyclically executes the steps of prediction, comparison, and data assimilation to continuously improve the accuracy of fire spread prediction and provide a more reliable basis for fire warning and decision-making.

[0026] Specifically, the fire awareness system collects fire monitoring point data in real time through temperature and humidity sensors and other equipment, and transmits it to the back end through a wireless sensor network. The system cleans and normalizes the monitoring data, and maps temperature, humidity and other data to the 0-1 interval. The standardized data is then input into the fire spread numerical simulation model. The model discretizes forest terrain, vegetation, wind direction and other parameters based on the finite element method, and uses the Rothermel semi-empirical model to calculate the flame spread speed and direction. The model outputs a prediction result every 30 seconds, which is compared with the actual fire data. If the prediction error exceeds 5%, data assimilation is triggered. The assimilation process uses the ensemble Kalman filter algorithm to generate 100 model parameter sets, predict in parallel, estimate the posterior distribution of parameters based on the error covariance matrix, and adjust key parameters such as increasing wind speed by 20% and reducing burning rate by 10%. The corrected parameters are updated to the model, predicted again, assimilated, and iteratively optimized, so that the prediction accuracy is increased to more than 90%, providing a reliable basis for fire warning.

[0027] S108. Automatically generate fire warning information based on the fire warning level and fire spread prediction results, and send the warning information to nearby forest rangers and fire departments in real time through the wireless communication network to provide reference for fire fighting decisions.

[0028] The fire warning level and fire spread prediction result data are obtained through the fire awareness system. The data fusion algorithm is used to conduct a comprehensive analysis of the fire warning level and fire spread prediction result data. According to the comprehensive analysis results, the fire risk level is judged. If the fire risk level exceeds the preset threshold, the fire warning information generation process is triggered. Through natural language generation technology, combined with fire warning level, fire spread direction, speed and other information, the fire warning information text is automatically generated. The communication address information of nearby forest rangers and fire departments is obtained. The wireless communication protocol is used to encapsulate the generated fire warning information text into a data packet. The fire warning information data packet is sent to the target communication address in real time through the wireless communication network. The sent fire warning information is synchronously displayed on the command and dispatch platform for reference in fire fighting decision-making. Specifically, the fire awareness system collects temperature distribution data at the fire scene through an infrared thermal imager, and uses a deep learning algorithm to classify the fire warning level, with an accuracy rate of more than 95%. At the same time, based on the fire spread model, the spread direction and speed of the fire in the next hour are predicted, and the average error is controlled within 10%. The data fusion algorithm adopts the weighted average method, comprehensively considering the fire warning level weight 6 and the fire spread prediction result weight 4, to obtain the comprehensive assessment value of the fire risk. When the comprehensive assessment value exceeds 7, the fire warning information generation process is triggered. The natural language generation module extracts key information such as the fire warning level, the direction of fire spread (such as the southeast direction), and the spread speed (such as 2 kilometers per hour) based on the preset information template, and automatically generates the warning information text "A fire occurred in the southeast area of ​​the forest, the warning level is level 1, the fire spreads to the southeast, and the spread speed is about 2 kilometers per hour. Please ask nearby forest rangers and fire departments to deal with it in time." The system automatically obtains the communication addresses of forest rangers and fire departments within a 5-kilometer range, and uses the MQTT protocol to encapsulate the warning information text into a JSON format data packet, which is sent in real time through the 4G wireless communication network. The command and dispatch platform is connected to the warning information sending module through the WebSocket interface, receives the sent warning information in real time, and displays it in the form of a list. At the same time, it marks the fire location on the electronic map to provide a reference for the fire department's fire fighting decision-making.

[0029] The present invention further provides a smoke and fire detection system based on deep reinforcement learning, comprising The acquisition unit is used to collect thermal imaging data of the forest area through a thermal imager to obtain the temperature distribution at different locations, and determine the location coordinates, number and specific temperature values ​​of the temperature abnormality points in the area with abnormally high temperature; The abnormal unit is used to determine that the temperature abnormal point is a suspected fire point if the temperature value exceeds a preset temperature threshold, and collect visible light video images of the area through a visible light camera according to the location of the suspected fire point; The image unit is used to perform smoke detection processing on the collected visible light video images, and judge whether there is smoke in the image by analyzing the image's color, texture, motion and other characteristics, and obtain the smoke's concentration, morphological characteristics and diffusion speed; The multi-source unit is used to conduct a comprehensive analysis of multi-source data based on the temperature value, number and smoke detection results of suspected fire points, using a data fusion algorithm to determine whether there is a forest fire. If there is a fire, the fire warning level is determined and the possible cause of the fire is analyzed; The monitoring unit is used to determine the main burning material type and its accumulation amount based on the tree species and density information around the fire location if a forest fire is detected, and to calculate the growth rate of the fire under different wind speed and direction conditions using a fire propagation model; The fire spread unit is used to obtain meteorological parameters such as wind speed and wind direction in the fire area through meteorological sensors, input the meteorological parameters into the fire spread model, and simulate the spread direction and range of the fire under the current wind field conditions; The prediction unit is used to assimilate the real-time temperature, humidity and other monitoring data of the fire monitoring point with the prediction results of the fire spread model, and to perform real-time correction on the parameters of the fire spread model to continuously improve the accuracy of fire spread prediction; The early warning unit is used to automatically generate fire warning information according to the fire warning level and fire spread prediction results, and send the warning information to nearby forest rangers and fire departments in real time through the wireless communication network to provide reference for fire fighting decisions.

[0030] The present invention further provides an electronic device, comprising a memory and a processor, wherein the memory is used to store information including program instructions, and the processor is used to control the execution of the program instructions, and when the program instructions are loaded and executed by the processor, the aforementioned fireworks detection method based on deep reinforcement learning is implemented.

[0031] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for detecting fireworks based on deep reinforcement learning, characterized in that: The method comprises: The thermal imaging data of the forest area is collected by a thermal imager to obtain the temperature distribution at different locations. For areas with abnormally high temperatures, the location coordinates, number and specific temperature values ​​of the temperature anomaly points are determined. If the temperature value of the temperature anomaly point exceeds the preset temperature threshold, it is determined to be a suspected fire point. According to the location of the suspected fire point, the visible light video image of the area is collected by a visible light camera. The collected visible light video image is processed for smoke detection. By analyzing the color, texture, motion and other features of the image, it is determined whether there is smoke in the image, and the concentration, morphological characteristics and diffusion speed of the smoke are obtained. According to the temperature value and number of the suspected fire point and the smoke detection results, a data fusion algorithm is used to conduct a comprehensive analysis of multi-source data to determine whether there is a forest fire. If there is a fire, the fire warning level is determined and the possible cause of the fire is analyzed. If the monitoring In the case of forest fire, the main types of combustible materials and their accumulation are determined based on the tree species and density information around the fire location, and the fire propagation model is used to calculate the growth rate of the fire under different wind speed and direction conditions; the wind speed, wind direction and other meteorological parameters in the fire area are obtained through meteorological sensors, and the meteorological parameters are input into the fire spread model to simulate the spread direction and range of the fire under the current wind field conditions; the real-time temperature, humidity and other monitoring data of the fire monitoring point are assimilated with the prediction results of the fire spread model, and the parameters of the fire spread model are corrected in real time to continuously improve the accuracy of fire spread prediction; according to the fire warning level and the fire spread prediction results, fire warning information is automatically generated, and the warning information is sent to nearby forest rangers and fire departments in real time through the wireless communication network to provide a reference for fire fighting decisions.

2. The method according to claim 1, characterized in that The thermal imaging data of the forest area is collected by a thermal imager to obtain the temperature distribution at different locations, and for the area with abnormally high temperature, the location coordinates, quantity and specific temperature value of the abnormal temperature points are determined, including: Collect thermal imaging data of forest areas through thermal imagers, obtain temperature distribution at different locations, preprocess the collected thermal imaging data, including denoising and correction, to improve data quality. Divide the preprocessed thermal imaging data into multiple small areas, and each small area corresponds to a specific location in the forest. Perform statistical analysis on the temperature data of each small area, and calculate the average temperature and temperature variance of the area. According to historical data and empirical thresholds, judge whether the temperature of each small area is abnormally high. If the temperature of a small area is abnormally high, mark the area as a temperature anomaly point. Summarize all temperature anomaly points to obtain the location coordinates, number and specific temperature values ​​of the temperature anomaly points. Use spatial clustering algorithm to cluster the temperature anomaly points and identify the abnormal temperature area. Visualize the location, range and temperature distribution of the abnormal temperature area, and generate alarm information to send to relevant personnel for disposal.

3. The method according to claim 1, characterized in that If the temperature value of the temperature abnormal point exceeds the preset temperature threshold, it is judged as a suspected fire point, and according to the location of the suspected fire point, a visible light video image of the area is collected by a visible light camera, including: The thermal imaging video images of the power plant area are collected by the thermal imaging camera to obtain the temperature value of each pixel point. According to the preset temperature threshold, the temperature value of each pixel point is compared, and the pixel points with temperature values ​​exceeding the threshold are screened out to determine the temperature abnormal points. The clustering algorithm is used to perform cluster analysis on the temperature abnormal points to obtain multiple temperature abnormal areas. The center position coordinates of each temperature abnormal area are calculated as the position coordinates of the suspected fire point. According to the position coordinates of the suspected fire point, the visible light camera is controlled to rotate and zoom and adjusted to the area of ​​the suspected fire point. The visible light video image of the suspected fire point area is collected by the visible light camera. Smoke detection and analysis are performed on the collected visible light video image to calculate the smoke concentration value in the image. If the smoke concentration value exceeds the preset smoke concentration threshold, the suspected fire point is judged to be a real fire point, and the fire alarm is triggered. The position coordinates of the real fire point and the visible light video image and other information are uploaded to the fire perception system for subsequent trend analysis and fire extinguishing decision-making.

4. The method according to claim 1, characterized in that: The smoke detection process is performed on the collected visible light video image, and by analyzing the image's color, texture, motion and other features, it is determined whether there is smoke in the image, and the concentration, morphological characteristics and diffusion speed of the smoke are obtained, including: Collect visible light video images to obtain the image's color, texture, motion and other feature information. Based on the acquired feature information, divide the image into different areas through the image segmentation algorithm. Perform color analysis on the divided areas to determine whether the color distribution of the pixels in the area conforms to the color characteristics of smoke. If the color distribution of the pixels in the area conforms to the smoke characteristics, perform texture analysis on the area to extract the texture features of the pixels in the area. Based on the extracted texture features, use the texture classification algorithm to determine whether the area is a smoke area. If it is determined to be a smoke area, obtain the motion trajectory of the area in continuous video frames through the target tracking algorithm. Based on the acquired motion trajectory, calculate the diffusion speed and direction of the smoke area. Calculate the number of pixels passing through the smoke area and the total number of image pixels, and calculate the concentration of the smoke. Fuse the detected smoke area's position, concentration, diffusion speed and other information to obtain the final smoke detection result, and trigger the corresponding alarm mechanism.

5. The method according to claim 1, characterized in that According to the temperature value, number and smoke detection results of the suspected fire points, a data fusion algorithm is used to comprehensively analyze the multi-source data to determine whether there is a forest fire. If there is a fire, the fire warning level is determined and the possible cause of the fire is analyzed, including: The temperature value of the suspected fire point is obtained by the thermal imaging camera, and the presence of abnormal temperature points is determined according to the temperature threshold. If there are abnormal temperature points, the visible light camera is triggered to detect smoke in the area, and the smoke density is analyzed by the image processing algorithm. According to the number of abnormal temperature points and the smoke density, the multi-source data is comprehensively analyzed by the data fusion algorithm to obtain the comprehensive evaluation result of the fire situation. If the comprehensive evaluation result shows that there is a forest fire, the fire warning level is determined according to the abnormal temperature value and the smoke density. According to the spatial distribution characteristics of the abnormal temperature points, the possible causes of the fire are analyzed in combination with the geographic information system and meteorological data. If the comprehensive evaluation result shows that there is no forest fire, it is determined whether the abnormal temperature point is caused by other factors, such as direct sunlight and equipment heating. If the abnormal temperature point is caused by other factors, the temperature threshold is updated, the fire judgment rules are optimized, and the accuracy of fire detection is improved. According to the fire warning level and the analysis results of the cause of the fire, the fire alarm information is automatically generated and sent to the command center and relevant departments through the communication network. The location, scope and development trend of the fire point are displayed in the GIS system of the command center to provide data support for fire fighting decisions.

6. The method according to claim 1, characterized in that If a forest fire is detected, the main burning material type and its accumulation amount are determined based on the tree species and density information around the fire location, and the fire propagation model is used to calculate the growth rate of the fire under different wind speed and direction conditions, including: According to the fire location coordinates obtained by the fire awareness system, the forest species and density information around the location are queried in the GIS system; Using remote sensing image interpretation technology, analyze satellite or drone images of the fire area to further determine parameters such as forest species and density and terrain characteristics; By consulting the forest fuel database, we can obtain combustion characteristic parameters such as fuel accumulation and moisture content of different forest species; Based on the forest fuel accumulation and moisture content and other parameters, the combustion calorific value and burning rate of the main combustible materials in the fire area are calculated; Obtain real-time meteorological data when a fire occurs, including parameters such as temperature, humidity, wind speed and direction; Use fire propagation models, such as the Rothermel model or the Farsite model, combined with combustible material parameters and meteorological parameters, to simulate and calculate the spread speed and direction of fire under different wind speed and direction conditions; If the fire spread trend calculated by the model is consistent with the actual monitoring data, the model is judged to be applicable and the model is continuously used for fire prediction; If the deviation is large, adjust the model parameters and re-perform the simulation calculation; According to the fire prediction results, the fire spread range and risk level map are drawn in the GIS system to provide reference for fire fighting decision-making; Continuously monitor the fire scene dynamics, obtain the latest fire parameters, and update the fire prediction results in real time until the fire is extinguished.

7. The method according to claim 1, characterized in that The meteorological parameters such as wind speed and wind direction in the fire area are obtained by meteorological sensors, and the meteorological parameters are input into the fire spread model to simulate the spread direction and range of the fire under the current wind field conditions, including: Meteorological parameters such as wind speed and wind direction in the fire area are obtained through meteorological sensors. Based on the obtained meteorological parameters, combined with fire environment factors and fire source characteristics and other factors, a mathematical model of fire spread is established. The convection-diffusion equation and atmospheric boundary layer theory are used to describe the spatial distribution of wind speed near the ground to obtain the wind field model. The Stefan-Boltzmann law is used to describe the thermal radiation intensity of the fire to obtain the thermal radiation model. Meteorological parameters such as wind speed and wind direction are input into the established mathematical model of fire spread. If the wind speed exceeds the preset threshold, it is judged that the wind field has an important influence on the spread of the fire, and the relevant parameters in the wind field model are adjusted. The thermal radiation intensity within the fire spread range is calculated according to the thermal radiation model to determine the fire impact range. The calculation results of the wind field model and the thermal radiation model are combined to simulate the spread direction and range of the fire under the current wind field conditions. The fire spread simulation results are compared and analyzed with the real-time monitoring data, and the model parameters are dynamically adjusted to improve the prediction accuracy.

8. The method according to claim 1, characterized in that The real-time temperature, humidity and other monitoring data of the fire monitoring point are assimilated with the prediction results of the fire spread model, and the parameters of the fire spread model are corrected in real time to continuously improve the accuracy of fire spread prediction, including: The real-time temperature and humidity monitoring data of the fire monitoring point are obtained through the sensing layer equipment, and the data is transmitted to the back-end platform through the transmission network; After receiving the monitoring data, the fire awareness system of the back-end platform pre-processes the data, including data cleaning and normalization, to obtain standardized monitoring data; The fire awareness system calls the fire spread model and calculates the fire spread prediction results based on the current model parameters and standardized monitoring data; Compare the prediction results of the fire spread model with the actual fire monitoring data and calculate the prediction error; If the prediction error exceeds the preset threshold, the data assimilation process is triggered; Otherwise, continue to use the current fire spread model parameters for prediction; In the data assimilation process, data assimilation algorithms such as Kalman filtering and ensemble Kalman filtering are used to fuse the real-time monitoring data with the prediction results of the fire spread model and to perform real-time correction of the model parameters; The data assimilation algorithm dynamically adjusts key parameters in the fire spread model, such as burning rate and wind speed and direction, based on the prediction error and the credibility of the monitoring data, so that the model prediction results are more consistent with the actual fire situation; The corrected fire spread model parameters are updated to the fire situation perception system for the next fire situation prediction; The fire spread perception system uses the corrected fire spread model and real-time monitoring data to cyclically execute prediction, comparison and data assimilation steps, continuously improving the accuracy of fire spread prediction and providing a more reliable basis for fire warning and decision-making.

9. A fire and smoke detection system based on deep reinforcement learning, characterized in that: include The acquisition unit is used to collect thermal imaging data of the forest area through a thermal imager to obtain the temperature distribution at different locations, and determine the location coordinates, number and specific temperature values ​​of the temperature abnormality points in the area with abnormally high temperature; The abnormal unit is used to determine that the temperature abnormal point is a suspected fire point if the temperature value exceeds a preset temperature threshold, and collect visible light video images of the area through a visible light camera according to the location of the suspected fire point; The image unit is used to perform smoke detection processing on the collected visible light video images, and judge whether there is smoke in the image by analyzing the image's color, texture, motion and other characteristics, and obtain the smoke's concentration, morphological characteristics and diffusion speed; The multi-source unit is used to conduct a comprehensive analysis of multi-source data based on the temperature value, number and smoke detection results of suspected fire points, using a data fusion algorithm to determine whether there is a forest fire. If there is a fire, the fire warning level is determined and the possible cause of the fire is analyzed; The monitoring unit is used to determine the main burning material type and its accumulation amount based on the tree species and density information around the fire location if a forest fire is detected, and to calculate the growth rate of the fire under different wind speed and direction conditions using a fire propagation model; The fire spread unit is used to obtain meteorological parameters such as wind speed and wind direction in the fire area through meteorological sensors, input the meteorological parameters into the fire spread model, and simulate the spread direction and range of the fire under the current wind field conditions; The prediction unit is used to assimilate the real-time temperature, humidity and other monitoring data of the fire monitoring point with the prediction results of the fire spread model, and to perform real-time correction on the parameters of the fire spread model to continuously improve the accuracy of fire spread prediction; The early warning unit is used to automatically generate fire warning information according to the fire warning level and fire spread prediction results, and send the warning information to nearby forest rangers and fire departments in real time through the wireless communication network to provide reference for fire fighting decisions.

10. An electronic device, characterized in that: The invention comprises a memory and a processor, wherein the memory is used to store information including program instructions, and the processor is used to control the execution of the program instructions, and when the program instructions are loaded and executed by the processor, the method for detecting fireworks based on deep reinforcement learning according to any one of claims 1 to 8 is implemented.

Images