Multi-sense-in-one forestry fire automatic identification and early warning system and method

Abstract

The invention discloses a multi-sense-in-one forestry fire automatic identification and early warning system and method, and belongs to the technical field of fire identification and early warning. The method comprises the following steps: S1, acquiring data to a system through terminal sensing equipment, and analyzing and processing the acquired data; s2, extracting characteristics related to smoke and fire from the preprocessed data; s3, building a convolutional neural network deep learning model, and performing smoke and fire identification on the real-time image or video data; s4, after the system automatically recognizes the fire sign, alarm information is sent out, and alarm point positioning information is confirmed through a positioning algorithm; and S5, judging the regional position of the alarm point based on the positioning information of the alarm point. Through the technologies of the unmanned aerial vehicle, the infrared sensor, intelligent identification and the like, forest fire can be monitored and early warned, a fire source is found in time, measures are taken to extinguish fire, and the harm of the fire to forest resources and the ecological environment is reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of fire identification and early warning, and specifically relates to a multi-sensory integrated forest fire automatic identification and early warning system and method. Background Art

[0002] Forest fire refers to a forest fire that is out of human control, spreads freely in the forest, and causes certain harm and losses to the forest, forest ecosystem and human beings. Forest fire is a natural disaster with strong suddenness, great destructiveness and relatively difficult handling and rescue. It is a type of fire.

[0003] Once a forest is hit by a fire, the most direct harm is the burning of trees. On the one hand, it causes a decrease in forest stock, and on the other hand, it also seriously affects forest growth. Forests are renewable resources with a long growth cycle. After a fire, it takes a long time for them to recover. Especially after a high-intensity and large-scale forest fire, it is difficult for the forest to recover its original appearance, and it is often replaced by low-value forests or shrubs.

[0004] In the existing technology, forest fire judgment is generally achieved through images or smoke sensors, among which the main method is to detect the presence of flames and smoke information. In this way, fires have already occurred. As for the prediction of possible fires, the current fire control means and prediction methods have limitations and deficiencies, resulting in inaccurate results. Summary of the invention

[0005] In response to the problems existing in the prior art mentioned in the background technology, the present invention proposes a multi-sensory integrated forest fire automatic identification and early warning system and method. Through technologies such as drones, infrared sensors and intelligent recognition, forest fires can be monitored and warned, the source of fire can be discovered in time and measures can be taken to extinguish the fire, thereby reducing the harm of fire to forest resources and the ecological environment.

[0006] Technical solution: In order to solve the above technical problems, the technical solution adopted by the present invention is as follows:

[0007] A multi-sensory integrated forest fire automatic identification and early warning method comprises the following steps:

[0008] S1: Collect data to the system through terminal sensing devices, and analyze and process the collected data;

[0009] S2: Extract smoke and fire features from the preprocessed data;

[0010] S3: Build a convolutional neural network deep learning model to perform fireworks recognition on real-time image or video data;

[0011] S4: After the system automatically identifies signs of fire, it issues an alarm message and uses the positioning algorithm to confirm the location information of the alarm point;

[0012] S5: Determine the regional position of the alarm point based on the positioning information of the alarm point.

[0013] Preferably, in S1, the specific contents of collecting data to the system through the terminal sensing device and analyzing and processing the collected data are:

[0014] The collected image or video data is processed by denoising, color space conversion, and image enhancement.

[0015] As a preferred method, in S3, a convolutional neural network deep learning model is built to perform fireworks recognition on real-time image or video data as follows:

[0016] The model includes 3 head convolution layers, 3 multi-scale feature extraction modules, 6 Transformer modules and 1 classification layer;

[0017] First, the preprocessed data is obtained and resized into 3×224×224 RGB images and input into three convolutional layers to generate spatial detail features;

[0018] Then, three multi-scale feature extraction modules are used to fully extract the fire feature information in the image, laying a feature foundation for the subsequent feature mapping of high-level spatial regions;

[0019] After the multi-scale feature extraction module, the feature maps with position embedding and class tags are input into the Transformer module to fully fuse the global information; finally, the class tokens are used to complete the fire detection through the classification layer composed of fully connected layers.

[0020] Preferably, the multi-scale feature extraction module consists of four different branches, and its specific calculation formula is as follows:

[0021] B=[H 1×1 (A), H 3×3 (A), H 5×5 (A), H MaxPool (A)],

[0022] Among them, A is the image after the convolution layer processing; B is the result of merging the four branches, H MaxPool () represents 3×3 maximum pooling, H n×n () represents n×n convolution operations;

[0023] Transformer module: The global feature extraction capability of the Transformer module mainly relies on the multi-head attention mechanism. The calculation formula of the self-attention mechanism is:

[0024]

[0025] Among them, Q, K, and V all refer to the submatrices obtained by passing the one-dimensional vector sequence input to the module through a transformation matrix, d k is a scaling factor, Softmax is a normalized exponential function; K T represents the transpose of the K matrix, and Attention represents the result of the self-attention mechanism;

[0026] MSA=Concat(head1, head2,...)W 0 ,

[0027] Among them, MSA represents the multi-head self-attention mechanism, head represents the result of the self-attention mechanism mentioned above, and Concat represents connecting different heads in series and multiplying them by the weight matrix W 0 .

[0028] As a preferred option, the Transformer module is improved:

[0029] Linear projection to convolution projection:

[0030] The token is reshaped into a 2D token map, and a convolution projection is implemented using a deep convolution with a convolution kernel of s. Finally, the projected token is flattened into 1D. The specific calculation formula is:

[0031] X q / k / v =ConvProj(X)=Flatten(DWConv(Reshape2D(X),s)),

[0032] Among them, X q / k / v is the labeled input of the q / k / v vector, X is the input label before convolution projection, DWConv() represents the depth convolution operation, s is the convolution kernel size; ConvProj() represents the convolution projection operation, Flatten() represents the flattening operation, and Reshape2D() represents the data reshaped into a two-dimensional form;

[0033] The feedforward network is improved to a convolutional residual feedforward network:

[0034] The original feed-forward network consists of two linear layers separated by a GeLU activation function; the specific expression is as follows:

[0035] FFN(X)=GeLU(XW1+b1)W2+b2,

[0036] Among them, X is the output of the multi-head attention mechanism of the previous layer, W1∈R D×4D , W2∈R 4D×D They represent the weights of the two linear layers, b1 and b2 are bias terms; GeLU is the activation function;

[0037] The improved feedforward network is a convolutional residual feedforward network, which includes an expansion layer, a depth convolution, a SE module, and a projection layer. The specific calculation formula is:

[0038] y=DWConv(x)+x,

[0039] z=SE(y),

[0040] CRFFN(x)=Conv(z),

[0041] Wherein, x represents the original data input to the residual module, DWConv() represents the deep convolution operation, y=DWConv(x)+x represents the addition of residual connection, SE() represents the SE module, which is used to adaptively recalibrate the channel feature response, and Conv() represents the convolution operation;

[0042] The final Transformer module is represented as:

[0043]

[0044] Z i =MHSA(ConvProj(LN(X i-1 )))+X i-1 i=1....L,

[0045] X i =CRFFN(LN(Z i ))+Z i i=1....L,

[0046]

[0047] Among them, X0 represents the multi-scale feature extraction module export parameter B after dimensional transformation, adding class label x class and position embedding E pos As the input of the first transformer module; X i-1 represents the input of the i-1th Transformer module, L represents the number of Transformer modules; LN() represents layer normalization, Z i represents the output of the first residual block, X i Represents the output of the i-th Transformer module; when continuing to execute the last Transformer module, y represents the output of the model; CRFFN() represents the convolutional residual feedforward network; MHSA() represents the multi-head self-attention mechanism, and ConvProj() represents the convolutional projection operation.

[0048] Preferably, in S4, after the system automatically identifies signs of fire, it issues an alarm message, and uses a positioning algorithm to confirm that the specific content of the alarm point positioning information is:

[0049] The system automatically identifies forest fires and initiates an alarm. Based on the location, status, and elevation angle data of the monitoring equipment, the system uses deep learning and convolutional neural networks to achieve automatic image-view registration and accurately locate forest fires. At the same time, combined with the map system, the location of the alarm point is presented on a three-dimensional map. When there are multiple monitoring points around the fire, cross-positioning is used to improve the accuracy of fire point positioning.

[0050] Preferably, in S5, the specific content of determining the regional position of the alarm point based on the positioning information of the alarm point is:

[0051] When receiving an alarm signal, the location information of the alarm point is obtained at the same time, and then the regional location is determined. The monitored area is divided into different geographical areas, and the location information of the alarm point is matched with the pre-divided areas. At the same time, based on the precise latitude and longitude coordinates, the geographic information system tools are used for position matching to determine the specific area where the alarm point is located.

[0052] A multi-sensing integrated forest fire automatic identification and early warning system, which realizes the above multi-sensing integrated forest fire automatic identification and early warning method, including terminal sensing equipment, smart forest fire prevention system and forest and grassland fire risk early warning system;

[0053] Terminal sensing devices include: satellite remote sensing, drones, high and low-level video surveillance, infrared cameras and sensors;

[0054] The smart forest fire prevention system includes a cockpit module, a picture module, a human defense module, a technical defense module, a physical defense module, a disposal module and a collaborative module;

[0055] The forest and grassland fire risk warning system includes user layer, application layer, platform service layer and Internet of Things perception layer.

[0056] Preferably, the cockpit module includes a geographic information system map, real-time video monitoring, and a point resource list;

[0057] The one-map module combines data of electronic maps, remote sensing images, digital elevation models, urban infrastructure, forest fire prevention facilities, administrative divisions, and responsibility grids;

[0058] The civil air defense module: collects statistics on personnel information, firefighting teams, emergency teams, and semi-professional teams, and displays them on a map;

[0059] The technical defense module collects statistics on the technical defense information of the PTZ probe, smart card port, and drone, and displays it on a map;

[0060] The physical defense module: realizes the display of point resource data on a three-dimensional map, including education facilities, water points, fire prevention construction and hazard source data;

[0061] The handling module: After the system discovers a fire, it pushes it to the grid responsible person to create a fully closed-loop process for forest fire prevention, including: fire discovery, alarm push, multi-cross collaboration, and fire handling;

[0062] The collaborative module includes department management, role management, user management, alarm management, resource management, equipment management and attendance management.

[0063] As a preferred embodiment, the user layer includes provincial forestry bureaus, municipal forestry bureaus, towns, and villages;

[0064] The application layer includes experimental analysis, information collection, fire risk factors, risk assessment, early warning forecast, early warning release, early warning response and system management;

[0065] The platform service layer includes transmission network and device access services, map engine services, application support services, and message sending services;

[0066] The IoT perception layer includes self-built monitoring stations, automatic weather stations, forestry weather stations and third-party access.

[0067] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0068] (1) The present invention forms a multi-sensory integrated ecological perception system based on terminal sensing devices such as satellite remote sensing, drones, high and low-position video surveillance, infrared cameras, sensors, etc., focusing on improving the level of intelligent monitoring and management in forest farms, wetland parks, nature reserves, scenic spots, forest parks, etc. The present invention uses big data analysis and AI technology to accurately identify and monitor forest fire prevention in real time, and gradually achieve more accurate forestry ecological governance, more scientific development trend prediction, more timely ecological security risk warning, and more effective support for forestry decision-making and deployment.

[0069] (2) The present invention improves the ability to deal with disasters in prevention and reduction. In the field of forest fire prevention, real-time supervision, drone applications cover more than 80% of key areas, and ground IoT monitoring covers more than 60% of key areas. Disaster factor identification based on the AI ​​algorithm library is achieved, and the identification efficiency reaches more than 95%. The comprehensive forestry data pool is complete, supporting the treatment and management process of disaster prevention and reduction, and reducing disaster risks.

[0070] (3) Big data visualization integrates multi-source perception monitoring data to provide decision analysis and visualization support for forestry ecological decision support.

[0071] (4) Through technologies such as drones, infrared sensors and intelligent recognition, forest fires can be monitored and warned, the source of fire can be discovered in a timely manner and measures can be taken to extinguish the fire, thereby reducing the harm of fire to forest resources and the ecological environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 It is a schematic diagram of a diagram of the smart forest fire prevention system of the present invention;

[0073] Figure 2 It is a schematic diagram of the human defense of the intelligent forest fire prevention system of the present invention;

[0074] Figure 3 It is a schematic diagram of the technical prevention of the smart forest fire prevention system of the present invention;

[0075] Figure 4 It is a schematic diagram of the treatment of the smart forest fire prevention system of the present invention;

[0076] Figure 5 It is a schematic diagram of video monitoring of a mobile terminal system of the smart forest fire prevention system of the present invention;

[0077] Figure 6 It is a schematic diagram of an alarm notification of a mobile terminal system of the smart forest fire prevention system of the present invention;

[0078] Figure 7 It is a schematic diagram of the patrol record of the mobile terminal system of the smart forest fire prevention system of the present invention;

[0079] Figure 8 It is a schematic diagram of the sign-in attendance of the mobile terminal system of the smart forest fire prevention system of the present invention;

[0080] Fig. 9 It is a schematic diagram of task management of a mobile terminal system of the smart forest fire prevention system of the present invention;

[0081] Fig.10 It is a schematic diagram of problem reporting of the mobile terminal system of the smart forest fire prevention system of the present invention;

[0082] Fig.11 It is a schematic diagram of resource collection of the mobile terminal system of the smart forest fire prevention system of the present invention;

[0083] Fig.12 It is a schematic diagram of a one-button alarm of a mobile terminal system of the smart forest fire prevention system of the present invention;

[0084] Fig.13 It is a schematic diagram of the large-screen interface of the automatic identification of fireworks in the smart forest fire prevention system of the present invention;

[0085] Fig.14It is a schematic diagram of the smoke and fire recognition background management interface of the smart forest fire prevention system of the present invention;

[0086] Fig.15 It is a schematic diagram of the smoke and fire recognition video monitoring interface of the smart forest fire prevention system of the present invention;

[0087] Fig.16 It is a schematic diagram of the alarm point location of the smart forest fire prevention system of the present invention;

[0088] Fig.17 It is a schematic diagram of the details of a fire event of the smart forest fire prevention system of the present invention;

[0089] Fig.18 It is a schematic diagram of the multi-span collaborative process of the smart forest fire prevention system of the present invention;

[0090] Fig.19 It is a schematic diagram of the video and map point marking of the smart forest fire prevention system of the present invention;

[0091] Fig. 20 It is a schematic diagram of the multi-directional observation of the fire situation of the smart forest fire prevention system of the present invention;

[0092] Fig.21 It is a schematic diagram of fire surrounding resource analysis of the smart forest fire prevention system of the present invention;

[0093] Fig. 22 It is the architecture diagram of the forest fire risk early warning system of the present invention;

[0094] Fig.23 It is a display diagram of objective factors of the risk assessment part of the present invention. DETAILED DESCRIPTION

[0095] The present invention is further illustrated below in conjunction with specific examples. The examples are implemented based on the technical solutions of the present invention. It should be understood that these examples are only used to illustrate the present invention and are not used to limit the scope of the present invention.

[0096] By aggregating data (video, water quality, soil, etc.) collected by terminal sensing devices such as satellite remote sensing, drones, high and low-position video surveillance, infrared cameras, sensors, etc. onto the platform, a multi-sensory monitoring mechanism and early warning system is established.

[0097] The multi-sensory integrated forest fire automatic identification and early warning method provided in this embodiment includes the following steps:

[0098] S1: Collect data to the system through terminal sensing devices, and analyze and process the collected data;

[0099] The collected image or video data is processed by denoising, color space conversion, and image enhancement to reduce the noise of the original image, remove redundant information, and reduce the data volume of the input image to facilitate subsequent detection and recognition processing.

[0100] In this embodiment, median filtering is used for denoising; in addition, since the background color information in the forest monitoring image is very rich and complex, and the color range of the smoke and flames of the fire is relatively small, the image is grayed.

[0101] S2: Extract smoke and fire features from the preprocessed data;

[0102] The features of smoke and fire are extracted from the preprocessed data, including color, texture, shape and motion information.

[0103] S3: Build a convolutional neural network deep learning model to perform fireworks recognition on real-time image or video data;

[0104] The model mainly consists of 3 head convolution layers, 3 multi-scale feature extraction modules (MFE Modules), 6 Transformer modules and 1 classification layer.

[0105] First, the preprocessed data is obtained and adjusted to 3×224×224 RGB images and input into three head convolutional layers to generate spatial detail features; then, the fire feature information in the image is fully extracted through three multi-scale feature extraction modules, laying a good feature foundation for the feature mapping of subsequent high-level spatial regions. After the MFE module, the feature map with position embedding and class labeling is input into the Transformer module to fully integrate the global information. Finally, the class token is used to complete the fire detection through the classification layer composed of fully connected layers.

[0106] CNN (Convolutional Neural Network) uses multiple layers of convolution and pooling operations to gradually extract features from images. The process of CNN performing feature extraction is as follows:

[0107] Convolutional Layer: In the initial convolutional layer of a CNN, a convolution kernel slides over the input image. It multiplies the kernel's weights with each small region of the input image and adds up the results to produce a feature map. This process captures low-level features of the image, such as edges and corners.

[0108] Activation function: After the convolution operation, a nonlinear activation function (ReLU, GELU, etc.) is usually applied to introduce nonlinear transformations. This helps introduce nonlinearity into the functionality of the network, enabling it to learn more complex functions.

[0109] Pooling layer: Pooling layers downsample feature maps. For example, max pooling retains the maximum value in each pooling window, thereby reducing the spatial resolution of the feature map. This helps reduce computational load while retaining essential features.

[0110] Layer stacking: CNNs are usually composed of multiple convolutional layers and pooling layers stacked alternately. Each layer introduces a new convolution kernel to capture higher-level features. Feature compression and nonlinear transformation are achieved through pooling and activation functions.

[0111] Multi-scale feature extraction module: This module consists of four different branches. The 1×1 convolution can be regarded as a simple linear transformation for data dimension reduction; compared with the 1×1 convolution, the 3×3 convolution and 5×5 convolution have a larger receptive field, so local spatial details can be obtained in a wider range. The fourth branch is a 3×3 maximum pooling with a step size of 1. The mathematical expression of multi-scale feature extraction can be expressed as follows:

[0112] B=[H 1×1 (A), H 3×3 (A), H 5×5 (A), H MaxPool (A)],

[0113] Among them, A is the image after the convolution layer processing; B is the result of merging the four branches, H MaxPool () represents 3×3 maximum pooling, H n×n () represents n×n convolution operations, that is, Σ i∈N ω n×n *x+b, where ω represents the weight matrix, b represents the bias, and x is the input feature map. The 1×1 convolution operation can mix different channels through a linear combination of each pixel without changing the spatial dimension of the feature map, thereby achieving dimensional transformation and dimensionality reduction of the feature. The 3×3 convolution operation captures the relationship between adjacent pixels and is used to extract local features in the image. Multiple superpositions of 3×3 convolutions can capture features at different levels, from edges to textures and so on. The 5×5 convolution operation is similar to the 3×3 convolution, but the larger convolution kernel is able to capture features over a larger range. Max pooling is a spatial downsampling operation that selects the maximum value within the window as the pooling output by sliding on the input feature map over the pooling window. This helps to retain the main features and reduce the size of the feature map, thereby reducing the amount of computation.

[0114] Transformer module: Transformer's powerful global feature extraction capability mainly relies on the multi-head attention mechanism. The calculation formula of the self-attention mechanism is:

[0115]

[0116] Among them, Q, K, and V all refer to the submatrices obtained by passing the one-dimensional vector sequence input to the module through a transformation matrix, d k is a scaling factor, Softmax is a normalized exponential function; K T represents the transpose of the K matrix, and Attention represents the result of the self-attention mechanism.

[0117] MSA=Concat(head1, head2,...)W 0 ,

[0118] Among them, head represents the result of the self-attention mechanism mentioned above, and Concat means connecting different heads in series and multiplying them by the weight matrix W 0 Different heads learn different styles of image targets, and finally the results of the self-attention mechanism are divided into different weights (W 0 ) combined together is the multi-head self-attention mechanism.

[0119] Improvements to the original Transformer module:

[0120] 1. Linear projection to convolution projection:

[0121] First, the token is reshaped into a 2D token map, and the convolution projection is implemented using a deep convolution with a convolution kernel of s. Finally, the projected token is flattened into 1D. The specific calculation formula is:

[0122] X q / k / v =ConvProj(X)=Flatten(DWConv(Redhape2D(X),s)),

[0123] Among them, X q / k / v Is the labeled input of the Q / K / V vector, X is the input label before convolution projection, DWConv() represents the depth convolution operation, s is the convolution kernel size. ConvProj() represents the convolution projection operation, Flatten() represents the flattening operation, and Reshape2D() represents the data reshaped into a two-dimensional form.

[0124] 2. The feedforward network is improved to a convolutional residual feedforward network:

[0125] The original feed-forward network (FFN) consists of two linear layers separated by a GeLU activation function. The first layer expands the dimension by a factor of 4, and the second layer reduces the dimension by the same percentage, as expressed below:

[0126] FFN(X)=GeLU(XW1+b1)W2+b2,

[0127] Among them, X is the output of the previous layer (multi-head attention mechanism), W1∈RD×4D , W2∈R 4D×D They represent the weights of the two linear layers respectively, b1 and b2 are bias terms; GeLU is the activation function.

[0128] The improved feedforward network is the convolutional residual feedforward network (CRFFN), which consists of an expansion layer, a depth convolution, a SE module, and a projection layer. The specific calculation formula is:

[0129] y=DWConv(x)+x,

[0130] z=SE(y),

[0131] CRFFN(x)=Conv(z),

[0132] Among them, x represents the input (the original data input to the residual module), DWConv() represents the deep convolution operation, y=DWConv(x)+x represents the addition of residual connection, SE( ) represents the SE module (Squeeze-and-Excitation module), which is used to adaptively recalibrate the channel feature response, and Conv( ) represents the convolution operation (i.e., the projection layer). Through such improvements, the characteristics of the input data can be better captured and the performance of the model can be improved.

[0133] In the above expression, deep convolution is used to extract local features, SE module is used to adjust the features in channel dimension to highlight important feature channels, and residual connection helps to alleviate the gradient vanishing problem, making the network easier to train and allowing the network to better learn subtle changes and complex patterns in the data. Finally, the output of the feedforward network is obtained by projection through convolution operation. This design combines the advantages of convolution operation and residual connection, and can better process data with spatial structure such as images while maintaining the expressive power of the model. The specific implementation method may be adjusted and optimized according to actual applications and needs.

[0134] The final Transformer module can be expressed as:

[0135]

[0136] Z i =MHSA(ConvProj(LN(X i-1 )))+X i-1 i=1....L,

[0137] X i =CRFFN(LN(Z i ))+Z i i=1....L,

[0138]

[0139] Among them, X0 represents the multi-scale feature extraction module export parameter B after dimensional transformation, adding class label x class and position embedding E pos As the input of the first transformer module; X i-1 Represents the input of the i-1th Transformer module, whose size is R (N+1)×D , L represents the number of Transformer modules; LN() represents layer normalization, Z i represents the output of the first residual block, X i Represents the output of the i-th Transformer module. When the last Transformer module is executed, y represents the output of the model. CRFFN() represents the convolutional residual feedforward network; MHSA() represents the multi-head self-attention mechanism, and ConvProj() represents the convolutional projection operation.

[0140] S4: After the system automatically identifies signs of fire, it issues an alarm message and uses the positioning algorithm to confirm the location information of the alarm point;

[0141] The system automatically identifies forest fires and initiates an alarm. Based on the location, status, and elevation angle data of the monitoring equipment, the system uses deep learning and convolutional neural network technology to achieve automatic image-viewing registration and accurately locate forest fires. Combined with the map system, the location of the alarm point is presented on a three-dimensional map. When there are multiple monitoring points around a fire, cross-positioning can be used to improve the accuracy of fire location.

[0142] S5: Determine the regional position of the alarm point based on the positioning information of the alarm point.

[0143] When receiving an alarm signal, the location information of the alarm point is also obtained; then the regional location is determined, and the monitored area is divided into different geographical areas. According to the location information of the alarm point, it is matched with the pre-divided areas; if there are precise latitude and longitude coordinates, the geographic information system (GIS) tool can be used for location matching to determine the specific area where the alarm point is located.

[0144] The multi-sensory integrated forest fire automatic identification and warning system provided in this embodiment includes terminal sensing equipment, a smart forest fire prevention system and a forest and grassland fire risk warning system.

[0145] Terminal sensing devices include: satellite remote sensing, drones, high and low-position video surveillance, infrared cameras and sensors, etc.

[0146] The smart forest fire prevention system includes a cockpit module, a picture module, a human defense module, a technical defense module, a physical defense module, a disposal module and a collaborative module.

[0147] Cockpit module: All data displayed can be counted by month or year, including:

[0148] GIS map: The relevant introduction and corresponding point information of each nature reserve, wetland park, and forest area can be displayed on the map.

[0149] Real-time video monitoring: You can select points for random patrol.

[0150] List of subsystems for forest fire prevention, pine nematode pest control, wildlife protection, deforestation for tea planting, forest fire risk level, and tourist flow monitoring at scenic spots.

[0151] List of monitoring equipment, facilities and resources involved in management: including but not limited to videos and sensors.

[0152] According to the actual situation, you can customize the display of environmental factor data (including the current and recent 7-day trends).

[0153] Event picture display (real-time update), you can customize the event type for display.

[0154] The current event list is scrolled (the area and type can be customized), showing the following contents: time, location, event, status; details can be opened for processing.

[0155] Display rankings for each region (ranking method can be customized).

[0156] like Figure 1 As shown in the "One Map" module: The "One Map" of forest resources effectively combines electronic maps, remote sensing images, digital elevation models, urban infrastructure, forest fire prevention facilities, administrative divisions, responsibility grids and other data, and displays them in two-dimensional and three-dimensional forms.

[0157] In addition to supporting commonly used 2D and 3D GIS functions, it can also support forest fire monitoring and early warning functions as follows:

[0158] (1) Map services and management: Realize geographic data management such as collection, import, editing, analysis, statistics, and output of multi-source geographic information data such as raster data, vector data, and table attribute data; build map services based on WebGIS (network geographic information system) technology; and provide map service support for the operation of forest fire prevention video monitoring systems on multiple terminals such as PCs and mobile devices.

[0159] (2) Basic map operations: 3D map scaling, rotation, zooming in, zooming out, keyboard operation, ground distance measurement, spatial distance function, height measurement function, area measurement, etc.

[0160] (3) Layer management: Implement the 3D model layer control function, realize the layered display and overlay of data models through simple operations, and control whether the data is visible on the map.

[0161] (4) Global search: Automatically locates the main forest fire prevention monitoring points, road checkpoints, resource points, personnel, etc. based on the three-dimensional terrain, and supports querying monitoring points, watchtowers and other resource information.

[0162] (5) Surrounding resource analysis: The distribution of key protected objects in the surrounding areas with a radius of 10 kilometers (circular selection by default) with the fire point as the center can be viewed on the map, including resource data (elements) such as water sources, materials, sources of danger, buildings, historical sites, gas stations, fire prevention command organizations, fire inspection stations, and fire prevention material storage depots.

[0163] (6) Look here: Select a point on the map, and the cameras within a 10-kilometer radius of the point will turn simultaneously to achieve multi-directional and accurate observation of the selected area, and display the straight-line distance between the selected point and the surrounding cameras. This is more intuitively reflected in the ability to quickly process video information monitored by the front end, and more accurately and comprehensively identify forest fires.

[0164] like Figure 2 As shown in the figure, the civil air defense module: collects statistics on civil air defense information such as personnel information, firefighting teams, emergency teams, semi-professional teams, etc., and displays them on the map, as follows:

[0165] Statistics: You can view statistics on civil defense teams, ranger patrols, and the number of personnel in the area.

[0166] Among them, the statistics of civil defense teams include: fire-fighting teams (branches), semi-professional teams (branches), emergency teams (branches) and forest rangers.

[0167] Ranger patrol statistics: including patrol mileage (km), patrol time (h), number of check-ins and patrol report events.

[0168] Number of regional personnel: including departmental areas, forest rangers, firefighting teams, emergency teams and semi-professional teams.

[0169] Personnel positioning: Supports selecting personnel lists, viewing all personnel, and viewing the current online personnel positioning status, and directly positioning to the personnel's latitude and longitude. Supports viewing personnel details and personnel trajectory records.

[0170] Firefighting team: You can select the firefighting team list, view all firefighting teams, and directly locate the longitude and latitude of the firefighting team. You can also view the details of the firefighting team.

[0171] Emergency Team: Supports selecting the emergency team list, can view all emergency teams, can directly locate the longitude and latitude of the emergency team. Supports viewing the details of the emergency team.

[0172] Semi-professional teams: Supports selecting the semi-professional team list, can view all semi-professional teams, can directly locate the longitude and latitude of the semi-professional team. Supports viewing the details of the semi-professional team.

[0173] like Figure 3 As shown in the figure, the technical defense module collects statistics on technical defense information such as PTZ probes, smart card ports, and drones, and displays them on the map, including statistics on smart devices and the proportion of technical defense.

[0174] Smart device statistics include pan-tilt probes, smart card ports, drones, forest monitoring and walkie-talkies.

[0175] PTZ Probe: Supports selection of PTZ probe list, and can view all PTZ probes. Displays the PTZ position in a 3D map, and can view the PTZ real-time video as needed; can control the device, and can view the detailed information of the device. Supports full-screen mode, and can control the video device screen in full-screen mode. Can view device alarm data.

[0176] Smart card port: Supports selection of smart card port list, and can view all smart card ports. The location of smart card ports is displayed in the 3D map, and the real-time video of smart card ports can be viewed as needed.

[0177] Drones: Supports selecting a list of drones and viewing the status of all drones. The drone locations are displayed on a 3D map, and drones can be called and live video can be viewed as needed.

[0178] Physical defense module: Display point resource data on a three-dimensional map, including education facilities, water points, fire prevention construction, hazard sources and other data.

[0179] like Figure 4 As shown in the figure, the disposal module: after the fire is discovered through the system, the grid responsible person is accurately pushed, and the four major responsibilities of "local management responsibility, industry supervision responsibility, business entity responsibility, and forest ranger management responsibility" are firmly implemented. Build a system and mechanism of "multi-cross collaboration, co-construction and sharing, reconstruction and reshaping" to achieve "early, small, and effective" forest fire prevention, and create a full closed-loop process for forest fire prevention, including: fire discovery, alarm push, multi-cross collaboration, and fire disposal.

[0180] 1. Fire discovery:

[0181] (1) Video surveillance intelligent recognition and alarm.

[0182] The dual-spectrum PTZ automatically cruises and scans 24 hours a day, 7 days a week. When smoke or heat sources are detected, an alarm is generated in a timely manner, pictures or video records of the alarm point are taken, and the corresponding alarm information is pushed to the relevant handling personnel in a timely manner according to the handling process.

[0183] The camera adopts automatic cruise mode and manual control mode. Under normal circumstances, the former is adopted. When a user logs in to control, it switches to manual control mode. When there is no control operation in manual mode within 1 minute, the camera automatically switches back to automatic cruise mode. When multiple users log in to control the same camera, the user with higher priority obtains control rights. When the user priorities are the same, the "first come, first served" principle is followed.

[0184] (2) The mini program takes a photo and sends an alarm.

[0185] Patrol personnel conduct manual patrols and can use the mini program to alert the public if they discover any fire information during the patrol process.

[0186] (3) Other alarm sources.

[0187] Through multi-cross collaboration and other means, the platform provides corresponding entrances to different functional departments based on user permissions. When these departments receive calls from the public to report the incident, they will simultaneously enter the alarm information into the system and enter the automatic handling process.

[0188] 2. Alarm push:

[0189] After an alarm "event" occurs, it is necessary to record the time of occurrence, longitude and latitude, equipment name, distance between the fire point and the equipment, marking of the fire point on the map, and the name of the department where the fire point is located. Add the photos and videos taken when the alarm is triggered to form complete basic alarm information.

[0190] After an alarm "incident" occurs, the alarm information will be immediately pushed to the grid forest rangers and early response teams via mini-programs, text messages, and voice according to the local grid authority, so that the forest rangers can grasp the alarm information at the first time and verify the relevant alarm information in a timely manner.

[0191] If the forest ranger fails to handle the alarm information in a timely manner within a short period of time, the alarm information will need to be pushed to the relevant competent departments of the township people's government, forest farm, and nature reserve. The competent department will again notify the captain of the early response team of the alarm information, and the early response team will go to the alarm location to verify the fire situation.

[0192] The accuracy of alarms needs to be continuously improved through AI intelligent learning to reduce the false alarm rate, but no "missed alarms" should occur. After an alarm "event" occurs, relevant personnel at all levels can view the "event" information and the current processing process.

[0193] 3. Multi-span collaboration:

[0194] After receiving the alarm information, the forest ranger or relevant personnel can make a preliminary judgment on the alarm information to determine whether the current alarm is an alarm in the forest area. If it is not an alarm in the forest area, the alarm information will be divided according to the relevant functional authority and pushed to different functional departments, and other functional departments will handle the alarm information. For forest area alarms, forest rangers, early disposal teams, township people's governments, forest farms, nature reserves and other competent departments will handle them in a hierarchical and multi-level manner according to the flow chart. According to the development of the fire situation, support the emergency management department to launch the emergency plan.

[0195] 4. Fire disposal:

[0196] When the forest ranger receives the relevant alarm information and determines it to be an alarm in the forest area, he will verify the alarm information (PTZ monitoring, on-site investigation). If it is confirmed to be non-fire alarm, the alarm information can be blocked and handled. If it is a fire alarm, the relevant fire information will be notified to the competent departments at all levels, the forestry department, and the emergency management bureau, and the early fire handling team will be notified to handle the fire according to the fire handling method, and generate relevant fire handling reports. If the fire cannot be controlled in the end and a fire is formed, the fire information will be pushed to the emergency management department, and the emergency management department will dispatch the corresponding forest fire fighting team to extinguish the fire according to the forest fire emergency plan.

[0197] Collaboration module: including department management, role management, user management, alarm management, resource management, equipment management and attendance management.

[0198] Department management: In the system department management, you can add departments. For example, if you add a department's parent department, you can add department details.

[0199] Role management: In the system role management, roles can be added. The added role details support displaying the authorized functions under the role and configuring the functions authorized by the role.

[0200] User management: In system user management, you can fill in relevant user information on the new interface. Fill in user information including: user account, password, department, user name, role authority (including: forest ranger, deputy leader, supervisor), push role, whether device control, control level 0-5 (the higher the control level, the priority of device control), personnel initial location, associated equipment, etc.

[0201] Alarm management: Unified management of historical alarms, including the ability to view alarm locations, image information, event tracking, process tracking, etc.

[0202] Resource management: There are two sources for resource lists: collection and addition by mini program rangers (reviewed by administrators) or independent addition by administrators.

[0203] Fill in the relevant information on the new interface. Resource information includes: resource name, resource type, resource collector, collection date, collection location, collection quantity, collector contact information, department and resource image. You can view resource details in the list and decide whether to approve the resources uploaded by the mini program or added by the administrator.

[0204] Device management: In system device management, you can add or view corresponding device information.

[0205] Attendance management: unified management of historical attendance, you can view the attendance time, department, longitude and latitude, name of the person punching the clock in, punching type, etc.

[0206] This embodiment also includes a mobile terminal system. The forest fire prevention mobile terminal is installed on the forest ranger's mobile phone, which can meet the needs of forest fire prevention patrols, explorations, and data collection. The system specifically includes the following functions:

[0207] like Figure 5 As shown in the figure, video monitoring: enter the video viewing function, you can search in the video list, or enter the real-time video screen through related videos. The console can be used to control the device screen and view the detailed information of the device. It supports full-screen mode and can control the video device screen in full-screen mode.

[0208] like Figure 6 As shown, alarm notification: enter the alarm function, you can view the alarm notification list, alarm pictures, etc., and support the operation and processing of alarm data. At the same time, you can filter and view the alarm category, and the alarm information includes: alarm type, alarm latitude and longitude, map location of the fire point, and alarm time.

[0209] like Figure 7 As shown, daily patrol: enter patrol mode and record the trajectory and upload it to the background.

[0210] like Figure 8 As shown, punch in and sign in: Enter the punch in function, you can do daily punch in and upload the punch in location, time, and punch in person to the background.

[0211] like Fig. 9 As shown, My Tasks: Enter the task management, you can view the unfinished, in progress, waiting to be viewed, and completed tasks in the task list. You can also enter the unfinished tasks (new tasks), view the task description, and then start the task.

[0212] like Fig.10 As shown, problem reporting: Enter the problem reporting function, in the problem reporting interface, take photos as required, and submit the report to the command center backend.

[0213] like Fig.11As shown, resource collection: Enter the resource collection function, the collection interface information includes: resource name, resource type, resource description, quantity, longitude and latitude (automatically locate the current location), resource picture photography. After uploading, it can be submitted to the command center background for review, and after passing the review, it can be displayed on the map point.

[0214] like Fig.12 As shown, one-key reporting: In the one-key reporting function, you can quickly report an alarm. The quick alarm content includes: alarm description, alarm snapshot, video recording, and current location positioning.

[0215] This embodiment integrates computer vision image technology, AI face recognition, big data analysis, cloud computing and other related advanced technologies, and uses ground surveillance cameras and high-position cameras to form high and low complementarity.

[0216] Fireworks are intelligently identified based on factors such as their color, shape, outline, texture, spectral characteristics, spatial geometric characteristics, and motion trajectory. Video analysis scene algorithm models are implanted, alarm areas are delineated, alarm rules are set, and dynamic information in video surveillance images is intelligently analyzed. Once suspected fireworks occur, they will be automatically identified and an alarm will be generated in the system.

[0217] Through big data analysis, historical climate data, fire occurrence records and forest conditions can be analyzed to predict the fire risk in a specific area. This analysis can help pre-deploy resources and strengthen monitoring of high-risk areas.

[0218] Fireworks automatic identification system: Conventional forest fire prevention video surveillance systems require manual supervision and cannot detect firework in time when no one is on duty. Therefore, deploy a firework automatic identification alarm system. When no one is on duty, the system automatically sends an alarm text message to the duty personnel, and the command center automatically pops up a sound alarm to achieve the purpose of rapid response and disposal.

[0219] like Fig.13 As shown in the figure, the fireworks recognition algorithm is able to digitize the video images collected by the camera without compression, and intelligently identify fireworks through multiple factors such as the color, shape, outline, texture, etc. of the smoke and fire. Once a suspected forest fire occurs, it can automatically identify it and send an alarm to the monitoring center.

[0220] The recognition accuracy of fireworks and fire is no less than 97% under the condition of satisfactory visibility, and the thermal imaging forest fire positioning error can be up to less than 40 meters within the monitoring range.

[0221] Thermal imaging fire point identification: The thermal imaging front end uses the embedded DSP (digital signal processor) temperature analysis fire point detection automatic alarm module to automatically detect environmental heat sources. The automatic alarm device follows the PTZ scanning process to detect fire points in the field of view. The main functions include:

[0222] Equipment management and settings: Manage and set up front-end equipment to ensure that the operating status of front-end equipment is known at the first time;

[0223] Software application interface module: This part includes video display window, PTZ information display, PTZ and lens control panel, fireworks position display, alarm signal issuance and automatic / manual monitoring switching, etc.;

[0224] Automatic recognition module for fireworks in forest background: integrated visible light automatic recognition of fireworks;

[0225] PTZ control module: The function of this module is to automatically or manually control the PTZ motion state (rotation speed, stop and pitch angle). In the automatic monitoring state, the PTZ and lens are automatically controlled by the program. When the fireworks recognition module recognizes the fireworks target, the PTZ automatically stops at the corresponding position and angle, and reads the current PTZ state data;

[0226] Geographic Information System Interface Module: Provides parameters used to accurately locate the geographic information system and its output interface.

[0227] Early fire verification by drones: drones can shoot real-time from high altitudes, and according to early fire alarms, they can specify the GPS (Global Positioning System) coordinates of forest areas, conduct automatic flight inspections, and transmit real-time synchronous monitoring images to ground stations, where ground personnel can verify forest fires based on the monitoring images. In the process of fire verification by drones, all-round and no-dead-angle detection can be achieved, which can not only improve verification efficiency, but also effectively improve verification results.

[0228] like Fig.18 As shown, alarm information push: following the "decentralized" design concept, the relevant departments of forest fire prevention can obtain fire information in the first time, and publish it through mobile phones in the form of text messages or WeChat public account messages. The platform will send relevant information of the fire point, such as longitude and latitude, geographical location, fire time, etc. to fire prevention and fighting personnel in the form of text messages, organize forest fire fighting in time, and minimize economic losses.

[0229] Hierarchical treatment and multi-level linkage: The first recipient of the alarm text message is the local staff responsible for monitoring the alarm area. The staff will first process the fire alarm information to avoid the district forest fire prevention command center from bearing too much pressure to process the alarm information. The forest fire prevention command center can retrieve and view the alarm information processed by everyone, so that the processing process can be traced. The above methods can achieve the effect of "hierarchical treatment and multi-level linkage".

[0230] like Fig.19 As shown, the video and the map interact in a two-way manner: according to the pan / tilt angle parameters returned by the camera, the longitude and latitude of the point facing the video image are calculated.

[0231] 1. Select any point on the map, the system can quickly search for surrounding video points, and automatically adjust the camera angle to aim the camera at the location on the map.

[0232] 2. When browsing real-time video on the map interface, click a certain place on the video screen and the system will mark and display the selected point in the video screen on the map.

[0233] UAV monitoring of fire situation changes: When the UAV flies above the fire scene, it can transmit the outline, area, spread speed and other data of the fire scene back to the ground control center in real time, monitor the occurrence and development of forest fires, and provide reliable information for ground fire fighting command. Relevant command and dispatch staff can reasonably adjust the command method based on the real-time data transmitted by the UAV, improve the efficiency of fire fighting operations, and prevent the fire from further expanding and spreading.

[0234] like Fig. 20 As shown in the “Look Here” monitoring screen, some fire situations are observed through identified equipment, but are severely obscured, and the actual situation cannot be observed. At this time, it can be observed through other nearby equipment, so as to achieve multi-directional and accurate observation of the selected area, more intuitive collection of fire information, and more comprehensive identification of forest fires.

[0235] like Fig.21 As shown, surrounding resource analysis: When a fire occurs, the surrounding resources are disordered and numerous, and it is impossible to quickly extract the desired information. Surrounding resource analysis can effectively classify and organize various resources. In different situations, decision makers can choose the resource information they need according to their needs.

[0236] Mobile terminal feedback: Through the APP mobile terminal, the fire scene can be automatically positioned and reported with one click, allowing managers to grasp the accurate on-site situation at the first time, including latitude and longitude and on-site pictures, audio and video, and reflect the fire extinguishing results in real time.

[0237] The fireworks automatic recognition system uses fireworks automatic recognition algorithm, intelligent shielding algorithm, high-precision positioning algorithm, event area determination algorithm, and single-machine multi-scene multiplexing scheduling algorithm.

[0238] like Fig.13 As shown in the figure, the automatic smoke and fire recognition algorithm: the cloud, edge, and end combined technical architecture and big data analysis algorithm use multiple factors such as the color and shape of smoke and fire to accurately and intelligently identify clouds, rain, fog, and smoke in forest areas, realizing the automation and intelligence of forest fire monitoring and early warning.

[0239] The collected image or video data is preprocessed, such as denoising, color space conversion, image enhancement, etc., and the features of smoke and fire are extracted from the preprocessed data. These features can include color, texture, shape, motion information, etc. By building a convolutional neural network (CNN) deep learning model, fireworks and fire are recognized in real-time image or video data. When signs of fire are identified, an early warning signal is issued in time to notify relevant personnel to take emergency measures. At the same time, through the technical architecture combined with cloud, edge, and end, the rapid dissemination and efficient execution of the fireworks and fire recognition algorithm can be achieved. The cloud is responsible for big data analysis and high-performance computing, the edge is responsible for real-time data collection and preliminary processing, and the terminal device is responsible for displaying early warning information and receiving instructions.

[0240] like Fig.14 , Fig.15 As shown in the figure, intelligent shielding algorithm: by combining PTZ (all-round movement, lens magnification and zoom control), smoke source positioning algorithm and image feature point matching algorithm, it can realize accurate shielding of the monitoring area, identify the weather, automatically distinguish and shield the clouds and fog in the forest area, and automatically turn off the rain alarm.

[0241] For repeated alarms from the same device, it is possible to generate only one effective alarm at the same location within a period of time, thereby optimizing the efficiency of forest fire risk warning, reducing the workload of grassroots workers, and thus improving work efficiency.

[0242] like Fig.16 As shown in the figure, the high-precision positioning algorithm: the system automatically identifies forest fires and initiates an alarm. Based on the location, status, and pitch angle of the monitoring equipment, the system uses deep learning and convolutional neural network technologies to achieve automatic image registration and accurate positioning of forest fires. At the same time, combined with the map system, the alarm point location is presented on a three-dimensional map. When there are multiple monitoring points around a fire, cross positioning can be used to improve the accuracy of fire point positioning.

[0243] like Fig.17 As shown in the figure, the event area determination algorithm: the event warning report image is identified through the intelligent algorithm, and the analysis and identification results are combined with the positioning algorithm to obtain the regional location of the fire event, so as to determine whether the alarm point belongs to a certain area, and classify the alarm information according to the determination result. For example, the event point is located in a forest area, non-forest area, or protected area.

[0244] The event area determination algorithm can monitor the occurrence area of ​​a specific event in real time, which helps to detect potential risks in a timely manner. By identifying and segmenting the event area, the algorithm can provide decision makers with information about the location, scale and evolution trend of the event, thereby assisting in decision-making and formulating response measures.

[0245] When the event area determination algorithm identifies an abnormal event, it can issue a warning signal in a timely manner to remind relevant personnel to take emergency measures and reduce potential risks.

[0246] Single-machine multi-scenario multiplexing scheduling algorithm: Provide effective PTZ scheduling services according to different monitoring needs. By judging the current scene, intelligently adjust the cruise plan and call the corresponding internal resources. If forest fire monitoring is required, it can be switched to automatic scheduling and coordination of the corresponding algorithm and cruise mode.

[0247] In this embodiment, forest fire prevention management includes real-time monitoring, alarm handling, fire situation analysis, fire alarm inspection, fire prevention duty, and work calendar.

[0248] Real-time monitoring: Comprehensive use of ground-based intelligent monitoring stations, intelligent identification, and patrol methods, integrating forestry resources, fire protection resources, natural geography, meteorology and other professional data resources, to achieve forest fire monitoring dynamics and real-time early warning of hidden dangers and ignition conditions; instant reminder of alarm information, intuitive display of alarm target information and location, time, to achieve effective alarm message reminders, the system broadcasts alarm messages in real time in the form of pop-up windows (accompanied by voice broadcast) combined with map flashing prompts.

[0249] It supports visual display of alarm information map details, and uses GIS map services to support one-click positioning of fire alarms and map viewing of details, including alarm source, fire location, monitoring time, and relevant resource attribute information such as land type and landform.

[0250] Alarm handling: By building a monitoring system, we can realize real-time monitoring of forest fires, identification and evaluation of abnormal situations, and timely report fire information to the command center. According to different monitoring types such as video monitoring, patrol and management, we can conduct real-time statistics on the total number of fire alarms, the number of real alarms, the number of false alarms, and the number of alarm processing feedbacks. We can also make thematic maps based on the analysis of alarm locations and alarm types, and form a visual display of map-number linkage, so that the response management department can grasp the forest fire situation in a timely and accurate manner and realize dynamic management of forest fire prevention.

[0251] By collecting and processing video monitoring data, manual inspections and other means, dynamic statistics of forest fire alarm information and disposal situations can be achieved. It supports multi-dimensional, diversified and multi-granular query and retrieval based on alarm location, alarm area, alarm time, disposal status and alarm nature (real alarm / false alarm). It supports alarm information editing, viewing, details display and disposal distribution, thereby realizing the control and management of alarm information.

[0252] It supports visual display of fire handling maps, and uses GIS map services to support one-click positioning of fire points and map viewing of details, including on-site verification images or videos, alarm time, verification time, fire location information, as well as handling methods, handling personnel, handling records and other information.

[0253] The disposal record information generated during the fire handling process shall be reported to the relevant departments and personnel, so that managers can intuitively and clearly understand the occurrence and reporting of forest fires.

[0254] Fire situation analysis: After discovering a fire through the automatic identification and alarm function of forest fires, the system can quickly and accurately lock the location of the fire point and obtain the small place name where the fire point is located. It supports custom settings of the surrounding range of the fire point, and can automatically calculate the fire prevention resources (fire prevention facilities, fire prevention materials, fire prevention personnel, etc.) and disaster-bearing bodies (community people, living facilities, transportation and electricity, protected objects, etc.) around the fire point, and quickly analyze and calculate the nearest / best path for each firefighting force to the fire scene, and conduct dynamic simulation of forest fire fighting, so that fire fighting commanders can organize firefighting teams and materials nearby to carry out effective firefighting.

[0255] Fire inspection: Statistical analysis of fire inspection resources around the fire point is performed, and the intelligent monitoring stations, drone nests, and inspection personnel within a certain range around the fire point are displayed in a list format combined with a visual icon symbol design. It supports sorting the fire inspection resource information from near to far, forming a visual display of the inspection resource thematic map, and supports the linkage of map and data. By online dispatching of video monitoring points around the fire point, the surrounding fire situation can be viewed in real time to ensure the timely discovery and rapid disposal of fires, improve the fire source control and firefighting capabilities, and ensure forest fire safety.

[0256] Fire prevention duty: Unified management of forest fire prevention duty departments and personnel in forest areas, and reasonable scheduling plans, intuitively displaying information such as shift leaders, duty personnel, and duty tasks on the calendar. The duty personnel need to patrol forest fires around the clock, report fire hazards in a timely manner, and do a good job of on-site disposal. Support statistical management of duty personnel according to different management zones to help managers arrange duty tasks more reasonably.

[0257] Work calendar: used to record the arrangement and execution of fire prevention inspections, fire prevention audits, fire prevention publicity, fire prevention duty and other work in the forest area. It records the work arrangement, duty personnel, duty time, fire prevention material allocation and other contents of each fire prevention work in units of weeks or months. With this function, staff can easily check and arrange daily work tasks to ensure that various fire prevention work is carried out in an orderly manner. At the same time, after the work task is completed, it can be recorded directly on the calendar, which is convenient for later data aggregation and analysis, and improves management efficiency. It urges relevant staff to complete the task on time, thereby ensuring the fire safety of the forest in the forest area.

[0258] In order to effectively carry out forest and grassland fire risk warning work, improve the accuracy of fire risk forecasting, and build a localized forest fire risk warning system, we will conduct sampling analysis and combustion experiments on combustibles in forest areas and grasslands, integrate objective factors such as combustible load, combustible moisture content, meteorology, topography, forest vegetation, and real-time monitoring data, combine subjective factors such as human activities and fire use, and use information technology such as artificial intelligence and big data to establish a forest fire risk warning model suitable for the local area, a fire risk warning AI composite model, and a fire risk warning demonstration system.

[0259] like Fig. 22 , Fig.23 As shown in the figure, the forest and grassland fire risk warning system includes user layer, application layer, platform service layer and Internet of Things perception layer.

[0260] The user layer includes provincial forestry bureaus, municipal forestry bureaus, townships, villages and other relevant units.

[0261] The application layer includes experimental analysis, information collection, fire risk factors, risk assessment, early warning forecast, early warning release, early warning response and system management;

[0262] The fire risk factor module includes information reporting, forest meteorology, and forest phenology.

[0263] Information reporting: reporting and management of combustible material dryness and wetness and human activity data.

[0264] Forest meteorology: The real-time meteorological data collected by the forest and grassland fire risk monitoring stations and the real-time and forecast data obtained from the third-party meteorological interface are connected to the forest meteorological module, and the data are displayed in the form of charts from different dimensions.

[0265] Forest phenology: The moisture content of combustibles and soil moisture content monitored by forest and grassland fire risk monitoring stations, as well as the combustible load survey data of risk surveys are connected to the system as forest phenology factors and displayed through charts.

[0266] The fire risk warning module includes risk assessment, forecast and warning, warning release and warning response.

[0267] like Fig.23 As shown, risk assessment includes forest area meteorological analysis, forest area phenological analysis, and human activity analysis.

[0268] Meteorological analysis of forest areas: Based on the real-time and forecast data of meteorological factors such as forest area temperature, humidity, wind speed, wind direction and rainfall collected by the fire risk factor module, combined with GIS spatial analysis technology, it completes the scalability analysis from "point" to "surface", and presents the dynamic change process of each factor through layers.

[0269] Forest phenology analysis: Based on the correlation between forest meteorological factors and phenological factors, the dynamic changes of fuel moisture content and soil moisture content under different forest meteorological factors are displayed in the form of layers.

[0270] Analysis of human activities: Statistical analysis is conducted on outdoor fire data from forest fire video surveillance, as well as data on spring ploughing, autumn harvest, major events, and major matters, and corresponding heat maps are generated based on the frequency of fire use and the influence of human subjective factors.

[0271] Forecasts and warnings include:

[0272] Threshold alarm: Implement custom threshold alarms, such as alarm reminders when the temperature exceeds 38°C or the moisture content of combustibles is less than 10%.

[0273] Under different meteorological and phenological conditions in the forest area, the forest fire risk level forecast, trend prediction and early warning signal effect are displayed according to the time series layer.

[0274] Warning release: Release forest fire risk level forecasts, trend predictions or warning signal information automatically or manually in the background.

[0275] Early warning response: record whether each district and county has taken corresponding measures in response to different forest fire risk levels and early warning signal issuances.

[0276] Experimental analysis module: Import ignition point measurement data, combustible load survey data in 2022 and 2023, combustible moisture content and other experimental observation data into the system. Its experimental data include: moisture content change analysis experiment, combustible type classification experiment, combustible combustion analysis experiment.

[0277] The system management module includes:

[0278] User management: management of different users of the system.

[0279] Role management: management of role function permissions.

[0280] Zone management: management of user zone levels.

[0281] Access to layer data such as combustible type zoning data and administrative division data.

[0282] Layer management: management of basic information of monitoring stations and operation status of equipment (meteorological and moisture content monitoring equipment).

[0283] Equipment management: Each monitoring station generates a QR code to record the basic information, operation and maintenance records and real-time data of the monitoring station (unauthenticated users do not have the authority to view data and can only view basic information).

[0284] Address Book: You can query the address book status at the personnel level.

[0285] Announcement release: system announcements and message management.

[0286] The platform service layer includes transmission networks (dedicated network, 4G, 5G) and device access services, map engine services, application support services, message sending services, etc.

[0287] 1. Data services:

[0288] (1) Public basic data services:

[0289] The public basic data service consists of major big data sets such as sensor equipment archives, sensor data archives, warning event data, and administrative directory data. Through the support of private cloud resources and virtualized resource pools, the basic framework support for distributed storage and data streaming processing of big data is realized. It provides an efficient and reliable basic environment for big data access and processing for multiple system services in the upper basic service layer.

[0290] Big data platform: Based on the support of private cloud resources and virtualized resource pools, big data engines are deployed for efficient storage and access services of output data of various front-end sensing devices, sensing data and related data analysis, and data mining. Among them, the main database engines include: graph database, spatiotemporal database, time series database, key-value database, document database, relational database and other database engines for different data fields and data applications, and the big data platform can effectively carry out data circulation and governance to organically combine various database engines.

[0291] Perception equipment archives: Through unified perception equipment archives, the equipment type, region, equipment protocol, management account, installation location, maintenance person, maintenance status, maintenance log and other record information related to various front-end perception equipment deployed throughout the province are managed. It can conveniently and accurately provide basic data support for the daily inspection and maintenance of front-end perception equipment.

[0292] Perception data archiving: For different types of front-end perception data, the collected front-end perception data is saved in a timely and effective manner through a suitable database engine, and efficient and accurate data retrieval and extraction support is provided for upper-level services and applications that need to be applied to perception data.

[0293] Warning event data: By recording the warning event data generated by each AI algorithm service and effectively storing it through the spatiotemporal database engine, it can provide effective spatiotemporal data set support for subsequent big data analysis and trend prediction. It can also provide quantifiable data sample support for subsequent business decision-making and optimization of disposal plans from a macro perspective.

[0294] (2) Forestry data services:

[0295] Forestry data services consist of big data sets in major forestry-specific fields, such as forestry basic data, forestry thematic data, and forestry comprehensive data. They are used to store and carry the main forestry-related data analysis results generated by big data analysis and processing of public basic data, and can effectively provide corresponding data service support for forestry-related businesses.

[0296] Forestry basic data: Through unified management of forestry basic data, unified standard basic metadata support is provided for business applications in the forestry field.

[0297] Forestry Thematic Data: Through customizable and automatically executed data statistical analysis task scheduling, it provides data support for automatic generation and publication of thematic data reports for business applications in the forestry field.

[0298] Comprehensive forestry data: Through customizable data processing pipelines, it provides automated data processing processes for business applications in the forestry field, which are used to associate data information from other large databases, thereby providing comprehensive data retrieval, data extraction and other related data service support for upper-level applications.

[0299] (3)GIS data services:

[0300] GIS data services consist of a variety of geographic information basic data sets, such as vector maps, raster maps, and image maps. Due to the particularity of the spatial relationships carried by GIS data services, corresponding dedicated spatial indexes are required for fast retrieval and access. GIS data services can be used to provide upper-level GIS services and business applications with various types of geographic spatial information data and index retrieval support.

[0301] Vector map: Vector map contains 2D or 3D vector data, which can be used for geographic information data retrieval and analysis based on spatial relationships. At the same time, the road topology data and terrain elevation data contained in the vector map can also be used for GIS navigation auxiliary functions such as fast path calculation.

[0302] Raster map: Raster map uses layered tile images pre-rendered based on vector maps, which can provide simple and easy-to-use map display support for low-performance, low-bandwidth front-end devices.

[0303] Image map: Image map uses layered tile images obtained through aerial photography or satellite remote sensing, which can provide a more intuitive display of map content or provide richer information content through layer overlay.

[0304] 2. Basic services:

[0305] (1) Perception data interconnection services:

[0306] The perception data interconnection service consists of three main system modules: data processing pipeline, data verification pipeline and data interconnection log. The perception data interconnection service mainly provides pipeline processing such as data format conversion, data validity verification and data integrity verification for the original output data collected from the perception devices. The standardized valid data after processing and verification is stored in the public basic data service to provide to other system services for further analysis and business applications.

[0307] (2) Data processing pipeline:

[0308] Through the customizable data processing pipeline, the perception data output by front-end perception devices of different types and protocols can be flexibly accessed and converted into a unified perception data type to prepare for subsequent data verification.

[0309] (3) Data verification pipeline:

[0310] Through the customizable data verification pipeline, you can define a variety of data validity and integrity verification processes to verify and make necessary repairs on the data after the data format is standardized through the data processing pipeline to ensure that the data is valid when it enters the large database for storage.

[0311] (4) Data interconnection log:

[0312] Since the platform needs to be interconnected with various data sources, a unified data interconnection log is used to log the entire process of processing and verifying the data obtained by the platform to ensure that the source of the perception data in all large databases can be traced. It can also provide effective log support for troubleshooting data processing problems during daily operation.

[0313] 3. Big data analysis services:

[0314] The big data analysis service consists of three main system modules: multi-dimensional spatial big data analysis, full-text data search, and thematic data report generation. The big data analysis service mainly provides services related to big data analysis, retrieval, and refinement, such as multi-dimensional spatial big data analysis, full-text data index generation and search, and business-based thematic data report editing and automatic generation for the perception data, warning events, and other related data sources stored in the public basic data service. The big data analysis service can provide efficient and unified big data analysis results support for other system modules.

[0315] (1) Multi-dimensional spatial big data analysis:

[0316] With the support of customizable data dimensions and spatial dimensions, it is possible to perform independent or related data statistical analysis on various data sets stored in the big database, and produce corresponding multi-dimensional spatial big data analysis results. These data analysis results can be used as full-text retrieval, thematic data reports or training sets for AI models. It can effectively simplify and improve the implementation efficiency and operation efficiency of subsequent big data applications.

[0317] (2) Full-text data indexing and search:

[0318] By generating full-text data indexes for big data analysis results and documented unstructured large data sets, efficient full-text data search capabilities can be provided for upper-level services or business applications. By vectorizing full-text data indexes, basic data corpus and knowledge base support can be provided for the subsequent application of large language models.

[0319] (3) Generation and publication of special data reports:

[0320] Through customizable thematic data reports, unified data report automatic generation and publishing basic service support can be provided for thematic data reporting needs in different business fields. With the support of multi-dimensional spatial big data analysis and basic data services, more complex and effective thematic data reports can be customized to simplify the implementation of upper-level business applications and improve operating efficiency.

[0321] 4.AI algorithm warehouse service:

[0322] The AI ​​algorithm warehouse service consists of three main system modules: AI algorithm reasoning, AI algorithm training, and AI algorithm deployment. The AI ​​algorithm warehouse service mainly provides online / offline model training task scheduling and execution based on various AI algorithms such as machine learning, deep neural networks, and visual recognition, AI model effect verification and evaluation, and online deployment and update of AI models. At the same time, the AI ​​algorithm warehouse service also provides real-time reasoning task scheduling and load balancing for AI models. The AI ​​algorithm warehouse can provide integrated AI algorithm service support for big data analysis and business applications.

[0323] (1) AI algorithm reasoning:

[0324] AI algorithm reasoning is based on the support of private cloud resources and virtualized resource pools, and containerized clustered AI computing power is dynamically allocated and scheduled, thereby making full use of the AI ​​computing power in the cluster to improve the timeliness of AI predictions and processing throughput.

[0325] (2) AI algorithm training:

[0326] AI algorithm training can be divided into two methods: online training and offline training according to different AI models. Online training AI models can be configured to allocate more resources in the specified virtualized container to complete incremental AI training while completing AI reasoning. Offline training AI models can complete the optimization training of AI models by allocating dedicated computing resources and offline data sets required for training. AI algorithm engineers can also monitor and manage the execution of AI algorithm training tasks in different containers through the console of the AI ​​algorithm warehouse, and make corresponding configurations and parameter adjustments as needed to obtain the best AI model training results.

[0327] (3) AI algorithm deployment:

[0328] AI algorithm deployment pre-runs the AI ​​model to be upgraded and deployed in a small-scale container, using the same online data as the online AI model. By comparing the differences in the effectiveness and completeness of the prediction results generated by different versions of AI models when processing the same online data, the optimization effect of the new version of the AI ​​model can be more accurately quantified and evaluated. When it is confirmed that the optimization of the new version of the AI ​​model is effective, the update of all related online AI models can be automatically completed to simplify the daily maintenance process.

[0329] 5.GIS services:

[0330] GIS services are mainly composed of two main system modules: geographic coordinate conversion and geographic information search. GIS services can provide algorithm support for mutual conversion between different geographic coordinate systems, projection conversion between 2D and 3D space coordinates, etc. At the same time, GIS services can also provide support for comprehensive information query based on keywords, spatial relationships, etc. for various geographic information data such as points, lines, and surfaces. GIS services can provide unified and effective geographic information service support for big data analysis, AI algorithm warehouses, and business applications.

[0331] (1) Geographic coordinate conversion:

[0332] When integrating geographic information data with other business data, it provides accurate algorithm support for interchange between different geographic coordinate systems, projection conversion between 2D and 3D space coordinates, etc. It can simplify data interconnection with external systems and the implementation of upper-level services and business applications.

[0333] (2) Geographic information search:

[0334] Geographic information search provides the service capability support for querying various geographic related information and its extended and associated perception data, warning events or business data in a comprehensive way based on keywords or spatial relationships. Upper-level services or business applications can create the required geospatial data index with the support of geographic information search services, providing basic support for improving search performance.

[0335] 6. Business support services:

[0336] (1) Perception data support:

[0337] Perception data support consists of major system modules such as real-time on-demand, historical playback, data report retrieval, and multi-channel data distribution. Real-time on-demand and historical playback of perception data can effectively support business applications' needs for the timeliness and effectiveness of perception data acquisition.

[0338] Data report retrieval can help business applications simplify the complexity of data analysis and report generation, reduce the development cost of business applications, and increase the flexibility of business applications in responding to different business needs.

[0339] Through multi-channel data distribution, the demand for data transmission bandwidth within the system platform can be effectively reduced, and the transmission efficiency of end users obtaining data can be improved to improve the user experience of business applications.

[0340] (2) Real-time on-demand perception data:

[0341] Real-time on-demand perception data can provide support for upper-layer business applications to push the real-time collected data of designated front-end perception devices to the application front-end for real-time playback and display.

[0342] (3) Historical playback of perception data:

[0343] The historical playback of perception data can provide support for upper-layer business applications to push the perception data collected by the specified front-end perception device within a specified time range in history to the application front-end for playback and display.

[0344] (4) Data report retrieval:

[0345] Through the support of the lower-level thematic data report service, the upper-level business application is provided with the function support of retrieving the generated data report according to the specified conditions. And the upper-level business application is provided with the support of extracting the data of the specified thematic data report and pushing it to the application front end for interactive display.

[0346] (5) Multi-channel data distribution:

[0347] Based on the characteristics of upper-layer business applications accessing perception data, there are a large number of terminals accessing the perception data of the same front-end perception device at the same time. The multi-channel data distribution service improves the response speed of the application front end through mechanisms such as multiple copies of perception data replication and distribution and local caching. At the same time, it can effectively reduce the access pressure on the lower-layer perception data extraction service and reduce the bandwidth occupation of high-cost links in the platform.

[0348] 7. Business application support:

[0349] Business application support consists of major system modules such as front-end interactive components, custom business process management, custom business catalog management, custom thematic data reports, and subscribeable warning message queues.

[0350] Through the front-end interactive components, business applications can be quickly integrated with user interactive interfaces for data on-demand and playback. Other commonly used front-end interactive components, including electronic maps and thematic data reports, are also provided.

[0351] Through customized business process management, business applications can provide customized management of business processes such as perception data distribution, warning event triggering, and notification to their end users. This simplifies the complexity of business applications in related functions and improves the efficiency of related data processing and transmission.

[0352] Through customized business catalog management, business applications can provide their end users with customized management of catalogs in other fields besides administrative catalogs, and can be applied to a variety of data processing and application scenarios such as big data analysis and thematic data reports. It simplifies the complexity of business applications in related functions and improves the efficiency and flexibility of related data processing.

[0353] By customizing thematic data reports, business applications can provide customizable thematic data reports to their end users, and complete the corresponding data analysis and report generation in big data services. While simplifying the complexity of business applications, the efficiency and flexibility of corresponding data processing are improved.

[0354] By subscribing to the warning message queue, business applications can provide timely and reliable warning message notification functions to their terminal users to ensure that warning messages of different priorities can be pushed to the corresponding business application terminals in a timely manner.

[0355] (1) Customized business processes:

[0356] By customizing business processes, some business processes that can be automatically processed in the platform can be pre-arranged. This simplifies the complexity of front-end business application implementation and improves operating efficiency. Business processes are implemented using an event-driven model. Through customizable event message queues and corresponding event processing nodes, the implementation of automated business processes can be flexibly completed.

[0357] (2) Customized business directory:

[0358] By customizing the business catalog, you can integrate business-related catalog definitions into multiple scenarios such as big data processing pipelines, verification pipelines, multi-dimensional spatial analysis, and thematic data report generation. While ensuring the standardization of big data data sets, you can achieve flexibility in business applications through different business catalog mappings.

[0359] (3) Customized special data reports:

[0360] By customizing thematic data reports, you can define the statistical dimensions, analysis indicators, time, spatial range, etc. of the corresponding data reports according to different business needs to generate data reports. At the same time, you can customize the generation cycle and release path of thematic data reports. With the support of custom thematic data reports, you can simplify the complexity of business applications to implement related functions and optimize the generation performance of data reports.

[0361] (4) You can subscribe to the warning message queue:

[0362] Through subscription, front-end business applications can obtain early warning message notifications in a timely manner and realize platform integration and docking under a unified standard message queue interface. This can effectively ensure the efficiency and reliability of early warning message push distribution.

[0363] 8. Access control support:

[0364] Access control support consists of three main system modules: identity authentication, access authorization, and access log. Through identity authentication, it can connect to the user identity authentication service of the digital forestry system to realize single sign-on identity recognition for users. Through access authorization, it can connect to the user authority authentication service of the digital forestry system to realize unified management and effective authentication of user function operations and data access. Through access log, it can connect to the user access log service of the digital forestry system to timely record and synchronize user access records to the platform to realize unified user access log supervision and audit.

[0365] (1) Identity Authentication:

[0366] The platform provides two authentication mechanisms: built-in and external. The built-in authentication mechanism uses the connected front-end business applications, external system platforms, and internal users of the platform as authentication objects, and completes authentication based on digital keys or password keys. The external authentication mechanism uses the connected front-end business applications and external users of the external system platform as authentication objects, and completes external user authentication based on the single sign-on protocol.

[0367] (2) Access authentication:

[0368] The platform provides two access authentication mechanisms: built-in and external. The built-in access authentication mechanism uses the connected front-end business applications, external system platforms, and internal users of the platform as authentication objects, and completes access authentication based on the role and permission allocation table. The external access authentication mechanism uses the connected front-end business applications and external users of the external system platform as authentication objects, and implements access authentication for external users based on the Open Authorization protocol.

[0369] (3) Access log:

[0370] The platform performs unified access log recording for all users’ identity authentication and access authorization, and provides log recording support for daily user access log supervision and auditing.

[0371] The IoT perception layer includes self-built monitoring stations, automatic weather stations, forestry weather stations, and third-party access.

[0372] The present invention is based on satellite remote sensing, drones, high and low-position video surveillance, infrared cameras, sensors and other terminal perceptions to build a multi-sensory integrated ecological perception system, focusing on improving the level of intelligent monitoring and management in forest farms, wetland parks, nature reserves, scenic spots, forest parks and other areas. The present invention uses big data analysis and AI technology to accurately identify and monitor forest fire prevention in real time, and gradually achieve more accurate forestry ecological governance, more scientific development trend prediction, more timely ecological security risk warning, and more effective support for forestry decision-making and deployment.

[0373] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A multi-sensor integrated forest fire automatic identification and early warning method, characterized in that: The following steps are involved: S1: Collect data to the system through terminal sensing devices, and analyze and process the collected data; S2: Extract smoke and fire features from the preprocessed data; S3: Build a convolutional neural network deep learning model to perform fireworks recognition on real-time image or video data; S4: After the system automatically identifies signs of fire, it issues an alarm message and uses the positioning algorithm to confirm the location information of the alarm point; S5: Determine the regional position of the alarm point based on the positioning information of the alarm point.

2. The multi-sensor integrated forest fire automatic identification and early warning method according to claim 1 is characterized by: In S1, the terminal sensing device collects data to the system, and the specific contents of analyzing and processing the collected data are as follows: The collected image or video data is processed by denoising, color space conversion, and image enhancement.

3. The multi-sensor integrated forest fire automatic identification and early warning method according to claim 1 is characterized by: In S3, the specific contents of building a convolutional neural network deep learning model to identify fireworks in real-time image or video data are as follows: The model includes 3 head convolution layers, 3 multi-scale feature extraction modules, 6 Transformer modules and 1 classification layer; First, the preprocessed data is obtained and resized into 3×224×224 RGB images and input into three convolutional layers to generate spatial detail features; Then, three multi-scale feature extraction modules are used to fully extract the fire feature information in the image, laying a feature foundation for the subsequent feature mapping of high-level spatial regions; After the multi-scale feature extraction module, the feature maps with position embedding and class tags are input into the Transformer module to fully fuse the global information; finally, the class tokens are used to complete the fire detection through the classification layer composed of fully connected layers.

4. The multi-sensor integrated forest fire automatic identification and early warning method according to claim 3 is characterized by: The multi-scale feature extraction module consists of four different branches, and its specific calculation formula is as follows: B=[H 1×1 (A),H 3×3 (A),H 5×5 (A),H MarPoot (A)], Among them, A is the image after the convolution layer processing; B is the result of merging the four branches, H MaxPool () represents 3×3 maximum pooling, H n×n () represents n×n convolution operations; Transformer module: The global feature extraction capability of the Transformer module mainly relies on the multi-head attention mechanism. The calculation formula of the self-attention mechanism is: Among them, Q, K, and V all refer to the submatrices obtained by passing the one-dimensional vector sequence input to the module through a transformation matrix, d k is a scaling factor, Softmax is a normalized exponential function; K T represents the transpose of the K matrix, and Attention represents the result of the self-attention mechanism; MSA=Concat(head1,head2,...)W 0 , Among them, MSA represents the multi-head self-attention mechanism, head represents the result of the self-attention mechanism mentioned above, and Concat represents connecting different heads in series and multiplying them by the weight matrix W 0 .

5. The multi-sensor integrated forest fire automatic identification and early warning method according to claim 4 is characterized by: Improvements to the Transformer module: Linear projection to convolution projection: The token is reshaped into a 2D token map, and a convolution projection is implemented using a deep convolution with a convolution kernel of s. Finally, the projected token is flattened into 1D. The specific calculation formula is: X q / k / v =ConuProj(X)=Flatten(DWConv(Reshape2D(X),s)), Among them, X q / k / v is the labeled input of the q / k / v vector, X is the input label before convolution projection, DWConv() represents the depth convolution operation, s is the convolution kernel size; ConvProj() represents the convolution projection operation, Flatten() represents the flattening operation, and Reshape2D() represents the data reshaped into a two-dimensional form; The feedforward network is improved to a convolutional residual feedforward network: The original feed-forward network consists of two linear layers separated by a GeLU activation function; the specific expression is as follows: FFN(X)=GeLU(XW1+b1)W2+b2, Among them, X is the output of the multi-head attention mechanism of the previous layer, W1∈R D×4D , W2∈R 4D×D They represent the weights of the two linear layers, b1 and b2 are bias terms; GeLU is the activation function; The improved feedforward network is a convolutional residual feedforward network, which includes an expansion layer, a depth convolution, a SE module, and a projection layer. The specific calculation formula is: y=DWConv(x)+x, z=SE(y), CRFFN(x)=Conv(z), Wherein, x represents the original data input to the residual module, DWConv() represents the deep convolution operation, y=DWConv(x)+x represents the addition of residual connection, SE() represents the SE module, which is used to adaptively recalibrate the channel feature response, and Conv() represents the convolution operation; The final Transformer module is represented as: AND pos ∈R (N+1)×D , Z i =MHSA(ConuProj(LN(X i-1 )))+X i-1 i=1....L, X i =CRFFN(LN(Z i ))+Z i i=1....L, Among them, X0 represents the multi-scale feature extraction module export parameter B after dimensional transformation, adding class label x class and position embedding E pos As the input of the first transformer module; X i-1 represents the input of the i-1th Transformer module, L represents the number of Transformer modules; LN() represents layer normalization, Z i represents the output of the first residual block, X i Represents the output of the i-th Transformer module; when continuing to execute the last Transformer module, y represents the output of the model; CRFFN() represents the convolutional residual feedforward network; MHSA() represents the multi-head self-attention mechanism, and ConvProj() represents the convolutional projection operation.

6. The multi-sensor integrated forest fire automatic identification and early warning method according to claim 1 is characterized by: In S4, after the system automatically identifies signs of fire, it issues an alarm message and uses the positioning algorithm to confirm that the specific content of the alarm point positioning information is: The system automatically identifies forest fires and initiates an alarm. Based on the location, status, and elevation angle data of the monitoring equipment, the system uses deep learning and convolutional neural networks to achieve automatic image-view registration and accurately locate forest fires. At the same time, combined with the map system, the location of the alarm point is presented on a three-dimensional map. When there are multiple monitoring points around the fire, cross-positioning is used to improve the accuracy of fire point positioning.

7. The multi-sensor integrated forest fire automatic identification and early warning method according to claim 1 is characterized by: In S5, the specific content of determining the regional position of the alarm point based on the positioning information of the alarm point is: When receiving an alarm signal, the location information of the alarm point is obtained at the same time, and then the regional location is determined. The monitored area is divided into different geographical areas, and the location information of the alarm point is matched with the pre-divided areas. At the same time, based on the precise latitude and longitude coordinates, the geographic information system tools are used for position matching to determine the specific area where the alarm point is located.

8. A multi-sensor integrated forest fire automatic identification and early warning system, which realizes the multi-sensor integrated forest fire automatic identification and early warning method described in claims 1-7, characterized in that: Including terminal sensing equipment, smart forest fire prevention system and forest and grassland fire risk early warning system; Terminal sensing devices include: satellite remote sensing, drones, high and low-level video surveillance, infrared cameras and sensors; The smart forest fire prevention system includes a cockpit module, a picture module, a human defense module, a technical defense module, a physical defense module, a disposal module and a collaborative module; The forest and grassland fire risk warning system includes user layer, application layer, platform service layer and Internet of Things perception layer.

9. The multi-sensor integrated forest fire automatic identification and early warning system according to claim 8 is characterized by: The cockpit module includes a geographic information system map, real-time video monitoring, and a point resource list; The one-map module combines data of electronic maps, remote sensing images, digital elevation models, urban infrastructure, forest fire prevention facilities, administrative divisions, and responsibility grids; The civil air defense module: collects statistics on personnel information, firefighting teams, emergency teams, and semi-professional teams, and displays them on a map; The technical defense module collects statistics on the technical defense information of the PTZ probe, smart card port, and drone, and displays it on a map; The physical defense module: realizes the display of point resource data on a three-dimensional map, including education facilities, water points, fire prevention construction and hazard source data; The handling module: After the system discovers a fire, it pushes it to the grid responsible person to create a fully closed-loop process for forest fire prevention, including: fire discovery, alarm push, multi-cross collaboration, and fire handling; The collaborative module includes department management, role management, user management, alarm management, resource management, equipment management and attendance management.

10. The multi-sensor integrated forest fire automatic identification and early warning system according to claim 8, characterized in that: The user layer includes provincial forestry bureaus, municipal forestry bureaus, townships, and villages; The application layer includes experimental analysis, information collection, fire risk factors, risk assessment, early warning forecast, early warning release, early warning response and system management; The platform service layer includes transmission network and device access services, map engine services, application support services, and message sending services; The IoT perception layer includes self-built monitoring stations, automatic weather stations, forestry weather stations and third-party access.

Images