Multi-Sense Integrated Automatic Forest Fire Identification and Early Warning System and Method

Abstract

The present invention discloses a multi-sense integrated automatic forest fire recognition and early warning system and method, belonging to the technical field of fire recognition and early warning. The present invention includes: S1: collecting data to the system through terminal sensing devices and analyzing and processing the collected data; S2: extracting features related to smoke and fire from the preprocessed data; S3: building a convolutional neural network deep learning model to recognize smoke and fire in real-time image or video data; S4: after the system automatically recognizes the fire signs, sending out an alarm message and using a positioning algorithm to confirm the positioning information of the alarm point; S5: judging the regional position of the alarm point based on the positioning information of the alarm point. Through technologies such as drones, infrared sensors, and intelligent recognition, the present invention can monitor and early warn forest fires, timely discover the fire source and take measures to extinguish the fire, reducing the harm of fires to forest resources and the ecological environment.

Description

Technical Field

[0001] The present invention belongs to the technical field of fire recognition and early warning, and particularly relates to a multi-sensor integrated automatic forest fire recognition and early warning system and method. Background Art

[0002] Forest fire refers to the forest fire behavior that gets out of human control, spreads and expands freely in the forest land, and brings certain harm and losses to the forest, forest ecosystem and human beings. Forest fire is a natural disaster with strong suddenness, great destructiveness and relatively difficult disposal and rescue. It is a kind of fire.

[0003] Once the forest suffers from a fire, the most intuitive harm is to burn or scorch the forest trees. On the one hand, it reduces the forest stock volume, and on the other hand, it also seriously affects the growth of the forest. Forest is a renewable resource with a long growth cycle. After suffering from a fire, its recovery takes a long time. Especially after a high-intensity large-area forest fire, it is very difficult for the forest to return to its original appearance and is often replaced by low-value forests or shrubs.

[0004] In the prior art, the judgment of forest fires is generally achieved through images or smoke sensors, etc. Among them, it mainly detects whether there is a flame and detects smoke information. In this way, most of the situations are that a fire has already occurred. For the prediction of possible fires, the current fire control means and prediction methods have limitations and deficiencies, resulting in inaccurate results. Summary of the Invention

[0005] Aiming at the problems existing in the prior art mentioned in the background art, the present invention proposes a multi-sensor integrated automatic forest fire recognition and early warning system and method. Through technologies such as unmanned aerial vehicles, infrared sensors and intelligent recognition, it can monitor and early warn forest fires, timely discover the fire source and take measures to extinguish the fire, and reduce the harm of the fire to forest resources and the ecological environment.

[0006] Technical Solution: To solve the above technical problems, the technical solution adopted by the present invention is as follows:

[0007] A multi-sensor integrated automatic forest fire recognition and early warning method, including the following steps:

[0008] S1: Collect data to the system through terminal sensing devices and analyze and process the collected data;

[0009] S2: Extract features related to smoke and fire from the preprocessed data;

[0010] S3: Build a convolutional neural network deep learning model to recognize smoke and fire in real-time image or video data;

[0011] S4: After the system automatically recognizes the fire signs, send out an alarm message and use a positioning algorithm to confirm the positioning information of the alarm point;

[0012] S5: Determine the regional location of the alarm point based on the positioning information of the alarm point.

[0013] Preferably, in S1, the specific content of collecting data to the system through the terminal sensing device and analyzing and processing the collected data is:

[0014] Perform denoising, color space conversion, and image enhancement on the collected image or video data.

[0015] Preferably, in S3, the specific content of building a convolutional neural network deep learning model to identify fireworks in real-time images or video data is:

[0016] The model includes 3 head convolutional layers, 3 multi-scale feature extraction modules, 6 Transformer modules, and 1 classification layer;

[0017] First, obtain the preprocessed data, adjust the preprocessed data into 3×224×224 RGB images, and input them into the three convolutional layers to generate spatial detail features;

[0018] Then, fully extract the fire feature information in the image through the three multi-scale feature extraction modules, laying a feature foundation for the feature mapping of subsequent high-level spatial regions;

[0019] After passing through the multi-scale feature extraction module, input the feature map with position embedding and class tokens into the Transformer module to fully fuse the global information; finally, use the class token to complete fire detection through the classification layer composed of fully connected layers.

[0020] Preferably, the multi-scale feature extraction module consists of four different branches, and its specific calculation formula is as follows:

[0021] B = [H 1×1 (A), H 3×3 (A), H 5×5 (A), H MaxPool (A)],

[0022] where A is the image processed by the convolutional layer; B is the result of merging the four branches, and H MaxPool () represents 3×3 max pooling, and H n×n () represents n×n convolutional operations;

[0023] Transformer module: The global feature extraction ability of the Transformer module mainly depends on the multi-head attention mechanism. The calculation formula of the self-attention mechanism is:

[0024]

[0025] Among them, Q, K, and V all refer to sub - matrices obtained by passing a one - dimensional vector sequence input into the module through a transformation matrix, and d k is a scaling factor, and Softmax is a normalization exponential function; K T represents the transpose of the K matrix, and Attention represents the result of the self - attention mechanism;

[0026] MSA = Concat(head1, head2,...)W 0 ,

[0027] Among them, MSA represents the multi - head self - attention mechanism, head represents the result of the self - attention mechanism mentioned above, Concat represents concatenating different heads, and multiplying by the weight matrix W 0 .

[0028] Preferably, the Transformer module is improved:

[0029] Linear projection to convolutional projection:

[0030] Reshape the tokens into a 2D token map, implement convolutional projection using depth - wise convolution with a convolution kernel of s, and finally, flatten the projected tokens into 1D. The specific calculation formula is:

[0031] X q / k / v = ConvProj(X)= Flatten(DWConv(Reshape2D(X), s)),

[0032] where X q / k / v is the token input of the q / k / v vector, X is the input token before convolutional projection, DWConv() represents the depth - wise convolution operation, s is the convolution kernel size; ConvProj() represents the convolutional projection operation, Flatten() represents the flattening operation, and Reshape2D() represents reshaping the data into a two - dimensional form;

[0033] The feed - forward network is improved to a convolutional residual feed - forward network:

[0034] The original feed - forward network consists of two linear layers, separated by the GeLU activation function; the specific expression is as follows:

[0035] FFN(X)= GeLU(XW1 + b1)W2 + b2,

[0036] where X is the output of the multi - head attention mechanism of the previous layer, W1 ∈ R D×4D , W2 ∈ R 4D×D represent the weights of the two linear layers respectively, b1 and b2 are bias terms; GeLU is the activation function;

[0037] The improved feed - forward network is a convolutional residual feed - forward network, which includes an expansion layer, depth convolution, SE module, and projection layer. The specific calculation formula is as follows:

[0038] y = DWConv(x)+x,

[0039] z = SE(y),

[0040] CRFFN(x)=Conv(z),

[0041] where x represents the original data input to the residual module, DWConv() represents the depth convolution operation, y = DWConv(x)+x represents adding a residual connection, SE() represents the SE module, which is used to adaptively recalibrate the channel feature response, and Conv() represents the convolution operation;

[0042] The final Transformer module is expressed as:

[0043]

[0044] Z i = MHSA(ConvProj(LN(X i-1 )))+X i-1 i = 1....L,

[0045] X i = CRFFN(LN(Z i ))+Z i i = 1....L,

[0046]

[0047] where X0 represents the addition of the class token x class and the position embedding E pos as the input of the first transformer module after the dimensionality transformation of the parameters exported by the multi - scale feature extraction module; X i-1 represents the input of the (i - 1) - th Transformer module, L represents the number of Transformer modules; LN() represents layer normalization, Z i represents the output of the first residual block, X i represents the output of the i - th Transformer module; when continuing to execute the last Transformer module, y represents the output of the model; CRFFN() represents the convolutional residual feed - forward network; MHSA() represents the multi - head self - attention mechanism, and ConvProj() represents the convolutional projection operation.

[0048] Preferably, in S4, after the system automatically identifies a fire sign, it sends an alarm message. The specific content of the positioning information of the alarm point confirmed by using the positioning algorithm is as follows:

[0049] The system automatically identifies forest fires and smokes, and after initiating an alarm, based on the location, device status, and pitch angle data of the monitoring equipment, and uses deep learning and convolutional neural networks to achieve automatic image registration, realizing the precise positioning of forest fires. At the same time, combined with the map system, the location of the alarm point is presented on a 3D map; when there are multiple monitoring points around the fire, cross-positioning is used to improve the positioning accuracy of the fire point.

[0050] Preferably, in S5, the specific content of judging the regional location of the alarm point based on the positioning information of the alarm point is as follows:

[0051] When receiving the alarm signal, the positioning information of the alarm point is obtained at the same time, and then the regional location is determined. The monitoring area is divided into different geographical regions, and according to the positioning information of the alarm point, it is matched with the pre-divided regions; at the same time, according to the accurate longitude and latitude coordinates, a geographic information system tool is used for position matching to determine the specific area where the alarm point is located.

[0052] A multi-sensor integrated forest fire automatic identification and early warning system that implements the above multi-sensor integrated forest fire automatic identification and early warning method, including terminal sensing devices, a smart forest fire prevention system, and a forest and grassland fire risk early warning system;

[0053] The terminal sensing devices include: satellite remote sensing, unmanned aerial vehicles, high and low position video monitoring, infrared cameras, and sensors;

[0054] The smart forest fire prevention system includes a cockpit module, a single map module, a human defense module, a technical defense module, a physical defense module, a disposal module, and a collaboration module;

[0055] The forest and grassland fire risk early warning system includes a user layer, an application layer, a platform service layer, and an Internet of Things sensing layer.

[0056] Preferably, the cockpit module: includes a geographic information system map, real-time video monitoring, and a point resource list;

[0057] The single map module: combines data of electronic maps, remote sensing image maps, digital elevation models, urban infrastructure, forest fire prevention facilities, administrative divisions, and responsibility grids;

[0058] The human defense module: statistically analyzes the human defense information of personnel information, fire fighting teams, emergency teams, and semi-professional teams, and displays it on the map;

[0059] The technical defense module: statistically analyzes the technical defense information of pan-tilt cameras, intelligent checkpoints, and unmanned aerial vehicles, and displays it on the map;

[0060] The physical defense module: It realizes the display of point resource data on a 3D map, including publicity and education facilities, water intake points, fire prevention construction, and hazard source data;

[0061] The disposal module: After the system detects a fire, it pushes it to the grid responsible person, creating a full closed-loop process for forest fire prevention, specifically including: fire detection, alarm push, multi-department collaboration, and fire disposal;

[0062] The collaboration module: It includes department management, role management, user management, alarm management, resource management, equipment management, and attendance management.

[0063] Preferably, the user layer includes the provincial forestry bureau, the municipal forestry bureau, townships, and village communities;

[0064] The application layer includes experimental analysis, information collection, fire risk factors, risk judgment, early warning forecasting, early warning release, early warning response, and system management;

[0065] The platform service layer includes a transmission network, as well as device access services, map engine services, application support services, and message sending services;

[0066] The Internet of Things perception layer includes self-built monitoring stations, automatic weather stations, forestry weather stations, and third-party access.

[0067] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0068] (1) Based on terminal sensing devices such as satellite remote sensing, unmanned aerial vehicles, high and low position video monitoring, infrared cameras, and sensors, the present invention forms a multi-sensing integrated ecological perception system, focusing on improving the intelligent monitoring and management level in areas such as forest farms, wetland parks, nature reserves, scenic spots, and forest parks. By combining big data analysis and AI technology, the present invention accurately identifies and real-time monitors forest fires, gradually realizing more precise forestry ecological governance, more scientific development trend prediction, more timely ecological security risk early warning, and more effective support for forestry decision-making and deployment.

[0069] (2) The present invention improves the disposal ability in disaster prevention and reduction. In the real-time supervision in the field of forest fire prevention, the application of unmanned aerial vehicles covers more than 80% of key areas, and the ground Internet of Things monitoring covers more than 60% of key parts. It realizes the identification of disaster factors based on the AI algorithm library, and the identification efficiency reaches more than 95%. The comprehensive forestry data pool is complete, supporting the disposal and management processes of disaster prevention and reduction, and reducing disaster risks.

[0070] (3) Big data visualization integrates multi-source perception monitoring data, providing decision analysis and visualization support for forestry ecological decision-making support.

[0071] (4) Through technologies such as drones, infrared sensors, and intelligent recognition, forest fires can be monitored and warned, the fire sources can be detected in time and measures can be taken to extinguish the fires, reducing the harm of fires to forest resources and the ecological environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 is a schematic diagram of a drawing of the intelligent forest fire prevention system of the present invention;

[0073] Figure 2 is a schematic diagram of the civil air defense of the intelligent forest fire prevention system of the present invention;

[0074] Figure 3 is a schematic diagram of the technical defense of the intelligent forest fire prevention system of the present invention;

[0075] Figure 4 is a schematic diagram of the disposal of the intelligent forest fire prevention system of the present invention;

[0076] Figure 5 is a schematic diagram of the video monitoring of the mobile terminal subsystem of the intelligent forest fire prevention system of the present invention;

[0077] Figure 6 is a schematic diagram of the alarm notification of the mobile terminal subsystem of the intelligent forest fire prevention system of the present invention;

[0078] Figure 7 is a schematic diagram of the patrol record of the mobile terminal subsystem of the intelligent forest fire prevention system of the present invention;

[0079] Figure 8 is a schematic diagram of the sign-in and attendance of the mobile terminal subsystem of the intelligent forest fire prevention system of the present invention;

[0080] Figure 9 is a schematic diagram of the task management of the mobile terminal subsystem of the intelligent forest fire prevention system of the present invention;

[0081] Figure 10 is a schematic diagram of the problem reporting of the mobile terminal subsystem of the intelligent forest fire prevention system of the present invention;

[0082] Figure 11 is a schematic diagram of the resource collection of the mobile terminal subsystem of the intelligent forest fire prevention system of the present invention;

[0083] Figure 12 is a schematic diagram of the one-key alarm of the mobile terminal subsystem of the intelligent forest fire prevention system of the present invention;

[0084] Figure 13 is a schematic diagram of the interface of the large screen for automatic recognition of smoke and fire of the intelligent forest fire prevention system of the present invention;

[0085] Figure 14It is a schematic diagram of the background management interface for smoke and fire recognition of the intelligent forest fire prevention system of the present invention;

[0086] Figure 15 It is a schematic diagram of the video monitoring interface for smoke and fire recognition of the intelligent forest fire prevention system of the present invention;

[0087] Figure 16 It is a schematic diagram of the location of the alarm points of the intelligent forest fire prevention system of the present invention;

[0088] Figure 17 It is a schematic diagram of the details of the fire incident of the intelligent forest fire prevention system of the present invention;

[0089] Figure 18 It is a schematic diagram of the multi - cross collaborative process of the intelligent forest fire prevention system of the present invention;

[0090] Figure 19 It is a schematic diagram of the video and map point marking of the intelligent forest fire prevention system of the present invention;

[0091] Figure 20 It is a schematic diagram of the multi - azimuth observation of the fire situation of the intelligent forest fire prevention system of the present invention;

[0092] Figure 21 It is a schematic diagram of the analysis of the resources around the fire situation of the intelligent forest fire prevention system of the present invention;

[0093] Figure 22 It is the architecture diagram of the forest fire risk warning system of the present invention;

[0094] Figure 23 It is a diagram showing the objective factors of the risk judgment part of the present invention. Specific implementation manners

[0095] The following further clarifies the present invention in conjunction with specific embodiments. The embodiments are implemented on the premise of the technical solution of the present invention. It should be understood that these embodiments are only used to illustrate the present invention and not to limit the scope of the present invention.

[0096] By aggregating the data (videos, water quality, soil, etc.) collected by terminal sensing devices such as satellite remote sensing, unmanned aerial vehicles, high - and low - position video monitoring, infrared cameras, and sensors to the platform, a multi - sensor integrated monitoring mechanism and warning system are established.

[0097] The multi - sensor integrated forest fire automatic recognition and warning method provided by this embodiment includes the following steps:

[0098] S1: Collect data to the system through terminal sensing devices and analyze and process the collected data;

[0099] Perform denoising, color space conversion, and image enhancement on the collected image or video data to reduce the noise of the original image, remove redundant information, and at the same time reduce the data volume of the input image, facilitating subsequent detection and recognition processing.

[0100] In this embodiment, median filtering is selected for denoising; in addition, since the background color information in forest monitoring images is very rich and complex, while the color ranges of fire smoke and flames are relatively small, the images are grayscale processed.

[0101] S2: Extract features related to smoke and fire from the preprocessed data;

[0102] Extracting features related to smoke and fire from the preprocessed data includes color, texture, shape, and motion information.

[0103] S3: Build a convolutional neural network deep learning model to perform smoke and fire recognition on real-time images or video data;

[0104] The model mainly consists of 3 head convolutional layers, 3 multi-scale feature extraction modules (MFE Module), 6 Transformer modules, and 1 classification layer.

[0105] First, obtain the preprocessed data, adjust the preprocessed data into 3×224×224 RGB images and input them into the three head convolutional layers to generate spatial detail features; then, fully extract the fire feature information in the image through the three multi-scale feature extraction modules to lay a good feature foundation for the feature mapping of subsequent high-level spatial regions. After passing through the MFE module, the feature map with position embedding and class tokens is input into the Transformer module to fully fuse the global information. Finally, use the class token to complete fire detection through the classification layer composed of fully connected layers.

[0106] CNN (Convolutional Neural Network) uses multiple layers of convolution and pooling operations to gradually extract features from images. The process of CNN performing feature extraction:

[0107] Convolutional layer: In the initial convolutional layer of the CNN, the convolutional kernel slides on the input image. It multiplies the weights of the kernel with each small region of the input image and sums the results to generate a feature map. This process captures the low-level features of the image, such as edges and corners.

[0108] Activation function: After the convolution operation, a non-linear activation function (ReLU, GELU, etc.) is usually applied to introduce non-linear transformation. This helps to introduce non-linearity into the function of the network, enabling it to learn more complex functions.

[0109] Pooling layer: The pooling layer downsamples the feature map. For example, max pooling retains the maximum value in each pooling window, thereby reducing the spatial resolution of the feature map. This helps reduce the computational load while retaining the basic features.

[0110] Layer stacking: A CNN is typically composed of multiple convolutional layers and pooling layers stacked alternately. Each layer introduces new convolutional kernels to capture higher-level features. Feature compression and non-linear transformation are achieved through pooling and activation functions.

[0111] Multi-scale feature extraction module: This module consists of four different branches. 1×1 convolution can be regarded as a simple linear transformation for data dimensionality reduction; compared with 1×1 convolution, 3×3 convolution and 5×5 convolution have a larger receptive field, so local spatial details can be obtained within a wider range. The fourth branch is 3×3 max pooling with a stride of 1. The mathematical expression for multi-scale feature extraction can be represented as follows:

[0112] B = [H 1×1 (A), H 3×3 (A), H 5×5 (A), H MaxPool (A)],

[0113] where A is the image processed by the convolutional layer; B is the result of merging the four branches, and H MaxPool () represents 3×3 max pooling, and H n×n () represents n×n convolutional operations, that is, Σ i∈N ω n×n *x + b, where ω represents the weight matrix, b represents the bias, and x is the input feature map. 1×1 convolution operation can mix different channels through the linear combination of each pixel without changing the spatial dimension of the feature map, thereby realizing the dimensionality transformation and reduction of features. 3×3 convolution operation captures the relationship between adjacent pixels and is used to extract local features in the image. Multiple stacks of 3×3 convolution can capture different levels of features, from edges to textures, etc. 5×5 convolution operation is similar to 3×3 convolution, but a larger convolution kernel can capture features within a larger range. Max pooling is a spatial downsampling operation that slides over the input feature map in the pooling window and selects the maximum value within the window as the pooling output. This helps retain the main features and reduce the size of the feature map, thereby reducing the computational amount.

[0114] Transformer module: The powerful global feature extraction ability of the Transformer mainly depends on the multi-head attention mechanism. The calculation formula of the self-attention mechanism is:

[0115]

[0116] Among them, Q, K, and V all refer to the sub - matrices obtained after a one - dimensional vector sequence input into the module passes through a transformation matrix, and d k is a scaling factor, and Softmax is a normalized exponential function; K T represents the transpose of the K matrix, and Attention represents the result of the self - attention mechanism.

[0117] MSA = Concat(head1, head2,...)W 0 ,

[0118] where head represents the result of the self - attention mechanism mentioned above, Concat means concatenating different heads, and multiplying by the weight matrix W 0 . Different heads learn different styles of image targets, and finally the results of the self - attention mechanism are combined according to different weights (W 0 ) together to form the multi - head self - attention mechanism.

[0119] Improve the original Transformer module:

[0120] 1. Linear projection to convolutional projection:

[0121] First, reshape the tokens into a 2D token map, implement convolutional projection using depth - wise convolution with a convolution kernel of s. Finally, flatten the projected tokens into 1D. The specific calculation formula is:

[0122] X q / k / v = ConvProj(X) = Flatten(DWConv(Redhape2D(X), s)),

[0123] where X q / k / v is the token input of the Q / K / V vector, X is the input token before convolutional projection, DWConv() represents the depth - wise convolution operation, and s is the convolution kernel size. ConvProj() represents the convolutional projection operation, Flatten() represents the flattening operation, and Reshape2D() represents reshaping the data into a two - dimensional form.

[0124] 2. Improve the feed - forward network to a convolutional residual feed - forward network:

[0125] The original feed - forward network (FFN) consists of two linear layers separated by the GeLU activation function. The first layer expands the dimension by 4 times, and the second layer reduces the dimension by the same percentage. The expression is as follows:

[0126] FFN(X) = GeLU(XW1 + b1)W2 + b2,

[0127] where X is the output of the previous layer (multi - head attention mechanism), W1 ∈ RD×4D , \(W_2 \in \mathbb{R}\) 4D×D represent the weights of two linear layers respectively, and \(b_1\) and \(b_2\) are bias terms; GeLU is the activation function.

[0128] The improved feed-forward network is the Convolutional Residual Feed-Forward Network (CRFFN), which consists of an expansion layer, depthwise convolution, SE module, and projection layer. The specific calculation formula is:

[0129] \(y = DWConv(x)+x\),

[0130] \(z = SE(y)\),

[0131] \(CRFFN(x)=Conv(z)\),

[0132] where \(x\) represents the input (the original data input to the residual module), \(DWConv()\) represents the depthwise convolution operation, \(y = DWConv(x)+x\) represents adding a residual connection, \(SE()\) represents the SE module (Squeeze-and-Excitation module), which is used to adaptively recalibrate the channel feature responses, and \(Conv()\) represents the convolution operation (i.e., the projection layer). Through such improvement, it can better capture the features of the input data and improve the performance of the model.

[0133] In the above expressions, depthwise convolution is used to extract local features, the SE module is used to adjust the features in the channel dimension to highlight important feature channels, and the residual connection helps to alleviate the vanishing gradient problem, making the network easier to train and enabling the network to better learn the subtle changes and complex patterns in the data. Finally, projection is performed through the convolution operation to obtain the output of the feed-forward network. Such a design combines the advantages of convolution operations and residual connections, and can better process data with spatial structures such as images while maintaining the model's expressive ability. The specific implementation method may be adjusted and optimized according to actual applications and requirements.

[0134] The final Transformer module can be expressed as:

[0135]

[0136] \(Z\) i \(= MHSA(ConvProj(LN(X\) i-1 )))+X\) i-1 \(i = 1....L\),

[0137] \(X\) i \(= CRFFN(LN(Z\) i ))+Z\) i \(i = 1....L\),

[0138]

[0139] Among them, X0 represents adding the class token x class and the position embedding E pos as the input of the first transformer module; X i-1 represents the input of the (i - 1)-th Transformer module, with its size being R (N+1)×D , L represents the number of Transformer modules; LN() represents layer normalization, and Z i represents the output of the first residual block, and X i represents the output of the i-th Transformer module. When continuing to execute the last Transformer module, y represents the output of the model. CRFFN() represents the convolutional residual feed-forward network; MHSA() represents the multi-head self-attention mechanism, and ConvProj() represents the convolutional projection operation.

[0140] S4: After the system automatically identifies the fire signs, it sends out an alarm message and uses the positioning algorithm to confirm the positioning information of the alarm point;

[0141] The system automatically identifies the forest area smoke and fire, and after initiating an alarm, according to the location, device status, and pitch angle data of the monitoring device, and uses deep learning and convolutional neural network technologies to achieve visual automatic registration, realize the precise positioning of forest fires, and at the same time, in combination with the map system, present the location of the alarm point on the 3D map; when there are multiple monitoring points around the fire, cross-positioning can be used to improve the fire point positioning accuracy.

[0142] S5: Judge the regional location of the alarm point based on the positioning information of the alarm point.

[0143] When receiving the alarm signal, obtain the positioning information of the alarm point at the same time; then determine the regional location, divide the monitoring area into different geographical regions, and match according to the positioning information of the alarm point with the pre-divided regions; if there are accurate longitude and latitude coordinates, the geographical information system (GIS) tool can be used for location matching to determine the specific region where the alarm point is located.

[0144] The multi-sensor integrated forest fire automatic identification and early warning system provided by this embodiment includes terminal sensing devices, a smart forest fire prevention system, and a forest and grassland fire risk early warning system.

[0145] The terminal sensing devices include: satellite remote sensing, unmanned aerial vehicles, high and low position video monitoring, infrared cameras, and sensors, etc.

[0146] The smart forest fire prevention system includes a cockpit module, a one-map module, a civil defense module, a technical defense module, a physical defense module, a disposal module, and a collaboration module.

[0147] Cockpit module: All data display content can be statistically analyzed monthly and annually, specifically including:

[0148] GIS map: It can display the relevant introductions and corresponding point information of nature reserves, wetland parks, and forest areas on the map.

[0149] Real-time video monitoring: Points can be selected for random round-robin.

[0150] List of subsystems for forest fire prevention, pine wire pest control, wildlife protection, deforestation for tea planting, forest fire risk level, and scenic spot passenger flow monitoring.

[0151] List of point resources of monitoring equipment and facilities involved in management: including but not limited to videos and sensors.

[0152] According to the actual situation, environmental factor data can be customized for display (which can include the current situation and the trend in the past 7 days).

[0153] Display of event pictures (real-time update), and the event types can be customized for display.

[0154] The current event list scrolls for display (the area and type can be customized), and the display content includes: time, location, event, status; the details can be opened for processing.

[0155] Display of rankings for each region (the ranking method can be customized).

[0156] As Figure 1 shown, the "One Map" module for forest resources effectively combines data such as electronic maps, remote sensing images, digital elevation models, urban infrastructure, forest fire prevention facilities, administrative divisions, and responsibility grids, and displays them in two-dimensional and three-dimensional forms.

[0157] In addition to supporting common two-dimensional and three-dimensional GIS functions, the specific functions that can support forest fire prevention monitoring and early warning are as follows:

[0158] (1) Map service and management: Realize the collection, import, editing, analysis, statistics, output, etc. of multi-source geographic information data such as raster data, vector data, and tabular attribute data, and construct a map service based on WebGIS (Web Geographic Information System) technology to provide map service support for the operation of forest fire prevention video monitoring systems on multiple terminals such as PCs and mobile devices.

[0159] (2) Basic map operations: Functions based on three-dimensional map zooming, rotation, magnification, reduction, keyboard operations, ground distance measurement, spatial distance function, height measurement function, area measurement, etc.

[0160] (3) Layer Management: Implement the layer control function for 3D models, and through simple operations, achieve the hierarchical display and superposition of data models, and control whether the data is visible on the map.

[0161] (4) Global Search: Automatically locate to the main monitoring points, road checkpoint points, resource points, personnel, etc. for forest fire prevention based on 3D terrain, and support querying resource information such as monitoring points and watchtowers.

[0162] (5) Peripheral Resource Analysis: It is possible to view the distribution of key protected objects in the surrounding area within a radius of 10 kilometers (default circular selection area) centered on the fire point on the map, including resource data (elements) such as water sources, materials, hazard sources, buildings, historical sites, gas stations, fire prevention command agencies, fire prevention checkpoints, and fire prevention material storage depots.

[0163] (6) Look Here: Select a point on the map, and the cameras within 10 kilometers of this point will turn simultaneously, achieving multi-directional and accurate observation of the selected area, and displaying the straight-line distance between the selected point and the surrounding cameras. It is more intuitive to reflect that the video information monitored at the front end can be processed quickly, and forest fires can be identified more accurately and comprehensively.

[0164] Such as Figure 2 shown, Civil Defense Module: Statistically analyze civil defense information such as personnel information, fire-fighting teams, emergency teams, semi-professional teams, etc., and display it on the map. Specifically as follows:

[0165] Statistical Data: It is possible to view the civil defense team statistics, forest ranger patrol statistics, and the number of personnel in the area.

[0166] Among them, Civil Defense Team Statistics: Include the number of fire-fighting teams (units), semi-professional teams (units), emergency teams (units), and forest rangers.

[0167] Forest Ranger Patrol Statistics: Include patrol mileage (km), patrol time (h), check-in times, and patrol report events.

[0168] Number of Personnel in the Area: Include department areas, forest rangers, fire-fighting teams, emergency teams, and semi-professional teams.

[0169] Personnel Location: Support selecting the personnel list, view all personnel, and view the location status of currently online personnel, and directly locate to the longitude and latitude of the person. Support viewing personnel details and viewing personnel trajectory records.

[0170] Fire-Fighting Team: Support selecting the fire-fighting team list, view all fire-fighting teams, and directly locate to the longitude and latitude of the fire-fighting team. Support viewing the details of the fire-fighting team.

[0171] Emergency teams: Support selecting the emergency team list, viewing all emergency teams, and directly locating to the longitude and latitude of the emergency team. Support viewing the details of the emergency team.

[0172] Semi-professional teams: Support selecting the semi-professional team list, viewing all semi-professional teams, and directly locating to the longitude and latitude of the semi-professional team. Support viewing the details of the semi-professional team.

[0173] As Figure 3 shown, the technical prevention module: Statistics of technical prevention information such as pan-tilt cameras, intelligent checkpoints, drones, etc. are carried out and displayed on the map, specifically including: statistics of intelligent devices and the proportion of technical prevention.

[0174] The statistics of intelligent devices include pan-tilt cameras, intelligent checkpoints, drones, underforest monitoring, and walkie-talkies.

[0175] Pan-tilt cameras: Support selecting the pan-tilt camera list and viewing all pan-tilt cameras. The position of the pan-tilt is displayed in the 3D map, and the real-time video of the pan-tilt can be viewed as needed; the device can be controlled, and the detailed information of the device can be viewed. Support full-screen mode, and the video device screen can be controlled in full-screen mode. The alarm data of the device can be viewed.

[0176] Intelligent checkpoints: Support selecting the intelligent checkpoint list and viewing all intelligent checkpoints. The position of the intelligent checkpoint is displayed in the 3D map, and the real-time video of the intelligent checkpoint can be viewed as needed.

[0177] Drones: Support selecting the drone list and viewing the status of all drones. The position of the drone is displayed in the 3D map, and the drone can be called and the real-time video can be viewed as needed.

[0178] The physical prevention module: Realize the display of point resource data in the 3D map, including data such as publicity and education facilities, water intake points, fire prevention construction, and hazard sources.

[0179] As Figure 4 shown, the disposal module: After the system discovers a forest fire, it accurately pushes the grid responsible person, tightly compressing the "four major responsibilities of territorial management responsibility, industry supervision responsibility, business entity responsibility, and forest ranger management responsibility". Build a system and mechanism of "multi-cross collaboration, co-construction and sharing, reconstruction and reshaping" to achieve "striking early, striking small, and striking out" for forest fires, and create a full closed-loop process for forest fire prevention, specifically including: fire discovery, alarm push, multi-cross collaboration, and fire disposal.

[0180] 1. Fire discovery:

[0181] (1) Intelligent identification and alarm of video monitoring.

[0182] The dual-spectrum pan-tilt automatically cruises and scans 24 hours a day, 7 days a week. When smoke or heat sources are detected, it generates alarms in a timely manner, takes pictures or videos of the alarm site, and pushes the corresponding alarm information to relevant handlers in accordance with the disposal process in a timely manner.

[0183] The camera adopts an automatic cruise mode and a manual control mode. Under normal circumstances, the former is used. When a user logs in to control, it switches to the manual control mode. When there is no control operation within 1 minute in the manual mode, the camera automatically switches back to the automatic cruise mode. When multiple users log in to control the same camera, the user with a higher priority obtains the control right. When the user priorities are the same, the principle of "first come, first served" is followed.

[0184] (2) Alarm by taking pictures in the mini-program.

[0185] During the manual patrol, if the patrol personnel discover fire information, they can report an alarm through the mini-program.

[0186] (3) Other alarm sources.

[0187] Through means such as multi-cross collaboration, the platform provides corresponding entrances to different functional departments according to user permissions. When these departments receive calls from the public for alarms, they synchronize the alarm information into the system and enter the automatic disposal process.

[0188] 2. Alarm push:

[0189] After an alarm "event" occurs, it is necessary to record the occurrence time, longitude and latitude, device name, distance between the fire point and the device, the marking of the fire point on the map, and the name of the department where the fire point is located. Together with the pictures and videos taken during the alarm, complete basic alarm information is formed.

[0190] After an alarm "event" occurs, according to the jurisdiction grid permissions, the alarm information is immediately pushed to grid forest rangers and the early disposal team in the form of mini-programs, text messages, and voices, so that the forest rangers can master the alarm information in the first time and verify the relevant alarm information in a timely manner.

[0191] If within a short period of time, the forest rangers fail to dispose of the alarm information in a timely manner, the alarm information needs to be pushed to the relevant competent departments of the township people's government, forest farm, and nature reserve. The competent department notifies the team leader of the early disposal team of the alarm information again, and the early disposal team goes to the alarm location to verify the fire situation.

[0192] The alarm accuracy rate needs to be continuously improved through AI intelligent learning to reduce the false alarm rate, but "missed alarms" shall not occur. After an alarm "event" occurs, relevant personnel at all levels can view the "event" information and the current handling process.

[0193] 3. Multi-cross collaboration:

[0194] After receiving the alarm information, forest rangers or relevant personnel can make a preliminary judgment on the alarm information to determine whether the current alarm is a forest area alarm. If it is not a forest area alarm, the alarm information will be pushed to different functional departments according to the relevant functional authority division, and other functional departments will handle the alarm information. For forest area alarms, the forest rangers, early disposal teams, township people's governments, forest farms, nature reserves and other competent departments will handle them at different levels and in a multi-level linkage manner according to the flow chart. According to the development of the fire situation, the emergency management department is supported to activate the emergency plan.

[0195] 4. Fire situation disposal:

[0196] After receiving the relevant alarm information and determining it as a forest area alarm, the forest ranger will verify the alarm information (cloud platform monitoring, on-site investigation). If it is confirmed as a non-fire alarm, the alarm information can be blocked and disposed of. If it is a fire alarm, the forest ranger will notify the relevant fire situation information to the competent departments at all levels, the forestry competent department, and the emergency management bureau, and notify the early fire disposal team to dispose of the fire according to the fire disposal method, and generate relevant fire disposal reports. If the fire cannot be controlled finally and a fire occurs, the fire information will be pushed to the emergency management department, and the emergency management department will dispatch the corresponding forest fire fighting teams to fight the fire according to the forest fire emergency plan.

[0197] Collaboration module: including department management, role management, user management, alarm management, resource management, equipment management and attendance management.

[0198] Department management: In the system department management, departments can be added. For example, if adding the superior department of a department, the detailed information of the department can be newly added.

[0199] Role management: In the system role management, roles can be added. The detailed content of the added role supports displaying the authorized functions under the role and configuring the functions authorized by the role.

[0200] User management: In the system user management, relevant user information can be filled in the new interface. The user information to be filled in includes: user account, password, department, user name, role permissions (including: forest ranger, deputy leader in charge, leader in charge), push role, whether equipment control is available, control level 0-5 (the higher the control level, the higher the priority to enjoy the equipment control right), initial position of the personnel, associated equipment, etc.

[0201] Alarm management: Uniformly manage historical alarms, and the alarm location, picture information, event tracking, process tracking, etc. can be viewed.

[0202] Resource management: There are two channels for the source of the resource list: newly added by the applet patrolman (reviewed by the administrator) or newly added by the administrator independently.

[0203] Fill in the relevant information on the new interface. The resource information includes: resource name, resource type, resource collector, collection date, collection location, collection quantity, collector contact information, department, and resource image. You can view the resource details in the list and determine whether to approve the resources uploaded by the applet or newly added by the administrator.

[0204] Device Management: In the system device management, you can add or view the corresponding device information.

[0205] Attendance Management: Uniformly manage historical attendance. You can view the attendance time, department, longitude and latitude, name of the person punching the card, punching type, etc.

[0206] This embodiment also includes a mobile terminal subsystem. The forest fire prevention mobile terminal is installed on the mobile phones of forest rangers and is a comprehensive system that can meet the needs of forest fire prevention patrol exploration and data collection. It specifically includes the following functions:

[0207] As Figure 5 shown, Video Surveillance: Enter the video viewing function. You can search in the video list or enter the live video screen through relevant videos. The console can control the device's screen and view the detailed information of the device. Full-screen mode is supported, and the video device screen can be controlled in full-screen mode.

[0208] As Figure 6 shown, Alarm Notification: Enter the alarm function. You can view the alarm notification list, alarm pictures, etc., and support operation and processing of alarm data. At the same time, you can filter and view alarm categories. The alarm information includes: alarm type, alarm longitude and latitude, map location of the fire point, and alarm time.

[0209] As Figure 7 shown, Daily Patrol: Enter the patrol mode and record the trajectory and upload it to the background.

[0210] As Figure 8 shown, Punch-in Sign-in: Enter the punch-in function. You can perform daily punch-in and upload the punch-in location, time, and person punching the card to the background.

[0211] As Figure 9 shown, My Tasks: Enter the task management. You can view the tasks that have not been carried out, are in progress, are pending review, and have been completed in the task list. You can also enter the tasks that have not been carried out (new tasks), view the task description, and start the task.

[0212] As Figure 10 shown, Problem Reporting: Enter the problem reporting function. On the problem reporting interface, after describing and taking pictures as needed, submit the report to the command center background.

[0213] As Figure 11As shown in the figure, resource collection: Enter the resource collection function. The information on the collection interface includes: resource name, resource type, resource description, quantity, longitude and latitude (automatically locate the current position), and take pictures of resource pictures. After uploading, it can be submitted to the command center background for review, and after passing the review, it can be displayed on the map points.

[0214] As Figure 12 shown in the figure, one - key reporting: In the one - key reporting function, you can quickly report an alarm. The content of the quick alarm includes: alarm description, alarm capture, video recording, and current position positioning.

[0215] This embodiment integrates related advanced technologies such as computer vision image technology, AI face recognition, big data analysis, and cloud computing, and uses ground monitoring cameras and high - position cameras to form high - low complementarity.

[0216] Based on various factors such as the color, shape, contour, texture, spectral characteristics, spatial geometric characteristics, and movement trajectory of the smoke and fire, it intelligently discriminates the smoke and fire, implants the video analysis scene algorithm model, delimits the alarm area and sets the alarm rules, and intelligently analyzes the dynamic information in the video surveillance screen. Once there is a suspected smoke and fire, it will be automatically recognized and an alarm will be generated in the system.

[0217] Through big data analysis, by analyzing historical climate data, fire occurrence records, and forest conditions, the fire risk in a specific area can be predicted. This kind of analysis can help pre - deploy resources and strengthen the monitoring of high - risk areas.

[0218] Smoke and fire automatic recognition system: Conventional forest fire prevention video surveillance systems require manual monitoring and cannot detect smoke and fire in a timely manner when there is no one on duty. Therefore, deploy a smoke and fire automatic recognition and alarm system. When there is no one on duty, the system automatically sends an alarm text message to the duty personnel, and the command center will automatically pop up a sound alarm to achieve the purpose of rapid response and disposal.

[0219] As Figure 13 shown in the figure, the smoke and fire recognition algorithm can digitize the video images collected by the camera without compression, and then intelligently discriminate the smoke and fire based on various factors such as the color, shape, contour, and texture of the smoke and fire. Once there is a suspected forest smoke and fire, it can be automatically recognized and an alarm will be sent to the monitoring center.

[0220] Under the condition of meeting the visibility, the recognition accuracy of the smoke and fire recognition is not less than 97%, and the thermal imaging forest fire positioning error can be as low as 40 meters at most within the monitoring range.

[0221] Thermal imaging fire point recognition: The thermal imaging front - end uses an embedded DSP (Digital Signal Processor) temperature analysis fire point detection automatic alarm module to automatically detect environmental heat sources. The automatic alarm device detects fire points in the field of view during the pan - tilt scanning process. The main functions include:

[0222] Device Management and Settings: Manage and set the front-end devices to ensure that the operation status of the front-end devices can be understood in a timely manner;

[0223] Software Application Interface Module: This part includes a video display window, pan-tilt information display, pan-tilt and lens control panels, fire position display, alarm signal emission, and automatic / manual monitoring switching, etc.;

[0224] Automatic Fire and Smoke Recognition Module in the Forest Background: Integrate visible light automatic fire and smoke recognition;

[0225] Pan-Tilt Control Module: The function of this module is to automatically or manually control the motion state of the pan-tilt (rotation speed, stop, and pitch angle). In the automatic monitoring state, the pan-tilt and lens are automatically controlled by the program. When the fire and smoke recognition module recognizes a fire and smoke target, the pan-tilt is automatically stopped at the corresponding position and angle, and the current pan-tilt status data is read;

[0226] Geographic Information System Interface Module: Provide parameters for accurately positioning the geographic information system and its output interface.

[0227] Early Fire Verification by UAV: It can perform real-time aerial photography operations, specify the GPS (Global Positioning System) coordinate points of the forest area according to the early fire alarm, conduct automatic flight inspections and verifications, and synchronously transmit the monitoring video to the ground station in real time. Ground personnel can verify the forest fire alarm situation based on the monitoring video. During the process of UAV fire verification, all-round and non-blind-spot detection can be achieved, which can not only improve the verification efficiency but also effectively improve the verification effect.

[0228] As Figure 18 shown, Alarm Information Push: Following the "decentralized" design concept, relevant departments for forest fire prevention in the forest area can obtain fire information in a timely manner and publish it through mobile phones in the form of text messages or WeChat official account messages. The platform will send relevant information about the fire point, such as longitude and latitude, geographical orientation, fire time, etc., to the fire prevention and suppression staff in the form of text messages, and organize the suppression of forest fires in a timely manner to minimize economic losses to the greatest extent.

[0229] Hierarchical Disposal and Multi-Level Linkage: The first recipient of the alarm text message is the local staff responsible for monitoring the alarm area. The staff will first process the fire alarm information to avoid placing too much pressure on the forest fire prevention command center for handling alarm information. The forest fire prevention command center can retrieve and view all the alarm information processed by everyone, ensuring that the processing process can be traced. Through the above methods, the effect of "hierarchical disposal and multi-level linkage" can be achieved.

[0230] As Figure 19 shown, Two-Way Interaction between Video and Map: Calculate the longitude and latitude of the point directly opposite to the video image according to the pan-tilt angle parameters returned by the camera.

[0231] 1. Select any point on the map, and the system can quickly search for surrounding video points and automatically adjust the camera angle to align the camera with that position on the map.

[0232] 2. When viewing real-time video on the map interface, click on a certain place on the video screen, and the system can mark and display the selected point on the video screen on the map.

[0233] UAV monitoring of the fire situation changes: When the UAV is flying above the fire site, data such as the outline, area, and spread speed of the fire site can be transmitted back to the ground control center in real time, monitoring the occurrence of forest fires and the development dynamics of the fire site, providing reliable information for ground fire fighting command. Relevant command and dispatch staff can reasonably adjust the command method based on the real-time data transmitted by the UAV to improve the efficiency of fire fighting operations and prevent the further spread of the fire.

[0234] Such as Figure 20 shown, "Look here" monitoring screen view: Some fire situations are severely blocked by the identified equipment and the actual situation cannot be observed. At this time, it can be observed through other nearby equipment, achieving multi-directional and accurate observation of the selected area, more intuitively collecting fire information, and more comprehensively identifying forest fires.

[0235] Such as Figure 21 shown, surrounding resource analysis: After a fire occurs, the surrounding resources are disorderly and numerous, and it is impossible to quickly extract the desired information. Through surrounding resource analysis, various resources are effectively classified and sorted. In different situations, decision-makers can select the resource information they need according to their needs.

[0236] Mobile terminal feedback: Through automatic positioning of the APP mobile terminal, take a photo of the fire situation on the scene and report it with one click, enabling managers to master the accurate on-site situation in the first time, including longitude and latitude, on-site pictures, voice and video, and reflecting the fire extinguishing results in real time.

[0237] In the smoke and fire automatic recognition system, smoke and fire automatic recognition algorithms, intelligent shielding algorithms, high-precision positioning algorithms, event area determination algorithms, and single-machine multi-scenario reuse scheduling algorithms are adopted.

[0238] Such as Figure 13 shown, smoke and fire automatic recognition algorithm: The cloud-edge-end combined technical architecture and big data analysis algorithm accurately and intelligently identify clouds, rain, fog, and smoke in the forest area through various factors such as the color and shape of smoke and fire, realizing the automation and intelligence of forest fire prevention monitoring and early warning.

[0239] Preprocess the collected image or video data, such as denoising, color space conversion, image enhancement, etc., and extract features related to smoke and fire from the preprocessed data. These features can include color, texture, shape, motion information, etc. By building a convolutional neural network (CNN) deep learning model, perform smoke and fire recognition on real-time image or video data. When a fire sign is recognized, send out a warning signal in a timely manner to notify relevant personnel to take emergency measures. At the same time, through a technical architecture that combines cloud, edge, and terminal, achieve the rapid dissemination and efficient execution of the smoke and fire recognition algorithm. The cloud is responsible for big data analysis and high-performance computing, the edge side is responsible for real-time data collection and preliminary processing, and the terminal device is responsible for displaying warning information and receiving instructions.

[0240] Such as Figure 14 , Figure 15 As shown in the figure, the intelligent shielding algorithm: By combining algorithms such as pan-tilt-zoom (PTZ) of the pan-tilt head (omnidirectional movement, lens zoom, and zoom control), smoke source positioning algorithm, and image feature point matching, etc., achieve precise shielding of the monitoring area, and identify the weather, automatically distinguish and shield clouds and fog in the forest area, and automatically turn off the alarm in rainy days, etc.

[0241] For repeated alarms of the same device, it can ensure that only one valid alarm is generated at the same location within a certain period of time, optimize the early warning efficiency of forest fires, reduce the work burden of grass-roots workers, and thus improve work efficiency.

[0242] Such as Figure 16 As shown in the figure, the high-precision positioning algorithm: After the system automatically identifies the forest area smoke and fire and initiates an alarm, based on data such as the location of the monitoring device, device status, pitch angle, etc., and uses technologies such as deep learning and convolutional neural network to achieve automatic registration of graphics and vision, realize the precise positioning of forest fires, and at the same time combine with the map system to present the location of the alarm point on a 3D map. When there are multiple monitoring points around the fire, cross-positioning can be used to improve the positioning accuracy of the fire point.

[0243] Such as Figure 17 As shown in the figure, the event area determination algorithm: Identify the event warning reported image through an intelligent algorithm, and combine the analysis and recognition results with the positioning algorithm to obtain the regional location where the fire event is located, so as to determine which area the alarm point belongs to, and classify the alarm information according to the determination result. Such as the event point is located in the forest area, non-forest area, protected area, etc.

[0244] The event area determination algorithm can monitor the occurrence area of specific events in real time, which helps to detect potential risks in a timely manner. Through the recognition and segmentation of the event area, the algorithm can provide decision-makers with information about the location, scale, and evolution trend of the event, so as to assist decision-making and formulate response measures.

[0245] When the event area determination algorithm identifies an abnormal event, it can issue a warning signal in a timely manner to remind relevant personnel to take emergency measures and reduce potential risks.

[0246] Single-machine multi-scenario reuse scheduling algorithm: Provide effective pan-tilt scheduling services according to different monitoring requirements. By judging the current scenario, intelligently adjust the cruise plan and call the corresponding internal resources. For example, if forest fire prevention monitoring is required, the corresponding algorithm and cruise mode can be automatically scheduled and coordinated.

[0247] In this embodiment, forest fire prevention management includes real-time monitoring, alarm handling, fire situation analysis, fire alarm inspection, fire prevention duty, and work calendar.

[0248] Real-time monitoring: Comprehensively utilize ground intelligent monitoring stations, intelligent recognition, and patrol means to converge professional data resources such as forestry resources, fire protection resources, natural geography, and meteorology, realize dynamic monitoring of forest fires, and real-time early warning of potential fire ignition situations; immediately remind alarm information, intuitively display alarm target information, location, and time, and realize effective alarm message reminders. The system broadcasts alarm messages in real time in the form of pop-up windows (accompanied by voice announcements) combined with map flashing prompts.

[0249] Support the visual detail display of alarm information on the map. Utilize GIS map services to support one-key positioning of fire prevention alarms and view details on the map, including alarm sources, fire locations, monitoring times, and relevant resource attribute information such as land types and landforms.

[0250] Alarm handling: Through building a monitoring system, realize real-time monitoring of forest fire situations, identification and assessment of abnormal situations, and timely report fire situation information to the command center. According to different monitoring types such as video monitoring and patrol management, conduct real-time statistics on the total number of fire alarms, the number of actual alarms, the number of false alarms, and the number of feedbacks on alarm handling. Based on the alarm location and alarm type analysis, produce thematic maps to form a visual display of map-data linkage, facilitating response management departments to timely and accurately grasp the forest fire situation in the forest area and realize dynamic management of forest area fire prevention.

[0251] Through means such as collecting and processing video monitoring data and manual inspections, realize dynamic statistics on forest fire prevention alarm information and handling situations, support multi-dimensional, diversified, and multi-granularity query and retrieval based on alarm location, alarm area, alarm time, handling status, and alarm nature (actual alarm / false alarm), support functions such as alarm information editing, viewing, detail display, and handling issuance, and realize the control and management of alarm information.

[0252] Support the visual detail display of fire situation handling on the map. Utilize GIS map services to support one-key positioning of fire points and view details on the map, including on-site verification pictures or videos, alarm time, verification time, fire location information, and information such as handling methods, handling personnel, and handling records.

[0253] Report the disposal record information formed during the fire incident handling process to relevant departments and personnel, enabling managers to intuitively and clearly understand the occurrence and reporting of forest fires in the forest area.

[0254] Fire situation analysis: After a fire is detected through the automatic forest fire recognition and alarm function, the system can quickly and accurately lock the location of the fire point and obtain the small place name where the fire point is located. It supports customizing the range around the fire point and can automatically calculate the fire prevention resources (fire prevention facilities, fire prevention materials, fire prevention personnel, etc.), disaster-bearing bodies (community population, living facilities, transportation and electricity, protected objects, etc.) around the fire point, and quickly analyze and calculate the nearest / best path for each fire fighting force to the fire scene, and conduct dynamic simulation of forest fire fighting, enabling fire fighting commanders to organize fire fighting teams and materials nearby for effective fire fighting.

[0255] Fire alarm inspection: Statistically analyze the fire alarm inspection resources around the fire point, and display the intelligent monitoring and monitoring stations, UAV nests, and inspection personnel within a certain range around the fire point in the form of a list combined with visualized icon symbols. It supports sorting the fire alarm inspection resource information in the order from near to far, forming a visualized display of the inspection resource thematic map, and supports the linkage between the map and data. By online dispatching the video monitoring points around the fire point, the surrounding fire situation can be viewed in real time, ensuring the timely discovery and rapid disposal of fires, improving the fire source control and fire fighting capabilities, and ensuring forest fire prevention safety.

[0256] Fire prevention duty: Unifiedly manage the forest fire prevention duty departments and personnel in the forest area, and reasonably formulate a shift plan. Intuitively display information such as the shift leader, duty personnel, and duty tasks on the calendar. The duty personnel need to patrol the forest fire situation all day long, report any fire hazards in a timely manner, and do a good job in on-site disposal. It supports statistical management of duty personnel according to different management zones to help managers arrange duty tasks more reasonably.

[0257] Work calendar: Used to record the work arrangements and implementation status of fire prevention inspections, fire prevention inspections, fire prevention publicity, fire prevention duties, etc. in the forest area. Record the work arrangements, duty personnel, duty time, fire prevention material allocation, etc. of each fire prevention work on a weekly or monthly basis. With the help of this function, staff can easily consult and arrange daily work tasks to ensure the orderly progress of various fire prevention work. At the same time, after the work tasks are completed, they can be directly recorded on the calendar, facilitating later data summary and analysis, improving management efficiency. Urge relevant staff to complete tasks on time, thereby ensuring the forest fire prevention safety in the forest area.

[0258] To earnestly do a good job in forest and grassland fire risk early warning, improve the accuracy of fire risk forecasting, and build a localized forest fire risk early warning system. Through sampling analysis and combustion experiments on combustibles in forest areas and grasslands, integrating objective factors such as combustible load, moisture content of combustibles, meteorology, topography and geomorphology, forest vegetation, and real-time monitoring data, and combining subjective factors such as human activities and fire use behaviors, using information technology means such as artificial intelligence and big data, establish a forest fire risk early warning model applicable to the local area, and establish a fire risk early warning AI composite model and a fire risk early warning demonstration system.

[0259] As Figure 22 , Figure 23 shown, the forest and grassland fire risk early warning system includes a user layer, an application layer, a platform service layer, and an Internet of Things perception layer.

[0260] The user layer includes the provincial forestry bureau, the municipal forestry bureau, townships, village communities, and other relevant units.

[0261] The application layer includes experimental analysis, information collection, fire risk factors, risk judgment, early warning forecasting, early warning release, early warning response, and system management, etc.;

[0262] The fire risk factor module includes information reporting, forest area meteorology, and forest area phenology.

[0263] Information reporting: The function of reporting and managing the dry-wet degree of combustibles and human activity data.

[0264] Forest area meteorology: Connect the real-time meteorological data collected by the forest and grassland fire risk monitoring station and the real-time and forecast data obtained from the third-party meteorological interface to the forest area meteorology module, and display the data in the form of charts from different dimensions.

[0265] Forest area phenology: Take the moisture content of combustibles, soil moisture content monitored by the forest and grassland fire risk monitoring station, and the combustible load survey data of the risk census as forest area phenology factors and connect them to the system, and display them through charts.

[0266] The fire risk early warning module includes risk judgment, forecasting and early warning, early warning release, and early warning response.

[0267] As Figure 23 shown, the risk judgment includes forest area meteorology analysis, forest area phenology analysis, and human activity analysis.

[0268] Forest area meteorology analysis: Based on the real-time and forecast data of meteorological factors such as temperature, humidity, wind speed, wind direction, and rainfall in the forest area collected by the fire risk factor module, combined with GIS spatial analysis technology, complete the expandable analysis from "point" to "surface", and present the dynamic change process of each element through layers.

[0269] Forest phenology analysis: Based on the correlation between meteorological factors and phenological factors in the forest area, the dynamic change processes of fuel moisture content and soil moisture content under different meteorological factors in the forest area are presented in the form of layers.

[0270] Analysis of human activities: Statistically analyze the data of outdoor fires monitored by forest fire videos, as well as the data of spring plowing, autumn harvest, major activities, and major events, and generate corresponding heat maps according to the fire frequency and the influence of subjective human factors.

[0271] Forecast and early warning include:

[0272] Threshold alarm: Implement custom threshold alarms, such as sending alarm reminders when the temperature exceeds 38°C or the fuel moisture content is lower than 10%.

[0273] Under different meteorological and phenological conditions in the forest area, display the forest fire risk level forecast, trend prediction, and early warning signal effects according to the time series layers.

[0274] Early warning release: Release the forest fire risk level forecast, trend prediction, or early warning signal information automatically or manually through the background.

[0275] Early warning response: Record whether each district and county has taken corresponding work measures in response to different forest fire risk levels and early warning signal releases.

[0276] Experimental analysis module: Import experimental observation data such as ignition point measurement data, fuel load survey data in 2022 and 2023, and fuel moisture content into the system. Its experimental data includes: moisture content change analysis experiment, fuel type classification experiment, and fuel combustion analysis experiment.

[0277] The system management module includes:

[0278] User management: Management of different users of the system.

[0279] Role management: Management of role function permissions.

[0280] Regional management: Management of user regional levels.

[0281] Access to layer data such as fuel type zoning data and administrative division data.

[0282] Layer management: Management of the basic information of monitoring stations and the operation status of equipment (meteorological and moisture content monitoring equipment).

[0283] Equipment management: Generate a QR code for each monitoring station, record the basic information, operation and maintenance records, and real-time data of the monitoring station (unauthenticated users do not have the permission to view data and can only view basic information).

[0284] Contact list: The contact list situation at the personnel level can be queried.

[0285] Announcement Release: System Announcement, Message Management.

[0286] The platform service layer includes transmission networks (private lines, 4G, 5G), as well as device access services, map engine services, application support services, message sending services, etc.

[0287] 1. Data Services:

[0288] (1) Public Basic Data Services:

[0289] Public basic data services consist of major big data sets such as perception device files, perception data archives, early warning event data, administrative directory data, etc. Through the support of private cloud resources and virtualized resource pools, a basic framework support for distributed storage of big data and data streaming processing is realized. It provides an efficient and reliable basic environment for big data access and processing for multiple system services in the upper-level basic service layer.

[0290] Big Data Platform: Based on the support of private cloud resources and virtualized resource pools, a big data engine is deployed for efficient access services to the output data of various front-end perception devices, perception data, and related data analysis and data mining. Among them, the main database engines include: graph databases, spatio-temporal databases, time-series databases, key-value databases, document databases, relational databases, and other database engines for different data fields and data applications, and data can be effectively transferred and governed under the big data platform, and various database engines are organically combined.

[0291] Perception Device File: Through a unified perception device file, record information such as device type, affiliated region, device protocol, management account, installation location, maintenance responsible person, maintenance status, maintenance log, etc. related to various front-end perception devices deployed across the province. It can conveniently and accurately provide basic data support for the daily inspection and maintenance of front-end perception devices.

[0292] Perception Data Archive: For different types of front-end perception data, the collected front-end perception data is saved in a timely and effective manner through a suitable database engine, and data support for efficient and accurate data retrieval and extraction is provided for upper-level services and applications that need to apply perception data.

[0293] Early Warning Event Data: By recording the early warning event data generated by each AI algorithm service and effectively storing it through a spatio-temporal database engine, it can provide support for subsequent big data analysis and trend prediction with an effective spatio-temporal data set. Furthermore, it provides quantifiable data sample support for subsequent business decision-making and optimization of disposal plans from a macro level.

[0294] (2) Forestry Data Services:

[0295] Forestry data services consist of large datasets in major forestry - exclusive fields such as forestry basic data, forestry thematic data, and forestry comprehensive data. It is used to store and carry the main data analysis results related to forestry generated after big - data analysis and processing of public basic data, and can effectively provide corresponding data service support for forestry - related operations.

[0296] Forestry basic data: Through the uniformly - managed forestry basic data, it provides basic metadata support with unified standards for business applications in the forestry field.

[0297] Forestry thematic data: Through customizable automated data statistical analysis task scheduling, it provides data support for automatically generating and publishing thematic data reports for business applications in the forestry field.

[0298] Forestry comprehensive data: Through customizable data processing pipelines, it provides automated data processing processes for business applications in the forestry field, used to associate data information from other large databases, and thus provides comprehensive data service support such as data retrieval and data extraction for upper - layer applications.

[0299] (3) GIS data services:

[0300] GIS data services consist of various geographical information basic datasets such as vector maps, raster maps, and image maps. Due to the particularity of the spatial relationships it carries, GIS data services require corresponding dedicated spatial indexes for fast retrieval and access. GIS data services can be used to provide various geographical space information data and their index retrieval support required by upper - layer GIS services and business applications.

[0301] Vector map: A vector map contains 2D or 3D vector data and can be used for retrieving and analyzing geographical information data based on spatial relationships. At the same time, the road topology data and terrain elevation data contained in the vector map can also be used for GIS navigation - assisting functions such as fast path calculation.

[0302] Raster map: A raster map uses pre - rendered hierarchical tile maps based on vector maps and can provide simple and easy - to - use map display function support for low - performance and low - bandwidth front - end devices.

[0303] Image map: An image map uses hierarchical tile maps obtained from aerial photography or satellite remote sensing, and can provide more intuitive map content display or richer information content display through layer superposition.

[0304] 2. Basic services:

[0305] (1) Perceived data interconnection service:

[0306] The perception data interconnection service consists of three main system modules: a data processing pipeline, a data verification pipeline, and a data interconnection log. The perception data interconnection service mainly provides pipeline processing work such as data format conversion, data validity verification, and data integrity verification for the original output data of the aggregated perception devices. And the standardized and valid data after processing and verification is stored in the common basic data service for further analysis and business applications by other system services.

[0307] (2) Data processing pipeline:

[0308] Through a customizable data processing pipeline, it is possible to flexibly access the perception data output by front-end perception devices of different types and protocols and convert it into a unified perception data type to prepare for subsequent data verification.

[0309] (3) Data verification pipeline:

[0310] Through a customizable data verification pipeline, it is possible to define multiple data validity and integrity verification processes to verify and perform necessary repairs on the data that has completed data format standardization through the data processing pipeline to ensure that the data is valid when entering the large database for storage.

[0311] (4) Data interconnection log:

[0312] Since it is necessary to interconnect and dock the platform with multiple different types of data sources, a unified data interconnection log is used to log the entire process of data processing and verification of the data obtained by the platform, which can ensure the traceability of the sources of all perception data in the large database. It can also provide effective log support for troubleshooting data processing problems during daily operation.

[0313] 3. Big data analysis service:

[0314] The big data analysis service consists of three main system modules: multi-dimensional spatial big data analysis, full-text data search, and thematic data report generation. The big data analysis service mainly provides services related to big data analysis, retrieval, and refinement, such as multi-dimensional spatial big data analysis, full-text data index generation and search, and thematic data report editing and automatic generation based on business for the perception data, warning events, and other relevant data sources stored in the common basic data service. The big data analysis service can provide efficient and unified big data analysis result support for other system modules.

[0315] (1) Multi-dimensional spatial big data analysis:

[0316] With the support of customizable data dimensions and spatial dimensions, various data sets stored in large databases can be statistically analyzed independently or in association, and corresponding multi-dimensional spatial big data analysis results can be produced. These data analysis results can be used as training sets for full-text retrieval, thematic data reports, or AI models. It can effectively simplify and improve the implementation efficiency and operation efficiency of subsequent big data applications.

[0317] (2) Full-text data indexing and search:

[0318] By generating full-text data indexes for big data analysis results and documented unstructured big data sets, it can provide efficient full-text data search capabilities for upper-layer services or business applications. Through the vectorization of full-text data indexes, it can provide basic data corpus and knowledge base support for the application of subsequent language large models.

[0319] (3) Thematic data report generation and release:

[0320] Through customizable thematic data reports, it can provide basic service support for the automated generation and release of unified data reports for the thematic data report requirements of different business fields. With the support of multi-dimensional spatial big data analysis and basic data services, more complex and effective thematic data reports can be customized, thus simplifying the implementation of upper-layer business applications and improving operation efficiency.

[0321] 4. AI algorithm repository service:

[0322] The AI algorithm repository service consists of three main system modules: AI algorithm inference, AI algorithm training, and AI algorithm deployment. The AI algorithm repository service mainly provides online / offline model training task scheduling and execution based on various AI algorithms such as machine learning, deep neural networks, and visual recognition, AI model effect verification and evaluation, and online deployment and update of AI models. At the same time, the AI algorithm repository service also provides real-time inference task scheduling and load balancing for AI models. The AI algorithm repository can provide integrated AI algorithm service support for big data analysis and business applications.

[0323] (1) AI algorithm inference:

[0324] Based on the support of private cloud resources and virtualized resource pools, AI algorithm inference performs containerized dynamic allocation and scheduling of clustered AI computing power, so as to make full use of the AI computing power in the cluster to improve the timeliness and processing throughput of AI prediction.

[0325] (2) AI algorithm training:

[0326] AI algorithm training can be divided into two methods: online training and offline training according to different AI models. The AI models for online training can, through configuration, allocate more resources in the specified virtualized containers to complete incremental AI training using online data while completing AI inference. The AI models for offline training can complete the optimized training of the AI models by allocating dedicated computing resources and the offline data sets required for training. AI algorithm engineers can also monitor and manage the execution of AI algorithm training tasks in different containers through the console of the AI algorithm repository, and make corresponding configurations and parameter adjustments as needed to obtain the best AI model training results.

[0327] (3) AI algorithm deployment:

[0328] AI algorithm deployment pre-runs the AI model to be upgraded and deployed in a small-scale containerized manner, using the same online data as the AI model running online. By comparing the differences in the effectiveness and completeness of the prediction results generated by different versions of the AI model when processing the same online data, the optimization effect of the new version of the AI model can be more accurately quantified and evaluated. When it is confirmed that the optimization of the new version of the AI model is effective, all related AI models running online can be automatically updated to simplify the daily maintenance process.

[0329] 5. GIS services:

[0330] GIS services mainly consist of two main system modules: geographic coordinate conversion and geographic information search. GIS services can provide algorithm support for the mutual conversion between different geographic coordinate systems, the projection conversion of 2D and 3D spatial coordinates, etc. At the same time, GIS services can also provide support for comprehensive information queries of various geographic information data such as points, lines, and surfaces based on keywords, spatial relationships, etc. GIS services can provide unified and effective geographic information service support for big data analysis, AI algorithm repositories, and business applications.

[0331] (1) Geographic coordinate conversion:

[0332] When integrating geographic information data with other business data, it provides accurate algorithm support for the interchange between different geographic coordinate systems, the projection conversion of 2D and 3D spatial coordinates, etc. It can simplify data interconnection with external systems and the implementation of upper-layer services and business applications.

[0333] (2) Geographic information search:

[0334] Geographic information search provides service capability support for querying various types of geography-related information and its extended associated perception data, warning events, or business data in a comprehensive manner based on keywords or spatial relationships, etc. Upper-layer services or business applications can create the required geographic space data index through the support of the geographic information search service, providing basic support for improving search performance.

[0335] 6. Business support services:

[0336] (1) Perception data support:

[0337] The perception data support consists of main system modules such as real-time on-demand, historical playback, data report retrieval, and multi-channel data distribution. Through real-time on-demand and historical playback of perception data, it can effectively support the requirements of business applications for the timeliness and effectiveness of obtaining perception data.

[0338] Through data report retrieval, it can help business applications simplify the complexity of data analysis and report generation, reduce the development cost of business applications, and improve the flexibility of business applications in responding to different business requirements.

[0339] Through multi-channel data distribution, it can effectively reduce the demand for the internal data transmission bandwidth of the system platform, and improve the transmission efficiency of end-users to obtain data, thereby enhancing the user experience of business applications.

[0340] (2) Real-time on-demand of perception data:

[0341] Real-time on-demand of perception data can provide support for upper-layer business applications to push the real-time acquisition data of specified front-end perception devices to the application front-end for real-time playback and display.

[0342] (3) Historical playback of perception data:

[0343] Historical playback of perception data can provide support for upper-layer business applications to push the perception data collected by specified front-end perception devices within a specified time range in history to the application front-end for playback and display.

[0344] (4) Data report retrieval:

[0345] With the support of the lower-layer thematic data report service, it provides functional support for upper-layer business applications to retrieve the generated data reports according to specified conditions. And it provides support for upper-layer business applications to extract the data of specified thematic data reports and push them to the application front-end for interactive display.

[0346] (5) Multi-channel data distribution:

[0347] Based on the characteristic that upper-layer business applications access perception data and there are many terminals accessing the perception data of the same front-end perception device simultaneously. The multi-channel data distribution service improves the response speed to the application front-end through mechanisms such as multi-copy replication and local caching of perception data. At the same time, it can effectively reduce the access pressure on the lower-layer perception data extraction service and reduce the bandwidth occupancy of high-cost links in the platform.

[0348] 7. Business application support:

[0349] The business application support consists of main system modules such as front-end interaction components, custom business process management, custom business directory management, custom topic data reports, and subscribable warning message queues.

[0350] Through the front-end interaction components, it can help the business application quickly integrate the user interaction interface for perceiving data on-demand and playback. At the same time, it also provides other common front-end interaction components including electronic maps, topic data reports, etc.

[0351] Through the custom business process management, it can help the business application provide custom management of business processes such as perceiving data distribution, warning event triggering, and notification to its end-users. It simplifies the complexity of the business application in related functions and improves the efficiency of related data processing and transmission.

[0352] Through the custom business directory management, it can help the business application provide custom management of other domain directories except the administrative directory to its end-users, and can be applied to various data processing and application scenarios such as big data analysis and topic data reports. It simplifies the complexity of the business application in related functions and improves the efficiency and flexibility of related data processing.

[0353] Through the custom topic data reports, it can help the business application provide the generation of customizable topic data reports to its end-users, and complete the corresponding data analysis and report generation in the big data service. While simplifying the complexity of the business application, it improves the efficiency and flexibility of the corresponding data processing.

[0354] Through subscribing to the warning message queue, it can help the business application provide a timely and reliable warning message notification function to its end-users, so as to ensure that warning messages of different priorities can be pushed to the corresponding business application terminals in a timely manner.

[0355] (1) Custom business process:

[0356] Through the custom business process, some business processes that can be automatically processed in the platform can be pre-arranged. Thus, it simplifies the complexity of the front-end business application implementation and improves the operation efficiency. The business process is implemented using an event-driven model. Through the customizable event message queue and the corresponding event processing nodes, the implementation of the automated business process can be flexibly completed.

[0357] (2) Custom business directory:

[0358] Through the custom business directory, the directory definitions related to the business can be integrated into multiple scenarios such as the data processing pipeline, verification pipeline, multi-dimensional spatial analysis, and topic data report generation of big data. While ensuring the standardization of the big data data set, the flexibility of the business application is achieved through different business directory mappings.

[0359] (3) Customized Special Topic Data Report:

[0360] Through the customized special topic data report, the statistical dimensions, analysis indicators, time, spatial scope, etc. of the corresponding data report can be defined according to different business requirements to generate a data report. At the same time, the generation cycle and release channels of the specified special topic data report can be customized. With the support of the customized special topic data report, the complexity of implementing related functions in business applications can be simplified, and the generation performance of the data report can be optimized.

[0361] (4) Subscribable Early Warning Message Queue:

[0362] Through the subscription method, the front-end business application can obtain timely notifications of early warning messages and achieve integrated docking of the platform under the unified standard message queue interface. This can effectively ensure the efficiency and reliability of the push and distribution of early warning messages.

[0363] 8. Access Control Support:

[0364] The access control support consists of three main system modules: identity authentication, access authorization, and access logs. Through identity authentication, the user identity authentication service of the digital forestry system can be docked to achieve single sign-on identity recognition of users. Through access authorization, the user permission authorization service of the digital forestry system can be docked to achieve unified management and effective authorization of users' functional operations and data access. Through access logs, the user access log service of the digital forestry system can be docked to record and synchronize users' access records to the platform in a timely manner to achieve unified supervision and auditing of user access logs.

[0365] (1) Identity Authentication:

[0366] The platform provides two identity authentication mechanisms: built-in and external. The built-in identity authentication mechanism takes the front-end business applications, external system platforms, and internal users of this platform docked as the authentication objects and completes identity authentication based on digital keys or password keys. The external identity authentication mechanism takes the front-end business applications, external system platforms, and external users docked as the authentication objects and completes the identity authentication of external users based on the single sign-on protocol.

[0367] (2) Access Authorization:

[0368] The platform provides two access authorization mechanisms: built-in and external. The built-in access authorization mechanism takes the front-end business applications, external system platforms, and internal users of this platform docked as the authorization objects and completes access authorization based on the role and permission allocation table. The external access authorization mechanism takes the front-end business applications, external system platforms, and external users docked as the authorization objects and realizes the access authorization of external users based on the Open Authorization protocol.

[0369] (3) Access Log:

[0370] The platform uniformly records access logs for the identity authentication and access authorization of all users, providing log record support for the daily supervision and auditing of user access logs.

[0371] The Internet of Things perception layer includes self-built monitoring stations, automatic weather stations, forestry weather stations, third-party access, etc.

[0372] Based on terminal perception such as satellite remote sensing, unmanned aerial vehicles, high and low position video monitoring, infrared cameras, sensors, etc., the present invention constructs an ecological perception system integrating multiple senses, focusing on improving the intelligent monitoring and management level in areas such as forest farms, wetland parks, nature reserves, scenic spots, and forest parks. The present invention combines big data analysis and AI technology to accurately identify and real-time monitor forest fire prevention, gradually realizing more accurate forestry ecological governance, more scientific prediction of development trends, more timely early warning of ecological security risks, and more effective support for forestry decision-making and deployment.

[0373] The above are only the preferred embodiments of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention.

Claims

1. A multi-sensor integrated forest fire automatic identification and early warning method, characterized in that: The following steps are involved: S1: Collect data to the system through terminal sensing devices, and analyze and process the collected data; S2: Extract smoke and fire features from the preprocessed data; S3: Build a convolutional neural network deep learning model to perform fireworks recognition on real-time image or video data; The model includes 3 head convolution layers, 3 multi-scale feature extraction modules, 6 Transformer modules and 1 classification layer; First, the preprocessed data is obtained and resized into 3×224×224 RGB images and input into three convolutional layers to generate spatial detail features; Then, three multi-scale feature extraction modules are used to fully extract the fire feature information in the image, laying a feature foundation for the subsequent feature mapping of high-level spatial regions; After the multi-scale feature extraction module, the feature map with position embedding and class labeling is input into the Transformer module to fully integrate the global information; finally, the class tokens are used to complete the fire detection through the classification layer composed of fully connected layers; The multi-scale feature extraction module consists of four different branches, and its specific calculation formula is as follows: B=[H 1×1 (A),H 3×3 (A),H 5×5 (A),H MaxPool (A)], Among them, A is the image after the convolution layer processing; B is the result of merging the four branches, H MaxPool () represents 3×3 maximum pooling, H n×n () represents n×n convolution operations; Transformer module: The global feature extraction capability of the Transformer module mainly relies on the multi-head attention mechanism. The calculation formula of the self-attention mechanism is: Among them, Q, K, and V all refer to the submatrices obtained by passing the one-dimensional vector sequence input to the module through a transformation matrix, d k is a scaling factor, Softmax is a normalized exponential function; K T represents the transpose of the K matrix, and Attention represents the result of the self-attention mechanism; MSA=Concat(head1,head2,...)W 0 , Among them, MSA represents the multi-head self-attention mechanism, head represents the result of the self-attention mechanism mentioned above, and Concat represents connecting different heads in series and multiplying them by the weight matrix W 0 ; Improvements to the Transformer module: Linear projection to convolution projection: The token is reshaped into a 2D token map, and a convolution projection is implemented using a deep convolution with a convolution kernel of s. Finally, the projected token is flattened into 1D. The specific calculation formula is: X q / k / v =ConvProj(X)=Flatten(DWConv(Reshape2D(X),s)), Among them, X q / k / v is the labeled input of the q / k / v vector, X is the input label before convolution projection, DWConv() represents the depth convolution operation, s is the convolution kernel size; ConvProj() represents the convolution projection operation, Flatten() represents the flattening operation, and Reshape2D() represents the data reshaped into a two-dimensional form; The feedforward network is improved to a convolutional residual feedforward network: The original feed-forward network consists of two linear layers separated by a GeLU activation function; the specific expression is as follows: FFN(X)=GeLU(XW1+b1)W2+b2, Among them, X is the output of the multi-head attention mechanism of the previous layer, W1∈R D×4D , W2∈R 4D×D They represent the weights of the two linear layers, b1 and b2 are bias terms; GeLU is the activation function; The improved feedforward network is a convolutional residual feedforward network, which includes an expansion layer, a depth convolution, a SE module, and a projection layer. The specific calculation formula is: y=DWConv(x)+x, z=SE(y), CRFFN(x)=Conv(z), Wherein, x represents the original data input to the residual module, DWConv() represents the deep convolution operation, y=DWConv(x)+x represents the addition of residual connection, SE() represents the SE module, which is used to adaptively recalibrate the channel feature response, and Conv() represents the convolution operation; The final Transformer module is represented as: Among them, X0 represents the multi-scale feature extraction module export parameter B after dimensional transformation, adding class label x class and position embedding E pos As the input of the first transformer module; X i-1 represents the input of the i-1th Transformer module, L represents the number of Transformer modules; LN() represents layer normalization, Z i represents the output of the first residual block, X i Represents the output of the i-th Transformer module; when continuing to execute the last Transformer module, y represents the output of the model; CRFFN() represents the convolutional residual feedforward network; MHSA() represents the multi-head self-attention mechanism, and ConvProj() represents the convolutional projection operation; S4: After the system automatically identifies signs of fire, it issues an alarm message and uses the positioning algorithm to confirm the location information of the alarm point; S5: Determine the regional position of the alarm point based on the positioning information of the alarm point.

2. The multi-sensor integrated forest fire automatic identification and early warning method according to claim 1 is characterized by: In S1, the terminal sensing device collects data to the system, and the specific contents of analyzing and processing the collected data are as follows: The collected image or video data is processed by denoising, color space conversion, and image enhancement.

3. The multi-sensor integrated forest fire automatic identification and early warning method according to claim 1 is characterized by: In S4, after the system automatically identifies signs of fire, it issues an alarm message and uses the positioning algorithm to confirm that the specific content of the alarm point positioning information is: The system automatically identifies forest fires and initiates an alarm. Based on the location, status, and elevation angle data of the monitoring equipment, the system uses deep learning and convolutional neural networks to achieve automatic image-view registration and accurately locate forest fires. At the same time, combined with the map system, the location of the alarm point is presented on a three-dimensional map. When there are multiple monitoring points around the fire, cross-positioning is used to improve the accuracy of fire point positioning.

4. The multi-sensor integrated forest fire automatic identification and early warning method according to claim 1 is characterized by: In S5, the specific content of determining the regional position of the alarm point based on the positioning information of the alarm point is: When receiving an alarm signal, the location information of the alarm point is obtained at the same time, and then the regional location is determined. The monitored area is divided into different geographical areas, and the location information of the alarm point is matched with the pre-divided areas. At the same time, based on the precise latitude and longitude coordinates, the geographic information system tools are used for position matching to determine the specific area where the alarm point is located.

5. A multi-sensory integrated forest fire automatic identification and early warning system, which realizes the multi-sensory integrated forest fire automatic identification and early warning method as claimed in any one of claims 1 to 4, characterized in that: Including terminal sensing equipment, smart forest fire prevention system and forest and grassland fire risk early warning system; Terminal sensing devices include: satellite remote sensing, drones, high and low-level video surveillance, infrared cameras and sensors; The smart forest fire prevention system includes a cockpit module, a picture module, a human defense module, a technical defense module, a physical defense module, a disposal module and a collaborative module; The forest and grassland fire risk warning system includes user layer, application layer, platform service layer and Internet of Things perception layer.

6. The multi-sensor integrated forest fire automatic identification and early warning system according to claim 5 is characterized by: The cockpit module includes a geographic information system map, real-time video monitoring, and a point resource list; The one-map module combines data of electronic maps, remote sensing images, digital elevation models, urban infrastructure, forest fire prevention facilities, administrative divisions, and responsibility grids; The civil air defense module: collects statistics on personnel information, firefighting teams, emergency teams, and semi-professional teams, and displays them on a map; The technical defense module collects statistics on the technical defense information of the PTZ probe, smart card port, and drone, and displays it on a map; The physical defense module: realizes the display of point resource data on a three-dimensional map, including education facilities, water points, fire prevention construction and hazard source data; The handling module: After the system discovers a fire, it pushes it to the grid responsible person to create a fully closed-loop process for forest fire prevention, including: fire discovery, alarm push, multi-cross collaboration, and fire handling; The collaborative module includes department management, role management, user management, alarm management, resource management, equipment management and attendance management.

7. The multi-sensor integrated forest fire automatic identification and early warning system according to claim 5 is characterized by: The user layer includes provincial forestry bureaus, municipal forestry bureaus, townships, and villages; The application layer includes experimental analysis, information collection, fire risk factors, risk assessment, early warning forecast, early warning release, early warning response and system management; The platform service layer includes transmission network and device access services, map engine services, application support services, and message sending services; The IoT perception layer includes self-built monitoring stations, automatic weather stations, forestry weather stations and third-party access.

Images