Forest fire prevention monitoring system and method based on multi-mode AI and intelligent cooperation

Abstract

The invention discloses a forest fire prevention monitoring system and method based on multi-modal AI and intelligent cooperation, and relates to the technical field of fire monitoring, and the system comprises an information collection module, a multi-layer layout module, a range analysis module, a layout optimization module, a preprocessing module, and an anomaly analysis module. The system has the advantages that monitoring equipment can be reasonably arranged through the multi-layer layout module according to the topography and actual requirements of the forest, the monitoring efficiency is improved, effective monitoring of all areas of the forest is ensured, monitoring overlapping and missing areas are reduced, smoke and noise interference of different scales is effectively coped with through the preprocessing module, and the real-time performance of the forest is improved. Meanwhile, self-adaptive Gamma correction is utilized to dynamically adjust image brightness distribution, the quality and detail definition of the image are improved, the optimized image is converted into an optimized monitoring video, high-quality video materials are provided for an anomaly analysis module, and abnormal conditions such as fire disasters and the like can be detected more accurately.

Description

Technical Field

[0001] The present invention relates to the field of fire monitoring technology, and specifically to a forest fire monitoring system and method based on multimodal AI and intelligent collaboration. Background Art

[0002] Forest fire prevention is the prevention of forest fires. Forest fires are the most dangerous enemy of forests and the most terrible disaster in forestry. Forest fires can lead to devastating consequences such as ecosystem collapse and loss of biodiversity. Forest fires not only burn large tracts of forests and harm animals in the forests, but also reduce the reproductive capacity of forests, cause soil infertility, destroy forest water conservation, and even cause the ecological environment to lose balance. Common forest fire monitoring systems and methods have low monitoring efficiency when in use, and there are overlapping and missed areas of monitoring, resulting in low quality of monitoring videos in some locations, affecting the judgment of forest fires. At the same time, the anti-interference ability of smoke and noise of different scales is insufficient, and the brightness of the monitoring video cannot be dynamically adjusted. For this reason, we propose a forest fire monitoring system and method based on multimodal AI and intelligent collaboration. Summary of the invention

[0003] The purpose of the present invention is to provide a forest fire monitoring system based on multimodal AI and intelligent collaboration.

[0004] To achieve the above objectives, the present invention provides the following technical solutions: a forest fire monitoring system and method based on multimodal AI and intelligent collaboration, the forest fire monitoring system comprising: The information collection module is used to collect the forest range and the monitoring equipment information required for forest fire prevention, and obtain the spatial effective range of the monitoring equipment information to obtain the spatial effective range; The effective spatial range refers to the maximum distance that the monitoring device can monitor under the condition of set video clarity; The multi-layer layout module obtains monitoring equipment, builds a three-dimensional model for the forest, builds different monitoring layout layers above the virtual forest in the model, sets the initial monitoring angle of the monitoring equipment, analyzes the central angle parallel plane of the monitoring equipment according to the initial monitoring angle, and extracts the position where the angle parallel plane contacts the forest, which is regarded as the monitoring center boundary, determines the upper and lower effective monitoring angles, obtains the upper monitoring plane and the lower monitoring plane, obtains the forest monitoring range according to the upper monitoring plane and the lower monitoring plane, extracts the contact line between the upper monitoring plane and the lower monitoring plane and the forest, obtains the upper boundary and the lower boundary, and arranges the monitoring equipment in different monitoring layout layers in sequence according to the monitoring center boundary, the upper boundary and the lower boundary, and the upper boundary of the X+1th monitoring equipment coincides with the monitoring center boundary of the Xth monitoring equipment, so as to obtain a hierarchical monitoring layout; Range analysis module, which obtains surveillance videos according to the surveillance devices in the hierarchical surveillance layout, analyzes the clarity of different surveillance devices based on the surveillance videos, and obtains the execution clarity; Layout optimization module, which generates an optimization plan according to the execution clarity of the surveillance devices and the distances between different surveillance devices and the boundary of the surveillance center, and optimizes the hierarchical surveillance layout according to the optimization plan; Preprocessing module, which extracts forest images, establishes a 4-level pyramid structure, decomposes the input image into a 4-level pyramid structure, and the resolutions of the 4-level pyramid structure are successively the original Figure 1 / 2, 1 / 4, 1 / 8, 1 / 16, which are used for hierarchical processing of smoke and noise interference at different scales, dynamically adjusts the image brightness distribution by using adaptive Gamma correction, obtains the enhanced forest image, and converts the enhanced forest image into an optimized surveillance video through Adobe Premiere Pro.

[0005] As a further solution of the present invention: the forest fire prevention surveillance system further includes an anomaly analysis module, which is used to compare the delayed images and generate a fire alarm signal. The anomaly analysis module extracts the optimized surveillance video, establishes a delayed comparison function, intercepts the forest video in real time to obtain the delayed images, records the delayed images, sets the delay time, compares the delayed images with the delayed images after the delay time, and uses AI to analyze whether a fire has occurred in the delayed images after the delay time. If a fire is detected, an alarm signal is generated and the responsible person is notified, otherwise no alarm signal is generated; When a fire occurs, different ways of alarm can be carried out; The first way is to trigger group messages + SMS + phone calls, and the phone calls can be made through Alibaba Cloud communication services; The second way is only group messages + enterprise WeChat reminders, and the enterprise WeChat reminders integrate the DingTalk API; The third way is only group messages, and the group messages have a delayed confirmation mechanism; At the same time, a DingTalk alarm push engine is set. The DingTalk robot can select the push channels according to the fire level, where the push channels include group messages, phone calls and SMS, and call the Yuanbao large model to generate a Markdown report. The Markdown report includes a fire heat map, disposal suggestions and to-do items for the responsible person; Generate a fire heat map through Matplotlib + Plotly, match and generate disposal suggestions based on the historical case library, and analyze the to-do items for the responsible person according to the automatic association of the enterprise organizational structure; As a further solution of the present invention: when the hierarchical monitoring layout in the multi-layer layout module is obtained, clarity boundaries are set for different monitoring arrangement layers, and the monitoring distance ranges of different monitoring devices within different clarity boundaries are analyzed. The intermediate distance of the monitoring distance range is calculated to obtain the execution distance. The distance between the monitoring device and the center point of the monitoring center boundary under the current monitoring angle is analyzed to obtain the angular distance. Then, the execution angle of the monitoring device is calculated based on the angular distance, the monitoring angle, and the execution distance, and the monitoring calibration is adjusted according to the execution angle.

[0006] As a further solution of the present invention: when calculating the intermediate distance of the multi-layer layout module, let the monitoring distance range be , and let the intermediate distance be : ; The intermediate distance is calculated through the above formula. At the same time, regarding the intermediate distance as the hypotenuse, the distance from the central angle parallel plane to the monitoring device is measured to obtain the horizontal distance, and the execution height of the monitoring device is calculated based on the intermediate distance and the horizontal distance.

[0007] As a further solution of the present invention: when calculating the execution height of the monitoring device in the multi-layer layout module, let the horizontal distance be , let the intermediate distance be , and let the execution height of the monitoring device be : ; The execution height of the monitoring device is calculated through the above formula. The height of the first monitoring arrangement layer is determined, and then the monitoring clarity is set for the remaining monitoring arrangement layers to determine the height positions of the remaining monitoring arrangement layers.

[0008] As a further solution of the present invention: after determining the height positions of the remaining monitoring arrangement layers in the multi-layer layout module, the left and right effective monitoring ranges of the monitoring device are set, the intermediate value of the left and right effective monitoring ranges is calculated, and it is used as the interval distance between adjacent left and right monitoring devices.

[0009] As a further solution of the present invention: when the preprocessing module extracts the forest image, it analyzes the video frames of the monitoring video, collects the GAN network dehazing enhancement technology, and uses the GAN network dehazing enhancement technology to repair the monitoring video. At this time, ResNet-50 and 3D-CNN are obtained. The preprocessing module uses ResNet-50 to extract spatial features, 3D-CNN to extract motion features, and uploads the dehazed enhanced video, spatial features, and motion features to the cloud using the MQTT protocol.

[0010] As a further solution of the present invention: the preprocessing module fuses the features of the RGB image and the depth map through the view fusion module of Transformer to eliminate forest occlusion. At the same time, it is linked with the DingTalk robot. The DingTalk robot has the permission to customize the message template according to the fire elements, and the DingTalk robot has the permission to push the WebSocket to the command large screen and automatically generate work orders.

[0011] In addition, a forest fire prevention monitoring method based on multi-modal AI and intelligent collaboration is also provided, and the method is applicable to the forest fire prevention monitoring system based on multi-modal AI and intelligent collaboration described above.

[0012] Adopting the above technical solutions, compared with the prior art, the beneficial effects of the present invention are as follows: 1. Through the multi-layer layout module, the present invention can reasonably arrange monitoring devices according to the terrain and actual needs of the forest, improve the monitoring efficiency, ensure the effective monitoring of all areas of the forest, reduce the overlapping and omitted areas of monitoring, effectively cope with smoke and noise interference of different scales by using the preprocessing module, and at the same time dynamically adjust the image brightness distribution by using adaptive Gamma correction, improve the quality and detail clarity of the image, convert the optimized image into an optimized monitoring video, provide high-quality video materials for the anomaly analysis module, and help to more accurately detect anomalies such as fires. 2. Through the multi-layer layout module, the present invention can make each monitoring device be in the best monitoring angle, optimize the monitoring field of view, improve the clarity of the monitoring picture and the amount of effective information obtained, thereby improving the monitoring quality of the entire monitoring system for the forest area, helping to reasonably plan the spatial layout of the monitoring devices, ensuring that the monitoring ranges of different monitoring devices cooperate with each other, forming a multi-level and all-round monitoring system, enhancing the monitoring ability for complex terrains and forest environments, avoiding resource waste caused by overly dense monitoring devices, and improving the layout rationality and resource utilization efficiency of the monitoring system. 3. Through the preprocessing module, the present invention can improve the video quality, make the monitoring picture clearer, and the extracted multiple features help to more comprehensively analyze the forest situation. Uploading to the cloud is convenient for data storage, sharing and further analysis, facilitating multi-department collaborative work and comprehensive judgment of forest fires, further optimizing the processing effect of the monitoring video, improving the ability to obtain detailed information of the image, and helping to more accurately identify potential fire hazards and other anomalies in the forest. Description of the Drawings

[0013] Figure 1 It is a schematic diagram of the system process in the embodiment of the present invention. Detailed Embodiments

[0014] The following further describes the specific embodiments of the present invention in conjunction with the accompanying drawings. It should be noted here that the description of these embodiments is used to help understand the present invention, but does not constitute a limitation on the present invention.

[0015] In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0016] Embodiment 1: The forest fire prevention monitoring system of the present invention based on multimodal AI and intelligent collaboration. In vast forest areas, forest fires are the most dangerous enemies of forests, having a great destructive effect on the forest ecosystem. Traditional forest fire prevention monitoring means have many deficiencies and are difficult to effectively meet the needs of forest fire prevention. The forest fire prevention monitoring system and method based on multimodal AI and intelligent collaboration can play an important role in the forest; When there are fire hazards in the forest, for example, there is a fire caused by human negligence leaving a fire or being struck by lightning. At this time, a large amount of smoke will appear. Due to the interference of the smoke, it will affect the monitoring system's judgment of the fire, and at the same time, it is impossible to adjust the image brightness of the monitoring video in real time according to the forest brightness, thus unable to provide a clear video; Therefore, in order to effectively solve the above problems, this application proposes a forest fire prevention monitoring system based on multimodal AI and intelligent collaboration, as shown in the accompanying drawings of the specification Figure 1 As shown, the forest fire prevention monitoring system includes: An information collection module, used to collect the information of the forest area and the monitoring equipment required for forest fire prevention, and obtain the spatial effective range of the monitoring equipment information to obtain the spatial effective range; A multi-layer layout module, which acquires the monitoring equipment, constructs a three-dimensional model of the forest, constructs different monitoring layout layers above the virtual forest in the model, sets the initial monitoring angle of the monitoring equipment, analyzes the central angle parallel plane of the monitoring equipment according to the initial monitoring angle, and extracts the position where the angle parallel plane contacts the forest, regarding it as the monitoring center boundary line, determines the upper and lower effective monitoring angles, obtains the upper monitoring plane and the lower monitoring plane, obtains the forest monitoring range according to the upper monitoring plane and the lower monitoring plane, and extracts the contact lines between the upper monitoring plane and the lower monitoring plane and the forest to obtain the upper boundary line and the lower boundary line. The monitoring equipment is arranged in different monitoring layout layers in turn according to the monitoring center boundary line, the upper boundary line and the lower boundary line, and the upper boundary line of the (X + 1)th monitoring equipment coincides with the monitoring center boundary line of the Xth monitoring equipment to obtain a hierarchical monitoring layout; Where X is a natural number; A range analysis module, which acquires the monitoring video according to the monitoring equipment in the hierarchical monitoring layout, analyzes the clarity of different monitoring equipment according to the monitoring video, and obtains the execution clarity; The layout optimization module generates an optimization plan based on the execution clarity of the monitoring devices and the distances between different monitoring devices and the boundaries of the monitoring center, and optimizes the hierarchical monitoring layout according to the optimization plan; The preprocessing module extracts forest images, constructs a 4-level pyramid structure, decomposes the input image into a 4-level pyramid structure, and the resolutions of the 4-level pyramid structure are successively 1 / 2, 1 / 4, 1 / 8, 1 / 16 of the original Figure 1 to process smoke and noise interference at different scales in layers, dynamically adjust the image brightness distribution using adaptive Gamma correction, obtain the enhanced forest image, and convert the enhanced forest image into an optimized monitoring video through Adobe Premiere Pro; Through the bidirectional mapping feature of the HFM model, the intensity range 0 - 255 of the enhanced forest image is divided into 3 dynamic regions: Low brightness region 0 - 50: Apply logarithmic transformation to expand details; Medium brightness region 51 - 200: Maintain linear mapping; High brightness region 201 - 255: Use gamma compression to prevent overexposure; Then, perform color correction on the enhanced forest image through the 3A algorithm of the Retinex theory; The forest fire prevention monitoring system also includes an anomaly analysis module for comparing delay images and generating fire alarm signals. The anomaly analysis module extracts the optimized monitoring video, establishes a delay comparison function, captures forest videos in real time to obtain delay images, records the delay images, sets the delay time, compares the delay images with the delay images after the delay time, and uses AI to analyze whether a fire has occurred in the delay images after the delay time. If a fire is detected, an alarm signal is generated and the responsible person is notified; otherwise, no alarm signal is generated; At the same time, it is also possible to use the GAN network to achieve smoke penetration and low - illumination enhancement, improve the PSNR of the image to 34dB, and use a dynamic resolution compensation algorithm to construct a Yuanbao AI fire detection model. Then, input forest videos, thermal images, and voiceprint data into the Yuanbao AI fire detection model, and then collect the Yuanbao large model. Use the Yuanbao large model to generate a Markdown report, and the Markdown report includes a fire heat map, disposal suggestions, and to - do items for the responsible person; Adaptive Gamma divides the input image into several small blocks or regions. For each small block, calculate its relevant statistical information. Common statistics include mean, variance, etc. According to the statistical information of the small block, calculate the Gamma value suitable for the small block, and then use the calculated Gamma value to correct the pixels within each small block. Combine the small blocks after Gamma correction to form the final corrected image; The GAN network structure consists of a generator, a discriminator, and adversarial training. The generator takes random noise as input and transforms it into generated data that is as similar as possible to real data through a neural network. The discriminator is responsible for distinguishing whether the input data is real data or fake data generated by the generator. It is a binary classifier. In adversarial training, the generator tries to generate fake data that can deceive the discriminator, while the discriminator tries its best to accurately distinguish between real and fake data. During the training process, the discriminator is first fixed, and the generator is updated according to its judgment of the generated data to make the generator generate more realistic data. Then, the generator is fixed, and the discriminator is updated according to the performance of the discriminator to make its discrimination ability stronger. This process is repeated until the data generated by the generator is realistic enough that the discriminator has difficulty distinguishing between true and false. At this time, the GAN reaches a balanced state, and the generator can generate high-quality data that meets the requirements. When converting the enhanced forest images into optimized surveillance videos using Adobe Premiere Pro, it is necessary to import the sequence of enhanced forest images. Premiere Pro will recognize them as frame-by-frame materials. Then, create a sequence parameter suitable for the surveillance video, such as frame rate, resolution, etc., through the "Sequence" function to match the display requirements of the surveillance device. Next, drag the image materials to the timeline and use the "Speed and Duration" function to adjust the playback speed of the images to make the image sequence form a smooth video effect. At the same time, tools such as color correction and sharpening in the "Effects" panel can be used to further optimize the video quality to make it clearer and easier to identify in the surveillance scenario. Finally, through the "Export" function, the edited video is output in a format suitable for playback by the surveillance system, such as MP4, etc., to complete the conversion from enhanced images to optimized surveillance videos. For the HFM model, preprocess the input data, then use a neural network, etc. to extract features at different levels and with different degrees of abstraction. Organize the features hierarchically. Start from the high level and match the features by calculating similarity. After successful high-level matching, refine to the low level, and use high-level semantics to guide low-level matching. Make a decision by integrating the matching information of each layer. For example, in image classification, judge based on the matching degree between the features and the category template, and a threshold may also be introduced to control the reliability of the matching. Specifically, collect the information of monitoring devices required for forest range and forest fire prevention, and obtain the spatial effective range of the monitoring device information. Construct different monitoring layout layers above the forest. Analyze the central angle parallel plane of the monitoring device according to the initial monitoring angle, and extract the position where the angle parallel plane contacts the forest, which is regarded as the monitoring center boundary. Obtain the upper monitoring plane and the lower monitoring plane. Obtain the forest monitoring range according to the upper monitoring plane and the lower monitoring plane, and extract the contact lines between the upper monitoring plane and the lower monitoring plane and the forest. Arrange the monitoring devices into different monitoring layout layers in sequence according to the monitoring center boundary, the upper boundary and the lower boundary. The upper boundary of the (X + 1)-th monitoring device coincides with the monitoring center boundary of the X-th monitoring device. Obtain the monitoring video, analyze the clarity of different monitoring devices according to the monitoring video, generate an optimization plan, optimize the hierarchical monitoring layout according to the optimization plan, extract the forest image, establish a 4-level pyramid structure, decompose the input image into a 4-level pyramid structure, and the resolutions of the 4-level pyramid structure are successively 1 / 2, 1 / 4, 1 / 8, 1 / 16 of the original Figure 1 to process the smoke and noise interference at different scales in a layered manner, use adaptive Gamma correction to dynamically adjust the image brightness distribution, obtain the enhanced forest image, use Adobe Premiere Pro to convert the enhanced forest image into an optimized monitoring video, compare the time-lapse image with the time-lapse image after the time-lapse time, and use AI to analyze whether a fire has occurred in the time-lapse image after the time-lapse time. If a fire is detected, generate an alarm signal and notify the responsible person, and make a notification. When no fire occurs, no alarm signal will be generated.

[0017] Embodiment 2: When obtaining the hierarchical monitoring layout in the multi-layer layout module, set the clarity limit for different monitoring layout layers, analyze the monitoring distance range of different monitoring devices within different clarity limits, calculate the intermediate distance of the monitoring distance range to obtain the execution distance, analyze the distance between the monitoring device and the center point of the monitoring center boundary at the current monitoring angle to obtain the angular distance, and then calculate the execution angle of the monitoring device according to the angular distance, the monitoring angle and the execution distance, and adjust the monitoring calibration according to the execution angle; When calculating the intermediate distance of the multi-layer layout module, set the monitoring distance range as and set the intermediate distance as : ; Calculate the intermediate distance through the above formula. At the same time, regard the intermediate distance as the hypotenuse, measure the distance from the central angle parallel plane to the monitoring device to obtain the horizontal distance, and calculate the execution height of the monitoring device according to the intermediate distance and the horizontal distance; Calculate the execution height of the monitoring device according to the intermediate distance and the horizontal distance, where the calculated execution height of the monitoring device is the height of the first monitoring layout layer from bottom to top; When calculating the execution height of the monitoring device in the multi-layer layout module, set the horizontal distance as and set the intermediate distance as and set the execution height of the monitoring device as : ; Calculate the execution height of the monitoring device through the above formula, determine the height of the first monitoring layout layer, then set the monitoring clarity for the remaining monitoring layout layers, and determine the height positions of the remaining monitoring layout layers; After determining the height positions of the remaining monitoring layout layers in the multi-layer layout module, set the left and right effective monitoring ranges of the monitoring device, calculate the intermediate value of the left and right effective monitoring ranges, and use it as the interval distance between adjacent left and right monitoring devices; Specifically, set clarity boundaries for different monitoring layout layers, analyze the monitoring distance ranges of different monitoring devices within different clarity boundaries, calculate the intermediate distance of the monitoring distance ranges, analyze the distance between the monitoring device and the center point of the monitoring center boundary under the current monitoring angle, then calculate the execution angle of the monitoring device according to the angular distance, monitoring angle and execution distance, and adjust the monitoring calibration according to the execution angle, calculate the intermediate distance, and at the same time regard the intermediate distance as the hypotenuse, measure the distance from the central angle parallel plane to the monitoring device to obtain the horizontal distance, calculate the execution height of the monitoring device according to the intermediate distance and the horizontal distance, determine the height of the first monitoring layout layer, set the monitoring clarity for the remaining monitoring layout layers, determine the height positions of the remaining monitoring layout layers, set the left and right effective monitoring ranges of the monitoring device, calculate the intermediate value of the left and right effective monitoring ranges, and use it as the interval distance between adjacent left and right monitoring devices.

[0018] Embodiment 3: When the preprocessing module extracts the forest image, it analyzes the video frames of the monitoring video, collects the GAN network dehazing enhancement technology, and uses the GAN network dehazing enhancement technology to repair the monitoring video. At this time, ResNet-50 and 3D-CNN are obtained. The preprocessing module uses ResNet-50 to extract spatial features and 3D-CNN to extract motion features, and uploads the dehazed enhanced video, spatial features and motion features to the cloud using the MQTT protocol; The preprocessing module fuses the RGB image and depth map features through the view fusion module of Transformer to eliminate forest occlusion, and at the same time links with the DingTalk robot. The DingTalk robot has the permission to customize the message template according to the fire elements, and the DingTalk robot has the permission to push WebSocket to the command big screen and automatically generate work orders; ResNet-50. As the depth of the neural network increases, problems such as vanishing gradients, difficult training, and degradation will occur. The main path uses multiple convolutional layers for feature extraction and transformation, and the shortcut connection directly adds the input to the output of the main path, enabling the network to learn residuals. This makes it easier to train, alleviates the problem of vanishing gradients, and makes the training of extremely deep networks possible. For image preprocessing, convolutional layers use convolution and pooling to extract initial features. Multiple residual blocks are stacked to gradually increase the feature dimension and the degree of abstraction. Different types of residual blocks are used to reduce the computational amount. The global average pooling layer compresses the feature map to obtain a fixed-length feature vector. The fully connected layer maps the feature vector to the output space for classification or other tasks; 3D-CNN extends the two-dimensional convolutional kernel to three dimensions and performs convolutional operations on three-dimensional data such as videos and volume data to extract features in space and time. Through multiple layers of networks, features are continuously abstracted for tasks such as object recognition and action analysis; MQTT protocol. The client establishes a connection with the server. The client can publish messages to the server to a specified topic and can also subscribe to topics of interest from the server. The server is responsible for receiving, storing, and forwarding messages, and pushing the messages published to a certain topic to the clients subscribed to that topic; ResNet-18 consists of 18 stacked convolutional layers, pooling layers, residual blocks, and fully connected layers. First is a 7×7 convolutional layer with a stride of 2 for extracting the initial features of the image. Then is a max pooling layer for reducing the size of the feature map. Then come the residual blocks, which are the core structure of ResNet-18. Finally are the global average pooling layer and the fully connected layer. The global average pooling layer compresses the feature map into a vector, and the fully connected layer is used for classifying the features or other tasks; The residual blocks of ResNet-18 are composed of 12 densely connected residual blocks; A 16-channel depth feature map is this kind of feature representation with 16 channels, which contains various features extracted by the model from the input data, such as low-level features like edges, textures, and colors, as well as higher-level semantic features like object categories and shapes; The deformable convolutional network, based on traditional convolution, makes the sampling points of the convolutional kernel adaptively changeable by introducing additional offsets, so as to better adapt to the shape and scale changes of the target and more flexibly extract features in the image; In the image dehazing task, the predicted scene depth distribution is used to estimate the scene depth map of the input hazy image by using relevant algorithms or models. The depth map represents the distance information between each point in the scene and the camera. Generally, the farther the distance, the more obvious the influence of fog. According to the atmospheric scattering model, the hazy image is regarded as composed of scene radiation, atmospheric light, and scattered light. Based on the above model and depth information, the hazy image is processed, and the estimated scattered light is subtracted from the hazy image to restore a clear image without fog; The hierarchical depth map represents the depth information of the scene in layers according to certain rules. In image dehazing, regions with different depths are affected by fog differently. The hierarchical depth map can divide the scene into multiple regions with different depth levels, and each level corresponds to a specific depth range; The view fusion module of the Transformer obtains a more comprehensive and accurate feature representation by fusing different view information. Specifically, this module uses the attention mechanism to calculate the correlation between different views and weighted-fuses the important information. For example, when fusing RGB images and depth images, the view fusion module can effectively incorporate the fog-removal-related part of the depth information into the features of the RGB image according to the attention weights, thereby improving the effect of dehazing enhancement; The Yuanbao AI model accepts multi-modal data inputs such as text, speech, images, and videos, supports mixed input methods, and realizes immersive interactions of "conversation as a service". Based on the Tencent Hunyuan large model, it adopts a mixture-of-experts model architecture, which can dynamically call sub-models in different fields, improving the accuracy of professional tasks while ensuring the response speed. Through knowledge distillation and reinforcement learning, using the massive user feedback data in the Tencent ecosystem, such as WeChat conversations and document editing behaviors, it continuously optimizes the model's ability to understand user intentions. It can perform corresponding processing and generation in natural language processing, image recognition, speech recognition, etc. according to the input content and user needs. For example, it can perform operations such as text generation, knowledge answering, image analysis, and speech-to-text, and through a multi-modal creation engine, it can realize functions such as mixed text and image creation and video script generation. At the same time, it can also directly connect to the latest information on the entire network through a search engine to solve the pain point of "knowledge lag" in traditional large models; Specifically, analyze the video frames of the surveillance video, collect the GAN network dehazing enhancement technology, and use the GAN network dehazing enhancement technology to repair the surveillance video. At this time, obtain ResNet-50 and 3D-CNN, and use ResNet-50 and 3D-CNN to extract spatial features and motion features from the surveillance video respectively. Then, use the MQTT protocol to upload the dehazed enhanced video, spatial features, and motion features to the cloud. Use ResNet-18 to extract multi-scale features to obtain a 16-channel depth feature map. Predict the scene depth distribution through a deformable convolutional network to generate a hierarchical depth map. Use the view fusion module of the Transformer to remove the complex geometric structures that block the forest in the surveillance video.

[0019] Working principle: First, collect the information of the surveillance equipment required for the forest area and forest fire prevention, and obtain the spatial effective range of the surveillance equipment information. Construct different surveillance layout layers above the forest. Analyze the central angle parallel plane of the surveillance equipment according to the initial surveillance angle, and extract the position where the angle parallel plane contacts the forest, which is regarded as the surveillance center boundary line, to obtain the upper surveillance plane and the lower surveillance plane. Obtain the forest surveillance range according to the upper surveillance plane and the lower surveillance plane, and extract the contact lines between the upper surveillance plane and the lower surveillance plane and the forest. Arrange the surveillance equipment into different surveillance layout layers in turn according to the surveillance center boundary line, the upper boundary line, and the lower boundary line. The upper boundary line of the (X + 1)-th surveillance equipment coincides with the surveillance center boundary line of the X-th surveillance equipment. Set the clarity boundary for different surveillance layout layers, and analyze the surveillance distance range of different surveillance equipment within different clarity boundaries. Calculate the middle distance of the surveillance distance range, analyze the distance between the surveillance equipment and the center point of the surveillance center boundary line at the current surveillance angle, and then calculate the execution angle of the surveillance equipment according to the angular distance, the surveillance angle, and the execution distance, and adjust the surveillance calibration according to the execution angle. Calculate the middle distance, and at the same time regard the middle distance as the hypotenuse, measure the distance from the central angle parallel plane to the surveillance equipment to obtain the horizontal distance. Calculate the execution height of the surveillance equipment according to the middle distance and the horizontal distance, determine the height of the first surveillance layout layer, set the surveillance clarity for the remaining surveillance layout layers, determine the height positions of the remaining surveillance layout layers, set the left and right effective surveillance ranges of the surveillance equipment, calculate the middle value of the left and right effective surveillance ranges, and use it as the interval distance between the left and right adjacent surveillance equipment. Obtain the surveillance video, analyze the clarity of different surveillance equipment according to the surveillance video, generate an optimization plan, optimize the hierarchical surveillance layout according to the optimization plan, extract the forest image, establish a 4-level pyramid structure, decompose the input image into a 4-level pyramid structure, and the resolutions of the 4-level pyramid structure are successively the original Figure 1 / 2, 1 / 4, 1 / 8, 1 / 16, used to hierarchically process smoke and noise interference of different scales, use adaptive Gamma correction to dynamically adjust the image brightness distribution, obtain the enhanced forest image, use Adobe Premiere Pro to convert the enhanced forest image into an optimized surveillance video, analyze the video frames of the surveillance video, collect the GAN network defogging enhancement technology, and use the GAN network defogging enhancement technology to repair the surveillance video. At this time, ResNet-50 and 3D-CNN are obtained, and ResNet-50 and 3D-CNN are used to extract spatial features and motion features from the surveillance video respectively, and the defogging enhanced video, spatial features and motion features are uploaded to the cloud using the MQTT protocol. ResNet-18 is used to extract multi-scale features to obtain a 16-channel depth feature map, and the scene depth distribution is predicted by the deformable convolutional network to generate a layered depth map , use the Transformer's view fusion module to remove the complex geometric structure of the forest obstruction in the surveillance video, compare the time-lapse image with the time-lapse image after the delay time, and use AI to analyze whether a fire has occurred in the time-lapse image after the delay time. If a fire is detected, an alarm signal is generated and the responsible person is notified. When no fire occurs, no alarm signal will be generated. At the same time, it is linked with the DingTalk robot. The DingTalk robot has the authority to customize message templates according to fire elements. The DingTalk robot has the authority to push WebSocket to the command screen and automatically generate work orders. At this point, the entire workflow ends.

[0020] Although the present invention is disclosed as above in terms of preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art may make possible changes and modifications without departing from the spirit and scope of the present invention. Therefore, any modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention without departing from the content of the technical solution of the present invention shall fall within the scope of protection defined by the claims of the present invention.

Claims

1. A forest fire prevention monitoring system based on multi-modal AI and intelligent collaboration, characterized in that, The forest fire prevention monitoring system includes: An information collection module, which is used to collect the information of the forest range and the monitoring equipment required for forest fire prevention, and obtain the spatial effective range of the monitoring equipment information; A multi-layer layout module, which acquires monitoring equipment, constructs a three-dimensional model of the forest, constructs different monitoring layout layers above the virtual forest in the model, sets the initial monitoring angles of the monitoring equipment, analyzes the central angle parallel planes of the monitoring equipment according to the initial monitoring angles, and extracts the positions where the angle parallel planes contact the forest, regarding them as the monitoring center boundaries, determines the upper and lower effective monitoring angles, obtains the upper monitoring plane and the lower monitoring plane, obtains the forest monitoring range according to the upper monitoring plane and the lower monitoring plane, extracts the contact lines between the upper monitoring plane and the lower monitoring plane and the forest, obtains the upper boundary and the lower boundary, and arranges the monitoring equipment into different monitoring layout layers in sequence according to the monitoring center boundary, the upper boundary and the lower boundary, and the upper boundary of the (X + 1)-th monitoring equipment coincides with the monitoring center boundary of the X-th monitoring equipment, obtaining a hierarchical monitoring layout; A range analysis module, which acquires monitoring videos according to the monitoring equipment in the hierarchical monitoring layout, analyzes the clarity of different monitoring equipment according to the monitoring videos, and obtains the execution clarity; A layout optimization module, which generates an optimization plan according to the execution clarity of the monitoring equipment and the distances between different monitoring equipment and the monitoring center boundary, and optimizes the hierarchical monitoring layout according to the optimization plan; A preprocessing module, which extracts forest images, establishes a 4-level pyramid structure, decomposes the input image into a 4-level pyramid structure, and the resolutions of the 4-level pyramid structure are 1 / 2, 1 / 4, 1 / 8, and 1 / 16 of the original image in sequence, which is used for layered processing of smoke and noise interference at different scales, dynamically adjusts the image brightness distribution by using adaptive Gamma correction, obtains the enhanced forest image, and converts the enhanced forest image into an optimized monitoring video through Adobe Premiere Pro.

2. The forest fire prevention monitoring system based on multi-modal AI and intelligent collaboration according to claim 1, characterized in that, The forest fire prevention monitoring system further includes an anomaly analysis module, which is used to compare the delayed images and generate a fire alarm signal. The anomaly analysis module extracts the optimized monitoring video, establishes a delay comparison function, intercepts the forest video in real time, obtains the delayed images, records the delayed images, sets the delay time, compares the delayed images with the delayed images after the delay time, and uses AI to analyze whether a fire has occurred in the delayed images after the delay time. If a fire is detected, an alarm signal is generated and the person in charge is notified, otherwise no alarm signal is generated.

3. The forest fire prevention monitoring system based on multi-modal AI and intelligent collaboration according to claim 1, characterized in that: When obtaining the hierarchical monitoring layout in the multi-layer layout module, clarity boundaries are set for different monitoring layout layers, and the monitoring distance ranges of different monitoring equipment in different clarity boundaries are analyzed, the middle distance of the monitoring distance range is calculated to obtain the execution distance, the distance between the monitoring equipment and the center point of the monitoring center boundary is analyzed at the current monitoring angle to obtain the angular distance, and then the execution angle of the monitoring equipment is calculated according to the angular distance, the monitoring angle and the execution distance, and the monitoring calibration is adjusted according to the execution angle.

4. The forest fire prevention monitoring system based on multi-modal AI and intelligent collaboration according to claim 3, characterized in that: When calculating the intermediate distance of the multi-layer layout module, assume that the monitoring distance range is , and assume that the intermediate distance is : ； Calculate the intermediate distance through the above formula. At the same time, regard the intermediate distance as the hypotenuse, measure the distance from the center angle parallel plane to the monitoring device to obtain the horizontal distance, and calculate the execution height of the monitoring device according to the intermediate distance and the horizontal distance.

5. The forest fire prevention monitoring system based on multi-modal AI and intelligent collaboration according to claim 4, characterized in that: When calculating the execution height of the monitoring device in the multi-layer layout module, let the horizontal distance be , let the middle distance be , and let the execution height of the monitoring device be : ； Calculate the execution height of the monitoring device through the above formula, determine the height of the first monitoring layout layer, and then set the monitoring clarity for the remaining monitoring layout layers to determine the height positions of the remaining monitoring layout layers.

6. The forest fire prevention monitoring system based on multi-modal AI and intelligent collaboration according to claim 5, characterized in that: After determining the height positions of the remaining monitoring layout layers in the multi-layer layout module, set the left and right effective monitoring ranges of the monitoring device, calculate the intermediate value of the left and right effective monitoring ranges, and use it as the interval distance between adjacent left and right monitoring devices.

7. The forest fire prevention monitoring system based on multi-modal AI and intelligent collaboration according to claim 1, characterized in that: When the preprocessing module extracts the forest image, it analyzes the video frames of the monitoring video, collects the GAN network dehazing enhancement technology, and uses the GAN network dehazing enhancement technology to repair the monitoring video. At this time, ResNet-50 and 3D-CNN are obtained. The preprocessing module uses ResNet-50 to extract spatial features, 3D-CNN to extract motion features, and uploads the dehazing enhanced video, spatial features, and motion features to the cloud using the MQTT protocol.

8. The forest fire prevention monitoring system based on multi-modal AI and intelligent collaboration according to claim 7, characterized in that: The preprocessing module fuses the features of the RGB image and the depth map through the view fusion module of Transformer to eliminate forest occlusion. At the same time, it is linked with the DingTalk robot. The DingTalk robot has the permission to customize the message template according to the fire elements, and the DingTalk robot has the permission to push WebSocket to the command large screen and automatically generate work orders.

9. A forest fire prevention monitoring method based on multimodal AI and intelligent collaboration, characterized in that: This method is applicable to the forest fire prevention monitoring system based on multi-modal AI and intelligent collaboration described in any one of claims 1-8.

Images