Forest fire monitoring method and device, computer equipment and storage medium

Abstract

The invention relates to the field of fire monitoring, in particular to a forest fire monitoring method and device, computer equipment and a storage medium. The forest fire monitoring method comprises the following steps: acquiring a forest fire monitoring image of a target monitoring place; extracting target forest fire image features of the forest fire monitoring image based on an image feature extraction model; obtaining a similarity weight matrix based on the target forest fire image features and the similarity of each cache key in a preset key value library; wherein the key value library comprises a cache key and a cache value, the cache key is an image feature vector of a forest fire image sample of a sample monitoring place, the cache value represents a smoke and fire type label of the forest fire image sample, and the cache key and the cache value support dynamic updating; performing weighted fusion on the cache values based on a similarity weight matrix to obtain first smoke and fire type prediction distribution; and determining a target identification result based on the first smoke and fire type prediction distribution. According to the invention, new forest fire monitoring requirements can be quickly met, and the accuracy of forest fire monitoring is guaranteed.

Description

Technical Field

[0001] This application relates to the field of fire monitoring, specifically to a forest fire monitoring method, device, computer equipment, and storage medium. Background Technology

[0002] Forest fires are a severe ecological disaster worldwide, causing immense damage to forest resources, biodiversity, ecosystem balance, and human life and property. To mitigate the harm caused by fires, forest fire monitoring technologies have emerged and are continuously evolving.

[0003] In forest fire monitoring technology, deep learning-based image recognition methods have become an important technical tool. Compared with traditional thresholding or spectral analysis methods, deep learning models can more effectively uncover the intrinsic connections between spatial and temporal information, significantly improving the accuracy and robustness of fire identification.

[0004] However, model training typically relies on specific datasets, while the distribution of data in real-world scenarios may differ significantly from the training data. For traditional deep learning models to recognize new scenarios, they need to collect a large number of samples of that type of scenario and perform professional annotations, followed by full fine-tuning or retraining of the model. This makes continuous optimization difficult to achieve in order to quickly adapt to new monitoring needs. Summary of the Invention

[0005] In view of the above, embodiments of this application provide a forest fire monitoring method, apparatus, computer equipment, and computer-readable storage medium, which facilitates continuous optimization of the forest fire monitoring process, enabling rapid adaptation to constantly changing environmental conditions, meeting new forest fire monitoring needs, and ensuring the accuracy of forest fire monitoring.

[0006] In a first aspect, embodiments of this application provide a forest fire monitoring method, which includes: Acquire forest fire monitoring images of the target monitoring area; Image features of the forest fire monitoring images are extracted based on a pre-trained image feature extraction model to obtain the target forest fire image features; Based on the similarity between the target forest fire image features and each cache key in the preset key value library, a similarity weight matrix is ​​obtained; wherein, the key value library also includes cache values ​​corresponding to the cache keys, the cache keys are the image feature vectors of forest fire image samples from the sample monitoring location, and the cache values ​​represent the smoke type label of the forest fire image samples; the cache keys and cache values ​​in the key value library support dynamic updates. The cached values ​​are weighted and fused based on the similarity weight matrix to obtain the predicted distribution of the first type of fireworks; The target identification result is determined based on the first predicted distribution of the firework type, and the target identification result is used to represent the firework type of the target monitoring location.

[0007] Secondly, embodiments of this application also provide a forest fire monitoring device, comprising: The image acquisition module is used to acquire forest fire monitoring images of the target monitoring area; The feature extraction module is used to extract image features from the forest fire monitoring images based on a pre-trained image feature extraction model to obtain the target forest fire image features; The similarity calculation module is used to obtain a similarity weight matrix based on the similarity between the target forest fire image features and each cache key in the preset key value library; wherein, the key value library also includes cache values ​​corresponding to the cache keys, the cache keys are image feature vectors of forest fire image samples from the sample monitoring location, and the cache values ​​represent the smoke type label of the forest fire image samples; The fireworks prediction module is used to perform weighted fusion of the cached values ​​based on the similarity weight matrix to obtain the first fireworks type prediction distribution; The smoke and fire identification module is used to determine the target identification result based on the predicted distribution of the first smoke and fire type, and the target identification result is used to represent the smoke and fire type of the target monitoring location.

[0008] Thirdly, embodiments of this application also provide a computer device, the computer device including a processor and a memory, the memory being used to store instructions, and the processor being used to call the instructions in the memory, causing the computer device to execute the forest fire monitoring method described in the first aspect.

[0009] Fourthly, a computer-readable storage medium stores computer instructions that, when executed on a computer device, cause the computer device to perform the forest fire monitoring method described in the first aspect.

[0010] The forest fire monitoring method in this application adopts a key-value library. The key-value library stores the image feature vectors (i.e., cache keys) of forest fire image samples in the sample monitoring area and cache values ​​representing smoke type labels. Therefore, in the case of newly added smoke recognition scenarios, it is not necessary to perform full fine-tuning or retraining of the pre-trained image feature extraction model. Instead, the key-value library is updated, and the feature vectors and corresponding labels of forest fire image samples in the newly added smoke scenarios are added to the key-value library. By calculating the similarity between the target forest fire image features and the cache keys and constructing a weight matrix, the cache values ​​are weighted and fused to obtain the predicted distribution of the first smoke type, thereby determining the target recognition result. This helps to reduce the resource investment required for large-scale sample annotation and model retraining during the adaptation process of new scenarios, reduce the cost of incremental sample integration and shorten the adaptation cycle. It enables the model to quickly respond to new monitoring needs brought about by changes in forest environment, vegetation type, and climate conditions, effectively avoiding the recognition blind spots, false positives and false negatives caused by traditional models not covering new scenarios or data distribution differences. While quickly adapting to new scenarios, it continuously ensures the accuracy and robustness of forest fire monitoring. Attached Figure Description

[0011] Figure 1 This is a schematic diagram of a forest fire monitoring system provided according to an embodiment of this application.

[0012] Figure 2 This is a flowchart of the steps of a forest fire monitoring method according to an embodiment of this application.

[0013] Figure 3 This is a schematic diagram of the model structure of an image feature extraction model provided according to an embodiment of this application.

[0014] Figure 4 This is a schematic diagram of the model structure of a semantic feature extraction model provided according to an embodiment of this application.

[0015] Figure 5 This is a schematic block diagram of the structure of a forest fire monitoring device provided according to an embodiment of this application.

[0016] Figure 6 This is a schematic diagram of the structure of a computer device provided according to an embodiment of this application. Detailed Implementation

[0017] To better understand the above-mentioned objectives, features, and advantages of this application, the application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0018] The following description sets forth many specific details to provide a full understanding of this application. The described embodiments are only some, not all, of the embodiments of this application.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein in the specification of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application.

[0020] It should be further noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0021] In this application, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and drawings of this application are used to distinguish similar objects, not to describe a specific order or sequence.

[0022] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0023] For ease of understanding, some concepts related to the embodiments of this application are illustrated and explained by way of example for reference.

[0024] 1. Caching Mechanism: A cache mechanism is an efficient information storage and retrieval technique that enables fast querying and retrieval by pre-storing key data in an easily accessible cache. In visual and language models, the cache mechanism uses visual features from a few-shot training set as "keys" and corresponding labels as "values" to construct a key-value pair database. During the inference phase, the features of the test image are used as queries. By calculating the similarity with the keys in the cache, the most relevant values ​​(i.e., label information) are quickly retrieved, thus achieving efficient integration of new knowledge. This mechanism not only avoids complex training processes but also significantly improves the efficiency and performance of the model, especially suitable for few-shot learning scenarios, enabling rapid adaptation to new tasks and reducing computational resource consumption.

[0025] 2. Patch Embedding Sublayer: Patch embedding sublayers are commonly used in image processing tasks, especially in the Vision Transformer architecture. The basic idea is to segment the input image into several fixed-size "patches" and map these local image patches into one-dimensional vector representations through linear transformations or convolutions, thereby converting two-dimensional pixel information into a one-dimensional sequence representation that can be processed by the Transformer. Through this segmentation and embedding method, the model can capture the local structural features of the image, providing richer contextual information for subsequent attention calculations.

[0026] 3. Position Embedding Sublayer: The Position Embedding Sublayer is a crucial component in the Transformer architecture used to represent the positional information of sequence elements. Since self-attention does not explicitly capture positional information when capturing dependencies within the input sequence, the Position Embedding Sublayer injects positional encoding into the sequence representation by adding sine or cosine functions or learnable parameters, enabling the model to distinguish elements at different positions within the sequence. This is essential for modeling natural language processing, image block sequences, and other sequence data.

[0027] 4. Multi-Head Self-Attention Layer: The multi-head self-attention layer is a core module of the Transformer architecture. By executing multiple self-attention mechanisms in parallel, the model can learn the dependencies between elements in a sequence in different subspaces. Each attention head maps the input sequence to queries, keys, and values, then calculates an attention score, and finally combines the outputs of multiple heads. In fields such as natural language processing and computer vision, multi-head self-attention layers help models flexibly and efficiently capture global dependencies and contextual information.

[0028] 5. Layer Normalization: Layer normalization plays a crucial role in stabilizing the training process and accelerating convergence in deep neural networks. By normalizing the activation values ​​in a layer to have zero mean and unit variance, it effectively prevents problems such as gradient explosion and gradient vanishing. In self-attention mechanisms and feedforward networks, placing the normalization layer after the input or residual connections further enhances the model's stability and generalization performance.

[0029] 6. Feed-Forward Neural Network (FFN): The basic structure of a feedforward neural network consists of two linear transformation layers with an activation function in between, used for nonlinear mapping and feature extraction of the representation at each position in the sequence. Through the processes of dimensionality upscaling and downscaling, feedforward neural networks can deeply combine and abstract local information, providing richer semantic representations for the subsequent attention calculation.

[0030] 7. Residual Connection: Residual connections, originally proposed by deep residual networks (ResNet), were designed to mitigate the vanishing gradient and model degradation problems associated with increasing network depth. In the Transformer, residual connections directly add the input to each sublayer (e.g., multi-head self-attention, MLP) to its output, creating a "skip" path. This approach enables identity transfer of information, avoids gradient decay caused by multiple layers stacking, and promotes more efficient feature learning and training stability.

[0031] In forest fire monitoring, deep learning-based image recognition methods have become an important technological tool. The core of this approach lies in acquiring multi-source data, such as video streams and infrared images, and using convolutional neural network (CNN) models to extract the color, texture, and motion features of smoke and flames. By analyzing the activation values ​​output by the network, the system can determine the presence of abnormal heat sources or smoke, thus achieving real-time fire monitoring. Compared to traditional thresholding or spectral analysis methods, deep learning models can more effectively uncover the intrinsic connections between spatial and temporal information, significantly improving the accuracy and robustness of fire identification. However, despite the excellent performance of deep learning models in laboratory environments, they still face many challenges in practical deployment.

[0032] However, model training typically relies on specific datasets, while the distribution of data in real-world scenarios can differ significantly from the training data. For example, during actual monitoring, forest environments, vegetation types, and climate conditions change over time, and new environmental conditions may constantly emerge. This discrepancy can lead to false positives or false negatives in practical applications, especially in complex natural environments such as forest fire monitoring scenarios.

[0033] For traditional deep learning models to recognize new scenarios, they need to collect a large number of samples of such scenarios and perform professional annotations, followed by full fine-tuning or retraining of the model. However, updating and optimizing the model requires significant computational resources and time, which is often difficult to implement in practice. Furthermore, the large number of parameters and deep layers in the model further increases the difficulty of optimization. Therefore, improving the ease of optimization for forest fire monitoring processes to adapt to constantly changing environmental conditions is an urgent problem to be solved.

[0034] In view of the above, embodiments of this application provide a forest fire monitoring method, apparatus, computer equipment, and computer-readable storage medium.

[0035] Please refer to Figure 1 , Figure 1 This is a schematic diagram illustrating the implementation environment of a forest fire monitoring method provided in an embodiment of this application.

[0036] Please refer to this document. Figure 1 , Figure 1 This is a schematic diagram of a forest fire monitoring system provided in an embodiment of this application. The forest fire monitoring system may include a computer device 100, which integrates a forest fire monitoring device, such as... Figure 1 Computer equipment in the country.

[0037] In this embodiment, the computer device 100 is mainly used to acquire forest fire monitoring images of the target monitoring site; extract image features of the forest fire monitoring images based on a pre-trained image feature extraction model to obtain target forest fire image features; obtain a similarity weight matrix based on the similarity between the target forest fire image features and each cache key in a preset key-value library; wherein, the key-value library also includes cache values ​​corresponding to the cache keys, the cache keys are image feature vectors of forest fire image samples of the sample monitoring site, and the cache values ​​represent the smoke type label of the forest fire image samples; the cache keys and cache values ​​in the key-value library support dynamic updates; perform weighted fusion of the cache values ​​based on the similarity weight matrix to obtain a first smoke type prediction distribution; determine the target identification result based on the first smoke type prediction distribution, and the target identification result is used to represent the smoke type of the target monitoring site.

[0038] In this embodiment, the computer device 100 can be a standalone server, a server network, or a server cluster. For example, the computer device 100 described in this embodiment includes, but is not limited to, a computer, a network host, a single network server, a set of multiple network servers, or a cloud server composed of multiple servers. The cloud server is composed of a large number of computers or network servers based on cloud computing.

[0039] The forest fire monitoring system may also include a mobile terminal 200, which can communicate with the computer device 100.

[0040] It is understood that the terminal 100 used in this application embodiment can be a device that includes both receiving and transmitting hardware, that is, a device having receiving and transmitting hardware capable of performing bidirectional communication on a bidirectional communication link. Such a device may include: cellular or other communication devices having a single-line display, a multi-line display, or a cellular or other communication device without a multi-line display. Specifically, the terminal 100 may be a desktop terminal or a mobile terminal, and the terminal 100 may also be one of a mobile phone, tablet computer, laptop computer, etc.

[0041] The terminal 100 can receive target recognition results from the server 100 and display them.

[0042] Those skilled in the art will understand that Figure 1 The application environment shown is merely one application scenario of the solution in this application and does not constitute a limitation on the application scenario of the solution in this application. Other application environments may include those that are more specific to this application. Figure 1 The number of computer devices shown is more or less, for example Figure 1Only one computer device is shown in the diagram. It is understood that the forest fire monitoring system may also include one or more other services, which are not specified here.

[0043] In addition, the forest fire monitoring system may also include image acquisition equipment for acquiring images of forest fires. This image acquisition equipment can be a high-definition network camera deployed at the target monitoring site, a drone equipped with a gimbal camera, or a satellite remote sensing system, but is not limited to these.

[0044] refer to Figure 2 As shown, Figure 2 This is a flowchart illustrating the steps of a forest fire monitoring method according to an embodiment of this application. The order of the steps in this flowchart can be changed, and some steps can be omitted, depending on different needs. This forest fire monitoring and early warning method can be applied to... Figure 1 The computer device 100 shown is not limited to this.

[0045] This forest fire monitoring method may include the following steps: Step 201: Obtain forest fire monitoring images of the target monitoring area.

[0046] For example, computer device 100 can acquire forest fire monitoring images from image acquisition equipment deployed at the target monitoring site. These forest fire monitoring images may be image frames from a forest fire monitoring video stream or infrared images, but are not limited to these, and this application embodiment does not limit them.

[0047] Step 202: Extract image features from forest fire monitoring images based on a pre-trained image feature extraction model to obtain target forest fire image features.

[0048] The target forest fire image features are the image feature vectors of the forest fire monitoring images, which are vectorized representations of the forest fire monitoring images.

[0049] The image feature extraction model can be a convolutional neural network (CNN) model, such as the ResNet series models. Convolution operations can effectively extract local texture features of forest fire images, such as flame edges and smoke density, which are highly matched with key visual cues for forest fire identification.

[0050] This image feature extraction model can also be a Transformer-type model, such as the Vision Transformer (ViT) model. The VIT model can capture the global semantic associations of forest fire images, such as the pattern of smoke spread and the spatial relationship with vegetation, through block embedding and multi-head attention mechanisms. Moreover, this model supports pre-trained weight transfer, eliminating the need to train from scratch and significantly shortening the development cycle.

[0051] For example, refer to Figure 3As shown, the image feature extraction model can include a linear layer, a second projection layer, and a Transformer stack.

[0052] The second projection layer may include a block embedding layer and a position encoding layer.

[0053] The Transformer stack includes at least one Transformer layer, which may include a second multi-head attention layer, a third normalization layer, a second feedforward neural network, and a fourth normalization layer.

[0054] When the Transformer stack includes multiple Transformer layers, each Transformer layer receives the image features of the previous layer as input and outputs the sequence representation of the next layer.

[0055] To illustrate the computation process of the Transformer layer, the following will use the first... Taking the multi-head self-attention mechanism in the Transformer layer as an example, the corresponding formula is given.

[0056] For the l-th Transformer layer, assume the input Transformer layer sequence is as follows (1): Then, the query (Q), key (K), and value (V) are generated as shown in the following formula (2): ; in, These are the projection matrices.

[0057] Then, the output of self-attention is calculated. As shown in the following formula (3): ; in, Indicates the vector dimension (or scaling factor).

[0058] The attention output is then residually connected to the input sequence and layered, as shown in the following formula (4): ; Finally, By feeding the data into a feed-forward network (FFN) and performing residual connections and normalization again, the first result can be obtained. The final output of the Transformer layer is shown in the following formula (5): .

[0059] The following is combined Figure 3 The image feature extraction model shown illustrates the implementation of step 202. Specifically, step 202 may include the following sub-steps: Step a1: The computer device inputs the forest fire monitoring image into the linear layer to obtain the first image processing intermediate features.

[0060] Step a2: Based on the second projection layer, the first image processing intermediate features are block-embedded and position-encoded to obtain the second image processing intermediate features.

[0061] Step a3: After obtaining the intermediate features of the second image, the second image processing features can be input into the Transformer layer, that is, steps a4 to a6 are executed below.

[0062] Step a4: Input the second image processing intermediate feature into the second multi-head attention layer to obtain the third image processing intermediate feature; perform residual connection between the third image processing intermediate feature and the second image processing intermediate feature to obtain the fourth image processing intermediate feature; perform normalization processing on the fourth image processing intermediate feature based on the third normalization layer to obtain the fifth image processing intermediate feature.

[0063] Step a5: Input the fifth image processing intermediate feature into the second feedforward neural network to obtain the sixth image processing intermediate feature; perform residual connection between the sixth image processing intermediate feature and the fifth image processing intermediate feature to obtain the seventh image processing intermediate feature; perform normalization processing on the seventh image processing intermediate feature based on the fourth normalization layer to obtain the eighth image processing intermediate feature.

[0064] Step a6: Determine the target forest fire image features based on the eighth intermediate image processing feature. For example, when the image feature extraction model includes multiple transformer layers, the eighth intermediate image processing feature can be input into the next transformer layer until the last transformer layer is reached. When the image feature extraction model includes one transformer layer, the eighth intermediate image processing feature can be used as the target forest fire image feature.

[0065] The above-described model structure and processing steps for image feature extraction are merely examples. In practical applications, the image feature extraction model can be set up according to specific requirements.

[0066] Step 203: Based on the similarity between the target forest fire image features and the cached keys in the preset key value library, obtain the similarity weight matrix.

[0067] The key-value library also includes the cache key. Corresponding cache value Key-value libraries can employ a caching mechanism.

[0068] The cache key is the image feature vector of the forest fire image sample from the monitored area. For example, the forest fire image sample can be input into the image feature extraction model described above to obtain the image feature vector of the forest fire image sample, and the image feature vector of the forest fire image sample can be used as the cache key.

[0069] The cached value represents the smoke type label of the forest fire image sample. The smoke type label can represent wildfire and non-wildfire. The wildfire label can further include "burning stubble and miscellaneous materials", and the non-wildfire label can be represented as "cooking smoke", "industrial emissions", etc. This smoke type label is only an example. In actual application, it can be set according to the forest fire classification needs of forest fire monitoring. This application embodiment does not limit it in this way.

[0070] This application embodiment can convert fireworks type labels represented in text form into one-hot codes, and use these one-hot codes as cached values. For example, the fireworks type labels represented in text form can be input into a pre-trained semantic feature extraction model to obtain the one-hot codes.

[0071] The key-value library can be built in the following ways: Obtain a sample set, which includes multiple forest fire monitoring samples. Each forest fire monitoring sample includes a forest fire image sample and a smoke type label for the forest fire image sample. Split the sample set into a first dataset for building a key-value library and a second dataset for model evaluation. Then, input the forest fire monitoring samples from the first dataset into a pre-trained image feature extraction model to obtain cache keys, and input the smoke type labels of the forest fire monitoring samples into a pre-trained semantic feature extraction model to obtain cache values. Construct cache key-value pairs in this way, and then build a key-value library based on the cache key-value pairs built on the first dataset.

[0072] In some embodiments, cached keys and cached values ​​in the key-value store can be dynamically updated.

[0073] For example, when a new forest fire identification scenario needs to be added, the forest fire monitoring samples of the new scenario can be uploaded to a computer device. The computer device obtains the new forest fire monitoring samples, which include new forest fire image samples and smoke type labels of the new forest fire image samples. Based on the image feature extraction model, the image features of the new forest fire image samples are extracted to obtain a new cache key. Based on the smoke type labels of the new forest fire image samples, a new cache value is determined. Based on the new cache key and the new cache value, a new key-value pair is created and added to the key-value library.

[0074] The embodiments of this application can dynamically expand the key value library, continuously enrich the sample features and label reserves, so that the forest fire monitoring method can be continuously adapted to new types of smoke and fire scenarios for optimization, improve the ability to identify rare or newly added types of smoke and fire, and extend the life cycle and scope of application of the forest fire monitoring method.

[0075] In some embodiments, step 203 can be implemented as follows: the computer device 100 can obtain a similarity weight matrix using the target weight calculation formula and the similarity between the target forest fire image features and each cached key in a preset key value library, wherein the target weight calculation formula is as follows: ; Among them, the The similarity weight matrix is ​​represented by exp(); exp() represents the exponential function; the The target similarity sharpness adjustment factor; The target forest fire image features; The transpose matrix formed by the cached keys in the key-value library, This indicates the similarity between the features of the target forest fire image and each cached key in the preset key-value library.

[0076] As can be seen from the above, when the key value library is updated, the similarity weight matrix will be updated accordingly, thereby affecting the final target recognition result. Therefore, the embodiments of this application can continuously optimize the monitoring process of forest fires through the updating of the key value library.

[0077] Step 204: Based on the similarity weight matrix, the cached values ​​in the key value library are weighted and fused to obtain the predicted distribution of the first fireworks type.

[0078] For example, computer device 100 can obtain the predicted distribution of the first fireworks type using the following formula. : ; in, This represents the similarity weight matrix; This represents the cached values ​​in the key-value store.

[0079] Step 205: Determine the target identification result based on the predicted distribution of the first firework type. The target identification result is used to represent the firework type of the target monitoring location.

[0080] In some embodiments, the computer device may select the firework type with the highest probability from the first firework type prediction distribution as the target recognition result.

[0081] In other embodiments, step 205 can also be implemented as follows: Step b1: The computer device obtains a set of answers, which includes candidate text labels for describing various types of fireworks.

[0082] The target recognition result in this embodiment is a candidate text label in the answer set. The candidate text labels in the answer set may include, but are not limited to, "burning stubble and miscellaneous materials," "cooking smoke," and "industrial emissions."

[0083] Step b2: Extract the semantic features of the candidate text tags based on the pre-trained semantic feature extraction model to obtain the tag semantic features.

[0084] In some embodiments, the semantic feature extraction model may be a large language model, Word2Vec model, GloVe model, ELMo model, BERT model, etc., but is not limited to these.

[0085] In other embodiments, reference is made to... Figure 4 As shown, the semantic feature extraction model includes a first projection layer and a Transformer stack.

[0086] The first projection layer may include a text embedding layer and a location encoding layer.

[0087] The Transformer stack includes at least one Transformer layer, which may include a first multi-head attention layer, a first normalization layer, a first feedforward neural network, and a second normalization layer.

[0088] When the Transformer stack includes multiple Transformer layers, each Transformer layer receives the text vector features of the previous layer as input and outputs the sequence representation of the next layer.

[0089] The calculation process of this Transformer layer is roughly the same as that of the Transformer layer in the image feature extraction model, and will not be described in detail here.

[0090] The following is combined Figure 4 The implementation method of step b2 is described, and step b2 further includes the following sub-steps b2.1 to b2.4.

[0091] In sub-step b2.1, the computer device performs text embedding and position encoding on the candidate text tags based on the first projection layer to obtain the first text processing intermediate features.

[0092] Sub-step b2.2: Input the first text processing intermediate feature into the first multi-head attention layer to obtain the second text processing intermediate feature; perform residual connection between the second text processing intermediate feature and the first text processing intermediate feature to obtain the third text processing intermediate feature; perform normalization processing on the third text processing intermediate feature based on the first normalization layer to obtain the fourth text processing intermediate feature.

[0093] Sub-step b2.3: ​​Input the fourth text processing intermediate feature into the first feedforward neural network to obtain the fifth text processing intermediate feature; perform residual connection between the fifth text processing intermediate feature and the fourth text processing intermediate feature to obtain the sixth text processing intermediate feature; perform normalization processing on the sixth text processing intermediate feature based on the second normalization layer to obtain the seventh text processing intermediate feature.

[0094] Sub-step b2.4: Determine the label semantic features based on the seventh text processing intermediate features. For example, when the semantic feature extraction model includes multiple transformer layers, the seventh text processing intermediate features can be input into the next transformer layer until the last transformer layer is reached. When the image feature extraction model includes one transformer layer, the seventh text processing intermediate features can be used as the label semantic features.

[0095] The semantic features of the tags extracted using the above technical solution can capture the deep semantic information of candidate text tags, providing accurate data support for similarity calculation in step b3, thereby improving the accuracy of the predicted distribution of the second type of fireworks.

[0096] Step b3: Based on the similarity between the target forest fire image features and the tag semantic features, obtain the predicted distribution of the second fire type.

[0097] The second firework type prediction distribution refers to the probability distribution of various firework types calculated based on the similarity between the target forest fire image features and the tag semantic features.

[0098] For example, the cosine similarity between the target forest fire image features and the label semantic features can be calculated. Then, the label semantic features can be normalized to obtain the predicted distribution of the second type of fire.

[0099] Step b4: Perform residual fusion of the first firework type prediction distribution and the second firework type prediction distribution to obtain the target recognition result.

[0100] In some embodiments, the first fireworks type prediction distribution and the second fireworks type prediction distribution are residually fused using a target residual fusion formula to obtain the target fireworks type prediction distribution. The target residual fusion formula is as follows: ; Among them, the This represents the predicted distribution of the target fireworks type; The target is to balance the weights; This represents the predicted distribution of the first type of fireworks; the This represents the predicted distribution of the second type of fireworks.

[0101] This application embodiment performs residual fusion between the first fireworks type prediction distribution and the second fireworks type prediction distribution, fully combining the dual advantages of image feature matching and text semantic association, effectively improving the accuracy and robustness of fireworks type recognition, and reducing the risk of misjudgment from single feature matching.

[0102] Furthermore, the embodiments of this application can balance the contribution ratios of the first firework type prediction distribution and the second firework type prediction distribution by using target balancing weights, thereby further improving the accuracy of target recognition results.

[0103] The above embodiments employ target residual fusion formulas and target weight calculation formulas for corresponding calculations. The target similarity sharpness adjustment factor in the target weight calculation formula and the target balance weight in the target residual fusion formula can be set according to actual application needs, for example, based on experience, or obtained through the following methods: Step c1: Obtain parameter update image samples and the fireworks type labels of the parameter update image samples.

[0104] The parameter-updated image sample is a forest fire image sample used to update the target similarity sharpness adjustment factor and the target balance weight. This parameter-updated image sample and its smoke type label can be obtained from a cache library, or from other databases or devices; this application embodiment does not limit this.

[0105] Step c2: By using the initial weight calculation formula and the parameters to update the image feature vector of the image sample and the similarity between each cached key in the preset key value library, a similarity weight matrix sample is obtained. The initial weight calculation formula is as follows: ; Among them, the The similarity weight matrix represents the samples; exp() represents the exponential function; the This is the initial similarity sharpness adjustment factor; the... Update the image feature vector of the image sample for the parameters; The transpose matrix formed by the cached keys in the preset key-value library, the This represents the similarity between the image feature vector of the updated image sample and each cached key in the key-value library.

[0106] When step c2 is executed for the first time, the initial similarity sharpness adjustment factor can be a preset value or a randomly generated value. When step c2 is executed in subsequent iterations, the current initial similarity sharpness adjustment factor is the initial similarity sharpness adjustment factor updated in the previous iteration.

[0107] Step c3: Based on the similarity weight matrix sample, perform weighted fusion of each cached value in the key value library to obtain the first fireworks type prediction distribution sample.

[0108] Step c4: Based on the similarity between the cache key and the semantic features of the tag, obtain the second fireworks type prediction distribution sample.

[0109] Step c5: The first fireworks type prediction distribution sample and the second fireworks type prediction distribution sample are residually fused using an initial residual fusion formula to obtain the target fireworks type prediction distribution sample. The initial residual formula is as follows: ; Among them, the This represents the predicted distribution sample of the target fireworks type; The initial balance weights; This represents the predicted distribution sample of the first type of fireworks; the This represents the predicted distribution sample for the second type of fireworks.

[0110] When step c5 is executed for the first time, the initial balance weight can be a pre-set value or a randomly generated value. When step c2 is executed in the subsequent iteration, the current initial balance weight is the initial balance weight updated in the previous iteration.

[0111] Step c6: Determine the identification result sample based on the predicted distribution sample of the target fireworks type; Step c7: Based on the recognition result sample and the parameter update image sample, update the initial similarity sharpness adjustment factor and the initial balance weight to obtain the target similarity sharpness adjustment factor and the target balance weight.

[0112] For example, based on a preset loss function, the loss value of the fireworks type label of the recognition result sample and the parameter update image sample is calculated; based on the loss value, gradient descent is performed on the initial similarity sharpness adjustment factor and the initial balance weight to update the initial similarity sharpness adjustment factor and the initial balance weight. Then, step b1 is executed again until the preset training completion condition is reached, and the target similarity sharpness adjustment factor in the target weight calculation formula and the target balance weight in the target residual fusion formula are obtained.

[0113] The training completion condition can be the convergence of the loss function, or the loss value being lower than a preset threshold, or the training reaching a preset number of rounds. This application embodiment does not limit this to any of these conditions.

[0114] The loss function can be set according to the actual application requirements, and this application embodiment does not limit it. For example, the loss function can be the cross-entropy loss function, etc.

[0115] This application's embodiments update image samples and corresponding labels through parameter updates, generate recognition result samples based on the initial weight formula and the initial residual fusion formula, and then update the initial similarity and sharpness adjustment factor according to the difference between the recognition result samples and the true labels. and initial balance weights To obtain target parameters that are adapted to the actual scenario. and This makes the weight calculation and distribution fusion more closely match the characteristics of the monitoring data, and significantly improves the generalization ability and recognition stability of the method in different monitoring scenarios.

[0116] In one embodiment, after generating the target similarity sharpness adjustment factor and the target balance weight, it is also necessary to verify the accuracy and reliability of the target prediction model. The target prediction model includes the aforementioned pre-trained image feature extraction model, semantic feature extraction model, target weight calculation formula, and target residual fusion formula, etc.

[0117] Specifically, this step may include: First, obtain the target identification results (hereinafter referred to as prediction results) predicted by the target prediction model for the forest fire image samples in the second dataset. This step is roughly the same as steps 201 to 205 above, except that steps 201 to 205 predict target identification results for forest fire monitoring images of the target monitoring area, while this step predicts target identification results for forest fire image samples in the second dataset.

[0118] Then, the evaluation results are determined based on the prediction results. The evaluation results include: consistent results and inconsistent results.

[0119] Consistent results indicate that the smoke type label and prediction result are the same for the forest fire image sample in the second dataset, while inconsistent results indicate that the smoke type label and prediction result are different for the forest fire image sample in the second dataset.

[0120] Next, based on the evaluation results, the reliability parameters of the target prediction model are calculated. The reliability parameters include at least accuracy, recall, and F1 score. Based on the reliability parameters, the accuracy of the target prediction model is evaluated.

[0121] This embodiment uses an evaluation metric based on the confusion matrix to calculate accuracy, recall, and F1 score.

[0122] The accuracy rate is calculated using the following formula (8): Accuracy=(TP+TN) / (TP+FN+FP+TN); The recall rate is calculated using the following formula (9): Recall = TP / (TP + FN); The formula for calculating the F1 score is as follows (10): F1score=(2*Precision*Recall) / (Precision+Recall); Where Precision = TP / (TP+FP).

[0123] TP represents the number of samples with positive labels and positive model predictions. TN represents the number of samples with negative labels and negative model predictions. FP represents the number of samples with negative labels but positive model predictions. FN represents the number of samples with positive labels but negative model predictions.

[0124] A positive label indicates wildfire-type smoke that requires an emergency warning; a negative label indicates non-wildfire-type smoke that does not require a warning, such as residential fires, but is not limited to these.

[0125] In a test of a real-world application scenario, the target prediction model obtained through the above method achieved an accuracy of 0.936, a recall of 0.929, and an F1 score of 0.932, indicating that the embodiments of this application can extract smoke and fire image features relatively accurately, thereby completing the monitoring of forest fires.

[0126] The forest fire monitoring method in this application adopts a key-value library. The key-value library stores the image feature vectors (i.e., cache keys) of forest fire image samples in the sample monitoring area and cache values ​​representing smoke type labels. Therefore, in the case of newly added smoke recognition scenarios, it is not necessary to perform full fine-tuning or retraining of the pre-trained image feature extraction model. Instead, the key-value library is updated, and the feature vectors and corresponding labels of forest fire image samples in the newly added smoke scenarios are added to the key-value library. By calculating the similarity between the target forest fire image features and the cache keys and constructing a weight matrix, the cache values ​​are weighted and fused to obtain the predicted distribution of the first smoke type, thereby determining the target recognition result. This helps to reduce the resource investment required for large-scale sample annotation and model retraining during the adaptation process of new scenarios, reduce the cost of incremental sample integration and shorten the adaptation cycle. It enables the model to quickly respond to new monitoring needs brought about by changes in forest environment, vegetation type, and climate conditions, effectively avoiding the recognition blind spots, false positives and false negatives caused by traditional models not covering new scenarios or data distribution differences. While quickly adapting to new scenarios, it continuously ensures the accuracy and robustness of forest fire monitoring.

[0127] Based on the same idea as the forest fire monitoring method in the above embodiments, this application also provides a forest fire monitoring device, which can be used to execute the above forest fire monitoring method. For ease of explanation, the structural schematic diagram of the cable operation parameter prediction device embodiment only shows the parts related to the embodiments of this application. Those skilled in the art will understand that the illustrated structure does not constitute a limitation on the device, and may include more or fewer components than illustrated, or combine certain components, or have different component arrangements.

[0128] like Figure 5 As shown, the forest fire monitoring device includes an image acquisition module 501, a feature extraction module 502, a similarity calculation module 503, a smoke and fire prediction module 504, and a smoke and fire recognition module 505. In some embodiments, the above modules can be programmable software instructions stored in memory and executable by a processor. It is understood that in other embodiments, the above modules can also be program instructions or firmware embedded in the processor.

[0129] Among them, the image acquisition module 501 is used to acquire forest fire monitoring images of the target monitoring site; Feature extraction module 502 is used to extract image features of the forest fire monitoring image based on a pre-trained image feature extraction model to obtain target forest fire image features; The similarity calculation module 503 is used to obtain a similarity weight matrix based on the similarity between the target forest fire image features and each cache key in the preset key value library; wherein, the key value library also includes cache values ​​corresponding to the cache keys, the cache keys are image feature vectors of forest fire image samples of the sample monitoring location, and the cache values ​​represent the smoke type label of the forest fire image samples; Fireworks prediction module 504 is used to perform weighted fusion of the cached values ​​based on the similarity weight matrix to obtain a first fireworks type prediction distribution; The smoke and fire identification module 505 is used to determine the target identification result based on the first smoke and fire type prediction distribution, and the target identification result is used to represent the smoke and fire type of the target monitoring location.

[0130] In some embodiments, determining the target identification result based on the predicted distribution of smoke types includes: Obtain a set of answers, which includes candidate text labels for describing various types of fireworks; The semantic features of the candidate text labels are extracted based on the pre-trained semantic feature extraction model to obtain the label semantic features; Based on the similarity between the target forest fire image features and the tag semantic features, the predicted distribution of the second type of smoke is obtained; The first fireworks type prediction distribution and the second fireworks type prediction distribution are residually fused to obtain the target fireworks type prediction distribution. The firework type with the highest probability is selected from the predicted distribution of target firework types as the target identification result.

[0131] In some embodiments, the semantic feature extraction model includes a first projection layer, a first multi-head attention layer, a first normalization layer, a first feedforward neural network, and a second normalization layer; The pre-trained semantic feature extraction model extracts the semantic features of the candidate text tags to obtain tag semantic features, including: Based on the first projection layer, the candidate text labels are embedded and positionally encoded to obtain the first text processing intermediate features; The first text processing intermediate features are input into the first multi-head attention layer to obtain the second text processing intermediate features; The second text processing intermediate feature and the first text processing intermediate feature are residually connected to obtain the third text processing intermediate feature; The third text processing intermediate feature is normalized based on the first normalization layer to obtain the fourth text processing intermediate feature. The fourth text processing intermediate feature is input into the first feedforward neural network to obtain the fifth text processing intermediate feature; The fifth text processing intermediate feature is residually connected with the fourth text processing intermediate feature to obtain the sixth text processing intermediate feature. The sixth text processing intermediate feature is normalized based on the second normalization layer to obtain the seventh text processing intermediate feature; The semantic features of the tag are determined based on the seventh text processing intermediate features.

[0132] In some embodiments, a similarity weight matrix is ​​obtained based on the similarity between the target forest fire image features and each cached key in a preset key-value library, including: By using the target weight calculation formula and the similarity between the target forest fire image features and each cached key in the preset key value library, a similarity weight matrix is ​​obtained. The target weight calculation formula is as follows: ; Among them, the The similarity weight matrix is ​​represented by exp(); exp() represents the exponential function; the The target similarity sharpness adjustment factor; The target forest fire image features; The transpose matrix formed by the cached keys in the key-value library, This indicates the similarity between the target forest fire image features and each cached key in a preset key-value library; The step of residually fusing the first fireworks type prediction distribution with the second fireworks type prediction distribution to obtain the target fireworks type prediction distribution includes: The first fireworks type prediction distribution and the second fireworks type prediction distribution are residually fused using a target residual fusion formula to obtain the target fireworks type prediction distribution. The target residual fusion formula is as follows: ; Among them, the This represents the predicted distribution of the target fireworks type; The target is to balance the weights; This represents the predicted distribution of the first type of fireworks; the This represents the predicted distribution of the second type of fireworks.

[0133] In some embodiments, the target similarity sharpness adjustment factor and the target balance weight are obtained as follows: Obtain parameter-updated image samples and the fire type labels of the parameter-updated image samples, wherein the parameter-updated image samples are forest fire image samples used to update the target similarity sharpness adjustment factor and the target balance weight; By using the initial weight calculation formula and the similarity between the image feature vector of the updated image sample and each cached key in the key value library, a similarity weight matrix sample is obtained. The initial weight calculation formula is as follows: ; Among them, the The similarity weight matrix represents the samples; exp() represents the exponential function; the This is the initial similarity sharpness adjustment factor; the... Update the image feature vector of the image sample for the parameters; The transpose matrix formed by the cached keys in the preset key-value library, the This indicates the similarity between the image feature vector of the updated image sample and each cached key in the key-value library; Based on the similarity weight matrix sample, the cached values ​​in the key value library are weighted and fused to obtain the first fireworks type prediction distribution sample; Based on the similarity between the cache key and the semantic features of the tag, a second fireworks type prediction distribution sample is obtained; The first and second predicted fireworks type distribution samples are residually fused using an initial residual fusion formula to obtain the target fireworks type prediction distribution sample. The initial residual formula is as follows: ; Among them, the This represents the predicted distribution sample of the target fireworks type; The initial balance weights; This represents the predicted distribution sample of the first type of fireworks; the This represents the predicted distribution sample of the second type of fireworks; Based on the predicted distribution samples of the target fireworks type, the identification result samples are determined; Based on the recognition result sample and the parameter update image sample, the initial similarity sharpness adjustment factor and the initial balance weight are updated to obtain the target similarity sharpness adjustment factor and the target balance weight.

[0134] In some embodiments, the image feature extraction model includes a linear layer, a second projection layer, a second multi-head attention layer, a third normalization layer, a second feedforward neural network, and a fourth normalization layer; The pre-trained image feature extraction model extracts image features from the forest fire monitoring images to obtain target forest fire image features, including: The forest fire monitoring image is input into the linear layer to obtain the first image processing intermediate features; Based on the second projection layer, the first image processing intermediate features are block-embedded and position-encoded to obtain the second image processing intermediate features; The second image processing intermediate features are input into the second multi-head attention layer to obtain the third image processing intermediate features; The third image processing intermediate feature and the second image processing intermediate feature are residually connected to obtain the fourth image processing intermediate feature; The fourth image processing intermediate feature is normalized based on the third normalization layer to obtain the fifth image processing intermediate feature; The fifth image processing intermediate feature is input into the second feedforward neural network to obtain the sixth image processing intermediate feature; The sixth image processing intermediate feature is residually connected with the fifth image processing intermediate feature to obtain the seventh image processing intermediate feature; The seventh image processing intermediate feature is normalized based on the fourth normalization layer to obtain the eighth image processing intermediate feature. The target forest fire image features are determined based on the eighth image processing intermediate features.

[0135] In some embodiments, the forest fire monitoring method further includes: Acquire new forest fire monitoring samples, which include new forest fire image samples and smoke type labels for the new forest fire image samples; Based on the image feature extraction model, image features of the new forest fire image samples are extracted to obtain a new cache key; A new cache value is determined based on the smoke type label of the new forest fire image sample; Create a new key-value pair based on the new cache key and the new cache value, and add the new key-value pair to the key-value library.

[0136] Figure 6 This is a schematic diagram of an embodiment of the computer device of this application.

[0137] Computer device 100 includes a memory 20, a processor 30, and a computer program 40 stored in the memory 20 and executable on the processor 30. When the processor 30 executes the computer program 40, it implements the steps described in the above-described forest fire monitoring method embodiment, for example... Figure 2 Steps 201 to 205 are shown.

[0138] For example, computer program 40 can also be divided into one or more modules / units, which are stored in memory 20 and executed by processor 30. The one or more modules / units can be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of computer program 40 in computer device 100.

[0139] Those skilled in the art will understand that the schematic diagram is merely an example of the computer device 100 and does not constitute a limitation on the computer device 100. It may include more or fewer components than shown, or combine certain components, or different components. For example, the computer device 100 may also include input / output devices, network access devices, buses, etc.

[0140] Processor 30 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. General-purpose processors can be microprocessors, single-chip microcomputers, or any conventional processor.

[0141] The memory 20 can be used to store computer programs 40 and / or modules / units. The processor 30 implements various functions of the computer device 100 by running or executing the computer programs and / or modules / units stored in the memory 20 and by calling data stored in the memory 20. The memory 20 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device 100 (such as audio data), etc. In addition, the memory 20 may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, RAM, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other non-volatile solid-state storage device.

[0142] If the modules / units integrated in the computer device 100 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately added to or subtracted from the content as required by the legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium may not include electrical carrier signals and telecommunication signals.

[0143] In the several embodiments provided in this application, it should be understood that the disclosed computer devices and methods can be implemented in other ways. For example, the computer device embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and other division methods may be used in actual implementation.

[0144] Furthermore, the functional units in the various embodiments of this application can be integrated into the same processing unit, or each unit can exist physically separately, or two or more units can be integrated into the same unit. The integrated units described above can be implemented in hardware or in the form of hardware plus software functional modules.

[0145] It will be apparent to those skilled in the art that this application is not limited to the details of the exemplary embodiments described above, and that this application can be implemented in other specific forms without departing from the spirit or essential characteristics of this application. Therefore, the embodiments should be considered exemplary and not restrictive in all respects. Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or computer devices recited in the computer device claims may also be implemented by the same unit or computer device through software or hardware. The terms "first," "second," etc., are used to denote names and do not indicate any particular order.

[0146] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application and are not intended to limit it. Although this application has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of this application without departing from the spirit and scope of the technical solutions of this application.

Claims

1. A method for monitoring forest fires, characterized in that, include: Acquire forest fire monitoring images of the target monitoring area; Image features of the forest fire monitoring images are extracted based on a pre-trained image feature extraction model to obtain the target forest fire image features; Based on the similarity between the target forest fire image features and each cache key in the preset key value library, a similarity weight matrix is ​​obtained; wherein, the key value library also includes cache values ​​corresponding to the cache keys, the cache keys are the image feature vectors of forest fire image samples from the sample monitoring location, and the cache values ​​represent the smoke type label of the forest fire image samples; the cache keys and cache values ​​in the key value library support dynamic updates. The cached values ​​are weighted and fused based on the similarity weight matrix to obtain the predicted distribution of the first type of fireworks; The target identification result is determined based on the first predicted distribution of the firework type, and the target identification result is used to represent the firework type of the target monitoring location.

2. The forest fire monitoring method as described in claim 1, characterized in that, The determination of target identification results based on the predicted distribution of the smoke type includes: Obtain a set of answers, which includes candidate text labels for describing various types of fireworks; The semantic features of the candidate text labels are extracted based on the pre-trained semantic feature extraction model to obtain the label semantic features; Based on the similarity between the target forest fire image features and the tag semantic features, the predicted distribution of the second type of smoke is obtained; The first fireworks type prediction distribution and the second fireworks type prediction distribution are residually fused to obtain the target fireworks type prediction distribution. The firework type with the highest probability is selected from the predicted distribution of target firework types as the target identification result.

3. The forest fire monitoring method as described in claim 2, characterized in that, The semantic feature extraction model includes a first projection layer, a first multi-head attention layer, a first normalization layer, a first feedforward neural network, and a second normalization layer; The pre-trained semantic feature extraction model extracts the semantic features of the candidate text tags to obtain tag semantic features, including: Based on the first projection layer, the candidate text labels are embedded and positionally encoded to obtain the first text processing intermediate features; The first text processing intermediate features are input into the first multi-head attention layer to obtain the second text processing intermediate features; The second text processing intermediate feature and the first text processing intermediate feature are residually connected to obtain the third text processing intermediate feature; The third text processing intermediate feature is normalized based on the first normalization layer to obtain the fourth text processing intermediate feature. The fourth text processing intermediate feature is input into the first feedforward neural network to obtain the fifth text processing intermediate feature; The fifth text processing intermediate feature is residually connected with the fourth text processing intermediate feature to obtain the sixth text processing intermediate feature. The sixth text processing intermediate feature is normalized based on the second normalization layer to obtain the seventh text processing intermediate feature; The semantic features of the tag are determined based on the seventh text processing intermediate features.

4. The forest fire monitoring method as described in claim 2, characterized in that, The similarity weight matrix obtained based on the similarity between the target forest fire image features and each cached key in the preset key-value library includes: By using the target weight calculation formula and the similarity between the target forest fire image features and each cached key in the preset key value library, a similarity weight matrix is ​​obtained. The target weight calculation formula is as follows: ； Among them, the The similarity weight matrix is ​​represented by exp(); exp() represents the exponential function; the The target similarity sharpness adjustment factor; The target forest fire image features; The transpose matrix formed by the cached keys in the key-value library, This indicates the similarity between the target forest fire image features and each cached key in a preset key-value library; The step of residually fusing the first fireworks type prediction distribution with the second fireworks type prediction distribution to obtain the target fireworks type prediction distribution includes: The first fireworks type prediction distribution and the second fireworks type prediction distribution are residually fused using a target residual fusion formula to obtain the target fireworks type prediction distribution. The target residual fusion formula is as follows: ； Among them, the This represents the predicted distribution of the target fireworks type; The target is to balance the weights; This represents the predicted distribution of the first type of fireworks; the This represents the predicted distribution of the second type of fireworks.

5. The forest fire monitoring method as described in claim 4, characterized in that, The target similarity sharpness adjustment factor and the target balance weight are obtained as follows: Obtain parameter-updated image samples and the fire type labels of the parameter-updated image samples, wherein the parameter-updated image samples are forest fire image samples used to update the target similarity sharpness adjustment factor and the target balance weight; By using the initial weight calculation formula and the similarity between the image feature vector of the updated image sample and each cached key in the key value library, a similarity weight matrix sample is obtained. The initial weight calculation formula is as follows: ； Among them, the The similarity weight matrix represents the samples; exp() represents the exponential function; the This is the initial similarity sharpness adjustment factor; the... Update the image feature vector of the image sample for the parameters; The transpose matrix formed by the cached keys in the preset key-value library, the This indicates the similarity between the image feature vector of the updated image sample and each cached key in the key-value library; Based on the similarity weight matrix sample, the cached values ​​in the key value library are weighted and fused to obtain the first fireworks type prediction distribution sample; Based on the similarity between the cache key and the semantic features of the tag, a second fireworks type prediction distribution sample is obtained; The first and second predicted fireworks type distribution samples are residually fused using an initial residual fusion formula to obtain the target fireworks type prediction distribution sample. The initial residual formula is as follows: ； Among them, the This represents the predicted distribution sample of the target fireworks type; The initial balance weights; This represents the predicted distribution sample of the first type of fireworks; the This represents the predicted distribution sample of the second type of fireworks; Based on the predicted distribution samples of the target fireworks type, the identification result samples are determined; Based on the recognition result sample and the parameter update image sample, the initial similarity sharpness adjustment factor and the initial balance weight are updated to obtain the target similarity sharpness adjustment factor and the target balance weight.

6. The forest fire monitoring method as described in claim 1, characterized in that, The image feature extraction model includes a linear layer, a second projection layer, a second multi-head attention layer, a third normalization layer, a second feedforward neural network, and a fourth normalization layer. The pre-trained image feature extraction model extracts image features from the forest fire monitoring images to obtain target forest fire image features, including: The forest fire monitoring image is input into the linear layer to obtain the first image processing intermediate features; Based on the second projection layer, the first image processing intermediate features are block-embedded and position-encoded to obtain the second image processing intermediate features; The second image processing intermediate features are input into the second multi-head attention layer to obtain the third image processing intermediate features; The third image processing intermediate feature and the second image processing intermediate feature are residually connected to obtain the fourth image processing intermediate feature; The fourth image processing intermediate feature is normalized based on the third normalization layer to obtain the fifth image processing intermediate feature; The fifth image processing intermediate feature is input into the second feedforward neural network to obtain the sixth image processing intermediate feature; The sixth image processing intermediate feature is residually connected with the fifth image processing intermediate feature to obtain the seventh image processing intermediate feature; The seventh image processing intermediate feature is normalized based on the fourth normalization layer to obtain the eighth image processing intermediate feature. The target forest fire image features are determined based on the eighth image processing intermediate features.

7. The forest fire monitoring method as described in claim 1, characterized in that, The forest fire monitoring method also includes: Acquire new forest fire monitoring samples, which include new forest fire image samples and smoke type labels for the new forest fire image samples; Based on the image feature extraction model, image features of the new forest fire image samples are extracted to obtain a new cache key; A new cache value is determined based on the smoke type label of the new forest fire image sample; Create a new key-value pair based on the new cache key and the new cache value, and add the new key-value pair to the key-value library.

8. A forest fire monitoring device, characterized in that, include: The image acquisition module is used to acquire forest fire monitoring images of the target monitoring area; The feature extraction module is used to extract image features from the forest fire monitoring images based on a pre-trained image feature extraction model to obtain the target forest fire image features; The similarity calculation module is used to obtain a similarity weight matrix based on the similarity between the target forest fire image features and each cache key in the preset key value library; wherein, the key value library also includes cache values ​​corresponding to the cache keys, the cache keys are image feature vectors of forest fire image samples from the sample monitoring location, and the cache values ​​represent the smoke type label of the forest fire image samples; The fireworks prediction module is used to perform weighted fusion of the cached values ​​based on the similarity weight matrix to obtain the first fireworks type prediction distribution; The smoke and fire identification module is used to determine the target identification result based on the predicted distribution of the first smoke and fire type, and the target identification result is used to represent the smoke and fire type of the target monitoring location.

9. A computer device, the computer device comprising a processor and a memory, characterized in that, The memory is used to store instructions, and the processor is used to call the instructions in the memory, causing the computer device to execute the forest fire monitoring method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed on a computer device, cause the computer device to perform the forest fire monitoring method as described in any one of claims 1 to 7.

Images