Forest fire smoke monitoring method based on false detection feedback and model self-evolution and related device

Abstract

The invention discloses a forest fire smoke monitoring method and related device based on false detection feedback and model self-evolution, and the method comprises the steps: obtaining a video stream of a monitoring region, carrying out the preprocessing, inputting a target detection model and a scene understanding model, and judging whether a candidate alarm event is generated or not through consistency analysis; when a candidate alarm event is generated, a work order is generated for manual judgment, a real alarm executes a disposal process, and a false alarm is stored in a database; screening high-value false detection samples based on an information entropy priority strategy, storing the samples into a dynamic memory bank, selecting the samples to form an incremental training set, performing incremental training on a pre-training teacher model, and introducing low-rank constraint stability parameters; migrating the discrimination ability of the teacher model to the student model through knowledge distillation, wherein alignment of a channel dimension and a multi-scale space dimension is carried out; and redeploying the updated student model to execute target detection. The invention aims to solve the problems that the current monitoring system depends on manpower, the coverage capability is limited, the response is slow and the cost is high, and the traditional method is high in false alarm rate, easy to leak and misjudge, difficult in large-scale network edge deployment, limited in light-weight model discrimination capability, lack of linkage between alarm processing and model updating and the like, and more accurate and efficient forest fire smoke monitoring is realized.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and intelligent monitoring, specifically relating to a method and related device for monitoring wildfire smoke based on false detection feedback and model self-evolution. Background Technology

[0002] With the advancement of new power systems and smart grid construction, transmission lines, as the core channels for energy transmission, play a vital role in ensuring the safe and stable operation of the social economy. Especially in transmission corridor areas prone to high temperatures and droughts, and densely forested and grassland areas, the smoke and flame hazards caused by wildfires are characterized by multiple sources and rapid dynamic spread, posing a threat to the safety of line equipment and power grid operation. Currently, the prevention and control system still largely relies on manual inspections and routine video surveillance, which suffers from limited coverage, slow response speed, and high labor costs, making it difficult to meet the practical needs of "accurate identification and rapid response."

[0003] In real-world monitoring environments, the visual appearance of wildfire smoke is easily confused with various interference sources such as morning fog, cloud shadows, halos, vehicle exhaust, and industrial emissions. These interferences are highly similar to early smoke in terms of morphology, brightness, and motion characteristics, making it difficult for traditional detection methods based on thresholds or single-frame features to effectively distinguish them, resulting in a high false alarm rate. Meanwhile, early fires often appear as small-scale, low-contrast targets such as light smoke and thin fog. These targets have limited information content and weak spatiotemporal consistency in videos, making them easy for lightweight models to miss or misjudge, leading to delayed warnings and impacting response timeliness. Currently, there are two contradictions at the algorithm and engineering application levels: on the one hand, large-scale multi-class detection networks have strong discriminative capabilities in complex backgrounds, but their computational and communication resource requirements make large-scale edge deployment difficult; on the other hand, while lightweight single-class models adapted to edge devices meet real-time and computational constraints, their discriminative capabilities are limited, making it difficult to maintain a low false alarm rate and high recall rate under complex interference. In addition, there is often a lack of effective linkage between the alarm handling link and the model update mechanism: although false detection samples are recorded, there is a lack of a systematic recycling mechanism, which cannot be effectively converted into training data; model updates rely on full retraining, which has high computational costs and long cycles, and is difficult to adapt to environmental changes and new interferences in a timely manner. Summary of the Invention

[0004] To address the problems existing in the prior art, this invention provides a wildfire smoke monitoring method and related device based on false detection feedback and model self-evolution. Its purpose is to overcome the current wildfire smoke monitoring and control system's reliance on manual inspections and conventional video surveillance, which suffers from limited coverage, slow response speed, high labor costs, and difficulty in accurate identification and rapid response. It also solves the problems of traditional detection methods' inability to effectively distinguish wildfire smoke from various interference sources, resulting in high false alarm rates and easy omissions or misjudgments of early small-scale, low-contrast targets, leading to delayed early warnings. Furthermore, it alleviates the difficulties in deploying large-scale detection networks at the edge, the limited discriminative power of lightweight models, and the contradiction between maintaining low false alarm rates and high recall rates under complex interference. Finally, it improves the lack of effective linkage between the alarm handling link and the model update mechanism, the inability to effectively convert false detection samples into training data, and the high computational cost and long cycle of model updates, making it difficult to adapt to environmental changes and new interference in a timely manner, thereby achieving more accurate and efficient wildfire smoke monitoring.

[0005] To solve the above-mentioned technical problems, the present invention is achieved through the following technical solution: According to a first aspect of the present invention, a method for monitoring wildfire smoke based on false detection feedback and model self-evolution is provided, comprising the following steps: S1. Acquire the video stream of the monitoring area, preprocess the video stream, and input it into the target detection model and the scene understanding model respectively; the target detection model outputs the area information of suspected smoke or flame, and the scene understanding model outputs the semantic description information of the image; perform consistency analysis based on the area information and the semantic description information, and determine whether to generate a candidate alarm event based on the analysis results; S2. When a candidate alarm event is generated, an alarm work order is generated and submitted to manual judgment. Based on the manual judgment result, if it is a real alarm, the handling process is executed; if it is a false alarm, the corresponding data is stored in the database as a false detection sample. S3. Based on the information entropy priority strategy, high-value samples are selected from the false detection samples and stored in a dynamic memory bank; samples are selected from the dynamic memory bank to form an incremental training set, and a pre-trained teacher model is incrementally trained. During the training process, low-rank constraints are introduced to stabilize the model parameters. S4. The discriminative ability of the pre-trained teacher model, which has been incrementally trained, is transferred to the student model, which serves as the target detection model, through knowledge distillation; the knowledge distillation process includes aligning the feature map distribution of the teacher and student models in the channel dimension and aligning the feature representations of the teacher and student models in the multi-scale spatial dimension. S5. The student model updated by knowledge distillation is redeployed to perform the function of the target detection model in step S1.

[0006] In one possible implementation of the first aspect, the consistency analysis based on the region information and the semantic description information in step S1 specifically includes: Calculate the consistency score between the region information and the semantic description information; When the consistency score is lower than a preset threshold, multiple consecutive frames of images associated with the current image are extracted for temporal correlation analysis. Based on the consistency score and the results of the time-series correlation analysis, a comprehensive judgment is made as to whether to generate the candidate alarm event.

[0007] In one possible implementation of the first aspect, when incrementally training a pre-trained teacher model as described in step S3, the semantic description information generated by the model is also combined with the scenario understanding, and difficult example samples are synthesized by generating adversarial networks.

[0008] In one possible implementation of the first aspect, when aligning the feature representations of the teacher-student model across multiple scale spatial dimensions as described in step S4, a random masking mechanism is introduced.

[0009] In one possible implementation of the first aspect, the knowledge distillation process described in step S4 is further combined with a feature pyramid network to align the multi-scale features after the teacher-student model fusion.

[0010] In one possible implementation of the first aspect, the dynamic memory is managed using a class-level circular buffer structure, and the sample eviction policy is determined based on its information entropy priority.

[0011] In one possible implementation of the first aspect, when incrementally training a pre-trained teacher model as described in step S3, the low-rank constraint is implemented by introducing the kernel norm of the model weight matrix into the verification loss function.

[0012] In one possible implementation of the first aspect, the alignment of the feature map distributions of the teacher-student model in the channel dimension as described in step S4 is achieved by minimizing the KL divergence between the probability distributions of the teacher-student model features after processing by the activation function.

[0013] According to a second aspect of the present invention, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the aforementioned wildfire smoke monitoring method based on false detection feedback and model self-evolution.

[0014] According to a third aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program, which, when executed by a processor, implements the aforementioned wildfire smoke monitoring method based on false detection feedback and model self-evolution.

[0015] According to a fourth aspect of the present invention, a computer program product is provided, which, when executed by a processor, implements the aforementioned wildfire smoke monitoring method based on false detection feedback and model self-evolution.

[0016] Compared with the prior art, the present invention has at least the following beneficial effects: This invention provides a wildfire smoke monitoring method based on false positive feedback and model self-evolution. Through collaborative analysis of a target detection model and a scene understanding model, it performs multi-dimensional consistency verification on suspected smoke or fire areas, effectively distinguishing wildfire smoke from common interference sources such as morning fog, cloud shadows, and halos, thus reducing the false positive rate. Simultaneously, this method can capture small-scale, low-contrast targets such as light smoke and fog in early-stage fires, enhancing the ability to identify weak features, reducing the risk of missed detections, and shortening response time. Employing a teacher-student model architecture and a knowledge distillation mechanism, the high discriminative power of complex models is transferred to a lightweight student model. This enables the student model to achieve both high inference speed and strong anti-interference performance on edge devices, effectively balancing computational resource constraints and detection accuracy requirements, and resolving the contradiction between the difficulty of deploying large-scale networks at the edge and the insufficient discriminative power of lightweight models. Through a systematic collection of false positive samples and a high-value sample screening mechanism, a dynamic memory is constructed, and the teacher model is stably optimized based on incremental training and low-rank constraints, avoiding the computational overhead and latency caused by full retraining. This self-evolutionary mechanism enables the model to quickly incorporate new interference samples and scene change characteristics, continuously improving its adaptability to complex environments. By deeply integrating the alarm handling chain with the model update mechanism, manual judgment feedback directly drives model iteration, forming a closed-loop process of detection-feedback-learning-deployment, reducing the frequency of manual review and operational costs. Through the overall synergy of the above technical means, this invention can achieve early detection, rapid identification, and low false alarm warnings of wildfire smoke in complex power transmission corridor environments such as high temperature and drought, and dense vegetation. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the specific embodiments of the present invention, the drawings used in the description of the specific embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is a flowchart of a wildfire smoke monitoring method based on false detection feedback and model self-evolution according to the present invention; Figure 2 This is a flowchart of a wildfire smoke monitoring method based on false detection feedback and model self-evolution, according to an embodiment of the present invention. Figure 3 This is a diagram of the multi-scale fusion module after knowledge distillation in the model update part of an embodiment of the present invention; Figure 4 This invention provides a model for extracting semantic feature maps from text and image information in a multimodal task. Figure 5 The images show a comparison of the monitoring of the same false detection scenario by the model before and after the update in the embodiment of the present invention. (a), (b), and (c) are the actual false detection scenarios, and (d), (e), and (f) are the correct monitoring after the update. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Combination Figures 1 to 5 As shown in the figure, the wildfire smoke monitoring method based on false detection feedback and model self-evolution provided by the embodiments of the present invention specifically includes the following steps: S1. Acquire the video stream of the monitoring area, preprocess the video stream, and input it into the target detection model and the scene understanding model respectively; the target detection model outputs the area information of suspected smoke or flame, and the scene understanding model outputs the semantic description information of the image; perform consistency analysis based on the area information and the semantic description information, and determine whether to generate a candidate alarm event based on the analysis results.

[0021] In this step, real-time video streams are acquired from front-end camera equipment deployed at monitoring points along power transmission lines and in forest areas. The video streams undergo preprocessing operations, including frame sampling, size cropping, normalization, and floating-point format conversion. The processed video frames are then used to construct short time segments for real-time detection and correlated frame backtracking analysis.

[0022] The preprocessed video frames are input into the YOLO Lightweight Student Model and the CogVLM Model for parallel processing. The YOLO Lightweight Student Model is responsible for real-time inference, outputting bounding boxes for suspected smoke or flame, class confidence scores, and multi-level feature representations. The CogVLM model generates semantic descriptions of images through a deep vision-language fusion mechanism. Specifically, keyframes are fed into the CogVLM visual encoder to obtain a set of visual tokens. It is embedded into several Transformer layers of the language backbone, generating modulation terms at the attention and FFN paths respectively. and .

[0023] The visual expert's calculation process is as follows: Let the text query be... Visual key value , The text and visual global context are used for joint calculation:

[0024]

[0025] That, For visual global context vectors, For the sigmoid function, gated This ensures the controlled injection of visual information. In this way, visual information can both deeply condition the language representation and feed the language context back to visual attention, improving grounding and fine-grained descriptive capabilities.

[0026] The output of visual experts and scene features are further aggregated into a cross-modal semantic embedding. These embeddings are used for decision-making in multiple locations. Consistency analysis specifically includes calculating a consistency score between region information and semantic description information. It can be calculated using the following formula:

[0027] in, For the visual features of the detector, The semantic vector corresponding to the scene-level text description generated for CogVLM. , These are the weighting coefficients.

[0028] When consistency score Below the preset threshold At that time, the system automatically extracts multiple consecutive frames of images associated with the current image for temporal correlation analysis, and extracts N frames before and after to form a frame sequence. Temporal correlation analysis includes calculating temporal consistency terms. Consistency with VE layers As auxiliary information to reduce misjudgments caused by transient noise, among which This represents inter-frame alignment operations based on optical flow or motion estimation. The results of consistency scores and temporal correlation analysis are combined to determine whether to generate candidate alarm events.

[0029] S2. When a candidate alarm event is generated, an alarm work order is generated and submitted to a human for judgment. Based on the human judgment result, if it is a real alarm, the handling process is executed; if it is a false alarm, the corresponding data is stored in the database as a false detection sample.

[0030] Specifically, the system automatically generates alarm work orders, which include time, location, detection box information, and confidence vector. Multi-level feature representation The alarm and related video indexes are then pushed to the operations and maintenance terminal. Operations and maintenance personnel manually verify the authenticity of the alarm. If the alarm is valid, the emergency response process is immediately triggered, such as dispatching a work order for on-site handling and recording the handling time, location, and result in the work order system. If the alarm is a false alarm, the corresponding video clips, detection results, semantic description information, etc., are stored as false detection samples in the false detection database, and the reason for the manual verification is recorded simultaneously.

[0031] S3. Based on the information entropy priority strategy, high-value samples are selected from the false detection samples and stored in a dynamic memory bank; samples are selected from the dynamic memory bank to form an incremental training set, and a pre-trained teacher model is incrementally trained. During the training process, low-rank constraints are introduced to stabilize the model parameters.

[0032] After false positive samples are stored in the database, the system uses an information entropy priority strategy to filter high-value samples. (Information entropy) The calculation formula is:

[0033] in, It is the confidence distribution of the sample. It represents the number of categories. High-entropy samples indicate high model uncertainty and thus have higher information value.

[0034] The dynamic memory is managed using a class-level circular buffer architecture, defined as follows: Each sub-library With fixed capacity Sample updates follow an information entropy-based prioritization strategy. Sample importance is assessed by calculating sample entropy values, and samples with high entropy values ​​are retained first to maximize the information density of the memory. The memory maintenance process is formally represented as:

[0035] in It is the sample with the lowest entropy value in the current memory.

[0036] Incremental training sets are constructed by selecting samples from the dynamic memory bank to incrementally train the pre-trained teacher model. A low-rank constraint is introduced during training to optimize the incremental learning process. This is achieved by incorporating the kernel norm of the model weight matrix into the validation loss function, thereby stabilizing model parameters and preventing overfitting. The low-rank constraint is specifically implemented by adding [the kernel norm of the model weight matrix] to the loss function. Item, of which The nuclear norm of a matrix is ​​denoted by .

[0037] Furthermore, incremental training also incorporates semantic description information generated by the CogVLM model to synthesize difficult example samples (light smoke, dense smoke, flames, etc.) through adversarial network generation. Specifically, a cross-class adversarial generation module is constructed. The false positive samples are fed into the cross-class adversarial generation module according to the scenario and confusion type. This generator uses category conditions. With the context embedding provided by CogVLM As input, the synthesized samples are semantically and structurally closer to real ambiguous samples. Besides determining the authenticity of samples, the discriminator D also needs to determine whether the samples satisfy the semantic constraints described by VE and the text, thus guiding the generator towards a semantically consistent but falsely induced sample space, forming a targeted expansion of the training set. The corresponding adversarial loss term is... .

[0038] In incremental training, the system also utilizes CogVLM to construct a pseudo-label pool by combining the scene description and grounding information generated by the trigger frame with the detector output, and calculates a credibility score for each pseudo-label. Pseudo-labels with higher credibility are included in the semi-supervised loss with greater weight. The weight is defined as .

[0039] To ensure that incremental data is used robustly to update the teacher-level multi-class network during incremental learning, and that the model can effectively integrate new data and maintain stability, this invention introduces low-rank constraints to optimize the incremental training process. Through low-rank constraints, the update of the weight matrix is ​​restricted, preventing overfitting and maintaining the structural stability of the model. Specifically, low-rank constraints maximize high-entropy samples in the memory bank while maintaining low-rank updates of model parameters, thereby achieving balanced learning of historical samples and new data. This ensures that after the teacher recovers or enhances its discriminative ability in new scenarios, the teacher's high-quality knowledge is transferred to the lightweight single-class student end under controlled conditions through distillation. Furthermore, the subsequent distillation stage is not conducted in parallel with incremental training; in this stage, the student model parameters... Teacher model parameters During the update process, a low-rank constraint is introduced, and the total loss function for incremental training is:

[0040] in, For the detection loss of labeled samples, , , , For regularization hyperparameters, Describe the Frobenius norm. and These are the weight matrices for the teacher and student models, respectively. is the nuclear norm of the matrix (i.e., the low-rank constraint).

[0041] The system incorporates samples that are manually verified as false alarms into the regression incremental training process as the first type of key samples.

[0042] S4. The discriminative ability of the pre-trained teacher model, which has been incrementally trained, is transferred to the student model, which serves as the target detection model, through knowledge distillation; the knowledge distillation process includes aligning the feature map distribution of the teacher and student models in the channel dimension and aligning the feature representations of the teacher and student models in the multi-scale spatial dimension.

[0043] The knowledge distillation process employs a cross-category knowledge transfer mechanism, leveraging generative adversarial networks (GANs) to enhance the student model's ability to discriminate between different target categories. By using a trained multi-category teacher model as the knowledge source, the problem of false detections easily generated by single-category student models in complex environments is addressed. The teacher model, trained on large-scale multi-category data, possesses rich semantic information and global feature representation, while the student model, due to its narrower category range, suffers from insufficient feature representation. To bridge this gap, Channel-Wise Distillation Loss (CWDLoss) and Multi-Scale Guided Distillation Loss (MGDLoss) are combined with a Feature Pyramid Network (FPN) to achieve multi-scale feature alignment, enabling effective transfer of teacher knowledge to the student model. Simultaneously, the GAN further enhances the student model's ability to discriminate between different categories of smoke and flame targets, improving its recognition performance in complex environments.

[0044] Aligning the feature map distributions of the teacher-student model along the channel dimension is achieved using CWDLoss. For teacher and student features with shapes [N,C,H,W], the spatial dimension of the teacher features is first flattened, and a smooth probability distribution is obtained using the softmax function (scaled via the temperature parameter T):

[0045] in, This represents the feature value of the teacher model at a certain spatial location. Then, LogSoftmax is applied to both the teacher and student features, and the KL divergence between them is calculated:

[0046] in, and represents the spatial probability distributions of the teacher and student models after softmax, respectively. This loss allows the student model to align with the spatial distribution of the teacher model in each channel, thereby enhancing its ability to perceive fine-grained smoke edges and interference areas.

[0047] Aligning the feature representations of the teacher-student model across multiple spatial dimensions is achieved using MGDLoss. In this process, a generative module is introduced to map student features to the semantic space of teacher features, and a random masking mechanism is employed to reduce background interference. Finally, the difference between the output of the generative module and the teacher features is measured using mean squared error loss, with the following formula:

[0048] in, For the features of the teacher model at a certain layer, The corresponding features output by the generation module. The total number of feature elements. The total multiscale guided loss is derived from the sum of the scale losses of each layer.

[0049] Within the overall knowledge distillation framework, these two loss functions constrain the feature learning of the student model from different perspectives. CWDLoss focuses on enabling the student model to capture the spatial distribution information of the teacher model in the channel dimension, while MGDLoss aims to establish a semantic mapping between teachers and students across multi-scale features. The two are integrated through the FeatureLoss module.

[0050] The knowledge distillation process also incorporates a Feature Pyramid Network (FPN) to align the multi-scale features after the teacher-student model fusion. An FPN fusion structure is constructed in both the teacher and student networks. First, 1×1 lateral convolutions are used to adjust the channel count of features from different scales. Then, upsampling is used to map low-resolution high-level features to the same spatial size as high-resolution low-level features. Lateral connections are then performed using pixel-wise addition. Finally, a 3×3 smooth convolution is used to process the fused feature map.

[0051] The FPN alignment loss is calculated as follows:

[0052] in, and These represent the multi-scale features of the teacher and student models after being fused by their respective FPN modules, where N is the batch size.

[0053] This invention constructs an FPN fusion structure in both the teacher and student networks. In traditional knowledge distillation methods, the teacher and student models typically align features only in one or a few intermediate layers. While this method can convey some high-level semantic information, its effectiveness is often limited when dealing with small object detection and complex background interference. This is because features at different scales are complementary for object detection tasks: low-level features retain high-resolution local details, which helps to accurately capture small objects, while high-level features contain rich global semantic information, playing a crucial role in distinguishing objects from the background. Single-scale alignment can easily lead to the loss of information at different scales in the teacher model, thus preventing the student model from fully learning the feature representation of the teacher model. To address this, a multi-scale feature fusion module is introduced, drawing inspiration from Feature Pyramid Network (FPN). By integrating features from different levels, it provides a unified and comprehensive supervision target for knowledge distillation.

[0054] Specifically, this multi-scale feature fusion module first uses 1×1 lateral convolutions to adjust the number of channels for features from different scales in the teacher model. This unifies features at each scale to the same number of channels, allowing subsequent feature fusion operations to be performed within a unified dimension. Next, through upsampling, low-resolution high-level features are mapped to the same spatial size as high-resolution low-level features, and lateral connections are performed using pixel-wise addition to achieve effective fusion of low- and high-level features. Subsequently, to further eliminate noise that may be introduced by upsampling and fusion operations, the module also employs 3×3 smooth convolutions to process the fused feature map, thereby obtaining a fused feature map that retains both detail and global semantic information.

[0055] During knowledge distillation, not only are the layer-by-layer features of the teacher and student models aligned, but corresponding FPN modules are also constructed for the teacher and student models using the aforementioned multi-scale feature fusion module. Through the FPN module, features from different levels in the teacher model are fused into a unified multi-scale fused feature representation. This representation includes not only low-level local details but also high-level global semantic information. Then, L2 loss is used to align the fused features of the teacher and student models after processing by the FPN module.

[0056] In the feature alignment process, the number of channels for student features is first adjusted using a 1×1 convolutional layer, and then the teacher and student features are normalized using a BatchNorm layer. The normalization process is described as follows:

[0057] in, Indicates input features, and These are the mean and variance, respectively. It is a small constant to prevent division by zero, and and These are the learnable parameters. Next, the aligned features are fed into the corresponding CWDLoss or MGDLoss for layer-by-layer loss calculation, and the losses of each layer are weighted and summed to obtain the final feature loss.

[0058] The total knowledge distillation loss is the traditional layer-by-layer feature alignment loss L. feature Fusion loss with FPN L FPN The weighted sum is integrated through the FeatureLoss module, enabling the student model to learn the feature representation of the teacher model from both local details and global semantics during the distillation process, effectively making up for the shortcomings of single-scale alignment methods in multi-scale information transmission.

[0059] S5. The student model updated by knowledge distillation is redeployed to perform the function of the target detection model in step S1.

[0060] Specifically, the updated student model is redeployed to the real-time monitoring link on edge devices for detection and alerting of subsequent video streams. The system continuously collects new false alarm samples and incorporates them into subsequent update cycles, outputting statistical reports on monitoring accuracy, alarms, and handling. Over long-term operation, a closed-loop optimization process of data collection—multimodal detection—dispatch—feedback—incremental retraining—deployment is formed, thereby steadily reducing the false alarm rate, improving the early smoke recall rate, and enhancing robustness in complex scenarios.

[0061] Through the above steps, this invention achieves automation and self-evolution in wildfire smoke monitoring, effectively overcoming the problems of limited coverage, slow response speed, and high labor costs caused by the reliance on manual inspections and conventional video surveillance in the current wildfire smoke monitoring and control system. It also solves the problem of high false alarm rate caused by the difficulty of traditional detection methods in effectively distinguishing wildfire smoke from various interference sources, alleviates the contradiction between the difficulty of deploying large-scale detection networks at the edge and the limited discrimination capability of lightweight models at the algorithm and engineering application levels, improves the situation of lack of effective linkage between alarm handling links and model update mechanisms, and achieves more accurate and efficient wildfire smoke monitoring.

[0062] Through multimodal collaboration of text and images, not only is qualitative information about the target provided, but the model's ability to understand complex backgrounds is also enhanced.

[0063] The false positive recovery and model update section incorporates false positive samples into the incremental training set of the teacher model. It then uses text information generated by a cross-class adversarial generative network and the CogVLM model for auxiliary training, further enhancing the model's transferability and recognition capabilities in different scenarios. The training set employs dynamic sample library management, using low-rank constraints to optimize the incremental learning process and saving and replaying historical key samples to ensure data diversity and balance. During retraining, channel-level distillation (CWDLoss) aligns the differences in feature channel distribution between teachers and students. Multi-scale guided distillation (MGDLoss) incorporates a random masking mechanism to reduce background interference, and Feature Pyramid Network (FPN) aligns the features of teachers and students at multiple scale levels. This transfers the multi-class discrimination capability of the teacher model to the lightweight single-class student model, thereby enhancing the student model's ability to perceive thin smoke at small scales and dense smoke over large areas. The updated student model is redeployed to the monitoring link, forming a closed-loop process of data collection, detection, dispatching, feedback, and optimization.

[0064] This invention utilizes cross-category transfer learning, leveraging a multi-category teacher model to transfer rich discriminative knowledge to the student model, effectively improving the student model's robustness in recognizing complex backgrounds and interfering targets. By incorporating generative adversarial networks, it enhances the model's adaptability to different categories and complex environmental scenarios.

[0065] By employing the CogVLM model to semantically align image information and generated text, the system's ability to discriminate in complex backgrounds is improved. Furthermore, the system extracts contextual information about the target by associating frames, and by combining image sequences from preceding and following frames, it further enhances its understanding of the target scene.

[0066] CWDLoss emphasizes channel-level spatial distribution alignment to enhance fine-grained feature learning; MGDLoss introduces a random masking mechanism in multi-scale features to effectively reduce background noise interference and improve the detection performance of thin smoke and edge smoke.

[0067] By combining feature pyramid (FPN) for multi-level semantic feature fusion, the student model can simultaneously capture local details and understand the global scene.

[0068] The system uses an incremental learning mechanism to support the model in continuously adapting to new scenarios and interference patterns during long-term operation. At the same time, it maintains the memory of historical knowledge through dynamic knowledge distillation and regularization techniques, thereby maintaining high detection accuracy and system reliability in continuous evolution.

[0069] In one specific embodiment, a wildfire smoke monitoring method based on multimodal collaboration and cross-category knowledge distillation includes the following steps: Step 1: Data acquisition and preprocessing. Real-time video streams are acquired by monitoring cameras deployed on power transmission lines and in forest areas. The video streams are preprocessed by frame segmentation, cropping, normalization, and data format conversion to obtain preprocessed images that meet the input requirements of the detection model.

[0070] Step 2: Multimodal real-time detection. The preprocessed image is simultaneously input into the YOLO Lightweight Student Model and the CogVLM Model. The YOLO Lightweight Student Model outputs candidate regions for suspected smoke or flames and their corresponding confidence information. The CogVLM Model generates a scene text description corresponding to the preprocessed image. The consistency score between the visual features output by the YOLO Lightweight Student Model and the semantic features of the text description generated by the CogVLM Model is calculated. If the consistency score is lower than a preset threshold, N frames before and after the preprocessed image are extracted from the video stream corresponding to the preprocessed image. The temporal consistency and inter-layer consistency between the N frames before and after the preprocessed image are calculated. The consistency score, temporal consistency, and inter-layer consistency are combined to determine whether to generate a candidate alarm event.

[0071] The following formula is used to calculate the consistency score:

[0072] in, For consistency score, Visual features output for the YOLO lightweight student model. The cross-modal semantic embeddings output by the CogVLM model. This is the semantic vector corresponding to the scene text description. This indicates the feature pooling operation. , These are the weighting coefficients. This indicates the calculation of cosine similarity.

[0073] The following formula is used to calculate timing consistency:

[0074] in, For timing consistency loss, The coordinates of the suspected target in the preprocessed image. These are the coordinates of the suspected target in the previous frame image. Represents the coordinate mapping function, ∥ ∥² represents the calculation of the L2 norm.

[0075] Step 3: Alarm Work Order Processing and Sample Collection. If a candidate alarm event is generated, an alarm work order containing the event time, location, and associated video index is automatically generated and pushed to the maintenance personnel's terminal. The maintenance personnel manually judge the alarm work order. If it is determined to be a real alarm, the work order is dispatched to the site for processing and the processing result is recorded. If it is determined to be a false alarm, the preprocessed image corresponding to the false alarm, the images of the preceding and following N frames, and the false alarm annotation information are stored in the false detection database.

[0076] Step 4: Dynamic Sample Library Construction. A dynamic memory library in the form of a class-level circular buffer is constructed. The dynamic memory library includes several sub-libraries corresponding to different scene categories, each with a fixed storage capacity. The information entropy of false detection samples in the false detection database is calculated, and sample priorities are determined based on the information entropy. High-priority false detection samples are stored in their corresponding sub-libraries according to scene category. When the storage capacity of a sub-library reaches the fixed storage capacity, the sample with the lowest information entropy in the sub-library is deleted, thus realizing the updating and maintenance of the dynamic memory library.

[0077] The following formula is used to calculate the information entropy of a sample:

[0078] in, For sample information entropy, This represents the total number of scene categories. For the sample to belong to the first The probability of similar scenarios This is a logarithmic operation.

[0079] Step 5: Cross-class incremental training. Extract historical high-entropy samples from the dynamic memory and combine them with false detection samples from the false detection database to form an incremental training set. Input the incremental training set into the multi-class teacher model and use the scene text descriptions generated by the cross-class adversarial generative network and the CogVLM model for auxiliary training. During the training process, a low-rank constraint is introduced to limit the update magnitude of the teacher model's weight matrix, so as to enable the teacher model to learn new sample knowledge and retain historical knowledge.

[0080] When a low-rank constraint is introduced, the parameter update loss function of the teacher model satisfies:

[0081] in, For the detection loss of labeled samples, , , , For regularization hyperparameters, For pseudo-label weights, For semi-supervised loss, To combat the losses, , These are the weight matrices for the teacher model and the student model, respectively. For the Frobenius norm, For nuclear norm.

[0082] Step 6: Knowledge distillation and model deployment. Channel-level distillation loss and multi-scale guided distillation loss are used in combination with a feature pyramid network to perform knowledge distillation on the trained teacher model and YOLO lightweight student model. Channel-level distillation loss is used to align the spatial distribution of teacher and student models in the feature channel dimension. Multi-scale guided distillation loss is used in combination with a random masking mechanism to reduce background interference and achieve multi-scale feature alignment between teacher and student models. The distilled and updated YOLO lightweight student model is redeployed to the detection link corresponding to the monitoring camera. Steps 1 to 6 are repeated to form a closed-loop optimization process for wildfire smoke monitoring. Channel-level distillation losses satisfy:

[0083] in, For channel-stage distillation losses, The spatial probability distribution of teacher model features after softmax processing. The spatial probability distribution of student model features after softmax processing.

[0084] Multi-scale guided distillation loss satisfies:

[0085] in, To guide distillation losses at multiple scales, The total number of characteristic elements. For a feature of a certain layer of the teacher model, The corresponding features output by the generation module.

[0086] When combining a feature pyramid network for multi-scale feature alignment, the feature alignment loss satisfies:

[0087] in, The feature alignment loss is used for the feature pyramid network. The teacher model consists of multi-scale features fused using a feature pyramid network. The student model consists of multi-scale features fused by a feature pyramid network.

[0088] In another embodiment of the present invention, a computer device is provided, comprising a processor and a memory. The memory stores a computer program, which includes program instructions. The processor executes the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions in the computer storage medium to achieve a corresponding method flow or corresponding function. The processor described in this embodiment of the present invention can be used in the operation of a wildfire smoke monitoring method based on false detection feedback and model self-evolution.

[0089] In another embodiment of the present invention, a storage medium is provided, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device used to store programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the computer device and extended storage media supported by the computer device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, the storage space also stores one or more instructions suitable for loading and execution by a processor. These instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be Random Access Memory (RAM) or non-volatile memory, such as at least one disk storage device. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the wildfire smoke monitoring method based on false detection feedback and model self-evolution in the above embodiments.

[0090] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, etc.) containing computer-usable program code.

[0091] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0092] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0093] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0094] This invention also provides a computer program product, which is used to execute any of the above-described methods for monitoring wildfire smoke based on false detection feedback and model self-evolution. Since the computer program product provided by this invention belongs to the same inventive concept as the above-described method for monitoring wildfire smoke based on false detection feedback and model self-evolution, it possesses all the advantages of the above-described method. Therefore, the beneficial effects of the computer program product provided by this invention will not be elaborated upon here.

[0095] In this invention, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to a specific feature, structure, material, or characteristic described in connection with that embodiment or example, which is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0096] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit them. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the scope of the technology disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention.

Claims

1. A method for monitoring wildfire smoke based on false detection feedback and model self-evolution, characterized in that, Includes the following steps: S1. Obtain the video stream of the monitoring area, and after preprocessing the video stream, input it into the target detection model and the scene understanding model respectively; The target detection model outputs region information that is suspected to be smoke or flame, and the scene understanding model outputs semantic description information of the image; a consistency analysis is performed based on the region information and the semantic description information, and a candidate alarm event is generated based on the analysis results; S2. When a candidate alarm event is generated, an alarm work order is generated and submitted to a human for judgment. Based on the results of manual judgment, if it is a real alarm, the handling procedure will be executed; if it is a false alarm, the corresponding data will be stored in the database as a false detection sample. S3. Based on the information entropy priority strategy, high-value samples are selected from the false detection samples and stored in a dynamic memory bank; samples are selected from the dynamic memory bank to form an incremental training set, and a pre-trained teacher model is incrementally trained. During the training process, low-rank constraints are introduced to stabilize the model parameters. S4. The discriminative ability of the pre-trained teacher model, which has been incrementally trained, is transferred to the student model, which serves as the target detection model, through knowledge distillation; the knowledge distillation process includes aligning the feature map distribution of the teacher and student models in the channel dimension and aligning the feature representations of the teacher and student models in the multi-scale spatial dimension. S5. The student model updated by knowledge distillation is redeployed to perform the function of the target detection model in step S1.

2. The wildfire smoke monitoring method based on false detection feedback and model self-evolution as described in claim 1, characterized in that, The consistency analysis based on the region information and the semantic description information in step S1 specifically includes: Calculate the consistency score between the region information and the semantic description information; When the consistency score is lower than a preset threshold, multiple consecutive frames of images associated with the current image are extracted for temporal correlation analysis. Based on the consistency score and the results of the time-series correlation analysis, a comprehensive judgment is made as to whether to generate the candidate alarm event.

3. The wildfire smoke monitoring method based on false detection feedback and model self-evolution as described in claim 1, characterized in that, In step S3, when incrementally training a pre-trained teacher model, the semantic description information generated by the model is also combined with the scene understanding, and difficult example samples are synthesized by generating adversarial networks.

4. The wildfire smoke monitoring method based on false detection feedback and model self-evolution as described in claim 1, characterized in that, In step S4, when aligning the feature representations of the teacher-student model across multiple spatial dimensions, a random masking mechanism is introduced.

5. The wildfire smoke monitoring method based on false detection feedback and model self-evolution according to claim 1, characterized in that, The knowledge distillation process described in step S4 also incorporates a feature pyramid network to align the multi-scale features after the teacher-student model fusion.

6. The wildfire smoke monitoring method based on false detection feedback and model self-evolution as described in claim 1, characterized in that, The dynamic memory is managed using a class-level circular buffer structure, and the sample elimination strategy is determined based on its information entropy priority.

7. A wildfire smoke monitoring method based on false detection feedback and model self-evolution as described in claim 1, characterized in that, In step S3, when incrementally training a pre-trained teacher model, the low-rank constraint is implemented by introducing the kernel norm of the model weight matrix into the verification loss function.

8. The wildfire smoke monitoring method based on false detection feedback and model self-evolution according to claim 1, characterized in that, The alignment of the feature map distribution of the teacher-student model in the channel dimension, as described in step S4, is achieved by minimizing the KL divergence between the probability distributions of the teacher-student model features after processing by the activation function.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements a wildfire smoke monitoring method based on false detection feedback and model self-evolution as described in any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements a wildfire smoke monitoring method based on false detection feedback and model self-evolution as described in any one of claims 1 to 8.

Images