Self-adaptive multi-scale dynamic fire detection method and system based on few-sample learning

Abstract

The invention relates to the technical field of computer vision and public safety, in particular to a self-adaptive multi-scale dynamic fire detection method and system based on few-sample learning. The method comprises the following steps: constructing a multi-scale semantic attribute library; generating a multi-scale dynamic feature map fusing color, time domain motion and multi-scale optical flow information based on the video sequence; obtaining visual and text feature vectors, and performing cross-modal semantic alignment through comparative learning to construct a basic model; a visual prototype and a semantic prototype are calculated and fused to obtain a scene prototype, and the basic model is finely adjusted to generate a scene special model; and acquiring a real-time video sequence, calculating to obtain a real-time multi-scale dynamic feature map, and extracting cosine similarity between a real-time feature visual vector and a scene prototype, thereby performing real-time fire detection. Due to the fact that multi-scale optical flow information is extracted, full-scale dynamic characteristics from microscopic flickering to macroscopic spreading can be accurately captured, and fire disasters can be efficiently detected.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer vision and public safety technology, in particular to a few-shot learning adaptive multi-scale dynamic fire detection method and system. BACKGROUND

[0002] In fire safety monitoring, image or video-based fire detection technology aims to achieve early and accurate alarm of fire by analyzing visual data. Traditional fire detection systems often rely on physical sensors, while vision-based detection methods have attracted widespread attention due to their non-contact and wide monitoring range. Most current image-based fire detection methods focus on analyzing single-frame static images, but this static analysis mode cannot effectively capture and utilize dynamic behavior characteristics unique to fire, such as the irregular flickering of flames, the spreading trend, and the dynamic diffusion pattern of smoke. Due to the lack of analysis of temporal dynamic characteristics, such methods have weak discrimination ability for static interference objects with similar colors or textures, resulting in high false alarm rates in actual applications, which seriously affects the reliability of the system.

[0003] In fire detection, the mainstream technology mostly adopts a supervised learning paradigm, whose performance heavily depends on large-scale, high-quality, and accurately labeled training datasets. However, fire scenes are diverse, and a model may perform well in the scenarios covered by the training set (such as specific viewpoints, lighting, and geographical environments), but it is difficult to effectively generalize to new scenarios that have not been seen before. For example, changes in camera models, changes in viewing angles, lighting differences caused by day and night or seasons, and different natural environment backgrounds can all cause significant performance degradation of the model. This contradiction between high dependence on data and insufficient generalization ability limits the universality of the technology in the diverse real world. Existing models mostly use fixed network structures or perform feature analysis at a single scale, which makes it difficult to effectively handle the large scale variation of fire targets in images, thereby affecting the comprehensiveness and accuracy of detection. When a trained model needs to be deployed in a new and significantly different monitoring scenario, its performance often cannot meet the requirements.

[0004] Therefore, traditional fire detection methods often have low data utilization, high detection cost, and high false alarm rates due to insufficient integration of dynamic characteristics of fire, insufficient utilization of dynamic information, and lack of adaptive mechanism design for scale differences of fire targets. SUMMARY

[0005] Therefore, in order to solve the above technical problems, a few-shot learning adaptive multi-scale dynamic fire detection method and system is provided, which can balance efficient data utilization and dynamic feature mining, and achieve high-performance and low-false adaptive fire detection.

[0006] An adaptive multi-scale dynamic fire detection method based on few-sample learning, the method comprising: Collect fire data, determine the atomic attribute set, scale context set, and attribute relationship set based on the fire data, and construct a multi-scale semantic attribute library based on the atomic attribute set, scale context set, and attribute relationship set; The video sequence to be analyzed is obtained, and multi-scale dynamic features are extracted from the video sequence to generate a multi-scale dynamic feature map that integrates color, temporal motion and multi-scale optical flow information. Visual feature vectors are obtained based on the multi-scale dynamic feature map, and text feature vectors are obtained based on the multi-scale semantic attribute library. The visual feature vectors and text feature vectors are then semantically aligned across modalities using contrastive learning loss to construct a pre-trained basic model. The visual prototype and semantic prototype are calculated and fused according to the multi-scale semantic attribute library to obtain the scene prototype. The parameters of the basic model are optimized to generate a scene-specific model. A real-time video sequence is acquired and a real-time multi-scale dynamic feature map is calculated. The visual encoder in the scene-specific model is used to determine the real-time visual feature vector, and the cosine similarity between the real-time visual feature vector and the scene prototype is calculated. Real-time fire detection is performed based on the cosine similarity.

[0007] In one embodiment, determining the set of atomic attributes, the set of scale contexts, and the set of attribute relationships based on the fire data includes: Based on the fire data, static visual attributes and dynamic behavioral attributes are defined, and the static visual attributes and dynamic behavioral attributes are combined into an atomic attribute set; Each observation scale is defined based on the fire data, and a scale context set is constructed based on each observation scale; Establish a mapping function between scale and attribute, construct a combination between scale and attribute, and generate semantic description text as a set of attribute relationships.

[0008] In one embodiment, multi-scale dynamic feature extraction is performed on the video sequence to generate a multi-scale dynamic feature map that integrates color, temporal motion, and multi-scale optical flow information, including: For the video sequence, the RGB color channels of the keyframes are extracted; The frame difference between video frames in the video sequence is calculated using the three-frame difference method; Based on the video sequence, a lightweight deep learning optical flow network is used to calculate optical flow on a multi-scale spatial pyramid to generate multi-scale optical flow information; The RGB color channels and frame difference multi-scale optical flow information are spliced ​​and fused to generate a multi-scale dynamic feature map.

[0009] In one embodiment, based on the video sequence, a lightweight deep learning optical flow network is used to calculate optical flow on a multi-scale spatial pyramid to generate multi-scale optical flow information, including: Based on the video sequence, a lightweight deep learning optical flow network is used to calculate the fine-scale optical flow, the mid-scale optical flow, and the macro-scale optical flow through scale transformation operations and optical flow calculation functions, respectively. The fine-scale optical flow, mid-scale optical flow, and macro-scale optical flow are fused using adaptive weights based on channel attention learning to obtain multi-scale optical flow information.

[0010] In one embodiment, a visual feature vector is obtained based on the multi-scale dynamic feature map, and a text feature vector is obtained based on the multi-scale semantic attribute library. The visual feature vector and the text feature vector are then semantically aligned across modalities using a contrastive learning loss, including: The multi-scale dynamic feature map is input into the visual encoder, and the visual feature vector is obtained by encoding through the visual encoder. The attribute relationship set in the multi-scale semantic attribute library is input into the text encoder, and the text feature vector is obtained by encoding through the text encoder; The contrastive learning loss function is optimized in a shared semantic space, and the visual feature vector and the text feature vector are semantically aligned across modalities using the contrastive learning loss function.

[0011] In one embodiment, a scene prototype is obtained by calculating and fusing a visual prototype and a semantic prototype based on the multi-scale semantic attribute library, and the parameters of the base model are optimized to generate a scene-specific model, including: Obtain the target scene and its corresponding support set samples, extract visual features from the support set samples, and calculate the visual prototype; Select a semantic description corresponding to the target scene from the multi-scale semantic attribute library, and encode and calculate the semantic prototype; The visual prototype and the semantic prototype are weighted and fused to generate a scene prototype of the target scene; Based on the scene prototype, the parameters of the base model are optimized using a prompting fine-tuning technique to generate a scene-specific model.

[0012] In one embodiment, a real-time video sequence is acquired and a real-time multi-scale dynamic feature map is calculated. A real-time visual feature vector is determined using a visual encoder in the scene-specific model. The cosine similarity between the real-time visual feature vector and the scene prototype is calculated. Real-time fire detection is performed based on the cosine similarity, including: The real-time video sequence of the fire to be detected is obtained, and multi-scale dynamic features are extracted frame by frame from the real-time video sequence to obtain a real-time multi-scale dynamic feature map. The real-time multi-scale dynamic feature map is mapped into a real-time visual feature vector using the visual encoder in the scene-specific model. Calculate the cosine similarity between the real-time visual feature vector and the scene prototype, and determine the dynamic decision threshold; The cosine similarity is compared with the dynamic decision threshold. If the cosine similarity is greater than the dynamic decision threshold, the detection result is a fire.

[0013] In one embodiment, the method further includes: Define a confidence interval. When the cosine similarity is within the confidence interval, add the real-time multi-scale dynamic feature map to the hard sample set. The scenario prototype and scenario-specific model are periodically optimized using the difficult sample set.

[0014] An adaptive multi-scale dynamic fire detection system with few-sample learning, the system comprising: The attribute library construction module is used to collect fire data, determine the atomic attribute set, scale context set, and attribute relationship set based on the fire data, and construct a multi-scale semantic attribute library based on the atomic attribute set, scale context set, and attribute relationship set. The feature extraction module is used to acquire the video sequence to be analyzed, perform multi-scale dynamic feature extraction on the video sequence, and generate a multi-scale dynamic feature map that integrates color, temporal motion and multi-scale optical flow information. The semantic alignment module is used to obtain visual feature vectors based on the multi-scale dynamic feature map, obtain text feature vectors based on the multi-scale semantic attribute library, and perform cross-modal semantic alignment of the visual feature vectors and the text feature vectors through contrastive learning loss to build a pre-trained basic model. The Few-Shot adaptation module is used to calculate and fuse the visual prototype and semantic prototype based on the multi-scale semantic attribute library to obtain the scene prototype, optimize the parameters of the basic model, and generate a scene-specific model. The real-time detection module is used to acquire real-time video sequences and calculate real-time multi-scale dynamic feature maps, use the visual encoder in the scene-specific model to determine real-time visual feature vectors, calculate the cosine similarity between the real-time visual feature vectors and the scene prototype, and perform real-time fire detection based on the cosine similarity.

[0015] The aforementioned adaptive multi-scale dynamic fire detection method and system based on few-shot learning can accurately capture dynamic characteristics across all scales, from microscopic flickering to macroscopic spread, by extracting multi-scale optical flow information, significantly improving the detection rate of flames and smoke. By generating multi-scale dynamic feature maps and combining them with contrastive learning for model training, scene prototypes and scene-specific models are constructed, overcoming the limitations of single-scale models and enabling efficient detection of both small nearby fire sources and large-scale fires in the distance. Attached Figure Description

[0016] Figure 1 This is a diagram illustrating the application environment of an adaptive multi-scale dynamic fire detection method based on few-sample learning in one embodiment. Figure 2 This is a flowchart illustrating an adaptive multi-scale dynamic fire detection method based on few-sample learning in one embodiment. Figure 3 This is a schematic diagram of the multi-scale dynamic feature extraction process in one embodiment; Figure 4 This is a schematic diagram of the application system structure of an adaptive multi-scale dynamic fire detection method based on few-sample learning in one embodiment. Figure 5 This is a block diagram of an adaptive multi-scale dynamic fire detection system with few-sample learning in one embodiment. Figure 6 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0018] The fire detection method based on multi-scale dynamic perception and few-shot learning provided in this application embodiment can be applied to, for example... Figure 1 The application environment shown. For example... Figure 1As shown, the application environment includes computer equipment 110. Computer device 110 can collect fire data, determine atomic attribute sets, scale context sets, and attribute relationship sets based on the fire data, and construct a multi-scale semantic attribute library based on the atomic attribute sets, scale context sets, and attribute relationship sets; computer device 110 can acquire video sequences to be analyzed, perform multi-scale dynamic feature extraction on the video sequences, and generate multi-scale dynamic feature maps that fuse color, temporal motion, and multi-scale optical flow information; computer device 110 can obtain visual feature vectors based on the multi-scale dynamic feature maps and text feature vectors based on the multi-scale semantic attribute library, and perform cross-modal semantic alignment of the visual feature vectors and text feature vectors through contrastive learning loss to construct a pre-trained basic model; computer device 110 can calculate and fuse visual prototypes and semantic prototypes according to the multi-scale semantic attribute library to obtain a scene prototype, optimize the parameters of the basic model, and generate a scene-specific model; computer device 110 can acquire real-time video sequences and calculate real-time multi-scale dynamic feature maps, use the visual encoder in the scene-specific model to determine real-time visual feature vectors, calculate the cosine similarity between the real-time visual feature vectors and the scene prototypes, and perform real-time fire detection based on the cosine similarity. Among them, computer equipment 110 may include, but is not limited to, various personal computers, laptops, smartphones, robots, unmanned aerial vehicles, tablets, and other devices.

[0019] In one embodiment, such as Figure 2 As shown, an adaptive multi-scale dynamic fire detection method based on few-shot learning is provided, including the following steps: Step 202: Collect fire data, determine the atomic attribute set, scale context set, and attribute relationship set based on the fire data, and construct a multi-scale semantic attribute library based on the atomic attribute set, scale context set, and attribute relationship set.

[0020] Computer devices can first collect datasets and prior knowledge in the field of fire in text form, define and collect atomic properties, and thus build a structured knowledge base.

[0021] In one embodiment, the provided adaptive multi-scale dynamic fire detection method based on few-shot learning may further include the process of constructing a multi-scale dynamic semantic attribute library. The specific process includes: defining static visual attributes and dynamic behavioral attributes based on fire data, and constructing an atomic attribute set by combining the static visual attributes and dynamic behavioral attributes; defining various observation scales based on fire data, and constructing a scale context set based on each observation scale; establishing a mapping function between scale and attribute, constructing a combination between scale and attribute, and generating semantic description text as an attribute relationship set.

[0022] Computer devices can define subsets of static visual attributes and subsets of dynamic behavioral attributes. The static visual attribute subset can be represented as: , Let represent the i-th static attribute, such as "orange-red" or "gray fluffy". A subset of dynamic behavioral attributes can be represented as: , Let j represent the j-th dynamic attribute, such as "blinking" or "diffusion". The resulting set of atomic attributes can be represented as: .

[0023] Computer devices can define three observation scales: fine, medium, and macro, and construct a set of scale contexts: .in, This indicates the fine observation scale, corresponding to the original resolution; This indicates the mid-level observation scale, corresponding to 1 / 2 resolution; This indicates the macroscopic observation scale, corresponding to 1 / 4 resolution.

[0024] Next, the computer device can define a set of attribute relationships. It contains a scale-attribute mapping function. This is used to generate a specific semantic description D. The defined set of attribute relationships can implement a scale-attribute mapping function. ; ; ; in This represents a string concatenation operation. The constructed multi-scale semantic attribute library can be represented as...

[0025] Step 204: Obtain the video sequence to be analyzed, perform multi-scale dynamic feature extraction on the video sequence, and generate a multi-scale dynamic feature map that integrates color, temporal motion and multi-scale optical flow information.

[0026] In one embodiment, the provided adaptive multi-scale dynamic fire detection method based on few-shot learning may further include a multi-scale dynamic feature extraction process, specifically including: for a video sequence, extracting the RGB color channels of key frames; calculating the frame difference of video frames in the video sequence using a three-frame difference method; based on the video sequence, calculating optical flow on a multi-scale spatial pyramid using a lightweight deep learning optical flow network to generate multi-scale optical flow information; and concatenating and fusing the RGB color channels and frame difference multi-scale optical flow information to generate a multi-scale dynamic feature map.

[0027] Computer equipment can acquire the raw video sequences that need to be analyzed. in, Let t represent the video image at frame t. Next, the computer device can normalize each frame: ,in Here are the mean and standard deviation of the dataset.

[0028] like Figure 3 As shown, the computer device can perform triple feature extraction and fusion on the input raw video sequence. The first step is to extract keyframes from the raw video sequence. The RGB color channels; then the frame difference is calculated. ,in, This refers to pixel-by-pixel absolute value calculation. To address the "hole" phenomenon in two-frame differences and extract a more complete internal region of the moving object, the computer can preferentially use the three-frame difference method to calculate the frame difference. Next, the computer can calculate optical flow on a multi-scale spatial pyramid, generating a scale-adaptive optical flow field. The RGB color channels, frame difference, and scale-adaptive optical flow field are stitched together to form a unified feature map, i.e., a multi-scale dynamic feature map. ;in This indicates a splicing operation along the channel dimension.

[0029] In one embodiment, the provided adaptive multi-scale dynamic fire detection method based on few-shot learning may further include a process of calculating multi-scale optical flow. The specific process includes: based on the video sequence, using a lightweight deep learning optical flow network, calculating fine-scale optical flow, mid-scale optical flow, and macro-scale optical flow respectively through scale transformation operations and optical flow calculation functions; fusing the fine-scale optical flow, mid-scale optical flow, and macro-scale optical flow through adaptive weights based on channel attention learning to obtain multi-scale optical flow information.

[0030] like Figure 3 As shown, computer devices can choose the lightweight deep learning optical flow network FastFlowNet based on their computing resources. FastFlowNet is a deep optical flow estimation network focused on high speed and high energy efficiency. Its core goal is to significantly reduce computational cost, model complexity, and inference time while maintaining accuracy comparable to state-of-the-art large models, making it well-suited for real-time operation on edge devices with limited computing resources. Its core idea is to replace the complex and time-consuming modules in traditional optical flow networks with carefully designed, efficient core components, achieving a balance between speed and accuracy.

[0031] Specifically, computer devices can define optical flow calculation functions: Define the scaling operation: This indicates scaling image I by a factor of s. In multi-scale optical flow, the calculation processes for fine-scale optical flow, meso-scale optical flow, and macro-scale optical flow are as follows: ; Finally, the fine-scale optical flow, the meso-scale optical flow, and the macro-scale optical flow are fused at multiple scales to obtain multi-scale optical flow information. The calculation formula can be expressed as: in These are the adaptive weights for each scale. The adaptive weights are calculated using channel attention-based learning.

[0032] Step 206: Obtain visual feature vectors based on multi-scale dynamic feature maps and text feature vectors based on multi-scale semantic attribute libraries. Use contrastive learning loss to perform cross-modal semantic alignment between visual feature vectors and text feature vectors to build a pre-trained basic model.

[0033] In one embodiment, the provided adaptive multi-scale dynamic fire detection method based on few-shot learning may further include a semantic alignment process, specifically including: inputting the multi-scale dynamic feature map into a visual encoder and encoding it to obtain a visual feature vector; inputting the attribute relationship set in the multi-scale semantic attribute library into a text encoder and encoding it to obtain a text feature vector; optimizing the contrastive learning loss function in a shared semantic space, and performing cross-modal semantic alignment between the visual feature vector and the text feature vector through the contrastive learning loss function.

[0034] The visual encoder used in this embodiment Based on the ViT-B / 16 architecture, the input channels have been modified to 6. ViT-B / 16 is a standard benchmark model within the Vision Transformer (ViT) architecture. Its core idea is to directly apply the Transformer architecture, used in natural language processing, to computer vision tasks, completely abandoning traditional convolutional operations. By treating images as a sequence of patches and leveraging the powerful global modeling capabilities of the pure Transformer architecture, ViT-B / 16, after pre-training on sufficient data (such as JFT-300M), achieved performance comparable to or even better than the state-of-the-art convolutional networks at the time on multiple image recognition benchmarks.

[0035] The text encoder used in this embodiment is: Based on the Transformer architecture (such as BERT). The core idea of ​​Transformer-based pre-trained text encoders is to build a "pre-training + fine-tuning" paradigm. It leverages the powerful context modeling capabilities of the Transformer architecture to compress linguistic knowledge into the model parameters through self-supervised pre-training on unlabeled big data, thereby forming a general and powerful foundation for text feature extraction.

[0036] Computer devices can input multi-scale dynamic feature maps F into a visual encoder. Obtain visual feature vectors Computer devices can also sample text descriptions from a multi-scale semantic attribute library K and input them into a text encoder. Obtain the text feature vector Next, by comparing the learning loss Closer matching within a shared semantic space By pushing away mismatched pairs and contrastively learning the loss function, the model is forced to understand the abstract concept of fire, ultimately outputting a pre-trained base model.

[0037] Specifically, during the comparative learning training process, it is assumed that there are N image-text pairs in the batch. Calculate the similarity matrix in The contrast loss function is ,in This refers to temperature hyperparameters.

[0038] Step 208: Calculate the visual prototype and semantic prototype based on the multi-scale semantic attribute library and fuse them to obtain the scene prototype. Optimize the parameters of the basic model to generate a scene-specific model.

[0039] In one embodiment, the provided adaptive multi-scale dynamic fire detection method using few-shot learning may further include a Few-Shot rapid adaptation process, specifically including: acquiring the target scene and its corresponding support set samples; extracting visual features from the support set samples to calculate a visual prototype; selecting a semantic description corresponding to the target scene from a multi-scale semantic attribute library and encoding and calculating a semantic prototype; performing weighted fusion of the visual prototype and the semantic prototype to generate a scene prototype of the target scene; and optimizing the parameters of the basic model based on the scene prototype using cue-based fine-tuning techniques to generate a scene-specific model. .

[0040] Few-Shot fast adaptation refers to a technique that allows a model to quickly learn and adapt to new tasks or domains using only a small number of samples. In this embodiment, the target scene and its corresponding support set samples can be represented as follows: in K is typically 3-10.

[0041] Computer devices can use a basic model Extracting support set samples Features and calculate its visual prototype ExtractFeatures is a multi-scale dynamic feature extraction method. Simultaneously, the computer device can obtain relevant semantic priors from the multi-scale dynamic semantic attribute library K and calculate the semantic prototype. in The relevant descriptions of the target scene are ultimately fused to obtain the scene prototype. in This is the fusion coefficient.

[0042] In this embodiment, the computer device can be based on a basic model. As the initial state, using support set samples Use prompt templates to quickly fine-tune the model and optimize parameters. Generate scenario-specific decision boundaries. Specifically, computer devices can define prompt template functions: The fine-tuning objective of the model can be expressed as: in This represents the cross-entropy loss.

[0043] Step 210: Obtain the real-time video sequence and calculate the real-time multi-scale dynamic feature map. Use the visual encoder in the scene-specific model to determine the real-time visual feature vector and calculate the cosine similarity between the real-time visual feature vector and the scene prototype. Perform real-time fire detection based on the cosine similarity.

[0044] In one embodiment, the provided adaptive multi-scale dynamic fire detection method based on few-shot learning may further include an online adaptive fire detection process, specifically including: acquiring a real-time video sequence of the fire to be detected, and extracting multi-scale dynamic features frame by frame from the real-time video sequence to obtain a real-time multi-scale dynamic feature map; mapping the real-time multi-scale dynamic feature map to a real-time visual feature vector using a visual encoder in a scene-specific model; calculating the cosine similarity between the real-time visual feature vector and the scene prototype, and determining a dynamic decision threshold; comparing the cosine similarity with the dynamic decision threshold, and if the cosine similarity is greater than the dynamic decision threshold, the detection result is a fire.

[0045] Computer equipment can acquire real-time video sequences to be analyzed and detected. For each frame in the real-time video sequence, Multi-scale dynamic feature extraction can be performed to generate real-time multi-scale dynamic feature maps. Next, computer devices can use scene-specific models. The visual encoder in the middle provides real-time multi-scale dynamic feature maps Mapped to feature vectors Then calculate the eigenvectors. With scene prototype The cosine similarity is calculated using the following formula: Computer devices can be configured with decision functions: ,in This is a dynamic decision threshold that can be adaptively adjusted based on the complexity of the scenario. If If the dynamic decision threshold is exceeded, it is determined to be a fire. Ultimately, the computer equipment outputs a binary detection result. and confidence level .

[0046] In one embodiment, the provided adaptive multi-scale dynamic fire detection method based on few-shot learning may further include an online learning process, specifically including: defining a confidence interval; adding real-time multi-scale dynamic feature maps to the hard sample set when the cosine similarity is within the confidence interval; and periodically optimizing the scene prototype and scene-specific model using the hard sample set.

[0047] Computer devices can define confidence intervals: when When this happens, the sample is added to the hard sample set. Regular updates: And re-execute the Few-Shot quick adaptation process.

[0048] In one embodiment, a few-sample learning adaptive multi-scale dynamic fire detection method is provided that can be applied to applications such as... Figure 4 The system architecture shown mainly includes Phase 1: Basic pre-training; Phase 2: Few-Shot adaptation and inference. Specifically, as... Figure 4 As shown, the basic pre-training stage may include: Step S1: Constructing a multi-scale dynamic semantic attribute library K; Step S2: Multi-scale dynamic feature extraction to output feature maps F; Step S3: Visual-language semantic alignment training to output a base model M_base. The Few-Shot adaptation and inference stage may include: Step S4: Few-Shot fast adaptation to output the target model M_target and the prototype p_target; Step S5: Online adaptive detection.

[0049] In this embodiment, multi-scale optical flow analysis can accurately capture dynamic characteristics across all scales, from microscopic flickering to macroscopic spread, significantly improving the detection rate of flames and smoke. It overcomes the limitations of single-scale models, enabling simultaneous and efficient detection of small nearby fire sources and large-scale fires in the distance, especially further improving detection performance in complex multi-scale scenarios. With the help of few-sample learning, new scene adaptation requires only a small number of samples and a few minutes, greatly reducing annotation costs. By integrating dynamic semantic prior knowledge, the model has stronger resistance to changes in illumination, partial occlusion, and complex background interference.

[0050] It should be understood that although the steps in the flowchart above are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart above may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0051] In one embodiment, such as Figure 5 As shown, an adaptive multi-scale dynamic fire detection system based on few-shot learning is provided, comprising: an attribute library construction module 510, a feature extraction module 520, a semantic alignment module 530, a few-shot adaptation module 540, and a real-time detection module 550, wherein:

[0052] The attribute library construction module 510 is used to collect fire data, determine the atomic attribute set, scale context set, and attribute relationship set based on the fire data, and construct a multi-scale semantic attribute library based on the atomic attribute set, scale context set, and attribute relationship set.

[0053] The feature extraction module 520 is used to acquire the video sequence to be analyzed, perform multi-scale dynamic feature extraction on the video sequence, and generate a multi-scale dynamic feature map that integrates color, temporal motion and multi-scale optical flow information.

[0054] The semantic alignment module 530 is used to obtain visual feature vectors based on multi-scale dynamic feature maps and text feature vectors based on multi-scale semantic attribute libraries. It performs cross-modal semantic alignment of visual feature vectors and text feature vectors through contrastive learning loss to build a pre-trained basic model.

[0055] The Few-Shot adaptation module 540 is used to calculate and fuse visual and semantic prototypes based on a multi-scale semantic attribute library to obtain a scene prototype, optimize the parameters of the basic model, and generate a scene-specific model.

[0056] The real-time detection module 550 is used to acquire real-time video sequences and calculate real-time multi-scale dynamic feature maps. It uses the visual encoder in the scene-specific model to determine the real-time visual feature vector and calculates the cosine similarity between the real-time visual feature vector and the scene prototype. Based on the cosine similarity, it performs real-time fire detection.

[0057] In one embodiment, the attribute library construction module 510 is further configured to define static visual attributes and dynamic behavioral attributes based on fire data, and to construct an atomic attribute set by combining the static visual attributes and dynamic behavioral attributes; define various observation scales based on fire data, and construct a scale context set based on each observation scale; establish a mapping function between scales and attributes, construct a combination between scales and attributes, and generate semantic description text as an attribute relationship set.

[0058] In one embodiment, the feature extraction module 520 is further configured to extract the RGB color channels of key frames from the video sequence; calculate the frame difference of video frames in the video sequence using the three-frame difference method; calculate optical flow on a multi-scale spatial pyramid based on the video sequence using a lightweight deep learning optical flow network to generate multi-scale optical flow information; and splice and fuse the RGB color channels and frame difference multi-scale optical flow information to generate a multi-scale dynamic feature map.

[0059] In one embodiment, the feature extraction module 520 is further configured to calculate fine-scale optical flow, mid-scale optical flow, and macro-scale optical flow based on the video sequence using a lightweight deep learning optical flow network through scale transformation operations and optical flow calculation functions; and to fuse the fine-scale optical flow, mid-scale optical flow, and macro-scale optical flow through adaptive weights based on channel attention learning to obtain multi-scale optical flow information.

[0060] In one embodiment, the semantic alignment module 530 is further configured to input a multi-scale dynamic feature map into a visual encoder, and encode it to obtain a visual feature vector; input the attribute relationship set in the multi-scale semantic attribute library into a text encoder, and encode it to obtain a text feature vector; optimize the contrastive learning loss function in the shared semantic space, and semantically align the visual feature vector with the text feature vector through the contrastive learning loss function.

[0061] In one embodiment, the Few-Shot adaptation module 540 is further configured to acquire the target scene and its corresponding support set samples, extract visual features from the support set samples, and calculate a visual prototype; select a semantic description corresponding to the target scene from a multi-scale semantic attribute library, encode and calculate a semantic prototype; perform weighted fusion of the visual prototype and the semantic prototype to generate a scene prototype of the target scene; and optimize the parameters of the basic model based on the scene prototype using a prompting fine-tuning technique to generate a scene-specific model.

[0062] In one embodiment, the real-time detection module 550 is further configured to acquire a real-time video sequence of the fire to be detected, and extract multi-scale dynamic features frame by frame from the real-time video sequence to obtain a real-time multi-scale dynamic feature map; use a visual encoder in the scene-specific model to map the real-time multi-scale dynamic feature map into a real-time visual feature vector; calculate the cosine similarity between the real-time visual feature vector and the scene prototype, and determine the dynamic decision threshold; compare the cosine similarity with the dynamic decision threshold, and if the cosine similarity is greater than the dynamic decision threshold, the detection result is a fire.

[0063] In one embodiment, the Few-Shot adaptation module 540 is also used to define a confidence interval, and when the cosine similarity is within the confidence interval, the real-time multi-scale dynamic feature map is added to the hard sample set; the hard sample set is used periodically to optimize the scene prototype and scene-specific model.

[0064] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 6 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a few-sample learning adaptive multi-scale dynamic fire detection method. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device casing, or an external keyboard, touchpad, or mouse.

[0065] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0066] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement steps of an adaptive multi-scale dynamic fire detection method based on few-sample learning.

[0067] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program being executed by a processor to implement steps of an adaptive multi-scale dynamic fire detection method based on few-sample learning.

[0068] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0069] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0070] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A few-shot learning adaptive multi-scale dynamic fire detection method, characterized in that, The method includes: Collect fire data, determine the atomic attribute set, scale context set, and attribute relationship set based on the fire data, and construct a multi-scale semantic attribute library based on the atomic attribute set, scale context set, and attribute relationship set; The video sequence to be analyzed is obtained, and multi-scale dynamic features are extracted from the video sequence to generate a multi-scale dynamic feature map that integrates color, temporal motion and multi-scale optical flow information. Visual feature vectors are obtained based on the multi-scale dynamic feature map, and text feature vectors are obtained based on the multi-scale semantic attribute library. The visual feature vectors and text feature vectors are then semantically aligned across modalities using contrastive learning loss to construct a pre-trained basic model. The visual prototype and semantic prototype are calculated and fused according to the multi-scale semantic attribute library to obtain the scene prototype. The parameters of the basic model are optimized to generate a scene-specific model. A real-time video sequence is acquired and a real-time multi-scale dynamic feature map is calculated. The visual encoder in the scene-specific model is used to determine the real-time visual feature vector, and the cosine similarity between the real-time visual feature vector and the scene prototype is calculated. Real-time fire detection is performed based on the cosine similarity.

2. The adaptive multi-scale dynamic fire detection method based on few-shot learning according to claim 1, characterized in that, Based on the fire data, the set of atomic attributes, the set of scale contexts, and the set of attribute relationships are determined, including: Based on the fire data, static visual attributes and dynamic behavioral attributes are defined, and the static visual attributes and dynamic behavioral attributes are combined into an atomic attribute set; Each observation scale is defined based on the fire data, and a scale context set is constructed based on each observation scale; Establish a mapping function between scale and attribute, construct a combination between scale and attribute, and generate semantic description text as a set of attribute relationships.

3. The adaptive multi-scale dynamic fire detection method based on few-shot learning according to claim 1, characterized in that, Multi-scale dynamic feature extraction is performed on the video sequence to generate a multi-scale dynamic feature map that integrates color, temporal motion, and multi-scale optical flow information, including: For the video sequence, the RGB color channels of the keyframes are extracted; The frame difference between video frames in the video sequence is calculated using the three-frame difference method; Based on the video sequence, a lightweight deep learning optical flow network is used to calculate optical flow on a multi-scale spatial pyramid to generate multi-scale optical flow information; The RGB color channels and frame difference multi-scale optical flow information are spliced ​​and fused to generate a multi-scale dynamic feature map.

4. The adaptive multi-scale dynamic fire detection method based on few-shot learning according to claim 3, characterized in that, Based on the video sequence, a lightweight deep learning optical flow network is used to calculate optical flow on a multi-scale spatial pyramid to generate multi-scale optical flow information, including: Based on the video sequence, a lightweight deep learning optical flow network is used to calculate the fine-scale optical flow, the mid-scale optical flow, and the macro-scale optical flow through scale transformation operations and optical flow calculation functions, respectively. The fine-scale optical flow, mid-scale optical flow, and macro-scale optical flow are fused using adaptive weights based on channel attention learning to obtain multi-scale optical flow information.

5. The adaptive multi-scale dynamic fire detection method based on few-shot learning according to claim 1, characterized in that, A visual feature vector is obtained based on the multi-scale dynamic feature map, and a text feature vector is obtained based on the multi-scale semantic attribute library. Cross-modal semantic alignment of the visual feature vector and the text feature vector is performed using a contrastive learning loss, including: The multi-scale dynamic feature map is input into the visual encoder, and the visual feature vector is obtained by encoding through the visual encoder. The attribute relationship set in the multi-scale semantic attribute library is input into the text encoder, and the text feature vector is obtained by encoding through the text encoder; The contrastive learning loss function is optimized in a shared semantic space, and the visual feature vector and the text feature vector are semantically aligned across modalities using the contrastive learning loss function.

6. The adaptive multi-scale dynamic fire detection method based on few-shot learning according to claim 1, characterized in that, Based on the multi-scale semantic attribute library, a visual prototype and a semantic prototype are calculated and fused to obtain a scene prototype. The parameters of the basic model are optimized to generate a scene-specific model, including: Obtain the target scene and its corresponding support set samples, extract visual features from the support set samples, and calculate the visual prototype; Select a semantic description corresponding to the target scene from the multi-scale semantic attribute library, and encode and calculate the semantic prototype; The visual prototype and the semantic prototype are weighted and fused to generate a scene prototype of the target scene; Based on the scene prototype, the parameters of the base model are optimized using a prompting fine-tuning technique to generate a scene-specific model.

7. The adaptive multi-scale dynamic fire detection method based on few-shot learning according to claim 1, characterized in that, Acquire real-time video sequences and calculate real-time multi-scale dynamic feature maps; use the visual encoder in the scene-specific model to determine real-time visual feature vectors; calculate the cosine similarity between the real-time visual feature vectors and the scene prototype; and perform real-time fire detection based on the cosine similarity, including: The real-time video sequence of the fire to be detected is obtained, and multi-scale dynamic features are extracted frame by frame from the real-time video sequence to obtain a real-time multi-scale dynamic feature map. The real-time multi-scale dynamic feature map is mapped into a real-time visual feature vector using the visual encoder in the scene-specific model. Calculate the cosine similarity between the real-time visual feature vector and the scene prototype, and determine the dynamic decision threshold; The cosine similarity is compared with the dynamic decision threshold. If the cosine similarity is greater than the dynamic decision threshold, the detection result is a fire.

8. The adaptive multi-scale dynamic fire detection method based on few-shot learning according to claim 7, characterized in that, The method further includes: Define a confidence interval. When the cosine similarity is within the confidence interval, add the real-time multi-scale dynamic feature map to the hard sample set. The scenario prototype and scenario-specific model are periodically optimized using the difficult sample set.

9. A few-sample learning adaptive multi-scale dynamic fire detection system, characterized in that, The system includes: The attribute library construction module is used to collect fire data, determine the atomic attribute set, scale context set, and attribute relationship set based on the fire data, and construct a multi-scale semantic attribute library based on the atomic attribute set, scale context set, and attribute relationship set. The feature extraction module is used to acquire the video sequence to be analyzed, perform multi-scale dynamic feature extraction on the video sequence, and generate a multi-scale dynamic feature map that integrates color, temporal motion and multi-scale optical flow information. The semantic alignment module is used to obtain visual feature vectors based on the multi-scale dynamic feature map, obtain text feature vectors based on the multi-scale semantic attribute library, and perform cross-modal semantic alignment of the visual feature vectors and the text feature vectors through contrastive learning loss to build a pre-trained basic model. The Few-Shot adaptation module is used to calculate and fuse the visual prototype and semantic prototype based on the multi-scale semantic attribute library to obtain the scene prototype, optimize the parameters of the basic model, and generate a scene-specific model. The real-time detection module is used to acquire real-time video sequences and calculate real-time multi-scale dynamic feature maps, use the visual encoder in the scene-specific model to determine real-time visual feature vectors, calculate the cosine similarity between the real-time visual feature vectors and the scene prototype, and perform real-time fire detection based on the cosine similarity.

Images