A video abnormal flame detection method, device, equipment and medium

Abstract

The application provides a video abnormal flame detection method, device and equipment and a medium, and relates to the technical field of computer vision processing. The method comprises the following steps: establishing an initial detection model, training a target model; obtaining video data to be identified; in the target model, obtaining a first feature vector set through a first encoder according to N frame data, and obtaining reconstruction data through image reconstruction; calculating a reconstruction loss; performing feature extraction on the reconstruction data to obtain a second feature vector set, and calculating a feature distribution alignment loss; inputting the Nth frame data and the reconstruction data into a discriminator to obtain an adversarial loss; outputting an abnormality judgment result according to the weighted sum of the reconstruction loss, the feature distribution alignment loss and the adversarial loss; segmenting the first feature vector set to obtain an identification result containing a flame area; and obtaining a target result based on the abnormality judgment result and the identification result. The application solves the problem of low accuracy of the abnormal flame detection result in the prior art.

Description

Technical Field

[0001] This invention relates to the field of computer vision processing technology, and in particular to a method, apparatus, device, and medium for detecting abnormal flames in video. Background Technology

[0002] Fires have caused significant loss of life and property to human society. With the application of information technology and intelligent technologies in the security field, intelligent security technologies effectively assist in fire prevention and post-disaster emergency response. For example, remote sensing and image recognition technologies enable large-scale, wide-field, and 24 / 7 real-time monitoring of forest areas, effectively preventing and controlling forest fires in real time. Intelligent monitoring equipment using smoke, light, temperature, and video images, combined with computer processing power and intelligent analysis algorithms, identifies and prevents fires in office buildings, factory workshops, and residential buildings, minimizing fire safety hazards.

[0003] Early fire detection typically involved installing sensor devices to monitor characteristics such as smoke, high temperatures, and bright light produced by a fire. However, limitations in device height and coverage resulted in limited detection range. Adding more equipment to cover multiple scenarios inevitably increased costs. Furthermore, hardware-based flame detection methods suffer from a lag in flame detection, only confirming a fire when it reaches a certain stage. Moreover, the coverage of this hardware is limited, leading to extremely high costs when a wide area needs to be covered.

[0004] Existing flame detection methods utilize the channel and edge features of flames for identification, but they heavily rely on manually designed features, resulting in weak anti-interference capabilities, poor adaptability, and a high likelihood of false negatives and false negatives. In recent years, several deep learning-based flame detection solutions have emerged. However, training typical convolutional neural networks requires a large amount of training data, and the samples in the training data significantly impact the model's robustness. Furthermore, given the motion characteristics of flames and smoke in real-world scenarios, relying solely on single-frame detection yields limited accuracy, and current models cannot achieve real-time performance. Methods involving cross-frame recognition lose the inter-frame correlation, further affecting the final flame recognition rate. Summary of the Invention

[0005] In order to overcome the above-mentioned technical defects, the purpose of this invention is to provide a video abnormal flame detection method, device, equipment and medium, and to solve the problem of low accuracy of abnormal flame detection results in the prior art.

[0006] This invention discloses a method for detecting abnormal flames in videos, comprising:

[0007] An initial detection model is established, and the initial detection model is trained using training samples to obtain the target model;

[0008] Acquire the video data to be identified and input it into the target model, wherein the video data to be identified includes N consecutive frame data;

[0009] In the target model, a first feature vector set is obtained through a first encoder based on N frames of data, and the image is reconstructed through a first decoder based on the first feature vector set to obtain reconstructed data.

[0010] Calculate the reconstruction loss based on the Nth frame data and the reconstructed data;

[0011] The reconstructed data is subjected to feature extraction by a second encoder to obtain a second feature vector set, and the feature distribution alignment loss is calculated based on the first feature vector set and the second feature vector set.

[0012] The Nth frame data and the reconstructed data are input into the discriminator to obtain adversarial loss;

[0013] The anomaly determination result is output based on the weighted sum of the reconstruction loss, feature distribution alignment loss, and adversarial loss.

[0014] The first feature vector set is segmented using a second decoder to obtain an identification result that includes the flame area;

[0015] Based on the anomaly determination result and the identification result, a target result including whether an abnormal flame exists is obtained.

[0016] Preferably, training the initial detection model using training samples to obtain the target model includes:

[0017] The training samples include a first training set for anomaly detection and a second training set for image segmentation;

[0018] During the training process, a first training set is used in advance to perform a first training process to fix the parameters of the first encoder, the second encoder, the first decoder, and the discriminator;

[0019] After fixing the parameters of the first encoder, a second training process is performed using the second training set to adjust the parameters of the second decoder until the model converges, and then the parameters of the second decoder are fixed.

[0020] The first encoder, second encoder, first decoder, discriminator, and second decoder with fixed parameters are fused to obtain the target model.

[0021] Preferably, during the first training process, the weights of the reconstruction loss, feature alignment loss, and adversarial loss are adjusted until the model converges, thereby fixing the weights of the reconstruction loss, feature alignment loss, and adversarial loss.

[0022] Preferably, the step of calculating the feature distribution alignment loss based on the first feature vector set and the second feature vector set includes:

[0023] The feature distribution alignment loss is calculated using KL divergence based on the first feature vector set and the second feature vector set.

[0024] Preferably, the step of outputting anomaly determination results based on the weighted sum of the reconstruction loss, feature distribution alignment loss, and adversarial loss includes:

[0025] The anomaly score is determined by a weighted sum of the reconstruction loss, feature distribution alignment loss, and adversarial loss.

[0026] Determine whether the abnormal score exceeds a preset threshold to determine the abnormality determination result.

[0027] Preferably, the step of segmenting the first feature vector set using a second decoder to obtain a recognition result including the flame area includes:

[0028] The first feature vector set is segmented using a second decoder to obtain the segmentation result.

[0029] Based on the proportion of pixel values ​​corresponding to the flame region in the segmentation results, the flame area is determined by mapping, and the recognition result is obtained.

[0030] Preferably, the first encoder and the second encoder have the same structure or are a shared encoder;

[0031] The first encoder and / or the second encoder employs a ResNet18 network.

[0032] The present invention also provides a video abnormal flame detection device, comprising:

[0033] The model building module is used to build an initial detection model, and to train the initial detection model using training samples to obtain the target model;

[0034] An input module is used to acquire video data to be identified and input it into the target model, wherein the video data to be identified includes N consecutive frame data;

[0035] An anomaly detection module is used to obtain a first feature vector set based on N frames of data through a first encoder, and to reconstruct the image based on the first feature vector set through a first decoder to obtain reconstructed data.

[0036] Calculate the reconstruction loss based on the Nth frame data and the reconstructed data;

[0037] The reconstructed data is subjected to feature extraction by a second encoder to obtain a second feature vector set, and the feature distribution alignment loss is calculated based on the first feature vector set and the second feature vector set.

[0038] The Nth frame data and the reconstructed data are input into the discriminator to obtain adversarial loss;

[0039] The anomaly determination result is output based on the weighted sum of the reconstruction loss, feature distribution alignment loss, and adversarial loss.

[0040] The image recognition module is used to perform image segmentation on the first feature vector set using a second decoder to obtain a recognition result containing the flame area;

[0041] The processing module is used to calculate a target result containing whether an abnormal flame exists, based on the anomaly determination result and the identification result.

[0042] The present invention also provides a computer device, the computer device comprising:

[0043] Memory, used to store executable program code; and

[0044] A processor is configured to call the executable program code in the memory and execute steps including the video abnormal flame detection method.

[0045] The present invention also provides a computer-readable storage medium having a computer program stored thereon.

[0046] When the computer program is executed by the processor, it implements the steps of the video abnormal flame detection method.

[0047] Compared with existing technologies, the above technical solution has the following advantages:

[0048] In this application, video flame detection is performed using the temporal context information of continuous frame data in the video. Based on the flame anomaly detection with a network structure of a first encoder, a first decoder, a second encoder, and a discriminator, and combined with image segmentation formed by a network structure of a first encoder and a second decoder, video flame target detection is performed, and anomaly judgment results and identification results including the flame area are output. This makes the model more accurate in recognizing abnormal flames or smoke scenes and improves the accuracy of abnormal flame detection results. Attached Figure Description

[0049] Figure 1 This is a flowchart of an embodiment of the video abnormal flame detection method of the present invention;

[0050] Figure 2 This is a schematic diagram of the target model network structure of an embodiment of the video abnormal flame detection method of the present invention;

[0051] Figure 3 This is a schematic diagram of a module in a second embodiment of the video abnormal flame detection device of the present invention;

[0052] Figure 4 This is a schematic diagram of the modules of the computer device described in this invention.

[0053] Figure label:

[0054] 10-Video abnormal flame detection device; 101-Model building module; 102-Input module; 103-Abnormal detection module; 104-Image detection module; 105-Processing module; 11-Computer equipment; 111-Memory; 112-Processor. Detailed Implementation

[0055] The advantages of the present invention will be further illustrated below with reference to the accompanying drawings and specific embodiments.

[0056] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0057] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The singular forms “a,” “the,” and “the” as used in this disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0058] It should be understood that although the terms first, second, third, etc., may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are used only to distinguish information of the same type from one another. For example, without departing from the scope of this disclosure, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0059] In the description of this invention, it should be understood that the terms "longitudinal", "lateral", "up", "down", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0060] In the description of this invention, unless otherwise specified and limited, it should be noted that the terms "installation", "connection" and "linking" should be interpreted broadly. For example, they can refer to mechanical or electrical connections, or internal connections between two components. They can be direct connections or indirect connections through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms according to the specific circumstances.

[0061] In the following description, suffixes such as "module," "part," or "unit" used to denote elements are used only for the convenience of the description of the invention and have no specific meaning in themselves. Therefore, "module" and "part" can be used interchangeably.

[0062] Example 1: This example discloses a method for detecting abnormal flames in video, including the following steps:

[0063] S100: Establish an initial detection model, train the initial detection model using training samples, and obtain the target model;

[0064] In this embodiment, the initial detection module has the same structural configuration as the target model described below. The training process is used to update the model parameters. Specifically, the initial detection module also includes the first encoder, second encoder, first decoder, discriminator, and second decoder in the target model. Optionally, the first encoder and the second encoder have the same structure or are a shared encoder; the first encoder and / or the second encoder adopt a ResNet18 network. For example, it can be a ResNet-18 network containing Conv-LSTM based on RNN improvement.

[0065] S200: Acquire the video data to be identified and input it into the target model, wherein the video data to be identified includes N consecutive frame data;

[0066] In this embodiment, flame detection is applied in video analysis scenarios. Since flames and smoke have motion characteristics in actual situations, the recognition accuracy is very limited if only single-frame detection of image frames is used. Furthermore, the method of cross-frame recognition loses the correlation between frames, which may affect the final flame recognition rate. Therefore, in this application, multiple consecutive frame data are used for flame recognition and anomaly determination. The Nth frame data is the most real-time frame data at present.

[0067] S300: In the target model, a first feature vector set (A) is obtained through a first encoder based on N frames of data, and the image is reconstructed through a first decoder based on the first feature vector set to obtain reconstructed data;

[0068] In the above steps, deep feature extraction is performed using the first decoder, and then the most real-time frame data is reconstructed based on the extracted features. That is, the input includes the previous N frames, including the current frame ~Fn. After the deep feature vector is obtained by the encoder, the original image ~Fn of the current frame is reconstructed based on the vector.

[0069] S400: Calculate the reconstruction loss based on the Nth frame data and the reconstructed data;

[0070] In the above steps, the reconstruction loss is the loss between the Nth frame data and the reconstructed data, i.e., the loss between ~Fn and Fn. Specifically, the loss between the two feature images can be calculated using existing loss functions or loss calculation methods, including but not limited to cross-entropy loss algorithms, to concretely represent the difference between the Nth frame data and the reconstructed data.

[0071] S500: The reconstructed data is subjected to feature extraction by the second encoder to obtain a second feature vector set (B), and the feature distribution alignment loss is calculated based on the first feature vector set and the second feature vector set;

[0072] Specifically, the step of calculating the feature distribution alignment loss based on the first feature vector set and the second feature vector set includes: using KL divergence to calculate the feature distribution alignment loss based on the first feature vector set and the second feature vector set.

[0073] After step S400 above, the image is reconstructed by passing it through the same encoder (i.e., the second encoder can be shared with the first encoder) to obtain an encoded deep vector. The feature distribution alignment loss is calculated based on the two sets of deep vectors. In addition to the KL divergence calculation method, other existing algorithms can also be used to calculate the loss between the feature distributions of the two sets of vectors.

[0074] S600: Input the Nth frame data and the reconstructed data into the discriminator to obtain adversarial loss;

[0075] In the above steps, the input to the discriminator is the reconstructed frame ~Fn and the original frame Fn, and the classic cross-entropy loss is used as the adversarial loss.

[0076] S700: Output the anomaly determination result based on the weighted sum of the reconstruction loss, feature distribution alignment loss, and adversarial loss;

[0077] The reconstruction loss, feature distribution alignment loss, and adversarial loss calculated in steps S400-S600 above are all used to reflect the difference between the reconstructed image (which can be regarded as an image predicted based on N frames of data) and the Nth frame of video acquisition data, and are used to detect whether there are any abnormalities in the Nth frame of data.

[0078] Specifically, the step of outputting anomaly determination results based on the weighted sum of the reconstruction loss, feature distribution alignment loss, and adversarial loss includes: determining an anomaly score based on the weighted sum of the reconstruction loss, feature distribution alignment loss, and adversarial loss; and determining whether the anomaly score exceeds a preset threshold to determine the anomaly determination result.

[0079] That is, during the inference phase, the weighted sum of the three loss functions is calculated to obtain an anomaly detection score. After setting a threshold, the anomaly of whether flames occur can be determined based on this score. It should be noted that the weights of the reconstruction loss, feature distribution alignment loss, and adversarial loss are determined through the training process.

[0080] S800: The first feature vector set is segmented using the second decoder to obtain the recognition result containing the flame area;

[0081] In this embodiment, step S700 is used to determine whether there are any abnormalities in the video frame data, in order to determine whether there is a flame. In order to further increase the accuracy of the recognition result, the solution provided in this application also performs image segmentation simultaneously to determine the flame area in the frame data. When the flame area > 0, there is an abnormal flame.

[0082] Specifically, the step of segmenting the first feature vector set using a second decoder to obtain a recognition result including the flame area includes:

[0083] S810: The first feature vector set is segmented using the second decoder to obtain the segmentation result (C);

[0084] In the above steps, a UNET-like encoder-decoder structure can be used. The input is a series of consecutive frames of images in time. The encoder-decoder containing RNN extracts features and segments the images to obtain the segmentation results.

[0085] S820: Based on the proportion of pixel values ​​corresponding to the flame region in the segmentation result, the flame area is determined by mapping, and the recognition result is obtained.

[0086] In the above steps, based on the mapping relationship between the pixel values ​​of the flame target in the above segmentation results and the actual environment, the area of ​​the flame is calculated. The fire severity level can be determined according to the area size, and the fire situation can be monitored in real time according to the flame area size, further improving the accuracy of abnormal flame detection results, so as to facilitate users to take corresponding measures.

[0087] S900: Based on the anomaly determination result and the identification result, obtain a target result including whether an abnormal flame exists.

[0088] During target model processing, the entire model extracts the first N frames based on N (i.e., time steps) for video analysis. Based on the anomaly detection branch, a judgment and confidence level are given regarding the presence of flames in the current frame (i.e., step S700 above). Based on the semantic segmentation branch, the identification result of the flame area in the current frame is obtained (i.e., step S800 above). Based on the image segmentation result and the anomaly detection result, a prediction of whether a fire exists in the current frame can be given. If it exists, the estimated area of ​​the fire can also be determined to facilitate the user in implementing corresponding strategies.

[0089] As explained, this application utilizes semantic segmentation technology to perform pixel-level prediction of images. Furthermore, since flames and smoke exhibit motion characteristics in real-world scenarios, relying solely on single-frame detection results in limited accuracy, and current models cannot achieve real-time performance. Cross-frame recognition methods lose the inter-frame correlation, affecting the final flame recognition rate. Therefore, several adjacent consecutive frames are used as input to segment the flame target (specifically, implemented through the first encoder and second decoder in the aforementioned model). In addition, this application treats scene flame target detection as a special type of abnormal event detection. That is, flames are considered as an abnormal target intruding into the normal monitoring environment. When a fire occurs, it is necessary to determine the existence of an "abnormal intrusion of flames" in the current scene and provide a confidence level for the fire occurrence. A flame anomaly detection model is set up (specifically implemented through the first encoder, second encoder, first decoder, and discriminator in the aforementioned model), combined with the aforementioned flame target segmentation, to improve the model's flame recognition accuracy and enhance its adaptability.

[0090] As described above, in this embodiment, the established initial detection model includes two operations: one for image segmentation and one for anomaly detection. These two operations are reflected in the model as the sharing of a portion of the module (the first encoder). Therefore, during training, the initial detection model can be trained directly, or the two operations can be trained separately and then fused. Specifically, training the initial detection model using training samples to obtain the target model includes:

[0091] The training samples include a first training set for anomaly detection and a second training set for image segmentation. During training, a first training process is performed using the first training set to fix the parameters of the first encoder, second encoder, first decoder, and discriminator. After fixing the parameters of the first encoder, a second training process is performed using the second training set to adjust the parameters of the second decoder until the model converges, and the parameters of the second decoder are fixed. The first encoder, second encoder, first decoder, discriminator, and second decoder with fixed parameters are fused to obtain the target model.

[0092] That is, the two operations of the initial detection model (or coefficient target model) mentioned above share the deep vector (i.e., the first vector set below) and the first encoder structure. The training method can be as follows: first train the flame anomaly detection model (i.e., the network formed by the first encoder, the first encoder, the second encoder and the discriminator mentioned above) until convergence, then fix the first encoder module or train the first encoder with a small learning rate (its parameters are regarded as fixed), and train the upsampling part of the segmentation model network (i.e., the second encoder part) normally.

[0093] Specifically, in the first training process described above, the weights of reconstruction loss, feature alignment loss, and adversarial loss in the target model processing process are adjusted until the model converges, so as to fix the weights of reconstruction loss, feature alignment loss, and adversarial loss. That is, the weights are adjusted according to different training data in the training samples so that the target model is actually used for processing in the above step S700 and the judgment score is output.

[0094] In this embodiment, video flame detection is performed using the temporal context information of the video. The flame anomaly detection model based on a generative adversarial network (i.e., the network formed by the first encoder, the first encoder, the second encoder, and the discriminator) enhances the network's sensitivity to flame and smoke targets, achieving a higher accuracy in fire prediction. Combined with semantic segmentation (i.e., the second encoder part) for video flame target detection, the model has a higher scene recognition rate for small target flames or smoke, improving the accuracy of abnormal flame detection results and providing the estimated area of ​​the flame, thus offering more detailed disaster information.

[0095] Example 2: This example also provides a video abnormal flame detection device 10, including:

[0096] The model building module 101 is used to build an initial detection model and train the initial detection model using training samples to obtain a target model;

[0097] Specifically, the initial detection module described above has the same structural configuration as the target model described below. The training process is used to update the model parameters. Specifically, the initial detection module also includes the first encoder, second encoder, first decoder, discriminator, and second decoder in the target model.

[0098] The input module 102 is used to acquire video data to be identified and input it into the target model, wherein the video data to be identified includes N consecutive frame data;

[0099] Specifically, in this embodiment, flame detection is applied to video analysis scenarios, and multiple consecutive frame data are used for flame recognition and anomaly determination.

[0100] An anomaly detection module 103 is configured to: obtain a first feature vector set based on N frames of data using a first encoder; reconstruct an image using a first decoder based on the first feature vector set to obtain reconstructed data; calculate a reconstruction loss based on the Nth frame of data and the reconstructed data; extract features from the reconstructed data using a second encoder to obtain a second feature vector set; calculate a feature distribution alignment loss based on the first and second feature vector sets; input the Nth frame of data and the reconstructed data into a discriminator to obtain an adversarial loss; and output an anomaly determination result based on a weighted sum of the reconstruction loss, the feature distribution alignment loss, and the adversarial loss.

[0101] The specific anomaly detection module is used to determine whether a flame exists in a current frame and the confidence level. That is, it reconstructs the Nth frame data based on N frames of data and compares it with the original Nth frame data. By checking whether the weighted sum of reconstruction loss, feature distribution alignment loss and adversarial loss exceeds the threshold, it determines whether an anomaly occurs in the Nth frame data, thereby determining whether an abnormal flame exists.

[0102] The image recognition module 104 is used to perform image segmentation on the first feature vector set using a second decoder to obtain a recognition result containing the flame area;

[0103] Specifically, the image recognition module employs an encoder-decoder structure (a first encoder and a second decoder). It takes multiple consecutive frames of images as input, extracts features (i.e., the aforementioned first feature vector set), and segments the image to obtain the segmentation result. Based on the mapping relationship between the pixel values ​​of the flame target in the segmentation result and the actual environment, the area of ​​the flame is calculated. This, combined with the aforementioned anomaly detection module, improves the accuracy of abnormal flame detection results.

[0104] The processing module 105 is used to calculate a target result containing whether an abnormal flame exists based on the anomaly determination result and the identification result.

[0105] Specifically, the processing module can predict whether there is an abnormal flame in the current scene based on the identification results (i.e., flame area) and the anomaly determination results. If there is, it can also determine the estimated area of ​​the fire to facilitate the user in implementing response strategies.

[0106] In this embodiment, an initial detection model is established through a model building module. The initial detection model is trained using training samples to obtain a target model. The video data to be identified is input into the input module of the target model. Then, in the target model, an anomaly detection module (with a network structure of a first encoder, a first decoder, a second encoder, and a discriminator) is used to reconstruct the Nth frame data based on N frames of data. The reconstruction loss, feature distribution alignment loss, and adversarial loss between the reconstructed data and the Nth frame data are calculated to obtain an anomaly judgment result (detection score). After setting a threshold, the presence of an anomaly such as flame occurrence can be determined based on this score. Then, an image recognition module (with a first decoder and a second decoder structure) is used to extract features and segment the image, mapping to calculate the flame area. Finally, the processing module determines the target result based on the anomaly judgment result and the flame area. If the flame area is 0, the anomaly judgment result is normal, and no fire has occurred. If the flame area is >0, the anomaly judgment result is abnormal, and an alert is issued. A corresponding strategy is formulated based on the flame area. By utilizing the temporal context information of videos for video flame detection, and combining a flame anomaly detection model based on generative adversarial networks with semantic segmentation for video flame target detection, the model achieves higher scene recognition rates for small target flames or smoke, thereby improving the accuracy of anomaly flame detection results.

[0107] Example 3: To achieve the above objectives, the present invention also provides a computer device 11, such as... Figure 4 As shown, the computer device can be a smartphone, tablet computer, laptop computer, desktop computer, etc., that executes programs. The computer device in this embodiment includes, but is not limited to, a memory 111 and a processor 112 that can be interconnected via a device bus, such as... Figure 4 As shown. It should be noted that, Figure 4Only computer devices with components are shown; however, it should be understood that implementation of all shown components is not required, and more or fewer components may be implemented instead.

[0108] In this embodiment, the memory 111 can be an internal storage unit of a computer device, such as the hard disk or memory of the computer device. In other embodiments, the memory 111 can also be an external storage device of the computer device, such as a plug-in hard disk equipped on the computer device. In this embodiment, the memory 111 is typically used to store operating devices and various application software installed on the computer device, such as the program code, training samples, images to be processed, and target results of a video abnormal flame detection method according to Embodiment 1. In addition, the memory 111 can also be used to temporarily store various types of data that have been output or will be output.

[0109] In some embodiments, processor 102 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. Processor 112 is typically used to control the overall operation of a computer device. In this embodiment, processor 112 is used to run program code stored in memory 111 or process data, for example, to run a video abnormal flame detection method according to Embodiment 1.

[0110] Example 4: To achieve the above objectives, the present invention also provides a computer-readable storage device, which includes multiple storage media, such as flash memory, hard disk, multimedia card, card-type memory (e.g., SD or D* memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, server, App application store, etc., storing computer programs thereon. When the program is executed by processor 112, it implements corresponding functions. In this embodiment, the computer-readable storage medium is used to store data, and when executed by processor 112, it implements a video abnormal flame detection method of Example 1 and a video abnormal flame detection device of Example 2.

[0111] It should be noted that the embodiments of the present invention have better implementability and are not intended to limit the present invention in any way. Any person skilled in the art may use the above-disclosed technical content to change or modify it into equivalent effective embodiments. However, any modifications or equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention shall still fall within the scope of the technical solution of the present invention.

Claims

1. A method for detecting abnormal flames in a video, characterized in that, include: An initial detection model is established, and the initial detection model is trained using training samples to obtain the target model; Acquire the video data to be identified and input it into the target model, wherein the video data to be identified includes N consecutive frame data; In the target model, a first feature vector set is obtained through a first encoder based on N frames of data, and the image is reconstructed through a first decoder based on the first feature vector set to obtain reconstructed data. Calculate the reconstruction loss based on the Nth frame data and the reconstructed data; The reconstructed data is subjected to feature extraction by a second encoder to obtain a second feature vector set, and the feature distribution alignment loss is calculated based on the first feature vector set and the second feature vector set. The Nth frame data and the reconstructed data are input into the discriminator to obtain adversarial loss; The anomaly determination result is output based on the weighted sum of the reconstruction loss, feature distribution alignment loss, and adversarial loss. The first feature vector set is segmented using a second decoder to obtain an identification result that includes the flame area; Based on the anomaly determination result and the identification result, a target result including whether an abnormal flame exists is obtained.

2. The detection method according to claim 1, characterized in that, The step of training the initial detection model using training samples to obtain the target model includes: The training samples include a first training set for anomaly detection and a second training set for image segmentation; During the training process, a first training set is used in advance to perform a first training process to fix the parameters of the first encoder, the second encoder, the first decoder, and the discriminator; After fixing the parameters of the first encoder, a second training process is performed using the second training set to adjust the parameters of the second decoder until the model converges, and then the parameters of the second decoder are fixed. The first encoder, second encoder, first decoder, discriminator, and second decoder with fixed parameters are fused to obtain the target model.

3. The detection method according to claim 2, characterized in that: During the first training process, the weights of the reconstruction loss, feature alignment loss, and adversarial loss are adjusted until the model converges, thereby fixing the weights of the reconstruction loss, feature alignment loss, and adversarial loss.

4. The detection method according to claim 1, characterized in that, The step of calculating the feature distribution alignment loss based on the first feature vector set and the second feature vector set includes: The feature distribution alignment loss is calculated using KL divergence based on the first feature vector set and the second feature vector set.

5. The detection method according to claim 1, characterized in that, The step of outputting anomaly determination results based on the weighted sum of the reconstruction loss, feature distribution alignment loss, and adversarial loss includes: The anomaly score is determined by a weighted sum of the reconstruction loss, feature distribution alignment loss, and adversarial loss. Determine whether the abnormal score exceeds a preset threshold to determine the abnormality determination result.

6. The detection method according to claim 1, characterized in that, The step of segmenting the first feature vector set using a second decoder to obtain a recognition result including the flame area includes: The first feature vector set is segmented using a second decoder to obtain the segmentation result. Based on the proportion of pixel values ​​corresponding to the flame region in the segmentation results, the flame area is determined by mapping, and the recognition result is obtained.

7. The detection method according to claim 1, characterized in that: The first encoder and the second encoder have the same structure or are a shared encoder; The first encoder and / or the second encoder employs a ResNet18 network.

8. A video-based abnormal flame detection device, characterized in that, include: The model building module is used to build an initial detection model, and to train the initial detection model using training samples to obtain the target model; An input module is used to acquire video data to be identified and input it into the target model, wherein the video data to be identified includes N consecutive frame data; An anomaly detection module is used to obtain a first feature vector set based on N frames of data through a first encoder, and to reconstruct the image based on the first feature vector set through a first decoder to obtain reconstructed data. Calculate the reconstruction loss based on the Nth frame data and the reconstructed data; The reconstructed data is subjected to feature extraction by a second encoder to obtain a second feature vector set, and the feature distribution alignment loss is calculated based on the first feature vector set and the second feature vector set. The Nth frame data and the reconstructed data are input into the discriminator to obtain adversarial loss; The anomaly determination result is output based on the weighted sum of the reconstruction loss, feature distribution alignment loss, and adversarial loss. The image recognition module is used to perform image segmentation on the first feature vector set using a second decoder to obtain a recognition result containing the flame area; The processing module is used to calculate a target result containing whether an abnormal flame exists, based on the anomaly determination result and the identification result.

9. A computer device, characterized in that: The computer device includes: Memory, used to store executable program code; and A processor is configured to invoke the executable program code in the memory to perform the steps including the video abnormal flame detection method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the video abnormal flame detection method according to any one of claims 1 to 7.

Images