A fire monitoring method, system, electronic device and storage medium

Abstract

The application discloses a fire monitoring method and system, an electronic device and a storage medium. The method is applied to the electronic device, specifically, a series of images obtained by image collection on a region to be monitored are filtered based on multi-dimensional features to obtain multiple image features of each image; the multiple image features are weighted and fused based on weights to obtain a comprehensive confidence of each image; the series of images are filtered based on the comprehensive confidence, and the images with a comprehensive confidence lower than a preset threshold are removed to obtain a series of effective images; and the series of effective images are recognized based on a multi-modal large model to obtain monitoring information. In the process of processing images obtained by a camera, the images are filtered based on multiple image features, and images containing strong reflection spots, light and the like which are easy to be misjudged as flames are filtered out in advance, so that false fire reports will not be caused in further recognition, and unnecessary troubles caused to production and life are avoided.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and more specifically, to a fire monitoring method, system, electronic device, and storage medium. Background Technology

[0002] Fire is a sudden, destructive, and widespread disaster. It can occur anytime and anywhere, including residential areas, industrial parks, forests and grasslands, historical buildings, and data centers. It not only causes enormous casualties and property damage but also harms the ecological environment, triggers secondary disasters, and poses a serious threat to public safety and economic development. With accelerated urbanization, an increase in high-risk environments, and frequent extreme weather events, the uncertainty and difficulty of fire prevention and control have further increased, placing higher demands on the timeliness, accuracy, and comprehensiveness of fire detection and forecasting. Fire detection, as the front-end of fire prevention and control, can quickly identify abnormal signals such as smoke, temperature rise, and characteristic gases in the early stages of a fire, issuing timely alarms and buying valuable time for evacuation and initial firefighting.

[0003] The occurrence and spread of fires exhibit distinct stages, from initial smoldering and overheating warnings to the spread of open flames, often within a specific time window. This window is crucial for minimizing casualties and property damage. However, accurate fire monitoring is paramount; false alarms can negatively impact daily life and production. Currently, conventional fire monitoring relies on visible / infrared cameras to capture images, which are then detected using appropriate flame detection models. However, traditional flame detection models are prone to misinterpreting strong reflective spots or lights as flames, leading to false alarms and unnecessary disruption to daily life and production. Summary of the Invention

[0004] In view of this, this application provides a fire monitoring method, system, electronic device and storage medium for monitoring fires to avoid false alarms.

[0005] To achieve the above objectives, the following solution is proposed: A fire monitoring method, applied to electronic equipment, the fire monitoring method comprising the following steps: Multidimensional feature filtering is performed on a series of images collected in the area to be monitored to obtain multiple image features for each image. The various image features are weighted and fused to obtain the overall confidence level of each image. The series of images are filtered based on the comprehensive confidence level. By removing images with a comprehensive confidence level lower than a preset threshold, a series of valid images are obtained. Based on a multimodal large model, the series of effective images are identified to obtain monitoring information.

[0006] Optionally, the multiple image features include some or all of the dynamic features, color features, morphological features, and geometric features.

[0007] Optionally, the step of performing multi-dimensional feature filtering on the series of images obtained from image acquisition of the area to be monitored to obtain multiple image features for each image includes the following steps: The dynamic features are obtained by detecting the pixel readout rate of change in consecutive frames of the series of images. The brightness and chromaticity of the image are separated by orthogonal transformation to obtain the color features; The Hu matrix is ​​constructed based on the image center matrix of the image to obtain the morphological features; The geometric features are obtained by calculating the path of sunlight and the angle of view of the camera used to capture the images.

[0008] Optionally, obtaining the dynamic features by detecting the pixel readout change rate of consecutive frames of the series of images includes the following steps: Extract the sequence of consecutive average values ​​of the continuous frames of the series of images; Calculate the inter-frame difference sequence based on the continuous mean sequence; Calculate the variance of the inter-frame difference sequence; The variance volatility is based on the variance, and the dynamic characteristic includes the variance volatility.

[0009] Optionally, the step of separating the brightness and chromaticity of the image through orthogonal transformation to obtain the color features includes the following steps: Perform orthogonal chromaticity transformation on the image; Region extraction is performed on the image after orthogonal chromatic transformation to obtain candidate flame regions; Calculate the ambient light intensity based on the non-flame region outside the flame candidate region; Calculate the signal-to-noise ratio of the chromaticity channel based on the ambient light intensity; The color features are obtained by calculating based on the signal-to-noise ratio of the chroma channel.

[0010] Optionally, the step of constructing a Hu matrix based on the image center matrix of the image to obtain the morphological features includes the following steps: Based on the binary membrane advance contour point set of the flame region in the image; Construct the Hu matrix based on the set of contour points; The cosine similarity is obtained by performing a similarity calculation on the Hu matrix. The contour complexity is calculated based on the cosine similarity. The morphological features are obtained by calculating based on the contour complexity.

[0011] Optionally, obtaining the geometric features by calculating the sunlight radiation path and the viewing angle of the camera used to acquire the image includes the following steps: Calculate the specular reflection direction vector of the image; The angle between the camera's line of sight and the reflected light is calculated based on the mirror reflection direction vector; The included angle is processed based on a specular reflection probability model to obtain the geometric features.

[0012] Optionally, the weighted fusion of the multiple image features based on weights to obtain the comprehensive confidence level of each image includes the following steps: The comprehensive confidence level is obtained by performing static weighted fusion calculation on the multiple image features based on fixed weights. Alternatively, the comprehensive confidence level can be obtained by dynamically weighting and fusing the various image features based on adaptive weights.

[0013] A fire monitoring system is applied to electronic devices. The fire monitoring system includes an edge server and a cloud connected to the edge server. The edge server is configured with a feature dynamic filter, a feature weighted fusion module, and an image filtering module. The cloud is configured with a cloud analysis module. The feature dynamic filter is configured to perform multi-dimensional feature filtering on a series of images acquired in the area to be monitored, thereby obtaining multiple image features for each image. The feature weighted fusion module is configured to perform weighted fusion of the multiple image features based on weights to obtain the comprehensive confidence level of each image; The image filtering module is configured to filter the series of images based on the comprehensive confidence score, and obtain a series of valid images by removing the images whose comprehensive confidence score is lower than a preset threshold. The cloud-based analysis module is configured to identify the series of valid images based on a multimodal large model to obtain monitoring information.

[0014] An electronic device includes at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs or instructions; The processor is used to execute the computer program or instructions to enable the electronic device to implement the fire monitoring method as described above.

[0015] A computer-readable storage medium is applied to an electronic device, the storage medium carrying one or more computer programs or instructions, the one or more computer programs being able to be implemented by the electronic device using the fire monitoring method described above.

[0016] As can be seen from the above technical solution, this application discloses a fire monitoring method, system, electronic device, and storage medium. The method is applied to an electronic device, specifically involving multi-dimensional feature filtering of a series of images acquired from image acquisition of the area to be monitored, obtaining multiple image features for each image; weighted fusion of these multiple image features based on weights to obtain the comprehensive confidence level of each image; filtering of the series of images based on the comprehensive confidence level, removing images with a comprehensive confidence level below a preset threshold to obtain a series of valid images; and identifying the series of valid images based on a multimodal large model to obtain monitoring information. In processing images acquired by a camera, this application filters images based on multiple image features, pre-filtering images containing strong reflective spots, lights, etc., which are easily misidentified as flames. This prevents false fire alarms during further identification, thus avoiding unnecessary disruption to production and daily life. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of a fire monitoring method according to an embodiment of this application; Figure 2 This is a block diagram of a fire monitoring device according to an embodiment of this application; Figure 3 This is a block diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0020] To address the technical problems raised in this application, the following technical solution is proposed. The specific details of this solution are as follows.

[0021] Figure 1 This is a flowchart of a fire monitoring method according to an embodiment of this application.

[0022] like Figure 1 As shown, the fire monitoring method provided in this embodiment is applied to electronic equipment for fire monitoring in areas to be monitored, such as factories, public places, and power plants. This electronic equipment can be understood as a computer, server, server cluster, or cloud platform with data computing and information processing capabilities. The specific steps of this monitoring method include: S1. Perform multi-dimensional feature filtering on the series of images collected in the area to be monitored to obtain multiple image features for each image.

[0023] The area to be monitored here refers to locations requiring fire monitoring, such as production sites, public places, and power plants. When monitoring the area, images are acquired using cameras capable of capturing visible light / infrared images, resulting in a time-series of images, which are then processed by the system. The various image features in this application include, but are not limited to, dynamic features, color features, morphological features, and geometric features, or may be a subset of these. This application obtains each feature through the following methods.

[0024] 1. For dynamic features, dynamic features are obtained by detecting the pixel readout rate of change in consecutive frames of a series of images. The so-called dynamic feature refers to the irregular flickering of flames in the 1-10 Hz range caused by turbulent combustion. This can be detected by pixel brightness change rate. The average brightness of suspected areas is extracted from 5-10 consecutive frames of images, and the inter-frame difference variance is calculated. If the variance fluctuation range is greater than a threshold (e.g., 30%), it can be identified as a flame. The dynamic feature is obtained through the following steps: 1) Extract the mean brightness sequence of consecutive frames. For 10 consecutive frames of images of the suspected flame region (ROI), calculate the mean pixel brightness μ frame by frame. t : = (t=1,2...,10) in, is the brightness value of the i-th pixel in the t-th frame, such as grayscale value or RGB mean; N is the total number of pixels within the ROI.

[0025] 2) Calculate the inter-frame difference sequence to generate the brightness change sequence Δμ between adjacent frames. t : =| - | (t= 1,2,...,9), Where, μ tThe brightness of the current frame, μ t+1 This is the brightness for the next frame.

[0026] 3) Calculate the variance of the difference sequence, specifically the variance of the difference sequence Δμ. : = , in, is the mean of the difference sequence.

[0027] 4) Calculate the relative fluctuation of the current variance compared to the historical variance baseline (such as the moving window mean): Volatility = , 5) Dynamic settings Values ​​based on turbulent flame characteristics (variance fluctuation > 30%): = .

[0028] 2. Regarding color features, this application separates the brightness and chromaticity of the image through orthogonal transformation to obtain color features. .

[0029] 1) Orthogonal chromaticity transformation generate.

[0030] Physical meaning: Flames exhibit the characteristic of R > G > B in the RGB color space, with significant differences between the red and blue channels. Orthogonal transformation Enhance flame chromaticity characteristics and suppress brightness interference: = (Value range: [-127.5, 127.5]) It can eliminate the influence of light intensity (such as cloudy days / strong light) and enhance the red-blue contrast of the flame (flame) The value was significantly higher than the background value.

[0031] 2) Flame candidate region extraction and flame pixel determination criteria (based on RGB statistical model):

[0032] in = , , This is the normalized channel value.

[0033] 3) Ambient light interference estimation.

[0034] Background modeling method: Take the image edge region (non-flame region) or the background of the reference frame and calculate the average ambient light intensity. .

[0035] Ambient light interference factor : = (Normalized to [0, 1]) 0 indicates low interference (such as at night). 1 indicates interference (such as strong sunlight).

[0036] 4) Calculation of chroma channel signal-to-noise ratio (SNR).

[0037] Signal definition: within the flame area The variance (characterizing the fluctuation of flame chromaticity): S = Var( ) Noise definition: background area Standard deviation (characterizing the intensity of environmental disturbance): N = Std( ) SNR formula: SNR = 10 = (prevention except zero) Among them, the higher the SNR, the more significant the flame color characteristics and the less affected by ambient light.

[0038] 5) Reliability Indicators Dynamic mapping.

[0039] Nonlinear normalization (adapting to SNR range): =

[0040] 3. Regarding morphological features, this application constructs a Hu matrix based on the image center matrix of the image to obtain the morphological features. .

[0041] 1) Contour preprocessing and normalization: Extract the contour point set from the binary mask of the flame region, denoted as...

[0042] 2) Hu matrix calculation (translation / rotation / scaling invariant), original moments (Spatial Moments): = × ) in, This is the pixel value (1 for binarization). Let A be the contour surface.

[0043] Central moment (eliminating translation effects): = · , Normalized central moments (eliminating scaling effects): = , = , Hu invariant moments (see the Hu invariant moment formulas in Table 1 for details):

[0044]

[0045] Table 1

[0046] Table 1 shows the expressions for the seven Hu matrices and their sensitivity to shape: Applications in photovoltaic fire detection: Flame: The Hu matrix values ​​of 3, 4, and 7 are relatively high (reflecting irregular distortion and asymmetry). Reflected light spot: High values ​​of Hu matrix 1, 2, and 5 (reflecting a symmetrical circular or elliptical shape). Area-to-perimeter ratio: a measure of the compactness of a profile. The shape compactness is calculated by combining the area (S) and perimeter (C) of the region enclosed by the outline, allowing for a quick distinction between flames and regularly reflected light. Key metrics: Circularity: P = C² / (4πS), P≈1: Perfect circle (typical reflected light spot). P 1: Complex and irregular shape (typical flame).

[0047] 3) Calculate the Hu matrix similarity to obtain the cosine similarity, and perform a logarithmic transformation of the Hu matrix (to resolve differences in magnitude): = sign(H) × , With template Hu matrix Cosine similarity: similarity = , The value range is [-1, 1]. The closer it is to 1, the more similar the shape is to the flame template.

[0048] 4) Contour complexity calculation (circularity), contour area A and perimeter P: A = , P = , Roundness formula: circularity = , Range (0,1]: Flame outline: ≈ 0.3 0.6 (edge ​​breakage) Reflective spot: →1.0 (smooth circle).

[0049] 5) Reliability Indicators Fusion formula: rm = λ similarity+(1- λ ) circularity Dynamic weight λ: λ = ， Physical meaning: →1: Shape matches template and outline is broken (real fire). →0: Shape deviation or excessively rounded outline (interference).

[0050] 4. Regarding geometric features, this application obtains the geometric features by calculating the sunlight radiation path and the viewing angle of the camera used to acquire the images. .

[0051] For example, in a photovoltaic power plant, the reflection probability can be calculated based on the solar altitude angle (α), panel tilt angle (β), and camera viewing angle (γ). If |α + β - γ| ≤ 5°, there is a high probability of reflection interference.

[0052] This formula quantifies the angle between the reflected sunlight path and the camera's line of sight, used to determine whether the reflected light enters the camera's field of view. When sunlight shines on a panel with an angle of incidence α and a tilt angle β, the reflection angle = α (specular reflection law). If the angle difference between the reflected light and the camera's viewing angle γ is ≤ 5, then reflection interference is highly probable.

[0053] 1) Calculation of the direction vector of mirror reflection: Solar incident vector (local coordinate system, Z-axis perpendicular to the ground and upward): S = , α: Solar altitude angle : Solar azimuth (relative to due north).

[0054] Panel normal vector (determined by tilt angle β and orientation angle θ): N = , β: Panel tilt angle (0° is horizontal, 90° is vertical). θ: Panel orientation angle (0° is due north, increasing clockwise).

[0055] Reflection vector (specular reflection law): R=S-2(S N)N, S N: The dot product of the incident vector and the normal vector.

[0056] 2) Angle between the reflected ray and the camera's line of sight; camera observation vector (from the panel to the camera): V = , γ: Camera angle (angle with the ground). Camera azimuth angle included angle (Reflected light and line-of-sight deviation): = arccos( ) 3) Specular reflection probability model, probability density function (angle) The larger the value, the lower the probability of reflection interference. p( ) = k=0.5 2.0 (attenuation coefficient) Reliability indicators Mapping: = 1- p( ) = 1-

[0057] Physical meaning: 0: Reflected light directly hits the camera. p 1 0 (High Interference) 1: Reflected light directly hits the camera p 0 1 (Low interference).

[0058] S2. Based on the weights, multiple image features are weighted and fused to obtain the comprehensive confidence level of each image.

[0059] This application, based on different specific scenarios and situations, employs static weighted fusion calculation of multiple image features with fixed weights in scenarios where prior knowledge is clear, and dynamic weighted fusion calculation of multiple image features with adaptive weights in complex scenarios, thereby obtaining the comprehensive confidence level F under different scenarios. fused .

[0060] A scenario with clear prior knowledge refers to a scenario where the environmental interference source is relatively singular, stable, and predictable, such that the reliability and importance weights of different feature dimensions in judging the fire situation can be pre-set and remain fixed over a relatively long period. A comprehensive judgment can be made based on the following dimensions: time and astronomical conditions, space and installation conditions, environment and interference source, and expected fire type. In this case, the formula is: F fused =αF d +βF c +γF m +δF g , Where α, β, γ and δ are weight coefficients, satisfying α+β+γ+δ=1; the weights can be allocated according to the importance of features, and in flame judgment, the weight allocation is set to β>α>γ>δ.

[0061] The technical formula for complex scenarios is: F fused =w d F d +w c F c +w m F m +w g F g Where the adaptive weight w d w c w m and w g Satisfying ∑w i =1, through the dynamic reliability index r i generate: w i =r i / ∑ j∈{d,c,m,g} r j 。 This application generates the above adaptive weights using a lightweight learner (such as a two-layer fully connected network), expressed by the following formula: w=softmax(W2·ReLU(W1·GAP([F d ·F c ·F m ·F g ]))) Where W1 and W2 are learnable parameters.

[0062] The complete calculation formula example is as follows: Feature alignment layer: F i ′ =W align,i F i (i∈{d,c,m,g}) Adaptive weight generation: g=GAP([F d ′ ·F c ′ ·F m ′ ·F g ′ ]) w = softmax(W2·ReLU(W1g)) Weighted fusion output: F fused =w d F d ′ + w c F c ′ + w m F m ′ + w g Fg ′ .

[0063] S3. Filter the series of images based on the comprehensive confidence level to obtain a series of valid images.

[0064] That is, by removing images whose overall confidence level is lower than a preset threshold, all series of images are filtered to obtain a portion of the effective images.

[0065] S4. Based on the multimodal large model, identify a series of effective images to obtain monitoring information.

[0066] By processing a series of valid images using a multimodal large model, that is, by combining the context of the image scene with reasoning, valid fire information is identified and alerts are sent to users, thereby reducing the occurrence of false alarms.

[0067] As can be seen from the above technical solution, this embodiment provides a fire monitoring method. This method is applied to electronic devices, specifically by performing multi-dimensional feature filtering on a series of images acquired from image acquisition of the area to be monitored, obtaining multiple image features for each image; weighted fusion of these multiple image features based on weights to obtain the comprehensive confidence level of each image; filtering the series of images based on the comprehensive confidence level, removing images with a comprehensive confidence level below a preset threshold to obtain a series of valid images; and identifying the series of valid images based on a multimodal large model to obtain monitoring information. In processing images acquired by a camera, this application filters images based on multiple image features, pre-filtering images containing strong reflective spots, lights, etc., which are easily misidentified as flames. This prevents false fire alarms during further identification, thus avoiding unnecessary disruption to production and daily life.

[0068] In addition, during the process of processing images to achieve fire monitoring, the fire information identified by this multimodal large model can be used to further optimize the model, thereby making the large model more accurate.

[0069] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0070] Although the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous.

[0071] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0072] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer.

[0073] Figure 2 This is a block diagram of a fire monitoring system according to an embodiment of this application.

[0074] like Figure 2 As shown, the fire monitoring system provided in this embodiment is applied to electronic devices for fire monitoring in areas to be monitored, such as factories, public places, and power plants. This electronic device can be understood as a computer, server, server cluster, or cloud platform with data computing and information processing capabilities. Specifically, the monitoring system includes an edge server 10 and a cloud platform 20. The edge server is equipped with a feature dynamic filter 11, a feature weighted fusion module 12, and an image filtering module 13, while the cloud platform is equipped with a cloud analysis module 21.

[0075] The feature dynamic filter is used to perform multi-dimensional feature filtering on a series of images collected in the area to be monitored, so as to obtain multiple image features for each image.

[0076] The area to be monitored here refers to locations requiring fire monitoring, such as production sites, public places, and power plants. When monitoring the area, images are acquired using cameras capable of capturing visible light / infrared images, resulting in a time-series of images, which are then processed by the system. The various image features in this application include, but are not limited to, dynamic features, color features, morphological features, and geometric features, or may be a subset of these. This application obtains each feature through the following methods.

[0077] 1. For dynamic features, dynamic features are obtained by detecting the pixel readout rate of change in consecutive frames of a series of images. The so-called dynamic feature refers to the irregular flickering of flames at 1-10 Hz caused by turbulent combustion. This can be detected by the pixel brightness change rate. The average brightness of suspected areas can be extracted from 5-10 consecutive frames of images, and the inter-frame difference variance can be calculated. If the variance fluctuation range is greater than the threshold (e.g., 30%), it can be determined as a flame.

[0078] 2. Regarding color features, this application separates the brightness and chromaticity of the image through orthogonal transformation to obtain color features. .

[0079] 3. Regarding morphological features, this application constructs a Hu matrix based on the image center matrix of the image to obtain the morphological features. .

[0080] 4. Regarding geometric features, this application obtains the geometric features by calculating the sunlight radiation path and the viewing angle of the camera used to acquire the images. .

[0081] The feature weighted fusion module is used to perform weighted fusion of multiple image features based on weights to obtain the comprehensive confidence level of each image.

[0082] This application, based on different specific scenarios and situations, employs static weighted fusion calculation of multiple image features with fixed weights in scenarios where prior knowledge is clear, and dynamic weighted fusion calculation of multiple image features with adaptive weights in complex scenarios, thereby obtaining the comprehensive confidence level F under different scenarios. fused .

[0083] A scenario with clear prior knowledge refers to a scenario where the environmental interference source is relatively singular, stable, and predictable, such that the reliability and importance weights of different feature dimensions in judging the fire situation can be pre-set and remain fixed over a relatively long period. A comprehensive judgment can be made based on the following dimensions: time and astronomical conditions, space and installation conditions, environment and interference source, and expected fire type.

[0084] The image filtering module is used to filter a series of images based on a comprehensive confidence level to obtain a series of valid images.

[0085] That is, by removing images whose overall confidence level is lower than a preset threshold, all series of images are filtered to obtain a portion of the effective images.

[0086] The cloud-based analytics module is used to identify a series of valid images based on a multimodal large model to obtain monitoring information.

[0087] By processing a series of valid images using a multimodal large model, that is, by combining the context of the image scene with reasoning, valid fire information is identified and alerts are sent to users, thereby reducing the occurrence of false alarms.

[0088] As can be seen from the above technical solution, this embodiment provides a fire monitoring system. This system is applied to electronic devices and specifically involves multi-dimensional feature filtering of a series of images acquired from the area to be monitored, obtaining multiple image features for each image; weighted fusion of these multiple image features based on weights to obtain the comprehensive confidence level of each image; filtering of the series of images based on the comprehensive confidence level, removing images with a comprehensive confidence level below a preset threshold to obtain a series of valid images; and identifying the series of valid images based on a multimodal large model to obtain monitoring information. In processing images acquired by a camera, this application filters images based on multiple image features, pre-filtering images containing strong reflective spots, lights, etc., that are easily misidentified as flames. This prevents false fire alarms during further identification, thus avoiding unnecessary disruption to production and daily life.

[0089] The units described in the embodiments of this disclosure can be implemented in software or in hardware. The name of a unit does not necessarily limit the unit itself; for example, the first acquisition unit can also be described as "a unit that acquires at least two Internet Protocol addresses".

[0090] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0091] Figure 3 This is a block diagram of an electronic device according to an embodiment of this application.

[0092] The following is for reference. Figure 3 This document illustrates a structural diagram suitable for implementing the electronic device in the embodiments of this disclosure. The terminal device in the embodiments of this disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. This electronic device is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this disclosure.

[0093] The electronic device may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from an input device 306 into a random access memory (RAM) 303. The RAM also stores various programs and data required for the operation of the electronic device. The processing unit, ROM, and RAM are interconnected via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0094] Typically, the following devices can be connected to the I / O interface: input devices including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 307 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 308 including, for example, magnetic tapes, hard disks, etc.; and communication devices 309. Communication device 309 allows the electronic device to communicate wirelessly or wiredly with other devices to exchange data. Although electronic devices with various devices are shown in the figures, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0095] This application also provides an embodiment of a readable storage medium.

[0096] The aforementioned computer-readable storage medium is applied to an electronic device and carries one or more computer programs. When the electronic device executes these programs, it performs multi-dimensional feature filtering on a series of images acquired from image acquisition of the area to be monitored, obtaining multiple image features for each image. It then performs weighted fusion of these multiple image features to obtain a comprehensive confidence score for each image. Based on the comprehensive confidence score, it filters the series of images, removing images with a comprehensive confidence score below a preset threshold to obtain a series of valid images. Finally, it identifies the series of valid images using a multimodal large model to obtain monitoring information. In processing images acquired by a camera, this application filters images based on multiple image features, pre-filtering images containing strong reflective spots, lights, or other elements easily misidentified as flames. This prevents false fire alarms during further identification, thus avoiding unnecessary disruption to production and daily life.

[0097] It should be noted that the computer-readable medium described above in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0098] In this disclosure, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0099] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0100] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present invention.

[0101] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0102] The technical solution provided by the present invention has been described in detail above. Specific examples have been used to illustrate the principle and implementation of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation and application scope based on the idea of ​​the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A fire monitoring method, applied to electronic equipment, characterized in that, The fire monitoring method includes the following steps: Multidimensional feature filtering is performed on a series of images collected in the area to be monitored to obtain multiple image features for each image. The various image features are weighted and fused to obtain the overall confidence level of each image. The series of images are filtered based on the comprehensive confidence level. By removing images with a comprehensive confidence level lower than a preset threshold, a series of valid images are obtained. Based on a multimodal large model, the series of effective images are identified to obtain monitoring information.

2. The acquisition and monitoring method as described in claim 1, characterized in that, The various image features include some or all of the dynamic features, color features, morphological features, and geometric features.

3. The fire monitoring method as described in claim 2, characterized in that, The step of performing multi-dimensional feature filtering on a series of images obtained from image acquisition of the area to be monitored to obtain multiple image features for each image includes the following steps: The dynamic features are obtained by detecting the pixel readout rate of change in consecutive frames of the series of images. The brightness and chromaticity of the image are separated by orthogonal transformation to obtain the color features; The Hu matrix is ​​constructed based on the image center matrix of the image to obtain the morphological features; The geometric features are obtained by calculating the path of sunlight and the angle of view of the camera used to capture the images.

4. The fire monitoring method as described in claim 3, characterized in that, The process of obtaining the dynamic features by detecting the pixel readout rate of consecutive frames of the series of images includes the following steps: Extract the sequence of consecutive average values ​​of the continuous frames of the series of images; Calculate the inter-frame difference sequence based on the continuous mean sequence; Calculate the variance of the inter-frame difference sequence; The variance volatility is based on the variance, and the dynamic characteristic includes the variance volatility.

5. The fire monitoring method as described in claim 3, characterized in that, The step of separating the brightness and chromaticity of the image through orthogonal transformation to obtain the color features includes the following steps: Perform orthogonal chromaticity transformation on the image; Region extraction is performed on the image after orthogonal chromatic transformation to obtain candidate flame regions; Calculate the ambient light intensity based on the non-flame region outside the flame candidate region; Calculate the signal-to-noise ratio of the chromaticity channel based on the ambient light intensity; The color features are obtained by calculating based on the signal-to-noise ratio of the chroma channel.

6. The fire monitoring method as described in claim 3, characterized in that, The process of constructing a Hu matrix based on the image center matrix to obtain the morphological features includes the following steps: Based on the binary membrane advance contour point set of the flame region in the image; Construct the Hu matrix based on the set of contour points; The cosine similarity is obtained by performing a similarity calculation on the Hu matrix. The contour complexity is calculated based on the cosine similarity. The morphological features are obtained by calculating based on the contour complexity.

7. The fire monitoring method as described in claim 3, characterized in that, The process of obtaining the geometric features by calculating the path of sunlight and the viewing angle of the camera used to acquire the image includes the following steps: Calculate the specular reflection direction vector of the image; The angle between the camera's line of sight and the reflected light is calculated based on the mirror reflection direction vector; The included angle is processed based on a specular reflection probability model to obtain the geometric features.

8. The fire monitoring method as described in claim 1, characterized in that, The weighted fusion of the multiple image features based on weights to obtain the comprehensive confidence level of each image includes the following steps: The comprehensive confidence level is obtained by performing static weighted fusion calculation on the multiple image features based on fixed weights. Alternatively, the comprehensive confidence level can be obtained by dynamically weighting and fusing the various image features based on adaptive weights.

9. A fire monitoring system, applied to electronic equipment, characterized in that, The fire monitoring system includes an edge server and a cloud connected to the edge server. The edge server is equipped with a dynamic feature filter, a feature weighted fusion module, and an image filtering module. The cloud is equipped with a cloud analysis module. The feature dynamic filter is configured to perform multi-dimensional feature filtering on a series of images acquired in the area to be monitored, thereby obtaining multiple image features for each image. The feature weighted fusion module is configured to perform weighted fusion of the multiple image features based on weights to obtain the comprehensive confidence level of each image; The image filtering module is configured to filter the series of images based on the comprehensive confidence score, and obtain a series of valid images by removing the images whose comprehensive confidence score is lower than a preset threshold. The cloud-based analysis module is configured to identify the series of valid images based on a multimodal large model to obtain monitoring information.

10. An electronic device, characterized in that, The electronic device includes at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs or instructions; The processor is used to execute the computer program or instructions to enable the electronic device to implement the fire monitoring method as described in any one of claims 1 to 8.

11. A computer-readable storage medium for use in electronic devices, characterized in that, The storage medium carries one or more computer programs or instructions, which can be implemented by the electronic device as described in any one of claims 1 to 8.

Images