Fire detection method and device

Abstract

The embodiment of the invention provides a fire detection method and device.The fire detection method comprises the steps that an infrared image and a visible light image collected by an unmanned aerial vehicle for a target area are input into a fire detection model, and the fire detection model comprises an infrared processing unit, a visible light processing unit and a feature fusion unit; extracting infrared image features of the infrared image through the infrared processing unit, and extracting visible light image features of the visible light image through the visible light processing unit; performing feature fusion on the infrared image features and the visible light image features by using the feature fusion unit, and determining fire detection information according to a feature fusion result; and when determining that a fire event occurs in the target area according to the fire detection information, correcting the fire detection information into target fire detection information by using a filter.

Description

Technical Field

[0001] The embodiments of this specification relate to the field of image processing technology, and in particular to a fire detection method and device. Background Art

[0002] The development of drone technology has brought new solutions to fire monitoring. Drones offer advantages such as maneuverability, high real-time performance, and the ability to fly at low altitudes. They are particularly suitable for use in complex terrain, such as densely populated areas, forests, and confined spaces. They can play a vital role in areas where traditional monitoring methods are insufficient. Currently, drones are often equipped with single sensors for fire detection, such as visible light cameras and infrared thermal imagers. Visible light cameras can effectively capture flame characteristics during the day, but their effectiveness is limited at night or in heavy smoke. Infrared thermal imagers can identify the high-temperature characteristics of fire sources, but their data is susceptible to interference from ambient background temperatures. While both sensors have their strengths, single-sensor monitoring methods still struggle to guarantee high fire detection accuracy, potentially leading to false alarms and missed detections due to factors such as environmental fluctuations and lighting conditions. Therefore, an effective solution is urgently needed to address these issues. Summary of the Invention

[0003] In view of this, embodiments of this specification provide a fire detection method. One or more embodiments of this specification also relate to a fire detection device, a computing device, a computer-readable storage medium, and a computer program product to address technical deficiencies in the prior art.

[0004] According to a first aspect of an embodiment of this specification, a fire detection method is provided, comprising: Inputting infrared images and visible light images collected by the drone of the target area into a fire detection model, wherein the fire detection model includes an infrared processing unit, a visible light processing unit, and a feature fusion unit; extracting infrared image features of the infrared image by the infrared processing unit, and extracting visible light image features of the visible light image by the visible light processing unit; Using the feature fusion unit to perform feature fusion on the infrared image features and the visible light image features, and determining fire detection information based on the feature fusion result; When it is determined that a fire event has occurred in the target area according to the fire detection information, the fire detection information is corrected into target fire detection information using a filter.

[0005] According to a second aspect of the embodiments of this specification, a fire detection device is provided, comprising: an input module configured to input infrared images and visible light images collected by the drone of the target area into a fire detection model, wherein the fire detection model includes an infrared processing unit, a visible light processing unit, and a feature fusion unit; an extraction module configured to extract infrared image features of the infrared image through the infrared processing unit, and to extract visible light image features of the visible light image through the visible light processing unit; a fusion module configured to perform feature fusion on the infrared image features and the visible light image features using the feature fusion unit, and determine fire detection information based on the feature fusion result; The correction module is configured to use a filter to correct the fire detection information into target fire detection information when it is determined that a fire event has occurred in the target area based on the fire detection information.

[0006] According to a third aspect of an embodiment of this specification, a computing device is provided, including: memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the above-mentioned fire detection method are implemented.

[0007] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores computer-executable instructions. When the instructions are executed by a processor, the steps of the above-mentioned fire detection method are implemented.

[0008] According to a fifth aspect of the embodiments of this specification, a computer program product is provided, comprising a computer program or instructions, which implement the steps of the above-mentioned fire detection method when executed by a processor.

[0009] The fire detection method provided in this embodiment can be adapted to a more complex monitoring environment and improve the accuracy of fire detection by using a fusion method of visible light images and infrared images for fire detection. Specifically, the infrared image and visible light image collected by the drone for the target area can be input into a fire detection model, wherein the fire detection model integrates an infrared processing unit, a visible light processing unit, and a feature fusion unit; it is used to extract infrared image features of the infrared image through the infrared processing unit, and to extract visible light image features of the visible light image through the visible light processing unit; after obtaining the infrared image features and the visible light image features, considering that the two image features belong to different types, the feature fusion unit can be used to fuse the infrared image features and the visible light image features, thereby determining the fire detection information based on the feature fusion results; and improving the accuracy of fire detection through fusion processing. Afterwards, if a fire incident is determined to have occurred in the target area based on the fire detection information, in order to accurately detect the fire situation, the fire detection information can be corrected in combination with a filter to reduce the interference of other noises in the environment, thereby obtaining accurate target fire detection information. In the drone fire detection scenario, the fire detection accuracy is effectively improved, so that a reasonable response can be made at the moment of fire occurrence to avoid excessive economic losses. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 This is a flow chart of a fire detection method provided by one embodiment of this specification; Figure 2 This is a structural diagram of a fire detection model in a fire detection method provided in one embodiment of this specification; Figure 3 This is a process flow chart of a fire detection method provided by one embodiment of this specification; Figure 4 This is a structural diagram of a fire detection device provided by one embodiment of this specification; Figure 5 This is a structural block diagram of a computing device provided by one embodiment of this specification. DETAILED DESCRIPTION

[0011] The following description sets forth many specific details to facilitate a thorough understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the scope of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0012] The terms used in one or more embodiments of this specification are for the purpose of describing specific embodiments only and are not intended to limit one or more embodiments of this specification. The singular forms "a," "the," and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0013] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0014] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0015] First, the terms involved in one or more embodiments of this specification are explained.

[0016] CNN model: A convolutional neural network (CNN) is a deep learning model used to process two-dimensional or three-dimensional data (such as images, audio, and video). Its core is the use of convolution and pooling operations for feature extraction and dimensionality reduction. A CNN consists of an input layer, convolutional layers, pooling layers, fully connected layers, and an output layer. The convolutional layer is the core of a CNN, responsible for extracting features from the input data; the fully connected layer integrates the features output by the previous layers and maps them to the sample's labeled space; the output layer outputs the final classification result or regression value based on the task requirements.

[0017] DS Evidence Theory, also known as the Dempster-Shafer Theory, is a mathematical tool for dealing with uncertainty and incomplete information. Its core principle is to address the synthesis of uncertain information from multiple sources through concepts such as basic probability assignments (BPAs), belief functions, and likelihood functions. BPAs are functions defined on all subsets of a recognition framework, representing the degree of belief in each proposition; belief functions represent the overall degree of belief in a proposition; and likelihood functions indicate the degree to which a proposition is not rejected. When there are multiple independent sources of evidence, the basic probability assignments of these sources can be combined using Dempster's combination rule to obtain a comprehensive belief assignment.

[0018] The Kalman filter is a state estimator that improves system accuracy and reduces uncertainty in the estimated state by optimally combining the system's state transition equations with the measurement system's measurements (weighted average). The Kalman filter iterates prediction and update steps, combining the system's dynamic model with observations to obtain an optimal estimate of the system's state. It assumes that the system's state changes follow a linear dynamic model and that observations are linear transformations of the system's state. Both processes are accompanied by noise, and the Kalman filter achieves the optimal estimate by minimizing the covariance of the estimation errors.

[0019] In this specification, a fire detection method is provided. This specification also relates to a fire detection device, a computing device, a computer-readable storage medium, and a computer program product, which are described in detail one by one in the following embodiments.

[0020] In practice, traditional fire monitoring methods include ground patrols, satellite remote sensing, and observations from fixed monitoring points. However, these methods have significant limitations. Ground patrols and fixed monitoring points have limited coverage and cannot capture real-time fire information in distant or inaccessible areas. Satellite remote sensing, while capable of covering large areas, is affected by weather conditions and image resolution, making it difficult to provide timely and accurate early warnings of fires. Due to these limitations, traditional monitoring methods cannot meet the real-time, accurate, and high-coverage requirements of modern forest fire monitoring. The rapid development of drones has brought new solutions to fire monitoring, enabling rapid fire detection and processing. However, in more complex scenarios, detection accuracy may be affected by environmental factors, making it difficult to guarantee accuracy. Therefore, an effective solution is urgently needed to address these issues.

[0021] The fire detection method provided in this embodiment can be adapted to a more complex monitoring environment and improve the accuracy of fire detection by using a fusion method of visible light images and infrared images for fire detection. Specifically, the infrared image and visible light image collected by the drone for the target area can be input into a fire detection model, wherein the fire detection model integrates an infrared processing unit, a visible light processing unit, and a feature fusion unit; it is used to extract infrared image features of the infrared image through the infrared processing unit, and to extract visible light image features of the visible light image through the visible light processing unit; after obtaining the infrared image features and the visible light image features, considering that the two image features belong to different types, the feature fusion unit can be used to fuse the infrared image features and the visible light image features, thereby determining the fire detection information based on the feature fusion results; and improving the accuracy of fire detection through fusion processing. Afterwards, if a fire incident is determined to have occurred in the target area based on the fire detection information, in order to accurately detect the fire situation, the fire detection information can be corrected in combination with a filter to reduce the interference of other noises in the environment, thereby obtaining accurate target fire detection information. In the drone fire detection scenario, the fire detection accuracy is effectively improved, so that a reasonable response can be made at the moment of fire occurrence to avoid excessive economic losses.

[0022] See also Figure 1 , Figure 1 A flow chart of a fire detection method provided according to an embodiment of this specification is shown, which specifically includes the following steps.

[0023] In step S102 , the infrared image and the visible light image collected by the drone for the target area are input into a fire detection model, wherein the fire detection model includes an infrared processing unit, a visible light processing unit, and a feature fusion unit.

[0024] The fire detection method provided in this embodiment can be applied to any fire detection scenario, such as fire detection in residential and building scenarios, fire detection in public places and transportation hubs, fire detection in industrial facilities and chemical enterprises, fire detection in warehousing and logistics centers, and forest fire detection. This embodiment uses the application of the fire detection method in the forest fire detection scenario as an example to illustrate the fire detection method. For descriptions of other scenarios, refer to the same or corresponding descriptions in this embodiment, and this embodiment will not be elaborated on here.

[0025] Specifically, the target area specifically refers to the area that currently needs to be processed for fire detection. Accordingly, the infrared image specifically refers to the image collected by the infrared sensor configured by the drone, and the visible light image specifically refers to the image collected by the optical sensor configured by the drone. Accordingly, the fire detection model specifically refers to a model that includes an infrared processing unit, a visible light processing unit, and a feature fusion unit. The model can be implemented through a CNN convolutional neural network, which is used to input infrared images and visible light images and output information for fire detection in the target area. Among them, the infrared processing unit specifically refers to the neural network that processes the infrared image, and the visible light processing unit specifically refers to the neural network that processes the visible light image. The infrared processing unit and the visible light processing unit belong to different channels in the fire detection model, thereby ensuring their own image processing accuracy and efficiency. The feature fusion unit specifically refers to a processing unit that fuses the processing results of different channels and outputs fire detection information.

[0026] In practical applications, in order to enable the fire detection model to have strong and sufficiently accurate prediction capabilities, it can be trained before application. Specifically, a sample set associated with the initial fire detection model is constructed, samples are extracted from the sample set and input into the model for processing, the prediction results output by the model are obtained, the model parameters are adjusted based on the labels corresponding to the samples and the prediction results, and the adjusted model is tested to see whether it meets the training stop conditions. If not, new samples are selected for training until the model meets the conditions in a certain training cycle. The model can then be used as a fire detection model. The training stop conditions can be the number of iterations, validation set verification, or loss value comparison, etc. This embodiment does not impose any restrictions on this.

[0027] Based on this, in order to adapt to more complex monitoring environments and improve the accuracy of fire detection, the infrared images and visible light images collected by the drone for the target area can be input into the fire detection model, wherein the fire detection model includes an infrared processing unit, a visible light processing unit and a feature fusion unit; the infrared image features of the infrared image are extracted by the infrared processing unit, and the visible light image features of the visible light image are extracted by the visible light processing unit; the feature fusion unit is used to perform feature fusion on the infrared image features and the visible light image features, and fire detection information is determined according to the feature fusion result; when it is determined that a fire event has occurred in the target area according to the fire detection information, the fire detection information is corrected to target fire detection information using a filter.

[0028] That is, the fire detection method provided in this embodiment, after collecting infrared images and visible light images through a drone, will process the infrared images and visible light images through a dual-channel fire detection model. Among them, the infrared processing unit in the fire detection model will focus on capturing temperature changes, suitable for detecting the high-temperature characteristics of the fire source; while the visible light processing unit focuses on appearance information such as flame shape and color. By extracting different characteristic information of the fire through the two processing units and performing weighted splicing at a high level through a feature fusion unit, complementary fusion of fire information can be achieved, ensuring that the collected images are not affected by ambient light, and fire detection accuracy can be guaranteed regardless of day or night.

[0029] In practice, the fire detection model provided in this embodiment can be implemented using dual-channel convolutional neural network fusion (Dual-Channel CNN Fusion). This utilizes a dual-channel architecture to process data from different sensors separately, fully exploiting the characteristics of both types of data. This fusion of features improves the accuracy and robustness of fire detection. Infrared images excel at capturing temperature changes and are suitable for detecting temperature characteristics of fire sources and their surroundings; visible light images excel at capturing appearance features such as shape and color, complementing each other. The structure and training strategy of the dual-channel CNN model effectively integrates these two types of image information, significantly improving fire detection capabilities.

[0030] Within the dual-channel CNN model architecture, the infrared processing unit includes a processing unit that processes infrared images captured by the infrared sensor, focusing primarily on temperature information. The convolutional layer extracts high-temperature areas and temperature differences in the infrared image to identify high-temperature characteristics of the fire source and its surroundings. The visible light processing unit processes visible light images and identifies visual features such as flame shape and smoke. During image feature extraction, the convolutional layer identifies key information such as flame shape and color. The feature fusion unit fuses visible light and infrared image features, combining this information through weighted feature concatenation to obtain more comprehensive fire characteristics. Furthermore, the model includes a classification layer, which makes the final fire determination. Using classification algorithms such as softmax, the fused features can be classified to produce more accurate fire detection results.

[0031] Step S104 : extracting infrared image features of the infrared image through the infrared processing unit, and extracting visible light image features of the visible light image through the visible light processing unit.

[0032] Specifically, after the infrared image and visible light image are collected by the drone and input into the fire detection model, which includes an infrared processing unit and a visible light processing unit, the infrared processing unit can extract infrared image features from the infrared image, and the visible light processing unit can extract visible light image features from the visible light image. This allows the infrared image and visible light image to be processed separately so that the features of the two channels can be combined to complete fire detection prediction.

[0033] Among them, infrared image features specifically refer to the vector expression of temperature information and fire source location extracted from infrared images; visible light image features specifically refer to the vector expression of visual information such as flames and smoke extracted from visible light images.

[0034] Furthermore, when the infrared processing unit is used to process the infrared image, considering that the infrared processing unit includes an infrared input layer, an infrared convolution layer, an infrared pooling layer, an infrared mining layer, and an infrared fully connected layer, the image processing operation can be completed in the following manner to obtain infrared image features that accurately represent the image features: A first infrared image feature corresponding to the infrared image is constructed through the infrared input layer, and the infrared convolution layer is used to perform region recognition on the first infrared image feature to obtain a second infrared image feature; based on the infrared pooling layer, the second infrared image feature is reduced to a third infrared image feature, and the infrared mining layer is used to perform feature mining on the third infrared image feature to obtain a fourth infrared image feature; and the fourth infrared image feature is processed through the infrared fully connected layer to obtain the infrared image feature of the infrared image.

[0035] Specifically, the infrared input layer receives infrared images and performs image preprocessing. Normalization is performed to normalize the pixel values ​​of the infrared image to the range [0, 1] to ensure consistent data distribution, thereby obtaining the first infrared image feature. The infrared convolution layer combines multiple convolution kernels to extract temperature features. For example, 3x3 and 5x5 convolution kernels can be used to identify high-temperature centers and diffuse regions in local areas, respectively. In practical applications, the convolution operation can be followed by a ReLU activation function to enhance nonlinear properties, thereby obtaining the second infrared image feature. The infrared pooling layer combines max-pooling to reduce feature dimensionality while retaining high-temperature regions. Pooling can help reduce computational complexity while enhancing the model's robustness to displacement, thereby obtaining the third infrared image feature. The infrared mining layer can be understood as consisting of deep convolutional and pooling layers. Multiple layers of convolution and pooling layers are alternating to extract deeper features, focusing on fire temperature characteristics, such as the temperature gradient of the fire source, thereby obtaining the fourth infrared image feature. The infrared fully connected layer is used to process the features output by the convolutional layer to obtain infrared image features for subsequent input feature fusion unit processing.

[0036] Based on this, after inputting the infrared image into the fire detection model, the infrared input layer can first construct the first infrared image features corresponding to the infrared image. The first infrared image features can ensure consistent data distribution. The infrared convolution layer can then perform regional recognition on the first infrared image features to determine the high-temperature center and extended area, thereby obtaining the second infrared image features. The infrared pooling layer then reduces the dimensionality of the second infrared image features to the third infrared image features, thereby reducing computational complexity while preserving the high-temperature areas. The infrared mining layer then performs feature mining on the third infrared image features, focusing them more on fire temperature characteristics, to obtain the fourth infrared image features. Finally, the infrared fully connected layer processes the fourth infrared image features to obtain the infrared image features of the infrared image, which are then used for feature fusion processing.

[0037] In summary, by processing the infrared image separately through the infrared processing unit, it can be ensured that the infrared image features can better reflect the temperature changes and fire source location information, thereby facilitating subsequent fusion and use.

[0038] Similarly, the visible light processing unit includes a visible light input layer, a visible light convolution layer, a visible light pooling layer, a visible light encoding layer, and a visible light fully connected layer. It can complete image processing operations through the following processing methods to obtain visible light image features that accurately represent image features: A first visible light image feature corresponding to the visible light image is constructed through the visible light input layer, and a flame attribute feature is extracted from the first visible light image feature using the visible light convolution layer to obtain a second visible light image feature; the second visible light image feature is reduced into a third visible light image feature based on the visible light pooling layer, and the third visible light image feature is encoded through the visible light encoding layer to obtain a fourth visible light image feature; the fourth visible light image feature is processed through the visible light fully connected layer to obtain the visible light image feature of the visible light image.

[0039] Specifically, the visible light input layer receives the visible light image and scales and normalizes it to meet the neural network's input requirements, thereby obtaining the first visible light image feature. The visible light convolution layer uses a 5x5 convolution kernel in the first layer to extract the shape, outline, and color of the flame. Subsequent convolution layers use 3x3 convolution kernels to extract detailed features, such as the edge of the fire source and the shape of the smoke, thereby obtaining the second visible light image feature. The visible light pooling layer combines max-pooling to retain regions with high activation values ​​and reduce feature size. This helps the network focus on the core area of ​​the flame and mitigates background interference, thereby obtaining the third visible light image feature. The visible light encoding layer performs feature extraction and encoding. Through multiple convolutional layers, it gradually extracts high-order fire-related features, forming a characteristic representation of the flame's shape and color, the fourth visible light image feature. The visible light fully connected layer further processes the extracted image features to obtain high-dimensional visible light image features.

[0040] Based on this, after the visible light image is input into the fire detection model, the visible light input layer included in the visible light processing unit of the fire detection model can be used to construct the first visible light image features corresponding to the visible light image, so that the features are adapted to the model processing. The visible light convolution layer can then be used to extract flame attribute features from the first visible light image features, obtaining second visible light image features associated with the flame shape contour and forehead. At this point, the second visible light image features are then reduced to third visible light image features using the visible light pooling layer, thereby reducing background information interference. The third visible light image features are then encoded using the visible light encoding layer to extract high-order features associated with flame information, namely fourth visible light image features. Finally, the fourth visible light image features are processed using the visible light fully connected layer to obtain the visible light image features of the visible light image, which are then used in subsequent feature fusion processing to improve fire prediction accuracy.

[0041] For example, when a drone is monitoring a fire-prone area in a forest, it will use its infrared and optical sensors to collect infrared and visible light images of the area. Figure 2 As shown in the schematic diagram, infrared images and visible light images can be input into a dual-channel CNN fusion model. The dual-channel CNN fusion model includes an infrared channel convolutional network and a visible light channel convolutional network. Furthermore, the infrared image can be processed using the input layer, convolutional layer, pooling layer, deep convolutional layer and pooling layer, and fully connected layer included in the infrared channel convolutional network. Based on the processing results, infrared image features V1 representing temperature change information and the location of the fire source can be obtained. Similarly, visible light images can be processed using the input layer, convolutional layer, pooling layer, feature extraction and encoding layer, and fully connected layer included in the visible light channel convolutional network. Based on the processing results, visible light image features V2 representing visual information such as flames and smoke can be obtained. Subsequently, the infrared image features V1 and visible light image features V2 can be combined to complete fire detection processing for the current area.

[0042] In summary, by processing the visible light image separately through the visible light processing unit, it can be ensured that the visible light image features can better reflect the visualization characteristics of the flame, thereby facilitating subsequent fusion and use.

[0043] Step S106 : Utilize the feature fusion unit to perform feature fusion on the infrared image features and the visible light image features, and determine fire detection information according to the feature fusion result.

[0044] Specifically, after the infrared image features and visible light image features are extracted by the infrared processing unit and the visible light processing unit, in order to accurately feedback the fire detection results of the target area by combining the image features of the two dimensions, the feature fusion unit can be used to fuse the infrared image features and the visible light image features. The fusion result reflects the fire visualization information and fire temperature information of the associated target area, thereby determining the fire detection information based on the feature fusion result, so that the fire detection information can reflect the detection result of whether a fire has occurred in the target area, thereby achieving the purpose of prevention and timely warning. Among them, the fire detection information specifically refers to information that represents the fire detection results of the target area. When it reflects a high probability of fire occurrence, it can record information such as the location of the fire source, the fire credibility, and the fire source temperature.

[0045] Furthermore, when the feature fusion unit is used to fuse image features, in order to improve the accuracy of fire detection, the feature fusion unit can be implemented in combination with the DS evidence theory. In this embodiment, the specific implementation method is as follows: The feature fusion unit is used to calculate the fire credibility of the infrared image features and the visible light image features respectively to obtain infrared credibility information of the infrared image and visible light credibility information of the visible light image; the infrared credibility information and the visible light credibility information are fused according to a preset information synthesis rule, and fire detection information is determined based on the feature fusion result.

[0046] Specifically, infrared credibility information refers to credibility description information that characterizes the occurrence of a fire determined based on infrared image features, and visible light credibility information refers to credibility description information that characterizes the occurrence of a fire determined based on visible light image features. During specific implementation, the feature fusion unit can be realized through DS evidence theory, thereby ensuring the accuracy of fire detection information.

[0047] Based on this, in order to be able to accurately calculate the fire detection information corresponding to the target area by combining the image features of two different dimensions after obtaining the infrared image features and the visible light image features, the DS evidence theory can be used to implement the feature fusion unit, and the feature fusion unit can be used to perform fire credibility calculation on the infrared image features and the visible light image features respectively, and the infrared credibility information of the infrared image and the visible light credibility information of the visible light image are obtained based on the calculation results; on this basis, the infrared credibility information and the visible light credibility information can be fused according to the information synthesis rules of the DS evidence theory, so as to determine the fire detection information according to the feature fusion results for subsequent use.

[0048] In practical applications, the DS evidence theory can be understood as a feature fusion unit that performs credibility calculation and evidence synthesis. Credibility calculation involves calculating the credibility of each channel, known as the basic probability assignment (BPA), based on infrared and visible light image features. Evidence synthesis, on the other hand, uses the Dempster rule to fuse evidence from different channels into a unified fire detection result, ensuring the model's robustness in uncertain environments. Ultimately, the fused results output a fire confidence score, thereby determining the probability of fire occurrence.

[0049] In summary, by combining the DS evidence theory to realize the feature fusion unit, the robustness in uncertain environments can be guaranteed when calculating fire detection information, thereby ensuring the accuracy of fire detection.

[0050] Furthermore, when performing credibility calculation, it can be achieved by calculating the probability and then combining it with the distribution function. In this embodiment, the specific implementation method is as follows: The feature fusion unit is used to calculate the fire probability of the infrared image features and the visible light image features to obtain the infrared fire probability of the infrared image and the visible light fire probability of the visible light image; and the credibility of the infrared fire probability and the visible light fire probability are respectively assigned according to the probability distribution function in the feature fusion unit to obtain the infrared credibility information of the infrared image and the visible light credibility information of the visible light image.

[0051] Specifically, the infrared fire probability refers to the probability of a fire occurring or not occurring in a target area calculated based on infrared image features. Similarly, the visible light fire probability refers to the probability of a fire occurring or not occurring in a target area calculated based on visible light image features. Accordingly, the probability distribution function refers to a function that combines fire probabilities to calculate credibility information.

[0052] Based on this, when combining DS evidence theory with feature fusion, a basic confidence assignment (BPA) calculation is required. Specifically, the feature fusion unit can first calculate the fire probability of the infrared image features and the visible light image features to obtain the infrared fire probability of the infrared image and the visible light fire probability of the visible light image. Then, the credibility of the infrared fire probability and the visible light fire probability can be assigned according to the probability distribution function in the feature fusion unit, thereby obtaining the infrared credibility information of the infrared image and the visible light credibility information of the visible light image, which can be used for subsequent evidence synthesis processing.

[0053] In practical applications, the basic belief allocation (BPA) calculation can quantify the confidence level of each channel in judging fire. Therefore, in the probability calculation of fire and no fire, the prediction probabilities of the infrared channel and the visible light channel for the "fire" and "no fire" states can be calculated respectively based on the infrared image features and the visible light image features. Assume that the fire prediction probability output by the infrared channel is , the probability of no fire is Similarly, the fire prediction probability output by the visible light channel is , the probability of no fire is .

[0054] After obtaining the fire probability calculation results corresponding to each channel, the basic probability distribution function can be constructed to construct the credibility distribution based on the predicted probabilities of fire and no fire. The basic probability distribution function m can be defined by the following formulas (1) and (2): (1) (2) It can be understood that the "fire" and "no fire" probabilities in the fire prediction probability are the credibility of the fire occurrence and the credibility of the fire not occurring. This information can be used for feature fusion later.

[0055] In the specific implementation, considering that uncertain factors may lead to a decrease in the accuracy of fire detection, such as the impact of strong light searchlights and the shadow of trees, a certain amount of unassigned trust value can be added to BPA to deal with the ambiguity of judgment caused by uncertainty, such as and , used to indicate unknown situations and avoid misjudgment due to conflicting data.

[0056] On this basis, when performing credibility information fusion, it is necessary to consider whether there are conflicting behaviors in the detection results of the two channels. In this embodiment, the specific implementation method is as follows: The infrared credibility information and the visible light credibility information are subjected to conflict detection according to preset information synthesis rules; when there is no conflict between the infrared credibility information and the visible light credibility information, the infrared credibility information and the visible light credibility information are normalized to obtain fire detection information; when there is a conflict between the infrared credibility information and the visible light credibility information, the infrared credibility information and the visible light credibility information are suppressed according to the information synthesis rules, and the suppressed infrared credibility information and the visible light credibility information are normalized to obtain fire detection information.

[0057] Specifically, the information fusion rule refers to the Dempster fusion rule in DS evidence theory. Correspondingly, conflict detection involves detecting inconsistencies in fire detection results between the infrared and visible light channels. Different approaches to feature fusion are employed for different situations, thereby improving fire detection accuracy.

[0058] Based on this, when performing feature fusion, the infrared credibility information and the visible light credibility information can be firstly detected for conflicts according to the preset information synthesis rules; when there is no conflict between the infrared credibility information and the visible light credibility information, it means that the fire detection results obtained based on the two features are the same, so the infrared credibility information and the visible light credibility information can be directly normalized to obtain the fire detection information.

[0059] When the infrared credibility information and the visible light credibility information conflict, it means that the detection results of Huozi Ajiu obtained by combining the infrared image features and the visible light image features are inconsistent. In order to determine the fire detection information in this situation, the infrared credibility information and the visible light credibility information can be suppressed according to the information synthesis rules. The suppressed infrared credibility information and the visible light credibility information can be normalized through the suppression processing to obtain the fire detection information.

[0060] In practical applications, after obtaining infrared credibility information and visible light credibility information, the Dempster synthesis rule can be used to fuse the two pieces of evidence to obtain a comprehensive degree of credibility. Specifically, assuming and are the basic probability distributions for infrared and visible light channels respectively, then the synthetic credibility of fire detection is The expression is as follows formula (3): (3) The conflict metric K represents the total weight of the conflicting data, which can be calculated using the following formula (4): (4) In other words, when the infrared and visible light credibility information are consistent, the Dempster synthesis rule multiplies and normalizes the credibility of the two, thereby increasing the weight of the consistent data and ensuring high-confidence fire detection results. If the infrared and visible light credibility information are inconsistent, the conflict metric K may be large. In this case, the Dempster synthesis rule can weaken the impact of the conflicting data to reduce the false alarm rate and adjust the weight of each channel based on the newcomer allocation.

[0061] Afterwards, through DS evidence theory fusion, the final fire judgment result includes fire confidence and no fire confidence Then, we can judge whether a fire has occurred based on the preset threshold. , it is determined to be a fire and the system issues a fire warning. For the judgment of no fire, if , then it is determined that there is no fire to avoid false alarms. For uncertainty judgment, if and If the system determines that the state is uncertain, it can prompt the system to conduct further testing or adjust the data collection frequency. This allows the system to output more accurate fire detection results for subsequent use.

[0062] Continuing with the above example, after obtaining the infrared image feature V1 and the visible light image feature V2, the credibility of the output results of the infrared channel and the visible light channel can be assigned using the DS evidence theory. Specifically, the fire probability m11 and the no-fire probability m12 corresponding to the infrared image are calculated based on the infrared image feature V1; the fire probability m21 and the no-fire probability m22 corresponding to the infrared image are calculated based on the visible light image feature V2. Then, according to the above formulas (1) and (2), the fire credibility p11 and no-fire credibility p12 corresponding to the infrared image are obtained; and the fire credibility p21 and no-fire credibility p22 corresponding to the visible light image are obtained.

[0063] Furthermore, the fire credibility p11 and no-fire credibility p12 corresponding to the infrared image, and the fire credibility p21 and no-fire credibility p22 corresponding to the visible light image are fused using the above formulas (3) and (4). At this time, the fire credibility of the corresponding forest area will be obtained. If the fire credibility is greater than the fire alarm threshold, it is determined that a fire has occurred in the area, and a fire warning can be issued through the system to quickly perform fire extinguishing operations.

[0064] In summary, based on the dual-channel fire detection model, the fire detection method provided in this embodiment utilizes the DS evidence theory to construct a feature fusion unit. This unit uses basic probability assignment (BPA) to assign confidence levels to the fire or no-fire results matched by the infrared processing unit and the visible light processing unit, respectively. The Dempster synthesis rule is then used to fuse the evidence from different channels. The DS evidence theory optimizes the confidence level of fire judgments in the presence of conflicting multimodal data, effectively reducing false positives and false negatives. In practical applications, the DS evidence theory provides a unified confidence level for fire detection, enabling the system to provide stable detection results in uncertain environments.

[0065] Step S108 : When it is determined that a fire event has occurred in the target area according to the fire detection information, the fire detection information is corrected into target fire detection information using a filter.

[0066] Specifically, after obtaining the fire detection information corresponding to the target area, if a fire event is determined to have occurred in the target area based on the fire detection information, fire extinguishing preparations are required. Considering that erroneous fire detection information may affect fire extinguishing efficiency, a filter can be used to correct the fire detection information to target fire detection information. This allows for rapid location and temperature determination of the fire source, enabling rapid fire extinguishing operations and minimizing fire damage. Target fire detection information refers to information obtained after correction of the fire detection information, resulting in a closer match to the actual fire information.

[0067] Furthermore, when the filter is used to correct the fire detection information, the information that needs to be fed back can be extracted from the fire detection information and then corrected. In this embodiment, the specific implementation is as follows: Extract the initial position information and initial temperature information of the fire source corresponding to the target area from the fire detection information; use a filter to correct the initial position information and initial temperature information of the fire source to obtain fire source position information and fire source temperature information; and use the fire source position information and the fire source temperature information as the target fire detection information corresponding to the target area.

[0068] Specifically, the initial fire source location refers to the geographical location of the fire source determined by the model based on the captured image after the target area is detected and confirmed to be a fire. The initial fire source temperature information refers to the temperature distribution information determined by combining the temperature data corresponding to the infrared image. The filter is a Kalman filter.

[0069] Based on this, when it is determined that a fire has occurred in the target area, considering that the fire detection information is predictive information and may have errors with the actual information, a Kalman filter can be used to correct the fire detection information. During the correction process, the initial position information of the fire source and the initial temperature information of the fire source corresponding to the target area can be extracted from the fire detection information; the initial position information of the fire source and the initial temperature information of the fire source can be corrected using the filter to obtain the fire source position information and the fire source temperature information; the fire source position information and the fire source temperature information can be used as the target fire detection information corresponding to the target area for subsequent use.

[0070] In practical applications, the initial position coordinates of the fire source and the temperature of the fire source can be used as the measurement input data of the Kalman filter, and its state variables can be defined as the following formula (5): (5) Where x and y represent the geographical coordinates of the fire source, which can be determined by geo-mapping using the drone’s GPS system and image processing technology. T represents the temperature of the fire source, which can be determined using infrared images. The system can predict the location of the fire source based on the drone’s current flight speed, direction, and the dynamic change trend of the fire source. Assuming that the fire source changes little in a short period of time, the state transition matrix is the unit matrix, or when the fire source is detected to move, the prediction matrix is ​​adjusted according to the direction of the fire source position change. Secondly, the fire source temperature will fluctuate due to the influence of the environment, and the process noise covariance matrix of the Kalman filter is used. To reflect the uncertainty of temperature prediction and ensure reasonable prediction of temperature changes. In the update step of the Kalman filter, the visual detection results (i.e., the measurement values ​​after CNN and DS fusion) are used to correct the fire source state prediction, thereby facilitating the subsequent determination of target fire detection information.

[0071] Furthermore, when correcting information, it is necessary to first calculate the gain information, and then complete the information correction process according to the gain information. In this embodiment, the specific implementation method is as follows: A filter is used to determine a state covariance matrix and a measurement noise covariance matrix corresponding to the target area, and gain information is calculated based on the state covariance matrix and the measurement noise covariance matrix; the initial position information of the fire source and the initial temperature information of the fire source are corrected based on the gain information to obtain fire source position information and fire source temperature information.

[0072] Specifically, the state covariance matrix refers to the variance matrix corresponding to the current predicted state. The measurement noise covariance matrix refers to the variance matrix used to quantify the noise level of visual detection. The gain information refers to the Kalman gain.

[0073] Based on this, when using the Kalman filter to correct fire detection information, the filter can be used to determine the state covariance matrix and measurement noise covariance matrix corresponding to the target area. At this time, the gain information can be calculated based on the state covariance matrix and the measurement noise covariance matrix; and then the initial position information of the fire source and the initial temperature information of the fire source can be corrected based on the gain information to determine the fire source position information and the fire source temperature information according to the correction results.

[0074] In practical applications, the Kalman filter corrects the temperature and position information by taking the fire source position and temperature output by the dual-channel CNN as the measurement value of the Kalman filter. ,in and is the visually detected position of the red fire source in the previous cycle (the most likely position determined after DS fusion). is the fire source temperature detected by the infrared channel in the previous cycle. Then, the Kalman gain can be calculated based on the current predicted state covariance matrix and the measurement noise covariance matrix. The size of the gain directly affects the degree of correction of the visual detection result to the prediction result. The Kalman gain is realized by the following formula (6): (6) in, Represents the measurement matrix, which maps the predicted state to the measurement space. represents the measurement noise covariance matrix, which is used to quantify the noise level of visual inspection.

[0075] Furthermore, the predicted fire source location and temperature can be corrected using the visual detection results to obtain an estimated value closer to the actual situation, which can be achieved through the following formula (7): (7) in, Represents the observations obtained from the visual results, namely the visually detected fire location and temperature.

[0076] Finally, the estimated error covariance matrix can be updated Adjust the system's prediction uncertainty of future states, which is achieved through the following formula (8): (8) In other words, the Kalman filter loops throughout the system, continuously processing each frame of visual detection data through prediction and updates. During drone flight, each time the system acquires a new infrared or visible light image, it feeds the visual detection results into the Kalman filter. By integrating these results, the fire source status is corrected, resulting in more stable and accurate estimates of the fire source location and temperature.

[0077] Continuing with the previous example, if a fire is detected in a forest area, in order to accurately feed the system the fire source's location and quickly extinguish it, the system can first determine the fire source's initial location, initial temperature, and fire confidence level based on the fire detection information. This information is then fed into a Kalman filter to correct the initial location and temperature collected during the current cycle. The corrected location determines the fire source's location as (x, y), the fire type as a surface fire, and the fire source temperature as 400°C. By feeding this information back to the system, rapid fire extinguishing planning can be implemented to prevent the fire from spreading throughout the forest.

[0078] In summary, to enhance the system's stability during real-time monitoring, a Kalman filter can be used to smooth visual inspection data and dynamically estimate the fire source's location and temperature. The Kalman filter performs predictions and updates within each frame of data, eliminating the effects of sensor noise and ensuring accurate tracking of the fire source. The prediction step estimates the current location and temperature of the fire source based on the drone's motion and the dynamic changes in the fire source. The update step, on the other hand, integrates visual inspection results to correct prediction errors and achieve more accurate fire source state estimation. The Kalman filter is particularly suitable for maintaining system stability in situations with missing data, delays, or significant noise interference, ensuring continuous and robust fire monitoring.

[0079] In addition, considering that fire detection information needs to be fed back to the server for judgment and use, the feedback information can be combined to drive the drone to continuously perform information detection, thereby quickly achieving the firefighting purpose. In this embodiment, the specific implementation method is as follows: The target fire detection information is sent to a server, and path planning information for the drone issued by the server is received; the drone is driven to move according to the path planning information; when the drone moves to an associated area, the associated area is used as the target area, and the infrared image and visible light image collected by the drone for the target area are input into the fire detection model.

[0080] Specifically, the server is capable of planning the drone's path based on its location and fire detection information, providing significant computing power. Path planning information, in turn, refers to information used to plan the drone's flight path, including flight speed, heading angle, altitude, and flight distance.

[0081] Based on this, and considering that the initial fire location detected may not be sufficient to prove the probability of a fire, fire detection can be determined by continuously collecting images. At this point, to make the drone's image collection more accurate, the target fire detection information can be sent to the server, and the server will receive path planning information for the drone. The drone can then be driven to move according to the path planning information. When the drone moves to the associated area, the associated area is used as the target area, and the infrared and visible light images collected by the drone for the target area are input into the fire detection model. This continuous control of the drone's mobile flight makes it easier for the server to determine the presence of a fire and confirm the fire's severity.

[0082] In summary, the fire detection method provided in this embodiment enables drones to perform real-time fire detection in complex environments. In practical applications, the drone carries a dual-channel sensor (infrared and visible light) and continuously inputs image data into a dual-channel CNN for feature extraction and fusion. The system then uses the DS evidence theory to synthesize the credibility of multimodal data, and finally smoothes the detection data using a Kalman filter. This system not only accurately locates fire sources and tracks fire development trends, but also dynamically adjusts to environmental noise and uncertainty factors, effectively improving the accuracy and response speed of fire monitoring.

[0083] The following combined Figure 3 , taking the application of the fire detection method provided in this specification in a fire detection scenario in an urban area as an example, the fire detection method is further described. Figure 3 A flowchart of a fire detection method according to an embodiment of the present specification is shown, which specifically includes the following steps.

[0084] In step S302, the infrared images and visible light images collected by the drone for the target area are input into the fire detection model, where the fire detection model includes an infrared processing unit, a visible light processing unit, and a feature fusion unit; the infrared processing unit includes an infrared input layer, an infrared convolution layer, an infrared pooling layer, an infrared mining layer, and an infrared fully connected layer; the visible light processing unit includes a visible light input layer, a visible light convolution layer, a visible light pooling layer, a visible light encoding layer, and a visible light fully connected layer.

[0085] Step S304: constructing a first infrared image feature corresponding to the infrared image through the infrared input layer, and performing region recognition on the first infrared image feature using the infrared convolution layer to obtain a second infrared image feature.

[0086] Step S306 : reducing the dimension of the second infrared image feature into a third infrared image feature based on the infrared pooling layer, and performing feature mining on the third infrared image feature through the infrared mining layer to obtain a fourth infrared image feature.

[0087] Step S308: Process the fourth infrared image feature through the infrared fully connected layer to obtain the infrared image feature of the infrared image.

[0088] Step S310 , constructing a first visible light image feature corresponding to the visible light image through the visible light input layer, and extracting flame attribute features from the first visible light image feature through the visible light convolution layer to obtain a second visible light image feature.

[0089] In step S312 , the second visible light image feature is reduced into a third visible light image feature based on the visible light pooling layer, and the third visible light image feature is encoded through the visible light coding layer to obtain a fourth visible light image feature.

[0090] Step S314 : Process the fourth visible light image feature through the visible light fully connected layer to obtain the visible light image feature of the visible light image.

[0091] Step S316 , using a feature fusion unit to calculate the fire probability of the infrared image features and the visible light image features, to obtain the infrared fire probability of the infrared image and the visible light fire probability of the visible light image.

[0092] Step S318 , performing credibility distribution on the infrared fire probability and the visible light fire probability respectively according to the probability distribution function in the feature fusion unit, and obtaining infrared credibility information of the infrared image and visible light credibility information of the visible light image.

[0093] Step S320 : Fusing the infrared credibility information and the visible light credibility information according to a preset information synthesis rule, and determining the fire detection information based on the feature fusion result.

[0094] Step S322 , when it is determined that a fire event has occurred in the target area according to the fire detection information, the fire source initial position information and the fire source initial temperature information corresponding to the target area are extracted from the fire detection information.

[0095] Step S324 : using the filter to determine the state covariance matrix and the measurement noise covariance matrix corresponding to the target area, and calculating gain information according to the state covariance matrix and the measurement noise covariance matrix.

[0096] Step S326: Correct the initial fire source position information and the initial fire source temperature information based on the gain information to obtain the fire source position information and the fire source temperature information.

[0097] Step S328: Use the fire source location information and the fire source temperature information as target fire detection information corresponding to the target area.

[0098] In summary, in order to adapt to more complex monitoring environments and improve the accuracy of fire detection, fire detection can be performed by fusing visible light images and infrared images. Specifically, the infrared images and visible light images collected by the drone for the target area can be input into the fire detection model, wherein the fire detection model integrates an infrared processing unit, a visible light processing unit, and a feature fusion unit; it is used to extract infrared image features from infrared images through the infrared processing unit, and to extract visible light image features from visible light images through the visible light processing unit; after obtaining the infrared image features and visible light image features, considering that the two image features belong to different types, the feature fusion unit can be used to fuse the infrared image features and the visible light image features, thereby determining the fire detection information based on the feature fusion results; and improving the accuracy of fire detection through fusion processing. Afterwards, if a fire incident is determined to have occurred in the target area based on the fire detection information, in order to accurately detect the fire situation, the fire detection information can be corrected in combination with a filter to reduce the interference of other noises in the environment, thereby obtaining accurate target fire detection information. In the drone fire detection scenario, the fire detection accuracy is effectively improved, so that a reasonable response can be made at the moment of fire occurrence to avoid excessive economic losses.

[0099] Corresponding to the above method embodiment, this specification also provides a fire detection device embodiment, Figure 4 FIG1 shows a schematic diagram of the structure of a fire detection device provided by an embodiment of this specification. Figure 4 As shown, the device includes: An input module 402 is configured to input the infrared image and visible light image collected by the drone of the target area into the fire detection model, wherein the fire detection model includes an infrared processing unit, a visible light processing unit, and a feature fusion unit; an extraction module 404 configured to extract infrared image features of the infrared image through the infrared processing unit, and to extract visible light image features of the visible light image through the visible light processing unit; a fusion module 406 configured to perform feature fusion on the infrared image features and the visible light image features using the feature fusion unit, and determine fire detection information based on the feature fusion result; The correction module 408 is configured to, when it is determined based on the fire detection information that a fire event has occurred in the target area, use a filter to correct the fire detection information into target fire detection information.

[0100] In an optional embodiment, the infrared processing unit includes an infrared input layer, an infrared convolution layer, an infrared pooling layer, an infrared mining layer, and an infrared fully connected layer, and the extraction module 404 is further configured to: A first infrared image feature corresponding to the infrared image is constructed through the infrared input layer, and the infrared convolution layer is used to perform region recognition on the first infrared image feature to obtain a second infrared image feature; based on the infrared pooling layer, the second infrared image feature is reduced to a third infrared image feature, and the infrared mining layer is used to perform feature mining on the third infrared image feature to obtain a fourth infrared image feature; and the fourth infrared image feature is processed through the infrared fully connected layer to obtain the infrared image feature of the infrared image.

[0101] In an optional embodiment, the visible light processing unit includes a visible light input layer, a visible light convolution layer, a visible light pooling layer, a visible light encoding layer, and a visible light fully connected layer, and the extraction module 404 is further configured to: A first visible light image feature corresponding to the visible light image is constructed through the visible light input layer, and a flame attribute feature is extracted from the first visible light image feature using the visible light convolution layer to obtain a second visible light image feature; the second visible light image feature is reduced into a third visible light image feature based on the visible light pooling layer, and the third visible light image feature is encoded through the visible light encoding layer to obtain a fourth visible light image feature; the fourth visible light image feature is processed through the visible light fully connected layer to obtain the visible light image feature of the visible light image.

[0102] In an optional embodiment, the fusion module 406 is further configured to: The feature fusion unit is used to perform fire credibility calculation on the infrared image features and the visible light image features respectively to obtain infrared credibility information of the infrared image and visible light credibility information of the visible light image; the infrared credibility information and the visible light credibility information are fused according to a preset information synthesis rule, and fire detection information is determined based on the feature fusion result.

[0103] In an optional embodiment, the fusion module 406 is further configured to: The feature fusion unit is used to calculate the fire probability of the infrared image features and the visible light image features to obtain the infrared fire probability of the infrared image and the visible light fire probability of the visible light image; and the credibility of the infrared fire probability and the visible light fire probability are respectively assigned according to the probability distribution function in the feature fusion unit to obtain the infrared credibility information of the infrared image and the visible light credibility information of the visible light image.

[0104] In an optional embodiment, the fusion module 406 is further configured to: The infrared credibility information and the visible light credibility information are subjected to conflict detection according to preset information synthesis rules; when there is no conflict between the infrared credibility information and the visible light credibility information, the infrared credibility information and the visible light credibility information are normalized to obtain fire detection information; when there is a conflict between the infrared credibility information and the visible light credibility information, the infrared credibility information and the visible light credibility information are suppressed according to the information synthesis rules, and the suppressed infrared credibility information and the visible light credibility information are normalized to obtain fire detection information.

[0105] In an optional embodiment, the correction module 408 is further configured to: Extract the initial position information and initial temperature information of the fire source corresponding to the target area from the fire detection information; use a filter to correct the initial position information and initial temperature information of the fire source to obtain fire source position information and fire source temperature information; and use the fire source position information and the fire source temperature information as the target fire detection information corresponding to the target area.

[0106] In an optional embodiment, the correction module 408 is further configured to: A filter is used to determine a state covariance matrix and a measurement noise covariance matrix corresponding to the target area, and gain information is calculated based on the state covariance matrix and the measurement noise covariance matrix; the initial position information of the fire source and the initial temperature information of the fire source are corrected based on the gain information to obtain fire source position information and fire source temperature information.

[0107] In an optional embodiment, the device further includes: The mobile module is configured to send the target fire detection information to a server and receive path planning information issued by the server for the drone; drive the drone to move according to the path planning information; when the drone moves to an associated area, use the associated area as the target area and execute the step of inputting the infrared image and visible light image collected by the drone for the target area into the fire detection model.

[0108] The fire detection device provided in this embodiment can adopt the method of fusing visible light images and infrared images for fire detection in order to adapt to more complex monitoring environments and improve the accuracy of fire detection. Specifically, the infrared image and visible light image collected by the drone for the target area can be input into the fire detection model, wherein the fire detection model integrates the infrared processing unit, the visible light processing unit and the feature fusion unit; it is used to extract the infrared image features of the infrared image through the infrared processing unit, and extract the visible light image features of the visible light image through the visible light processing unit; after obtaining the infrared image features and the visible light image features, considering that the two image features belong to different types, the feature fusion unit can be used to fuse the infrared image features and the visible light image features, thereby determining the fire detection information according to the feature fusion results; and improving the accuracy of fire detection through fusion processing. Afterwards, if a fire incident is determined to have occurred in the target area based on the fire detection information, in order to accurately detect the fire situation, the fire detection information can be corrected in combination with a filter to reduce the interference of other noises in the environment, thereby obtaining accurate target fire detection information. In the drone fire detection scenario, the fire detection accuracy is effectively improved, so that a reasonable response can be made at the moment of fire occurrence to avoid excessive economic losses.

[0109] The above is a schematic diagram of a fire detection device according to this embodiment. It should be noted that the technical solution of the fire detection device and the technical solution of the fire detection method described above are based on the same concept. For details not described in detail in the technical solution of the fire detection device, please refer to the description of the technical solution of the fire detection method described above.

[0110] Figure 5 The block diagram of a computing device 500 according to one embodiment of the present disclosure is shown. Components of the computing device 500 include, but are not limited to, a memory 510 and a processor 520. The processor 520 is connected to the memory 510 via a bus 530, and a database 550 is used to store data.

[0111] The computing device 500 also includes an access device 540 that enables the computing device 500 to communicate via one or more networks 560. Examples of such networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 540 may include one or more of any type of network interface (e.g., a network interface card (NIC)) whether wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, or a near field communication (NFC) interface.

[0112] In one embodiment of the present specification, the above components of the computing device 500 and Figure 5 Other components not shown in the figure may also be connected to each other, for example, via a bus. Figure 5 The computing device structure block diagram shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art may add or replace other components as needed.

[0113] Computing device 500 can be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, personal digital assistant, laptop computer, notebook computer, netbook computer, etc.), a mobile phone (e.g., a smartphone), a wearable computing device (e.g., a smartwatch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or personal computer (PC). Computing device 500 can also be a mobile or stationary server.

[0114] The processor 520 is configured to execute the following computer-executable instructions, which implement the steps of the fire detection method when executed by the processor.

[0115] The above is a schematic diagram of a computing device according to this embodiment. It should be noted that the technical solution of the computing device and the technical solution of the fire detection method described above are of the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the fire detection method described above.

[0116] An embodiment of the present specification further provides a computer-readable storage medium storing computer-executable instructions, which implement the steps of the above-mentioned fire detection method when executed by a processor.

[0117] The above is a schematic diagram of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of the storage medium and the technical solution of the fire detection method described above are based on the same concept. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the fire detection method described above.

[0118] An embodiment of the present specification further provides a computer program product, comprising a computer program or instructions, which implement the steps of the above-mentioned fire detection method when executed by a processor.

[0119] The above is a schematic diagram of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product and the technical solution of the fire detection method described above are based on the same concept. For details not described in detail in the technical solution of the computer program product, please refer to the description of the technical solution of the fire detection method described above.

[0120] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0121] The computer instructions include computer program code, which may be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium may include any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal, and software distribution medium. It should be noted that the content of the computer-readable medium may be appropriately increased or decreased based on the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media does not include electric carrier signals and telecommunication signals.

[0122] It should be noted that for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0123] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0124] The preferred embodiments disclosed above are intended only to help illustrate this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific embodiments described. Obviously, many modifications and variations are possible based on the content of the embodiments described herein. These embodiments are selected and described in detail in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification.

Claims

1. A fire detection method, characterized in that: include: Inputting the infrared image and the visible light image collected by the drone for the target area into the fire detection model, wherein the fire detection model includes an infrared processing unit, a visible light processing unit and a feature fusion unit; Extracting infrared image features of the infrared image by the infrared processing unit, and extracting visible light image features of the visible light image by the visible light processing unit; Using the feature fusion unit to perform feature fusion on the infrared image features and the visible light image features, and determining fire detection information according to the feature fusion result; When it is determined according to the fire detection information that a fire event has occurred in the target area, the fire detection information is corrected into target fire detection information using a filter.

2. The fire detection method according to claim 1, characterized in that: The infrared processing unit includes an infrared input layer, an infrared convolution layer, an infrared pooling layer, an infrared mining layer and an infrared fully connected layer. The infrared image features of the infrared image are extracted by the infrared processing unit, including: Constructing a first infrared image feature corresponding to the infrared image through the infrared input layer, and performing region recognition on the first infrared image feature by using the infrared convolution layer to obtain a second infrared image feature; Based on the infrared pooling layer, the second infrared image feature is reduced to a third infrared image feature, and the third infrared image feature is subjected to feature mining through the infrared mining layer to obtain a fourth infrared image feature; The fourth infrared image feature is processed by the infrared fully connected layer to obtain the infrared image feature of the infrared image.

3. The fire detection method according to claim 1, characterized in that: The visible light processing unit includes a visible light input layer, a visible light convolution layer, a visible light pooling layer, a visible light encoding layer and a visible light fully connected layer. The visible light image features of the visible light image are extracted by the visible light processing unit, including: Constructing a first visible light image feature corresponding to the visible light image through the visible light input layer, and extracting flame attribute features from the first visible light image feature using the visible light convolution layer to obtain a second visible light image feature; reducing the second visible light image feature into a third visible light image feature based on the visible light pooling layer, and encoding the third visible light image feature through the visible light encoding layer to obtain a fourth visible light image feature; The fourth visible light image feature is processed by the visible light fully connected layer to obtain the visible light image feature of the visible light image.

4. The fire detection method according to claim 1, characterized in that: The step of utilizing the feature fusion unit to fuse the infrared image features and the visible light image features, and determining fire detection information according to the feature fusion result, includes: Using the feature fusion unit to perform fire credibility calculations on the infrared image features and the visible light image features respectively, to obtain infrared credibility information of the infrared image and visible light credibility information of the visible light image; The infrared credibility information and the visible light credibility information are fused according to a preset information synthesis rule, and the fire detection information is determined according to the feature fusion result.

5. The fire detection method according to claim 4, characterized in that: The using the feature fusion unit to perform fire credibility calculations on the infrared image features and the visible light image features respectively to obtain infrared credibility information of the infrared image and visible light credibility information of the visible light image includes: Utilizing the feature fusion unit to calculate the fire probability of the infrared image features and the visible light image features, to obtain the infrared fire probability of the infrared image and the visible light fire probability of the visible light image; The infrared fire probability and the visible light fire probability are respectively allocated credibility according to the probability allocation function in the feature fusion unit to obtain infrared credibility information of the infrared image and visible light credibility information of the visible light image.

6. The fire detection method according to claim 4, characterized in that: The step of fusing the infrared credibility information and the visible light credibility information according to a preset information synthesis rule and determining the fire detection information according to the feature fusion result includes: Performing conflict detection on the infrared credibility information and the visible light credibility information according to a preset information synthesis rule; In the case where there is no conflict between the infrared credibility information and the visible light credibility information, normalizing the infrared credibility information and the visible light credibility information to obtain fire detection information; In case that the infrared credibility information and the visible light credibility information conflict, the infrared credibility information and the visible light credibility information are suppressed according to the information synthesis rule, and the suppressed infrared credibility information and the visible light credibility information are normalized to obtain fire detection information.

7. The fire detection method according to claim 1, characterized in that: The method of using a filter to correct the fire detection information into target fire detection information includes: Extracting the fire source initial position information and the fire source initial temperature information corresponding to the target area from the fire detection information; Using a filter to correct the fire source initial position information and the fire source initial temperature information to obtain fire source position information and fire source temperature information; The fire source location information and the fire source temperature information are used as target fire detection information corresponding to the target area.

8. The fire detection method according to claim 7, characterized in that: The method of using a filter to correct the fire source initial position information and the fire source initial temperature information to obtain the fire source position information and the fire source temperature information includes: Determine a state covariance matrix and a measurement noise covariance matrix corresponding to the target area by using a filter, and calculate gain information according to the state covariance matrix and the measurement noise covariance matrix; The fire source initial position information and the fire source initial temperature information are corrected based on the gain information to obtain fire source position information and fire source temperature information.

9. The fire detection method according to any one of claims 1 to 8, characterized in that: After the step of using a filter to correct the fire detection information to target fire detection information is performed, the method further includes: Sending the target fire detection information to a server, and receiving the path planning information issued by the server for the drone; Driving the UAV to move according to the path planning information; When the drone moves to the associated area, the associated area is used as the target area, and the infrared image and the visible light image collected by the drone for the target area are input into the fire detection model.

10. A fire detection device, characterized in that: include: An input module is configured to input the infrared image and the visible light image collected by the drone for the target area into the fire detection model, wherein the fire detection model includes an infrared processing unit, a visible light processing unit and a feature fusion unit; an extraction module, configured to extract infrared image features of the infrared image through the infrared processing unit, and to extract visible light image features of the visible light image through the visible light processing unit; a fusion module, configured to perform feature fusion on the infrared image features and the visible light image features using the feature fusion unit, and determine fire detection information according to the feature fusion result; The correction module is configured to correct the fire detection information into target fire detection information by using a filter when it is determined that a fire event occurs in the target area based on the fire detection information.

11. A computing device, characterized in that: include: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the method described in any one of claims 1 to 9 are implemented.

12. A computer-readable storage medium, characterized in that: It stores computer executable instructions, which, when executed by a processor, implement the steps of the method described in any one of claims 1 to 9.

13. A computer program product, characterized in that The method comprises a computer program or an instruction, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 9.

Images