Fire prediction optimization method and system based on YOLO, and storage medium

Abstract

The invention relates to the technical field of fire prevention, and discloses a YOLO-based fire prediction optimization method and system, and a storage medium. The method comprises the following steps: acquiring visible light images of a target area under various environmental conditions; identifying a fire area from the visible light image, establishing a first identification model based on a YOLOv5 model, training the first identification model by using the marked visible light image, generating a second label for the infrared thermal image by the first identification model, and marking the second label; constructing an expansion set based on the marked infrared thermal image to train a first recognition model, outputting a candidate region of the fire, obtaining an infrared thermal image sequence of the candidate region in a preset time period, and recognizing a fire region and a non-fire region; and establishing a second identification model based on the neural network model, taking the first statistical magnitude and the second statistical magnitude of the fire area and the non-fire area as training data of the second identification model, and outputting a fire prediction probability, thereby improving fire prediction accuracy.

Description

Technical Field

[0001] The present application relates to the field of fire prevention technology, and in particular to a fire prediction optimization method, system, and storage medium based on YOLO. Background Art

[0002] In recent years, with global climate change and intensified human activities, the frequency and intensity of disasters such as forest fires, industrial fires, and urban fires have increased. These fires not only cause severe damage to the ecological environment but also pose a significant threat to human life and property. Therefore, how to quickly and accurately detect and predict the occurrence of fires has become a critical issue that needs to be addressed.

[0003] Traditional fire detection methods primarily rely on physical devices such as smoke detectors and temperature sensors. While these methods can detect fires to a certain extent, they suffer from shortcomings such as slow response, high false alarm rates, and limited coverage. With the rapid development of computer vision and artificial intelligence technologies, fire detection methods based on image processing have gradually become a research hotspot.

[0004] For example, a Chinese patent application with publication number CN119625911A provides a smoke and fire detection method based on an enhanced YOLO algorithm, including uploading the acquired data to be labeled to a data labeling platform for labeling to construct a training data set, wherein the data to be labeled is smoke and fire image data to be labeled; inputting the training data set into a smoke and fire detection model based on enhanced YOLOv5 for training, wherein the smoke and fire detection model includes an input module, a backbone network, a neck network and a prediction module; accelerating the convergence of the smoke and fire detection model during training through a focal loss function to obtain a trained smoke and fire detection model; uploading the data to be predicted to a model prediction platform, applying the trained smoke and fire detection model for prediction, and outputting the final prediction result.

[0005] Another example is Chinese patent application CN114445729A, which provides a small-target fire detection method based on an improved YOLO algorithm. The method includes the following steps: S100: establishing a fire dataset, collecting images of fires and smoke, including images of flames and smoke under different weather conditions and lighting conditions; S200: labeling the dataset; S300: establishing an improved YOLO-V3 algorithm framework; and S400: pre-training the network. The improved YOLO-V3 model is pre-trained using the fire image training dataset to obtain prediction frames of different scales.

[0006] However, the above-mentioned existing technologies mainly rely on visible light image data, lack the fusion of multimodal data such as infrared thermal imaging and temperature sensors, and are not robust enough in dynamic environments, resulting in reduced fire prediction accuracy. Summary of the Invention

[0007] To solve the above technical problems, the present application provides an optimized method, system and storage medium for fire prediction based on YOLO, which is used to improve the prediction accuracy of fire areas.

[0008] In a first aspect, the present application provides an optimized method for fire prediction based on YOLO, and the method includes: Step S1: Obtain historical image data of a target area under various environmental conditions, where the historical image data includes visible light images and infrared thermal images, and perform preprocessing operations on the historical image data; Step S2: Identify the fire area from the visible light images and label it with a first label, establish a first recognition model based on the YOLOv5 model, train the first recognition model with the labeled visible light images, and generate and label a second label for the infrared thermal images based on the trained first recognition model; Step S3: Construct an extended set based on the labeled infrared thermal images, use the extended set to perform secondary training on the first recognition model, the first recognition model outputs candidate areas of the fire, obtain an infrared thermal image sequence of the candidate areas within a preset time period, and identify the fire areas and non-fire areas in the candidate areas; Step S4: Calculate the first statistic and the second statistic of the fire areas and non-fire areas in the infrared thermal image sequence, establish a second recognition model based on the neural network model, use the first statistic and the second statistic as the training data of the second recognition model, and output the fire prediction probability.

[0009] Combined with the first aspect, in the first implementation manner of the first aspect of the present application, generating and labeling a second label for the infrared thermal image includes: The first recognition model obtains a first detection result based on the visible light image with the first label, the first detection result includes the boundary information and position information of the fire area, constructs a multi-modal feature space based on various environmental conditions, the multi-modal feature space includes a first modal space and a second modal space, projects the first detection result onto the second modal space based on the homography transformation matrix to obtain a second detection result, and uses the second detection result as the second label of the infrared thermal image and performs labeling.

[0010] Combined with the first aspect, in the second implementation manner of the first aspect of the present application, constructing a multi-modal feature space based on various environmental conditions includes: The environmental conditions include time information, environmental temperature, shooting distance, sensor type, and lens cleanliness. The first type of features and the second type of features are respectively extracted from the visible light image and the infrared thermal image. Each environmental condition is used as the coordinate axis of the condition, and the first type of features and the second type of features are used as the coordinate axis of the feature quantity. The visible light image and the infrared thermal image are calibrated and aligned. The coordinate representations of the first modal space and the second modal space are respectively defined by combining the corresponding condition axis coordinates and feature quantity axis coordinates of the visible light image and the infrared thermal image.

[0011] Combined with the first aspect, in the third implementation manner of the first aspect of the present application, an extended set is constructed based on the annotated infrared thermal image, including: Project the infrared thermal image labeled with the second label into the first modal space to generate a first data set under nighttime visible light conditions. An extended set is constructed based on the first data set and the visible light image labeled with the first label.

[0012] Combined with the first aspect, in the fourth implementation manner of the first aspect of the present application, identifying the fire area and the non-fire area in the candidate area includes: The candidate area is divided into multiple area blocks, and the feature quantity of each area block in each frame of the infrared thermal image sequence is calculated. The feature quantity includes the target pixel density index and the dynamic target ratio. If the target pixel density index of the area block is greater than the first threshold, the area block is marked as a high-temperature area. If the dynamic target ratio of the area block is less than the second threshold, the area block is marked as an external area. The first number of frames and the second number of frames in which the area blocks in the infrared thermal image sequence are marked as the high-temperature area and the external area are counted. The area blocks that simultaneously satisfy the condition that the first number of frames is greater than the third threshold and the second number of frames is less than the fourth threshold are merged to obtain the fire area, otherwise it is defined as the non-fire area.

[0013] Combined with the first aspect, in the fifth implementation manner of the first aspect of the present application, calculating the feature quantity of each area block in each frame of the infrared thermal image sequence includes: A dynamic threshold is set based on the environmental data in the target area. The target pixel density index is the ratio of the number of pixels with a temperature higher than the dynamic threshold in each area block to the total number of pixels in the area block. In the normal state without fire, multiple frames of infrared thermal images of the target area are collected. The infrared thermal images are divided into multiple area blocks, and the kernel density estimation is calculated for the pixel values of each area block to obtain the background probability distribution. The difference value between the current frame and the background probability distribution is calculated. The pixels with a difference value greater than the fourth threshold are defined as dynamic target pixels. The ratio of the number of dynamic target pixels to the total number of pixels in the area block is defined as the dynamic target ratio.

[0014] In combination with the first aspect, in the sixth implementation manner of the first aspect of the present application, calculating the first statistic and the second statistic of the fire area and the non-fire area includes: The first statistic is the first mean value and the second mean value of the fire area, and the second statistic is the first mean value and the second mean value of the non-fire area. The first mean value is the average value of the proportion of all target pixels in the infrared thermal image sequence, and the second mean value is the average value of the proportion of dynamic targets in the infrared thermal image sequence.

[0015] In a second aspect, the present application provides an optimized fire prediction system based on YOLO. The system includes: A preprocessing module for obtaining historical image data of a target area under various environmental conditions. The historical image data includes visible light images and infrared thermal images, and performing preprocessing operations on the historical image data; A labeling module for identifying a fire area from the visible light image and labeling it with a first label, establishing a first recognition model based on the YOLOv5 model, training the first recognition model with the labeled visible light images, and generating and labeling a second label for the infrared thermal image based on the trained first recognition model; An identification module for constructing an extended set based on the labeled infrared thermal image, using the extended set to perform secondary training on the first recognition model. The first recognition model outputs candidate areas of a fire, obtaining an infrared thermal image sequence of the candidate areas within a preset time period, and identifying the fire area and the non-fire area in the candidate areas; A prediction module for calculating the first statistic and the second statistic of the fire area and the non-fire area in the infrared thermal image sequence, establishing a second recognition model based on a neural network model, using the first statistic and the second statistic as training data of the second recognition model, and outputting a fire prediction probability.

[0016] A third aspect of the present application provides a computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When it runs on a computer, it causes the computer to execute the above-mentioned optimized fire prediction method based on YOLO.

[0017] Compared with the prior art, the beneficial effects of the present invention are at least as follows: The present invention collects visible light and infrared thermal images containing various environmental conditions (such as day and night, temperature and humidity changes), covering different spectral information, enabling the model to adapt to complex scenarios (such as low light at night, rain and fog interference); uses the YOLOv5 model trained with visible light images to automatically generate labels (second labels) for infrared thermal images, avoiding the cumbersome process of manually annotating infrared data and significantly reducing the annotation cost; projects the visible light detection results into the infrared modal space through a homography transformation matrix to solve the spatial alignment problem between visible light and infrared images and achieve cross-modal feature fusion.

[0018] The present invention also projects the annotated infrared images into the visible light modal space to generate an extended dataset under night-time visible light conditions, solving the problem of night-time fire detection; uses the extended dataset to retrain the YOLO model to enhance the candidate region detection ability of the model under complex conditions such as low light and sensor contamination; distinguishes instantaneous interference from real fire sources by analyzing the candidate regions in the infrared thermal image sequence and combining multi-frame statistics (such as the number of consecutive frames of a high-temperature region); calculates the statistics of fire / non-fire regions (mean proportion of target pixels, mean proportion of dynamic targets), fuses spatio-temporal features, avoids the limitations of single-frame detection, and converts the statistics into a fire probability through a neural network model (second recognition model) to support hierarchical early warning (such as low risk, high risk) and improve decision-making flexibility. Description of the Drawings

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0020] Figure 1 It is a schematic diagram of an embodiment of the YOLO-based fire prediction optimization method in the embodiments of the present application; Figure 2 It is a flow chart of candidate region recognition in the embodiments of the present application; Figure 3 It is a flow chart of recognizing fire regions and non-fire regions in the candidate regions in the embodiments of the present application; Figure 4 It is a schematic diagram of an embodiment of the YOLO-based fire prediction optimization system in the embodiments of the present application. Detailed Embodiments

[0021] The embodiments of the present application provide an optimized method, system and storage medium for fire prediction based on YOLO. The terms "first", "second", "third", "fourth", etc. (if any) in the specification, claims and the above-mentioned drawings of the present application are used to distinguish similar objects and do not necessarily describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments described here can be implemented in an order different from that illustrated or described here. In addition, the term "including" or "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units does not necessarily limit to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0022] For ease of understanding, the specific process of the embodiments of the present application will be described below. Please refer to Figure 1 , an embodiment of the optimized method for fire prediction based on YOLO in the embodiments of the present application includes: Step S1, obtain historical image data of the target area under various environmental conditions. The historical image data includes visible light images and infrared thermal images, and perform preprocessing operations on the historical image data.

[0023] Specifically, the present invention is applicable to fire prediction in complex environments (such as outdoor factories, tunnels, forests, etc.), especially in scenarios where visible light is blocked by smoke or there are drastic changes in light. Historical image data refers to visible light images and infrared thermal images collected in the target area (the present invention takes a forest as an example) under different environmental conditions. Visible light images are images collected using ordinary cameras or video cameras, mainly relying on natural light or artificial light sources, but the effect is not good under conditions of smoke, darkness or strong light; infrared thermal images are images collected using infrared thermal imagers, which can capture the temperature distribution on the surface of objects. The advantage is that it is not affected by light conditions and can penetrate a certain degree of smoke; the disadvantage is that the image resolution is relatively low and the details are not as rich as visible light images.

[0024] Different environmental conditions include different times (such as day, night, dusk, dawn, etc.), different weathers (such as sunny, cloudy, rainy, snowy, etc.), different seasons (such as spring, summer, autumn, winter, etc.), different light conditions (such as strong light, weak light, shadow, etc.) and different smoke concentrations (such as no smoke, light smoke, heavy smoke, etc.). By obtaining image data under various environmental conditions, more accurate real-time monitoring and prediction can be achieved.

[0025] Step S2: Identify the fire area from the visible light image and label it with the first label. Establish a first recognition model based on the YOLOv5 model, train the first recognition model using the labeled visible light images, and generate and label a second label for the infrared thermal image based on the trained first recognition model.

[0026] Specifically, use a common YOLO recognition algorithm (such as YOLOv3) or deep learning method to identify the fire area (the area within the red box) in the picture, and label the bounding box of the fire area with the first label (fire). Suppose there is a dataset of 1000 visible light images of fires under different environmental conditions, and each image is labeled with the fire area. Establish a first recognition model based on the YOLOv5 model. The first recognition model consists of multiple convolutional neural network layers and can perform feature extraction and object detection simultaneously. Input the labeled visible light image dataset into the YOLOv5 model for training. After multiple iterations, the model can identify the fire area in the visible light image.

[0027] However, visible light images are easily affected by various factors, such as smoke, light changes, tree occlusion, etc. It is difficult to accurately identify the fire area in visible light images at night. Infrared thermal images are not affected by changes in lighting conditions, can penetrate a certain degree of smoke, and clearly show high-temperature areas. In the case of smoke or when the fire area is blocked by trees, they can still provide more reliable fire area information. Project the first detection result (the fire area bounding box) of the YOLOv5 model trained based on visible light images into the second modality space (infrared image), and the second label (such as the high-temperature area marker in the corresponding area of the infrared image). The specific process will be elaborated later.

[0028] Step S3: Construct an extended set based on the labeled infrared thermal images, use the extended set to retrain the first recognition model. The first recognition model outputs candidate areas of the fire, obtain the infrared thermal image sequence of the candidate areas within a preset time period, and identify the fire areas and non-fire areas in the candidate areas.

[0029] Specifically, as Figure 2 shown, it is the flowchart of candidate area recognition. Obtain the infrared thermal image sequence of the candidate area within a preset time (such as the past day and night) (assuming there are N frames of infrared thermal images), record the timestamp information of each infrared thermal image, extract the dynamic features of the candidate area, and based on the results of dynamic feature analysis, combined with the initial recognition results of the first recognition model, identify the fire areas and non-fire areas in the candidate area. The specific process will be elaborated later.

[0030] Step S4: Calculate the first statistic and the second statistic for the fire area and the non-fire area in the infrared thermal image sequence, establish a second recognition model based on the neural network model, use the first statistic and the second statistic as the training data of the second recognition model, and output the fire prediction probability.

[0031] Specifically, the first statistic refers to the change in the proportion of the high-temperature area and the change in the proportion of dynamic targets (flames, smoke) in the fire area within a preset time period, and the second statistic refers to the change in the proportion of the high-temperature area and the change in the proportion of dynamic targets (leaves, bird flocks) in the non-fire area within a preset time period. The second recognition model is trained using the first statistic and the second statistic. The second recognition model identifies the change trends of the first statistic and the second statistic, obtains a new infrared thermal image, first inputs the infrared thermal image into the first recognition model to obtain candidate regions, then identifies the fire area and the non-fire area of the candidate regions and inputs them into the second recognition model, and the second recognition model outputs the prediction probability of a future fire.

[0032] In a specific embodiment, generating a second label and annotating the infrared thermal image specifically includes the following steps: The first recognition model obtains a first detection result based on the visible light image of the first label. The first detection result includes the boundary information and the position information of the fire area. A multi-modal feature space is constructed based on various environmental conditions. The multi-modal feature space includes a first modal space and a second modal space. The first detection result is projected onto the second modal space based on the homography transformation matrix to obtain a second detection result, and the second detection result is used as the second label of the infrared thermal image and is annotated.

[0033] Specifically, the visible light image is input into a pre-trained first recognition model (YOLOv5 model), and the model outputs the first detection result of the fire area in each visible light image. The boundary information includes the bounding box coordinates and position information of the fire area. A multi-modal feature space is constructed considering different environmental conditions such as time, weather, lighting conditions, etc. The first modal space (the feature space of the visible light image) includes features such as color, texture, and shape, and the second modal space (the feature space of the infrared thermal image) mainly includes temperature distribution features. Feature points are matched between the visible light image and the infrared thermal image. For example, algorithms such as SIFT and SURF are used. According to the matched feature points, the homography transformation matrix from the visible light image to the infrared thermal image is calculated. Using the calculated homography transformation matrix, the coordinate points in the visible light image are projected into the coordinate system of the infrared thermal image. According to the transformed bounding box coordinates, the position information of the fire area in the infrared thermal image is updated. Suppose the bounding box coordinates of a fire area in the visible light image are [100, 150, 200, 300]. After homography transformation, the coordinates in the infrared thermal image are [120, 160, 220, 320]. The transformed coordinates are used as the second label to annotate the infrared thermal image at the same timestamp.

[0034] In a specific embodiment, constructing a multi-modal feature space based on various environmental conditions specifically includes the following steps: The environmental conditions include time information, environmental temperature, shooting distance, sensor type, and lens cleanliness. The first type of features and the second type of features are respectively extracted from the visible light image and the infrared thermal image. Each environmental condition is used as the conditional axis coordinate, and the first type of features and the second type of features are used as the feature quantity axis coordinates. The visible light image and the infrared thermal image are calibrated and aligned, and the combinations of the corresponding conditional axis coordinates and feature quantity axis coordinates of the visible light image and the infrared thermal image are respectively defined as the coordinate representations of the first modal space and the second modal space.

[0035] Specifically, a pre-trained convolutional neural network (such as ResNet, VGG, Inception, etc.) is used to extract the deep features of the visible light image. The first type of features includes color features, texture features, shape features, etc., and the second type of features includes temperature features, heat distribution features, etc. Each environmental condition is used as the conditional axis coordinate. For example, the time information is discretized into different time periods, such as day and night; the environmental temperature is discretized into different temperature ranges, such as low temperature, medium temperature, and high temperature; the shooting distance is discretized into different distance ranges, such as near, medium, and far; the sensor type is discretized into different sensor types, such as visible light sensor and infrared thermal sensor; the lens cleanliness is discretized into different cleanliness levels, such as clean, slightly polluted, and severely polluted.

[0036] Taking the first type of features (visible light image features) and the second type of features (infrared thermal image features) as the coordinate axes of feature quantities, and combining the conditional axis coordinates with the feature quantity axis coordinates, they are respectively defined as the coordinate representations of the first modal space (visible light image) and the second modal space (infrared thermal image). For example, the first modal space coordinates = [time information, ambient temperature, shooting distance, sensor type, lens cleanliness, visible light image features]; the second modal space coordinates = [time information, ambient temperature, shooting distance, sensor type, lens cleanliness, infrared thermal image features].

[0037] In a specific embodiment, constructing the extended set based on the annotated infrared thermal images specifically includes the following steps: Project the infrared thermal images labeled with the second label into the first modal space to generate the first data set under night visible light conditions, and construct the extended set based on the first data set and the visible light images labeled with the first label.

[0038] Specifically, for the first modal space [time information, ambient temperature, shooting distance, sensor type, lens cleanliness, visible light image features], using the homography transformation matrix, project the coordinate points of the infrared thermal image into the coordinate system of the visible light image, and map the temperature distribution of the projected infrared thermal image into the color space of the visible light image to generate a simulated night visible light image. Suppose in an infrared thermal image, the temperature of a certain area is 30 , after mapping to the color space of the visible light image, this area is displayed as blue. Merge the first data set (simulated night visible light images) with the existing data set of visible light images labeled with the first label to form an extended set, ensuring that the number of samples of each category in the extended set is relatively balanced. For example, the ratio of the fire area to the non-fire area is appropriate. Under night or low light conditions, the diversity of the data set is crucial for the training of the model. This method can provide more training samples and improve the generalization ability of the model.

[0039] In a specific embodiment, identifying the fire area and the non-fire area in the candidate area specifically includes the following steps: Divide the candidate area into multiple area blocks, calculate the feature quantities of each area block in each frame of the infrared thermal image sequence. The feature quantities include the target pixel density index and the proportion of dynamic targets. If the target pixel density index of the area block is greater than the first threshold, mark the area block as a high-temperature area. If the proportion of dynamic targets in the area block is less than the second threshold, mark the area block as a foreign area, and count the first number of frames and the second number of frames of the area blocks marked as high-temperature areas and foreign areas in the infrared thermal image sequence. Merge the area blocks that simultaneously satisfy the condition that the first number of frames is greater than the third threshold and the second number of frames is less than the fourth threshold to obtain the fire area, otherwise define it as a non-fire area.

[0040] Specifically, such asFigure 3 As shown in the figure, it is a flowchart for identifying fire areas and non-fire areas in candidate areas. The candidate areas are divided into fixed-size grids (such as 16×16 pixels). For each area block, the following three types of feature quantities are calculated. Among them, the target pixel density index represents the density of high-temperature pixels (exceeding the set dynamic threshold) in the area, which is used to better distinguish between fire areas and non-fire areas. Because fire areas appear as high-temperature hotspots in infrared images, high TPDI values indicate that there is a concentrated heat source in the area, while non-fire areas (such as vegetation and buildings) have uniform temperature distribution and low TPDI values.

[0041] The dynamic target ratio indicates the proportion of pixels within a region with drastic temperature changes, reflecting the dynamic characteristics of the target (such as moving objects or transient interference). It is used to eliminate dynamic interference. For example, moving objects such as birds and vehicles appear as short-term high-temperature areas in infrared images, but their temperature changes greatly.

[0042] If, among five infrared thermal images, area block A in three frames is marked as a high-temperature area, and in two frames is marked as an external area, then the area block is a non-fire area, because at most one of the five area blocks is allowed to be marked as an external area before the area is defined as a fire area.

[0043] In a specific embodiment, calculating the feature value of each region block in each frame of the infrared thermal image sequence specifically includes the following steps: A dynamic threshold is set based on the environmental data in the target area. The target pixel density index is the ratio of the number of pixels in each area block with a temperature higher than the dynamic threshold to the total number of pixels in the area block. Multiple frames of infrared thermal images of the target area are collected under normal conditions without fire. The infrared thermal images are divided into multiple area blocks. The kernel density estimation is calculated for the pixel values of each area block to obtain the background probability distribution. The difference between the current frame and the background probability distribution is calculated. The pixels with a difference value greater than the fourth threshold are defined as dynamic target pixels. The ratio of the number of dynamic target pixels to the total number of pixels in the area block is defined as the dynamic target ratio.

[0044] Specifically, considering the environmental conditions within the target area, such as environmental temperature, humidity, wind speed, etc., these factors will affect the temperature distribution of the infrared thermal image. Therefore, the temperature threshold is dynamically adjusted according to the environmental conditions. In the normal state without fire, multiple frames of infrared thermal images of the target area are collected (such as continuous 24-hour data). The collected infrared thermal images are divided into multiple region blocks, and the division method can be the same as the region block division method in step S4. Pixel values are extracted from each region block, and a suitable kernel function (such as a Gaussian kernel function) is selected to perform kernel density estimation on the pixel values of each region block to generate a background probability distribution. Each frame of the infrared thermal image sequence within a preset time is compared with the background probability distribution, and the difference value between each pixel and the background probability distribution is calculated. The difference value can be defined as the absolute value of the difference between the pixel value and the background mean. Assuming the fourth threshold is 10, if the difference value of a certain pixel is 15, then this pixel is defined as a dynamic target pixel. The number of dynamic target pixels in each region block is counted, and the ratio of the number of dynamic target pixels to the total number of pixels in the region block is defined as the dynamic target ratio. A dynamic target refers to an object or area that changes in position or shape in consecutive frame images. In the scenario of fire monitoring, dynamic targets may include flames, smoke, and moving objects. The dynamic changes of flames and smoke are important characteristics of a fire, while static high-temperature areas (such as the ground irradiated by sunlight) are not fires. Through frame difference calculation and comparison of the background probability distribution, dynamic targets can be effectively identified and their ratios calculated. These dynamic features combined with other features (such as temperature information, the ratio of frame-difference pixels, etc.) can more accurately identify the fire area, reduce false alarms and missed alarms, and improve the overall performance of the fire monitoring system.

[0045] Setting a dynamic threshold based on the environmental data within the target area specifically includes the following steps: Obtain the highest temperature value based on the environmental data of the target area, obtain the first infrared thermal image of the fire area taken by a pollution-free infrared thermal imager, determine the temperature distribution in the first infrared thermal image, determine the first temperature value based on the temperature distribution, and calculate the dynamic threshold based on the first formula The first formula is: where, is the total light energy at the first temperature value, is the total light energy at the environmental temperature, is the transmittance when the pollution on the infrared thermal imager used for the infrared thermal image reaches the preset limit, is the Stefan-Boltzmann law constant, is a preset constant.

[0046] Specifically, when the state where the optical system of the infrared thermal imaging is not contaminated is obtained through multiple experiments, the temperature threshold of the fire area, that is, the first temperature value, can be stably extracted. Assume that in the experiment, by collecting multiple frames of infrared thermal images and analyzing the temperature data of the flame area, the average temperature of the flame area is 250 , the maximum temperature of the flame area is 300 , the average temperature of the background area is 30 , then a reasonable temperature threshold can be determined, for example, 200 (80% of the average temperature of the flame area), as the first temperature value = 200 ), the total light energy is 568 The calculation process is as follows: First, convert the temperature = 200 Kelvin, = 200 + 273.15 = 473.15K. Based on the Stefan-Boltzmann law calculate , where, is the material emissivity. In a forest fire, the typical emissivity of dry wood is ε = 0.2, then = 1840 ; is the total light energy at the ambient temperature (such as 30 ). Similar to the above method, calculate and obtain = 412 , is the pollution transmittance (reflecting the degree of sensor pollution, such as light attenuation caused by lens dust accumulation). Assume = 0.2, , substitute to obtain , convert it to Celsius as T = 363K - 273.15K = 89.85 , so when the temperature threshold in the infrared thermal image is greater than 89.85 , it is set as a high-temperature pixel. Through the first formula, when the ambient temperature rises, the threshold is automatically increased (to avoid false alarms), and when the ambient temperature drops, the threshold is automatically decreased (to avoid missed alarms). The pollution coefficient is obtained through regular calibration, and the energy loss caused by pollution is automatically compensated in the first formula, reducing the error caused by sensor pollution.

[0047] In a specific embodiment, calculating the first statistic and the second statistic of the fire area and the non-fire area specifically includes the following steps: The first statistic is the first mean and the second mean of the fire area, and the second statistic is the first mean and the second mean of the non-fire area. The first mean is the average value of the proportion of all target pixels in the infrared thermal image sequence, and the second mean is the average value of the proportion of dynamic targets in the infrared thermal image sequence.

[0048] Specifically, the first mean of the fire area refers to the average value of the proportion of target pixels in all high-temperature areas in the infrared thermal image sequence. Traverse each frame of the infrared thermal image sequence, calculate the proportion of target pixels in each high-temperature area, and calculate the average value.

[0049] For example, assume that in 10 frames of images, the proportions of target pixels in the fire area are 60%, 65%, 70%, 55%, 80%, 75%, 65%, 70%, 68%, and 72% respectively. Then the first mean is (60% + 65% + 70% + 55% + 80% + 75% + 65% + 70% + 68% + 72%) / 10 = 68%. The second mean of the fire area refers to the proportion of dynamic target pixels (such as moving flames, smoke, etc.) in the total number of pixels in the area, and no more examples will be given. The first mean of the non-fire area is the average value of the proportion of target pixels in all high-temperature areas, and the second mean refers to the proportion of dynamic target pixels (such as blowing leaves, birds, etc.) in the total number of pixels in the area.

[0050] The above describes a YOLO-based fire prediction optimization method in the embodiments of the present application. Next, the YOLO-based fire prediction optimization system in the embodiments of the present application will be described. Please refer to Figure 4 , an embodiment of a YOLO-based fire prediction optimization system in the embodiments of the present application includes: A preprocessing module for obtaining historical image data of the target area under various environmental conditions. The historical image data includes visible light images and infrared thermal images, and performing preprocessing operations on the historical image data.

[0051] A labeling module for identifying the fire area from the visible light image and labeling it with the first label, establishing a first recognition model based on the YOLOv5 model, training the first recognition model with the labeled visible light image, and generating and labeling the second label for the infrared thermal image based on the trained first recognition model.

[0052] A recognition module for constructing an extended set based on the labeled infrared thermal image, using the extended set to perform secondary training on the first recognition model. The first recognition model outputs candidate areas of the fire, obtains the infrared thermal image sequence of the candidate areas within a preset time period, and identifies the fire area and the non-fire area in the candidate areas.

[0053] A prediction module, configured to calculate a first statistic and a second statistic for a fire area and a non-fire area in an infrared thermal image sequence, establish a second recognition model based on a neural network model, use the first statistic and the second statistic as training data for the second recognition model, and output a fire prediction probability.

[0054] The present application also provides a computer-readable storage medium. The computer-readable storage medium can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions run on a computer, the computer is caused to execute the steps of the optimized fire prediction method based on YOLO.

[0055] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described system, system, and unit can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0056] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The foregoing storage medium includes: various media such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disc that can store program codes.

[0057] As described above, the above embodiments are only used to illustrate the technical solutions of the present application and are not intended to limit them. Although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments or perform equivalent replacements for some of the technical features. However, such modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the various embodiments of the present application.

Claims

1. An optimized method for fire prediction based on YOLO, characterized in that The method includes: Step S1: Obtain historical image data of a target area under multiple environmental conditions, where the historical image data includes visible light images and infrared thermal images, and perform preprocessing operations on the historical image data; Step S2: Identify the fire area from the visible light images and label it with a first label. Establish a first recognition model based on the YOLOv5 model, train the first recognition model using the labeled visible light images, and generate and label a second label for the infrared thermal images based on the trained first recognition model; Step S3: Construct an extended set based on the labeled infrared thermal images, use the extended set to perform secondary training on the first recognition model, the first recognition model outputs candidate areas of the fire, obtain an infrared thermal image sequence of the candidate areas within a preset time period, and identify the fire areas and non-fire areas in the candidate areas; Step S4: Calculate a first statistic and a second statistic of the fire areas and non-fire areas in the infrared thermal image sequence, establish a second recognition model based on a neural network model, use the first statistic and the second statistic as training data of the second recognition model, and output a fire prediction probability.

2. The method according to claim 1, characterized in that, Generating and labeling a second label for the infrared thermal image includes the following steps: The first recognition model obtains a first detection result based on the visible light image with the first label. The first detection result includes boundary information and position information of the fire area. Construct a multi-modal feature space based on multiple environmental conditions. The multi-modal feature space includes a first modal space and a second modal space. Project the first detection result onto the second modal space based on a homography transformation matrix to obtain a second detection result, and use the second detection result as the second label of the infrared thermal image and label it.

3. The method according to claim 2, wherein Constructing a multi-modal feature space based on multiple environmental conditions includes the following steps: The environmental conditions include time information, environmental temperature, shooting distance, sensor type, and lens cleanliness. Extract first-type features and second-type features from the visible light images and the infrared thermal images respectively. Use each environmental condition as the coordinate of the condition axis, and the first-type features and the second-type features as the coordinate of the feature quantity axis. Calibrate and align the visible light images and the infrared thermal images, and define the combinations of the corresponding condition axis coordinates and feature quantity axis coordinates of the visible light images and the infrared thermal images as the coordinate representations of the first modal space and the second modal space respectively.

4. The method according to claim 3, wherein Constructing an extended set based on the labeled infrared thermal images includes the following steps: Project the infrared thermal image labeled with the second label onto the first modal space to generate a first data set under night visible light conditions, and construct an extended set based on the first data set and the visible light image labeled with the first label.

5. The method according to claim 1, wherein Identifying the fire areas and non-fire areas in the candidate areas includes the following steps: Divide the candidate region into multiple region blocks, calculate the feature quantities of each region block in each frame of the infrared thermal image sequence, where the feature quantities include the target pixel density index and the proportion of dynamic targets. If the target pixel density index of the region block is greater than the first threshold, mark the region block as a high-temperature region. If the proportion of dynamic targets in the region block is less than the second threshold, mark the region block as a foreign region. Count the first number of frames and the second number of frames in the infrared thermal image sequence where the region blocks are marked as the high-temperature region and the foreign region, respectively. Merge the region blocks that simultaneously satisfy the condition that the first number of frames is greater than the third threshold and the second number of frames is less than the fourth threshold to obtain the fire region, otherwise define it as a non-fire region.

6. The method according to claim 5, characterized in that Calculating the feature quantities of each region block in each frame of the infrared thermal image sequence includes the following steps: Set a dynamic threshold based on the environmental data within the target region. The target pixel density index is the ratio of the number of pixels with a temperature higher than the dynamic threshold in each region block to the total number of pixels in the region block. Collect multiple frames of infrared thermal images of the target region in a normal state without fire, divide the infrared thermal images into multiple region blocks, calculate the kernel density estimate for the pixel values of each region block to obtain the background probability distribution, calculate the difference value between the current frame and the background probability distribution, define the pixels with a difference value greater than the fourth threshold as dynamic target pixels, and define the ratio of the number of dynamic target pixels to the total number of pixels in the region block as the proportion of dynamic targets.

7. The method according to claim 1, wherein Calculating the first statistic and the second statistic of the fire region and the non-fire region includes the following steps: The first statistic is the first mean and the second mean of the fire region, and the second statistic is the first mean and the second mean of the non-fire region. The first mean is the average of the proportion of all target pixels in the infrared thermal image sequence, and the second mean is the average of the proportion of dynamic targets in the infrared thermal image sequence.

8. An optimized fire prediction system based on YOLO, which is used to implement the optimized fire prediction method based on YOLO as described in any one of claims 1-7, characterized in that, The system includes: A preprocessing module for obtaining historical image data of the target region under various environmental conditions, where the historical image data includes visible light images and infrared thermal images, and performing preprocessing operations on the historical image data; A labeling module for identifying the fire region from the visible light images and labeling it with a first label, establishing a first recognition model based on the YOLOv5 model, training the first recognition model using the labeled visible light images, and generating and labeling a second label for the infrared thermal images based on the trained first recognition model; An identification module for constructing an extended set based on the labeled infrared thermal images, using the extended set to perform secondary training on the first recognition model, where the first recognition model outputs candidate regions of the fire, obtaining the infrared thermal image sequence of the candidate regions within a preset time period, and identifying the fire region and the non-fire region in the candidate regions; A prediction module for calculating the first statistic and the second statistic of the fire region and the non-fire region in the infrared thermal image sequence, establishing a second recognition model based on a neural network model, using the first statistic and the second statistic as the training data of the second recognition model, and outputting the fire prediction probability.

9. A computer-readable storage medium having instructions stored thereon, characterized in that, When the instruction is executed by a processor, it implements the YOLO-based fire prediction optimization method according to any one of claims 1-7.

Images