A flame detection method and system based on image recognition

Abstract

This invention relates to the field of target detection technology, and more particularly to a flame detection method and system based on image recognition. The method includes: extracting industrial monitoring video frames and creating a sample set containing flame masks and label matrices; extracting multi-dimensional features of the flame and background to calculate flame saliency; simultaneously calculating background interference complexity and boundary confusion; and adaptively determining sample weights by fusing flame saliency, background interference complexity, and boundary confusion to construct a weighted loss function to train a CNN flame detection model. In field use, a normalized real-time monitoring image is input into the model, and after confidence filtering and coordinate inverse normalization, a standardized flame localization result is output. This invention improves the detection rate of occluded flames and reduces false alarms in industrial scenarios by quantifying flame and background interference features in multiple dimensions and adaptively weighting the training of the CNN flame detection model.

Description

Technical Field

[0001] This invention relates to the field of target detection technology, and more particularly to a flame detection method and system based on image recognition. Background Technology

[0002] Image recognition relies on image processing and target detection algorithms to extract, identify, and locate target features by capturing video footage from surveillance cameras. Compared to traditional contact sensors such as smoke and heat detectors, this technology offers a wider monitoring range, faster response speed, and the ability to retain visual monitoring footage for post-event traceability, making it highly valuable in industrial safety production and security early warning systems. Flame detection is a key means of industrial fire early warning, therefore image recognition has become the mainstream technology for flame monitoring.

[0003] Currently, industrial flame detection based on image recognition falls into two technical categories: traditional manual feature detection and deep learning-based target detection. While both approaches can achieve automated flame recognition, they suffer from fundamental technical flaws and cannot adapt to complex industrial monitoring scenarios. Existing algorithms train all samples using a standardized method, ignoring the varying detection difficulty across different monitoring feeds. Factories commonly experience interference conditions such as low light at night, dust, fog, and red reflective equipment. These harsh environments severely obscure and weaken flame features, significantly increasing the difficulty of recognition. Existing models do not explicitly utilize the unique optical, textural, and gradient physical properties of flames during training and lack specific criteria for flame discrimination. Consequently, they exhibit weak differentiation capabilities when faced with flame-like interference objects such as red lights, red metals, and reflective building materials.

[0004] The combination of these two defects directly leads to a significant decrease in the model's recognition performance: on the one hand, flame targets are easily obscured in complex scenarios, causing missed detections; on the other hand, similar red interference objects are easily misjudged as flames, resulting in a large number of false detections. These two problems seriously reduce the accuracy of industrial fire early warning and make it difficult to meet the high-standard safety production monitoring needs of industrial scenarios. Summary of the Invention

[0005] To address the issues of flames being easily obscured in complex scenarios, leading to missed detections, and the misjudgment of red flame-like interfering objects causing false detections, which significantly reduces the accuracy of industrial fire early warning and fails to meet the high standards of industrial safety production monitoring requirements, this invention provides solutions in the following aspects.

[0006] In the first aspect, a flame detection method based on image recognition includes: acquiring an industrial site monitoring video stream, extracting frames at a preset frame rate to obtain single-frame images; labeling the flame target in the single-frame image and generating a binary mask image, normalizing the labeled coordinates to construct a label matrix, and constructing a historical sample set by combining the single-frame image, the binary mask image, and the label matrix; dividing the flame region and background region according to the binary mask, converting the single-frame image into a color image and a grayscale image, and extracting multi-dimensional physical features of the flame region and background region respectively; comparing the differences between the features of each dimension to obtain the single-feature difference degree, and weighting and summing the single-feature difference degree based on preset weighting coefficients for each dimension to obtain the flame saliency used to characterize the degree of distinction between the flame and the background; and determining the flame visibility based on the flame visibility. The system uses optical spectrum rules to filter out flame-interference pixels in the background region and calculates the background interference complexity by counting the number of connected components of the interference pixels. It then extracts the annular neighborhood of the flame boundary and calculates the boundary confusion by counting the proportion of interference pixels within this neighborhood. The system adaptively calculates the training weights for the target samples by fusing flame saliency, background interference complexity, and boundary confusion. A weighted loss function is constructed based on the weights coupling classification loss and boundary regression loss, and the CNN flame detection model is iteratively trained. Real-time acquisition of single-frame images from industrial monitoring is performed, normalized, and input into the CNN flame detection model. The model outputs the predicted coordinates of each candidate region and the corresponding flame confidence score. Invalid candidate boxes are filtered based on a pre-set confidence threshold, and the predicted coordinates are de-normalized to output standardized flame localization detection results.

[0007] Preferably, the multi-dimensional physical features include color features, texture features, and edge gradient features; The color features include chromaticity mean and saturation variation coefficient: the chromaticity mean is obtained by processing HSV chromaticity channel data using the vector averaging method; the saturation variation coefficient is calculated by statistically analyzing the mean and standard deviation of saturation pixels. The texture feature is texture entropy: a pixel distribution histogram is generated using a preset neighborhood LBP operator, and the texture entropy is obtained by combining the Shannon entropy formula. The edge gradient feature is the region average gradient magnitude: the Sobel operator is used to solve the pixel gradient, and the average of all gradient magnitudes in the region is taken to obtain the region average gradient magnitude.

[0008] Preferably, the step of intercepting the annular neighborhood of the flame boundary includes: Using the rectangular bounding boxes of each flame target as a reference, an outer bounding rectangle is obtained by expanding it outward by a preset pixel width. The difference between the outer bounding rectangle and the original flame bounding box is used to generate the annular region of the flame boundary, thus obtaining the pixel set of the annular neighborhood of the flame boundary. If the flame bounding box extends to the edge of the image, only the effective annular neighborhood pixels within the image range are retained for statistical purposes.

[0009] Preferably, the step of filtering flame interference pixels in the background area according to the visible light spectrum rules of flames includes: traversing all pixels in the background area, filtering the RGB three-channel values ​​of a single frame image, and marking pixels that satisfy the condition that the red channel value is greater than the green channel value and the green channel value is greater than the blue channel value as flame interference pixels.

[0010] Preferably, the step of obtaining the boundary confusion degree includes: The number of flame-like interference pixels within the annular neighborhood of the flame boundary and the total number of all background pixels within the annular neighborhood are counted. The ratio of the number of flame-like interference pixels to the total number of background pixels is used to obtain the interference pixel ratio, which is then used as the boundary confusion degree.

[0011] Preferably, the target sample training weights are calculated as follows: The sample training weights are obtained by averaging the results of background interference complexity, boundary confusion, and negative exponent of flame saliency.

[0012] Preferably, the step of constructing the weighted loss function includes: Binary cross-entropy is used as the classification loss and CIoU as the boundary regression loss. Invalid sample rows with a value of -1 are filtered out in the label matrix so that invalid sample rows do not participate in the loss calculation. The training weights corresponding to each sample are retrieved, and the sum of the classification loss and boundary regression loss of a single sample is weighted. The average of the weighted loss of all valid samples in the batch is used as the weighted loss function.

[0013] Preferably, the CNN flame detection model includes a feature extraction backbone network, a multi-scale feature fusion module, and a detection head module connected in sequence; the feature extraction backbone network is used to extract multi-level basic features of the input image; the multi-scale feature fusion module is used to fuse multi-resolution feature maps to adapt to flame targets of different sizes; and the detection head module is used to output normalized detection box coordinates and corresponding flame confidence scores.

[0014] Secondly, a flame detection system based on image recognition includes a processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, the above-mentioned flame detection method based on image recognition is implemented.

[0015] The present invention has the following effects: 1. This invention measures the degree of distinction between flame and background features by measuring the saliency of flame, the complexity of background interference by measuring global flame interference, and the confusion of boundaries by measuring the mixed interference of flame edges. It integrates multiple dimensions to adaptively generate sample training weights, and assigns higher training weights to difficult samples with severe occlusion and low feature recognition. It iteratively optimizes the CNN model with a weighted loss function that couples classification and boundary regression losses, which enhances the model's ability to learn weak and occluded flame features, effectively solves the problem of missed detection caused by flame occlusion under complex working conditions, and improves the detection rate of fire targets.

[0016] 2. This invention relies on the inherent visible light spectrum rules of flames to screen flame-like interference pixels in the background area, combines the neighboring connected domain to count the number of global interference sources, and constructs a ring-shaped neighborhood of the flame boundary to count the proportion of edge interference pixels. It accurately quantifies the interference intensity of red walls, lights and other interference objects from two dimensions: global background and flame edge. During the model training stage, it fully learns the multi-dimensional feature differences (chroma, texture, edge gradient) between interference and real flames. During the inference stage, it filters low-confidence candidate boxes through confidence threshold. The dual mechanism greatly reduces the misjudgment of red interference objects and reduces the frequency of false alarms in industrial monitoring. Attached Figure Description

[0017] Figure 1 This is a flowchart of steps S1-S4 in an image recognition-based flame detection method according to an embodiment of the present invention.

[0018] Figure 2 This is a structural block diagram of a flame detection system based on image recognition according to an embodiment of the present invention. Detailed Implementation

[0019] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments.

[0020] Reference Figure 1 A flame detection method based on image recognition includes steps S1-S4, as detailed below: S1: Acquire the industrial site monitoring video stream, extract frames at a preset frame rate to obtain single-frame images; label the flame targets in the single-frame images and generate a binary mask image; normalize the coordinates of the labels to construct a label matrix; combine the single-frame images, binary mask images and label matrix to form a historical sample set.

[0021] Acquire historical video streams of industrial site monitoring that include real flame targets and cover a variety of complex industrial conditions. To ensure that the trained model has excellent scene generalization ability, the acquired video materials cover a variety of lighting environments and on-site interference environments; the lighting environments include strong daylight, backlight at dusk, and low light conditions at night, and the interference environments include common industrial interference conditions such as dust, fog, and rain.

[0022] For the aforementioned continuous video sequence containing real flame targets, frame extraction is performed using a preset fixed frame rate to obtain multiple single-frame images with RGB (Red, Green, Blue) three-channels containing the flame targets. For example, the sampling frame rate can be set to 1 frame / second and can be adaptively adjusted according to the rate of change of the flames on site. By reasonably setting the frame extraction frame rate, sample redundancy caused by highly similar adjacent frames can be effectively avoided, reducing training computational consumption while ensuring sample diversity. The resolution of all extracted single-frame images is consistent with the original output resolution of the monitoring equipment. Each single-frame image consists of three independent color channels (R, G, B), providing a standardized image data source for subsequent flame annotation and feature extraction.

[0023] Manual annotation is performed on each RGB single-frame image. Each independent and connected real flame region in the image is manually identified, and a bounding rectangle is drawn for each flame. The coordinates of the bounding rectangle are recorded, which are the original pixel coordinates of the upper left corner and the lower right corner.

[0024] A binary mask image with the exact same size as the original image is generated simultaneously based on the actual outline of the flame. Within the mask image, pixels in the area actually covered by the flame are uniformly marked as 1, while pixels in the remaining background area are uniformly marked as 0. This mask allows for precise differentiation between flame pixels and background pixels. When extracting multidimensional physical features from both flame and background regions, the statistical range can be accurately locked, ensuring the accuracy of feature calculation areas.

[0025] Because the coordinates of the bounding boxes in a single frame image can vary in numerical scale with changes in monitoring resolution, which is detrimental to stable model learning, Min-Max normalization is performed on the original pixel coordinates of all flame bounding boxes. The horizontal coordinate is converted based on the overall width of the image, and the vertical coordinate is converted based on the overall height of the image. After processing, all coordinate values ​​are uniformly mapped to the interval between 0 and 1 to obtain normalized annotation coordinates.

[0026] The number of flames contained in different single-frame images varies, and directly using these figures would result in inconsistent data dimensions. Therefore, we first count the maximum number of flame bounding boxes in all sample images, and use this maximum number as the fixed number of rows in the label matrix. All samples share the same size label matrix. This ensures that each frame image corresponds to a standardized label matrix with the maximum number of rows and 4 columns. The specific construction rules are as follows: The normalized coordinates of all flame bounding boxes in the current frame are sorted in ascending order of the x-coordinate of the top-left corner. Each flame bounding box corresponds to a row in the matrix, storing the normalized x and y coordinates of the top-left and bottom-right corners in sequence. If the actual number of flames in a single frame is less than the maximum, all blank rows in the matrix are filled with the value -1. Choosing -1 as an invalid fill indicator clearly distinguishes it from the image origin coordinate 0, preventing numerical confusion. During subsequent model training, the program can identify and mask matrix rows filled with -1, excluding these rows from loss calculations to avoid invalid fill data interfering with model parameter iteration updates.

[0027] A complete set of training samples is defined as consisting of three types of associated data: the original RGB single-frame image, the label matrix corresponding to the single-frame image, and the binary mask image corresponding to the single-frame image. After all images undergo frame extraction, flame annotation, mask generation, coordinate normalization, and label matrix standardization and filling operations in sequence, all the associated data are aggregated to obtain a complete, standardized historical sample set that can be directly used for model training.

[0028] After completing the standardized construction of the historical sample set, the flame pixels and background pixels are accurately segmented based on the binary mask image accompanying the sample. This segmentation is then used as the basis for multi-dimensional feature extraction and difference quantization calculation. The specific steps are as follows: S2: Divide the flame region and background region according to the binary mask, convert the single frame image into a color image and a grayscale image, and extract the multi-dimensional physical features of the flame region and background region respectively; compare the differences between the features of each dimension to obtain the single feature difference degree, and sum the single feature difference degree based on the preset weighting coefficients of each dimension to obtain the flame saliency degree used to characterize the degree of distinction between the flame and the background.

[0029] For any target sample in the historical sample set (i.e., the original RGB single-frame image, binary mask image, and label matrix), the binary mask image of the target sample is retrieved. Based on the pixel label values ​​of the mask image, binary region segmentation of the image pixels is completed to divide the real flame region and the background region for subsequent region extraction and comparison feature extraction. If there are multiple independent flame targets that are not connected to each other in a single sample image, the flame region is further split into smaller parts. The logic for each region segmentation is as follows: The actual flame region corresponds to the set of pixels in the binary mask image whose pixel marker value is equal to 1, and its spatial range completely overlaps with the coverage area of ​​the actual burning flame in the original image. When there are two or more independent flame targets that are separated from each other and have no connected pixels in the image, the overall flame region is divided into multiple rectangular sub-regions of equal number according to the independent flame targets. Each rectangular sub-region uniquely corresponds to an independent flame and stores a pixel sub-matrix composed of all flame pixels under the corresponding flame target. The length and width of each pixel sub-matrix adaptively change with the actual size of the outer rectangle of the corresponding flame, and there is no uniform fixed size. After splitting, the visual features of each independent flame can be calculated separately, avoiding interference between the feature statistics results of multiple flame targets.

[0030] The background region corresponds to the set of pixels in the binary mask image whose pixel marker value is equal to 0. Its spatial range is the remaining area of ​​the original image after removing all flame regions. This region serves as a reference point for numerical comparison with the flame region on the same feature dimension, calculating the difference in features across each dimension.

[0031] In this embodiment, multi-dimensional physical features are extracted from the flame region and the background region respectively. The multi-dimensional physical features are divided into three categories: color features, texture features, and edge gradient features. The specific steps are as follows: Color features include two indicators: chromaticity mean and saturation coefficient of variation. First, the RGB single-frame image is converted to the HSV color space, and the chromaticity channel data is separated. Then, the vector averaging method is used to calculate the chromaticity channel values ​​of all pixels in the target area divided by the mask, and the chromaticity mean of the target area is obtained. Based on the saturation values ​​of all pixels in the target area, the dispersion is calculated to obtain the saturation coefficient of variation, which is used to characterize the distribution and fluctuation of color saturation in the area.

[0032] Texture features are calculated using the LBP (Local Binary Pattern) operator to obtain texture entropy. All pixels within a region are traversed in units of a preset size neighborhood; for example, a 3×3 pixel neighborhood can be selected as the calculation window. The LBP operator is used to characterize the local texture variation patterns of pixels, generating corresponding texture codes. Then, based on the texture codes corresponding to all pixels within the region, the entropy values ​​are statistically analyzed to quantify the texture disorder of the flame and background regions.

[0033] Edge gradient features are the average gradient magnitude of the region. The Sobel operator is used to solve the horizontal and vertical gradient components of the image separately, and the gradient magnitude of a single pixel is obtained by fusion. The average gradient magnitude of all pixels within the masked region is calculated as the edge gradient feature of that region, which reflects the richness of the contours and edges within the region.

[0034] The three types of features described above depict the visual attributes of the flame area and the background area from three dimensions: color distribution, texture details, and edge contours. Subsequently, by comparing the feature values ​​of the flame area and the background area under the same dimension, the single-dimensional feature difference can be obtained.

[0035] Retrieve the first and second corresponding regions of the real flame area and the background area from the same target sample. For each dimension of feature values, the absolute difference between two sets of feature values ​​is calculated to obtain the single-feature dissimilarity degree corresponding to that dimension. The numerical range constraint for single feature difference is [0,1].

[0036] Single feature difference is used to quantify the ability to distinguish features between flame and background in a single physical dimension: the larger the single feature difference value, the more significant the feature difference between the flame area and the background area in that dimension, and the stronger the visual recognizability of the flame target in that dimension; conversely, it indicates that the flame and background features are highly similar in that dimension, and it is more difficult to distinguish them.

[0037] Based on the contribution of each dimension of physical features to the differentiation and recognition of flame targets, fixed weighting coefficients are assigned to the four feature dimensions: chromaticity mean, saturation variation coefficient, texture entropy, and region average gradient magnitude. The weighted coefficients of all dimensions satisfy the normalization constraint, that is, the sum of all weighted coefficients is 1, which ensures that the numerical scale of flame significance is uniform and the evaluation standard is stable.

[0038] The flame saliency of the target sample is obtained by multiplying the single feature difference scores of each dimension with the matching weighting coefficients one by one and summing the results. This weighted fusion method integrates the multi-dimensional feature difference information. The specific calculation formula is as follows: ; In the formula, Indicates the flame significance corresponding to the target sample; Indicates the target sample number Single-feature variance of each dimension; Indicates the target sample number Preset weighting coefficients corresponding to each dimension of features.

[0039] To adapt to the actual characteristics of industrial flame detection, this invention provides exemplary configurations for the weighting coefficients of each dimension. The values ​​of each coefficient are set according to the physical characteristics of the flame and the priority of scene recognition. The specific configuration rules are as follows: The color characteristics of the flame are the core basis for distinguishing the flame from the conventional background and interference objects. Among them, the chromaticity mean can characterize the flame's unique color tendency difference, and the saturation variation coefficient can reflect the color fluctuation characteristics of the flame's dynamic combustion. Both have the highest recognition priority, and their weighting coefficients are set to 0.3 for example. Texture entropy can accurately distinguish dynamic burning flames from static red interference objects, and can compensate for the recognition defects in scenes with similar color characteristics. As a core supplementary feature, its weighting coefficient is set to 0.25 for example. The regional average gradient amplitude is mainly used to characterize the flame edge contour features and assist in optimizing the flame boundary positioning effect. Its contribution to the overall distinction between the flame and the background is relatively weak, and its weighting coefficient is set to 0.15 for example. The sum of the above exemplary weighting coefficients is 1, and they can be adaptively fine-tuned according to the interference type and lighting environment of different industrial sites, and are not limited to the values ​​in this embodiment.

[0040] This invention comprehensively characterizes the visual differences between flames and the background through three complementary physical feature dimensions: color, texture, and edge gradient. Single-feature difference achieves precise quantification of the distinguishing ability of a single dimension, while weighted coefficients differentiate the contribution of different feature dimensions. The resulting flame saliency comprehensively and objectively reflects the overall difficulty of flame target recognition in the current sample scene: a higher flame saliency value indicates a more significant difference in the multi-dimensional comprehensive features between the flame and the background, and a lower difficulty in flame recognition; a lower flame saliency value indicates a higher similarity between the flame and the background features, stronger scene interference, and a higher difficulty in flame recognition.

[0041] However, in actual industrial monitoring scenarios, flame-like interference factors such as red walls, lights, and reflective fog are common. Relying solely on flame salience cannot comprehensively characterize the degree of scene interference and the difficulty of flame recognition, making it difficult to accurately distinguish between easy and difficult samples, resulting in insufficient model adaptability to complex interference scenarios. To solve this problem, this invention further refines the quantification of scene interference features from two dimensions: global background interference and local flame boundary interference. It combines flame salience to comprehensively evaluate the sample learning difficulty, adaptively generates sample training weights, and constructs a weighted loss function to complete the iterative training of the model. The specific implementation steps are as follows: S3: Based on the visible light spectrum rules of flames, filter flame interference pixels in the background region, count the number of connected components of interference pixels to calculate the background interference complexity; extract the annular neighborhood of the flame boundary, count the proportion of interference pixels in the annular neighborhood of the flame boundary to obtain the boundary confusion degree; integrate flame saliency, background interference complexity, and boundary confusion degree to adaptively solve the target sample training weights, construct a weighted loss function based on the weight coupling classification loss and boundary regression loss, and iteratively train to obtain the CNN flame detection model.

[0042] Background interference complexity is used to quantify the distribution quantity and overall interference intensity of flame-like interference sources within the global background region of an entire image. It can characterize the risk of false detection interference from the global environment in an industrial scene and is one of the core indicators for evaluating the overall detection difficulty of a sample. This invention, based on the inherent spectral laws of visible light from flames, obtains background interference complexity through a three-step process: interference pixel screening, connected component analysis, and complexity mapping calculation. The specific implementation method is as follows: Flames possess stable and unique spectral response characteristics in the visible light band. Specifically, the red channel response intensity of a burning flame is the highest, followed by the green channel, and the blue channel response intensity is the weakest. Based on this inherent physical characteristic, a pixel discrimination rule based on the magnitude of the RGB three-channel amplitude is established to achieve rapid and accurate screening of flame-like interference pixels in the background region. Specifically, the entire set of background pixels obtained by binary masking in the target sample is traversed, and the RGB three-channel values ​​of each pixel are compared one by one. If the current background pixel satisfies the condition that the red channel value is greater than the green channel value, and the green channel value is greater than the blue channel value (R>G>B), then the pixel is determined to have flame-like spectral characteristics and is marked as a flame-like interference pixel.

[0043] After filtering out interfering pixels in the background region, connected component analysis is performed on the interfering pixels to distinguish independent interference sources and avoid the impact of scattered, isolated pixels on the accuracy of background interference complexity statistics. If no flame-like interfering pixels are found in the background region of the target sample, it is determined that the sample has no global background interference, and the background interference complexity is directly set to 0. If flame-like interfering pixels exist in the background region, a two-pass scanning method is used to perform 8-neighbor connected component detection on all interfering pixels, segmenting them into several independent, non-pixel-connected flame-like interfering connected components. Each independent connected component corresponds to an independent background interference source.

[0044] The total number of all independent flame-like connected components within the background region of the current target sample is counted, and this number is defined as the number of interference sources. In industrial inspection scenarios, each independent flame-like connected component within the background region is a potential interference source that can be misidentified as a flame by the model. Regardless of its size or number of pixels, each independent flame-like connected component will bring a corresponding risk of false detection and reduce the accuracy of flame detection.

[0045] To address the issue of the number of interference sources taking values ​​within a non-negative integer range with no numerical boundary, a sigmoid function is introduced to normalize the number of interference sources, transforming the unbounded quantity parameter into a stable and controllable complexity range, thus obtaining the background interference complexity. The specific calculation formula is as follows: ; In the formula, This represents the background interference complexity corresponding to the target sample; This represents the total number of flame-like independent connected components within the background region of the target sample, i.e., the number of interference sources. This represents a preset adjustment coefficient used to control the weight of the influence of the number of interference sources on the complexity of background interference, adapting to the interference density characteristics of different scenarios. In this embodiment, the value is 0.2, but in actual applications, it can be adaptively adjusted according to the interference situation in the industrial field.

[0046] The more independent flame interference sources there are in the background region, the higher the probability of the model generating false detection candidate boxes, and the stronger the global background interference. By adjusting the weighted coefficients and mapping with the Sigmoid function, the number of unbounded interference sources can be transformed into a smoothly changing complexity index, allowing the background interference complexity to stably quantify the global detection difficulty of the sample. The larger the background interference complexity value, the more severe the global background interference of the current target sample, and the higher the overall difficulty of flame detection.

[0047] To accurately quantify the degree of interference between the flame edge and the surrounding background, this invention constructs a flame boundary annular neighborhood, specifically for statistically analyzing the distribution of flame-like interference pixels within the region immediately adjacent to the flame edge. The specific construction method is as follows: using the bounding box of each independent flame target in the image as the baseline boundary, a preset pixel width is uniformly expanded outward from the bounding box to generate a larger outward-expanding rectangular region; the region difference operation is performed between the outward-expanding rectangular region and the original flame bounding box, and the area covered by the original flame rectangle is removed, resulting in a flame boundary annular region surrounding the flame target. Pixels belonging to the flame target are further removed from the flame boundary annular region, ultimately retaining only the background region immediately adjacent to the flame edge, thus forming the flame boundary annular neighborhood.

[0048] At the same time, an image boundary fault tolerance mechanism is set: if the original flame annotation rectangle extends to the edge of the image, causing the outward rectangle to exceed the effective imaging range of the image, the invalid area that exceeds the image boundary is discarded, and only the valid ring-shaped neighboring pixels within the image range are retained to participate in subsequent statistical calculations, ensuring that the neighboring pixel statistics are valid and there is no invalid data that exceeds the boundary.

[0049] Based on the construction of the annular neighborhood of the flame boundary corresponding to a single flame target, the pixel sets of the annular neighborhood corresponding to the boundary of all independent flame rectangular regions within the target sample image are summarized and integrated to obtain the pixel set of the flame boundary neighborhood corresponding to the target sample of the current whole frame. This pixel set of the flame boundary neighborhood consists entirely of effective background pixels adjacent to the flame edge, which can completely characterize the local environmental features of the edges of all flame targets in the entire image.

[0050] Furthermore, within the total set of pixels in the neighborhood of the flame boundary, the number of flame-like interference pixels that have been screened is counted, and the proportion of flame-like interference pixels in the total number of pixels in the neighborhood is calculated. This proportion is defined as the boundary confusion degree, and its value is naturally constrained within the range of... Within the range.

[0051] Boundary confusion is used to quantify the degree of interference and clutter in the local area of ​​the flame target edge, specifically characterizing the fit and mixing of the flame outline boundary with surrounding flame-like interference. A higher boundary confusion value indicates that the proportion of flame-like interference pixels in the area immediately adjacent to the flame edge is larger, and the boundary features between the flame outline and the background interference are more blurred. The model is more prone to boundary missegmentation, missed detection, or false detection at the flame edge, making flame boundary recognition more difficult. Conversely, a lower boundary confusion value indicates a cleaner background at the flame edge, clearer boundary distinction, and lower recognition difficulty.

[0052] This invention integrates three core evaluation indicators—global background interference, local boundary confusion, and flame feature discrimination ability—and adaptively solves the exclusive training weights for each target sample to achieve weighted reinforcement learning for highly difficult interference samples.

[0053] The specific calculation logic is as follows: perform a negative exponential operation on the flame saliency to weaken the sample weights of high flame saliency and increase the sample weights of low flame saliency; sum the calculated flame saliency result, background interference complexity, and boundary confusion, and then take the arithmetic mean of the summation result to finally obtain the sample training weights of the target sample.

[0054] Specifically, the training weights of the target samples satisfy the following relationship: ; In the formula, Indicates the training weights of the target sample; This represents the background interference complexity of the target sample, with a value range of... ; The boundary confusion of the target sample is represented, and its value range is [range missing]. ; This represents the significance of the flame in the target sample, with a value range of [value missing]. .

[0055] Among them, the higher the background interference complexity, the more global flame interference sources there are in the image, the greater the risk of false detection, the higher the sample recognition difficulty, and the higher the corresponding sample training weight; the higher the boundary confusion, the more serious the mixing of flame edges and background interference, the worse the boundary recognition, the higher the sample detection difficulty, and the higher the corresponding sample training weight; the lower the flame saliency, the weaker the difference in multidimensional features between the flame and the background, the lower the feature discrimination, the greater the sample recognition difficulty, and the negative exponential transformation can effectively increase the weight ratio of such low discrimination samples.

[0056] This weight design method enables adaptive matching that is positively correlated with sample difficulty and training weight: it automatically assigns higher training weights to complex samples with strong interference, low discriminative power, and difficulty in identification, prompting the model to prioritize learning flame features under complex working conditions; at the same time, it assigns lower weights to simple samples with clear features and weak interference, avoiding simple samples dominating model training and effectively improving the model's generalization detection capability for complex industrial interference scenarios.

[0057] To adapt the aforementioned adaptively obtained sample training weights and achieve differentiated training of the model for samples of varying difficulty, balancing the accuracy of flame target classification and boundary localization, this invention integrates classification loss and boundary regression loss, and constructs an adaptive weighted loss function by combining sample-specific training weights. Simultaneously, a masking mechanism is implemented for invalid padding data in the label matrix to prevent invalid data from interfering with the model's iteration accuracy. The specific construction method is as follows.

[0058] Based on the label matrix construction rules, invalid data rows with a padding value of -1 exist in the sample label matrix. These rows lack real flame target coordinate information and have no training guidance value. Therefore, before loss calculation, invalid sample rows with a padding value of -1 are uniformly removed from the label matrix, and only valid samples containing real flame annotation information are retained for loss calculation. This completely avoids the loss calculation bias and model training interference caused by invalid padding data, ensuring the effectiveness and accuracy of the training process.

[0059] The core design logic of the weighted loss function is as follows: relying on adaptive sample weights, it automatically amplifies the loss proportion of high-interference, low-discrimination, and difficult samples, allowing the model to focus on learning the flame characteristics and boundary distribution patterns under complex working conditions; at the same time, it weakens the loss weight of simple samples to avoid simple samples excessively dominating model training. Through the coupled weighted optimization of classification loss and regression loss, the flame recognition accuracy and boundary localization accuracy of the model can be improved simultaneously, significantly enhancing the robustness and generalization ability of the model in complex industrial interference scenarios.

[0060] The weighted loss function constructed in this invention is composed of a classification loss term and a boundary regression loss term. The classification loss adopts the binary cross-entropy (BCE) loss, which corresponds to the flame binary classification task. The ground truth value of the flame mask pixel is 1 to represent the flame region and 0 to represent the background region. It is used to quantify the deviation between the model's pixel-level flame classification prediction probability and the real mask label. This loss relies on the Sigmoid output probability and has the global convergence property of a convex function, which will impose a stronger penalty on misclassifications with high confidence. The boundary regression loss adopts the CIoU loss, which specifically measures the difference between the detected predicted rectangle and the real flame label box. This loss comprehensively considers the intersection-union ratio of the two boxes, the Euclidean distance between the center points, and the consistency of the aspect ratio. Even when the predicted box and the real box do not overlap and the size difference is large, it can still output an effective gradient to optimize the localization accuracy. Combined with the sample training weights obtained by the adaptive solution above, the two types of losses are weighted and fused, and the CNN flame detection model is trained iteratively as a whole.

[0061] Specifically, the weighted loss function satisfies the following relationship: ; In the formula, This represents the weighted loss function of the CNN flame detection model; Indicates the total number of samples in the historical sample set; Indicates the first in the historical sample set Training weights for each historical sample; Indicates the first in the historical sample set The weighted classification loss of each historical sample; This represents the boundary regression loss of the j-th historical sample in the historical sample set.

[0062] The CNN flame detection model used in this invention consists of three functional modules connected in series. The complete chain is: feature extraction backbone network → multi-scale feature fusion module → detection head module. Each module has a clear division of labor and completes the entire process of image feature extraction, feature optimization, and detection result output step by step. The functions of each module are described as follows: Feature extraction backbone network: Receives input RGB single-frame image, performs multi-level basic visual feature extraction layer by layer, including shallow texture, mid-level contour, and high-level semantics, and outputs multiple sets of original feature maps at different resolutions to provide basic feature data for subsequent multi-scale fusion; Multi-scale feature fusion module: Receives multi-resolution feature maps output by the backbone network, performs splicing, upsampling, and downsampling fusion processing on features of different scales, unifies feature scale and complements strong and weak feature information, can adapt to flame targets of varying sizes in industrial sites, and takes into account the feature expression capabilities of small flames at a distance and large flames at a close distance. Detection head module: Receives the fused enhanced feature map, performs convolutional decoding operation to output the normalized detection box coordinate values ​​and the confidence level of the presence of flame targets in the corresponding area, and completes the flame target classification and boundary localization results output.

[0063] The CNN flame detection model in this embodiment is based on the YOLOv5 detection model. The feature extraction backbone network first slices and downsamples the received image through the shallowest layer to expand the channels, and then uses an activation layer to activate the shallow features of the image to capture the jagged edges of the flames, the brightness abrupt changes between the flames and the background, the fine texture of the flames, and the wrinkles in the smoke. Then, the downsampling module continues to extract the local edges and small bright areas of the image to assist in the enhancement of small fire features. Finally, the image is downsampled at a downsampling rate of 8 times to obtain the mid-level contour features of the image, at a downsampling rate of 16 times to obtain the mid-to-high-level contour features of the image, and at a downsampling rate of 32 times to obtain the high-level semantic features of the image. The aforementioned mid-level contour features have medium resolution and are used to perceive the overall outer contour of the flame and the shape of the combustion connected area. They are responsible for detecting flames of normal size in the image and are the main layer for small target detection. The receptive field of the mid-to-high-level contour features is further expanded, and details are weakened and regional semantics are strengthened. They are adapted to medium to large cluster flames. The high-level semantic features can further deepen and expand the receptive field, and identify large-area continuous flames at close range and large-scale open flames in the furnace.

[0064] The multi-scale feature fusion module performs FPN top-down fusion and PAN bottom-up fusion on mid-level contour features, mid-to-high-level contour features, and high-level semantic features. FPN top-down fusion injects strong semantic features of large-area flames into the shallow and mid-level layers, reducing the probability of misclassifying lights, reflections, and red workpieces as flames and improving classification reliability. PAN bottom-up fusion adds flame edges and subtle high-brightness textures preserved in the mid-level contour features to the large-scale feature layer, compensating for small target information lost in deep downsampling and significantly improving the recall rate of weak flames at long distances. Finally, the fused mid-level contour features, mid-to-high-level contour features, and high-level semantic features are followed by a 3×3 convolution to eliminate grid ghosting caused by upsampling interpolation, ultimately outputting a three-layer enhanced fused feature map which is then fed into the detection head module.

[0065] The detection head module identifies distant, small flames based on mid-level contour features, neutral-sized flames based on mid-to-high-level contour features, and large-area flames at close range based on high-level semantic features. Then, it outputs the original prediction parameters through coupled convolution, activates the probability mapping through the Sigmoid activation function, and restores the pixel coordinates of the flames through anchor box decoding.

[0066] The entire network architecture achieves accurate multi-scale flame detection through layered division of labor. Combined with the aforementioned adaptive weighted loss function, it completes targeted iterative training, which can significantly improve the accuracy of flame detection and boundary positioning in complex lighting and multi-interference industrial scenarios.

[0067] S4: Real-time acquisition of single-frame images from industrial monitoring, normalization processing, input into the CNN flame detection model, outputting the predicted coordinates of each candidate region and the corresponding flame confidence score, filtering invalid candidate boxes according to the preset confidence threshold, inverse normalizing the predicted coordinates, and outputting standardized flame localization detection results.

[0068] The device connects to the real-time video stream pushed by the industrial monitoring platform and uses the same frame extraction rate as in step S1 to capture RGB color single-frame images. The image resolution maintains the original output specifications of the monitoring camera to ensure that the size standard of the online inference image is consistent with that of the training sample image.

[0069] For the captured real-time images, the same Min-Max normalization rule used in the model training phase is applied to perform coordinate standardization preprocessing to eliminate numerical scale differences caused by different resolutions and ensure that the distribution of model input data is consistent with that of training data.

[0070] The preprocessed image is input into the trained CNN flame detection model, and the model outputs a prediction label matrix with a maximum of 4 rows and 4 columns.

[0071] Each row in the predicted label matrix corresponds to a candidate flame detection box. The four columns represent the normalized x and y coordinates of the top-left and bottom-right corners of the detection box, respectively. The model simultaneously outputs the flame confidence score for each candidate box, with the numerical range limited to [value missing]. The confidence score represents the probability that the model determines the area within the box to be a real flame. The closer the value is to 1, the higher the confidence that a flame exists in the area; the closer the value is to 0, the more likely the area is a flame-like object such as a light or a red wall.

[0072] The candidate boxes directly output by the model contain a large number of low-confidence false detection boxes. Post-processing is required to filter them and restore the original pixel coordinates in order to obtain usable final detection results. The specific steps are as follows: A confidence threshold is preset, and in this embodiment, it is 0.5. The confidence values ​​of all candidate boxes are compared one by one, and invalid candidate boxes with scores lower than the threshold are removed. Redundant interference boxes generated by the model are also removed, and only valid flame detection boxes with qualified confidence are retained.

[0073] For the valid detection boxes that are filtered and retained, Min-Max inverse normalization calculation is performed on the normalized coordinates within the label matrix, and the standardized coordinates in the 0~1 interval are converted inversely to pixel coordinates under the original resolution of the monitoring image.

[0074] After completing confidence filtering and coordinate restoration, the system outputs complete flame detection data. The output includes the original image pixel coordinates corresponding to all valid flame frames, as well as the flame confidence score matched for each detection frame. This data can be directly used for upper-level business logic such as monitoring screen annotation and flame anomaly alarm.

[0075] The confidence threshold filtering mechanism can filter out interference candidate boxes that are fuzzy in the model recognition, reducing meaningless false alarms in industrial sites; coordinate inverse normalization restores the model's internal calculations to the real pixel coordinates of the screen using standardized coordinates, enabling intuitive labeling of flame targets on the monitoring screen, and adapting to the needs of industrial visualization operation and maintenance.

[0076] The technical solution of this application was applied to an industrial furnace monitoring scenario (the main radiation band of the furnace flame is 800um-1000um), and the effect was verified for 30 days. The verification results are as follows: Recognition accuracy: A total of 10,000 fire scene tests were conducted, with recognition parameters including fire-specific low-frequency pulsations, the proportion of the fire area in the frame, morphological complexity recognition, inter-frame centroid displacement, and inter-frame area fluctuation. 9,920 correct recognitions were achieved, with an accuracy rate of 99.2%, higher than existing traditional fire recognition cameras (accuracy rate ≤95%). False alarm rate: A total of 5,000 tests were conducted in non-flame scenarios (the purpose of the non-flame scenario experiments was to reduce false alarms, identify specific interference sources such as lights, high-temperature pipes, and smoke; verify the distinguishability of features; optimize thresholds and discrimination logic; and expose some unexpected false alarm scenarios). There were 23 false alarms, with a false alarm rate of 0.46%, which is lower than existing products (false alarm rate ≥3%). False negative rate: A total of 1,000 weak flame scenarios (small, distant flames with a flame height of less than 0.3m, flame area of ​​less than 2%, flame temperature of less than 600℃; distance parameter of more than 30m) were tested, with 3 false negatives, resulting in a false negative rate of 0.3%, which meets the needs of industrial monitoring. Response speed: The average identification response time is 85ms, which can realize rapid alarm of flames and buy time for fire response.

[0077] This invention also provides a flame detection system based on image recognition. For example... Figure 2 As shown, the system includes a processor and a memory. The memory stores computer program instructions, which, when executed by the processor, implement a flame detection method based on image recognition according to the first aspect of the present invention. The system also includes other components well known to those skilled in the art, such as a communication bus and a communication interface, the settings and functions of which are known in the art and will not be described further here.

[0078] It should be noted that those skilled in the art can make various modifications and improvements without departing from the inventive concept, and these all fall within the scope of protection of this invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A flame detection method based on image recognition, characterized in that, include: Acquire industrial site monitoring video streams and extract single-frame images by frame extraction at a preset frame rate; The flame targets in a single frame image are labeled and a binary mask image is generated. The coordinates of the labels are normalized to construct a label matrix. The single frame image, the binary mask image and the label matrix are combined to form a historical sample set. The flame region and background region are divided according to the binary mask. The single frame image is converted into a color image and a grayscale image. The multi-dimensional physical features of the flame region and the background region are extracted respectively. The differences between features in each dimension are compared to obtain the single feature difference degree. The single feature difference degree is weighted and summed based on the preset weighting coefficients of each dimension to obtain the flame saliency degree used to characterize the degree of distinction between flame and background. Based on the visible light spectrum rules of flames, flame interference pixels in the background region are screened, and the background interference complexity is calculated by counting the number of connected components of the interference pixels. The ring neighborhood of the flame boundary is extracted, and the proportion of interference pixels in the ring neighborhood of the flame boundary is counted to obtain the boundary confusion degree. The flame saliency, background interference complexity, and boundary confusion degree are fused to adaptively solve the target sample training weights. A weighted loss function is constructed based on the weight coupling classification loss and boundary regression loss. The CNN flame detection model is obtained through iterative training. The system acquires single-frame images from industrial monitoring in real time, performs normalization processing, and inputs them into a CNN flame detection model. It outputs the predicted coordinates of each candidate region and the corresponding flame confidence score. Invalid candidate boxes are filtered out according to a preset confidence threshold, and the predicted coordinates are denormalized to output standardized flame localization detection results.

2. The flame detection method based on image recognition according to claim 1, characterized in that, The multi-dimensional physical features include color features, texture features, and edge gradient features; The color features include chromaticity mean and saturation variation coefficient: the chromaticity mean is obtained by processing HSV chromaticity channel data using the vector averaging method; the saturation variation coefficient is calculated by statistically analyzing the mean and standard deviation of saturation pixels. The texture feature is texture entropy: a pixel distribution histogram is generated using a preset neighborhood LBP operator, and the texture entropy is obtained by combining the Shannon entropy formula. The edge gradient feature is the region average gradient magnitude: the Sobel operator is used to solve the pixel gradient, and the average of all gradient magnitudes in the region is taken to obtain the region average gradient magnitude.

3. The flame detection method based on image recognition according to claim 1, characterized in that, The step of intercepting the annular neighborhood of the flame boundary includes: Using the rectangular bounding boxes of each flame target as a reference, an outer bounding rectangle is obtained by expanding it outward by a preset pixel width. The difference between the outer bounding rectangle and the original flame bounding box is used to generate the annular region of the flame boundary, thus obtaining the pixel set of the annular neighborhood of the flame boundary. If the flame bounding box extends to the edge of the image, only the effective annular neighborhood pixels within the image range are retained for statistical purposes.

4. The flame detection method based on image recognition according to claim 1, characterized in that, The step of filtering flame interference pixels in the background area according to the visible light spectrum rules of flames includes: traversing all pixels in the background area, filtering the RGB three-channel values ​​of a single frame image, and marking pixels that satisfy the condition that the red channel value is greater than the green channel value and the green channel value is greater than the blue channel value as flame interference pixels.

5. The flame detection method based on image recognition according to claim 1, characterized in that, The steps for obtaining the boundary confusion include: The number of flame-like interference pixels within the annular neighborhood of the flame boundary and the total number of all background pixels within the annular neighborhood are counted. The ratio of the number of flame-like interference pixels to the total number of background pixels is used to obtain the interference pixel ratio, which is then used as the boundary confusion degree.

6. The flame detection method based on image recognition according to claim 1, characterized in that, The target sample training weights are calculated as follows: The sample training weights are obtained by averaging the results of background interference complexity, boundary confusion, and negative exponent of flame saliency.

7. The flame detection method based on image recognition according to claim 1, characterized in that, The steps for constructing the weighted loss function include: Binary cross-entropy is used as the classification loss and CIoU as the boundary regression loss. Invalid sample rows with a value of -1 are filtered out in the label matrix so that invalid sample rows do not participate in the loss calculation. The training weights corresponding to each sample are retrieved, and the sum of the classification loss and boundary regression loss of a single sample is weighted. The average of the weighted loss of all valid samples in the batch is used as the weighted loss function.

8. The flame detection method based on image recognition according to claim 1, characterized in that, The CNN flame detection model includes a feature extraction backbone network, a multi-scale feature fusion module, and a detection head module connected in sequence. The feature extraction backbone network is used to extract multi-level basic features of the input image. The multi-scale feature fusion module is used to fuse multi-resolution feature maps to adapt to flame targets of different sizes. The detection head module is used to output normalized detection box coordinates and corresponding flame confidence scores.

9. A flame detection system based on image recognition, characterized in that, include: A processor and a memory, the memory storing computer program instructions that, when executed by the processor, implement the image recognition-based flame detection method according to any one of claims 1-8.

Images