A flame recognition method and system based on multi-feature fusion

Abstract

The present invention discloses a flame recognition method and system based on multi-feature fusion, which belongs to the field of image recognition. The method comprises the following steps: step S1, obtaining multi-modal data of flame, including flame image data and ambient temperature data; and performing preprocessing; step S2, extracting features from the multi-modal data of the flame; step S3, performing feature fusion on the extracted multiple features; step S4, classifying the flame state based on the feature fused data. The present invention realizes accurate recognition and real-time monitoring of the flame state through multi-feature fusion, and effectively improves the accuracy and reliability of flame recognition.

Description

Technical Field

[0001] The present invention belongs to the field of image recognition, and in particular, relates to a flame recognition method and system based on multi-feature fusion. Background Art

[0002] With the widespread application of waste incineration technology, how to effectively and accurately monitor and control the flame state during the incineration process has become an important topic in the field of waste treatment. The waste incineration process is complex, and the combustion conditions are affected by multiple factors such as waste composition, ambient temperature, and air supply, resulting in various changes in the flame state. Traditional flame monitoring mainly relies on single sensor means such as thermocouples and infrared sensors. Although these methods can detect basic parameters such as flame temperature, their accuracy and reliability are limited in complex incineration environments. The shape, color, temperature fluctuation, and spectral characteristics of the flame can all be used as important bases for judging the combustion efficiency and incineration state. Therefore, a single monitoring method is difficult to meet the needs of multiple flame characteristics in practical applications. With the development of machine vision and artificial intelligence technology, flame detection technology based on image processing and pattern recognition has gradually become a research hotspot. These technologies can realize flame recognition by extracting visual features such as flame color, shape, and dynamic texture. However, a single image processing method is easily interfered in the complex environment of high temperature, high noise, and high smoke in waste incineration, resulting in low recognition accuracy. To solve such problems, multimodal flame detection technology has emerged. By fusing multiple sensor data (such as images, temperature, spectrum, etc.), it can more comprehensively perceive and judge the flame state. These fusion technologies combine the advantages of different sensors and effectively improve recognition accuracy and robustness.

[0003] Existing multimodal fusion methods still face many challenges in the feature extraction and fusion stages, such as how to effectively deal with the heterogeneity of different modal data, and how to achieve efficient and real-time flame detection with limited computing resources under the condition of high-dimensional data.

[0004] Therefore, the present invention proposes a flame recognition method and system based on multi-feature fusion. Summary of the invention

[0005] The present invention aims to overcome the shortcomings of the prior art and proposes a flame recognition method and system based on multi-feature fusion to achieve the following objectives: through multi-feature fusion, accurate recognition and real-time monitoring of flame status can be achieved, effectively improving the accuracy and reliability of flame recognition.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is: a flame recognition method based on multi-feature fusion, the method comprising the following steps:

[0007] Step S1, obtaining multimodal data of flame, including flame image data and ambient temperature data; and performing preprocessing;

[0008] Step S2, extracting features from the multimodal data of the flame;

[0009] Step S3, performing feature fusion on the extracted multiple features;

[0010] Step S4: classify the flame state based on the feature fused data.

[0011] Preferably, the step S1 comprises:

[0012] Step S11, flame image data preprocessing, including image denoising and normalization;

[0013] Step S12, preprocessing the ambient temperature data, including using a Kalman filter to perform denoising on the ambient temperature data.

[0014] Preferably, in step S2, for feature extraction of flame image data, the flame image data must first be grayed. When perceived by human eyes, the grayed expression is as follows:

[0015] g(x,y)=0.11×R(x,y)+0.59×G(x,y)+0.30×B(x,y);

[0016] Among them, g(x,y) represents the grayscale value of the pixel at the coordinates (x,y) in the image; R(x,y) represents the brightness value of the red channel at the pixel at the coordinates (x,y); G(x,y) represents the brightness value of the green channel at the pixel at the coordinates (x,y); B(x,y) represents the brightness value of the blue channel at the pixel at the coordinates (x,y).

[0017] Preferably, in step S2, feature extraction is performed based on the grayscale image, and the extracted multiple features include flame effective area, average grayscale of flame image effective area, flame high temperature area, average grayscale of high temperature area, flame area rate, high temperature rate, flame centroid, flame area rate variance per unit time, image grayscale mean variance per unit time, and color component ratio.

[0018] Preferably, the multi-feature extraction methods of the image are:

[0019] (1) Flame effective area, which reflects the combustion efficiency. The corresponding extraction method is: set the flame effective point grayscale threshold. In the image, if the grayscale value of the coordinate point is greater than the flame effective point grayscale threshold, the corresponding coordinate point is considered to be an effective point. The area of ​​the region composed of all effective points is accumulated to be the flame effective area.

[0020] (2) The average grayscale of the flame image effective area reflects the overall temperature level of the flame. The corresponding extraction method is: calculate the grayscale values ​​of the coordinates of all effective points in the image and accumulate them. The accumulated value is divided by the flame effective area to obtain the average grayscale of the flame image effective area;

[0021] (3) The area of ​​the flame high temperature zone reflects the flame's anti-interference ability and stability. The corresponding extraction method is: set the flame high temperature point grayscale threshold. In the image, if the grayscale value of the coordinate point is greater than the flame high temperature point grayscale threshold, the corresponding coordinate point is considered to be a high temperature point. The area of ​​the region composed of all high temperature points is accumulated to be the flame high temperature zone area.

[0022] (4) Average grayscale of high temperature zone, which reflects the average temperature level of the high temperature zone. The corresponding extraction method is: calculate the grayscale values ​​of all high temperature point coordinates in the image and add them up. The accumulated value is divided by the area of ​​the flame high temperature zone to obtain the average grayscale of the high temperature zone.

[0023] (5) Flame area ratio, which reflects the ratio of flame filling the selected image area. The corresponding extraction method is:

[0024]

[0025] Among them, S v represents the flame area ratio; i×j represents the area of ​​the selected area; Z[] represents a step function, and Z takes 1 when the value in the square brackets is greater than 0, otherwise Z takes 0; g(x,y) represents the gray value of the image pixel coordinate point (x,y); g th Indicates the preset flame point grayscale threshold; g(x,y) is greater than g th When , it means the pixel coordinate point (x, y) is the flame point;

[0026] (6) High temperature rate, which reflects the proportion of the flame high temperature area to the entire image space. The corresponding extraction method is: the area of ​​the flame high temperature area divided by the area of ​​the flame effective area;

[0027] (7) Flame centroid, which reflects the spatial position information of flame combustion. The corresponding extraction method is:

[0028]

[0029] Among them, x c is the horizontal coordinate of the flame center of mass, y c is the ordinate of the flame center of mass;

[0030] (8) The variance of the flame area rate per unit time reflects the change of the flame area. The corresponding extraction method is:

[0031]

[0032] Among them, Ds represents the variance of flame area rate per unit time; S vi represents the flame area ratio of the image calculated for the i-th time; S vmean It represents the mean value of the flame area rate in the statistical time period; n represents the number of measurements;

[0033] (9) The mean variance of the grayscale of the image per unit time reflects the temperature fluctuation of the flame. The corresponding extraction method is:

[0034]

[0035] Among them, D g represents the mean variance of image grayscale per unit time; g meani Represents the grayscale mean of the image calculated for the i-th time; It represents the mean of the grayscale mean of the image in the statistical time period; n represents the number of measurements;

[0036] (10) Color component ratio reflects the flame characteristics, and the corresponding extraction method is:

[0037] K = G / R, or K = B / R;

[0038] Among them, K represents the color component ratio; R represents the brightness value of the red channel at the pixel coordinates (x, y); G represents the brightness value of the green channel at the pixel coordinates (x, y); B represents the brightness value of the blue channel at the pixel coordinates (x, y).

[0039] Preferably, the feature fusion adopts a weighted fusion method, which is expressed by the formula:

[0040]

[0041] Among them, F fusion Indicates the fused features; F i represents the i-th feature involved in the fusion; w i represents the i-th feature F i The fusion weight of ; N represents the number of features involved in the fusion.

[0042] Preferably, in step S4, flame state classification is performed by a multi-scale convolutional neural network MS-CNN, wherein MS-CNN uses convolution kernels of different scales to extract multi-level features from the fused features again, and the extracted multi-scale features are integrated through parallel convolution layers, and the integrated multi-scale features are fused and input into a classifier for classification, wherein the classification errors of different flame states are balanced by a multi-scale loss function, and the multi-scale loss function is expressed as follows:

[0043]

[0044] Among them, L MS-CNN Represents the value of the multi-scale loss function; L s is the classification loss of the s-th classification layer; s is the corresponding weight.

[0045] Preferably, the weights in the multi-scale loss function are adjusted by an adaptive algorithm.

[0046] Preferably, the adaptive algorithm comprises a minimum mean square error, namely:

[0047]

[0048] Where MSE stands for mean square error; L MS-CNN Represents the multi-scale loss function value of each sample, and n represents the number of samples; after obtaining the MSE, the λ of each classification layer is s After the update, the MSE is calculated again, and this cycle is repeated until a preset number of times. The classification layer weight with the smallest MSE is taken in the preset number of cycles.

[0049] At the same time, the present application also proposes a flame recognition system based on multi-feature fusion, which is constructed according to the above-mentioned flame recognition method based on multi-feature fusion, including a camera, a temperature sensor, and a computer, wherein the camera and the temperature sensor are used to collect flame image data and ambient temperature data, and the computer is used to perform flame recognition based on the flame image data and ambient temperature data.

[0050] The technical effects of the present invention are as follows: (1) The present invention utilizes multi-dimensional information by integrating multiple features of flame images (including RGB and HIS color space features) and multi-sensor data such as temperature. Combined with the adaptive threshold discrimination method, it can effectively cope with complex flame combustion environments and greatly improve the accuracy and robustness of flame detection. Compared with the traditional system that relies on a single sensor, the present invention reduces the impact of noise and environmental factors on the detection results. (2) In addition, in data processing, the present invention uses operations such as image grayscale, combined with the data filtering technology of the temperature sensor, to perform comprehensive denoising and optimization processing on the original data, thereby improving the credibility of the image and sensor data and providing high-quality input for subsequent feature extraction and working condition classification. (3) The present invention introduces a multi-scale convolutional neural network (MSCNN) and adaptive classification technology. The multi-scale convolution can effectively capture the flame features (such as shape, texture, brightness, etc.) of different regions, while the adaptive classification technology dynamically adjusts the classification criteria according to the flame combustion characteristics, ensuring the accuracy and real-time classification of the flame state, thereby realizing accurate judgment and automatic monitoring of the combustion state. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 A flow chart of a flame recognition method based on multi-feature fusion according to an embodiment of the present invention;

[0052] Figure 2 A flowchart of a specific method when the embodiment of the present invention is applied;

[0053] Figure 3 This is a structural diagram of a multi-scale convolutional neural network according to an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The following is a further detailed description of the specific implementation of the present invention by describing the embodiments with reference to the accompanying drawings, with the purpose of helping those skilled in the art to have a more complete, accurate and in-depth understanding of the inventive concept and technical solution of the present invention and to facilitate its implementation. In order to make the technical solution of the present invention clearer, the present invention is explained by the following embodiments.

[0055] In the existing flame recognition method, in a complex combustion environment, such as garbage incineration, a single sensor is easily affected by smoke, high temperature, high noise, etc., resulting in a decrease in recognition accuracy. Therefore, this embodiment proposes a flame recognition method based on multi-feature fusion, which can solve the shortcomings of the flame recognition process in a complex environment and improve the accuracy and robustness of recognition. Figure 1 As shown, the method comprises the following steps:

[0056] Step S1, obtaining multimodal data of flame, including flame image data and ambient temperature data; and performing preprocessing;

[0057] Step S2, extracting features from the multimodal data of the flame;

[0058] Step S3, performing feature fusion on the extracted multiple features;

[0059] Step S4: classify the flame state based on the feature fused data.

[0060] Combine the following Figure 2 This embodiment is described in detail.

[0061] Specifically, step S1 of this embodiment includes:

[0062] Step S11, flame image data preprocessing, including image denoising, normalization, etc.

[0063] First, the flame image data is acquired by using a camera or other equipment, so as to facilitate the acquisition of RGB and other image information in the subsequent steps. At the same time, the flame image is easily disturbed by noise, such as smoke, light, etc., so this embodiment uses a Gaussian filter method to denoise the flame image data. The formula of Gaussian filter is expressed as follows:

[0064] I'=∑ x,y I(x,y)·G(x,y);

[0065] Where I and y represent the pixel coordinates of the image, I(x,y) represents the pixel value of the image coordinate point (x,y), and G(x,y) represents the Gaussian filter function value of the image coordinate point (x,y). In specific implementation, other q denoising methods can also be flexibly selected according to actual conditions.

[0066] At the same time, in order to eliminate the impact of brightness differences, the image is further normalized so that the pixel values ​​fall into a fixed range, such as [0, 1]. The normalization formula is expressed as follows:

[0067]

[0068] Among them, I norm Represents the normalized value; I represents the pixel value of the image coordinate point; I max , I min They represent the maximum and minimum values ​​of the feature in the entire dataset, respectively.

[0069] Step S12, preprocessing the ambient temperature data, includes using a Kalman filter to perform denoising on the ambient temperature data to eliminate obviously abnormal temperatures and improve the reliability of the data.

[0070] Different features can be extracted not only from multimodal data, but also from single image data to extract features of multiple identification objects. In step S2 of this embodiment, since the temperature data itself is usually a continuous value and can directly reflect environmental changes, the original data has a certain amount of information and does not require additional feature extraction, and it can be used as a temperature feature. For flame image data, multiple different features that can be used for flame identification can be extracted from it. After extracting and fusing them, it is helpful to improve the accuracy of flame identification.

[0071] Specifically, in the feature extraction process of flame image data (including RGB and HIS color space features), the flame image data must first be grayscaled. When perceived by the human eye, the grayscale expression is as follows:

[0072] g(x, y)=0.11×R(x, y)+0.59×G(x, y)+0.30×B(x, y);

[0073] Among them, g(x, y) represents the grayscale value of the pixel at the coordinates (x, y) in the image; R(x, y) represents the brightness value of the red channel at the coordinates (x, y) of the pixel; G(x, y) represents the brightness value of the green channel at the coordinates (x, y) of the pixel; B(x, y) represents the brightness value of the blue channel at the coordinates (x, y) of the pixel.

[0074] In this embodiment, after the image is grayed, the extracted image features include flame effective area, average grayscale of flame image effective area, flame high temperature area, average grayscale of high temperature area, flame area rate, high temperature rate, flame centroid, flame area rate variance per unit time, image grayscale mean variance per unit time, and color component ratio. Among them:

[0075] (1) Flame effective area reflects the combustion efficiency. Generally, the higher the combustion efficiency, the larger the value. The corresponding extraction method is: set the flame effective point grayscale threshold. In the image, if the grayscale value of the coordinate point is greater than the flame effective point grayscale threshold, the corresponding coordinate point is considered to be a valid point. The area of ​​the region composed of all valid points is accumulated as the flame effective area.

[0076] (2) The average grayscale of the flame image effective area reflects the overall temperature level of the flame. Generally, the higher the temperature, the higher the average grayscale value of the flame image effective area. The corresponding extraction method is: calculate the grayscale values ​​of all effective point coordinates in the image and accumulate them. The accumulated value is divided by the flame effective area to obtain the average grayscale of the flame image effective area.

[0077] (3) The area of ​​the flame high temperature zone reflects the flame's anti-interference ability and stability. The corresponding extraction method is: set the flame high temperature point grayscale threshold. In the image, if the grayscale value of the coordinate point is greater than the flame high temperature point grayscale threshold, the corresponding coordinate point is considered to be a high temperature point. The area of ​​the region composed of all high temperature points is accumulated to be the flame high temperature zone area.

[0078] (4) Average grayscale of high temperature zone, which reflects the average temperature level of the high temperature zone. The corresponding extraction method is: calculate the grayscale values ​​of all high temperature point coordinates in the image and add them up. The accumulated value is divided by the area of ​​the flame high temperature zone to obtain the average grayscale of the high temperature zone.

[0079] (5) Flame area ratio reflects the ratio of flame filling the selected image area, which can represent the coverage of the burning area. The corresponding extraction method is:

[0080]

[0081] Among them, S vrepresents the flame area ratio; i×j represents the area of ​​the selected area; Z[] represents a step function, and Z takes 1 when the value in the square brackets is greater than 0, otherwise Z takes 0; g(x,y) represents the gray value of the image pixel coordinate point (x,y); g th Indicates the preset flame point grayscale threshold; g(x,y) is greater than g th When , it means the pixel coordinate point (x, y) is the flame point;

[0082] (6) High temperature rate, which reflects the proportion of the flame high temperature area to the entire image space. The high temperature rate is used to reduce the influence of smoke and dust on the determination of the effective area and high temperature area. The corresponding extraction method is: the area of ​​the flame high temperature area divided by the area of ​​the flame effective area;

[0083] (7) The center of mass of the flame reflects the spatial position information of the flame combustion, which can be used to determine whether the flame is in a reasonable position, which is conducive to the judgment of partial burning. The corresponding extraction method is:

[0084]

[0085] Among them, x c is the horizontal coordinate of the flame center of mass, y c is the ordinate of the flame center of mass;

[0086] (8) The variance of the flame area rate per unit time reflects the change of the flame area. Generally, the flame area of ​​a stable combustion does not fluctuate significantly over time, while the flame area of ​​an unstable combustion flame fluctuates greatly in some areas. The corresponding extraction method is:

[0087]

[0088] Among them, D s represents the variance of flame area rate per unit time; S vi represents the flame area ratio of the image calculated for the i-th time; S vmean It represents the mean value of the flame area rate in the statistical time period; n represents the number of measurements;

[0089] (9) The mean variance of the grayscale of the image per unit time reflects the temperature fluctuation of the flame. The corresponding extraction method is:

[0090]

[0091] Among them, D g represents the mean variance of image grayscale per unit time; g meani Represents the grayscale mean of the image calculated for the i-th time; It represents the mean of the grayscale mean of the image in the statistical time period; n represents the number of measurements;

[0092] (10) Color component ratio reflects flame characteristics. Affected by the flame background, the value of the red component R does not change much between various states, while the other two components G and R change greatly. Therefore, the color component ratio is used as a characteristic quantity to characterize the flame characteristics and state. The corresponding extraction method is:

[0093] K = G / R, or K = B / R;

[0094] Among them, K represents the color component ratio; R represents the brightness value of the red channel at the pixel coordinates (x, y); G represents the brightness value of the green channel at the pixel coordinates (x, y); B represents the brightness value of the blue channel at the pixel coordinates (x, y).

[0095] In step S3 of this embodiment, multiple features extracted in step S2 are fused, including multiple features of image data and temperature features. Common feature fusion methods include weighted fusion, feature-level fusion and decision-level fusion. In this embodiment, feature fusion adopts a weighted fusion method, assigning different weights to each feature for weighted fusion. It can be expressed as:

[0096]

[0097] Among them, F fusion Indicates the fused features; F i represents the i-th feature involved in the fusion; w i represents the i-th feature F i The fusion weight of ; N represents the number of features involved in the fusion.

[0098] Flame state classification is the ultimate goal of the present invention, which relies on fused multimodal features. To this end, in view of the different scale changes of the flame, in step S4 of the present embodiment, the flame image is fine-grainedly detected and classified by a multi-scale convolutional neural network MS-CNN. Based on this, on the one hand, the key areas in the flame image can be detected from the fused multimodal features to obtain the flame state detection result, and on the other hand, the flame can be classified in advance according to the working conditions when the flame is burning, and then the overall working conditions when the flame is burning can be identified from the fused multimodal features.

[0099] Among them, Figure 3 As shown in FIG. 1 , MS-CNN uses convolution kernels of different scales in the input layer to extract multi-level features from the fused features again; in the hidden layer, the extracted multi-scale features are integrated and fused again through parallel convolution layers; finally, the integrated and fused multi-scale features are input into the classifier of the output layer for classification, wherein the classification errors of different flame states (such as large flames and small flames) are balanced by a multi-scale loss function, and the multi-scale loss function is expressed as follows:

[0100]

[0101] Among them, L MS-CNN Represents the value of the multi-scale loss function; L s is the classification loss of the s-th classification layer; s Furthermore, the weights in the multi-scale loss function are adjusted by an adaptive algorithm, that is, the classification criteria are dynamically adjusted according to the flame combustion characteristics, ensuring the accuracy and real-time performance of the flame state classification, thereby achieving accurate judgment and automatic monitoring of the combustion state.

[0102] The adaptive algorithm used in this embodiment is the minimum mean square error, that is:

[0103]

[0104] Where MSE stands for mean square error; L MS-CNN Represents the multi-scale loss function value of each sample, and n represents the number of samples; after obtaining the MSE, the λ of each classification layer is s After the update, the MSE is calculated again, and the cycle is repeated until the preset number of times. The classification layer weight with the minimum MSE is taken in the preset number of cycles. This embodiment uses a multi-scale convolutional neural network MS-CNN to finely classify different flame states, and combines an adaptive algorithm to achieve real-time feedback and optimization adjustment of flame state classification. This multi-scale classification technology adapts to different flame combustion environments while ensuring real-time performance and accuracy.

[0105] Finally, the flame state detection and classification results can be visualized through the host computer.

[0106] At the same time, the present application also proposes a flame recognition system based on multi-feature fusion, which is constructed according to the above-mentioned flame recognition method based on multi-feature fusion, including a camera, a temperature sensor, and a computer, wherein the camera and the temperature sensor are used to collect flame image data and ambient temperature data, and the computer is used to perform flame recognition based on the flame image data and ambient temperature data.

[0107] The present invention uses multi-feature fusion technology to combine multiple sensor data such as images and temperature, so that the system can adaptively handle complex flame combustion environments, overcome the problem that traditional single sensors are easily disturbed by noise, smoke, etc., and significantly improve the accuracy and robustness of flame recognition. In addition, the present invention performs data preprocessing on the collected image data and temperature data to achieve denoising, effectively improving the reliability of the data. The present invention also uses multi-scale convolutional neural networks and adaptive classification technology to accurately classify flame states, thereby ensuring long-term operational stability and automated monitoring of the combustion process.

[0108] The present invention is described above by way of example in conjunction with the accompanying drawings. Obviously, the specific implementation of the present invention is not limited to the above-mentioned methods. As long as various non-substantial improvements are made using the method concept and technical solution of the present invention; or the above concept and technical solution of the present invention are directly applied to other occasions without improvement, they are all within the protection scope of the present invention.

Claims

1. A flame recognition method based on multi-feature fusion, characterized in that: The method comprises the following steps: Step S1, obtaining multimodal data of flame, including flame image data and ambient temperature data; and performing preprocessing; Step S2, extracting features from the multimodal data of the flame; Step S3, performing feature fusion on the extracted multiple features; Step S4: classify the flame state based on the feature fused data.

2. The flame recognition method based on multi-feature fusion according to claim 1 is characterized in that: The step S1 comprises: Step S11, flame image data preprocessing, including image denoising and normalization; Step S12, preprocessing the ambient temperature data, including using a Kalman filter to perform denoising on the ambient temperature data.

3. The flame recognition method based on multi-feature fusion according to claim 1 is characterized in that: In step S2, for feature extraction of flame image data, the flame image data must first be grayed. When perceived by human eyes, the grayed expression is as follows: g(x,y)=0.11×R(x,y)+0.59×G(x,y)+0.30×B(x,y); Among them, g(x,y) represents the grayscale value of the pixel at the coordinates (x,y) in the image; R(x,y) represents the brightness value of the red channel at the pixel at the coordinates (x,y); G(x,y) represents the brightness value of the green channel at the pixel at the coordinates (x,y); B(x,y) represents the brightness value of the blue channel at the pixel at the coordinates (x,y).

4. The flame recognition method based on multi-feature fusion according to claim 3 is characterized in that: In step S2, feature extraction is performed based on the grayscale image, and the extracted multiple features include flame effective area, average grayscale of flame image effective area, flame high temperature area, average grayscale of high temperature area, flame area rate, high temperature rate, flame centroid, flame area rate variance per unit time, image grayscale mean variance per unit time, and color component ratio.

5. The flame recognition method based on multi-feature fusion according to claim 4 is characterized in that: The multi-feature extraction methods of the image are: (1) Flame effective area, which reflects the combustion efficiency. The corresponding extraction method is: set the flame effective point grayscale threshold. In the image, if the grayscale value of the coordinate point is greater than the flame effective point grayscale threshold, the corresponding coordinate point is considered to be an effective point. The area of ​​the region composed of all effective points is accumulated to be the flame effective area. (2) The average grayscale of the flame image effective area reflects the overall temperature level of the flame. The corresponding extraction method is: calculate the grayscale values ​​of the coordinates of all effective points in the image and accumulate them. The accumulated value is divided by the flame effective area to obtain the average grayscale of the flame image effective area; (3) The area of ​​the flame high temperature zone reflects the flame's anti-interference ability and stability. The corresponding extraction method is: set the flame high temperature point grayscale threshold. In the image, if the grayscale value of the coordinate point is greater than the flame high temperature point grayscale threshold, the corresponding coordinate point is considered to be a high temperature point. The area of ​​the region composed of all high temperature points is accumulated to be the flame high temperature zone area. (4) Average grayscale of high temperature zone, which reflects the average temperature level of the high temperature zone. The corresponding extraction method is: calculate the grayscale values ​​of all high temperature point coordinates in the image and add them up. The accumulated value is divided by the area of ​​the flame high temperature zone to obtain the average grayscale of the high temperature zone. (5) Flame area ratio, which reflects the ratio of flame filling the selected image area. The corresponding extraction method is: Among them, S v represents the flame area ratio; i×j represents the area of ​​the selected area; Z[] represents a step function, and Z takes 1 when the value in the square brackets is greater than 0, otherwise Z takes 0; g(x,y) represents the grayscale value of the image pixel coordinate point (x,y); g th Indicates the preset flame point grayscale threshold; g(x,y) is greater than g th When , it means the pixel coordinate point (x, y) is the flame point; (6) High temperature rate, which reflects the proportion of the flame high temperature area to the entire image space. The corresponding extraction method is: the area of ​​the flame high temperature area divided by the area of ​​the flame effective area; (7) Flame centroid, which reflects the spatial position information of flame combustion. The corresponding extraction method is: Among them, x c is the horizontal coordinate of the flame center of mass, y c is the ordinate of the flame center of mass; (8) The variance of the flame area rate per unit time reflects the change of the flame area. The corresponding extraction method is: Among them, D s represents the variance of flame area rate per unit time; S vi represents the flame area ratio of the image calculated for the i-th time; S vmean It represents the mean value of the flame area rate in the statistical time period; n represents the number of measurements; (9) The mean variance of the grayscale of the image per unit time reflects the temperature fluctuation of the flame. The corresponding extraction method is: Among them, D g represents the mean variance of image grayscale per unit time; g meani Represents the grayscale mean of the image calculated for the i-th time; It represents the mean of the grayscale mean of the image in the statistical time period; n represents the number of measurements; (10) Color component ratio reflects the flame characteristics, and the corresponding extraction method is: K = G / R, or K = B / R; Among them, K represents the color component ratio; R represents the brightness value of the red channel at the pixel coordinates (x, y); G represents the brightness value of the green channel at the pixel coordinates (x, y); B represents the brightness value of the blue channel at the pixel coordinates (x, y).

6. A flame recognition method based on multi-feature fusion according to any one of claims 1 to 5, characterized in that: The feature fusion adopts a weighted fusion method, which is expressed by the formula: Among them, F fusion Indicates the fused features; F i represents the i-th feature involved in the fusion; w i represents the i-th feature F i The fusion weight of ; N represents the number of features involved in the fusion.

7. The flame recognition method based on multi-feature fusion according to claim 1 is characterized in that: In step S4, flame state classification is performed by a multi-scale convolutional neural network MS-CNN, wherein MS-CNN uses convolution kernels of different scales to extract multi-level features from the fused features again, integrates the extracted multi-scale features through parallel convolution layers, and then fuses the integrated multi-scale features and inputs them into a classifier for classification, wherein the classification errors of different flame states are balanced by a multi-scale loss function, and the multi-scale loss function is expressed as follows: Among them, L MS-CNN Represents the value of the multi-scale loss function; L s is the classification loss of the s-th classification layer; s is the corresponding weight.

8. The flame recognition method based on multi-feature fusion according to claim 7 is characterized in that: The weights in the multi-scale loss function are adjusted by an adaptive algorithm.

9. The flame recognition method based on multi-feature fusion according to claim 8 is characterized in that: The adaptive algorithm includes the minimum mean square error, namely: Where MSE stands for mean square error; L MS-CNN Represents the multi-scale loss function value of each sample, and n represents the number of samples; after obtaining the MSE, the λ of each classification layer is s After the update, the MSE is calculated again, and this cycle is repeated until a preset number of times. The classification layer weight with the smallest MSE is taken in the preset number of cycles.

10. A flame recognition system based on multi-feature fusion, characterized in that: The system is constructed according to a flame recognition method based on multi-feature fusion according to any one of claims 1 to 9, and includes a camera, a temperature sensor, and a computer, wherein the camera and the temperature sensor are used to collect flame image data and ambient temperature data, and the computer is used to perform flame recognition based on the flame image data and ambient temperature data.

Images