A factory flue gas emission monitoring and analyzing method based on multi-modal data

Abstract

The application discloses a kind of factory smoke exhaust monitoring and analysis method based on multi-modal data, including data acquisition, target identification model, AiNa radiant energy detection model, distance detection model, radiant energy compensation model, smoke detection model and trigger prediction and hierarchical warning model;The application belongs to the field of environmental field monitoring, solves the problem that the accuracy and timeliness is not high when environmental monitoring station monitors and analyzes the pollution situation caused by factory smoke exhaust;The advantages are: specific target and its temperature can be quickly and accurately predicted;Effectively improve the prediction accuracy of infrared camera;Effectively improve the situation that infrared camera cannot accurately detect due to insufficient received radiant energy when the installation position of infrared camera is far away from the ignition point;Using multi-modal data fusion, a variety of early warning signals are given for the pollution situation of factory smoke exhaust, to help staff monitor the atmospheric pollution caused by factory smoke exhaust and effectively guide the atmospheric pollution control work according to the early warning signal.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of environmental field monitoring, and particularly relates to a factory flue gas monitoring and analyzing method based on multi-modal data. BACKGROUND

[0002] The factory flue gas contains a large amount of particulate matter, gaseous pollutants and other harmful substances, which not only affects human health, but also causes damage to the ecological system; for example, inhalable particulate matter (PM10) and fine particulate matter (PM2.5) can cause serious harm to the human respiratory system and cause diseases such as asthma and bronchitis; sulfur dioxide (SO2) and nitrogen oxides (NOx) are one of the main causes of acid rain, which can cause damage to the soil, water and ecological system; if not monitored and controlled, these pollutants will be directly discharged into the atmosphere, resulting in a decrease in air quality, and the pollutants will also excessively interfere with the ecological system and affect biodiversity.

[0003] Through monitoring and analyzing the factory flue gas, the emission of atmospheric pollutants can be timely mastered, early warning of the atmospheric pollution situation can be provided, and a scientific basis for formulating atmospheric pollution prevention and control measures can be provided.

[0004] The data types for monitoring the factory flue gas mainly include the following: concentration, including PM2.5 and PM10 concentrations reflecting the content of smoke dust in the factory flue gas, and concentrations of various harmful gases such as sulfur dioxide SO2, nitrogen oxides NOx, carbon monoxide CO and hydrogen chloride HCl; temperature, reflecting the efficiency of the combustion process and the operating condition of the equipment; and diffusion range, which can be used to calculate the emission amount of the pollutants.

[0005] The factory flue gas can be monitored by a sensor or by image monitoring; usually, a smoke sensor is used to monitor the smoke content, or a gas sensor is used to monitor the harmful gas content, or a temperature sensor is used to monitor the flue gas temperature; and image monitoring mainly uses high-definition cameras, thermal imagers and other equipment to take real-time pictures of the factory flue gas outlet, and through analysis of the taken images, information such as the color, concentration and temperature of the flue gas is obtained, so as to determine whether the flue gas is normal and whether there is a pollution problem; for example, under normal circumstances, the factory flue gas may be white or light gray, if the color of the flue gas changes to black, yellow and other abnormal colors, it may mean that the flue gas contains a large amount of particulate matter or other pollutants, and through analysis of the diffusion range of the flue gas in the image, the concentration and emission amount of the flue gas can be determined.

[0006] The smoke sensor realizes the alarm function by monitoring the concentration of the smoke, and when the smoke concentration reaches a certain degree, the smoke sensor automatically issues an alarm, but due to the large amount of factory flue gas, the smoke sensor may be false alarmed.

[0007] The traditional computer vision-based smoke detection scheme is essentially a classification problem, that is, judging the information of the smoke image in the image based on the convolutional neural network. This way is relatively accurate in color and concentration information judgment, but not accurate in temperature information judgment.

[0008] The use of an infrared camera can solve the above problems. According to the characteristic that the higher the temperature of an object, the more infrared radiation energy it emits, the temperature of the smoke can be judged according to the radiation energy of the object. However, the change of the environmental temperature will affect the prediction accuracy of the infrared camera, and when the installation position of the infrared sensor is far away from the factory chimney, the situation of insufficient received radiation energy and inaccurate detection occurs from time to time. SUMMARY

[0009] Technical problems to be solved

[0010] To solve the above problems of the prior art, the present application provides a factory smoke monitoring and analysis method based on multi-modal data, which aims to:

[0011] (1) solve the problem of false alarm of smoke sensor triggered during factory smoke;

[0012] (2) solve the influence of environmental temperature on the working performance of the infrared camera and improve the prediction accuracy of the infrared camera;

[0013] (3) improve the situation that when the installation position of the infrared camera is far away from the ignition point, the received radiation energy is insufficient and accurate detection cannot be achieved;

[0014] (4) use the multi-modal data fusion technology of smoke sensor, gas sensor, binocular camera, infrared camera and temperature sensor to give multiple warning signals for the pollution situation of factory smoke, help workers effectively monitor the atmospheric pollution caused by factory smoke and effectively guide the atmospheric pollution control work according to the warning signals.

[0015] Technical scheme

[0016] In order to achieve the above purpose, a technical scheme adopted by the present application is: a factory smoke monitoring and analysis method based on multi-modal data, comprising data acquisition, target recognition model, AiNa radiation energy detection model, distance detection model, radiation energy compensation model, smoke detection model and trigger prediction and hierarchical warning model:

[0017] Further, the data acquisition includes monocular image collection and preprocessing, infrared image collection and preprocessing, environmental temperature acquisition, smoke concentration acquisition and gas pollutant concentration acquisition, and the data acquisition is for the data of the factory smoke at position index x:

[0018] 1) monocular image collection and preprocessing Obtain monocular image and perform preprocessing operation on monocular image, the steps of preprocessing operation on monocular image include increasing image contrast, Gaussian filtering and image sharpening, and then monocular preprocessing image SP is obtained;

[0019] 2) infrared image collection and preprocessing Obtain infrared image and perform preprocessing operation on infrared image, the steps of preprocessing operation on infrared image include increasing image contrast, Gaussian filtering, image sharpening and graying, and then infrared preprocessing image IP is obtained;

[0020] 3) ambient temperature acquisition is responsible for acquiring ambient temperature AT and outputting ambient temperature AT;

[0021] 4) smoke concentration acquisition is responsible for acquiring smoke concentration SC, when smoke concentration SC>500mg / m3, the output of smoke concentration acquisition unit is 1, otherwise the output is 0; The output of smoke concentration acquisition is recorded as Out_1;

[0022] 5) gas pollutant concentration acquisition is responsible for acquiring the concentration Out_2 of harmful gases such as sulfur dioxide SO2, nitrogen oxides NOx, carbon monoxide CO and hydrogen chloride HCl in smoke, Out_2={gas_i}, i∈[1,4], wherein gas_i=(a_i-m_i) / a_i, a_i represents the current concentration of the i-th harmful gas, and m_i represents the standard concentration of the i-th gas.

[0023] As preferred, the target recognition model is based on YOLO algorithm for deep learning to obtain the data set of factory exhaust smoke, and based on the data set for transfer learning training, using the trained target recognition model to detect the monocular preprocessing image SP, the bounding box and the corresponding confidence of the target object can be obtained, and the target object is the factory exhaust smoke.

[0024] Further, the AiNa radiant energy detection model processes the infrared preprocessing image IP, specifically including feature collection, feature extraction, up-sampling, channel merging, scale transformation, feature compression and combination convolution, because the size of the thermal radiation energy of the object is related to the temperature of the surface of the object, therefore the final output of the AiNa radiant energy detection model is the detection temperature T corresponding to the maximum position of the radiant energy of the factory exhaust smoke at position x.

[0025] The mathematical principle of the distance detection model is the binocular distance measurement principle, so the monocular camera and the infrared camera are required to be placed in parallel, to synchronously collect the front image, and the monocular camera and the infrared camera are required to have the same focal length, at this time the monocular image and the infrared image obtained can be applied to the distance detection model, the distance of the target object is calculated by using the pixel difference between the monocular image and the infrared image, and the distance detection model obtains the final predicted distance of the factory exhaust smoke at position xod .

[0026] As preferred, the ambient temperature AT, the final predicted distance od detected by the distance detection model, and the detected temperature T obtained by the radiant energy detection model are respectively input into the radiant energy compensation model, the radiant energy compensation model corrects the predicted temperature corresponding to the object to obtain the predicted temperature T C of the target object; the output of the radiant energy compensation model is denoted as Out_3= (T C-AT) / T C.

[0027] Further, the smoke detection model detects the monocular preprocessed image SP processed by the target recognition model to obtain the deepest color and diffusion range of the factory smoke in SP; the output of the smoke detection model is Out_4={color, area}; For the color output, the deeper the color, the more serious the pollution of the smoke, wherein R, G and B respectively represent the values corresponding to red, green and blue in RGB; area∈[0, 1], which is the area ratio of the smoke area in the monocular preprocessed image SP.

[0028] Further, the trigger prediction and hierarchical warning model is a combined neural network, including an input layer, a competitive layer and an output layer, the input layer contains 8 neurons, respectively corresponding to the data in the sample vector of the factory smoke at position x composed of Out_1, Out_2, Out_3 and Out_4; the competitive layer includes 8 neurons and each pair of neurons forms a group; the output layer contains 4 neurons, each neuron is only connected to one group of neurons in the competitive layer; the weight matrix from the input layer to the competitive layer is defined as W1, and the weight matrix from the competitive layer to the output layer is defined as W2; after the trigger prediction and hierarchical warning model is trained, the trigger prediction and hierarchical warning model can effectively predict according to the input data, and the four prediction results correspond to four alarm information, that is, normal smoke, abnormal alarm (slight pollution alarm), medium pollution alarm and serious pollution alarm.

[0029] Advantages

[0030] (1) The AiNa radiant energy detection model can quickly and accurately predict the temperature of a specific target and its corresponding temperature;

[0031] (2) The radiant energy compensation model can effectively solve the influence of ambient temperature on the working performance of the infrared camera and improve the prediction accuracy of the infrared camera;

[0032] (3) The radiant energy compensation model can effectively improve the situation that the infrared camera cannot accurately detect due to insufficient received radiant energy when the installation position of the infrared camera is far away from the smoke position;

[0033] (4) Trigger prediction and hierarchical warning model uses multi-modal data fusion of smoke sensor, gas sensor, binocular camera, infrared camera and temperature sensor to give various early warning signals at different stages of pollution, helping workers to effectively prevent atmospheric pollution caused by factory exhaust smoke and guiding the treatment of atmospheric pollution according to the early warning signals. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1 A flowchart of a factory exhaust smoke monitoring and analysis method based on multi-modal data is provided for the present application;

[0035] Figure 2 A detection step flowchart of the AiNa radiation energy detection model is provided for the present application;

[0036] Figure 3 The trigger prediction and hierarchical warning model is provided for the present application;

[0037] The accompanying drawings are included to provide a further understanding of the present application, and constitute a part of the specification, which together with the embodiments of the present application, serve to explain the present application, and do not constitute a limitation on the present application. DETAILED DESCRIPTION

[0038] A factory exhaust smoke monitoring and analysis method based on multi-modal data, as shown in Figure 1 , includes data acquisition, target recognition model, AiNa radiation energy detection model, distance detection model, radiation energy compensation model, exhaust smoke detection model and trigger prediction and hierarchical warning model.

[0039] Data acquisition includes monocular image collection and preprocessing, infrared image collection and preprocessing, environment temperature acquisition, smoke concentration acquisition and gas pollutant concentration acquisition, which are all for data of factory exhaust smoke at position index x:

[0040] 1) Monocular image collection and preprocessing acquires monocular images and performs preprocessing operations on the monocular images, the steps of preprocessing the monocular images include increasing image contrast, Gaussian filtering and image sharpening, and then obtaining monocular preprocessed images SP;

[0041] 2) Infrared image collection and preprocessing acquires infrared images and performs preprocessing operations on the infrared images, the steps of preprocessing the infrared images include increasing image contrast, Gaussian filtering, image sharpening and grayscale, and then obtaining infrared preprocessed images IP;

[0042] 3) Environment temperature acquisition is responsible for acquiring and outputting environment temperature AT;

[0043] 4) Smoke concentration acquisition is responsible for acquiring smoke concentration SC, when smoke concentration SC > 500 mg / m3, the output of smoke concentration acquisition unit is 1, otherwise the output is 0; the output of smoke concentration acquisition is recorded as Out_1;

[0044] 5) Gas pollutant concentration acquisition is responsible for acquiring the concentration Out_2 of harmful gases sulfur dioxide SO2, nitrogen oxides NOx, carbon monoxide CO, and hydrogen chloride HCl in the smoke, Out_2={gas_i}, i∈[1,4], wherein gas_i=(a_i-m_i) / a_i, a_i represents the current concentration of the i-th harmful gas, and m_i represents the standard concentration of the i-th gas.

[0045] The target recognition model obtains a data set of factory exhaust smoke based on deep learning of the YOLO algorithm, and performs transfer learning training based on the data set. The trained target recognition model is used for target detection on the monocular preprocessed image SP, and the bounding box and confidence of the target object, i.e. the factory exhaust smoke, can be obtained.

[0046] The AiNa radiation energy detection model includes feature acquisition, feature extraction, upsampling, channel merging, scale transformation, feature compression and combined convolution. Since the thermal radiation energy of an object is related to the temperature of the surface of the object, the final output of the AiNa radiation energy detection model is the detection temperature T corresponding to the maximum position of the radiation energy of the factory exhaust smoke at position x. To achieve this purpose, the features include feature acquisition, feature extraction, upsampling, channel merging, scale transformation, feature compression and combined convolution. The detection steps of the AiNa radiation energy detection model are as follows, as shown in Figure 2

[0047] (1) First, feature acquisition is performed on the infrared preprocessed image IP to obtain a feature image P1 with a scale of 20x20x256;

[0048] (2) The feature image P1 is subjected to a feature extraction operation to further extract the features in the image, and a feature image P2 with a scale of 10x10x256 is obtained;

[0049] (3) The feature image P2 is subjected to an upsampling operation, and the feature image P1 is subjected to channel merging to obtain a feature image P3, and the scale of the feature image P3 is 20x20x512;

[0050] (4) The feature image P3 is subjected to a scale transformation operation to obtain a feature vector V1, and the scale of the feature vector V1 is 1x1x200;

[0051] (5) The feature image P2 is subjected to a feature compression operation to obtain a feature vector V2, and the scale of the feature vector V2 is 1x1x200;

[0052] ​(6) the channel merging operation is performed on the feature vector V1 and the feature vector V2 to obtain a feature vector V3, and the size of the feature vector V3 is 1*1*400;

[0053] (7) the feature vector V3 is connected with an output neuron after passing through an intermediate hidden layer one and an intermediate hidden layer two, the intermediate hidden layer one contains 100 neurons, the intermediate hidden layer two contains 20 neurons, and the output neuron contains only one neuron, and the output value of the output neuron is the detection temperature T.

[0054] In the process of training the AiNa radiation energy detection model, a square loss function is used for the loss function, and the formula is as follows:

[0055]

[0056] In the formula, T represents the real temperature of the target object, y i T represents the detection temperature of the target object, and n represents the number of data in a batch. z i The distance detection model, the mathematical principle of the distance detection model is binocular distance measurement principle, so the monocular camera and the infrared camera for collecting images are required to be placed in parallel, the front image is required to be collected synchronously, and the monocular camera and the infrared camera are required to have the same focal length, so that the monocular image and the infrared image obtained can be applied to the distance detection model, and the distance of the target object is calculated by using the pixel difference between the monocular image and the infrared image; the distance calculation formula is:

[0057]

[0058]

[0059] In the formula, D-(r-l) is the parallax, (r-l) is the epipolar distance, D is the baseline distance of the monocular camera and the infrared camera, ƒ is the focal length of the monocular camera, and is also the focal length of the infrared camera, od is the vertical distance of a target pixel to the baseline. The traditional distance detection model needs to perform feature matching on each detected feature point, but not every feature point is useful information that we care about, therefore, the application provides a target-based feature point detection and matching method, which can further reduce the calculation amount and obtain more accurate calculation results, and the specific steps are as follows:

[0060] S1, target detection, target detection is performed on the monocular image and the binocular image respectively;

[0061] S2, target-based feature point detection;

[0062] S3, target-based feature point matching;

[0063] ​​​S4, target-based feature point distance prediction;

[0064] S5, calculate the average value and get the final predicted distance od .

[0065] The ambient temperature AT, the final predicted distance od detected by the distance detection model, and the detection temperature T obtained by the radiant energy detection model are respectively sent to the radiant energy compensation model. The radiant energy compensation model corrects the predicted temperature corresponding to the object to obtain the predicted temperature T_C of the target object. The output of the radiant energy compensation model is Out_3= (T_C-AT) / T_C.

[0066] The smoke detection model detects the monocular preprocessed image SP processed by the target recognition model to obtain the deepest color and diffusion range of the factory smoke in SP. The output of the smoke detection model is Out_4={color, area}. color=(R+G+B) / (256*3) is the output for color, and the deeper the color, the more serious the pollution of the smoke, where R, G and B represent the values of red, green and blue in RGB corresponding to the obtained color. area∈[0,1] is the area ratio of the smoke area in the monocular preprocessed image SP.

[0067] Trigger the prediction and hierarchical warning model, which is a combination of neural networks, as shown in Figure 3 The input layer includes 8 neurons, which correspond to the data in the sample vector A of the factory smoke at position x composed of Out_1, Out_2, Out_3 and Out_4. The competitive layer includes 8 neurons and is divided into four groups. The output layer includes 4 neurons, each of which is connected to one of the four groups of neurons in the competitive layer. The weight matrix from the input layer to the competitive layer is defined as W1, and the weight matrix from the competitive layer to the output layer is defined as W2. The specific algorithm steps are as follows:

[0068] S0, data preparation, in the form of:

[0069] ;

[0070] In the above formula, A represents the sample vector composed of Out_1, Out_2 and Out_3, and d1-d4 represents the expected output mode, which is:

[0071] ;

[0072] The above four formulas represent smoke normal, abnormal alarm (slight pollution alarm), medium pollution alarm and serious pollution alarm, a total of four mode classes.

[0073] S1, for each weight vector W1 in the weight matrix W1 from the input layer to the competition layer j where j ∈ [1, 8], initialized to a small decimal value around 0, given an initial learning rate α(t), which is a function of time monotonically decreasing:

[0074] ;

[0075] In the above formula, C is a small number between 0 and 1, B is a constant greater than 1, t m is a time constant greater than 1;

[0076] S2, based on the input sample vector A and the weight vector W1, find the winning neuron:

[0077] ;

[0078] When the winning neuron is found, the output of the winning neuron is 1, the output of the neurons in the same group as the winning neuron is also 1, and the output of all other neurons is 0; the output result is recorded as Y, where the output result Y has and only has four results:

[0079] ;

[0080] S3, define the weight matrix W2 from the competition layer to the output layer as:

[0081] ;

[0082] S4, according to the calculation result of the winning neuron, calculate the predicted output:

[0083] ;

[0084] S5, weight adjustment, only the winning neuron adjusts the weight vector, and the rest of the neurons do not adjust:

[0085] (1) When the predicted output is consistent with the expected output, the adjustment method of the weight vector corresponding to the winning neuron is as follows:

[0086] ;

[0087] (2) When the predicted output is inconsistent with the expected output, the adjustment method of the weight vector corresponding to the winning neuron is as follows:

[0088] ;

[0089] where k is the iteration number, and the initial value is 0.

[0090] After the trigger prediction and hierarchical warning model is trained, the trigger prediction and hierarchical warning model can effectively predict according to input data, and four prediction results correspond to four alarm information, that is, normal smoke exhaust, abnormal alarm (slight pollution alarm), medium pollution alarm and serious pollution alarm.

[0091] The above is the specific workflow of the present application, and the next time you use it, repeat this step.

[0092] It should be noted that in this paper, the relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or equipment.

[0093] Although the embodiments of the present application have been shown and described, it can be understood by those of ordinary skill in the art that various changes, modifications, replacements and variations can be made to the embodiments without departing from the principles and spirits of the present application, and the scope of the present application is defined by the appended claims and their equivalents.

[0094] The present application and its embodiments have been described above, and such description is not restrictive, and the embodiments shown in the drawings are only one of the embodiments of the present application, and the actual structure is not limited thereto. In general, if a person skilled in the art is inspired by it, without departing from the purpose of the present application, without creative design, similar structure and embodiments of the technical solution should belong to the protection scope of the present application.

Claims

1. A method for factory flue gas monitoring and analysis based on multi-modal data, the method comprising: The data acquisition, target recognition model, AiNa radiation energy detection model, distance detection model, radiation energy compensation model, smoke exhaust detection model, and trigger prediction and grading warning model are included. ​ The data acquisition includes monocular image collection and preprocessing, infrared image collection and preprocessing, environment temperature acquisition, smoke concentration acquisition, and gas pollutant concentration acquisition, and the data acquisition is all for data of factory smoke at position index x: 1) Monocular image collection and preprocessing acquires monocular images and performs preprocessing operations on the monocular images, the preprocessing operations on the monocular images include increasing image contrast, Gaussian filtering, and image sharpening, and then a monocular preprocessed image SP is obtained; 2) Infrared image collection and preprocessing acquires infrared images and performs preprocessing operations on the infrared images, the preprocessing operations on the infrared images include increasing image contrast, Gaussian filtering, image sharpening, and grayscale, and then an infrared preprocessed image IP is obtained; 3) Environment temperature acquisition is responsible for acquiring and outputting environment temperature AT; 4) Smoke concentration acquisition is responsible for acquiring smoke concentration SC, when the smoke concentration SC is greater than a predetermined value, the output of the smoke concentration acquisition unit is 1, otherwise the output is 0; the output of the smoke concentration acquisition is denoted as Out_1; 5) Gas pollutant concentration acquisition is responsible for acquiring the concentrations of harmful gases in smoke, such as sulfur dioxide SO2, nitrogen oxides NOx, carbon monoxide CO, and hydrogen chloride HCl, Out_2; The target recognition model is trained based on a factory smoke data set using transfer learning, and the trained target recognition model is used to detect targets in the monocular preprocessed image SP, which can obtain the bounding box and corresponding confidence of the target object, which is the factory smoke; The AiNa radiation energy detection model includes feature acquisition, feature extraction, up-sampling, channel merging, scale transformation, feature compression, and combined convolution. Since the thermal radiation energy of an object is related to the temperature of the object's surface, the final output of the AiNa radiation energy detection model is the detection temperature T corresponding to the maximum radiation energy position of the factory smoke at position x; The distance detection model calculates the distance of the target object using the pixel difference between the monocular image and the infrared image collected by the monocular camera, and obtains the final predicted distance od of the factory smoke at position x; the monocular camera and the infrared camera have the same focal length and are placed in parallel, and the front images are collected synchronously; The environment temperature AT, the final predicted distance od detected by the distance detection model, and the detection temperature T obtained by the radiation energy detection model are sent to the radiation energy compensation model respectively, the radiation energy compensation model corrects the detection temperature corresponding to the object, and obtains the predicted temperature T_C of the target object; the output of the radiation energy compensation model is denoted as Out_3= (T_C-AT) / T_C; The smoke emission detection model detects the monocular preprocessed image SP processed by the target recognition model to obtain the deepest color and diffusion range of the factory smoke in SP; the output of the smoke emission detection model is Out_4={color, area}; color is the output for color, and the deeper the color, the more serious the pollution of the smoke emission; area∈[0, 1] is the area ratio of the smoke area in the monocular preprocessed image SP; The trigger prediction and hierarchical warning model is a combined neural network including an input layer, a competitive layer and an output layer; the input layer includes 8 neurons corresponding to the data in the sample vector of the factory smoke at position x composed of Out_1, Out_2, Out_3 and Out_4; the competitive layer includes 8 neurons and each two neurons form a group; the output layer includes 4 neurons, each of which is connected to one of the four groups of neurons in the competitive layer; the weight matrix from the input layer to the competitive layer is defined as W1, and the weight matrix from the competitive layer to the output layer is defined as W2; after the training of the trigger prediction and hierarchical warning model is completed, the trigger prediction and hierarchical warning model can effectively predict according to the input data, and the four prediction results correspond to four alarm information, i.e. normal smoke emission, slight pollution alarm, medium pollution alarm and serious pollution alarm.

2. The method of claim 1, wherein: The monocular image collection and preprocessing includes the steps of increasing image contrast, Gaussian filtering and image sharpening, thereby obtaining a monocular preprocessed image SP.

3. The method of claim 1, wherein: The infrared image collection and preprocessing includes the steps of increasing image contrast, Gaussian filtering, image sharpening and grayscale, thereby obtaining an infrared preprocessed image IP.

4. The method of claim 1, wherein: The smoke concentration preset value is 500 mg / m3.

5. The method of claim 1, wherein: Out_2={gas_i}, i∈[1, 4], wherein gas_i=(a_i-m_i) / a_i, a_i represents the current concentration of the i-th harmful gas, and m_i represents the standard concentration of the i-th gas.

6. The method of factory flue gas monitoring and analysis based on multi-modal data according to claim 1, characterized in that: The target recognition model obtains the data set of the factory smoke based on the deep learning of the YOLO algorithm.

7. The method of claim 1, wherein: The detection steps of the AiNa radiant energy detection model are as follows: (1) first, feature collection is performed on the infrared preprocessed image IP to obtain a feature image P1 with a scale of 20×20×256; (2) the feature image P1 is subjected to feature extraction operation to further extract the features in the image, thereby obtaining a feature image P2 with a scale of 10×10×256; (3) the feature image P2 is subjected to upsampling operation, and is combined with the feature image P1 to obtain a feature image P3, and the scale of the feature image P3 is 20×20×512; (4) the feature image P3 is subjected to scale transformation operation to obtain a feature vector V1 with a scale of 1×1×200; (5) the feature image P2 is subjected to feature compression operation to obtain a feature vector V2 with a scale of 1×1×200; (6) the feature vector V1 and the feature vector V2 are subjected to channel merging operation to obtain a feature vector V3 with a scale of 1×1×400; (7) The characteristic vector V3 is connected with the output neuron after passing through the intermediate hidden layer one and the intermediate hidden layer two, the intermediate hidden layer one contains 100 neurons, the intermediate hidden layer two contains 20 neurons, the output neuron only contains one neuron, and the output value of the output neuron is the detection temperature T.

8. The method of claim 1, wherein: In the process of training the AiNa radiation energy detection model, a square loss function is used, and the formula is as follows: ; in the above formula, y i represents the real temperature of the target object, z i represents the detected temperature of the target object, and n represents the number of data in a batch.

9. The method of claim 1, wherein: The distance calculation formula of the distance detection model is: ; In the formula, D-(r-l) is the parallax, (r-l) is the extinction difference, D is the baseline distance of the monocular camera and the infrared camera, f is the focal length of the monocular camera and also the focal length of the infrared camera, and od is the vertical distance of a target pixel to the baseline.

10. The method of claim 1, wherein: where R, G and B represent the values corresponding to red, green and blue in the RGB color space, respectively.

11. The method of claim 1, wherein: The specific steps of the trigger prediction and hierarchical warning model are as follows: S0, data preparation, in the form of: ; In the above formula, A represents a sample vector composed of Out_1, Out_2, and Out_3, and d1-d4 represent desired output patterns, respectively as follows: ; The meanings of the four formulas above represent four modes, respectively, normal smoke exhaust, abnormal alarm (slight pollution warning), medium pollution warning and serious pollution warning; S1, for each weight vector W1 in the weight matrix W1 from the input layer to the competitive layer j wherein j ∈ [1, 8], initialized as a small decimal value around 0, given an initial learning rate α(t), which is a function monotonically decreasing with time: ; In the above formula, C is a decimal number between 0 and 1, B is a constant greater than 1, t m is a time constant greater than 1; S2, based on the input sample vector A and the weight vector W1, finds the winning neuron: ; when the winning neuron is found, the output of the winning neuron is 1, the outputs of the neurons in the same group as the winning neuron are also 1, and the outputs of all other neurons are 0; the output result is recorded as Y, where the output result Y has and only has four results: ; S3, define the weight matrix W2 from the competition layer to the output layer as: S4, calculate the predicted output according to the calculation result of the winning neuron: ; S5, weight adjustment, only the winning neuron adjusts the weight vector, and the remaining neurons do not adjust: (1) when the predicted output is consistent with the expected output, the winning neuron corresponding to the weight vector is adjusted as follows: ; (2) When the prediction output is inconsistent with the expected output, the weight vector corresponding to the winning neuron is adjusted as follows: ； Wherein, k is the iteration number, and the initial value is 0; After the trigger prediction and hierarchical warning model is trained, the trigger prediction and hierarchical warning model can effectively predict according to the input data, and the four prediction results correspond to four alarm information, that is, normal smoke exhaust, slight pollution alarm, moderate pollution alarm and serious pollution alarm.

Images