Factory smoke discharge monitoring and analysis method based on multi-modal data

Abstract

The invention discloses a factory smoke discharge monitoring and analysis method based on multi-modal data. The method comprises data acquisition, a target identification model, an AiNa radiation energy detection model, a distance detection model, a radiation energy compensation model, a smoke discharge detection model and a trigger prediction and grading warning model. The invention belongs to the field of environment field monitoring, and solves the problem that an environment monitoring station is not high in accuracy and timeliness when monitoring and analyzing the brought-up pollution condition during factory smoke exhaust. The method has the advantages that a specific target and the temperature thereof can be quickly and accurately predicted; the prediction accuracy of the infrared camera is effectively improved; the condition that accurate detection cannot be carried out due to insufficient received radiation energy when the installation position of the infrared camera is far away from an ignition point is effectively improved; by means of multi-modal data fusion, various early warning signals are given according to the pollution condition of factory smoke exhaust, workers are helped to monitor the atmospheric pollution condition caused by factory smoke exhaust, and the atmospheric pollution treatment work is effectively guided according to the early warning signals.

Description

Technical Field

[0001] The invention belongs to the field of environmental field monitoring, and specifically refers to a factory smoke exhaust monitoring and analysis method based on multimodal data. Background Art

[0002] Factory exhaust contains a large amount of harmful substances such as particulate matter and gaseous pollutants, which not only affect human health, but also damage the ecosystem. For example, inhalable particulate matter (PM10) and fine particulate matter (PM2.5) can cause serious harm to the human respiratory system and cause diseases such as asthma and bronchitis. Sulfur dioxide (SO2) and nitrogen oxides (NOx) are one of the main causes of acid rain and can damage soil, water bodies and ecosystems. If not monitored and controlled, these pollutants will be directly discharged into the atmosphere, resulting in a decline in air quality. Pollutants also excessively interfere with the ecosystem and affect biodiversity.

[0003] By monitoring and analyzing factory smoke exhaust, we can timely grasp the emission of air pollutants, provide early warning for air pollution, and provide a scientific basis for formulating air pollution prevention and control measures.

[0004] The data types for factory smoke exhaust monitoring are mainly the following: concentration, including PM2.5 and PM10 concentrations that reflect the smoke content in factory exhaust, and the concentrations of various harmful gases such as sulfur dioxide SO2, nitrogen oxides NOx, carbon monoxide CO, and hydrogen chloride HCl in factory exhaust; temperature, which reflects the efficiency of the combustion process and the operating status of the equipment; diffusion range, which can be used to calculate the emission of pollutants.

[0005] Factory smoke exhaust can be monitored through sensors or image monitoring. Smoke sensors are usually used to monitor smoke content, or gas sensors are used to monitor harmful gas content, or temperature sensors are used to monitor exhaust temperature. Image monitoring mainly uses high-definition cameras, thermal imagers and other equipment to take real-time photos of factory smoke exhaust ports. By analyzing the captured images, the color, concentration, temperature and other information of the exhaust smoke can be obtained to determine whether the exhaust is normal and whether there are pollution problems. For example, under normal circumstances, factory exhaust smoke may appear white or light gray. If the exhaust smoke changes to abnormal colors such as black or yellow, it may mean that the exhaust smoke contains a large amount of particulate matter or other pollutants. By analyzing the diffusion range of the exhaust smoke in the image, the concentration and emission of the exhaust smoke can be determined.

[0006] The smoke sensor realizes the alarm function by monitoring the smoke concentration. When the smoke concentration reaches a certain level, the smoke sensor automatically sounds an alarm. However, due to the excessive amount of smoke exhaust from the factory, the smoke sensor may cause a false alarm.

[0007] The traditional computer vision-based smoke detection solution is essentially a classification judgment problem. It uses a convolutional neural network to judge the information of the smoke image in the image. This method is relatively accurate in judging color and concentration information, but it is not accurate in judging temperature information.

[0008] The use of infrared cameras can solve the above problems. According to the characteristic that the higher the object's temperature, the more infrared radiation energy it emits, the exhaust temperature can be judged according to the size of the object's radiation energy. However, changes in ambient temperature will affect the prediction accuracy of the infrared camera, and when the installation location of the infrared sensor is far away from the factory chimney, it is often impossible to accurately detect due to insufficient received radiation energy. Summary of the invention

[0009] Technical issues to be solved In order to solve the above problems in the prior art, the present invention provides a factory smoke exhaust monitoring and analysis method based on multimodal data, the purpose of which is to: (1) Solve the problem of false alarms caused by smoke sensors when exhausting smoke in factories; (2) Solve the impact of ambient temperature on the working performance of infrared cameras and improve the prediction accuracy of infrared cameras; (3) Improve the situation where the infrared camera cannot accurately detect the fire due to insufficient radiation energy when it is installed far away from the fire point; (4) Using multimodal data fusion technology of smoke sensors, gas sensors, binocular cameras, infrared cameras and temperature sensors, a variety of early warning signals are given according to the pollution situation of factory exhaust smoke, helping staff to effectively monitor the air pollution caused by factory exhaust smoke and effectively guide air pollution control work based on early warning signals.

[0010] Technical Solution In order to achieve the above-mentioned purpose, a technical solution adopted by the present invention is: a factory smoke exhaust monitoring and analysis method based on multimodal data, including data acquisition, target recognition model, AiNa radiation energy detection model, distance detection model, radiation energy compensation model, smoke exhaust detection model and trigger prediction and graded warning model: Furthermore, the data acquisition includes monocular image collection and preprocessing, infrared image collection and preprocessing, ambient temperature acquisition, smoke concentration acquisition and gas pollutant concentration acquisition, and the data acquisition is performed on the factory exhaust data at the position index x: 1) Monocular image collection and preprocessing: Obtain a monocular image and perform preprocessing operations on the monocular image. The steps of preprocessing the monocular image include increasing image contrast, Gaussian filtering, and image sharpening, thereby obtaining a monocular preprocessed image SP; 2) Infrared image collection and preprocessing: Acquire infrared images and perform preprocessing operations on them. The steps of preprocessing infrared images include increasing image contrast, Gaussian filtering, image sharpening and grayscale conversion, and then obtain infrared preprocessing images IP; 3) Ambient temperature acquisition is responsible for obtaining the ambient temperature AT and outputting the ambient temperature AT; 4) Smoke density acquisition is responsible for obtaining the smoke density SC. When the smoke density SC>500mg / m3, the output of the smoke density acquisition unit is 1, otherwise the output is 0; the output of the smoke density acquisition is recorded as Out_1; 5) Gas pollutant concentration acquisition is responsible for obtaining the concentrations of harmful gases such as sulfur dioxide SO2, nitrogen oxides NOx, carbon monoxide CO, and hydrogen chloride HCl in smoke Out_2, Out_2={gas_i}, i∈[1,4], where gas_i=(a_i-m_i) / a_i, a_i represents the current concentration of the i-th harmful gas, and m_i represents the standard concentration of the i-th gas.

[0011] Preferably, the target recognition model performs deep learning based on the YOLO algorithm to obtain a data set of factory smoke exhaust, and performs transfer learning training based on the data set. The trained target recognition model is used to perform target detection on the monocular preprocessed image SP to obtain the bounding box of the target object and the corresponding confidence level, and the target object is factory smoke exhaust.

[0012] Furthermore, the AiNa radiation energy detection model processes the infrared preprocessed image IP, specifically including feature acquisition, feature extraction, upsampling, channel merging, scale transformation, feature compression and combined convolution. Since the thermal radiation energy of an object is related to the temperature of the object's surface, the final output of the AiNa radiation energy detection model is the detection temperature T corresponding to the maximum radiation energy position of the factory exhaust at position x.

[0013] The mathematical principle of the distance detection model is the binocular distance measurement principle, so the monocular camera for collecting images is required to be placed in parallel with the infrared camera, and the front image is collected synchronously. In addition, the monocular camera and the infrared camera are required to have the same focal length. Only then can the monocular image and the infrared image obtained be applied to the distance detection model. The pixel difference between the monocular image and the infrared image is used to calculate the distance of the target object. The distance detection model obtains the final predicted distance to the factory exhaust at position x. od .

[0014] Preferably, the ambient temperature AT, the final predicted distance od detected by the distance detection model and the detected temperature T acquired by the radiation energy detection model are respectively sent to the radiation energy compensation model, and the radiation energy compensation model corrects the predicted temperature corresponding to the object to obtain the predicted temperature T_C of the target object; the output of the radiation energy compensation model is recorded as Out_3=(T_C-AT) / T_C.

[0015] Furthermore, the smoke exhaust detection model detects the monocular pre-processed image SP after being processed by the target recognition model to obtain the darkest color and diffusion range of the factory exhaust smoke in SP; the output of the smoke exhaust detection model is Out_4={color, area}; , is the color output, the darker the color, the more serious the smoke pollution, where R, G and B represent the values ​​of red, green and blue in RGB respectively; area∈[0,1] is the area ratio of the smoke area in the monocular pre-processed image SP.

[0016] Furthermore, the trigger prediction and graded warning model is a combined neural network, including an input layer, a competition layer and an output layer, the input layer includes 8 neurons, corresponding to the data in the sample vector of factory smoke exhaust at position x composed of Out_1, Out_2, Out_3 and Out_4 respectively; the competition layer includes 8 neurons and is grouped in pairs; the output layer includes 4 neurons, each neuron is only weightedly connected to one group of four groups of neurons in the competition layer; the weight matrix from the input layer to the competition layer is defined as W1, and the weight matrix from the competition layer to the output layer is defined as W2; after the training of the trigger prediction and graded warning model is completed, the trigger prediction and graded warning model can make effective predictions based on the input data, and the four prediction results correspond to four types of alarm information, namely normal smoke exhaust, abnormal alarm (mild pollution alarm), moderate pollution alarm and severe pollution alarm.

[0017] Beneficial Effects (1) The AiNa radiation energy detection model can quickly and accurately predict specific targets and their corresponding temperatures; (2) The radiation energy compensation model can effectively solve the impact of ambient temperature on the working performance of infrared cameras and improve the prediction accuracy of infrared cameras; (3) The radiation energy compensation model can effectively improve the situation where the infrared camera is installed far away from the smoke exhaust location and cannot be accurately detected due to insufficient received radiation energy; (4) The trigger prediction and graded warning model uses multimodal data fusion of smoke sensors, gas sensors, binocular cameras, infrared cameras and temperature sensors to give multiple early warning signals at different stages of pollution, helping staff to effectively prevent air pollution caused by factory exhaust and guide air pollution control work based on early warning signals. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A flowchart of a factory smoke exhaust monitoring and analysis method based on multimodal data proposed by the present invention; Figure 2 This is a flow chart of the detection steps of the AiNa radiation energy detection model proposed by the present invention; Figure 3 The trigger prediction and graded warning model proposed by the present invention; The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention. DETAILED DESCRIPTION

[0019] A factory smoke exhaust monitoring and analysis method based on multimodal data, such as Figure 1 As shown, it includes data acquisition, target recognition model, AiNa radiation energy detection model, distance detection model, radiation energy compensation model, smoke exhaust detection model and trigger prediction and graded warning model.

[0020] Data acquisition includes monocular image collection and preprocessing, infrared image collection and preprocessing, ambient temperature acquisition, smoke concentration acquisition, and gas pollutant concentration acquisition. The data acquisition is all performed on the factory exhaust data at the position index x: 1) Monocular image collection and preprocessing: Obtain a monocular image and perform preprocessing operations on the monocular image. The steps of preprocessing the monocular image include increasing image contrast, Gaussian filtering, and image sharpening, thereby obtaining a monocular preprocessed image SP; 2) Infrared image collection and preprocessing: Acquire infrared images and perform preprocessing operations on them. The steps of preprocessing infrared images include increasing image contrast, Gaussian filtering, image sharpening and grayscale conversion, and then obtain infrared preprocessing images IP; 3) Ambient temperature acquisition is responsible for obtaining the ambient temperature AT and outputting the ambient temperature AT; 4) Smoke density acquisition is responsible for obtaining the smoke density SC. When the smoke density SC>500mg / m3, the output of the smoke density acquisition unit is 1, otherwise the output is 0; the output of the smoke density acquisition is recorded as Out_1; 5) Gas pollutant concentration acquisition is responsible for obtaining the concentrations of harmful gases such as sulfur dioxide SO2, nitrogen oxides NOx, carbon monoxide CO, and hydrogen chloride HCl in smoke Out_2, Out_2={gas_i}, i∈[1,4], where gas_i=(a_i-m_i) / a_i, a_i represents the current concentration of the i-th harmful gas, and m_i represents the standard concentration of the i-th gas.

[0021] The target recognition model performs deep learning based on the YOLO algorithm to obtain a data set of factory exhaust smoke, and performs transfer learning training based on the data set. The trained target recognition model is used to perform target detection on the monocular preprocessed image SP, and the bounding box and confidence of the target object can be obtained. The target object is factory exhaust smoke.

[0022] The AiNa radiation energy detection model includes feature acquisition, feature extraction, upsampling, channel merging, scale transformation, feature compression and combined convolution. Since the thermal radiation energy of an object is related to the temperature of the object's surface, the final output of the AiNa radiation energy detection model is the detection temperature T corresponding to the maximum radiation energy position of the factory exhaust smoke at position x. To achieve this goal, its features include feature acquisition, feature extraction, upsampling, channel merging, scale transformation, feature compression and combined convolution. The detection steps of the AiNa radiation energy detection model are as follows: Figure 2 As shown: (1) First, the features of the infrared preprocessed image IP are collected to obtain a feature image P1 with a scale of 20×20×256; (2) The feature image P1 is subjected to a feature extraction operation to further extract features in the image, and a feature image P2 with a scale of 10×10×256 is obtained; (3) Upsample the feature image P2 and merge the channels with the feature image P1 to obtain the feature image P3. The scale of the feature image P3 is 20×20×512. (4) Perform a scale transformation operation on the feature image P3 to obtain a feature vector V1, whose scale is 1×1×200; (5) Perform feature compression operation on the feature image P2 to obtain the feature vector V2, whose scale is 1×1×200; (6) Perform channel merging operation on feature vector V1 and feature vector V2 to obtain feature vector V3, and the scale of feature vector V3 is 1×1×400; (7) The feature vector V3 is connected to the output neuron after passing through the intermediate hidden layer 1 and the intermediate hidden layer 2. The intermediate hidden layer 1 contains 100 neurons, the intermediate hidden layer 2 contains 20 neurons, and the output neuron contains only one neuron. The output value of the output neuron is the detected temperature T.

[0023] In the process of training the AiNa radiation energy detection model, the loss function uses the square loss function, and the formula is as follows: ; In the above formula, y i Represents the actual temperature of the target object, z i Represents the detected temperature of the target object, and n represents the number of data in a batch.

[0024] Distance detection model, the mathematical principle of the distance detection model is the binocular distance measurement principle, so the monocular camera and infrared camera for collecting images are required to be placed in parallel, and the front image is collected synchronously, and the monocular camera and infrared camera are required to have the same focal length. At this time, the monocular image and infrared image obtained can be applied to the distance detection model, and the pixel difference between the monocular image and the infrared image is used to calculate the distance of the target object; the distance calculation formula is: ; In the above formula, D-(r - l) is the parallax, (r - l) is the annular difference, D is the baseline distance between the monocular camera and the infrared camera, ƒ is the focal length of the monocular camera, and is also the focal length of the infrared camera. od is the vertical distance from a target pixel to the baseline. The traditional distance detection model needs to perform feature matching on each detected feature point, but not every feature point is useful information that we care about. Therefore, the present invention proposes a target-based feature point detection and matching method, which can further reduce the amount of calculation and obtain more accurate calculation results. The specific steps are: S1, target detection, target detection is performed on monocular images and binocular images respectively; S2, target-based feature point detection; S3, target-based feature point matching; S4, target-based feature point distance prediction; S5. Calculate the average value and get the final predicted distance od .

[0025] The ambient temperature AT, the final predicted distance od detected by the distance detection model and the detected temperature T obtained by the radiation energy detection model are respectively sent to the radiation energy compensation model. The radiation energy compensation model corrects the predicted temperature corresponding to the object to obtain the predicted temperature T_C of the target object; the output of the radiation energy compensation model is recorded as Out_3=(T_C-AT) / T_C.

[0026] The smoke exhaust detection model detects the monocular preprocessed image SP after being processed by the target recognition model, and obtains the darkest color and diffusion range of the factory exhaust smoke in SP; the output of the smoke exhaust detection model is Out_4={color, area}; color=(R+G+B) / (256*3), which is the output for color, the darker the color, the more serious the smoke exhaust pollution, where R, G and B represent the values ​​corresponding to red, green and blue in RGB corresponding to the obtained color respectively; area∈[0,1] is the area ratio of the smoke area in the monocular preprocessed image SP.

[0027] The trigger prediction and classification warning model is a combined neural network, such as Figure 3 As shown, it includes an input layer, a competition layer and an output layer. The input layer includes 8 neurons, which correspond to the data in the sample vector A of the factory exhaust smoke at the position x composed of Out_1, Out_2, Out_3 and Out_4 respectively; the competition layer includes 8 neurons and two neurons are grouped together; the output layer includes 4 neurons, each neuron is only weightedly connected to one group of the four groups of neurons in the competition layer; the weight matrix from the input layer to the competition layer is defined as W1, and the weight matrix from the competition layer to the output layer is defined as W2; the specific algorithm steps are as follows: S0, data preparation, in the form of: ; In the above formula, A represents the sample vector composed of Out_1, Out_2 and Out_3, and d1-d4 represent the expected output modes, which are: ; The meanings of the above four formulas represent normal smoke exhaust, abnormal alarm (light pollution alarm), moderate pollution alarm and severe pollution alarm, a total of four mode categories; S1, for each weight vector W1 in the weight matrix W1 from the input layer to the competition layer j , where j∈[1,8] is initialized to a small value near 0, and the initial learning rate α(t) is given as a function that decreases monotonically over time: ; In the above formula, C is a decimal between 0 and 1, and B is a constant greater than 1. t m is a time constant greater than 1; S2. Find the winning neuron based on the input sample vector A and the weight vector W1: ; When the winning neuron is found, the output of the winning neuron is 1, the output of the neurons in the same group as the winning neuron is also 1, and the output of all other neurons is 0; the output result is recorded as Y, where the output result Y has only four results: ; S3. Define the weight matrix W2 from the competition layer to the output layer as: ; S4. Calculate the predicted output based on the calculation result of the winning neuron: ; S5, weight adjustment, only the winning neuron adjusts the weight vector, and the other neurons do not adjust: (1) When the predicted output is consistent with the expected output, the weight vector corresponding to the winning neuron is adjusted as follows: ; (2) When the predicted output is inconsistent with the expected output, the weight vector corresponding to the winning neuron is adjusted as follows: ; Where k is the number of iterations and its initial value is 0.

[0028] After the trigger prediction and graded warning model is trained, it can make effective predictions based on the input data. The four prediction results correspond to four types of alarm information, namely, normal smoke exhaust, abnormal alarm (mild pollution alarm), moderate pollution alarm and severe pollution alarm.

[0029] The above is the specific working process of the present invention, and you can repeat this step next time you use it.

[0030] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0031] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

[0032] The present invention and its embodiments are described above, which is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if ordinary technicians in the field are inspired by it and design structural methods and embodiments similar to the technical solution without creativity without departing from the purpose of the invention, they should all fall within the protection scope of the present invention.

Claims

1. A factory smoke exhaust monitoring and analysis method based on multimodal data, characterized in that: Including data acquisition, target recognition model, AiNa radiation energy detection model, distance detection model, radiation energy compensation model, smoke exhaust detection model and trigger prediction and graded warning model; The data acquisition includes monocular image collection and preprocessing, infrared image collection and preprocessing, ambient temperature acquisition, smoke concentration acquisition and gas pollutant concentration acquisition. The data acquisition is performed on the factory exhaust data at the position index x: 1) Monocular image collection and preprocessing: Obtain a monocular image and perform preprocessing operations on the monocular image. The steps of preprocessing the monocular image include increasing image contrast, Gaussian filtering, and image sharpening, thereby obtaining a monocular preprocessed image SP; 2) Infrared image collection and preprocessing: Acquire infrared images and perform preprocessing operations on them. The steps of preprocessing infrared images include increasing image contrast, Gaussian filtering, image sharpening and grayscale conversion, and then obtain infrared preprocessing images IP; 3) Ambient temperature acquisition is responsible for obtaining the ambient temperature AT and outputting the ambient temperature AT; 4) Smoke density acquisition is responsible for acquiring the smoke density SC. When the smoke density SC> the preset value, the output of the smoke density acquisition unit is 1, otherwise the output is 0; the output of the smoke density acquisition is recorded as Out_1; 5) Gas pollutant concentration acquisition is responsible for obtaining the concentration Out_2 of harmful gases in smoke, such as sulfur dioxide SO2, nitrogen oxides NOx, carbon monoxide CO, and hydrogen chloride HCl; The target recognition model is trained by transfer learning based on the factory exhaust data set. The trained target recognition model is used to detect the monocular pre-processed image SP, and the bounding box and corresponding confidence of the target object can be obtained. The target object is the factory exhaust. The AiNa radiation energy detection model includes feature acquisition, feature extraction, upsampling, channel merging, scale transformation, feature compression and combined convolution. Since the thermal radiation energy of an object is related to the temperature of the object surface, the final output of the AiNa radiation energy detection model is the detection temperature T corresponding to the maximum radiation energy position of the factory exhaust smoke at position x; The distance detection model calculates the distance of the target object using the pixel difference between the monocular image and the infrared image collected by the monocular camera, and obtains the final predicted distance od from the factory smoke exhaust at position x; the monocular camera and the infrared camera have the same focal length and are placed in parallel to synchronously collect the front image; The ambient temperature AT, the final predicted distance od detected by the distance detection model, and the detected temperature T obtained by the radiation energy detection model are respectively sent to the radiation energy compensation model. The radiation energy compensation model corrects the predicted temperature corresponding to the object to obtain the predicted temperature T_C of the target object; the output of the radiation energy compensation model is recorded as Out_3=(T_C-AT) / T_C; The smoke exhaust detection model detects the monocular pre-processed image SP after being processed by the target recognition model, and obtains the darkest color and diffusion range of the factory exhaust smoke in SP; the output of the smoke exhaust detection model is Out_4={color, area}; color is the output for color, the darker the color, the more serious the smoke pollution; area∈[0,1] is the area ratio of the smoke area in the monocular pre-processed image SP; The trigger prediction and graded warning model is a combined neural network, including an input layer, a competition layer and an output layer, the input layer includes 8 neurons, which correspond to the data in the sample vector of factory smoke exhaust at position x composed of Out_1, Out_2, Out_3 and Out_4 respectively; the competition layer includes 8 neurons and is grouped in pairs; the output layer includes 4 neurons, each neuron is only weightedly connected to one group of four groups of neurons in the competition layer; the weight matrix from the input layer to the competition layer is defined as W1, and the weight matrix from the competition layer to the output layer is defined as W2; after the training of the trigger prediction and graded warning model is completed, the trigger prediction and graded warning model can make effective predictions based on the input data, and the four prediction results correspond to four types of alarm information, namely normal smoke exhaust, slight pollution alarm, medium pollution alarm and severe pollution alarm.

2. The factory smoke exhaust monitoring and analysis method based on multimodal data according to claim 1 is characterized by: Monocular Image Collection and Preprocessing The steps of preprocessing the monocular image include increasing image contrast, Gaussian filtering and image sharpening, thereby obtaining a monocular preprocessed image SP.

3. The factory smoke exhaust monitoring and analysis method based on multimodal data according to claim 1 is characterized in that: Infrared image collection and preprocessing The steps of preprocessing infrared images include increasing image contrast, Gaussian filtering, image sharpening and grayscale, and then obtaining an infrared preprocessed image IP.

4. The factory smoke exhaust monitoring and analysis method based on multimodal data according to claim 1 is characterized in that: The preset value of smoke concentration is 500mg / m3.

5. The factory smoke exhaust monitoring and analysis method based on multimodal data according to claim 1 is characterized by: Out_2={gas_i}, i∈[1,4], where gas_i=(a_i-m_i) / a_i, a_i represents the current concentration of the i-th harmful gas, and m_i represents the standard concentration of the i-th gas.

6. The factory smoke exhaust monitoring and analysis method based on multimodal data according to claim 1 is characterized by: The target recognition model performs deep learning based on the YOLO algorithm to obtain a data set of factory smoke exhaust.

7. The factory smoke exhaust monitoring and analysis method based on multimodal data according to claim 1 is characterized by: The detection steps of the AiNa radiation energy detection model are as follows: (1) First, the features of the infrared preprocessed image IP are collected to obtain a feature image P1 with a scale of 20×20×256; (2) The feature image P1 is subjected to a feature extraction operation to further extract features in the image, and a feature image P2 with a scale of 10×10×256 is obtained; (3) Upsample the feature image P2 and merge the channels with the feature image P1 to obtain the feature image P3. The scale of the feature image P3 is 20×20×512. (4) Perform a scale transformation operation on the feature image P3 to obtain a feature vector V1, whose scale is 1×1×200; (5) Perform feature compression operation on the feature image P2 to obtain the feature vector V2, whose scale is 1×1×200; (6) Perform channel merging operation on feature vector V1 and feature vector V2 to obtain feature vector V3, and the scale of feature vector V3 is 1×1×400; (7) The feature vector V3 is connected to the output neuron after passing through the intermediate hidden layer 1 and the intermediate hidden layer 2. The intermediate hidden layer 1 contains 100 neurons, the intermediate hidden layer 2 contains 20 neurons, and the output neuron contains only one neuron. The output value of the output neuron is the detected temperature T.

8. The factory smoke exhaust monitoring and analysis method based on multimodal data according to claim 1 is characterized by: During the training of the AiNa radiation energy detection model, the loss function uses a square loss function, and the formula is as follows: ； In the above formula, y i Represents the actual temperature of the target object, z i Represents the detected temperature of the target object, and n represents the number of data in a batch.

9. The factory smoke exhaust monitoring and analysis method based on multimodal data according to claim 1 is characterized by: The distance calculation formula of the distance detection model is: ； In the above formula, D-(r - l) is the parallax, (r - l) is the annihilation error, D is the baseline distance between the monocular camera and the infrared camera, f is the focal length of the monocular camera, which is also the focal length of the infrared camera, and od is the vertical distance from a target pixel to the baseline.

10. The factory smoke exhaust monitoring and analysis method based on multimodal data according to claim 1, characterized in that: , where R, G, and B represent the values ​​of red, green, and blue in RGB, respectively.

11. The factory smoke exhaust monitoring and analysis method based on multimodal data according to claim 1 is characterized in that: The specific steps of the trigger prediction and graded warning model are: S0, data preparation, in the form of: ； In the above formula, A represents the sample vector composed of Out_1, Out_2 and Out_3, and d1-d4 represent the expected output modes, which are: ； The meanings of the above four formulas represent normal smoke exhaust, abnormal alarm (light pollution alarm), moderate pollution alarm and severe pollution alarm, a total of four mode categories; S1, for each weight vector W1 in the weight matrix W1 from the input layer to the competition layer j , where j∈[1,8] is initialized to a small value near 0, and the initial learning rate α(t) is given as a function that decreases monotonically over time: ； In the above formula, C is a decimal between 0 and 1, and B is a constant greater than 1. t m is a time constant greater than 1; S2. Find the winning neuron based on the input sample vector A and the weight vector W1: ； When the winning neuron is found, the output of the winning neuron is 1, the output of the neurons in the same group as the winning neuron is also 1, and the output of all other neurons is 0; the output result is recorded as Y, where the output result Y has only four results: ； S3. Define the weight matrix W2 from the competition layer to the output layer as: ； S4. Calculate the predicted output based on the calculation result of the winning neuron: ； S5, weight adjustment, only the winning neuron adjusts the weight vector, and the other neurons do not adjust: (1) When the predicted output is consistent with the expected output, the weight vector corresponding to the winning neuron is adjusted as follows: ； (2) When the predicted output is inconsistent with the expected output, the weight vector corresponding to the winning neuron is adjusted as follows: ； Among them, k is the number of iterations, and the initial value is 0; After the trigger prediction and graded warning model is trained, the trigger prediction and graded warning model can make effective predictions based on the input data. The four prediction results correspond to four types of alarm information, namely normal smoke exhaust, slight pollution alarm, moderate pollution alarm and severe pollution alarm.

Images