Garbage incineration working condition identification method based on multi-modal fusion and large language model

Abstract

The invention discloses a waste incineration working condition identification method based on multi-modal fusion and a large language model. The method comprises the following steps: step 1, constructing a waste incineration working condition knowledge base; 2, waste incineration working condition data are collected and preprocessed; 3, working condition state knowledge corresponding to the current waste incineration working condition data is obtained through retrieval in the waste incineration knowledge base; and 4, identifying and analyzing the working condition state knowledge by using a large language model, and feeding back an identification result. According to the scheme, the waste incineration working condition can be effectively and accurately recognized, and the working condition recognition result is fed back.

Description

Technical Field

[0001] The present invention relates to the field of waste incineration, and particularly to a method for identifying waste incineration conditions based on multimodal fusion and large language models. Background Art

[0002] As an important way to dispose of municipal solid waste, waste incineration has been widely used globally because it can effectively reduce the volume of waste, achieve harmless treatment, and recover heat energy.

[0003] During the waste incineration process, the accurate identification of operating conditions is crucial for ensuring incineration efficiency, reducing pollutant emissions, and ensuring the safe operation of equipment. Identifying waste incineration conditions involves real-time monitoring and analysis of various complex factors such as temperature, pressure, oxygen concentration, and waste composition in the incinerator. These factors are interrelated and dynamically changing, making the identification of operating conditions a very challenging task.

[0004] With the development of artificial intelligence technology, methods based on machine learning and deep learning have gradually been introduced into waste incineration condition identification. However, these methods still face many challenges: on the one hand, waste incineration data has highly complex and non-linear characteristics, and traditional machine learning models are difficult to effectively process it; on the other hand, deep learning models require a large amount of labeled data for training, while the labeling cost of waste incineration data is high and the distribution is uneven, which easily leads to overfitting of the model, thereby affecting the generalization ability of the model.

[0005] In addition, most existing methods lack effective utilization of historical data and knowledge, and cannot fully utilize the value of data, thus limiting the accuracy and reliability of waste incineration condition identification. Summary of the Invention

[0006] The purpose of the present invention is to overcome the deficiencies of the prior art and provide a method for identifying waste incineration conditions based on multimodal fusion and large language models, which can effectively and accurately identify waste incineration conditions.

[0007] To achieve the above purpose, the technical solution adopted by the present invention is: a method for identifying waste incineration conditions based on multimodal fusion and large language models, including the following steps:

[0008] Step 1: Construct a knowledge base for waste incineration conditions;

[0009] Step 2: Collect and preprocess waste incineration condition data;

[0010] Step 3: Retrieve knowledge similar to the waste incineration condition description in the waste incineration knowledge base;

[0011] Step 4: Use a large language model to identify the working conditions and generate an identification result.

[0012] Step 4 includes:

[0013] Construct the retrieval content for waste incineration condition recognition. Based on the constructed retrieval content for waste incineration condition recognition, utilize the language understanding ability of the large language model to generate condition status, descriptions, and suggestion information and output them as results.

[0014] The construction of the waste incineration condition knowledge base in Step 1 includes:

[0015] Obtain unstructured data related to waste incineration conditions, including: obtaining unstructured data corresponding to national environmental protection regulations, operation rules and regulations, and process expert experience through various channels.

[0016] Extract text data from the unstructured data, and perform corresponding text data cleaning to form M text sets; then, divide the M text sets into N text sets according to a length of n; then, vectorize and encode the N text sets using the Embedding model; finally, store the vectorized encoding using a vector database to form a waste incineration knowledge base.

[0017] The collection and preprocessing of waste incineration condition data in Step 2 include:

[0018] Use a fire-viewing camera to collect flame video data inside the furnace, and use a distributed control system to collect time-series data during the incineration process;

[0019] Perform frame extraction on the video image to obtain flame images, and extract feature data from the flame images to obtain flame feature data;

[0020] Preprocess the collected time-series data to obtain preprocessed time-series data;

[0021] Use the flame feature data and the preprocessed time-series data together as waste incineration condition data.

[0022] In Step 2, perform frame extraction on the flame video data at fixed time intervals to obtain flame images, and use machine learning to extract feature data, including: effective flame area, average gray level of the effective flame region, area of the high-temperature flame region, average gray level of the high-temperature region, flame area ratio, flame centroid, variance of the flame area ratio per unit time;

[0023] Use the OPC protocol of the distributed control system to obtain sensor time-series data during waste incineration at fixed time intervals, and preprocess the time-series data to ensure data quality and consistency.

[0024] Step 3 also includes

[0025] Vector-encode the waste incineration condition description information using an Embedding semantic encoding model, then perform similarity matching and retrieval in the waste incineration knowledge base to obtain K condition knowledge related to this condition, and select the top N most relevant condition knowledge from them.

[0026] Step four also includes:

[0027] Set the prompt words of the large language model, which are used to limit the environment where the current condition data is located and the answer range of the large language model; use the top N most relevant condition knowledge obtained as the context information of the large language model, and send the condition recognition retrieval content as the question content of the large language model to the large language model, and the large language model returns the recognition result after understanding.

[0028] Based on the input information, the large language model uses its understanding ability to generate the final condition status, description and suggestions, and returns the final result in JSON format.

[0029] The advantages of the present invention are as follows: By integrating the furnace flame video data, the time-series data such as the furnace temperature, grate speed, and oxygen content in the flue gas of the distributed control system, as well as the unstructured data such as the process expert experience knowledge, boiler operation specifications, and national environmental protection regulations, the dynamic recognition of the waste incineration condition is realized through the retrieval-enhanced generation technology, and at the same time, the current condition description and operation suggestions are provided. On the one hand, compared with the machine learning method, this method combines the flame image recognition result with the process expert experience, significantly improving the generalization ability of the condition recognition model; on the other hand, compared with the deep learning method, this method can obtain high condition recognition accuracy and robustness without a large amount of labeled data for training. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The following briefly describes the content expressed in each drawing of the present invention specification and the marks in the drawings:

[0031] Figure 1 It is the flowchart for constructing the waste incineration knowledge base of the present invention;

[0032] Figure 2 It is the flowchart for condition recognition of multi-modal fusion and large language model of the present invention;

[0033] Figure 3 It is the schematic diagram of the waste incineration boiler of the present invention;

[0034] Figure 4 It is the schematic diagram of the working principle of waste incineration condition recognition of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0035] The following further details the specific embodiments of the present invention by describing the optimal embodiments with reference to the drawings.

[0036] The present invention proposes a method for identifying the operating conditions of waste incineration based on multimodal fusion and large language models. This method integrates the furnace flame video data, the time-series data such as the furnace temperature, grate speed, and flue gas oxygen content in the distributed control system, as well as the unstructured data such as the experience knowledge of process experts, the boiler operation specifications, and the national environmental protection laws and regulations. Through the retrieval-augmented generation technology of the large language model, it realizes the accurate identification and real-time monitoring of the waste incineration operating conditions, solves the deficiencies existing in the existing machine learning and deep learning in the process of waste incineration operating conditions, and effectively improves the accuracy and robustness of the waste incineration operating conditions.

[0037] As Figures 1-4 shown, the method for identifying the operating conditions of waste incineration based on multimodal fusion and large language models provided in this embodiment includes the following steps:

[0038] Step 1: Construct a knowledge base for waste incineration operating conditions

[0039] 1. Obtain unstructured data related to waste incineration operating conditions: Through various forms such as the Internet, intranet, and on-site interviews, obtain unstructured data such as national environmental protection regulations, operation rules and regulations, and the experience of process experts. The unstructured data includes but is not limited to forms such as Word, PDF, PPT, and web pages.

[0040] 2. Construct a waste incineration knowledge database: First, extract text data from the unstructured data and perform corresponding text data cleaning to form M text sets; then, divide the M text sets into N text sets according to a length of n (by default 512 characters); then, vectorize and encode the N text sets using an Embedding model; finally, store the vectorized encoding using a vector database (such as Chroma, Faiss, Postgres, etc.).

[0041] Through the above steps, a waste incineration knowledge base is finally formed.

[0042] Step 2: Collect and preprocess waste incineration operating condition data

[0043] 1. Extract frames from the fire-viewing camera: Use the RTSP protocol of the furnace fire-viewing camera to extract 1 frame of flame image at a fixed interval time t1 (by default 3 seconds).

[0044] 2. Feature extraction of the flame: Use machine learning methods to extract features such as the effective area of the flame, the average gray level of the effective area of the flame, the area of the high-temperature area of the flame, the average gray level of the high-temperature area, the flame area ratio, the centroid of the flame, and the variance of the flame area ratio per unit time for quantitative analysis, and then qualitatively judge the current flame features, such as the occurrence of operating conditions such as left-side burning, right-side burning, and low fire line.

[0045] 3. Data acquisition of distributed control system: Using the OPC protocol of the distributed control system, obtain the sequential data of sensors such as furnace temperature, grate speed, and oxygen content in flue gas at a fixed interval time t2 (default is 3 seconds).

[0046] 4. Preprocess the acquired sequential data: Process the data acquired from the distributed control system (DCS), including outlier detection and handling, data standardization, etc., to ensure the quality and consistency of the data.

[0047] Through the above steps, complete the information of the operating condition parameters for one-time waste incineration operating condition identification.

[0048] Step 3: Retrieve relevant operating conditions in the waste incineration knowledge base

[0049] 1. Fill in the retrieval template for operating condition identification: Construct the operating condition description information according to the template of "The current operating condition information is as follows: {Name 1} is {Parameter 1}; {Name 2} is {Parameter 2};..." with the qualitative features of the extracted flame images and the quantitative data such as furnace temperature, grate speed, and oxygen content in flue gas obtained from the distributed control system (DCS). Here, the name is the specific name of the characteristic parameter, and the parameter refers to the specific parameter value.

[0050] 2. Vector encode the content of operating condition retrieval: Use the Embedding semantic vector model to vector encode the operating condition description information and convert it into a computable vector form for similarity matching in the knowledge base.

[0051] 3. Obtain K relevant operating condition knowledge: Use the text similarity algorithm to match and retrieve the encoded operating condition description information in the waste incineration knowledge base to obtain K (default is 15) pieces of operating condition knowledge related to the retrieved operating condition description information.

[0052] The text similarity calculation formula is as follows:

[0053]

[0054] Among them, v1 represents the vector of the operating condition description information and v2 represents the vector representation of the knowledge in the incineration operating condition knowledge base.

[0055] 4. Select the top N most relevant operating condition knowledge: Use the Rerank semantic ranking model to rank the relevance of the K pieces of incineration operating condition knowledge and select the top N (default is 5) most relevant operating condition description information.

[0056] Through the above steps, obtain N pieces of operating condition knowledge most relevant to the retrieved operating condition.

[0057] Step 4: Construct the retrieval content for waste incineration operating condition identification

[0058] 1. Setting and using prompt words: Setting and using the following prompt words: "You are an operator of a waste - to - energy control system. Use the following context to answer whether the current working condition is normal and give reasons. If you don't know the answer or the context doesn't have the answer, just say you don't know and don't try to fabricate an answer." can significantly improve the understanding ability of the large - language model for waste incineration working condition recognition and avoid the large - language model from fabricating answers. At the same time, specify to return the waste incineration working condition status in JSON format. The final prompt word template is as follows:

[0059] You are an operator of a waste - to - energy control system. Use the following context to answer whether the current working condition is normal and give reasons. If you don't know the answer or the context doesn't have the answer, just say you don't know and don't try to fabricate an answer.

[0060] {context}

[0061] Question: {question}

[0062] --------------

[0063] Answer in JSON format:

[0064] {

[0065] "Status":..., / / The status of the working condition ("normal" or "abnormal")

[0066] "Explanation":..., / / The reason for abnormality, as detailed as possible

[0067] "Suggestion":..., / / Suggestions for operation, giving operation steps

[0068] }

[0069] 2. Filling the working condition recognition retrieval template: Fill the top N most relevant working condition knowledge obtained into {context} as the context information of the large - language model. Fill the working condition recognition retrieval content into {question} as the question content of the large - language model. context is the top N most relevant working condition knowledge, and question is the working condition parameter data collected by sensors under the current disclosure, that is, the corresponding information parameters in the working condition recognition retrieval template.

[0070] Through the above steps, the complete waste incineration working condition recognition retrieval content is constructed.

[0071] Step Five: Generating the current waste incineration working condition recognition result

[0072] 1. Using the large - language model to recognize the working condition status: Send the filled - in prompt words to the large - language model.

[0073] 2. The large language model generates operating condition information: Utilize the understanding ability of the large language model to generate the final operating condition, description, and suggestions, and return the final result in JSON format as follows:

[0074] {

[0075] "Operating Condition": "Abnormal",

[0076] "Description": "The oxygen content in the current furnace is 8%, which is higher than the recommended standard range of 6% - 10%. Especially for the oxygen content at the economizer outlet under stable combustion conditions, it should be maintained between 3% - 5%. This indicates an excess of air, which may lead to a decrease in furnace temperature, a reduction in boiler load, and incomplete combustion of the garbage.",

[0077] "Suggestion": "It is necessary to adjust the damper opening to optimize the combustion condition.",

[0078] }

[0079] Through the above steps, a complete waste incineration operating condition identification process is completed. When the current operating condition is identified as "abnormal", the cause of the abnormality and relevant operation suggestions can be given.

[0080] 6. Advantages and Positive Effects

[0081] A waste incineration operating condition identification method based on multi-modal fusion and large language model proposed by the present invention has significant beneficial effects compared with the prior art, which are mainly reflected in the following aspects:

[0082] Improve the accuracy of operating condition identification: This method fuses multi-modal sensor data such as flame videos, furnace temperature, and grate speed to comprehensively and accurately capture various characteristic information in the waste incineration process. The multi-modal data fusion effectively makes up for the limitations of a single data source, enabling the model to accurately identify complex incineration operating conditions and significantly improving the accuracy of operating condition identification. Under complex scenarios such as different types of garbage, changes in incinerator structure, or adjustment of operating parameters, it still maintains high accuracy and robustness.

[0083] Enhance the generalization ability of the model: This method combines the flame video recognition results, distributed control system sensor parameters, and the experience of waste incineration experts, and uses retrieval-enhanced generation technology to dynamically adapt to changes in waste incineration operating conditions, improve the adaptability of the model to different operating conditions, and enhance the generalization ability. Facing unseen operating conditions, the model can accurately identify based on expert experience and existing data patterns, effectively solving the deficiencies of existing machine learning and deep learning methods in generalization ability.

[0084] Reduce the cost of data annotation: Compared with traditional deep learning methods, this method does not require a large amount of annotated data for training. By leveraging expert experience and retrieved relevant information for learning optimization, it can achieve a high accuracy in operating condition recognition without relying on large-scale annotated data, significantly reducing the manpower, time, and cost investment in data annotation.

[0085] Provide real-time dynamic recognition and operation suggestions: This method can dynamically recognize the waste incineration operating conditions in real time and provide a detailed description of the current conditions and specific operation suggestions based on expert experience. On-site operators can promptly understand the operating status of the incinerator, precisely adjust operations according to the suggestions, optimize the incineration process, improve waste incineration efficiency, and reduce pollutant emissions.

[0086] Obviously, the specific implementation of the present invention is not limited by the above methods. As long as various non-substantive improvements are made using the method concept and technical solution of the present invention, they are all within the protection scope of the present invention.

Claims

1. A method for identifying the operating conditions of waste incineration based on multimodal fusion and large language models, characterized in that: It includes the following steps: Step 1: Construct a knowledge base for waste incineration conditions; Step 2: Collect and preprocess waste incineration condition data; Step 3: Retrieve knowledge similar to the waste incineration condition description in the waste incineration knowledge base; Step 4: Use a large language model to identify the condition state and generate an identification result.

2. The method for identifying the waste incineration working conditions based on multi-modal fusion and large language models according to claim 1, wherein: Step 4 includes: Construct waste incineration condition identification retrieval content, and based on the constructed waste incineration condition identification retrieval content, use the language understanding ability of the large language model to generate information such as condition state, description, operation suggestions, etc. and output it as a result.

3. The method for identifying the waste incineration working conditions based on multi-modal fusion and large language model according to claim 1 or 2, characterized in that: Step 1 of constructing the waste incineration condition knowledge base includes: Obtain unstructured data related to waste incineration conditions, extract text data from the unstructured data, and perform corresponding text data cleaning to form M text sets; then, divide the M text sets into N text sets according to a length of n; then, vectorize and encode the N text sets using an Embedding semantic encoding model; finally, store the vectorized encoding in a vector database to form a waste incineration knowledge base.

4. The method for identifying the waste incineration working condition based on multi-modal fusion and large language model according to claim 3, wherein: Obtaining unstructured data related to waste incineration conditions includes: obtaining unstructured data corresponding to national environmental protection regulations, operation rules and regulations, and process expert experience through various channels.

5. The waste incineration condition identification method based on multimodal fusion and large language model according to claim 1 or 2, characterized in that: The collection and preprocessing of waste incineration condition data in Step 2 includes: Use a fire observation camera to collect flame video data in the furnace, and use a distributed control system DCS to collect time-series data during the incineration process; Perform frame extraction on the video image to obtain a flame image, and perform feature extraction on the flame image to obtain flame feature data; Preprocess the collected time-series data to obtain processed time-series data; Use the flame feature data and the processed time-series data together as waste incineration condition description information.

6. The method for identifying the waste incineration working condition based on multimodal fusion and large language model according to claim 5, characterized in that: In Step 2, perform frame extraction on the flame video data at a fixed interval time to obtain a flame image, and use machine learning to extract feature data. The feature data includes: flame effective area, average gray value of the flame effective area, area of the flame high-temperature area, average gray value of the high-temperature area, flame area ratio, flame centroid, variance of the flame area ratio per unit time; Use the OPC protocol of the distributed control system DCS to obtain sensor time-series data during waste incineration at a fixed interval time, and preprocess the time-series data to ensure the quality and consistency of the data.

7. The method for identifying the waste incineration working conditions based on multimodal fusion and large language models according to any one of claims 1-6, characterized in that: Encode the condition description information using an Embedding semantic encoding model, use a Rerank semantic correlation model to perform correlation ranking on the obtained K condition description information, and take the top N most relevant condition knowledge.

8. The method for identifying the waste incineration working conditions based on multimodal fusion and large language model according to any one of claims 1-6, characterized in that: Step 4 also includes: Set the prompt words of the large language model, and the prompt words are used to limit the environment where the current condition data is located and the answer range of the large language model; use the top N most relevant condition knowledge obtained as the context information of the large language model, and send the condition identification retrieval content as the question content of the large language model to the large language model, and the large language model returns an identification result after understanding.

9. The method for identifying the waste incineration working condition based on multimodal fusion and large language model according to claim 8, wherein: Based on the input information, the large language model utilizes its understanding ability to generate the final operating condition status, explanations, and suggestions, and returns the final result in JSON format.

Images