Atmospheric pollution process analysis method, electronic equipment and storage medium

Abstract

The invention provides an atmospheric pollution process analysis method, electronic equipment and a storage medium, and relates to the technical field of data processing, and the method comprises the following steps: receiving an atmospheric pollution problem sent by a user, and respectively carrying out vector database retrieval and knowledge graph retrieval based on the atmospheric pollution problem to obtain a first retrieval result and a second retrieval result; wherein the vector database retrieval comprises the following steps: calculating cosine similarity between background information vectors in a vector database and atmospheric pollution problem vectors, and determining multiple pieces of background information of which the cosine similarity accords with a preset threshold value as a first retrieval result; the knowledge graph retrieval comprises the following steps: performing entity recognition and standardization on the atmospheric pollution problem, and generating a target knowledge triple as a second retrieval result; and fusing and inputting the atmospheric pollution problem, the first retrieval result and the second retrieval result into a large language model, and outputting an analysis result of the atmospheric pollution problem. Through the method provided by the invention, the illusion problem of a general large language model in the atmosphere field is relieved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to an atmospheric pollution process analysis method, electronic equipment and storage medium. Background Art

[0002] Air pollution is a global problem that has a significant impact on human health and environmental safety. Analysis of air pollution processes is an important scientific basis for revealing pollution formation mechanisms, evaluating the contribution of pollution sources, optimizing prevention and control strategies, and promoting air quality improvements and public health protection.

[0003] An expert system is a computer system built based on domain expert knowledge and rules. It simulates the problem-solving ability of experts in a specific field through logical reasoning and decision support. Expert systems based on general large language models provide new possibilities for analyzing atmospheric pollution processes. However, general large language models suffer from serious problems of hallucinations and outdated information when analyzing atmospheric pollution processes, and their reasoning ability is weak, making it difficult to support theoretical research in the field of atmospheric science.

[0004] How to solve the problems of illusion and outdated information of large models in the field of atmospheric science, and improve their reasoning ability and professionalism in solving problems of analyzing and responding to atmospheric pollution processes, is a technical problem that needs to be solved at present. Summary of the Invention

[0005] The present invention provides an atmospheric pollution process analysis method, electronic equipment and storage medium, which are used to solve the defects in the prior art.

[0006] The present invention provides an atmospheric pollution process analysis method, comprising the following steps: Receive an air pollution problem sent by a user, perform a vector database search and a knowledge graph search based on the air pollution problem, and obtain a first search result and a second search result; wherein the vector database search comprises calculating the cosine similarity between the background information vector in the vector database and the air pollution problem vector, and determining multiple pieces of background information whose cosine similarity meets a preset threshold as the first search result; and the knowledge graph search comprises performing entity recognition and standardization on the air pollution problem to generate a target knowledge triple as the second search result; The air pollution problem, the first search result and the second search result are fused and input into a large language model, and an analysis result of the air pollution problem is output.

[0007] According to an air pollution process analysis method provided by the present invention, the method of calculating the cosine similarity between a background information vector and an air pollution problem vector in a vector database, and determining multiple pieces of background information whose cosine similarity meets a preset threshold as first search results, includes: The air pollution problem is converted into the air pollution problem vector through a text embedding model; wherein the text embedding model is: a pre-trained model based on Transformer; Calculating the cosine similarity between the air pollution problem vector and each background information vector in a vector database; wherein the vector database includes: a plurality of background information vectors, each background information vector being a vector representation of domain background information pre-processed by the text embedding model and stored in the vector database; The background information of the target quantity most relevant to the air pollution problem vector is recalled according to the cosine similarity sorting as the first retrieval result.

[0008] According to an air pollution process analysis method provided by the present invention, the entity recognition and standardization of the air pollution problem to generate a target knowledge triple as the second search result includes: An Aho-Corasick automaton is constructed based on a preset atmospheric science classification vocabulary, and multiple entities in the atmospheric pollution problem are extracted using a multi-pattern string search algorithm based on the Aho-Corasick automaton; Based on the preset mapping vocabulary, the extracted same entity is standardized into the same keyword and corrected through the fuzzy matching algorithm to obtain multiple standardized entities; For each of the plurality of standardized entities, identifying attributes or relationships related to a prompt word based on a large language model, and merging them using a regular expression to obtain a target attribute or target relationship related to the prompt word; wherein the prompt word is: a structured input instruction constructed based on the air pollution problem; Based on the standardized entities and the target attributes or target relationships in the standardized entities that are related to the prompt words, the subgraph corresponding to each standardized entity is recalled through a loop algorithm nested with preset statements, and the target knowledge triples are extracted based on the subgraphs corresponding to the standardized entities, and the target knowledge triples are used as the second retrieval results.

[0009] According to an atmospheric pollution process analysis method provided by the present invention, the process of constructing the vector database includes: Acquiring background information in the field of atmospheric science, and performing data cleaning and text segmentation on the background information to obtain preprocessed text data; Performing text vectorization on the preprocessed text data using the text embedding model to generate a vector representation corresponding to the preprocessed text data; The vector representation corresponding to the preprocessed text data is stored in a cloud-native database to construct the vector database.

[0010] According to an atmospheric pollution process analysis method provided by the present invention, the process of constructing the knowledge graph includes: Acquire target data; wherein the target data includes: standardized data, structured data, semi-structured data and unstructured data; Performing text annotation on the target data based on preset entity, relationship and attribute annotation specifications to obtain annotated data; Performing entity extraction, relationship extraction, and attribute distillation based on the labeled data to generate a document in a target format; The target format file is subjected to reference resolution, entity alignment and entity linking to generate knowledge triples, and the knowledge triples are stored in a graph database to construct the knowledge graph.

[0011] According to an air pollution process analysis method provided by the present invention, the air pollution problem, the first search result, and the second search result are integrated into a large language model and input, and the analysis result of the air pollution problem is output, comprising: The air pollution problem, the first search result, and the second search result are integrated into a structured prompt word according to a preset template; wherein the preset template is a prompt word template containing a thought chain structure; The structured prompt words are input into the large language model, and the analysis results of the air pollution problem are output.

[0012] According to a method for analyzing an air pollution process provided by the present invention, after fusing the air pollution problem, the first search result, and the second search result into a large language model and outputting the analysis result of the air pollution problem, the method further comprises: Based on the fidelity, answer relevance, context recall, and context precision indicators, the large language model is automatically evaluated through an automated evaluation framework to obtain an evaluation result; Based on the evaluation result, the preset threshold of the cosine similarity of the vector database and the retrieval depth of the knowledge graph are adjusted.

[0013] The present invention also provides an atmospheric pollution process analysis system, comprising the following modules: A retrieval module is configured to receive an air pollution problem sent by a user, and perform a vector database search and a knowledge graph search based on the air pollution problem, respectively, to obtain a first search result and a second search result; wherein the vector database search comprises calculating the cosine similarity between a background information vector in the vector database and an air pollution problem vector, and determining as the first search result a plurality of background information pieces whose cosine similarity meets a preset threshold; and the knowledge graph search comprises performing entity recognition and standardization on the air pollution problem to generate a target knowledge triple as the second search result; An analysis module is used to integrate the air pollution problem, the first search result and the second search result into a large language model, and output an analysis result of the air pollution problem.

[0014] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the atmospheric pollution process analysis method as described above is implemented.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described atmospheric pollution process analysis methods.

[0016] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned atmospheric pollution process analysis methods.

[0017] The present invention provides an atmospheric pollution process analysis method, electronic device, and storage medium. The method receives an atmospheric pollution problem sent by a user, performs a vector database search and a knowledge graph search based on the atmospheric pollution problem, and obtains a first search result and a second search result. The vector database search is as follows: calculating the cosine similarity between the background information vector and the atmospheric pollution problem vector in the vector database, and determining that multiple pieces of background information whose cosine similarity meets a preset threshold are the first search result. The knowledge graph search is as follows: performing entity recognition and standardization on the atmospheric pollution problem to generate a target knowledge triple as the second search result. The atmospheric pollution problem, the first search result, and the second search result are fused and input into a large language model, and the analysis result of the atmospheric pollution problem is output. It can be seen that the present invention adopts a retrieval enhancement method that fuses knowledge graphs and vector databases to alleviate the hallucination problem of general large language models in the atmospheric field, improve the professionalism and robustness of the large language model's answers in the atmospheric science field, and serve scientific research and information query in the atmospheric science field. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 It is a flow chart of the atmospheric pollution process analysis method provided by the present invention.

[0020] Figure 2 This is a schematic diagram of the architecture of the retrieval enhancement generation model based on the vector database and knowledge graph provided by the present invention.

[0021] Figure 3 This is the overall flow chart for constructing the knowledge graph in the field of atmospheric science provided by the present invention.

[0022] Figure 4 It is a structural schematic diagram of the atmospheric pollution process analysis system provided by the present invention.

[0023] Figure 5 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0024] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0025] The following combination Figure 1-Figure 5 The present invention describes an atmospheric pollution process analysis method, electronic equipment and storage medium.

[0026] It should be noted that retrieval-enhanced generation is a technology that combines advanced language models with real-time web search. It enables the language model to query the latest information through search engines when processing problems. It is of great significance in solving the problems of hallucination and outdated information in general large language models.

[0027] Existing technologies typically rely solely on vector databases for search-enhanced generation, resulting in weak multi-hop reasoning capabilities for large language models. Furthermore, approaches that align graph models with large language models have yet to be applied to air pollution prediction, making current general-purpose large language models incapable of handling complex logical reasoning problems in this field. Therefore, the present invention provides an air pollution process analysis method to address at least one of these issues.

[0028] Figure 1 It is a flow chart of the atmospheric pollution process analysis method provided by the present invention, such as Figure 1 As shown, the method includes the following: Step 100: Receive an air pollution problem sent by a user, and perform a vector database search and a knowledge graph search based on the air pollution problem to obtain a first search result and a second search result; wherein, the vector database search is to calculate the cosine similarity between the background information vector in the vector database and the air pollution problem vector, and determine that multiple pieces of background information whose cosine similarity meets a preset threshold are the first search result; the knowledge graph search is to perform entity recognition and standardization on the air pollution problem, and generate a target knowledge triple as the second search result.

[0029] Figure 2 This is a schematic diagram of the architecture of the retrieval enhancement generation model based on the vector database and knowledge graph provided by the present invention. Figure 2 As shown, this embodiment uses a parallel search method for knowledge graphs and vector databases. After the question is entered, two links of information retrieval are performed. The first link searches through the vector database, converting the prompt word into a 1536-dimensional vector using text-embedding-ada-002. The cosine similarity method is used to search for relevant background information in the vector database and recall it. The second link performs subgraph recall through the knowledge graph. After entity recognition and subgraph recall, the relevant information in the knowledge graph is converted into knowledge triples. The content retrieved from the knowledge graph and vector database, as well as the prompt word itself, are integrated through the prompt word project and input into the large language model to generate the result.

[0030] Specifically, the construction and retrieval process of the vector database are first explained.

[0031] The process of constructing the vector database includes: Step S110 : obtaining background information in the field of atmospheric science, and performing data cleaning and text segmentation on the background information to obtain pre-processed text data.

[0032] Step S120: performing text vectorization on the preprocessed text data through the text embedding model to generate a vector representation corresponding to the preprocessed text data.

[0033] Step S130: Store the vector representation corresponding to the preprocessed text data in a cloud-native database to construct the vector database.

[0034] In one embodiment, after collecting literature and reports in the field of atmospheric science, the text is cleaned and the text is segmented using the RecursiveCharacterTextSplitter method in the Python-based langchain.text_splitter library. The text is vectorized using the text embedding model text-embedding-ada-002 developed by OpenAI, and the cloud-native vector database Pinecone is used for vector storage.

[0035] In step 100, the cosine similarity between the background information vector and the air pollution problem vector in the vector database is calculated, and multiple pieces of background information whose cosine similarity meets a preset threshold are determined as first search results, including: Step 110: Convert the air pollution problem into the air pollution problem vector through a text embedding model; wherein the text embedding model is a pre-trained model based on Transformer.

[0036] Step 120: Calculate the cosine similarity between the air pollution problem vector and each background information vector in the vector database; wherein the vector database includes: multiple background information vectors, and the background information vector is: a vector representation of the domain background information that has been pre-processed by the text embedding model and stored in the vector database.

[0037] Step 130 : Recall the background information of the target quantity most relevant to the air pollution problem vector according to the cosine similarity sorting as the first search result.

[0038] In one embodiment, when a user asks a question to the system, the question is entered into the vector database using the same vectorization model (text-embedding-ada-002). After searching for the top k pieces of information using cosine similarity, the retrieved content is stored in the model system as additional knowledge, which is then used to generate prompt words for the large language model. The vector database remains external to the system, interacting with the model system as a search knowledge base. This entire process does not involve fine-tuning the model itself or adjusting the neural network parameters.

[0039] Furthermore, the construction and retrieval process of the knowledge graph are explained.

[0040] The construction process of the knowledge graph includes: Step S140: Acquire target data; wherein the target data includes: standardized data, structured data, semi-structured data and unstructured data.

[0041] Step S150: perform text annotation on the target data based on preset entity, relationship and attribute annotation specifications to obtain annotated data.

[0042] Step S160: Entity extraction, relationship extraction, and attribute distillation are performed based on the labeled data to generate a file in a target format.

[0043] Step S170: Perform reference resolution, entity alignment, and entity linking on the target format file to generate knowledge triples, and store the knowledge triples in a graph database to construct the knowledge graph.

[0044] In one embodiment, Figure 3 This is the overall flow chart for constructing the knowledge graph in the field of atmospheric science provided by the present invention, such as Figure 3 As shown in the figure, the overall process for constructing a knowledge graph in the field of atmospheric science includes: data acquisition, definition of annotation standards, knowledge modeling and text annotation, knowledge fusion, and knowledge storage and computation. Unstructured data is obtained from literature searches, while structured data is obtained from mind maps of various atmospheric environmental science textbooks, incorporating standardized data from the ConceptNet common sense knowledge base developed by the MIT Media Lab. The annotation standards require the determination of entities, relationships, and attributes based on atmospheric environmental chemical elements during heavy pollution events. Once the standards are determined, the text annotation tool Label-Studio is used for text annotation, followed by entity-relationship extraction. The results are exported as JSON files, subjected to coreference resolution, entity alignment, and entity linking, and then transferred to the graph database Neo4J for data storage. After quality review, the PageRank algorithm is used to evaluate the influence of nodes and determine the overall characteristics of the knowledge graph.

[0045] In step 100, entity recognition and standardization are performed on the air pollution problem to generate target knowledge triples as the second search results, including: Step 140: construct an Aho-Corasick automaton based on a preset atmospheric science field classification vocabulary, and extract multiple entities in the atmospheric pollution problem through a multi-mode string search algorithm based on the Aho-Corasick automaton.

[0046] Step 150 : Based on a preset mapping vocabulary, the extracted identical entities are standardized into identical keywords and corrected through a fuzzy matching algorithm to obtain a plurality of standardized entities.

[0047] Step 160: For each standardized entity among the multiple standardized entities, identify attributes or relationships related to the prompt word based on the large language model, and merge them through regular expressions to obtain target attributes or target relationships related to the prompt word; wherein the prompt word is: a structured input instruction constructed based on the air pollution problem.

[0048] Step 170: Based on the standardized entity and the target attribute or target relationship related to the prompt word in the standardized entity, recall the subgraph corresponding to each standardized entity through a loop algorithm nested preset statement, extract the target knowledge triple based on the subgraph corresponding to the standardized entity, and use the target knowledge triple as the second retrieval result.

[0049] In one embodiment, the entity recognition process is described in detail.

[0050] Entity recognition is achieved by creating the EntityClassifierRevise module. This module contains three core functions: build_actree, standardize_terms, and check_entity. It first reads a pre-created classification vocabulary for atmospheric pollution processes and uses an if statement in Python to create a dict-type domain vocabulary. This vocabulary is then constructed as an Aho-Corasick automaton to achieve accurate entity recall. The algorithm consists of two main phases: automaton construction and search. Automaton construction involves constructing a Trie tree and adding failure pointers. All entities in the atmospheric science field are used to construct a data structure called a Trie (prefix tree). Each node represents a character, and the path from the root node to any node represents a prefix. Failure pointers are added to the base of the Trie. These pointers point to other nodes in the Trie. If a character match fails at a node, the failure pointers allow the match to continue with another possible prefix. After the Trie is constructed, the Aho-Corasick automaton scans the text (i.e., the prompt word), starting from the root node and matching each character. If a matching character is found in the current node, the node branches to the corresponding child node. If a match fails, the node branches along the failure pointer until a match is found or the node returns to the root node. Efficient multi-pattern string searching is achieved through the Aho-Corasick automaton. After the search is complete, a custom check_entity function is used to match the retrieved query keywords to their corresponding labels using dictionary comprehension. This module ultimately returns a dictionary of the form "{keyword:label}" to facilitate entity recall. Because the Cypher statements used in Neo4j interactions have extremely low error tolerance, node names cannot have morphological variations. However, the same entity may have different representations (e.g., sulfur dioxide = SO2). To improve the generalization of entity recognition, this study added a mapping vocabulary to the EntityClassifierRevise function. This allows the searched entity to be normalized to a single keyword (e.g., all synonyms of SO2 are converted to SO2), significantly improving the robustness and generalization of the question-answering system. At the same time, in order to prevent users from having different spellings or parts of speech, fuzzy-wuzzy fuzzy matching is used to solve spelling errors or part of speech changes, thereby improving the fault tolerance of the entity recognition model.

[0051] In another embodiment, the subgraph recall process is described in detail.

[0052] The subgraph recall process is an important step in knowledge graph retrieval following entity recognition. This module is divided into three main parts: identifying entity attributes or relationships related to the prompt word, cypher statement retrieval, and knowledge triple creation. Traditionally, identifying attributes or relationships related to the prompt word often uses conditional statements for multi-classification, which makes the code cumbersome and less readable. Furthermore, if...then statements cannot cover all cases, resulting in rigid subgraph recall and weak generalization capabilities. However, with the improvement of the natural language understanding capabilities of large language models, this project uses multiple calls to the large model to achieve attribute or relationship recognition. Specifically, GPT-3.5-turbo is used to identify attributes or relationships related to the prompt word in the entities identified in the first step, and regular expressions are used to calculate the union of the entity attributes or relationships related to the prompt word. The for loop algorithm nested in the cypher language "MATCH p=(m:{entity_type})-[r*..{depth}]-(n) wherem.name = '{entity_name}' return p" recalls each entity-related subgraph and extracts query-related information into the form of knowledge triples.<subject, relation, object> Incorporate it into the prompt words of the subsequent question and answer module.

[0053] Step 200: The air pollution problem, the first search result, and the second search result are integrated and input into a large language model, and an analysis result of the air pollution problem is output.

[0054] Step 200 specifically includes: Step 210: The air pollution problem, the first search result, and the second search result are integrated into structured prompt words according to a preset template; wherein the preset template is a prompt word template including a thought chain structure.

[0055] Step 220: Input the structured prompt words into the large language model and output the analysis results of the air pollution problem.

[0056] In one embodiment, the information retrieved from the vector database and knowledge graph needs to be integrated with the user's initial instructions through a prompt word project. For this step, this embodiment designs the format_prompt function and the llm_server module. In the format_prompt step, a prompt word containing a "thought chain" is designed based on knowledge related to the prompt word project. The recalled knowledge triples and the prompt word are integrated into the same prompt word, which is then fed into the chat module of the large language model for real-time question and answer. Considering the needs of different users, this embodiment selects two large models, Qwen7B and GPT, as the base. Within the GPT module, you can choose the model type you want to use, including OpenAI open model bases such as gpt-4-turbo, gpt-4, gpt-3.5-turbo, and gpt-3.5-turbo-16k, providing greater flexibility in use. Simultaneously, the GPT memory module is written, in which the previous question and answer results are added to the prompt word for the next round of streaming question and answer to achieve contextual memory.

[0057] The above is a description of the steps of the atmospheric pollution process analysis method provided by the present invention. From the description of the above steps, it can be seen that according to the atmospheric pollution process analysis method provided by the present invention, by receiving the atmospheric pollution problem sent by the user, a vector database search and a knowledge graph search are performed based on the atmospheric pollution problem, respectively, to obtain a first search result and a second search result; wherein, the vector database search is: calculating the cosine similarity between the background information vector and the atmospheric pollution problem vector in the vector database, and determining that the multiple background information whose cosine similarity meets the preset threshold is the first search result; the knowledge graph search is: performing entity recognition and standardization on the atmospheric pollution problem, generating a target knowledge triple as the second search result; the atmospheric pollution problem, the first search result and the second search result are fused and input into the large language model, and the analysis result of the atmospheric pollution problem is output. It can be seen that the present invention adopts a retrieval enhancement method that fuses knowledge graphs and vector databases, which alleviates the hallucination problem of the general large language model in the atmospheric field, improves the professionalism and robustness of the large language model's answers in the field of atmospheric science, and serves scientific research and information query in the field of atmospheric science.

[0058] Based on the above embodiment, in this embodiment, after step 200 integrates the air pollution problem, the first search result, and the second search result into a large language model and outputs the analysis result of the air pollution problem, the method further includes: Step 300: Based on the fidelity, answer relevance, context recall, and context accuracy indicators, the large language model is automatically evaluated through an automated evaluation framework to obtain an evaluation result.

[0059] Step 400: Adjust the preset threshold of the cosine similarity of the vector database and the retrieval depth of the knowledge graph based on the evaluation result.

[0060] Specifically, to verify the effectiveness of retrieval augmented generation, this example uses RAGAS for model testing. RAGAS (Retrieval Augmented Generation Assessment) is an automated framework for evaluating RAG systems using a large model (GPT4). RAGAS evaluates RAGs by considering the retrieval system's ability to identify relevant and important contextual passages, the LLM's ability to faithfully utilize these passages, and the quality of the generation itself.

[0061] In this embodiment, four indicators, namely faithfulness, answer_relevancy, context_recall, and context_precision, are used to evaluate the model system. The calculation formula is as follows:

[0062] Among them, faithfulness measures the factual consistency of the generated answer with the given context, which is calculated based on the answer and the retrieved context. answer_relevancy evaluates the relevance of the generated answer to the given question or task, which is obtained by comparing the question, answer and retrieval information. Egi is the vectorized embedding of the generated question i, Eo is the vectorized embedding of the original question, N is the number of generated questions, and the default is 3. context_recall measures the consistency between the retrieved context and the annotated answer (considered as the basic facts), which is calculated based on the actual situation and the retrieved context. GT is ground_truth, which is the basic facts prepared in advance for the evaluation question. context_precision is a measure of whether all benchmark truth-related items are ranked higher in the context. The calculation formula for Precision@k is:

[0063] Here is the total number of chunks in the context, Is in the ranking The correlation index at .

[0064] The atmospheric pollution process analysis method provided in this embodiment proposes annotation specifications for constructing a knowledge graph in the field of atmospheric science, and constructs the first knowledge graph in the field of atmospheric science for atmospheric pollution prediction and response tasks. In addition, a retrieval enhancement method that integrates knowledge graphs and vector databases is adopted to alleviate the hallucination problem of general large language models in the atmospheric field, improve the professionalism and robustness of the answers given by large language models in the field of atmospheric science, and serve scientific research and information query in the field of atmospheric science. From a technical perspective, the method of entity recognition based on Aho-Corasick automata and mapping vocabulary and the method of related attribute identification based on large language models provide useful references for the retrieval and utilization of knowledge graphs. At the same time, the RAGAS evaluation framework is applied to evaluate the prediction of the large language model after retrieval enhancement generation.

[0065] The following describes the atmospheric pollution process analysis system provided by the present invention. The atmospheric pollution process analysis system described below and the atmospheric pollution process analysis method described above can be referenced to each other.

[0066] Figure 4 This is a schematic diagram of the structure of the atmospheric pollution process analysis system provided by the present invention. Figure 4 As shown, the atmospheric pollution process analysis system provided by the present invention includes: Retrieval module 401 is configured to receive an air pollution problem sent by a user, and perform a vector database search and a knowledge graph search based on the air pollution problem, respectively, to obtain a first search result and a second search result. The vector database search comprises calculating the cosine similarity between a background information vector in the vector database and an air pollution problem vector, and determining as the first search result a plurality of pieces of background information whose cosine similarity meets a preset threshold. The knowledge graph search comprises performing entity recognition and standardization on the air pollution problem to generate a target knowledge triple as the second search result. The analysis module 402 is used to integrate the air pollution problem, the first search result and the second search result into a large language model, and output an analysis result of the air pollution problem.

[0067] The atmospheric pollution process analysis system provided by the present invention receives atmospheric pollution problems sent by users, and performs vector database retrieval and knowledge graph retrieval based on the atmospheric pollution problems, respectively, to obtain a first retrieval result and a second retrieval result; wherein, the vector database retrieval is as follows: calculating the cosine similarity between the background information vector and the atmospheric pollution problem vector in the vector database, and determining that multiple pieces of background information whose cosine similarity meets a preset threshold are the first retrieval result; the knowledge graph retrieval is as follows: performing entity recognition and standardization on the atmospheric pollution problem, generating a target knowledge triple as the second retrieval result; the atmospheric pollution problem, the first retrieval result, and the second retrieval result are fused and input into a large language model, and the analysis result of the atmospheric pollution problem is output. It can be seen from this that the present invention adopts a retrieval enhancement method that fuses knowledge graphs and vector databases, which alleviates the hallucination problem of general large language models in the atmospheric field, improves the professionalism and robustness of the large language model's answers in the atmospheric science field, and serves scientific research and information query in the atmospheric science field.

[0068] Based on the above embodiment, in this embodiment, the retrieval module 401 is specifically used to: The air pollution problem is converted into the air pollution problem vector through a text embedding model; wherein the text embedding model is: a pre-trained model based on Transformer; Calculating the cosine similarity between the air pollution problem vector and each background information vector in a vector database; wherein the vector database includes: a plurality of background information vectors, each background information vector being a vector representation of domain background information pre-processed by the text embedding model and stored in the vector database; The background information of the target quantity most relevant to the air pollution problem vector is recalled according to the cosine similarity sorting as the first retrieval result.

[0069] Based on the above embodiment, in this embodiment, the retrieval module 401 is specifically used to: An Aho-Corasick automaton is constructed based on a preset atmospheric science classification vocabulary, and multiple entities in the atmospheric pollution problem are extracted using a multi-pattern string search algorithm based on the Aho-Corasick automaton; Based on the preset mapping vocabulary, the extracted same entity is standardized into the same keyword and corrected through the fuzzy matching algorithm to obtain multiple standardized entities; For each of the plurality of standardized entities, identifying attributes or relationships related to a prompt word based on a large language model, and merging them using a regular expression to obtain a target attribute or target relationship related to the prompt word; wherein the prompt word is: a structured input instruction constructed based on the air pollution problem; Based on the standardized entities and the target attributes or target relationships in the standardized entities that are related to the prompt words, the subgraph corresponding to each standardized entity is recalled through a loop algorithm nested with preset statements, and the target knowledge triples are extracted based on the subgraphs corresponding to the standardized entities, and the target knowledge triples are used as the second retrieval results.

[0070] Based on the above embodiment, in this embodiment, the system further includes a construction module, specifically configured to: Acquiring background information in the field of atmospheric science, and performing data cleaning and text segmentation on the background information to obtain preprocessed text data; Performing text vectorization on the preprocessed text data using the text embedding model to generate a vector representation corresponding to the preprocessed text data; The vector representation corresponding to the preprocessed text data is stored in a cloud-native database to construct the vector database.

[0071] Based on the above embodiment, in this embodiment, the building block is specifically used to: Acquire target data; wherein the target data includes: standardized data, structured data, semi-structured data and unstructured data; Performing text annotation on the target data based on preset entity, relationship and attribute annotation specifications to obtain annotated data; Performing entity extraction, relationship extraction, and attribute distillation based on the labeled data to generate a document in a target format; The target format file is subjected to reference resolution, entity alignment and entity linking to generate knowledge triples, and the knowledge triples are stored in a graph database to construct the knowledge graph.

[0072] Based on the above embodiment, in this embodiment, the analysis module 402 is specifically used to: The air pollution problem, the first search result, and the second search result are integrated into a structured prompt word according to a preset template; wherein the preset template is a prompt word template containing a thought chain structure; The structured prompt words are input into the large language model, and the analysis results of the air pollution problem are output.

[0073] Based on the above embodiment, in this embodiment, the system further includes an evaluation module, which is specifically configured to: After fusing the air pollution problem, the first search result, and the second search result into a large language model and outputting an analysis result of the air pollution problem, the large language model is automatically evaluated using an automated evaluation framework based on fidelity, answer relevance, background information recall, and background information accuracy indicators to obtain an evaluation result; Based on the evaluation result, the preset threshold of the cosine similarity of the vector database and the retrieval depth of the knowledge graph are adjusted.

[0074] Figure 5 An example of a physical structure diagram of an electronic device is shown below. Figure 5 As shown, the electronic device may be a robot or other electronic device, and may include: a processor 510, a communications interface 520, a memory 530, and a communication bus 540. The processor 510, the communications interface 520, and the memory 530 communicate with each other via the communication bus 540. The processor 510 may call logic instructions in the memory 530 to execute the atmospheric pollution process analysis method, including: Receive an air pollution problem sent by a user, perform a vector database search and a knowledge graph search based on the air pollution problem, and obtain a first search result and a second search result; wherein the vector database search comprises calculating the cosine similarity between the background information vector in the vector database and the air pollution problem vector, and determining multiple pieces of background information whose cosine similarity meets a preset threshold as the first search result; and the knowledge graph search comprises performing entity recognition and standardization on the air pollution problem to generate a target knowledge triple as the second search result; The air pollution problem, the first search result and the second search result are fused and input into a large language model, and an analysis result of the air pollution problem is output.

[0075] Furthermore, the logic instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product, stored in a storage medium, includes instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in at least one embodiment of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0076] On the other hand, the present invention further provides a computer program product, comprising a computer program, which may be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can perform the atmospheric pollution process analysis method provided by the above methods, including: Receive an air pollution problem sent by a user, perform a vector database search and a knowledge graph search based on the air pollution problem, and obtain a first search result and a second search result; wherein the vector database search comprises calculating the cosine similarity between the background information vector in the vector database and the air pollution problem vector, and determining multiple pieces of background information whose cosine similarity meets a preset threshold as the first search result; and the knowledge graph search comprises performing entity recognition and standardization on the air pollution problem to generate a target knowledge triple as the second search result; The air pollution problem, the first search result and the second search result are fused and input into a large language model, and an analysis result of the air pollution problem is output.

[0077] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the atmospheric pollution process analysis method provided by the above methods, including: Receive an air pollution problem sent by a user, perform a vector database search and a knowledge graph search based on the air pollution problem, and obtain a first search result and a second search result; wherein the vector database search comprises calculating the cosine similarity between the background information vector in the vector database and the air pollution problem vector, and determining multiple pieces of background information whose cosine similarity meets a preset threshold as the first search result; and the knowledge graph search comprises performing entity recognition and standardization on the air pollution problem to generate a target knowledge triple as the second search result; The air pollution problem, the first search result and the second search result are fused and input into a large language model, and an analysis result of the air pollution problem is output.

[0078] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0079] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0080] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for analyzing an atmospheric pollution process, characterized in that: include: Receive an air pollution problem sent by a user, perform a vector database search and a knowledge graph search based on the air pollution problem, and obtain a first search result and a second search result; wherein the vector database search comprises calculating the cosine similarity between the background information vector in the vector database and the air pollution problem vector, and determining multiple pieces of background information whose cosine similarity meets a preset threshold as the first search result; and the knowledge graph search comprises performing entity recognition and standardization on the air pollution problem to generate a target knowledge triple as the second search result; The air pollution problem, the first search result and the second search result are fused and input into a large language model, and an analysis result of the air pollution problem is output.

2. The atmospheric pollution process analysis method according to claim 1, characterized in that: The calculating cosine similarity between the background information vector and the air pollution problem vector in the vector database, and determining multiple pieces of background information whose cosine similarity meets a preset threshold as the first search result, includes: The air pollution problem is converted into the air pollution problem vector through a text embedding model; wherein the text embedding model is: a pre-trained model based on Transformer; Calculating the cosine similarity between the air pollution problem vector and each background information vector in a vector database; wherein the vector database includes: a plurality of background information vectors, each background information vector being a vector representation of domain background information pre-processed by the text embedding model and stored in the vector database; The background information of the target quantity most relevant to the air pollution problem vector is recalled according to the cosine similarity sorting as the first retrieval result.

3. The atmospheric pollution process analysis method according to claim 1, characterized in that: The performing entity recognition and standardization on the air pollution problem to generate a target knowledge triple as the second search result includes: An Aho-Corasick automaton is constructed based on a preset atmospheric science classification vocabulary, and multiple entities in the atmospheric pollution problem are extracted using a multi-pattern string search algorithm based on the Aho-Corasick automaton; Based on the preset mapping vocabulary, the extracted same entity is standardized into the same keyword and corrected through the fuzzy matching algorithm to obtain multiple standardized entities; For each of the plurality of standardized entities, identifying attributes or relationships related to a prompt word based on a large language model, and merging them using a regular expression to obtain a target attribute or target relationship related to the prompt word; wherein the prompt word is: a structured input instruction constructed based on the air pollution problem; Based on the standardized entities and the target attributes or target relationships in the standardized entities that are related to the prompt words, the subgraph corresponding to each standardized entity is recalled through a loop algorithm nested with preset statements, and the target knowledge triples are extracted based on the subgraphs corresponding to the standardized entities, and the target knowledge triples are used as the second retrieval results.

4. The atmospheric pollution process analysis method according to claim 2, characterized in that: The process of constructing the vector database includes: Acquiring background information in the field of atmospheric science, and performing data cleaning and text segmentation on the background information to obtain preprocessed text data; Performing text vectorization on the preprocessed text data using the text embedding model to generate a vector representation corresponding to the preprocessed text data; The vector representation corresponding to the preprocessed text data is stored in a cloud-native database to construct the vector database.

5. The atmospheric pollution process analysis method according to claim 3, characterized in that: The construction process of the knowledge graph includes: Acquire target data; wherein the target data includes: standardized data, structured data, semi-structured data and unstructured data; Performing text annotation on the target data based on preset entity, relationship and attribute annotation specifications to obtain annotated data; Performing entity extraction, relationship extraction, and attribute distillation based on the labeled data to generate a document in a target format; The target format file is subjected to reference resolution, entity alignment and entity linking to generate knowledge triples, and the knowledge triples are stored in a graph database to construct the knowledge graph.

6. The atmospheric pollution process analysis method according to claim 1, characterized in that: The step of fusing the air pollution problem, the first search result, and the second search result into a large language model and outputting an analysis result of the air pollution problem includes: The air pollution problem, the first search result, and the second search result are integrated into a structured prompt word according to a preset template; wherein the preset template is a prompt word template containing a thought chain structure; The structured prompt words are input into the large language model, and the analysis results of the air pollution problem are output.

7. The atmospheric pollution process analysis method according to claim 6, characterized in that: After fusing the air pollution problem, the first search result, and the second search result into a large language model and outputting an analysis result of the air pollution problem, the method further includes: Based on the fidelity, answer relevance, context recall, and context precision indicators, the large language model is automatically evaluated through an automated evaluation framework to obtain an evaluation result; Based on the evaluation result, the preset threshold of the cosine similarity of the vector database and the retrieval depth of the knowledge graph are adjusted.

8. An atmospheric pollution process analysis system, characterized in that: include: A retrieval module is configured to receive an air pollution problem sent by a user, and perform a vector database search and a knowledge graph search based on the air pollution problem, respectively, to obtain a first search result and a second search result; wherein the vector database search comprises calculating the cosine similarity between a background information vector in the vector database and an air pollution problem vector, and determining as the first search result a plurality of background information pieces whose cosine similarity meets a preset threshold; and the knowledge graph search comprises performing entity recognition and standardization on the air pollution problem to generate a target knowledge triple as the second search result; An analysis module is used to integrate the air pollution problem, the first search result and the second search result into a large language model, and output an analysis result of the air pollution problem.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the atmospheric pollution process analysis method according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the atmospheric pollution process analysis method according to any one of claims 1 to 7 is implemented.

Images