Policy and regulation intelligent question and answer method and system based on large language model

Abstract

The invention relates to the technical field of artificial intelligence, and particularly discloses a policy and regulation intelligent question answering method and system based on a large language model, and the method comprises the steps: segmenting a policy and regulation text into text blocks through a recursive character text segmentation method, and constructing a knowledge base in combination with expert labeling and automatic information extraction; performing semantic embedding on the text block by utilizing a pre-trained Transform model, generating a high-dimensional vector representation, and storing the high-dimensional vector representation in a Chroma database; the user question is converted into an embedded vector, relevant laws and regulations are retrieved through a Chroma database, a large language model and a knowledge fusion attention generation module are combined, and text embedding and knowledge graph information are fused to generate an answer; and the user obtains answers through the intelligent question-answer interaction interface and can view the original texts and explanations of laws and regulations. The system comprises a data acquisition and preprocessing module, a semantic embedding and storage module, a retrieval and generation module and a user interaction module. According to the method, large-size regulation texts are effectively processed, and the retrieval efficiency and the answer quality are improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology. More specifically, the present invention relates to a method and system for intelligent question answering of policies and regulations based on large language models. Background Art

[0002] Currently, with the acceleration of digital transformation, enterprises, governments, and the public often face problems such as numerous legal provisions, frequent updates, complex language, and great difficulty in understanding when dealing with policy and regulation texts.

[0003] Existing public document 1 (Research on the Construction of an Intelligent Question Answering System for Alzheimer's Disease Based on Large Language Models and Knowledge Bases, 2025) discloses an intelligent question answering system for Alzheimer's disease based on large language models and knowledge bases. This system uses the Langchain framework to integrate the ChatGLM3-6B model with the AD local knowledge base (ADKB) and realizes domain knowledge question answering through retrieval-augmented generation (RAG) technology. However, this system relies on an external knowledge base, resulting in slow response speed for interactive question answering.

[0004] Existing public document 2 (Design and Development of an Intelligent Question Answering Platform for Special Equipment Professional Knowledge Bases Based on Large Models, 2025) discloses an intelligent question answering platform for special equipment professional knowledge bases based on large language models. This platform adopts a three-layer architecture: the business layer supports multi-format document upload, OCR parsing, and session management; the model layer integrates named entity recognition, text similarity models, and large language models (such as GPT-4, Baichuan-13B) to achieve semantic parsing and knowledge fusion; the front-end UI layer provides a mini-program interaction interface that supports text / voice input, answer display, and user feedback. Its core functions include document tagging management, intelligent question answering (generating answers in combination with Prompt engineering and annotating the original source), multi-model scheduling, and a self-optimization mechanism driven by user feedback. However, this platform relies on the knowledge coverage ability of external large models. If the pre-trained data does not fully cover the content of the special equipment professional field, the accuracy of answers will be limited.

[0005] Therefore, there is an urgent need for a method to improve the accuracy of understanding regulatory texts and the response speed of interactive question answering. Summary of the Invention

[0006] To overcome the above-mentioned deficiencies of the prior art, the present invention provides a method and system for intelligent policy and regulation Q&A based on large language models. Through a recursive character text segmentation method, a regulation document is segmented into text blocks. A pre-trained Transformer model is used to perform semantic embedding on the regulation text blocks, generate high-dimensional vector representations, and store them in a Chroma database. Combining a large language model and a knowledge fusion attention generation module, by fusing the text embedding and the entity relationship information of the structured knowledge graph, the problems raised in the above background art are solved.

[0007] To achieve the above object, the present invention provides the following technical solutions: A method for intelligent policy and regulation Q&A based on large language models, comprising the following steps: Step S1, using a recursive character text segmentation method to process the collected policy and regulation text, and segmenting it into text blocks.

[0008] Step S2, using a pre-trained Transformer model to perform semantic embedding on the policy and regulation text blocks, generate high-dimensional vector representations, and store them in a vector database.

[0009] Step S3, converting the user's question into an embedded vector, retrieving relevant regulations using the Chroma database, and then combining the generation techniques of a large language model and a knowledge fusion attention generation module to generate a final answer.

[0010] Step S4, the user obtains an answer through a natural language question based on the intelligent Q&A interaction interface.

[0011] The knowledge fusion attention generation module adopts a dynamic knowledge fusion mechanism to achieve the dual fusion of text semantic embedding information and structured entity knowledge, and generate an answer. The specific steps are as follows: Step Y1, performing word segmentation processing and semantic encoding on the user's question through a pre-trained Transformer semantic embedding model, converting the natural language question raised by the user into a high-dimensional embedded vector representation with fine semantic features, and using this representation as the initial query input of the knowledge fusion attention generation module.

[0012] Step Y2, the knowledge fusion attention generation module uses the question embedded vector to retrieve the regulation text fragment most similar to the user's question semantics from the Chroma semantic vector database, and at the same time combines the pre-constructed structured knowledge graph. By matching the entity vocabulary in the question with the knowledge graph nodes, relevant structured entities, attributes, and relationship information involved in the regulation fragment are extracted to construct a preliminary multi-source knowledge candidate set.

[0013] Step Y3: The knowledge fusion attention generation module uses a cross-modal attention mechanism. Guided by the user question embedding vector, it dynamically calculates attention weights between the regulatory text embedding representation and the knowledge graph structured knowledge representation through the multi-head self-attention structure unique to the Transformer model, captures the semantic interaction and complementary information between the regulatory text and the knowledge graph, realizes the semantic fusion of unstructured text information and structured knowledge information, and forms a comprehensive feature representation integrating multiple sources.

[0014] Step Y4: The fused comprehensive feature representation is further fed into the pre-trained Transformer model. At this time, the multi-source semantic features obtained by the Transformer model are used to gradually generate answer segments that highly match the user question in an autoregressive generation manner. In the generation process, each step combines the context information generated in the previous step with the entity and relationship features in the structured knowledge graph, so that the answer has both clear logical reasoning paths and the interpretability of structured knowledge on the basis of semantic accuracy.

[0015] As a further solution of the present invention, in step S1, the collected policy and regulation texts are processed by a recursive character text segmentation method and segmented into text blocks, including the following specific contents: The data collection stage of the policy and regulation texts covers multi-source heterogeneous information sources of laws and regulations, including both structured and semi-structured sources such as national laws and regulations databases, local government websites, industry standards, and enterprise internal rules and regulations, and unstructured text data such as legal policy interpretations, expert opinions, and case analyses published on news media and social media platforms.

[0016] The collected data is preprocessed, and the preprocessing is a recursive character text segmentation method. The specific implementation steps of the recursive character text segmentation method are as follows: First, define the initial window size for segmentation, and preliminarily divide the text by sliding the window according to the semantic coherence and grammatical integrity of the sentence; secondly, for each generated text block after division, further perform in-depth splitting according to the word count limit to form finer-grained text blocks (Chunks), and the size of the text blocks is controlled between 200 and 2000 characters. After the segmentation is completed, the obtained text is converted into a structured or semi-structured form knowledge base by combining expert annotation and automated information extraction.

[0017] As a further solution of the present invention, in step S2, a pre-trained Transformer model is used to perform semantic embedding on the policy and regulation text blocks, generate a high-dimensional vector representation and store it in the vector database, including the following specific content: A deep pre-trained model with a Transformer architecture is used for the semantic representation of the policy and regulation text blocks, and the semantic embedding task of the policy and regulation text blocks is completed by calling the open-source library of the pre-trained Transformer model.

[0018] Before inputting into the Transformer model, preprocessing operations need to be performed on the policy and regulation text blocks. The preprocessing operation is to tokenize the text blocks using the tokenizer supporting the model, and decompose the text into the smallest semantic units (Tokens) that the model can understand and process.

[0019] The tokenized policy and regulation text blocks are input into the Transformer model in the form of a tokens sequence. The deep semantic feature extraction of the text is performed using the multi-layer Transformer encoder module included in the model, including the following steps: Step Q1, before the text input is sent into the Transformer encoder, it is decomposed into multiple smallest semantic units (Tokens) by the tokenizer. The smallest semantic units are mapped to an initial high-dimensional vector representation, and position encoding is superimposed to clarify the positional relationship of each Token in the text sequence.

[0020] Step Q2, the first layer in the Transformer encoder calculates the semantic relevance between each Token in the input text and all other Tokens through the multi-head self-attention mechanism, obtains a preliminary semantic relationship feature representation, maps the vector of each Token to a query vector (Query), a key vector (Key), and a value vector (Value), and calculates the mutual attention weights through the dot product between the query vector, the key vector, and the value vector to capture the semantic interaction relationship between the words in the text.

[0021] Step Q3, the vector output by the first layer of self-attention mechanism is non-linearly mapped and semantically abstracted through a feed-forward neural network to further enrich and integrate the preliminarily obtained semantic information.

[0022] Step Q4, the output obtained from the attention calculation and the feed-forward network processing is used as the input of the next layer of the Transformer encoder module. Each layer of the encoder module repeats the process of step Q1-step Q3. However, since each layer is independently trained, it can focus on higher-order and more abstract semantic features, and deeper and richer text semantic information can be captured for each layer going up.

[0023] Step Q5: After stacking multiple Transformer encoders, the model finally obtains deep and detailed semantic embedding vectors, which contain the semantics and context relationships at all levels of the text, and can express complex syntactic structures and long-distance semantic dependency relationships.

[0024] The high-dimensional semantic embedding vectors of the legal text blocks output by the Transformer model are stored in the Chroma database for rapid and efficient semantic retrieval and matching in subsequent intelligent question-answering tasks.

[0025] As a further solution of the present invention, in step S3, the user question is converted into an embedding vector, relevant regulations are retrieved using the Chroma database, and then combined with the generation technology of the large language model and the knowledge fusion attention generation module to generate the final answer, including the following specific content: After the user asks a question in natural language through the intelligent question-answering interaction interface, the question statement input by the user is converted into the form of a semantic embedding vector through a pre-trained Transformer model. Based on the obtained question semantic embedding vector, the Chroma vector database is called, and the text fragments related to the question that are semantically closest are quickly retrieved from the large-scale legal text embedding library in the way of nearest neighbor search.

[0026] The quickly retrieved legal text fragments are not directly used as answers, but are input into the knowledge fusion attention generation module together with the context semantic information as potential answer sources. The knowledge fusion attention generation module adopts a dynamic multi-source knowledge fusion mechanism, that is, first retrieves the embedding vectors of the legal fragments from the Chroma database, and simultaneously calls the corresponding entity relationship information in the structured professional domain knowledge graph, so as to achieve the dual fusion of text semantic embedding information and structured entity knowledge.

[0027] As a further solution of the present invention, in step S4, the user obtains an answer based on the intelligent question-answering interaction interface through a natural language question, including the following specific content: A front-end Web interface that supports natural language interaction is constructed. When the user inputs a question related to policies and regulations in natural language, the intelligent question-answering interaction interface immediately transmits the question to the knowledge fusion attention generation module at the back end. Then, answer fragments are generated and displayed to the user on the intelligent question-answering interaction interface. In addition, to meet the user's deeper information needs, the interface also provides a series of additional functions, including the function of positioning the original legal text, that is, providing a link to the legal document of the answer source, allowing the user to quickly jump to the corresponding original legal text by clicking the link for direct verification; and the function of explaining the answer, that is, providing the structured knowledge node information and professional term explanations involved in the answer generation process to help the user better understand the background and semantic connotation of the answer.

[0028] As a further solution of the present invention, a policy and regulation intelligent question-answering system based on a large language model includes a data collection and preprocessing module, a semantic embedding and storage module, a retrieval and generation module, and a user interaction module. The data collection and preprocessing module is used to collect policy and regulation texts from diverse sources such as national laws and regulations databases, local government websites, industry standards, enterprise internal rules and regulations, and news media, and uses a recursive character text segmentation method to split the text into blocks of 200-2000 characters, and converts them into a structured or semi-structured knowledge base through expert annotation and automated information extraction; the semantic embedding and storage module uses a pre-trained Transformer model to perform semantic embedding on the text blocks, generates a high-dimensional vector representation, and stores the results in a Chroma database; after receiving a user question, the retrieval and generation module converts the question into an embedded vector through a Transformer model, retrieves relevant regulation text blocks using the Chroma database, and combines a large language model and a knowledge fusion attention generation module to fuse the entity relationship information of text embedding and knowledge graph; the user interaction module receives the user's natural language question through an intelligent question-answering interaction interface, and displays the generated answer, the link to the original regulation text, and relevant knowledge explanations.

[0029] Technical effects and advantages of a policy and regulation intelligent question-answering method and system based on a large language model of the present invention: The present invention uses a recursive character text segmentation method to split the regulation document into blocks suitable for processing, effectively coping with the challenges of large text volume and high semantic analysis difficulty; the present invention uses a pre-trained Transformer model to perform semantic embedding on the regulation text blocks, generates a high-dimensional vector representation and stores it in a Chroma database, realizing efficient semantic retrieval and improving the response speed; the present invention combines a large language model and a knowledge fusion attention generation module, and by fusing the entity relationship information of text embedding and a structured knowledge graph, ensures the accuracy and logic of the answer. In addition, the intelligent question-answering interaction interface provides functions for positioning the original regulation text and explaining the answer, enhancing the user experience and information transparency. Description of the Drawings

[0030] Figure 1 It is a flowchart of a policy and regulation intelligent question-answering method based on a large language model of the present invention.

[0031] Figure 2 It is a general flowchart of an Alzheimer's disease intelligent question-answering system based on a large language model and a knowledge base in the prior art.

[0032] Figure 3 It is a general organizational construction architecture diagram of a special equipment professional knowledge base intelligent question-answering platform based on a large model in the prior art.

[0033] Figure 4Schematic diagram of the performance comparison of the knowledge fusion attention generation module of the present invention under different working conditions.

[0034] Figure 5 Schematic diagram of the comparison of the generation time of the knowledge fusion attention generation module of the present invention under different fusion conditions.

[0035] Figure 6 Schematic diagram of the intelligent Q&A interaction interface of the present invention Figure 1 。

[0036] Figure 7 Schematic diagram of the intelligent Q&A interaction interface of the present invention Figure 2 。

[0037] Figure 8 Schematic diagram of the intelligent Q&A interaction interface of the present invention Figure 3 。

[0038] Figure 9 Schematic diagram of the structure of a policy and regulation intelligent Q&A system based on a large language model of the present invention. Detailed implementation manners

[0039] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0040] Embodiment 1 Referring to Figure 1 the method flow chart shown, an embodiment of the present invention provides a policy and regulation intelligent Q&A method based on a large language model, which includes the following steps: Step S1, using a recursive character text segmentation method to process the collected policy and regulation texts and segment them into text blocks.

[0041] Step S2, using a pre-trained Transformer model to perform semantic embedding on the policy and regulation text blocks, generating high-dimensional vector representations and storing them in a vector database.

[0042] Step S3, converting the user's question into an embedded vector, retrieving relevant regulations using the Chroma database, and then combining the generation technologies of the large language model and the knowledge fusion attention generation module to generate a final answer.

[0043] Step S4, the user obtains the answer through a natural language question based on the intelligent Q&A interaction interface.

[0044] In this embodiment, referring to Figure 2, it is an Alzheimer's disease intelligent Q&A system based on large language models and knowledge bases disclosed in the prior art. This system uses the Langchain framework to integrate the ChatGLM3-6B model with the AD local knowledge base (ADKB), and realizes domain knowledge Q&A through retrieval-augmented generation (RAG) technology. However, this system relies on an external knowledge base, resulting in slow response speed for interactive Q&A. Refer to Figure 3 , it is an intelligent Q&A platform for special equipment professional knowledge bases based on large language models disclosed in the prior art. This platform adopts a three-layer architecture: the business layer supports multi-format document upload, OCR parsing, and session management; the model layer integrates named entity recognition, text similarity models, and large language models (such as GPT-4, Baichuan-13B) to achieve semantic parsing and knowledge fusion; the front-end UI layer provides a mini-program interaction interface, supporting text / voice input, answer display, and user feedback. Its core functions include document tagging management, intelligent Q&A (generating answers in combination with Prompt engineering and annotating the original source), multi-model scheduling, and a self-optimization mechanism driven by user feedback. However, this platform relies on the knowledge coverage ability of external large models. If the pre-training data does not fully cover the content of the special equipment professional field, the accuracy of answers will be limited.

[0045] Furthermore, in step S1, a recursive character text segmentation method is used to process the collected policy and regulation texts and cut them into text blocks, including: the data collection stage of policy and regulation texts covers multi-source heterogeneous information sources of laws and regulations, including both structured and semi-structured sources such as national laws and regulations databases, local government websites, industry standards, and enterprise internal rules and regulations, as well as unstructured text data such as legal policy interpretations, expert opinions, and case analyses published on news media and social media platforms.

[0046] After data collection, due to the large text volume of regulation documents, it is difficult to directly perform overall semantic analysis. Therefore, the collected data is preprocessed, and the preprocessing is a recursive character text segmentation method. The specific implementation steps of the recursive character text segmentation method are as follows: First, define the initial window size for segmentation, and based on the semantic coherence and grammatical integrity of sentences, preliminarily divide the text through a sliding window method; Second, for each generated text block after division, further perform in-depth splitting according to the word count limit to form more fine-grained text blocks (Chunks), and the size of the text blocks is controlled between 200 and 2000 characters.

[0047] After the segmentation is completed, the obtained text is transformed into a structured or semi-structured form knowledge base by combining expert annotation with automated information extraction. Specifically, the annotation work of experts in the professional field includes semantic category annotation of legal terms appearing in the text block, identification and association of synonymous terms, clarification of legal liability subjects and applicable conditions, and clear definition of the core points of policies and regulations and the scope of legal effect, so as to initially determine the semantic structure.

[0048] Further, in step S2, a pre-trained Transformer model is used to perform semantic embedding on the policy and regulation text block, generate a high-dimensional vector representation and store it in a vector database, including: using a deep pre-trained model with a Transformer architecture for semantic representation of the policy and regulation text block, and completing the semantic embedding task of the policy and regulation text block by calling the open-source library of the pre-trained Transformer model.

[0049] Before inputting into the Transformer model, preprocessing operations need to be performed on the policy and regulation text block. The preprocessing operation is to tokenize the text block using the tokenizer provided with the model, and decompose the text into the smallest semantic units (Tokens) that the model can understand and process, such as words, sub-words, or character fragments, etc.

[0050] The tokenized policy and regulation text block is input into the Transformer model in the form of a tokens sequence, and a multi-layer Transformer encoder module included in the model is used to extract deep semantic features from the text, including the following steps: Step Q1, before the text input is sent into the Transformer encoder, it is decomposed into multiple smallest semantic units (Tokens) by a tokenizer. The smallest semantic units are mapped to an initial high-dimensional vector representation, and positional encoding is added to clarify the positional relationship of each Token in the text sequence.

[0051] Step Q2, the first layer in the Transformer encoder calculates the semantic relevance of each Token in the input text to all other Tokens through a multi-head self-attention mechanism, obtains a preliminary semantic relationship feature representation, maps the vector of each Token to a query vector (Query), a key vector (Key), and a value vector (Value), and calculates the mutual attention weights through the dot product between the query vector, the key vector, and the value vector to capture the semantic interaction relationship between the words in the text.

[0052] Step Q3, the vector output by the first-layer self-attention mechanism is non-linearly mapped and semantically abstracted through a feed-forward neural network to further enrich and integrate the initially obtained semantic information.

[0053] In step Q4, the output obtained from the attention calculation and the feed-forward network processing is used as the input to the next-layer Transformer encoder module. The process of steps Q1 - Q3 is repeated for each encoder module. However, since each layer is independently trained, it can focus on higher-order and more abstract semantic features, and deeper and richer text semantic information can be captured with each upward layer.

[0054] In step Q5, after stacking multiple layers of Transformer encoders, the model finally obtains a deep and detailed semantic embedding vector. The semantic embedding vector contains the semantics and context relationships at all levels of the text, and can express complex syntactic structures and long-distance semantic dependency relationships.

[0055] The high-dimensional semantic embedding vector of the legal text block output by the Transformer model is stored in the Chroma database for fast and efficient semantic retrieval and matching in subsequent intelligent question-answering tasks.

[0056] Furthermore, in step S3, the user's question is converted into an embedding vector, relevant regulations are retrieved using the Chroma database, and then combined with the generation technology of the large language model and the knowledge fusion attention generation module to generate the final answer, including: after the user asks a question in natural language through the intelligent question-answering interaction interface, the question statement input by the user is converted into a semantic embedding vector form through the pre-trained Transformer model. Based on the obtained question semantic embedding vector, the Chroma vector database is called, and the text fragments most semantically similar to the question are quickly retrieved from the large-scale legal text embedding library in the way of nearest neighbor search.

[0057] The quickly retrieved legal text fragments are not directly used as answers, but are used as potential answer sources and context semantic information and are input into the knowledge fusion attention generation module together. The knowledge fusion attention generation module adopts a dynamic multi-source knowledge fusion mechanism, that is, first retrieves the embedding vector of the legal fragment from the Chroma database, and simultaneously calls the corresponding entity relationship information in the structured professional domain knowledge graph, so as to realize the dual fusion of text semantic embedding information and structured entity knowledge. The specific steps are as follows: In step Y1, the user's question is tokenized and semantically encoded through the pre-trained Transformer semantic embedding model, and the natural language question proposed by the user is converted into a high-dimensional embedding vector representation with fine semantic features, and this representation is used as the initial query input to the knowledge fusion attention generation module.

[0058] Step Y2, the knowledge fusion attention generation module uses the question embedding vector to retrieve the legal text fragment with the most similar semantics to the user's question from the Chroma semantic vector database. At the same time, in combination with the pre-constructed structured knowledge graph, by matching the entity vocabulary in the question with the knowledge graph nodes, the relevant structured entity, attribute, and relationship information involved in the legal fragment is extracted to construct a preliminary multi-source knowledge candidate set.

[0059] Step Y3, the knowledge fusion attention generation module applies the cross-modal attention mechanism. Through the multi-head self-attention structure unique to the Transformer model, guided by the user's question embedding vector, the attention weights are dynamically calculated between the legal text embedding representation and the structured knowledge representation of the knowledge graph, capturing the semantic interaction and complementary information between the legal text and the knowledge graph, and realizing the semantic fusion of unstructured text information and structured knowledge information to form a comprehensive feature representation of multi-source fusion.

[0060] Step Y4, the fused comprehensive feature representation is further fed into the pre-trained Transformer model. At this time, the multi-source semantic features obtained by the Transformer model are used to gradually generate answer fragments that highly match the user's question in an autoregressive generation manner. And in each step of the generation process, the context information generated in the previous step and the entity and relationship features in the structured knowledge graph are combined, so that the answer has both clear logical reasoning paths and the interpretability of structured knowledge on the basis of semantic accuracy.

[0061] Step Y5, finally, the knowledge fusion attention generation module outputs the generated legal answer fragment to the intelligent question-answering interaction interface, and at the same time provides, in an auxiliary manner, the original legal text source corresponding to the answer and the relevant structured knowledge node, attribute, and relationship links.

[0062] In this embodiment, the following is a Python language code example of the knowledge fusion attention generation module: from transformers import AutoModel, AutoTokenizer,AutoModelForCausalLM import chromadb import torch encoder_model_name = "bert-base-chinese" generator_model_name = "uer / gpt2-chinese-cluecorpussmall" tokenizer_encoder = AutoTokenizer.from_pretrained(encoder_model_name) encoder_model = AutoModel.from_pretrained(encoder_model_name) tokenizer_generator = AutoTokenizer.from_pretrained(generator_model_name) generator_model = AutoModelForCausalLM.from_pretrained(generator_model_name) client = chromadb.Client() collection = client.create_collection(name="policy_knowledge") sample_texts = ["Regulations on environmental protection in laws and regulations", "Details of overtime compensation in the Labor Law", "Relevant tax reduction and exemption policies for individual income tax"] embeddings = encoder_model(**tokenizer_encoder(sample_texts, return_tensors="pt", padding=True))[0][:, 0, :].detach().numpy() collection.add(documents=sample_texts, embeddings=embeddings.tolist(), ids=["1", "2", "3"]) def KFAG(question, top_k=1): question_embedding = encoder_model(**tokenizer_encoder(question,return_tensors="pt"))[0][:, 0, :].detach().numpy().tolist() results = collection.query(query_embeddings=question_embedding, n_results=top_k) retrieved_docs = results["documents"][0] fusion_context = ";".join(retrieved_docs) input_text = f"Question: {question}; Related regulations: {fusion_context}; Answer:" inputs = tokenizer_generator.encode(input_text, return_tensors="pt") outputs = generator_model.generate(inputs, max_length=256, do_sample=True, top_p=0.95, top_k=60) answer = tokenizer_generator.decode(outputs[0], skip_special_tokens=True) return answer if __name__ == "__main__": user_question = "How is overtime pay calculated?" response = KFAG(user_question) print("Answer generated by KFAG:", response) In this embodiment, see Figure 4 , the performance of the generated answer quality of the knowledge fusion attention generation module is evaluated using BLEU, ROUGE and Diversity indicators. The results show that the generated answer quality after adding structured knowledge graph information is significantly better than that of using only text embedding. The BLEU score is increased from 72.3% to 85.7%, the ROUGE score is increased from 68.5% to 81.4%, and the Diversity score is also increased from 45.2% to 67.8%. Figure 5, when only using text embeddings, the generation time is approximately 1.45 seconds. After integrating the structured knowledge graph information, the generation time slightly increases to 1.72 seconds. This shows that although the introduction of structured knowledge increases the computational complexity, the overall increase in generation time is not significant, and the system still maintains a high response speed.

[0063] Further, in step S4, the user obtains answers based on the intelligent Q&A interaction interface through natural language questions, including: Refer to Figures 6 - 8 , construct a front-end Web interface that supports natural language interaction. When the user inputs a policy and regulation-related question in the form of natural language, the intelligent Q&A interaction interface immediately transmits the question to the knowledge fusion attention generation module at the back end. Then, answer fragments are generated and displayed to the user on the intelligent Q&A interaction interface. In addition, to meet the user's deeper information needs, the interface also provides a series of additional functions, including the function of positioning the original text of regulations, that is, providing a link to the regulation document where the answer source is located, allowing the user to quickly jump to the corresponding original text of the regulation by clicking the link for direct verification; and the function of explaining the answer, that is, providing the structured knowledge node information and professional term explanations involved in the answer generation process to help the user better understand the background and semantic connotation of the answer.

[0064] Embodiment 2 Refer to Figure 9 Referring to the structural schematic diagram shown, an embodiment of the present invention provides a policy and regulation intelligent Q&A system based on a large language model, including a data collection and preprocessing module, a semantic embedding and storage module, a retrieval and generation module, and a user interaction module. The data collection and preprocessing module is used to collect policy and regulation texts from diverse sources such as national laws and regulations databases, local government websites, industry standards, enterprise internal rules and regulations, and news media, and use a recursive character text segmentation method to segment the text into blocks of 200 - 2000 characters, and convert them into a structured or semi-structured knowledge base through expert annotation and automated information extraction; the semantic embedding and storage module uses a pre-trained Transformer model to perform semantic embedding on the text blocks, generate high-dimensional vector representations, and store the results in a Chroma database; after receiving the user's question, the retrieval and generation module converts the question into an embedding vector through a Transformer model, retrieves relevant regulation text blocks using the Chroma database, and combines a large language model and a knowledge fusion attention generation module to fuse the entity relationship information of text embedding and the knowledge graph; the user interaction module receives the user's natural language question through the intelligent Q&A interaction interface, and displays the generated answer, the link to the original text of the regulation, and relevant knowledge explanations.

[0065] As described above, it is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of changes or substitutions, which should all be covered within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claimed rights.

[0066] Finally: The above are only the preferred embodiments of the present invention and are not used to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for intelligent question answering of policies and regulations based on large language models, characterized in that, It includes the following steps: Step S1: Process the collected policy and regulation texts using a recursive character text segmentation method, and segment them into text blocks; Step S2: Use a pre-trained Transformer model to perform semantic embedding on the policy and regulation text blocks, generate high-dimensional vector representations, and store them in a vector database; Step S3: Convert the user's question into an embedded vector, retrieve relevant regulations using the Chroma database, and then combine the generation techniques of a large language model and a knowledge fusion attention generation module to generate a final answer; Step S4: The user obtains an answer through a natural language question based on the intelligent question and answer interaction interface; The knowledge fusion attention generation module adopts a dynamic knowledge fusion mechanism to achieve the dual fusion of text semantic embedding information and structured entity knowledge, and generate an answer. The specific steps are as follows: Step Y1: Convert the user's question into an embedded vector; Step Y2: Retrieve relevant regulation text fragments from the vector database, and extract entity, attribute, and relationship information in combination with a structured knowledge graph; Step Y3: Use a cross-modal attention mechanism to dynamically calculate the attention weights between the text embedding and the knowledge graph, and form a multi-source fusion feature representation; Step Y4: Generate answer fragments through an autoregressive generation method, and in each step of the generation process, the context information generated in the previous step and the entity and relationship features in the structured knowledge graph will be combined.

2. The method for intelligent policy and regulation Q&A based on large language models according to claim 1, wherein , The recursive character text segmentation method is: Define an initial window size, and perform a preliminary division of the text through a sliding window; Deeply split the divided text blocks according to the word count limit to form fine-grained text blocks of 200-2000 characters.

3. The method for intelligent Q&A of policies and regulations based on large language models according to claim 1, characterized in that , The semantic embedding process is: Decompose the text block into the smallest semantic units through a tokenizer, and input it into the Transformer encoder after adding position encoding; The encoder extracts text semantic features through a multi-layer multi-head self-attention mechanism and a feed-forward neural network to generate a deep semantic embedding vector.

4. The intelligent Q&A method for policies and regulations based on large language models according to claim 1, characterized in that, The intelligent question and answer interaction interface provides a function for locating the original text of regulations and an answer explanation function, enabling users to verify the source of the answer and understand the background information.

5. The intelligent Q&A method for policies and regulations based on large language models according to claim 1, wherein The encoder of the Transformer model contains a multi-layer structure. Each layer calculates the semantic correlation between the smallest semantic units through a multi-head self-attention mechanism, and performs a non-linear mapping through a feed-forward neural network.

6. A method for intelligent Q&A of policies and regulations based on large language models according to claim 1, characterized in that, The cross-modal attention mechanism uses the multi-head self-attention structure of the Transformer model to dynamically capture the semantic interaction information between the regulation text and the knowledge graph guided by the user's question embedded vector.

7. A method for intelligent Q&A of policies and regulations based on a large language model according to claim 1, characterized in that, The structured knowledge graph contains entity, attribute, and relationship information in the policy and regulation text, and extracts relevant information by matching the entity vocabulary in the question with the nodes in the knowledge graph.

8. An intelligent Q&A system for policies and regulations based on large language models, characterized in that, Applied to a policy and regulation intelligent question and answer method based on a large language model according to any one of claims 1-7, it includes a data collection and preprocessing module, a semantic embedding and storage module, a retrieval and generation module, and a user interaction module.

9. The intelligent Q&A system for policies and regulations based on large language models according to claim 8, characterized in that, The data collection and preprocessing module is used to collect policy and regulation texts from diverse sources such as national laws and regulations databases, local government websites, industry standards, enterprise internal rules and regulations, and news media, and use a recursive character text segmentation method to segment the text into blocks of 200-2000 characters, and transform them into a structured or semi-structured knowledge base through expert annotation and automated information extraction; the semantic embedding and storage module uses a pre-trained Transformer model to perform semantic embedding on the text blocks, generate high-dimensional vector representations, and store the results in a Chroma database; after receiving a user question, the retrieval and generation module converts the question into an embedding vector through a Transformer model, retrieves relevant regulation text blocks using the Chroma database, and combines a large language model and a knowledge fusion attention generation module to fuse the entity relationship information of text embeddings and knowledge graphs; the user interaction module receives natural language questions from users through an intelligent question and answer interaction interface, and displays the generated answers, links to the original regulations, and relevant knowledge explanations.

Images