Private knowledge question-answering method in combination with knowledge graph

Abstract

The invention discloses a private knowledge question-answering method combined with a knowledge graph. Comprising the following steps: S1, collecting private basic corpora; s2, fusing the acquired basic corpus information; s3, establishing a knowledge graph; s4, partitioning the vector data information; s5, constructing a knowledge graph; s6, processing the problem of the user; s7, user question answer retrieval and generation; s8, outputting answers to the user questions; the knowledge graph is constructed and combined with the large language model to deeply understand and efficiently retrieve complex information in the private knowledge base, and the knowledge graph organizes entities and relationships, so that the system can better capture association between the information, the understanding ability of a user to put forward a question is enhanced, and the user experience is improved. The combination not only improves the accuracy and efficiency of information acquisition, but also provides new assistance for innovation and development of enterprises.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and more specifically, particularly relates to a private knowledge question-answering method combined with a knowledge graph. Background Art

[0002] With the rapid development of big data and artificial intelligence technologies, enterprises have accumulated a huge amount of private data in their daily operations. The types of these data are diverse, including technical documents, business reports, user feedback, and industry research materials. This information contains rich knowledge and insights, which can support enterprises' strategic decision-making, market analysis, and product development. However, traditional question-answering systems face a series of challenges when processing these private data.

[0003] First, traditional question-answering systems are often unsatisfactory in terms of retrieval efficiency. Due to the large amount of data, the system may spend too much time in obtaining relevant information, causing users to wait too long and affecting overall work efficiency. In addition, traditional systems are also insufficient in terms of depth of understanding, and often fail to fully analyze the background and context of users' questions. This lack of understanding makes it impossible for the system to generate high-quality answers, which in turn limits users' information acquisition and decision support capabilities.

[0004] In recent years, retrieval-augmented generation (RAG) technology, as an emerging method, has shown significant advantages. RAG technology effectively improves the performance of question-answering systems by combining knowledge sources with large language models (LLMs). This technology can use information in large-scale data sets to provide users with more accurate and context-related answers, greatly improving the accuracy of information retrieval. However, although RAG systems perform well in many scenarios, they still face challenges when processing complex queries, especially when deep understanding and integration of multiple information sources are required. These limitations result in users still not being able to get satisfactory answers in some business scenarios, affecting the effectiveness of decision-making. Summary of the invention

[0005] In view of the problems existing in the prior art, the purpose of the present invention is to provide a private knowledge question-answering method combined with a knowledge graph, which realizes the effective management and efficient utilization of private knowledge through a series of innovative means such as intelligent identification of entities and relationships, community division and construction of knowledge graphs, vectorized representation of knowledge graphs, and processing of user questions and generation of answers.

[0006] To achieve the above purpose, the present invention provides the following technical solution: a private knowledge question-answering method combined with a knowledge graph, comprising the following steps:

[0007] S1. Private basic corpus collection: According to the field of private knowledge question and answer, obtain the basic corpus information of the corresponding field;

[0008] S2. Fusing the collected basic corpus information: fusing the basic corpus information from different sources through a fusion algorithm, and converting the basic corpus information into text information;

[0009] S3. Establish a knowledge graph: perform entity recognition and relationship extraction on the generated text information, and store the vector data information in the form of a vectorized structure;

[0010] S4. Partitioning the vector data information: performing community division on the vector data information of the identified entities and relationships to obtain community introductions;

[0011] S5. Knowledge graph construction: vectorize the vector data information divided by the community, build a knowledge graph, and use the graph database to store and manage the knowledge graph;

[0012] S6. Processing user questions: Parsing user input questions through natural language processing technology to identify entities, relationships and intent in the questions;

[0013] S7. User question answer retrieval and generation: Based on the user's natural language question, it is converted into a graph query language to query the data in the graph; potential answers are inferred through the inference engine;

[0014] S8. Output of answers to user questions: Generate the final answer through graph query results or inference results, and display it in a user-friendly manner; based on user feedback, continuously optimize the system's query capabilities and the accuracy of the knowledge graph.

[0015] Specifically, the private basic corpus in S1 is a large amount of private data accumulated by the enterprise in daily operations, including technical documents, business reports, user feedback and industry research materials; and the private basic corpus comes from different formats, including text, tables and databases.

[0016] Specifically, after the private basic corpus is collected, the private basic corpus is preprocessed, and the preprocessing includes data filtering, cleaning and missing value compensation;

[0017] The calculation formula of the data filtering is as follows:

[0018] For each data information x i Use median filtering;

[0019]

[0020] Among them, y i represents the basic corpus information after filtering; S represents the number of scales; w sRepresents the weight of scale s, which is used to perform weighted averaging of filtering results of different scales; It represents the result of median filtering the data information at scale s. Specifically, It is all the data information within the window range at scale s, that is, the neighborhood data information at this scale; window s represents the neighborhood window corresponding to scale s, x j Represents the basic corpus information of the input.

[0021] Specifically, the cleaning is performed by dividing the difference of the mean by the standard deviation to obtain the Z-score of the data information, thereby detecting outliers in the basic corpus information;

[0022] Among them, X is the basic corpus information; μ is the mean of the basic corpus information; σ is the standard deviation of the basic corpus information;

[0023] Furthermore, if the outlier is determined as Z>2, the data information is determined to be an outlier; if Z<2, the basic corpus information is determined to be a normal value, and the outlier is removed;

[0024] And the vacancies for outlier removal and missing value compensation are filled by the mean, and the mean is calculated as follows:

[0025] Among them, x i is the i-th value in the basic corpus information; n is the total number of basic corpus information; Represents the sum of all basic corpus information.

[0026] Specifically, the text information processing in S2 includes:

[0027] Data cleaning: Use character set filters to process text to remove non-standard characters, including special symbols, control characters, and extra spaces, identify and delete characters that do not conform to the expected character set, and ensure that only standard characters such as letters, numbers, and spaces are retained in the text;

[0028] Text duplicate checking: Use the Jaccard similarity algorithm to check for text duplicates, which is defined as the ratio of the size of the intersection of two sets to the size of the union; given two text sets and, their Jaccard similarity is expressed by the following formula:

[0029] A is a text set, and B is another text set. First, the text is divided into a set of words or phrases. Then, the intersection |A∩B| and the union |A∪B| of any two text sets are calculated. If the Jaccard similarity J exceeds the preset threshold of 0.8, it is considered as a duplicate text and deleted.

[0030] Text formatting: During the text formatting stage, all texts are uniformly encoded in UTF-8 format to ensure processing under the same character encoding. At the same time, all characters in the text are converted to lowercase to improve the consistency of subsequent processing.

[0031] Specifically, the specific implementation method of entity recognition and relationship extraction involved in S3 is:

[0032] Use a fine-tuned large language model to perform entity recognition and entity relationship extraction on text units. It should include entity descriptions, entity relationships related to the entities, and be vectorized and saved.

[0033] Specifically, the specific implementation method of performing community division on the vector data information involved in S4 is as follows:

[0034] Use the community detection algorithm to divide the identified entities, divide the closely related entities into the same community, and obtain N communities. Theoretically, N ≥ 0. Add a community introduction to the identified community. The community introduction should include community information, entities within the community, and the relationship between entities.

[0035] Specifically, the specific implementation method of the knowledge graph construction involved in S5 is:

[0036] Construct a knowledge graph based on the identified entities, relationships between entities, and community introductions, and select a suitable graph database for storage.

[0037] Specifically, the specific implementation method of identifying the entities, relationships and intentions involved in the problem in S6 is:

[0038] User question preprocessing:

[0039] Clean the text of the questions input by the user, including removing extra spaces and punctuation marks to improve the accuracy of subsequent processing; perform word segmentation and part-of-speech tagging to better understand the user's intention and question structure;

[0040] Key Entity Identification:

[0041] Use a fine-tuned large language model to analyze user questions and identify key entities, including names, places, times, or events;

[0042] Identification of relationships between entities:

[0043] Identify the relationships between entities involved in the user's question to ensure the context of the question is understood;

[0044] Vectorized processing:

[0045] The identified key entities and their relationships are converted into vector representations, and their semantic information is retained using embedding technology; the vectorized results will be used for subsequent similarity calculations and information retrieval.

[0046] Specifically, the user question answer retrieval and generation involved in S7 and the user question answer output in S8 are implemented as follows:

[0047] Cosine similarity calculation: For the entities identified in the user's question, extract their vector representation and compare them with the entity vectors stored in the knowledge graph;

[0048] By calculating cosine similarity, we can identify entities related to the user’s question.

[0049] For the entities identified in the user's question, extract their vector representation and compare them with the entity vectors stored in the knowledge graph. By calculating the cosine similarity, identify the entities related to the user's question. The calculation formula for cosine similarity is:

[0050]

[0051] Where: V A and V B are two vectors, representing the entities in the user question and the entities in the knowledge graph, respectively. · represents the vector dot product, ||V A || represents the Euclidean length of a vector;

[0052] The calculation method is:

[0053] Among them, V i is the i-th element in the vector, and n is the dimension of the vector;

[0054] Relevant entity identification: Based on the calculation results of cosine similarity, the entities most relevant to the user's question are selected, which constitute the basic information for answer generation;

[0055] Setting similarity thresholds to ensure that only highly relevant entities are considered;

[0056] Information collation and preparation: Integrate the identified relevant entities and related information to build a context-rich information set;

[0057] Answer generation: The organized information is fed into a large, fine-tuned language model to generate answers to user questions.

[0058] Answer output: Return the generated answer to the user.

[0059] Technical effects and advantages of the present invention:

[0060] The present invention builds a knowledge graph and combines it with a large language model to deeply understand and efficiently retrieve complex information in private knowledge bases. The knowledge graph organizes entities and relationships, allowing the system to better capture the relationship between information and enhance the ability to understand questions raised by users. This combination not only improves the accuracy and efficiency of information acquisition, but also provides new impetus for the innovation and development of enterprises.

[0061] That is, the large language model is fine-tuned through private basic corpus to facilitate knowledge question-answering by enterprise users, and the private basic corpus is processed for text to facilitate the extraction of entities and relationships. The entities and relationships are first converted into vector data, and then the entities and relationships are classified through community division to improve the connectivity between entities and relationships. Then the vector data information is generated into a knowledge graph, and the large language model analyzes the user's questions to obtain the entities, relationships and intentions in the questions, so as to facilitate the retrieval of entities and relationships in the graph database, facilitate the rapid query of knowledge graph data information, and infer potential answers through the reasoning engine, so that the answers to the questions should be clear and accurate, meet the user's needs as much as possible, and make the answers to the questions richer;

[0062] Community division and knowledge graph construction:

[0063] Divide the identified entities into communities to form closely related entity groups, and add descriptive information to each community to obtain a community introduction;

[0064] Build a knowledge graph based on the identified entities and their relationships, select a suitable graph database for storage and management, and improve the efficiency and accuracy of information retrieval;

[0065] Processing of user questions and generation of answers:

[0066] Ability to pre-process questions raised by users, including text cleaning, word segmentation, and intent recognition;

[0067] Identify entities related to user questions and integrate relevant information by calculating the cosine similarity between the entities identified in the user questions and the entities in the knowledge graph;

[0068] The collated information is fed into the fine-tuned LLM to generate accurate answers to the user’s questions and return them to the user in a timely manner;

[0069] In summary, the present invention realizes the effective management and efficient utilization of private knowledge through a series of innovative means such as intelligent identification of entities and relationships, community division and construction of knowledge graphs, vectorized representation of knowledge graphs, and processing of user questions and generation of answers.

[0070] Further features and advantages of the present invention will become apparent from the following detailed description of exemplary embodiments of the present invention with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 It is a schematic diagram of the step flow provided by the present invention. DETAILED DESCRIPTION

[0072] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention is further described in detail below in conjunction with specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0073] like Figure 1 As shown, the private knowledge question answering method combined with the knowledge graph provided in the embodiment of the present invention includes the following steps:

[0074] S1. Private basic corpus collection: According to the field of private knowledge question and answer, obtain the basic corpus information of the corresponding field;

[0075] S2. Fusing the collected basic corpus information: fusing the basic corpus information from different sources through a fusion algorithm, and converting the basic corpus information into text information;

[0076] Perform systematic text processing on the basic corpus, including data cleaning, text duplication checking, formatting and standardization, to ensure data quality and consistency; cut the processed text into 800 token units, ensuring that each unit contains necessary contextual information, providing a basis for subsequent information extraction and understanding, preserving the semantic integrity of the text, and ensuring the relevance of information during retrieval;

[0077] S3. Establish a knowledge graph: perform entity recognition and relationship extraction on the generated text information, and store the vector data information in the form of a vectorized structure;

[0078] S4. Partitioning the vector data information: performing community division on the vector data information of the identified entities and relationships to obtain community introductions;

[0079] S5. Knowledge graph construction: vectorize the vector data information divided by the community, build a knowledge graph, and use the graph database to store and manage the knowledge graph;

[0080] S6. Processing user questions: Parsing user input questions through natural language processing technology to identify entities, relationships and intent in the questions;

[0081] S7. User question answer retrieval and generation: Based on the user's natural language question, it is converted into a graph query language to query the data in the graph; potential answers are inferred through the inference engine;

[0082] S8. Output of answers to user questions: Generate the final answer through graph query results or inference results, and display it in a user-friendly manner; based on user feedback, continuously optimize the system's query capabilities and the accuracy of the knowledge graph.

[0083] In this embodiment, preferably, the private basic corpus in S1 is a large amount of private data accumulated by the enterprise in daily operations, including technical documents, business reports, user feedback and industry research materials; and the private basic corpus comes from different formats, including text, tables and databases;

[0084] It should be noted that collecting private data accumulated in the daily operations of the enterprise, including technical documents, business reports, user feedback and industry research materials, forms a complete basic corpus, which facilitates fine-tuning of large language models and improves the accuracy of large language models.

[0085] In this embodiment, preferably, after the private basic corpus is collected, the private basic corpus is preprocessed, and the preprocessing includes data filtering, cleaning and missing value compensation;

[0086] The calculation formula of the data filtering is as follows:

[0087] For each data information x i Use median filtering;

[0088]

[0089] Among them, y i represents the basic corpus information after filtering; S represents the number of scales; w s Represents the weight of scale s, which is used to perform weighted averaging of filtering results of different scales; It represents the result of median filtering the data information at scale s. Specifically, It is all the data information within the window range at scale s, that is, the neighborhood data information at this scale; window s represents the neighborhood window corresponding to scale s, x j Represents the basic corpus information of the input;

[0090] It should be noted that filtering the basic corpus information can improve the accuracy of the basic corpus information, making the basic corpus information convenient for subsequent calculation and processing.

[0091] In this embodiment, preferably, the cleaning is performed by dividing the difference of the mean by the standard deviation to obtain the Z-score of the data information, so as to detect outliers in the basic corpus information;

[0092] Among them, X is the basic corpus information; μ is the mean of the basic corpus information; σ is the standard deviation of the basic corpus information;

[0093] Furthermore, if the outlier is determined as Z>2, the data information is determined to be an outlier; if Z<2, the basic corpus information is determined to be a normal value, and the outlier is removed;

[0094] And the vacancies for outlier removal and missing value compensation are filled by the mean, and the mean is calculated as follows:

[0095] Among them, x i is the i-th value in the basic corpus information; n is the total number of basic corpus information; Represents the sum of all basic corpus information;

[0096] It should be noted that by processing and cleaning the basic corpus information, the errors in the basic corpus information are reduced, and mean compensation is performed on outliers and missing values ​​to improve the completeness and accuracy of the basic corpus information.

[0097] In this embodiment, preferably, the text information processing in S2 includes:

[0098] Data cleaning: Use character set filters to process text to remove non-standard characters, including special symbols, control characters, and extra spaces, identify and delete characters that do not conform to the expected character set, and ensure that only standard characters such as letters, numbers, and spaces are retained in the text;

[0099] Text duplicate checking: Use the Jaccard similarity algorithm to check for text duplicates, which is defined as the ratio of the size of the intersection of two sets to the size of the union; given two text sets and, their Jaccard similarity is expressed by the following formula:

[0100] A is a text set, and B is another text set. First, the text is divided into a set of words or phrases. Then, the intersection |A∩B| and the union |A∪B| of any two text sets are calculated. If the Jaccard similarity J exceeds the preset threshold of 0.8, it is considered as a duplicate text and deleted.

[0101] Text formatting: During the text formatting phase, all texts are uniformly encoded in UTF-8 format to ensure that they are processed under the same character encoding. At the same time, all characters in the text are converted to lowercase to improve the consistency of subsequent processing;

[0102] It should be noted that by processing text information, it is convenient to provide accuracy of text information, ensure that only standard characters such as letters, numbers and spaces are retained in the text, and implement duplicate checking of the text, as well as maintain the formatting of the text, so as to facilitate the uniformity of text information and improve the consistency of subsequent processing.

[0103] In this embodiment, preferably, the specific implementation method of entity recognition and relationship extraction involved in S3 is:

[0104] Use a fine-tuned large language model to perform entity recognition and entity relationship extraction on text units. It should include entity descriptions and entity relationships related to the entities, and be vectorized and saved.

[0105] It should be noted that the processed text units are processed using a fine-tuned large language model (LLM), and the specific steps are as follows:

[0106] Entity recognition: Identify key entities in the text (such as names, places, organizations, etc.) to lay the foundation for subsequent relationship extraction;

[0107] Relationship extraction: determine the relationship between entities, including superior-subordinate relationships, association relationships, etc., to build a richer knowledge network;

[0108] Vectorization processing: The identified entities and their relationships are vectorized and saved for subsequent storage and retrieval. Vectorization can use technologies such as Word2Vec and BERT to ensure the retention of semantic information.

[0109] In this embodiment, preferably, the specific implementation method of performing community division on the vector data information involved in S4 is:

[0110] Use community detection algorithm to divide the identified entities, divide the entities with close connections into the same community, and obtain N communities. Theoretically, N ≥ 0. Add community introductions to the identified communities. The community introductions should include community information, entities within the community, and relationships between entities.

[0111] It should be noted that the community detection algorithm is used to divide the identified entities. The specific steps are as follows:

[0112] Dividing entities with close connections into the same community will theoretically form N communities (N ≥ 0);

[0113] For each identified community, add a community introduction, including the main information of the community, entities within the community and their relationships, the functions and uses of the community, etc., to enhance the comprehensibility and practicality of the knowledge graph.

[0114] In this embodiment, preferably, the specific implementation method of the knowledge graph construction involved in S5 is:

[0115] Build a knowledge graph based on the identified entities, relationships between entities, and community introductions, and select a suitable graph database to store them;

[0116] It should be noted that the knowledge graph is constructed based on the identified entities, relationships between entities and community introductions. The specific steps include:

[0117] Select a suitable graph database (such as Neo4j, JanusGraph, etc.) for storage;

[0118] Represent the nodes in the graph as entities and the edges as the relationships between entities, building a queryable knowledge network to support complex queries and data analysis.

[0119] In this embodiment, preferably, the specific implementation method of identifying the entities, relationships and intentions involved in the problem in S6 is:

[0120] User question preprocessing:

[0121] Clean the text of the questions input by the user, including removing extra spaces and punctuation marks to improve the accuracy of subsequent processing; perform word segmentation and part-of-speech tagging to better understand the user's intention and question structure;

[0122] Key Entity Identification:

[0123] Use a fine-tuned large language model to analyze user questions and identify key entities, including names, places, times, or events;

[0124] Identification of relationships between entities:

[0125] Identify the relationships between entities involved in the user's question to ensure the context of the question is understood;

[0126] Vectorized processing:

[0127] The identified key entities and their relationships are converted into vector representations, and their semantic information is retained using embedding technology; the vectorized results will be used for subsequent similarity calculations and information retrieval;

[0128] It should be noted that when the user enters a question, the following processing is performed:

[0129] User question preprocessing:

[0130] Clean the text of the questions input by the user, including removing extra spaces and punctuation marks, to improve the accuracy of subsequent processing; perform word segmentation and part-of-speech tagging, and use tools such as NLTK or spaCy for natural language processing to better understand the user's intention and question structure;

[0131] Key Entity Identification:

[0132] Use the fine-tuned LLM to analyze user questions and identify key entities. These entities may include names, places, times, events, etc. to ensure a comprehensive understanding of user questions;

[0133] Identification of relationships between entities:

[0134] Identify the relationships between entities involved in the user's question to ensure that the context of the question is understood. This can be achieved through a relation extraction model;

[0135] Vectorized processing:

[0136] The identified key entities and their relationships are converted into vector representations, and their semantic information is retained using embedding techniques (such as Word2Vec, BERT, etc.). The vectorized results facilitate subsequent similarity calculations and information retrieval to quickly match user questions.

[0137] In this embodiment, preferably, the specific implementation method of the user question answer retrieval and generation involved in S7 and the user question answer output in S8 is:

[0138] Cosine similarity calculation: For the entities identified in the user's question, extract their vector representation and compare them with the entity vectors stored in the knowledge graph;

[0139] By calculating cosine similarity, we can identify entities related to the user’s question.

[0140] For the entities identified in the user's question, extract their vector representation and compare them with the entity vectors stored in the knowledge graph. By calculating the cosine similarity, identify the entities related to the user's question. The calculation formula for cosine similarity is:

[0141]

[0142] Where: V A and V B are two vectors, representing the entities in the user question and the entities in the knowledge graph, respectively. · represents the vector dot product, ||V A || represents the Euclidean length of a vector;

[0143] The calculation method is:

[0144] Among them, V i is the i-th element in the vector, and n is the dimension of the vector;

[0145] Relevant entity identification: Based on the calculation results of cosine similarity, the entities most relevant to the user's question are selected, which constitute the basic information for answer generation;

[0146] Setting similarity thresholds to ensure that only highly relevant entities are considered;

[0147] Information collation and preparation: Integrate the identified relevant entities and related information to build a context-rich information set;

[0148] Answer generation: The organized information is fed into a large, fine-tuned language model to generate answers to user questions.

[0149] Answer output: return the generated answer to the user;

[0150] It should be noted that related entity identification:

[0151] Based on the calculation results of cosine similarity, select the entities most relevant to the user's question. These entities form the basic information for answer generation, and set the similarity threshold to 0.7 to ensure that only highly relevant entities are considered;

[0152] Information collation and preparation:

[0153] Integrate the identified related entities and related information to build a context-rich information set for subsequent answer generation. The organized information should include entity descriptions, attributes, and relationships between entities to form rich background information;

[0154] Answer generation:

[0155] The collated information is fed into a large, fine-tuned language model to generate answers to user questions. The generated answers should be clear, accurate, and as consistent as possible with user needs.

[0156] The specific operation process of the present invention is as follows:

[0157] Step 1: Private basic corpus collection: According to the field of private knowledge question and answer, obtain the basic corpus information of the corresponding field;

[0158] Step 2: Fusing the collected basic corpus information: fusing the basic corpus information from different sources through a fusion algorithm, and converting the basic corpus information into text information;

[0159] Step 3: Establish a knowledge graph: perform entity recognition and relationship extraction on the generated text information, and store the vector data information in the form of a vectorized structure;

[0160] The calculation of entity recognition and relationship extraction is as follows:

[0161]

[0162] Among them, TF(x i ,d) reflects the importance of a word in the data information. The more frequently it appears, the higher the TF value, x i represents the word for calculation, and d represents the word x i The number of times it appears in the data information, It is used to measure the prevalence of a word in the entire corpus. The higher the value, the less common the word is in the corpus, expressed as the total number of texts in the corpus, DF(x i ) means that it contains word x i The number of texts;

[0163] If a word appears frequently in the data information to be detected and calculated, but appears less frequently in other texts, TF(x i ,The value of d) will be very high, indicating the importance of the word to the data information that needs to be detected and calculated;

[0164] On the contrary, if the word frequency is low or the word appears in many texts, then TF(x i ,The value of d) will be low, indicating that the word is not important to the data information that needs to be detected and calculated.

[0165] Step 4: Partitioning the vector data information: Perform community division on the vector data information of the identified entities and relationships to obtain community introductions;

[0166] Step 5: Knowledge graph construction: vectorize the vector data information divided by the community, construct a knowledge graph, and use a graph database to store and manage the knowledge graph;

[0167] Step 6: Process the user's questions: parse the user's input questions through natural language processing technology to identify the entities, relationships and intent in the questions;

[0168] Step 7: Retrieve and generate answers to user questions: Based on the user's natural language questions, convert them into graph query language to query the data in the graph; infer potential answers through the inference engine;

[0169] Step 8: Output answers to user questions: Generate the final answer through graph query results or inference results, and display it in a user-friendly manner; based on user feedback, continuously optimize the system's query capabilities and the accuracy of the knowledge graph.

[0170] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A private knowledge question-answering method combined with a knowledge graph, characterized in that: The steps include: S1. Private basic corpus collection: According to the field of private knowledge question and answer, obtain the basic corpus information of the corresponding field; S2. Fusing the collected basic corpus information: fusing the basic corpus information from different sources through a fusion algorithm, and converting the basic corpus information into text information; S3. Establish a knowledge graph: perform entity recognition and relationship extraction on the generated text information, and store the vector data information in the form of a vectorized structure; S4. Partitioning the vector data information: performing community division on the vector data information of the identified entities and relationships to obtain community introductions; S5. Knowledge graph construction: vectorize the vector data information divided by the community, build a knowledge graph, and use the graph database to store and manage the knowledge graph; S6. Processing user questions: Parsing user input questions through natural language processing technology to identify entities, relationships and intent in the questions; S7. User question answer retrieval and generation: Based on the user's natural language question, it is converted into a graph query language to query the data in the graph; potential answers are inferred through the inference engine; S8. Output of answers to user questions: Generate the final answer through graph query results or inference results, and display it in a user-friendly manner; based on user feedback, continuously optimize the system's query capabilities and the accuracy of the knowledge graph.

2. The private knowledge question answering method combined with knowledge graph according to claim 1 is characterized by: The private basic corpus in S1 is a large amount of private data accumulated by the enterprise in its daily operations, including technical documents, business reports, user feedback, and industry research materials; and the private basic corpus comes from different formats, including text, tables, and databases.

3. The private knowledge question answering method combined with knowledge graph according to claim 2 is characterized by: After the private basic corpus is collected, the private basic corpus is preprocessed, and the preprocessing includes data filtering, cleaning and missing value compensation; The calculation formula of the data filtering is as follows: For each data information x i Use median filtering; Among them, y i represents the basic corpus information after filtering; S represents the number of scales; w s Represents the weight of scale s, which is used to perform weighted averaging of filtering results of different scales; It represents the result of median filtering the data information at scale s. Specifically, It is all the data information within the window range at scale s, that is, the neighborhood data information at this scale; window s represents the neighborhood window corresponding to scale s, x j Represents the basic corpus information of the input.

4. The private knowledge question answering method combined with knowledge graph according to claim 3 is characterized by: The cleaning is performed by dividing the difference of the mean by the standard deviation to obtain the Z-score of the data information, thereby detecting outliers in the basic corpus information; Among them, X is the basic corpus information; μ is the mean of the basic corpus information; σ is the standard deviation of the basic corpus information; Furthermore, if the outlier is determined as Z>2, the data information is determined to be an outlier; if Z<2, the basic corpus information is determined to be a normal value, and the outlier is removed; And the vacancies for outlier removal and missing value compensation are filled by the mean, and the mean is calculated as follows: Among them, x i is the i-th value in the basic corpus information; n is the total number of basic corpus information; Represents the sum of all basic corpus information.

5. The private knowledge question answering method combined with knowledge graph according to claim 1 is characterized in that: The text information processing in S2 includes: Data cleaning: Use character set filters to process text to remove non-standard characters, including special symbols, control characters, and extra spaces, identify and delete characters that do not conform to the expected character set, and ensure that only standard characters such as letters, numbers, and spaces are retained in the text; Text duplicate checking: Use the Jaccard similarity algorithm to check for text duplicates, which is defined as the ratio of the size of the intersection of two sets to the size of the union; given two text sets and, their Jaccard similarity is expressed by the following formula: A is a text set, and B is another text set. First, the text is divided into a set of words or phrases. Then, the intersection |A∩B| and the union |A∪B| of any two text sets are calculated. If the Jaccard similarity J exceeds the preset threshold of 0.8, it is considered as a duplicate text and deleted. Text formatting: During the text formatting stage, all texts are uniformly encoded in UTF-8 format to ensure processing under the same character encoding. At the same time, all characters in the text are converted to lowercase to improve the consistency of subsequent processing.

6. The private knowledge question answering method combined with knowledge graph according to claim 1 is characterized by: The specific implementation method of entity recognition and relationship extraction involved in S3 is: Use a fine-tuned large language model to perform entity recognition and entity relationship extraction on text units. It should include entity descriptions, entity relationships related to the entities, and be vectorized and saved.

7. The private knowledge question answering method combined with knowledge graph according to claim 1 is characterized by: The specific implementation method of community division of the vector data information involved in S4 is: Use the community detection algorithm to divide the identified entities, divide the closely related entities into the same community, and obtain N communities. Theoretically, N ≥ 0. Add a community introduction to the identified community. The community introduction should include community information, entities within the community, and the relationship between entities.

8. The private knowledge question answering method combined with knowledge graph according to claim 1, characterized in that: The specific implementation method of the knowledge graph construction involved in S5 is: Construct a knowledge graph based on the identified entities, relationships between entities, and community introductions, and select a suitable graph database for storage.

9. The private knowledge question answering method combined with knowledge graph according to claim 1, characterized in that: The specific implementation method of identifying the entities, relationships and intentions involved in the problem in S6 is: User question preprocessing: Clean the text of the questions input by the user, including removing extra spaces and punctuation marks to improve the accuracy of subsequent processing; perform word segmentation and part-of-speech tagging to better understand the user's intention and question structure; Key Entity Identification: Use a fine-tuned large language model to analyze user questions and identify key entities, including names, places, times, or events; Identification of relationships between entities: Identify the relationships between entities involved in the user's question to ensure the context of the question is understood; Vectorized processing: The identified key entities and their relationships are converted into vector representations, and their semantic information is retained using embedding technology; the vectorized results will be used for subsequent similarity calculations and information retrieval.

10. The private knowledge question answering method combined with knowledge graph according to claim 1, characterized in that: The specific implementation methods of the user question answer retrieval and generation involved in S7 and the user question answer output in S8 are: Cosine similarity calculation: For the entities identified in the user's question, extract their vector representation and compare them with the entity vectors stored in the knowledge graph; By calculating cosine similarity, we can identify entities related to the user’s question. For the entities identified in the user's question, extract their vector representation and compare them with the entity vectors stored in the knowledge graph. By calculating the cosine similarity, identify the entities related to the user's question. The calculation formula for cosine similarity is: Where: V A and V B are two vectors, representing the entities in the user question and the entities in the knowledge graph, respectively. · represents the vector dot product, ||V A || represents the Euclidean length of a vector; The calculation method is: Among them, V i is the i-th element in the vector, and n is the dimension of the vector; Relevant entity identification: Based on the calculation results of cosine similarity, the entities most relevant to the user's question are selected, which constitute the basic information for answer generation; Setting similarity thresholds to ensure that only highly relevant entities are considered; Information collation and preparation: Integrate the identified relevant entities and related information to build a context-rich information set; Answer generation: The organized information is fed into a large, fine-tuned language model to generate answers to user questions. Answer output: Return the generated answer to the user.

Images