Knowledge retrieval method and device, computer equipment, storage medium and product
By employing a dual-path parallel analysis of a natural language processing model and a keyword extraction model, the problem of separating knowledge graph construction and retrieval was solved, enabling real-time knowledge graph construction and efficient data construction and feedback, thereby improving knowledge retrieval performance.
Patent Information
- Application Number
- CN202510763091.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-09
- Publication Date
- 2025-11-11
Smart Images
Figure CN120930762A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of knowledge retrieval technology, and in particular to a knowledge retrieval method, apparatus, computer equipment, storage medium, and product. Background Technology
[0002] In related technical solutions, the construction of knowledge graphs and knowledge extraction are relatively independent and separate processes. That is, knowledge is first stored in a dedicated database table, and then a knowledge graph is generated based on the selected database and icons. Data mining is then performed based on the generated knowledge graph. However, this method of knowledge mining based on knowledge graphs is inefficient, has low data utilization efficiency, and cannot guarantee timeliness, resulting in poor knowledge retrieval performance. Summary of the Invention
[0003] This application provides a knowledge retrieval method, apparatus, computer equipment, storage medium, and product, which can solve the technical problem of low data utilization efficiency and inability to guarantee timeliness, resulting in poor knowledge retrieval performance. The technical solution is as follows.
[0004] On the one hand, a knowledge retrieval method is provided, the method comprising: The search is performed based on the search keywords entered by the user, and the search results are obtained. A relevance analysis is performed on the search results and the search keywords to obtain a search result dataset. The search result dataset contains the n search results with the highest relevance to the search keywords, where n is a positive integer. In response to a first search result containing no keyword record in the search result dataset, the first search result is sent to a natural language processing model and a keyword extraction model for word segmentation processing to obtain a first word segmentation result and a second word segmentation result corresponding to the first search result; the keyword extraction model is a model with text keyword extraction function. Based on the first word segmentation result and the second word segmentation result, keyword filtering is performed to generate a display keyword set. The display keyword set includes a first display keyword, which is a keyword that is common to both the first word segmentation result and the second word segmentation result. Based on the set of displayed keywords, a set of target keywords is determined, and a knowledge graph is constructed based on the set of target keywords. In response to receiving a request to view the knowledge graph, the knowledge graph is returned to the user.
[0005] On the other hand, a knowledge retrieval device is provided, the device comprising: The search module is used to perform searches based on user-input keywords and obtain search results. The analysis module is used to perform relevance analysis on the search results and the search keywords to obtain a search result dataset. The search result dataset contains the n search results with the highest relevance to the search keywords, where n is a positive integer. The word segmentation module is used to respond to a first search result in the search result dataset containing a record without keywords, and to send the first search result to the natural language processing model and the keyword extraction model respectively for word segmentation processing to obtain the first word segmentation result and the second word segmentation result corresponding to the first search result; the keyword extraction model is a model with text keyword extraction function. The filtering module is used to filter keywords based on the first word segmentation result and the second word segmentation result, and generate a set of display keywords. The set of display keywords includes a first display keyword, which is a keyword that is common to both the first word segmentation result and the second word segmentation result. The set determination module is used to determine the target keyword set based on the displayed keyword set, so as to construct a knowledge graph based on the target keyword set; The feedback module is used to respond to a received knowledge graph viewing request by providing the knowledge graph back to the user.
[0006] In one possible implementation, the device further includes: The relevance calculation module is used to calculate the relevance between other keywords and knowledge keywords. The other keywords are keywords other than the first displayed keywords in the first word segmentation result and the second word segmentation result. The knowledge keywords are known keywords corresponding to vertical domain data determined based on the search result dataset. The keyword determination module is used to identify other keywords with relevance values greater than the relevance threshold as second display keywords and add them to the display keyword set.
[0007] In one possible implementation, the set determination module includes: The consistency filtering submodule is used to perform consistency filtering on the keywords in the display keyword set through the entity constraint benchmark thesaurus corresponding to the search result dataset to obtain entity node keywords; the entity constraint benchmark thesaurus is constructed based on keywords extracted from industry domain data, vertical domain data and custom data; The set determination submodule is used to determine the target keyword set based on the entity node keywords.
[0008] In one possible implementation, the set determines the sub-module, including: The term frequency acquisition unit is used to acquire the term frequency of keywords for each entity node in the search result dataset. The filtering unit is used to filter entity node keywords based on the word frequency of keywords in each entity node to obtain the target keyword set.
[0009] In one possible implementation, the device further includes: The first adding module is used to add keywords from the keyword records corresponding to the second search result to the target keyword set in response to the second search result containing keyword records in the search result dataset.
[0010] In one possible implementation, the device further includes: The second adding module is used to add the remaining keywords to the preparatory library of the entity constraint benchmark terminology library corresponding to the search result dataset, so as to update the entity constraint benchmark terminology library corresponding to the search result dataset based on the keywords in the preparatory library; the remaining keywords are the keywords in the first word segmentation result and the second word segmentation result other than the first display keyword and the second display keyword.
[0011] In one possible implementation, the device further includes: The fuzzy matching module is used to perform fuzzy matching on non-entity node keywords in the display keyword set based on the entity constraint benchmark lexicon, and obtain the fuzzy matching results of each non-entity node keyword. The third addition module is used to add non-entity node keywords that indicate successful matching in the fuzzy matching results to the preparatory library of the entity constraint benchmark terminology corresponding to the retrieval result dataset.
[0012] In one possible implementation, the device further includes: An extension module is used to respond to receiving a selection operation on a target entity node in the knowledge graph, and to perform hierarchical extension of the knowledge graph based on the target entity node; the hierarchical extension refers to further constructing the knowledge graph using the keywords corresponding to the target entity node as search keywords.
[0013] On the other hand, a computer device is provided, the computer device including a processor and a memory, the memory storing at least one computer program, the at least one computer program being loaded and executed by the processor to implement the knowledge retrieval method described above.
[0014] On the other hand, a computer-readable storage medium is provided, wherein at least one computer program is stored in the computer-readable storage medium, the computer program being loaded and executed by a processor to implement the knowledge retrieval method described above.
[0015] On the other hand, a computer program product is provided, the computer program product including a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions, which, when executed by a computer, cause the computer to execute to implement the knowledge retrieval method provided in the various optional implementations described above.
[0016] The knowledge retrieval method provided in this application, after receiving user input and obtaining search keywords, performs a search based on those keywords to obtain search results. Then, it analyzes the search results based on relevance to obtain a search result dataset. For the first search result dataset containing no key records, it performs word segmentation using both a natural language processing model and a keyword extraction model, obtaining a first word segmentation result and a second word segmentation result. The keywords common to both the first and second word segmentation results are identified as the first display keywords of the first search result and added to a large display keyword set. Subsequently, a knowledge graph is constructed based on the target keyword set determined from the display keyword set. Upon receiving a request to view the knowledge graph... When a request is made, the knowledge graph is fed back to the user. This method deeply integrates knowledge graph construction with the real-time retrieval process, significantly improving the efficiency of knowledge mining and the timeliness of data use. Specifically, through parallel analysis of a natural language processing model and a keyword extraction model, highly relevant keywords are directly extracted from the user's search results, avoiding the lag problem of traditional static knowledge graphs. A cross-validation mechanism is used to filter and display the keyword set, ensuring the quality of the basic data for building the knowledge graph, thereby improving the quality of knowledge graph construction and the matching degree between the knowledge graph and user search needs. Overall, a closed-loop optimization of knowledge acquisition-analysis-construction-feedback is achieved, enabling knowledge services to have real-time response capabilities, thus improving the knowledge retrieval effect. It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description
[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0018] Figure 1 A flowchart of a knowledge retrieval method provided in an exemplary embodiment of this application is shown; Figure 2 This illustration shows a schematic diagram of the training process of a keyword extraction model provided in an exemplary embodiment of this application; Figure 3 A flowchart of a knowledge retrieval method provided by another exemplary embodiment of this application is shown; Figure 4This illustration shows a schematic diagram illustrating the creation of an entity constraint reference lexicon according to an exemplary embodiment of this application; Figure 5 A schematic diagram of a knowledge retrieval process provided by an exemplary embodiment of this application is shown; Figure 6 A block diagram of a knowledge retrieval apparatus provided in an exemplary embodiment of this application is shown; Figure 7 This application shows a structural block diagram of a computer device according to an exemplary embodiment. Figure 8 A structural block diagram of a computer device is shown in another exemplary embodiment of this application. Detailed Implementation
[0019] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of methods consistent with some aspects of this application as detailed in the appended claims.
[0020] Data mining and knowledge mining refer to the process of extracting useful information and knowledge from large amounts of data. Data mining includes data preprocessing, pattern discovery, data analysis, and result evaluation, while knowledge mining further transforms this information into actionable knowledge. Data mining focuses on discovering hidden patterns and relationships in data, while knowledge mining emphasizes applying these patterns and relationships to practical problems.
[0021] A knowledge graph is a structured database that represents entities and relationships. Entities in a knowledge graph can be people, organizations, locations, etc., and relationships can be attributes, categories, similarity, etc. Knowledge graphs can be used to store and query relationships between entities, and can also be used for reasoning to derive new knowledge.
[0022] Knowledge graph-based data mining, or knowledge mining, refers to the process of extracting meaningful information from large amounts of structured or unstructured data and integrating this information into a knowledge graph. In existing technologies, building a knowledge graph is a complex process involving multiple stages, including data processing, knowledge extraction, and storage management. Illustratively, the process can be roughly divided into several stages: data preprocessing, entity relationship identification, graph database storage, and optimization. Ultimately, the extracted entities and relationships are stored in a graph database, which provides efficient node and edge querying and storage capabilities. Common graph databases include Neo4j and ArangoDB. Deep learning technology is mainly used in knowledge graph construction for entity recognition, relationship extraction, and knowledge fusion. However, the construction of knowledge graphs and knowledge extraction are relatively independent and separate processes. That is, the knowledge graph is built first, and then retrieval is performed based on the knowledge graph. Since the knowledge graph is pre-built, the sorting of knowledge data requires the extraction of entities and relationships through manual labor, machine learning, and professional tools, which is inefficient and time-consuming. During debugging and optimization, since the entities and relationships that constitute the indicator graph are labeled manually and machine learning is used, when there are problems with the knowledge graph in terms of logic or performance, a lot of time and effort is needed to readjust the labeling rules or retrain the machine learning. When model performance is poor, users find it difficult to pinpoint the problem and make targeted improvements. Furthermore, knowledge graphs in related technologies, which involve human and machine extraction of entities and relationships, are typically designed for specific domains or tasks. When expanding to new domains or tasks, re-annotation is required, limiting the flexibility and scalability of knowledge graphs. Moreover, because knowledge graphs are pre-built and require the use of structured data sources to extract ontology and pattern information from high-quality data before adding it to the knowledge base, existing knowledge graph technologies cannot be dynamically updated in real time. This results in low data utilization and unreliable data timeliness, leading to poor knowledge retrieval performance.
[0023] Based on this, embodiments of this application provide a knowledge retrieval method that can solve all or part of the problems in the prior art. Figure 1 This application illustrates a flowchart of a knowledge retrieval method provided in an exemplary embodiment. This method can be executed by a computer device, which can be implemented as a server or a terminal, such as... Figure 1 As shown, the knowledge retrieval method may include the following steps.
[0024] Step 110: Perform a search based on the search keywords entered by the user and obtain the search results.
[0025] The search keywords can be input by the user through the interactive interface provided by the computer device, such as manually entering them in the search box, using voice recognition for voice input, importing and organizing files containing search keywords, or selecting them from the history, etc.
[0026] After receiving search keywords, the computer device can perform a search using a target search method to obtain search results. This target search method can be keyword search, exact match search, fuzzy search, etc. This application embodiment does not limit the search method. Exact match refers to directly querying files that completely contain the search keywords. Fuzzy match can be based on edit distance or semantic vector to perform file search and obtain search results.
[0027] After obtaining the search results, the computer device can return the search results to the user. In one possible implementation, if the number of search results exceeds a threshold, the computer device can sort the search results based on relevance and return the most relevant search results to the user.
[0028] Step 120: Perform a relevance analysis on the search results and the search keywords to obtain a search result dataset. This search result dataset contains the n search results with the highest relevance to the search keywords, where n is a positive integer.
[0029] The relevance between the search results and the search keywords is obtained during the search process. The process of conducting relevance analysis can be a process of filtering the search results based on the relevance of each search result. That is, the top n search results with the highest relevance to the search keywords are used as the filtering results to form a search result dataset, where n is a positive integer. The value of n can be set based on actual needs, and this application does not impose any restrictions on it.
[0030] Step 130: In response to the first search result containing no keyword records in the search result dataset, the first search result is sent to the natural language processing model and the keyword extraction model respectively for word segmentation processing to obtain the first word segmentation result and the second word segmentation result corresponding to the first search result; the keyword extraction model is a model with text keyword extraction function.
[0031] In this context, "keyword record" refers to a search result with explicit keyword annotations, while "no keyword record" refers to a search result without explicit keyword annotations. In this case, keyword extraction is required. In this embodiment, to improve the accuracy of keyword extraction, the computer device performs word segmentation using both a natural language processing model and a keyword extraction model to obtain two different word segmentation results. That is, during word segmentation, the computer device iterates through the first search result in the search result dataset and sequentially inputs each first search result into the natural language processing model and the keyword extraction model for word segmentation, thereby obtaining the first and second word segmentation results corresponding to each first search result.
[0032] Natural language processing models, such as BERT and GPT, are deep learning-based text processing models that can be used for tasks like word segmentation, part-of-speech tagging, and named entity recognition. Keyword extraction models are deep learning-based keyword extraction models. For example, the training process for a keyword extraction model can include three stages: training data preparation, text preprocessing, and model training. Figure 2 This illustration shows a schematic diagram of the training process of a keyword extraction model provided in an exemplary embodiment of this application, such as... Figure 2As shown, in the training data preparation stage, the computer device can perform data collection and data format cleaning to obtain the text data to be processed. Among them, when performing data collection, the computer device can collect data from multiple data sources. For example, it can collect various types and a certain volume of text data through journals, newspapers, public websites on the Internet, etc. When performing data format cleaning, it can unify the format of the text data. In the text preprocessing stage, the computer device can perform word segmentation on the text data to be processed through a word segmentation algorithm, splitting the text data into words or phrases. In this process, it can also remove stop words, such as stop words like "de" (of), "le" (particle indicating completion), etc., to improve the accuracy of subsequent text keyword extraction. This word segmentation algorithm can be the BBPE (Byte - Level Byte Pair Encoding) algorithm or the ULM (Unigram Language Model) algorithm. After the word segmentation process, the computer device can extract text features from the word segmentation results. These text features are used to indicate the word frequency or importance of the word segmentation results, etc. For example, it can determine the text features of the word segmentation results by calculating the TF-IDF (Term Frequency-Inverse Document Frequency) value. Among them, the TF-IDF value is used to represent the importance of the word segmentation results for the document. In the model training stage, the computer device can train the natural language model to extract keywords based on a general natural language large model, using the text data to be processed and the text features of each word segmentation result. It can optimize the model performance by adjusting algorithm parameters, increasing training, etc. After meeting the training completion conditions, a keyword extraction model is obtained.
[0033] Step 140: Based on the first word segmentation result and the second word segmentation result, perform keyword screening to generate a display keyword set. The display keyword set contains a first display keyword, and the first display keyword is the keyword shared by the first word segmentation result and the second word segmentation result.
[0034] Traverse each retrieval result in the retrieval result dataset. Each retrieval result has a first word segmentation result and a second word segmentation result. Each word segmentation result can contain multiple keywords corresponding to the retrieval result. When performing keyword screening, the computer device can determine the keyword shared by the first word segmentation result and the second word segmentation result as the first display keyword of the retrieval result. After completing the keyword screening of each first retrieval result in the above manner, the first display keyword of each first retrieval result is obtained and added to the display keyword set.
[0035] Step 150: Determine a target keyword set based on the display keyword set, and construct a knowledge graph based on the target keyword set.
[0036] In one possible implementation, the computer device can directly determine the set of display keywords as the set of target keywords, thereby identifying each keyword in the set of target keywords as an entity node of the knowledge graph, establishing relationships between the entity nodes, and thus completing the construction of the indicator graph. The relationships between the entity nodes can be determined based on the contextual relationships, co-occurrence relationships, grammatical rules, semantic patterns, etc. between keywords.
[0037] In another possible implementation, the computer device can further filter the set of displayed keywords to obtain a set of target keywords, thereby improving the relevance between the keywords in the search results and the search keywords, and thus improving the accuracy of the constructed knowledge graph.
[0038] Step 160: In response to receiving a knowledge graph viewing request, the knowledge graph is returned to the user.
[0039] In one possible implementation, the computer device can provide a knowledge graph viewing portal to the user while providing search results. Upon receiving a knowledge graph viewing request from the user via this portal, the device will provide the user with the generated knowledge graph corresponding to the search results. Alternatively, in another possible implementation, the knowledge graph viewing request can be automatically triggered based on user-input search keywords. In this case, after completing the search based on the search keywords and obtaining the knowledge graph corresponding to the search results, the computer device will automatically provide the knowledge graph to the user. In another possible implementation, the computer device can display the knowledge graph of the search results through a front-end visualization tool. For example, this front-end visualization tool could be a web-based visualization tool such as ECharts (Enterprise Charts) or AntV (Ant Visualization).
[0040] In summary, the knowledge retrieval method provided in this application, after receiving user input and obtaining search keywords, performs a search based on those keywords to obtain search results. Then, it analyzes the search results based on relevance to obtain a search result dataset. For the first search result containing no key records within the dataset, it performs word segmentation using both a natural language processing model and a keyword extraction model, obtaining a first word segmentation result and a second word segmentation result for the first search result. The keywords common to both the first and second word segmentation results are identified as the first display keywords for the first search result and added to a large set of display keywords. Subsequently, a knowledge graph is constructed based on the target keyword set determined from the display keyword set. Upon receiving a query from the knowledge graph... When a request is received, the knowledge graph is fed back to the user. This method deeply integrates knowledge graph construction with the real-time retrieval process, significantly improving the efficiency of knowledge mining and the timeliness of data usage. Specifically, through parallel analysis of a natural language processing model and a keyword extraction model, highly relevant keywords are directly extracted from the user's search results, avoiding the lag problem of traditional static knowledge graphs. A cross-validation mechanism is used to filter and display the keyword set, ensuring the quality of the basic data for building the knowledge graph, thereby improving the quality of knowledge graph construction and the matching degree between the knowledge graph and user search needs. Overall, a closed-loop optimization of knowledge acquisition-analysis-construction-feedback is achieved, enabling knowledge services to have real-time responsiveness and thus improving the knowledge retrieval effect.
[0041] In one possible implementation, after obtaining the set of display keywords, the computer device can further filter the keywords in the set of display keywords through entity constraints and / or word frequency constraints to obtain the set of target keywords, thereby improving the relevance between the keywords in the set of target keywords and the search keywords, and thus improving the accuracy of the knowledge graph constructed based on the set of target keywords and improving the knowledge retrieval effect. Figure 3 A flowchart of a knowledge retrieval method provided in another exemplary embodiment of this application is shown. This method can be executed by a computer device, which can be implemented as a server or a terminal, such as... Figure 3 As shown, the method may include the following steps.
[0042] Step 310: Perform a search based on the search keywords entered by the user and obtain the search results.
[0043] After receiving search keywords input by a user, a computer device can perform a search in a knowledge base to obtain search results. This knowledge base may include at least one type of structured database or unstructured database.
[0044] Step 320: Perform a relevance analysis on the search results and the search keywords to obtain a search result dataset. This search result dataset contains the n search results with the highest relevance to the search keywords, where n is a positive integer.
[0045] Step 330: In response to the first search result containing no keyword records in the search result dataset, the first search result is sent to the natural language processing model and the keyword extraction model respectively for word segmentation processing to obtain the first word segmentation result and the second word segmentation result corresponding to the first search result; the keyword extraction model is a model with text keyword extraction function.
[0046] For the first search results in the search results dataset that do not contain any keyword records, the computer device can use these first search results as a knowledge data source and send them to the natural language processing model and the keyword extraction model. The natural language processing model performs word segmentation on each first search result to obtain the first word segmentation result corresponding to each first search result; the keyword extraction model extracts keywords from each first search result to obtain the second word segmentation result corresponding to each first search result.
[0047] Step 340: Based on the first word segmentation result and the second word segmentation result, keyword filtering is performed to generate a set of display keywords. The set of display keywords includes the first display keyword, which is a keyword that is common to both the first word segmentation result and the second word segmentation result.
[0048] The computer device compares the first word segmentation result with the second word segmentation result, and retains the keywords that are the same in both (i.e. the first display keywords) and adds them to the display keyword set.
[0049] Step 350: Calculate the relevance between other keywords and knowledge keywords. Other keywords are keywords other than the first displayed keyword in the first and second word segmentation results. The knowledge keywords are known keywords corresponding to vertical domain data determined based on the search result dataset.
[0050] For keywords that are not displayed first, the computer device can determine whether other keywords have value. In this application embodiment, the relevance between each other keyword and the knowledge keyword can be calculated. The relevance between each other keyword and the knowledge keyword can be calculated by semantic similarity or word vector similarity. This application does not limit this.
[0051] The knowledge keywords are known keywords in the vertical domain data corresponding to the retrieval result dataset. For example, these knowledge keywords can be words that appear frequently in the vertical domain, words that exist in the knowledge base or standard terminology of the vertical domain, or words that have special meanings in the vertical domain. For example, in the field of medical imaging diagnosis, the corresponding knowledge keywords can include CT plain scan, lung nodules, MRI enhancement, etc. In the field of financial risk control, the corresponding knowledge keywords can include credit scoring, Basel Accords, probability of default, etc.
[0052] Step 360: Other keywords with relevance values greater than the relevance threshold are identified as second display keywords and added to the display keyword set.
[0053] Keywords with high relevance to knowledge keywords can be identified as new knowledge points. Other keywords with relevance values greater than the relevance threshold are added to the display keyword set to improve the real-time performance of the knowledge graph built based on this display keyword set.
[0054] Step 370: Determine the target keyword set based on the display keyword set, and construct a knowledge graph based on the target keyword set.
[0055] In one possible implementation, the computer device can further filter the displayed keyword set using at least one of entity constraints or word frequency constraints to determine the target keyword set. The following examples illustrate the sequential filtering of the displayed keyword set using entity constraints and word frequency constraints, but do not restrict the order in which the keyword filtering is performed.
[0056] The process of determining the target keyword set through entity constraints can be implemented as follows: The keywords in the displayed keyword set are screened for consistency by using the entity constraint benchmark thesaurus corresponding to the retrieval result dataset, and the entity node keywords are obtained. This entity constraint benchmark thesaurus is constructed based on keywords extracted from industry domain data, vertical domain data, and custom data. Determine the target keyword set based on entity node keywords.
[0057] This entity constraint benchmark thesaurus can be constructed based on industry or domain data. Figure 4 This illustration shows a schematic diagram illustrating the creation of an entity constraint reference lexicon according to an exemplary embodiment of this application, such as... Figure 4As shown, the creation of an entity constraint benchmark lexicon includes three stages: data preparation, entity extraction, and lexicon construction. In the data preparation stage, three types of data sources can be linked or input: industry domain data, vertical domain data, and custom data. Industry domain data can be general data such as industry standards and white papers; vertical domain data can be business-related proprietary datasets; and custom data can be user-defined supplementary data. In the entity extraction stage, the computer can perform entity extraction in parallel through dual channels: manual processing or large-scale model extraction. Manual processing can involve entity annotation by relevant personnel or extraction based on established entity extraction rules. Large-scale model extraction can involve automatic entity recognition using a large model. In the lexicon construction stage, the computer can first deduplicate and denoise the entities extracted in the entity extraction stage, such as merging synonyms and removing low-frequency words. Then, the deduplicated and denoised entities are stored in a structured manner to obtain the entity constraint benchmark lexicon. The entities in this entity constraint benchmark lexicon can be continuously updated based on data updates from the data sources.
[0058] By performing consistency screening on the keywords in the display keyword set through the entity constraint benchmark thesaurus corresponding to the retrieval result dataset, the entity constraint benchmark terms in the entity constraint benchmark thesaurus can be precisely matched with the keywords in the display keyword set. That is, if the keyword is completely consistent with the entity constraint benchmark terms, the keyword is determined as the entity node keyword, thereby improving the domain relevance and accuracy of the keywords.
[0059] In one possible implementation, the computer device can determine the entity node keywords obtained through entity constraint filtering as target keywords in the target keyword set, and construct a knowledge graph based on this.
[0060] Alternatively, in another possible implementation, to further improve the relevance between the knowledge graph and the search keywords, the computer device can further filter keywords through word frequency constraints. This process can be implemented as follows: Obtain the word frequency of keywords for each entity node in the search results dataset; The target keyword set is obtained by filtering entity node keywords based on the word frequency of each entity node keyword.
[0061] Among them, term frequency is used to indicate the frequency of occurrence of entity node keywords in the search results dataset. The higher the frequency, the higher the corresponding term frequency. In another possible implementation, when calculating term frequency, different occurrence positions of entity node keywords can be assigned different weights. For example, the weight of entity node keywords in the title is greater than the weight of entity node keywords in the content. Term frequency is calculated based on the number of occurrences, occurrence positions, and weights corresponding to each occurrence position of entity node keywords.
[0062] When filtering keywords for each entity node based on word frequency, in one possible implementation, the computer device can sort each entity node according to word frequency and determine the top few keywords with the highest word frequency as target keywords to form a target keyword set; or, in another possible implementation, the computer device can set a word frequency threshold and determine the entity node keywords that are higher than the word frequency threshold as target keywords to form a target keyword set.
[0063] Furthermore, during the knowledge graph construction process, for keywords in the first and second word segmentation results that are not identified as target keywords, the computer device can update the entity constraint benchmark vocabulary based on these keywords. In one possible implementation, the computer device can add the remaining keywords to the preparatory library of the entity constraint benchmark vocabulary corresponding to the retrieval result dataset, so as to update the entity constraint benchmark vocabulary corresponding to the retrieval result dataset based on the keywords in the preparatory library; the remaining keywords are the keywords in the first and second word segmentation results other than the first display keywords and the second display keywords.
[0064] For the remaining keywords after entity constraints, the computer device can perform fuzzy matching on the non-entity node keywords in the display keyword set based on the entity constraint benchmark thesaurus to obtain the fuzzy matching results of each non-entity node keyword. Add the keywords of non-entity nodes that indicate successful fuzzy matching results to the preparatory library of the entity constraint benchmark thesaurus corresponding to the retrieval result dataset.
[0065] In one possible implementation, when performing fuzzy matching on non-entity node keywords in the display keyword set based on the entity constraint benchmark lexicon, the computer device can further segment the entity constraint benchmark words in the entity constraint benchmark lexicon, compare the segmented entity constraint benchmark words with the non-entity node keywords, add the successfully matched non-entity node keywords to the preparatory library of the entity constraint benchmark lexicon corresponding to the retrieval result dataset, and discard the non-entity node keywords that fail to match. Alternatively, in another possible implementation, the computer device can calculate the semantic similarity between the non-entity node keywords and the entity constraint benchmark words, add the non-entity node keywords with semantic similarity higher than the semantic similarity threshold to the preparatory library of the entity constraint benchmark lexicon corresponding to the retrieval result dataset, and discard the non-entity node keywords with semantic similarity lower than the semantic similarity threshold.
[0066] For the preliminary database of entity constraint benchmark terminology, computer equipment can send keywords from the preliminary database to relevant personnel at certain intervals so that they can filter them, add keywords that are new knowledge points to the entity constraint benchmark terminology for the next matching, and discard keywords that are not new knowledge points.
[0067] Step 380: In response to a second search result containing a keyword record in the search result dataset, add the keyword from the keyword record corresponding to the second search result to the target keyword set.
[0068] For search results containing keyword records in the search result dataset, the computer device can obtain the keyword records of the search result by querying the keyword records, and add the keywords in the keyword records to the target keyword set. In other words, if the second search result corresponds to a keyword record, after retrieving the second search result, the computer device can directly obtain the keyword records corresponding to the second search result to extract the keywords in the keyword records for application.
[0069] Step 390: In response to receiving a request to view the knowledge graph, the knowledge graph is returned to the user.
[0070] This knowledge graph is the first-level knowledge graph constructed corresponding to the search keywords. The computer device feeds the knowledge graph back to the user, allowing the user to view entity nodes that are directly related to the search keywords through the graphical knowledge graph.
[0071] Furthermore, when providing the knowledge graph to the user, interactive functionality can be offered. This allows users to expand the knowledge graph's hierarchy by selecting individual entity nodes. This process can be implemented as follows: In response to receiving a selection operation for a target entity node in the knowledge graph, the knowledge graph is expanded hierarchically based on the target entity node; hierarchical expansion refers to further constructing the knowledge graph using the keywords corresponding to the target entity node as search keywords.
[0072] During hierarchical expansion, the knowledge graph is constructed in real time. The process of hierarchical expansion is similar to that of the first-layer knowledge graph, except that the graph is no longer constructed based on user-input search keywords. Instead, the expansion source is the target entity node in the first-layer knowledge graph that receives the user's selection operation. The computer device searches the knowledge base using the keywords corresponding to the target entity node, repeating the construction process of the first-layer knowledge graph. In other words, the graphical knowledge graph can be expanded by performing an expansion operation on a certain entity node. Since the entire process uses real-time computing, custom expansion construction can be performed on newly expanded entity nodes.
[0073] After completing the hierarchical expansion based on the target entity nodes, the computer device feeds back the expanded knowledge graph to the user so that the user can view it or continue to expand the knowledge graph hierarchically.
[0074] In summary, the knowledge retrieval method provided in this application, after receiving user input and obtaining search keywords, performs a search based on those keywords to obtain search results. Then, it analyzes the search results based on relevance to obtain a search result dataset. For the first search result containing no key records within the dataset, it performs word segmentation using both a natural language processing model and a keyword extraction model, obtaining a first word segmentation result and a second word segmentation result for the first search result. The keywords common to both the first and second word segmentation results are identified as the first display keywords for the first search result and added to a large set of display keywords. Subsequently, a knowledge graph is constructed based on the target keyword set determined from the display keyword set. Upon receiving a query from the knowledge graph... When a request is received, the knowledge graph is fed back to the user. This method deeply integrates knowledge graph construction with the real-time retrieval process, significantly improving the efficiency of knowledge mining and the timeliness of data usage. Specifically, through parallel analysis of a natural language processing model and a keyword extraction model, highly relevant keywords are directly extracted from the user's search results, avoiding the lag problem of traditional static knowledge graphs. A cross-validation mechanism is used to filter and display the keyword set, ensuring the quality of the basic data for building the knowledge graph, thereby improving the quality of knowledge graph construction and the matching degree between the knowledge graph and user search needs. Overall, a closed-loop optimization of knowledge acquisition-analysis-construction-feedback is achieved, enabling knowledge services to have real-time response capabilities, thus improving the knowledge retrieval effect.
[0075] Further keyword filtering through entity constraints and word frequency constraints can improve the accuracy of keywords in knowledge graph construction and enhance the overall construction effect. In addition, maintaining a benchmark vocabulary for entity constraints based on data extracted during knowledge graph construction can improve the utilization rate of data resources.
[0076] Taking the interaction between computer devices and search engines, large language model engines, keyword extraction models, entity constraint components, word frequency constraint components, data filtering components, and visualization tools to achieve knowledge retrieval as an example. Figure 5 The illustration shows a schematic diagram of a knowledge retrieval process provided in an exemplary embodiment of this application. This process includes two stages: the construction of the first-layer knowledge graph and the real-time construction of node cyclic expansion. Figure 5 As shown, in the first-layer knowledge graph construction stage, the system graphically displays entity nodes with expected direct relationships based on user-input search keywords. During this process, the user submits search keywords to the search engine; the search engine performs a search based on these keywords, obtaining search results. While returning the search results to the user, the system performs a relevance analysis between the search results and the search keywords, resulting in a search result dataset. Keywords corresponding to the first search result in the search result dataset containing keyword records are sent to a visualization tool. Second search results in the search result dataset without keyword records are sent to an NLP engine for word segmentation to obtain the first word segmentation result, and then sent to a keyword extraction model for keyword extraction to obtain the second word segmentation result. The first and second word segmentation results are then sent to a data filtering component, which compares the first word segmentation result with... The second word segmentation result filters out the first display keyword, and other keywords are filtered based on their relevance to knowledge keywords to obtain the second display keyword. The display keyword set containing the first and second display keywords is sent to the entity constraint component. The entity constraint component performs consistency filtering on the keywords in the display keyword set using the entity constraint benchmark thesaurus to obtain entity node keywords, and sends the entity node keywords to the word frequency constraint component. The word frequency constraint component performs word frequency calculation and low-frequency word cleaning to obtain the target keyword set, and sends the target keyword set to the visualization tool. The visualization tool graphically constructs entity nodes and relationship lines based on the received target keyword set and the keywords corresponding to the first search result. After receiving a knowledge graph viewing request sent by the user through the search engine, the knowledge graph is fed back to the user through the search engine.
[0077] In addition, during the keyword filtering process, the data filtering component can also send keywords whose relevance cannot be confirmed (i.e., remaining keywords) to the entity constraint component. The entity constraint component then performs fuzzy filtering through the entity constraint benchmark thesaurus and adds the remaining keywords that pass the fuzzy filtering to the reserve library of the entity constraint benchmark thesaurus. Furthermore, keywords that are inconsistent with the entity constraint benchmark thesaurus in subsequent verification are also added to the reserve library of the entity constraint benchmark thesaurus, so as to update the entity constraint benchmark thesaurus by judging the keywords in the reserve library.
[0078] By combining the knowledge retrieval process with the knowledge graph construction process, this approach leverages large language model segmentation and keyword extraction models to extract keywords based on user search content. It then rapidly constructs the knowledge graph in real-time using entity constraints and word frequency constraints, supporting user knowledge mining and improving data resource utilization. Specifically, the use of large language model segmentation and keyword extraction models for keyword extraction, through comparative analysis, improves the accuracy of knowledge extraction and ensures the quality of the generated knowledge graph. Keyword selection using entity constraints and word frequency constraints offers a high degree of automation and low maintenance, while also improving the debugging and optimization performance of the generated knowledge graph. The method of analyzing data and generating the knowledge graph in real-time based on user searches and search results focuses on the data scope of user interest, contributing to the effectiveness and timeliness of knowledge mining. The hierarchical, extensible, real-time knowledge graph construction interaction effectively improves user retrieval efficiency.
[0079] Furthermore, since the knowledge retrieval method based on knowledge graphs, which is triggered by user-input search keywords, is provided in this application embodiment, it can perform knowledge retrieval in various fields based on search keywords. Therefore, this method can be applied to knowledge retrieval in various fields to improve the knowledge retrieval effect.
[0080] Figure 6 This invention illustrates a block diagram of a knowledge retrieval apparatus provided in an exemplary embodiment of this application, which can perform functions such as... Figure 1 or Figure 3 All or part of the steps in the illustrated embodiments, such as Figure 6 As shown, the device may include the following modules.
[0081] The retrieval module 610 is used to perform retrieval based on the search keywords entered by the user and obtain the retrieval results; Analysis module 620 is used to perform relevance analysis on the search results and the search keywords to obtain a search result dataset. The search result dataset contains the n search results with the highest relevance to the search keywords, where n is a positive integer. The word segmentation module 630 is used to respond to a first search result in the search result dataset containing a record without keywords, and to send the first search result to the natural language processing model and the keyword extraction model respectively for word segmentation processing to obtain the first word segmentation result and the second word segmentation result corresponding to the first search result; the keyword extraction model is a model with text keyword extraction function. The filtering module 640 is used to filter keywords based on the first word segmentation result and the second word segmentation result, and generate a set of display keywords. The set of display keywords includes a first display keyword, which is a keyword that is common to the first word segmentation result and the second word segmentation result. The set determination module 650 is used to determine a target keyword set based on the displayed keyword set, so as to construct a knowledge graph based on the target keyword set; The feedback module 660 is used to respond to a received knowledge graph viewing request by providing the knowledge graph back to the user.
[0082] In one possible implementation, the device further includes: The relevance calculation module is used to calculate the relevance between other keywords and knowledge keywords. The other keywords are keywords other than the first displayed keywords in the first word segmentation result and the second word segmentation result. The knowledge keywords are known keywords corresponding to vertical domain data determined based on the search result dataset. The keyword determination module is used to identify other keywords with relevance values greater than the relevance threshold as second display keywords and add them to the display keyword set.
[0083] In one possible implementation, the set determination module 650 includes: The consistency filtering submodule is used to perform consistency filtering on the keywords in the display keyword set through the entity constraint benchmark thesaurus corresponding to the search result dataset to obtain entity node keywords; the entity constraint benchmark thesaurus is constructed based on keywords extracted from industry domain data, vertical domain data and custom data; The set determination submodule is used to determine the target keyword set based on the entity node keywords.
[0084] In one possible implementation, the set determines the sub-module, including: The term frequency acquisition unit is used to acquire the term frequency of keywords for each entity node in the search result dataset. The filtering unit is used to filter entity node keywords based on the word frequency of keywords in each entity node to obtain the target keyword set.
[0085] In one possible implementation, the device further includes: The first adding module is used to add keywords from the keyword records corresponding to the second search result to the target keyword set in response to the second search result containing keyword records in the search result dataset.
[0086] In one possible implementation, the device further includes: The second adding module is used to add the remaining keywords to the preparatory library of the entity constraint benchmark terminology library corresponding to the search result dataset, so as to update the entity constraint benchmark terminology library corresponding to the search result dataset based on the keywords in the preparatory library; the remaining keywords are the keywords in the first word segmentation result and the second word segmentation result other than the first display keyword and the second display keyword.
[0087] In one possible implementation, the device further includes: The fuzzy matching module is used to perform fuzzy matching on non-entity node keywords in the display keyword set based on the entity constraint benchmark lexicon, and obtain the fuzzy matching results of each non-entity node keyword. The third addition module is used to add non-entity node keywords that indicate successful matching in the fuzzy matching results to the preparatory library of the entity constraint benchmark terminology corresponding to the retrieval result dataset.
[0088] In one possible implementation, the device further includes: An extension module is used to respond to receiving a selection operation on a target entity node in the knowledge graph, and to perform hierarchical extension of the knowledge graph based on the target entity node; the hierarchical extension refers to further constructing the knowledge graph using the keywords corresponding to the target entity node as search keywords.
[0089] In summary, the knowledge retrieval device provided in this application, after receiving user input and obtaining search keywords, performs a search based on those keywords to obtain search results. It then analyzes the search results based on relevance to obtain a search result dataset. For the first search result dataset containing no key records, it performs word segmentation using both a natural language processing model and a keyword extraction model, obtaining a first word segmentation result and a second word segmentation result. The keywords common to both the first and second word segmentation results are identified as the first display keywords of the first search result and added to a large display keyword set. Subsequently, a knowledge graph is constructed based on the target keyword set determined from the display keyword set. The knowledge graph is then viewed... When a request is made, the knowledge graph is fed back to the user. Through the aforementioned device, the construction of the knowledge graph and the real-time retrieval process can be deeply integrated, significantly improving the efficiency of knowledge mining and the timeliness of data use. Specifically, by using a dual-path parallel analysis of a natural language processing model and a keyword extraction model, highly relevant keywords are directly extracted from the user's search results, avoiding the lag problem of traditional static knowledge graphs. The use of a cross-validation mechanism to filter and display the keyword set ensures the quality of the basic data for constructing the knowledge graph, thereby improving the quality of the knowledge graph construction and the matching degree between the knowledge graph and the user's search needs. Overall, a closed-loop optimization of knowledge acquisition-analysis-construction-feedback is achieved, enabling knowledge services to have real-time response capabilities, thus improving the knowledge retrieval effect.
[0090] Figure 7 This illustration shows a structural block diagram of a computer device 700 according to an exemplary embodiment of this application. This computer device can be implemented as a server in the above-described scheme of this application. The computer device 700 includes a Central Processing Unit (CPU) 701, a system memory 704 including Random Access Memory (RAM) 702 and Read-Only Memory (ROM) 703, and a system bus 705 connecting the system memory 704 and the CPU 701. The computer device 700 also includes a mass storage device 706 for storing an operating system 709, application programs 710, and other program modules 711. The system memory 704 and the mass storage device 706 can be collectively referred to as memory.
[0091] According to various embodiments of this application, the computer device 700 can also be connected to a remote computer on a network, such as the Internet. That is, the computer device 700 can be connected to a network 708 via a network interface unit 707 connected to the system bus 705, or the network interface unit 707 can be used to connect to other types of networks or remote computer systems (not shown).
[0092] The memory also includes at least one instruction, at least one program, code set, or instruction set, which are stored in the memory. The central processing unit 701 executes the at least one instruction, at least one program, code set, or instruction set to implement all or part of the steps in the knowledge retrieval methods shown in the above embodiments.
[0093] Figure 8 A structural block diagram of a computer device 800 illustrating another exemplary embodiment of this application is shown. This computer device 800 can be implemented as the aforementioned terminal. For example, the computer device can be an Android terminal device; typically, the computer device 800 includes a processor 801 and a memory 802. The memory 802 may include one or more computer-readable storage media for storing at least one instruction, which is executed by the processor 801 to implement all or part of the steps in the knowledge retrieval method shown in the method embodiments of this application.
[0094] In some embodiments, the computer device 800 may optionally include a peripheral device interface 803 and at least one peripheral device. The processor 801, memory 802, and peripheral device interface 803 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 803 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of a radio frequency circuit 804, a display screen 805, a camera assembly 806, an audio circuit 807, and a power supply 808. In some embodiments, the computer device 800 also includes one or more sensors 809. These sensors 809 include, but are not limited to, an accelerometer 810, a gyroscope 811, a pressure sensor 812, an optical sensor 813, and a proximity sensor 814. Those skilled in the art will understand that... Figure 8 The structure shown does not constitute a limitation on the computer device 800, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.
[0095] In one exemplary embodiment, a computer-readable storage medium is also provided, which stores at least one computer program that is loaded and executed by a processor to implement all or part of the steps in the knowledge retrieval method described above. For example, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, or optical data storage device, etc.
[0096] In one exemplary embodiment, a computer program product is also provided, the computer program product including a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions that, when executed by a computer, cause the computer to perform the above-described actions. Figure 1 or Figure 3 All or part of the steps of the embodiments shown in any of the embodiments.
[0097] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the claims.
[0098] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.
Claims
1. A knowledge retrieval method, characterized in that, The method includes: The search is performed based on the search keywords entered by the user, and the search results are obtained. A relevance analysis is performed on the search results and the search keywords to obtain a search result dataset. The search result dataset contains the n search results with the highest relevance to the search keywords, where n is a positive integer. In response to a first search result containing no keyword record in the search result dataset, the first search result is sent to a natural language processing model and a keyword extraction model for word segmentation processing to obtain a first word segmentation result and a second word segmentation result corresponding to the first search result; the keyword extraction model is a model with text keyword extraction function. Based on the first word segmentation result and the second word segmentation result, keyword filtering is performed to generate a display keyword set. The display keyword set includes a first display keyword, which is a keyword that is common to both the first word segmentation result and the second word segmentation result. Based on the set of displayed keywords, a set of target keywords is determined, and a knowledge graph is constructed based on the set of target keywords. In response to receiving a request to view the knowledge graph, the knowledge graph is returned to the user.
2. The method according to claim 1, characterized in that, The method further includes: Calculate the relevance between other keywords and knowledge keywords, wherein the other keywords are those other than the first displayed keywords in the first word segmentation result and the second word segmentation result; the knowledge keywords are known keywords corresponding to vertical domain data determined based on the search result dataset; Other keywords with relevance values greater than the relevance threshold are identified as second display keywords and added to the set of display keywords.
3. The method according to claim 1 or 2, characterized in that, The step of determining the target keyword set based on the displayed keyword set includes: The keywords in the displayed keyword set are subjected to consistency screening using the entity constraint benchmark thesaurus corresponding to the search result dataset to obtain entity node keywords; the entity constraint benchmark thesaurus is constructed based on keywords extracted from industry domain data, vertical domain data, and custom data; The target keyword set is determined based on the entity node keywords.
4. The method according to claim 3, characterized in that, Determining the target keyword set based on the entity node keywords includes: Obtain the word frequency of keywords for each entity node in the search results dataset; The target keyword set is obtained by filtering entity node keywords based on the word frequency of each entity node keyword.
5. The method according to claim 1, characterized in that, The method further includes: In response to a second search result containing a keyword record in the search result dataset, the keywords in the keyword record corresponding to the second search result are added to the target keyword set.
6. The method according to claim 2, characterized in that, The method further includes: The remaining keywords are added to the preparatory library of the entity constraint benchmark thesaurus corresponding to the search result dataset, so as to update the entity constraint benchmark thesaurus corresponding to the search result dataset based on the keywords in the preparatory library; the remaining keywords are the keywords in the first word segmentation result and the second word segmentation result other than the first display keyword and the second display keyword.
7. The method according to claim 3, characterized in that, The method further includes: Based on the entity constraint benchmark thesaurus, fuzzy matching is performed on the non-entity node keywords in the display keyword set to obtain the fuzzy matching results of each non-entity node keyword. The keywords of non-entity nodes that are successfully matched by fuzzy matching results are added to the preparatory library of the entity constraint benchmark vocabulary corresponding to the retrieval result dataset.
8. The method according to claim 1, characterized in that, The method further includes: In response to receiving a selection operation for a target entity node in the knowledge graph, the knowledge graph is expanded hierarchically based on the target entity node; the hierarchical expansion refers to further constructing the knowledge graph using the keywords corresponding to the target entity node as search keywords.
9. A knowledge retrieval device, characterized in that, The device includes: The search module is used to perform searches based on user-input keywords and obtain search results. The analysis module is used to perform relevance analysis on the search results and the search keywords to obtain a search result dataset. The search result dataset contains the n search results with the highest relevance to the search keywords, where n is a positive integer. The word segmentation module is used to respond to a first search result in the search result dataset containing a record without keywords, and to send the first search result to the natural language processing model and the keyword extraction model respectively for word segmentation processing to obtain the first word segmentation result and the second word segmentation result corresponding to the first search result; the keyword extraction model is a model with text keyword extraction function. The filtering module is used to filter keywords based on the first word segmentation result and the second word segmentation result, and generate a set of display keywords. The set of display keywords includes a first display keyword, which is a keyword that is common to both the first word segmentation result and the second word segmentation result. The set determination module is used to determine the target keyword set based on the displayed keyword set, so as to construct a knowledge graph based on the target keyword set; The feedback module is used to respond to a received knowledge graph viewing request by providing the knowledge graph back to the user.
10. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing at least one computer program, which is loaded and executed by the processor to implement the knowledge retrieval method as described in any one of claims 1 to 8.
11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one computer program, which is loaded and executed by a processor to implement the knowledge retrieval method as described in any one of claims 1 to 8.
12. A computer program product, characterized in that, The computer program product includes a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions that, when executed by a computer, cause the computer to perform the knowledge retrieval method as described in any one of claims 1 to 8.