Knowledge base information retrieval method and system of service platform

Abstract

The invention discloses a knowledge base information retrieval method and system of a service platform, and belongs to the technical field of data processing. The method comprises the steps that query information input by a user is recognized; identifying extended vocabularies related to the user query information from a predefined vocabulary library, and screening to obtain a target document based on the importance of the extended vocabularies in the source document; obtaining a first correlation score of each target document and the query information by using an abstract scoring algorithm; obtaining a second correlation score of each target document and the query information by using a co-word algorithm; based on the first correlation score and the second correlation score, obtaining a comprehensive correlation score of each target document and the query information through a weighting mode; and according to the comprehensive correlation scores, performing descending sort on the target documents and then displaying the target documents to a user. According to the method, the semantic association of the vocabularies and the matching of the vocabularies and the document themes are considered, and the accuracy of database retrieval results is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of data processing, and in particular relates to a knowledge base information retrieval method and system for a service platform. Background Art

[0002] In the field of information retrieval, with the rapid growth of data volumes and the increasing complexity of query requirements, efficiently extracting the most relevant content from large amounts of documents has become a research priority. Modern information retrieval systems typically rely on a variety of scoring algorithms to assess document relevance, including content-based scoring, co-word analysis, and the use of expanded terms.

[0003] While existing retrieval technologies have improved the relevance of search results to some extent, they still have some significant shortcomings. First, while traditional scoring algorithms perform well in calculating word and document frequencies, they often overlook the semantic associations between words and the topic of the document. This deficiency can lead to retrieval systems failing to accurately capture the actual semantics of documents when faced with complex queries, thus affecting the accuracy of retrieval results. Summary of the Invention

[0004] In view of the deficiencies in the prior art, the present invention provides a knowledge base information retrieval method and system for a service platform, which considers the semantic association of words and the matching with document topics, thereby improving the accuracy of database retrieval results.

[0005] The present invention provides the following technical solutions:

[0006] In a first aspect, a knowledge base information retrieval method for a service platform is provided, comprising:

[0007] Identify query information entered by the user;

[0008] Identify extended words related to user query information from a predefined vocabulary library, and select and obtain target documents based on the importance of the extended words in the source documents;

[0009] Using the summary scoring algorithm, a first relevance score between each target document and the query information is obtained; using the co-word algorithm, a second relevance score between each target document and the query information is obtained;

[0010] Based on the first and second relevance scores, a comprehensive relevance score between each target document and the query information is obtained in a weighted manner;

[0011] The target documents are sorted in descending order according to the size of the comprehensive relevance score and then displayed to the user.

[0012] Optionally, the identifying the query information input by the user specifically includes: preprocessing the query information input by the user, and performing intent analysis on the query information through natural language processing technology based on the preprocessed query information.

[0013] Optionally, the target document is screened and obtained based on the importance of the expanded vocabulary in the source document, specifically:

[0014] The importance score of the extended vocabulary in each source document is calculated according to the following formula, and the source documents with a score above the set threshold are output as target documents;

[0015]

[0016] Among them, Score word (Q, d) is the importance score of the expanded word of the query information Q in the source document d, tf word (Q, d) is the total word frequency of the expanded words of the query information Q in the source document d, k3 is the adjustment factor used to control the influence of word frequency on importance score, b2 is the adjustment factor used to control the influence of the length of the expanded word list on importance score, d wl is the length of the expanded word list in the source document d, avg dwl The average length of the expanded word lists for all source documents.

[0017] Optionally, the summary scoring algorithm is used to obtain a first relevance score between each target document and the query information, specifically:

[0018] Extract the summary part of each target document;

[0019] The first relevance score between the summary of each target document and the query information is calculated according to the following formula: abstract (Q,d);

[0020]

[0021] Among them, IDF(q i ) is the i-th expansion word q i The inverse document frequency, m is the total number of expanded words in the target document d summary, f i For the extended word q i The word frequency in the target document d summary, k1 is the adjustment factor used to control the effect of word frequency on the first relevance score, b1 is the adjustment factor used to control the effect of document summary length on the importance score, d l is the length of the target document d summary, avg dl is the average length of all target document summaries.

[0022] Optionally, the co-word algorithm is used to obtain a second relevance score between each target document and the query information, specifically:

[0023] For each target document, extracting co-word data related to the query information, wherein the co-word data is words in the target document that are related to the query information and frequently appear together;

[0024] Calculate the inverse document frequency (IDF) of each group of co-words corresponding to the target document gene (g i ,d);

[0025]

[0026] Among them, g i is the i-th group of co-word data, N is the total number of target documents, n(g i ) is the data g containing the i-th group of co-words i The number of documents;

[0027] Sum the inverse document frequency values ​​of all co-words to obtain the second relevance score Score between the target document d and the query information co-word (Q,d);

[0028]

[0029] Where n is the total number of co-word groups in the target document d.

[0030] Optionally, based on the first and second relevance scores, a comprehensive relevance score between each target document and the query information is obtained in a weighted manner, specifically:

[0031] Scoer 总 (Q,d)=λ1Score abstract (Q,d)+λ2Score co-word (Q,d)

[0032] Among them, Scoer 总 (Q, d) is the comprehensive relevance score between the target document d and the query information Q, Score abstract (Q,d) and Score co-word (Q, d) are the first and second relevance scores, respectively. λ1 and λ2 are the weights of the first and second relevance scores, respectively.

[0033] Optionally, before obtaining the comprehensive relevance score between each target document and the query information in a weighted manner, a normalization operation is performed on the first relevance score and the second relevance score;

[0034] The first relevance score and the second relevance score of each document are recorded in a database or a result list.

[0035] In a second aspect, a knowledge base information retrieval system for a service platform is provided, comprising:

[0036] Intent recognition module: identifies the query information entered by the user;

[0037] Document screening module: identifies extended terms related to user query information from a predefined vocabulary library and selects target documents based on the importance of the extended terms in the source documents;

[0038] Separate scoring module: uses the summary scoring algorithm to obtain the first relevance score of each target document and the query information; uses the co-word algorithm to obtain the second relevance score of each target document and the query information;

[0039] Fusion scoring module: Based on the first and second relevance scores, it obtains the comprehensive relevance score between each target document and the query information in a weighted manner;

[0040] Result presentation module: sorts each target document in descending order according to the size of the comprehensive relevance score and presents it to the user.

[0041] According to a third aspect, a computer device is provided, comprising a processor and a memory; wherein, when the processor executes a computer program stored in the memory, the steps of the knowledge base information retrieval method of a service platform described in any one of the first aspects are implemented.

[0042] In a fourth aspect, a computer-readable storage medium is provided for storing a computer program; when the computer program is executed by a processor, the steps of the knowledge base information retrieval method of a service platform described in any one of the first aspects are implemented.

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] This application uses extended word analysis to screen out target documents related to the query information. The first relevance score is calculated based on the degree of match between the target document summary and the query information, and the second relevance score is based on the co-word data related to the query information in the target document. The two scores respectively reflect the relevance of the target document in the summary content and the subject association, effectively improving the accuracy of the retrieval results; in addition, by weighted fusion of the first relevance score and the second relevance score, the comprehensive relevance score of each target document is determined, which does not rely on unilateral evaluation, further improving the accuracy of the retrieval results. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1It is the flowchart of the steps of the knowledge base information retrieval method of the service platform of the present invention;

[0046] Figure 2 It is the structural block diagram of the knowledge base information retrieval system of the service platform of the present invention. Specific implementation manner

[0047] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention. It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and do not have to be used to describe a specific order or sequence. In addition, the term "comprising" and any variation thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0048] Embodiment 1

[0049] As Figure 1 shown, a knowledge base information retrieval method for a service platform is provided, including the following steps:

[0050] S1: Identify the query information input by the user.

[0051] Specifically, preprocess the obtained query information input by the user, and perform intent analysis on the query information through natural language processing technology according to the preprocessed query information.

[0052] The preprocessing includes: removing stop words (such as "of", "is", "in"), punctuation marks, and other irrelevant characters; and size normalization, converting all characters into a unified case form.

[0053] Natural language processing (NLP) is a discipline that combines computer science, artificial intelligence, and linguistics, mainly studying how to make computers understand, interpret, and generate human natural language. The goal of NLP is to enable computers to understand and process language data in a way similar to humans, so as to achieve efficient automation in fields such as human-computer interaction, text analysis, and information retrieval.

[0054] S2: Identify the extended vocabulary related to the user's query information from the predefined vocabulary library, and filter and obtain the target document based on the importance of the extended vocabulary in the source document.

[0055] In an information retrieval system, using extended vocabulary to enhance the accuracy and relevance of queries is an important strategy to improve the retrieval effect.

[0056] Specifically, sub-step S21 includes: First, identifying expanded vocabulary related to the user's query from a predefined vocabulary library. These expanded vocabulary include synonyms, related field terms, and other relevant terms that can enhance the query. By expanding the vocabulary, we can cover more relevant documents and improve the comprehensiveness and accuracy of the search. The key to this step is to ensure that the expanded vocabulary selection fully reflects the query intent and covers relevant fields.

[0057] As an alternative, if the user searches for "heart disease", the expanded vocabulary may include synonyms and related terms such as "coronary heart disease" and "cardiovascular disease".

[0058] Sub-step S22: After identifying the relevant expanded words, the total scores of these expanded words are calculated to evaluate their importance in the target document.

[0059]

[0060] Among them, Score word (Q, d) is the importance score of the expanded word of the query information Q in the source document d, tf word (Q, d) is the total word frequency of the expanded words of the query information Q in the source document d, k3 is the adjustment factor used to control the influence of word frequency on importance score, b2 is the adjustment factor used to control the influence of the length of the expanded word list on importance score, b2∈[0,1], d wl is the length of the expanded word list in the source document d, avg dwl The average length of the expanded word lists for all source documents.

[0061] Sub-step S23: Based on the calculated expansion word scores, documents with scores higher than a preset threshold are retained, and these documents are considered as target documents that are highly relevant to the query information.

[0062] In this way, the documents that meet the query intent can be effectively screened out, improving the accuracy and relevance of the retrieval.

[0063] S3: Using a summary scoring algorithm, obtain a first relevance score between each target document and the query information; using a co-word algorithm, obtain a second relevance score between each target document and the query information.

[0064] In information retrieval systems, evaluating the relevance of the target document summary to the query information is an important step in ensuring the accuracy of retrieval results. Specifically, it includes the following steps:

[0065] (1) First relevance score

[0066] Sub-step S31: extracting a summary portion from the filtered target document.

[0067] A summary is typically a brief overview of a document, quickly providing the main information and themes. For extracting the summary, refer to existing technologies, such as BERT and GPT models.

[0068] Sub-step S32: Calculate the first relevance score Score between the abstract of each target document and the query information according to the following formula: abstract (Q,d).

[0069]

[0070] Among them, IDF(q i ) is the i-th expansion word q i The inverse document frequency, m is the total number of expanded words in the target document d summary, f i For the extended word q i The word frequency in the target document d summary, k1 is the adjustment factor used to control the effect of word frequency on the first relevance score, b1 is the adjustment factor used to control the effect of document summary length on the importance score, d l is the length of the target document d summary, avg dl is the average length of all target document summaries.

[0071] Frequency of the expanded word in the target document summary, expanded word q i The inverse document frequency, the length of the target document d summary, etc. can all be obtained by relying on existing technologies.

[0072] (2) Second relevance score

[0073] Sub-step S33: For each target document, extract the co-word data related to the query information.

[0074] Co-word data refers to words that are related to the query information and appear frequently together in the target document. They can reflect the topic structure of the document and its relationship with the query information.

[0075] The method of obtaining co-words can refer to existing technologies, such as building association strength, using latent Dirichlet allocation model or deep learning model, etc.

[0076] Sub-step S34: Calculate the inverse document frequency (IDF) of each group of co-words corresponding to the target document gene (g i ,d);

[0077]

[0078] Among them, g i is the i-th group of co-word data, N is the total number of target documents, n(g i) is the data g containing the i-th group of co-words i The number of documents.

[0079] Sub-step S35: Sum the inverse document frequency values ​​of all co-words to obtain the second relevance score Score between the target document d and the query information co-word (Q,d).

[0080]

[0081] Where n is the total number of co-word groups in the target document d.

[0082] It's important to note that after calculating the first and second relevance scores, each needs to be normalized to ensure that the score ranges for all target documents are consistent. Normalization converts the scores to a standardized range, making them comparable across documents and facilitating subsequent score sorting and analysis.

[0083] In some other embodiments, after calculating the first and second relevance scores, the first and second relevance scores are recorded in a database or result list to facilitate subsequent search result sorting and user presentation. Recording the scores not only provides a basis for document sorting but can also be used to further analyze and optimize the search algorithm.

[0084] S4: Based on the first and second relevance scores, a comprehensive relevance score between each target document and the query information is obtained in a weighted manner.

[0085] In information retrieval systems, in order to comprehensively evaluate the relevance of target documents, it is usually necessary to perform weighted fusion of different types of scores.

[0086] The specific steps for weighted fusion of the first relevance score (summary-based score) and the second relevance score (co-word-based score) are as follows:

[0087] Sub-step S41: First, collect the first and second relevance scores for each target document. The first relevance score is calculated based on the degree of match between the document summary and the query information, while the second relevance score is calculated based on the co-word data related to the query information in the document. These two scores reflect the relevance of the document in terms of summary content and topic relevance, respectively, providing basic data for comprehensively evaluating the overall relevance of the document.

[0088] Sub-step S42: Obtaining a comprehensive relevance score between each target document and the query information.

[0089] Scoer 总 (Q,d)=λ1Scoreabstract (Q,d)+λ2Score co-word (Q,d)

[0090] Among them, Scoer 总 (Q, d) is the comprehensive relevance score between the target document d and the query information Q, Score abstract (Q,d) and Score co-word (Q, d) are the first and second relevance scores, respectively. λ1 and λ2 are the weights of the first and second relevance scores, respectively.

[0091] By setting appropriate weighting factors, the influence of the two scores on the final weighted relevance score can be adjusted according to actual needs. This weighted fusion method helps to comprehensively consider the abstract content and topic relevance of the document, improving the overall relevance and accuracy of the retrieval results.

[0092] S5: Arrange the target documents in descending order according to the comprehensive relevance scores and present them to the user.

[0093] Specifically, the weighted relevance scores of all target documents are collected; all target documents are sorted in descending order according to the weighted relevance scores, that is, documents with higher scores are ranked first; a new document list is generated based on the sorting results, and the list is sorted from high to low according to the weighted relevance scores, and the target documents with higher rankings are displayed to the user, usually through the user interface or the search results page.

[0094] This application solves the problem of semantic association of words and the theme of documents that are ignored by traditional scoring algorithms, and at the same time solves the problem that existing technologies cannot accurately capture the actual semantics of documents, thereby improving the accuracy of retrieval results.

[0095] Example 2

[0096] like Figure 2 As shown, a knowledge base information retrieval system of a service platform includes:

[0097] Intent recognition module: identifies the query information entered by the user;

[0098] Document screening module: identifies extended terms related to user query information from a predefined vocabulary library and selects target documents based on the importance of the extended terms in the source documents;

[0099] Separate scoring module: uses the summary scoring algorithm to obtain the first relevance score of each target document and the query information; uses the co-word algorithm to obtain the second relevance score of each target document and the query information;

[0100] Fusion scoring module: Based on the first and second relevance scores, it obtains the comprehensive relevance score between each target document and the query information in a weighted manner;

[0101] Result presentation module: sorts each target document in descending order according to the size of the comprehensive relevance score and presents it to the user.

[0102] For more specific details of the above method, please refer to the corresponding contents disclosed in the aforementioned embodiments, which will not be repeated here.

[0103] Example 3

[0104] The present invention provides a computer device comprising a processor and a memory; wherein, when the processor executes a computer program stored in the memory, the steps of the knowledge base information retrieval method of the above-mentioned service platform are implemented.

[0105] For more specific details of the above method, please refer to the corresponding contents disclosed in the aforementioned embodiments, which will not be repeated here.

[0106] Example 4

[0107] The present invention provides a computer-readable storage medium for storing a computer program; when the computer program is executed by a processor, the steps of the knowledge base information retrieval method of the above-mentioned service platform are implemented.

[0108] For more specific details of the above method, please refer to the corresponding contents disclosed in the aforementioned embodiments, which will not be repeated here.

[0109] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from the other embodiments. References to the same or similar parts between the various embodiments will be sufficient. The systems, devices, and storage media disclosed in the embodiments are described briefly because they correspond to the methods disclosed in the embodiments. For relevant details, refer to the method description.

[0110] Those skilled in the art will clearly understand that the techniques in the embodiments of the present invention can be implemented using software plus a necessary general-purpose hardware platform. Based on this understanding, the technical solutions in the embodiments of the present invention, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments of the present invention or certain portions of the embodiments.

[0111] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions based on the principles of the present invention are within the scope of protection of the present invention. It should be noted that for those skilled in the art, various improvements and modifications that do not depart from the principles of the present invention should be considered within the scope of protection of the present invention.

Claims

1. A knowledge base information retrieval method for a service platform, characterized in that: include: Identify query information entered by the user; Identify extended words related to user query information from a predefined vocabulary library, and select and obtain target documents based on the importance of the extended words in the source documents; Using the summary scoring algorithm, a first relevance score between each target document and the query information is obtained; using the co-word algorithm, a second relevance score between each target document and the query information is obtained; Based on the first and second relevance scores, a comprehensive relevance score between each target document and the query information is obtained in a weighted manner; The target documents are sorted in descending order according to the size of the comprehensive relevance score and then displayed to the user.

2. The knowledge base information retrieval method of a service platform according to claim 1, characterized in that: The identifying of the query information input by the user specifically includes: preprocessing the query information input by the user, and performing intent analysis on the query information through natural language processing technology based on the preprocessed query information.

3. The knowledge base information retrieval method of a service platform according to claim 1, characterized in that: Based on the importance of the expanded vocabulary in the source document, the target document is screened and obtained, specifically: The importance score of the extended vocabulary in each source document is calculated according to the following formula, and the source documents with a score above the set threshold are output as target documents; Among them, Score word (Q, d) is the importance score of the expanded word of the query information Q in the source document d, tf word (Q, d) is the total word frequency of the expanded words of the query information Q in the source document d, k3 is the adjustment factor used to control the influence of word frequency on importance score, b2 is the adjustment factor used to control the influence of the length of the expanded word list on importance score, d wl is the length of the expanded word list in the source document d, avg dwl The average length of the expanded word lists for all source documents.

4. The knowledge base information retrieval method of a service platform according to claim 1, characterized in that: The summary scoring algorithm is used to obtain the first relevance score between each target document and the query information, specifically: Extract the summary part of each target document; The first relevance score between the summary of each target document and the query information is calculated according to the following formula: abstract (Q,d); Among them, IDF(q i ) is the i-th expansion word q i The inverse document frequency, m is the total number of expanded words in the target document d summary, f i For the extended word q i The word frequency in the target document d summary, k1 is the adjustment factor used to control the effect of word frequency on the first relevance score, b1 is the adjustment factor used to control the effect of document summary length on the importance score, d l is the length of the target document d summary, avg dl is the average length of all target document summaries.

5. The knowledge base information retrieval method of a service platform according to claim 1, characterized in that: The co-word algorithm is used to obtain the second relevance score between each target document and the query information, specifically: For each target document, extracting co-word data related to the query information, wherein the co-word data is words in the target document that are related to the query information and frequently appear together; Calculate the inverse document frequency (IDF) of each group of co-words corresponding to the target document gene (g i ,d); Among them, g i is the i-th group of co-word data, N is the total number of target documents, n(g i ) is the data g containing the i-th group of co-words i The number of documents; Sum the inverse document frequency values ​​of all co-words to obtain the second relevance score Score between the target document d and the query information co-word (Q,d); Where n is the total number of co-word groups in the target document d.

6. The knowledge base information retrieval method of a service platform according to claim 1, characterized in that: The comprehensive relevance score between each target document and the query information is obtained in a weighted manner based on the first and second relevance scores, specifically: Scoer 总 (Q,d)=λ1Score abstract (Q,d)+λ2Score co-word (Q,d) Among them, Scoer 总 (Q, d) is the comprehensive relevance score between the target document d and the query information Q, Score abstract (Q,d) and Score co-word (Q, d) are the first and second relevance scores, respectively. λ1 and λ2 are the weights of the first and second relevance scores, respectively.

7. The knowledge base information retrieval method of a service platform according to claim 1, characterized in that: Before obtaining the comprehensive relevance score between each target document and the query information in a weighted manner, performing a normalization operation on the first relevance score and the second relevance score; The first relevance score and the second relevance score of each document are recorded in a database or a result list.

8. A knowledge base information retrieval system for a service platform, characterized in that: include: Intent recognition module: identifies the query information entered by the user; Document screening module: identifies extended terms related to user query information from a predefined vocabulary library and selects target documents based on the importance of the extended terms in the source documents; Separate scoring module: uses the summary scoring algorithm to obtain the first relevance score of each target document and the query information; uses the co-word algorithm to obtain the second relevance score of each target document and the query information; Fusion scoring module: Based on the first and second relevance scores, it obtains the comprehensive relevance score between each target document and the query information in a weighted manner; Result presentation module: sorts each target document in descending order according to the size of the comprehensive relevance score and presents it to the user.

9. A computer device, characterized in that: It comprises a processor and a memory; wherein, when the processor executes the computer program stored in the memory, it implements the steps of the knowledge base information retrieval method of a service platform as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that Used to store computer programs; when the computer programs are executed by a processor, the steps of the knowledge base information retrieval method of a service platform described in any one of claims 1 to 7 are implemented.

Images