Knowledge base retrieval ranking method and system based on multi-dimensional content quality score

Abstract

The application discloses a knowledge base retrieval and sorting method and system based on multi-dimensional content quality score, the method comprises the following steps: performing mixed retrieval on the knowledge base according to a user query Q to obtain a candidate result set C, and obtaining the quality score of each document segment; determining the weight of the mixed retrieval score and the quality score according to the category of the user query Q; for each document segment in the candidate result set C, weighting and summing the mixed retrieval score and the quality score according to the determined weight to obtain a final score; sorting the document segments in the candidate result set C in descending order of the final score, and outputting the top k document segments as the retrieval result of the user query Q on the knowledge base. The application aims to keep the efficiency advantage of traditional mixed retrieval, preferentially return complete, reasonable and context clear document segments in the sorting result, and improve the first result accuracy and user experience.

Description

Technical Field

[0001] This invention relates to the fields of natural language processing (NLP) and information retrieval (IR), specifically to a knowledge base retrieval and ranking method and system based on multi-dimensional content quality scoring. Background Technology

[0002] With the improvement of enterprise digitalization, a large number of business documents, technical materials, rules and regulations and engineering plans are accumulated into electronic documents. Building an intelligent knowledge base retrieval and question answering system based on natural language questioning has become an important means to improve the efficiency of knowledge utilization. In the existing technology, common retrieval methods include: (1) Keyword-based retrieval: This type of method calculates the relevance between documents and queries through indicators such as word frequency statistics and inverse document frequency (TF-IDF), which has the advantages of simple implementation and high computational efficiency. However, this type of method is difficult to understand semantics, relies too much on word frequency, and is prone to the situation where a certain keyword is repeated in a table or short text, resulting in an artificially high score. At the same time, it ignores the structure and contextual integrity of the document. (2) Vector-based semantic retrieval (Embedding retrieval): By encoding the query and document into a high-dimensional vector and calculating cosine similarity or other distance measures, the semantic relevance calculation is realized. This method can support semantic similarity queries, but it is not sensitive to precise keyword matching and usually ignores the internal structure and information completeness of the document. In addition, when the document is long, the text needs to be truncated, resulting in potential information loss. (3) Hybrid Search: To balance precise keyword matching and semantic matching, existing technologies often linearly combine vector similarity with BM25 scores. This type of method improves overall recall and precision to some extent, but still has the following problems: 1) The hybrid weights are fixed and cannot adapt to different types of queries; 2) The ranking results only focus on "relevance" and do not explicitly model the content quality of document fragments; 3) In documents containing a large number of tables and structured fields, the BM25 score may still increase the score of some fragmented content due to the repeated appearance of keywords, causing users to have to browse the results multiple times to obtain a truly complete answer. In summary, existing technologies generally have the following defects: 1) They only consider relevance scores and do not explicitly model the content quality of document fragments; 2) They cannot identify whether fragments have a complete context and are prone to returning fragmented content; 3) The hybrid search weights are fixed and lack the ability to adapt to different query types. Summary of the Invention

[0003] The technical problem to be solved by this invention is to provide a knowledge base retrieval and ranking method and system based on multi-dimensional content quality scoring, in order to address the above-mentioned problems of the prior art. This invention aims to maintain the efficiency advantages of traditional hybrid retrieval while prioritizing the return of complete, well-structured and clearly contextualized document fragments in the ranking results, thereby improving the accuracy of the first result and the user experience.

[0004] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0005] A knowledge base retrieval and ranking method based on multi-dimensional content quality scoring includes the following steps:

[0006] S101, perform a hybrid search on the knowledge base based on user query Q to obtain a candidate result set C, wherein the candidate result set C includes multiple document fragments and their corresponding hybrid search scores;

[0007] S102, perform multi-dimensional quality evaluation on each document fragment in the candidate result set C to obtain its quality score, wherein the multi-dimensional evaluation includes length, structure, information density and some or all of the dimensions in the context.

[0008] S103, determine the category of user query Q, and determine the weight of the mixed retrieval score and quality score for each document fragment based on the category of user query Q;

[0009] S104, for each document fragment in the candidate result set C, the combined retrieval score and its quality score are weighted and summed according to the determined weights to obtain the final score of the document fragment;

[0010] S105, sort the document fragments in the candidate result set C in descending order according to the final score, and output the top k document fragments as the retrieval results of the user query Q against the knowledge base.

[0011] Optionally, step S101 includes: calculating a first similarity between the user query Q and the embedding vectors of document fragments in the knowledge base using vector retrieval, and normalizing the embedding vectors to obtain a first similarity score; calculating a second similarity between the user query Q and document fragments in the knowledge base using a similarity calculation method based on word frequency and inverse document frequency, and normalizing the second similarity score; weighted summing the first similarity score and the second similarity score to obtain a mixed retrieval score; sorting the document fragments in the knowledge base in descending order according to the mixed retrieval score; and selecting a corresponding number of document fragments according to the size of a preset candidate result set C to obtain a candidate result set C composed of the selected document fragments and their mixed retrieval scores.

[0012] Optionally, when the first similarity score and the second similarity score are weighted and summed to obtain the mixed search score, the weight coefficient used is obtained by querying a preset weight configuration table for the type of user query Q. The weight configuration table configures different values ​​for the weight coefficient for the two types of user query Q: question-type query and keyword-type query, and the weight coefficient corresponding to the keyword-type query is greater than the weight coefficient corresponding to the question-type query.

[0013] Optionally, in step S102, when performing multi-dimensional quality evaluation on each document fragment in the candidate result set C to obtain its quality score, the quality score is obtained by weighted summation of sub-scores of multiple dimensions. The sub-scores of multiple dimensions include some or all of the following: length score, structural integrity score, information density score, and contextual integrity score. The length score is the score corresponding to the length range to which the character length of the document fragment belongs. The structural integrity score is the score corresponding to whether the document fragment contains document structural features, including whether it contains a title, the number of paragraphs, whether it contains table or list markers, and whether it uses some or all of the natural language punctuation. The information density score is the score corresponding to the coverage of user query Q in the document fragment and whether there is a complete query string match. The contextual integrity score is the score corresponding to whether the document fragment contains hierarchical path information, whether it contains guiding text, and whether it ends with a complete ending mark.

[0014] Optionally, the functional expression for the structural integrity score is:

[0015] ;

[0016] in, For document fragments Structural integrity score, This is a flag indicating whether a title is included, with a value of 0 or 1. This is a flag indicating whether the number of paragraphs exceeds a preset threshold, with a value of 0 or 1. This is a flag indicating whether a list or table is included, with a value of 0 or 1. This is a flag indicating whether to use natural language punctuation, with a value of 0 or 1. This represents the number of valid structures in a document fragment.

[0017] Optionally, the calculation function expression for the information density score is:

[0018] ;

[0019] in, For document fragments Information density score These are the weighting coefficients. To query the number of keywords related to Q for users, For users to query Q and document fragments The number of keywords that appear together For document fragments The identifier for the complete string of the user's query Q is present, with a value of 0 or 1; the expression for the calculation function of the context integrity score is:

[0020] ;

[0021] in, For document fragments Context integrity score This is a flag indicating whether hierarchical path information is included, with a value of 0 or 1. This is a flag indicating whether or not the text contains guiding text; its value is 0 or 1. This is a flag indicating whether the item ends with a complete tag; its value is 0 or 1.

[0022] Optionally, determining the category of user query Q in step S103 includes:

[0023] S201, compare the user query Q with each question word in the preset question word set. If the user query Q contains a certain question word, then determine that the user query Q is a question type query and end; otherwise, proceed to step S202.

[0024] S202, compare the user query Q with each question symbol in the preset question symbol set. If the user query Q contains a question symbol, then the user query Q is determined to be a question type query and the process ends; otherwise, proceed to step S203.

[0025] S203, perform word segmentation and part-of-speech tagging on the user query Q. If the part-of-speech sequence obtained after word segmentation and part-of-speech tagging includes a continuous combination of verbs, nouns and auxiliary words, then the user query Q is determined to be a question query and the process ends; otherwise, proceed to step S204.

[0026] S204, the user's query Q is determined to be a keyword query.

[0027] The present invention also provides a knowledge base retrieval and ranking system based on multi-dimensional content quality scoring, including a microprocessor and a memory interconnected thereto, wherein the microprocessor is programmed or configured to execute the knowledge base retrieval and ranking method based on multi-dimensional content quality scoring.

[0028] The present invention also provides a computer-readable storage medium storing a computer program or instructions that are programmed or configured to execute the knowledge base retrieval and ranking method based on multi-dimensional content quality scoring by a processor.

[0029] The present invention also provides a computer program product, including a computer program or instructions, which are programmed or configured to execute the knowledge base retrieval and ranking method based on multi-dimensional content quality scoring via a processor.

[0030] Compared with existing technologies, the present invention mainly achieves the following beneficial effects: The method of the present invention includes performing a hybrid retrieval on a knowledge base based on a user query Q to obtain a candidate result set C, and obtaining a quality score for each document fragment therein; determining the weights of the hybrid retrieval score and the quality score based on the category of the user query Q; for each document fragment in the candidate result set C, weighting and summing its hybrid retrieval score and its quality score according to the determined weights to obtain a final score; sorting the document fragments in the candidate result set C in descending order according to the final score, and outputting the top k document fragments as the retrieval results of the user query Q on the knowledge base. While maintaining the efficiency advantages of traditional hybrid retrieval, the method of the present invention introduces a content quality metric and combines it with a query type adaptive weighting strategy, thereby enabling the sorted results to prioritize returning complete, structurally sound, and context-clear document fragments, improving the accuracy of the first result and the user experience. Attached Figure Description

[0031] Figure 1 This is a schematic diagram of the basic process of the method in an embodiment of the present invention.

[0032] Figure 2 The figures show a comparison of the experimental results of the method in the embodiments of the present invention, where (a) is the evaluation index of the mixed retrieval results and (b) is the evaluation index of the final retrieval results. Detailed Implementation

[0033] To enable those skilled in the art to better understand the technical solutions of the present invention, the technical solutions of the present invention will be further described in detail below with reference to the accompanying drawings in the embodiments of the present invention.

[0034] like Figure 1 As shown, the knowledge base retrieval and ranking method based on multi-dimensional content quality scoring in this embodiment includes the following steps:

[0035] S101, perform a hybrid search on the knowledge base based on user query Q to obtain a candidate result set C, wherein the candidate result set C includes multiple document fragments and their corresponding hybrid search scores;

[0036] S102, perform multi-dimensional quality evaluation on each document fragment in the candidate result set C to obtain its quality score, wherein the multi-dimensional evaluation includes length, structure, information density and some or all of the dimensions in the context.

[0037] S103, determine the category of user query Q, and determine the weight of the mixed retrieval score and quality score for each document fragment based on the category of user query Q;

[0038] S104, for each document fragment in the candidate result set C, the combined retrieval score and its quality score are weighted and summed according to the determined weights to obtain the final score of the document fragment;

[0039] S105, sort the document fragments in the candidate result set C in descending order according to the final score, and output the top k document fragments as the retrieval results of the user query Q against the knowledge base.

[0040] In this embodiment, step S101 includes: calculating the first similarity between the user query Q and the embedding vectors of document fragments in the knowledge base using vector retrieval, and normalizing the embedding vectors to obtain a first similarity score; calculating the second similarity between the user query Q and the document fragments in the knowledge base using a similarity calculation method based on word frequency and inverse document frequency, and normalizing the second similarity score; weighted summing the first similarity score and the second similarity score to obtain a mixed retrieval score; sorting the document fragments in the knowledge base in descending order according to the mixed retrieval score; and selecting a corresponding number of document fragments according to the size of a preset candidate result set C to obtain a candidate result set C composed of the selected document fragments and their mixed retrieval scores, which can be represented as: ,in This is a document fragment. The score is a hybrid retrieval score, where i is the index. The similarity calculation method based on term frequency and inverse document frequency can be chosen as needed; for example, the BM25 algorithm is used in this embodiment. The function expression for obtaining the hybrid retrieval score by weighted summation of the first similarity score and the second similarity score is:

[0041] ;

[0042] in, For mixed search scoring, This is a document fragment. For user queries, These are the weighting coefficients. The first similarity score, This is the second similarity score.

[0043] Among them, the first similarity score This is a normalized score for vector retrieval (Embedding), representing the semantic similarity score between document D and query Q calculated through vector retrieval (such as text embeddings generated based on BERT or other pre-trained language models), and then normalized. Specifically, This score reflects the semantic matching degree between query Q and document D, and is usually calculated using cosine similarity or other distance metrics. After normalization, the score is ensured to be within a certain range (usually [0,1]). This score captures the semantic relevance between the query and the document, and can handle lexical diversity, synonyms, and contextual semantic similarity.

[0044] Second similarity score The keyword matching score based on the BM25 algorithm represents the keyword matching score between document D and query Q, typically calculated using the BM25 algorithm. BM25 is a classic retrieval method based on term frequency (TF) and inverse document frequency (IDF). When calculating the relevance between a document and a query, it emphasizes the frequency of the query keywords contained in the document and the scarcity of these keywords among all documents. The second similarity score is also normalized to ensure that it is on the same order of magnitude as the first similarity score (usually also normalized to [0,1]). The BM25 algorithm ensures that keyword matching strongly influences the retrieval results, and is particularly suitable for retrieving queries with clear keywords. Its advantage lies in the precise matching of query terms, and it can handle short queries and keyword-based retrieval tasks well.

[0045] Weighting coefficients for hybrid retrieval This determines the relative contributions of the first similarity score and the second similarity score to the mixed score. The value of α typically ranges between [0,1]. Adjusting α can change the influence of vector retrieval and keyword retrieval on the final results. When When, it relies entirely on the first similarity score, that is, it only uses semantic-based vector retrieval results; when When, it relies entirely on the second similarity score, that is, it only uses the search results of keyword matching; when At the same time, the two will be weighted proportionally, taking into account both semantic matching and keyword matching. The hybrid retrieval score, through reasonable weighting of vector retrieval and keyword retrieval, can comprehensively consider semantic relevance and keyword matching, and adaptively adjust the weight of the hybrid score according to the query type, thus optimizing the retrieval results. To achieve adaptive adjustment of the hybrid score weight according to the query type, in this embodiment, when the first similarity score and the second similarity score are weighted and summed to obtain the hybrid retrieval score, the weight coefficient used is obtained by querying a preset weight configuration table for the type of user query Q. This weight configuration table configures different values ​​for the weight coefficients for question-type queries and keyword-type queries of user query Q, and the weight coefficient corresponding to keyword-type queries is greater than the weight coefficient corresponding to question-type queries. In this embodiment, The selection was not random, but determined based on experimental verification and the characteristics of the query types. Analysis of question-based and keyword-based queries revealed that for question-based queries, semantic understanding and content quality are more important; therefore, vector retrieval should contribute relatively more in this case. For keyword-based queries, accurate keyword matching is more important; therefore, a second similarity score should be added. The weights. Therefore, experimental data show that: 1) for question-type queries, The optimal choice is usually between 0.4 and 0.6. 2) For keyword queries, The optimal choice is typically between 0.6 and 0.75. In this embodiment, it is set... Setting the value to 0.5 as a general approach achieves a good balance between different query types: for question-based queries, A value of 0.5 ensures equal weighting for semantic and keyword matching, adapting to complex semantic understanding and response needs; for keyword-based queries, A score of 0.65 ensures that keyword matching is stronger than semantic matching, thereby improving the accuracy of retrieval.

[0046] Traditional information retrieval methods often rely solely on keyword matching or semantic similarity, neglecting the document's internal structure and contextual information. This can easily lead to fragmented content being incorrectly ranked at the top. In step S102, when performing multi-dimensional quality evaluation on each document fragment in the candidate result set C to obtain its quality score, the multi-dimensional factors include length, structure, information density, and context. By adding structural integrity and contextual integrity scores to the above four content quality dimensions, this problem is effectively solved, and more intelligent and user-friendly retrieval ranking results are provided. Combined with reasonable weight settings and weighted summation, the multi-faceted quality of the document fragment is comprehensively considered, significantly improving the accuracy and readability of the ranking results. In this embodiment, the quality score is obtained by weighted summation of sub-scores of multiple dimensions, including length score, structural integrity score, information density score, and contextual integrity score. Each sub-score is calculated based on the specific features of the document fragment, combined with actual application requirements, aiming to comprehensively measure the quality of the document fragment. (1) Length score: The length score is based on the character length of the document fragment. LThe document is divided into several intervals, including categories such as too short, short, moderate, long and too long, and the document is assigned a corresponding score for each interval. Short document fragments may lack sufficient contextual information, while excessively long document fragments may contain redundant information. Therefore, a reasonable length interval can effectively filter out suitable fragments; (2) Structural integrity score: The structural integrity score starts from the internal structure of the document and considers whether the document contains a title, the number of paragraphs, whether it contains table or list markers, and whether it uses natural language punctuation. The structural integrity of the document directly affects its readability and user experience. Documents with clear structure and distinct levels are usually more in line with retrieval needs. Therefore, the structural integrity score adds points to the readability and integrity of the document; (3) Information density score: The information density score is evaluated based on the coverage of query terms in the document and whether there is a complete query string match. The coverage of query terms reflects the relevance of the document to the query, while the match of the complete query string further verifies whether the document fragment accurately responds to the user's query needs. Information density scoring effectively improves the score of document fragments that match the query intent, ensuring the accuracy of the retrieval results; (4) Contextual integrity scoring: Contextual integrity scoring considers whether the document fragment contains hierarchical path information, whether it contains guiding text (such as "this project", "according to", "therefore", etc.), and whether it ends with a complete ending mark (such as a period, right bracket, etc.). Contextual integrity ensures that the document fragment not only meets semantic matching, but also has logical integrity. The more complete the contextual information of the document, the faster the user can understand the fragment, thereby improving user satisfaction.

[0047] In this embodiment, the length score is the score corresponding to the length range to which the character length of the document fragment belongs. Specifically, it is expressed using a piecewise function, and the function expression is as follows:

[0048] ;

[0049] in, Rate the length. The length of the document fragment is given; as can be seen from the function expression above, when... When, the length score is 1.0; when At that time, the length score was 0.7; when At that time, the length score was 0.3.

[0050] In this embodiment, the structural integrity score is a score corresponding to whether a document fragment contains document structural features. These features include whether it contains a title, the number of paragraphs, whether it contains table or list markers, and whether it uses natural language punctuation. The specific function expression used is:

[0051] ;

[0052] in, For document fragments Structural integrity score, This is a flag indicating whether a title is included, with a value of 0 or 1. This is a flag indicating whether the number of paragraphs exceeds a preset threshold (e.g., 2), with a value of 0 or 1. This is a flag indicating whether a list or table is included, with a value of 0 or 1. This is a flag indicating whether to use natural language punctuation, with a value of 0 or 1. This represents the number of valid structures in the document fragment; here, the value is 4.

[0053] In this embodiment, the information density score is the score corresponding to the coverage of user query Q in the document fragment and the existence of a complete query string match; the specific calculation function expression used is:

[0054] ;

[0055] in, For document fragments Information density score These are the weighting coefficients. To query the number of keywords related to Q for users, For users to query Q and document fragments The number of keywords that appear together For document fragments The identifier that appears in the complete string of the user's query Q is either 0 or 1;

[0056] In this embodiment, the contextual integrity score considers whether the document fragment contains hierarchical path information, whether it contains guiding text, and whether it ends with a complete tag. The specific calculation function expression used is as follows:

[0057] ;

[0058] in, For document fragments Context integrity score This is a flag indicating whether hierarchical path information (such as "1.2.3") is included, with a value of 0 or 1. This is a flag indicating whether the text contains guiding words (such as "In summary"), and its value is 0 or 1. This is a flag indicating whether to end with a complete marker (such as a period), and its value is 0 or 1.

[0059] In this embodiment, the quality score is obtained by weighted summation of sub-scores from multiple dimensions. The functional expression for obtaining the quality score by weighted summation of sub-scores from multiple dimensions is as follows:

[0060] ;

[0061] in, To rate the quality, ~ Here, the weighting coefficients are... , , , This reflects the relative importance of each dimension in the overall score. As an optional implementation method, weights can be set. =0.25, =0.30, =0.25, =0.20, and the sum of the four is 1. Through this weight allocation, this embodiment can reasonably balance the document length, structure, information density, and contextual integrity, thereby improving the overall quality of search results.

[0062] The meanings of the above weight values ​​are as follows:

[0063] =0.25: Document length is generally related to the completeness of its content. Too short a document fragment often lacks sufficient information to effectively answer queries; while too long a fragment may contain redundant information, degrading the user experience. Therefore, a reasonable length score weight should ensure that the document length is moderate. This embodiment, through experimental analysis, found that while the length score has an important impact on content, it is not the most crucial factor. Too short or too long document fragments are usually further optimized or eliminated by structural completeness and contextual completeness scores. Therefore, the length score is assigned... =0.25, to ensure it has sufficient influence on the overall score, while avoiding excessive interference with the overall ranking.

[0064] =0.30: The structural integrity of a document directly impacts its readability and logical clarity. Clear headings, paragraph breaks, tables, and lists—structured elements—significantly improve the user-friendliness of a document, especially important for complex queries. For example, structured content is crucial for question-based queries, as users typically expect clear, hierarchical answers. Therefore, structural integrity scores are highly important in content quality assessment and are given a high weight. =0.30, to highlight the advantages of structured document fragments. This score helps the system prioritize returning document fragments that are well-structured, easy to understand and process, thereby improving user satisfaction and retrieval efficiency.

[0065] =0.25: The information density score reflects the relevance between document fragments and queries, especially the coverage of query terms and the completeness of query string matching. The degree of query term matching is crucial to retrieval accuracy and can effectively filter out document fragments that are highly relevant to the query. Since information density is one of the key factors for retrieval accuracy, but not the only determining factor in long documents, this embodiment sets the weight of the information density score to 0.25. =0.25, to ensure that it plays an important role in the sorting, while avoiding the incorrect placement of fragments with excessive information density at the beginning, thereby ensuring the rationality and accuracy of the final sort.

[0066] =0.20: The contextual integrity score assesses whether a document fragment possesses complete contextual information, such as hierarchical paths, guiding text, and a complete ending. Contextual integrity directly impacts the comprehensibility of the document fragment and the coherence of the answer. While contextual integrity is important for long documents and question-based queries, its impact is relatively smaller for keyword-based queries, as keyword queries focus more on exact matching. Therefore, this embodiment sets the weight of the contextual integrity score to 0.20. =0.20, to ensure that it plays a moderate role in the overall score, while avoiding excessive influence on the final ranking results.

[0067] In summary, this embodiment achieves a comprehensive score for document fragments by assigning different weights to four quality dimensions. Structural integrity and information density play a more important role in the content quality of a document, and therefore they are given relatively higher weights. =0.30, =0.25). Meanwhile, considering that length scoring plays a significant role in the suitability of document content, it is set to... =0.25, ensuring it carries an appropriate weight in content scoring. While contextual integrity is important, its impact on keyword-based queries is relatively small, therefore it is assigned... =0.20, ensuring that an appropriate contextual quality assessment is provided without affecting other scores. These weight settings are not only based on the actual needs of document quality, but their optimality has also been verified through experiments, enabling the scoring mechanism to more accurately and effectively optimize document retrieval ranking results.

[0068] This embodiment analyzes real-world knowledge base retrieval behavior and finds significant differences in users' natural language queries in terms of semantic structure and information needs, which can be divided into two main types: question queries and keyword queries. Question queries typically contain interrogative words or question marks, have relatively complete syntactic structures, and complex semantic expressions. Users intend to obtain directly readable, natural paragraph-style answers, so the ranking process relies more on semantic similarity and the content quality of document fragments. In contrast, keyword queries are mostly composed of noun phrases, focusing on term matching and coverage. Their main goal is to locate relevant fragments containing specific keywords or phrases. Relevance indicators based on word frequency, such as the BM25 algorithm, have a stronger influence on their effectiveness, while content quality is only a secondary factor. Based on the differences in semantic features and retrieval needs between these two types of queries, this embodiment adopts a differentiated weight allocation strategy for different query types to achieve retrieval ranking results that better match user intentions. The query type is determined based on features such as the text length of query Q and whether it contains question marks or interrogative words: when the query length is greater than a preset threshold, or contains question marks and interrogative words, the query is identified as a specific question query; otherwise, it is identified as a keyword query. In this embodiment, step S103, determining the category of user query Q includes:

[0069] S201: Compare the user query Q with each question word in the preset question word set. If the user query Q contains a certain question word, then the user query Q is determined to be a question-type query, and the process ends; otherwise, proceed to step S202. The question word set can be set to any value as needed; for example, in this embodiment, a question word set is defined. For: {what, how, why, whether, how, who, where, how much};

[0070] S202, compare the user query Q with each question symbol in the preset question symbol set. If the user query Q contains a certain question symbol, it is determined that the type of the user query Q is a question type query and the process ends; otherwise, proceed to step S203. The question symbol set can be set to values ​​as needed. For example, in this embodiment, the question symbol set is defined as a set of both Chinese question marks and English question marks.

[0071] S203, perform word segmentation and part-of-speech tagging on the user query Q. If the part-of-speech sequence obtained after word segmentation and tagging includes a continuous combination of verbs, nouns, and auxiliary words, then the user query Q is determined to be a question query and the process ends; otherwise, proceed to step S204. The word segmentation and part-of-speech tagging are existing methods, such as using word segmentation tools like jieba. If the continuous combination of "verb (V) + noun (N) + auxiliary word (U)" appears at least once in the tagged part-of-speech sequence, then the user query Q is determined to be a question query.

[0072] S204, the user's query Q is determined to be a keyword query.

[0073] In step S103, when determining the weights of the mixed search score and quality score for each document fragment based on the category of the user query Q, the type of the user query Q can be queried from a preset weight configuration table to obtain the weights of the mixed search score and quality score. The weight configuration table configures different values ​​for the weights of the mixed search score and quality score for the two types of user query Q: question-type query and keyword-type query.

[0074] In step S104 of this embodiment, for each document fragment in the candidate result set C, the function expression for obtaining the final score of the document fragment by weighting and summing its mixed retrieval score and quality score according to the determined weights is as follows:

[0075] ;

[0076] in, For document fragments The final score, As a weight for quality scoring, This involves a mixed retrieval weighting. As an optional implementation, this embodiment sets a mixed retrieval weighting for specific question-type queries. =0.5, weight of quality score =0.5; For keyword-based queries, set a mixed retrieval weight. =0.65, weight of quality score =0.35. The fundamental reason for using differentiated weight allocation in this embodiment is that the relevance judgment criteria for different types of queries in information retrieval are significantly different. For specific question-type queries, their query expressions usually contain interrogative words or complete syntactic structures, and users expect to obtain answers with coherent semantics, complete context, and direct readability—natural paragraphs. In such query scenarios, simply relying on keyword relevance indicators such as BM25 can easily lead to table fields, short sentence fragments, or fragmented content receiving high scores due to the frequent appearance of keywords, but these fragments do not constitute effective answers; while semantic vector retrieval can capture semantic references better, it may still return structurally incomplete fragments. Therefore, in question-type queries, this embodiment sets the mixed retrieval score and content quality score with equal weight. =0.5、 =0.5, so that semantic relevance and content completeness jointly dominate the ranking, thus ensuring that complete paragraphs and well-defined fragments are presented first. For keyword queries, user input usually consists of several noun phrases, and the core information requirement is to locate document fragments that "contain the keyword", rather than to obtain natural paragraph-style explanations. In existing information retrieval (IR) theories, term relevance models such as BM25 have higher discriminative power when processing short word queries, and their sensitivity to term frequency, inverse document frequency (IDF), and term position is much higher than that of semantic vector models; while embedding semantic similarity is prone to semantic generalization in short queries, resulting in the retrieval of fragments that do not contain the query term but are semantically related to it, thereby reducing the retrieval accuracy of keyword exact matching tasks. Based on the above principles, this embodiment, through statistical analysis of a large number of real query logs, found that in keyword queries, the optimal combination of BM25 relevance and embedding semantic similarity usually tends to be dominated by BM25. Experiments show that when mixed retrieval is weighted When the index is in the range of 0.60 to 0.75, the Top 1 accuracy for keyword queries is the highest, while the quality score has the highest weight. In this scenario, it mainly serves a fine-tuning function, preventing short, fragmented segments from abnormally increasing scores due to repeated keywords. Further parameter sensitivity experiments show that when When the value is less than 0.60, the embedding weight is too large, resulting in paragraphs with "semantic similarity but lacking keywords" in the sorting results, leading to a significant decrease in accuracy; when... When the value is greater than 0.70, the proportion of BM25 is too high, the influence of content quality factors is weakened, and table fields or isolated short sentences are easily ranked incorrectly at the top due to keyword density. =0.65、 =0.35 showed the best performance in both average Top1 and Top3 accuracy across multiple rounds of experiments, and had the lowest proportion of fragmented segments reaching the top. Therefore, this embodiment selects... =0.65、 The weighting of 0.35 is not arbitrary; it is the optimal solution obtained through experiments based on the statistical patterns of relevance feature distribution in keyword query tasks. This ensures that the retrieval ranking maintains the dominance of keyword matching while appropriately utilizing content quality scores to suppress fragmentation noise, thus achieving an optimal balance between query understanding and ranking performance. In summary, the weight allocation design in this embodiment is not arbitrary but based on the fundamental differences between question-based and keyword-based queries in terms of semantic structure, user intent, and retrieval mechanisms. By modeling the contribution of semantic relevance and content quality to different query types, the ranking strategy is adaptively optimized, significantly improving the accuracy and readability of search results.

[0077] In step S105 of this embodiment, the document fragments in the candidate result set C are sorted in descending order according to the final score, and the top k document fragments are output as the retrieval results of the user query Q against the knowledge base.

[0078] To verify the effectiveness of the knowledge base retrieval and ranking method based on multi-dimensional content quality scoring in this embodiment, commonly used performance evaluation metrics in information retrieval and recommendation systems are employed, including first-place accuracy (P@1), normalized discount cumulative gain (NDCG) 3, and mean reciprocal rank (MRR). Figure 2 The figures show a comparison of the experimental results of the methods in this embodiment of the invention, where (a) represents the evaluation index of the mixed retrieval results, and (b) represents the evaluation index of the final retrieval results. (Comparison) Figure 2 As shown in (a) and (b), the method in this embodiment achieves significant improvements in first-order accuracy P@1, normalized discount cumulative gain NDCG 3, and average reciprocal ranking MR through multi-dimensional content quality scoring and a weighted adaptive mechanism of multi-dimensional content quality scoring and mixed retrieval results.

[0079] In summary, the knowledge base retrieval and ranking method based on multi-dimensional content quality scoring in this embodiment includes: performing a hybrid retrieval on the knowledge base according to user query Q to obtain a candidate result set C; obtaining a quality score for each document fragment in the set based on four dimensions: length, structure, information density, and context; determining the weights of the hybrid retrieval score and the quality score according to the category of user query Q; for each document fragment in the candidate result set C, weighting and summing its hybrid retrieval score and its quality score according to the determined weights to obtain the final score; sorting the document fragments in the candidate result set C in descending order according to the final score, and selecting the top k document fragments as... The search results output for Q to the knowledge base have the following advantages: (1) The content quality of document fragments is explicitly modeled through four dimensions: length, structure, information density and context, which improves the completeness and readability of the ranking results; (2) By detecting the completeness of guiding text and ending, complete natural paragraphs are effectively identified and fragmented content is distinguished, so that key descriptive fragments are prioritized; (3) Through query type identification and adaptive weight allocation, better ranking results can be obtained in specific question and keyword query scenarios; (4) Heuristic rules are adopted, no additional labeled data and model training are required, the deployment cost is low and the interpretability is strong.

[0080] Furthermore, this embodiment also provides a knowledge base retrieval and ranking system based on multi-dimensional content quality scoring, including a microprocessor and a memory interconnected, wherein the microprocessor is programmed or configured to execute the knowledge base retrieval and ranking method based on multi-dimensional content quality scoring. This embodiment also provides a computer-readable storage medium storing a computer program or instructions programmed or configured to execute the knowledge base retrieval and ranking method based on multi-dimensional content quality scoring via a processor. This embodiment also provides a computer program product including a computer program or instructions programmed or configured to execute the knowledge base retrieval and ranking method based on multi-dimensional content quality scoring via a processor.

[0081] Those skilled in the art will understand that the technical solutions provided by this invention may take the form of a method, system, or computer program product. Therefore, this invention may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this invention may take the form of a computer program product embodied on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, produce an implementation of the flowchart... Figure 1 One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0082] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A knowledge base retrieval and ranking method based on multi-dimensional content quality scoring, characterized in that, Includes the following steps: S101, perform a hybrid search on the knowledge base based on user query Q to obtain a candidate result set C, wherein the candidate result set C includes multiple document fragments and their corresponding hybrid search scores; The calculation of the hybrid retrieval score includes: calculating the first similarity between the user query Q and the embedding vectors of document fragments in the knowledge base using vector retrieval, and normalizing it to obtain the first similarity score of the embedding vectors; calculating the second similarity between the user query Q and document fragments in the knowledge base using a similarity calculation method based on term frequency and inverse document frequency, and normalizing it to obtain the second similarity score; and weighted summing the first similarity score and the second similarity score to obtain the hybrid retrieval score. S102, perform multi-dimensional quality evaluation on each document fragment in the candidate result set C to obtain its quality score. The multi-dimensional evaluation includes length, structure, information density, and some or all of the contextual dimensions. The quality score is obtained by weighted summation of the sub-scores of multiple dimensions, including length score, structural integrity score, information density score, and contextual integrity score. The length score is the score corresponding to the length range to which the character length of the document fragment belongs. The structural integrity score is the score corresponding to whether the document fragment contains document structural features, including whether it contains a title, the number of paragraphs, whether it contains table or list markers, and whether it uses natural language punctuation. The information density score is the score corresponding to the coverage of user query Q in the document fragment and whether there is a complete query string match. The contextual integrity score is the score corresponding to whether the document fragment contains hierarchical path information, whether it contains guiding text, and whether it has a complete ending mark. The functional expression for the structural integrity score is: ； in, For document fragments Structural integrity score, This is a flag indicating whether a title is included, with a value of 0 or 1. This is a flag indicating whether the number of paragraphs exceeds a preset threshold, with a value of 0 or 1. This is a flag indicating whether a list or table is included, with a value of 0 or 1. This is a flag indicating whether to use natural language punctuation, with a value of 0 or 1. The number of effective structures in a document fragment; the calculation function expression for the information density score is: ； in, For document fragments Information density score These are the weighting coefficients. To query the number of keywords related to Q for users, For users to query Q and document fragments The number of keywords that appear together For document fragments The identifier for the complete string of the user's query Q is present, with a value of 0 or 1; the expression for the calculation function of the context integrity score is: ； in, For document fragments Context integrity score This is a flag indicating whether hierarchical path information is included, with a value of 0 or 1. This is a flag indicating whether or not the text contains guiding text; its value is 0 or 1. This is a flag indicating whether the tag ends with a complete sequence; its value is 0 or 1. S103, determine the category of user query Q, and determine the weight of the mixed retrieval score and quality score for each document fragment based on the category of user query Q; S104, for each document fragment in the candidate result set C, the combined retrieval score and its quality score are weighted and summed according to the determined weights to obtain the final score of the document fragment; S105, sort the document fragments in the candidate result set C in descending order according to the final score, and output the top k document fragments as the retrieval results of the user query Q against the knowledge base.

2. The knowledge base retrieval and ranking method based on multi-dimensional content quality scoring according to claim 1, characterized in that, Step S101 includes: calculating the first similarity between the user query Q and the embedding vectors of document fragments in the knowledge base using vector retrieval, and normalizing the embedding vectors to obtain the first similarity score; calculating the second similarity between the user query Q and document fragments in the knowledge base using a similarity calculation method based on word frequency and inverse document frequency, and normalizing the second similarity score; weighted summing the first similarity score and the second similarity score to obtain a mixed retrieval score; sorting the document fragments in the knowledge base in descending order according to the mixed retrieval score; and selecting a corresponding number of document fragments according to the size of a preset candidate result set C to obtain a candidate result set C composed of the selected document fragments and their mixed retrieval scores.

3. The knowledge base retrieval and ranking method based on multi-dimensional content quality scoring according to claim 2, characterized in that, When the first similarity score and the second similarity score are weighted and summed to obtain the mixed search score, the weight coefficient used is obtained by querying a preset weight configuration table for the type of user query Q. The weight configuration table configures different values ​​for the weight coefficient for the two types of user query Q: question-type query and keyword-type query, and the weight coefficient corresponding to the keyword-type query is greater than the weight coefficient corresponding to the question-type query.

4. The knowledge base retrieval and ranking method based on multi-dimensional content quality scoring according to any one of claims 1 to 3, characterized in that, In step S103, determining the category of user query Q includes: S201, compare the user query Q with each question word in the preset question word set. If the user query Q contains a certain question word, then determine that the user query Q is a question type query and end; otherwise, proceed to step S202. S202, compare the user query Q with each question symbol in the preset question symbol set. If the user query Q contains a question symbol, then the user query Q is determined to be a question type query and the process ends; otherwise, proceed to step S203. S203, perform word segmentation and part-of-speech tagging on the user query Q. If the part-of-speech sequence obtained after word segmentation and part-of-speech tagging includes a continuous combination of verbs, nouns and auxiliary words, then the user query Q is determined to be a question query and the process ends; otherwise, proceed to step S204. S204, the user's query Q is determined to be a keyword query.

5. A knowledge base retrieval and ranking system based on multi-dimensional content quality scoring, comprising interconnected microprocessors and memory, characterized in that, The microprocessor is programmed or configured to execute the knowledge base retrieval and ranking method based on multi-dimensional content quality scoring as described in any one of claims 1 to 4.

6. A computer-readable storage medium storing a computer program or instructions, characterized in that, The computer program or instructions are programmed or configured to execute, via a processor, the knowledge base retrieval and ranking method based on multi-dimensional content quality scoring as described in any one of claims 1 to 4.

7. A computer program product, comprising a computer program or instructions, characterized in that, The computer program or instructions are programmed or configured to execute, via a processor, the knowledge base retrieval and ranking method based on multi-dimensional content quality scoring as described in any one of claims 1 to 4.

Images