A resource recommendation method and system based on hybrid retrieval RAG

Abstract

The application relates to the technical field of intelligent recommendation, and discloses a resource recommendation method and system based on a hybrid retrieval RAG, which comprises the following steps: collecting resource text data, constructing a vector library and a label library; expanding a user question based on a language model to obtain a plurality of semantic expansion questions; performing intention recognition to determine whether the user question is a resource recommendation question, and determining a target classification type if the user question is a resource recommendation question; filtering data according to field definitions in the label library to obtain a candidate knowledge segment set; obtaining candidate vectors, mapping the user question and the semantic expansion questions into query vectors, calculating the similarity between the query vectors and each candidate vector, and selecting knowledge supplement content; and performing resource splicing on all the knowledge supplement content to generate a resource recommendation answer. The application combines a structured label filtering mechanism and a semantic vector precision sorting mechanism, and overcomes the problems of recall redundancy and matching deviation caused by the fact that the existing RAG system only depends on semantic similarity retrieval.

Description

Technical Field

[0001] This invention relates to the field of intelligent recommendation technology, and more specifically, to a resource recommendation method and system based on hybrid retrieval (RAG). Background Technology

[0002] With the development of artificial intelligence, especially Large Language Model (LLM) technology, Retrieval-Augmented Generation (RAG) has become an important means to realize intelligent question answering and knowledge recommendation systems. RAG combines information retrieval and natural language generation technologies, and can inject external knowledge as context into the language model, thereby improving the accuracy, contextual relevance, and content coverage of question answering in specific domains.

[0003] In areas with high semantic complexity, such as government policies, scientific research services, and science and technology innovation resources, users often raise highly targeted resource query needs. Existing RAGs mostly rely on semantic vectors for similarity recall, failing to utilize the structured tag information already present in the resource data for precise conditional filtering, resulting in redundant returned content or matching biases. They cannot automatically identify the user's intent type or the corresponding resource category, nor can they rewrite user questions in diverse ways, leading to insufficient recall coverage. User questions are difficult to automatically convert into actionable structured search logic, preventing the system from performing field-level filtering of the resource database. Simply relying on vector space recall and directly using it to generate answers, without setting clear semantic thresholds and fine-tuning strategies, easily introduces irrelevant or low-quality text.

[0004] Therefore, it is necessary to design a resource recommendation method and system based on hybrid retrieval RAG to solve the problems existing in the current technology. Summary of the Invention

[0005] In view of this, the present invention proposes a resource recommendation method and system based on hybrid retrieval RAG, aiming to solve the problems of lack of structured constraints, semantic expansion and intent judgment capabilities, and low accuracy of semantic filtering.

[0006] In one aspect, this invention proposes a resource recommendation method based on hybrid retrieval RAG, comprising:

[0007] Collect resource text data, and construct a vector library and a tag library based on the resource text data;

[0008] Collect user questions, expand the user questions based on a language model, and obtain several semantically extended questions;

[0009] For all the semantic expansion questions, perform intent recognition to determine whether the user question is a resource recommendation question; if so, determine the target category type to which the resource belongs.

[0010] The user questions and semantic expansion questions are transformed into structured query statements. The data is then filtered according to the field definitions in the tag library to obtain a set of candidate knowledge fragments that meet the tag conditions.

[0011] Semantic vector encoding is performed on each resource text in the candidate knowledge fragment set to obtain candidate vectors; based on semantic encoding, the user question and semantic extension question are mapped to query vectors, the similarity between the query vector and each candidate vector is calculated, and fragments with similarity greater than a similarity threshold are selected as knowledge supplementary content;

[0012] All the aforementioned supplementary knowledge is combined to generate resource-recommended answers.

[0013] Furthermore, when collecting resource text data and constructing vector and tag libraries based on the resource text data, the process includes:

[0014] The resource text data is preprocessed, including text segmentation, format regularization, and noise removal.

[0015] Constructing the vector library: The preprocessed resource text data is segmented into blocks according to semantic logic. The segmentation includes segmentation by title level and segmentation by parent-child semantic structure.

[0016] Each text block is vectorized and encoded using a semantic embedding model to obtain a semantic vector.

[0017] The semantic vectors are mapped to the original resource text data, and the mapping relationship is stored in the database to obtain the vector library;

[0018] Building a tag library: Extracting keywords and entities related to application conditions from preprocessed resource text data, and constructing tag names and tag values;

[0019] Resource content is tagged and labeled based on rule templates to form a tag expression;

[0020] Based on business logic, several tag expressions are combined to build a tag condition tree, and the tag content is bound to the corresponding resource fragment;

[0021] The bound tag information is organized into a structured table format and stored in a database to obtain the tag library.

[0022] Furthermore, when collecting user questions and expanding the user questions based on a language model to obtain several semantically extended questions, these include:

[0023] Based on a language model, the user's question is rewritten to generate multiple semantically different but meaningful query expressions. Figure 1 The way to express the problem;

[0024] Before rewriting the question, the language model is guided to generate diverse questions by constructing a prompt word template, which includes instruction information, language style requirements and context memory loading fields.

[0025] During the question rewriting process, the current session history information is taken into account.

[0026] Furthermore, when performing intent recognition on all the aforementioned semantic expansion questions to determine whether the user question is a resource recommendation question, the process includes:

[0027] Construct a predefined list of intent categories, including resource recommendations and non-resource inquiries. Input all the semantic expansion questions and generate intent classification results.

[0028] A majority vote is conducted on the identification results of multiple semantic expansion questions. If the result belongs to the resource recommendation intent, the target classification type of the resource is determined based on the semantic keywords; otherwise, the recommendation process is terminated and a rejection is triggered.

[0029] Furthermore, when transforming the user question into a structured query statement and filtering the data according to the field definitions in the tag library to obtain a set of candidate knowledge fragments that meet the tag conditions, the process includes:

[0030] Extract the table structure information of the tag library, which includes: field name, field type, field comment, tag name and tag value enumeration set;

[0031] Based on the user question and the semantic extension question, the field descriptions and annotations in the tag library are semantically matched to generate a structured SQL statement that satisfies the semantic intent. The structured SQL statement includes a WHERE statement to specify the matching conditions between the tag field and the tag value.

[0032] If the structured SQL statement cannot be parsed or the query fails, a fallback strategy rule is used to match the rules for mapping keywords and tag names, and the SQL query conditions are manually constructed.

[0033] Based on the structured SQL statement, a query operation is performed in the tag library to obtain a set of candidate knowledge fragments that match the semantic conditions of the user's question.

[0034] Furthermore, semantic vector encoding is performed on each resource text in the candidate knowledge fragment set to obtain candidate vectors, including:

[0035] Each resource text in the candidate knowledge fragment set is normalized, and the normalization process includes removing HTML tags, standardizing punctuation, standardizing spaces, and pruning the length of the text.

[0036] Candidate knowledge fragments are encoded using a semantic embedding model consistent with the vector library construction phase;

[0037] The semantic embedding model maps each selected knowledge fragment to a fixed-dimensional semantic vector to obtain the candidate vector. The candidate vector is obtained during the encoding process using mean pooling or CLS token pooling strategies.

[0038] Furthermore, based on semantic encoding, the user question and semantic expansion question are mapped into query vectors. The similarity between the query vector and each candidate vector is calculated. When selecting segments with similarity greater than a similarity threshold as supplementary knowledge content, the process includes:

[0039] The user question and the semantic extension question are encoded based on the semantic embedding model, and the user question is mapped to a query vector. The query vector and the candidate vector have the same dimension and semantic space.

[0040] For each candidate vector, a cosine similarity is calculated with the query vector to obtain the similarity of each candidate vector;

[0041] All the aforementioned similarities are compared with a similarity threshold to obtain the supplementary knowledge content.

[0042] Furthermore, when comparing all the aforementioned similarities with a similarity threshold to obtain the supplementary knowledge content, the process includes:

[0043] When the similarity is greater than or equal to the similarity threshold, the resource text corresponding to the similarity is used as the knowledge supplement content. If multiple resource texts meet the condition, semantic relevance analysis is performed on the multiple resource texts, and they are sorted from high to low similarity and from high to low relevance. The top n texts are selected as the knowledge supplement content.

[0044] Furthermore, when generating resource-recommended answers by combining all the aforementioned supplementary knowledge content, the process includes:

[0045] The selected knowledge supplements are sorted from high to low according to their similarity to the query vector, and the sorted content is concatenated into the context input. During the concatenation process, a separator is inserted between each content, and the original user question and semantic expansion question are embedded in the concatenation context.

[0046] The generated answer output is post-processed, including removing invalid statements, standardizing terminology style, deduplicating and classifying, to obtain the recommended answer for the resource.

[0047] Compared with existing technologies, the beneficial effects of this invention are as follows: This application integrates structured tag filtering and semantic vector fine-ranking mechanisms, overcoming the problems of recall redundancy and matching bias caused by relying solely on semantic similarity retrieval in existing RAG systems. It performs dual modeling of resource text: on the one hand, it constructs a semantic vector library to capture deep semantic information of resource content; on the other hand, it constructs a tag library to extract conditional attribute information of policies or resources, supporting precise matching based on field rules. It uses a language model to rewrite and semantically expand user questions in various ways, improving recall coverage; simultaneously, it ensures the accuracy of recommendation logic through intent recognition and resource type classification. It automatically converts user intent into SQL form, enabling conditional filtering of tag fields to obtain highly relevant candidate knowledge fragments. It performs semantic encoding and similarity calculation on candidate content, sets thresholds, and performs fine-ranking filtering, generating structurally reasonable and semantically targeted recommended answers through contextual concatenation. This improves the accuracy of resource recommendations, contextual matching degree, and controllability of generated content.

[0048] On the other hand, this application also provides a resource recommendation system based on hybrid retrieval RAG, for applying the above-mentioned resource recommendation method based on hybrid retrieval RAG, including:

[0049] The acquisition unit is configured to acquire resource text data and construct a vector library and a tag library based on the resource text data.

[0050] The processing unit is configured to collect user questions, expand the user questions based on a language model, and obtain several semantically extended questions;

[0051] The identification unit is configured to perform intent recognition on all the semantic extension questions, determine whether the user question is a resource recommendation question, and if so, determine the target category type to which the resource belongs.

[0052] The filtering unit is configured to transform the user questions and semantic extension questions into structured query statements, filter the data according to the field definitions in the tag library, and obtain a set of candidate knowledge fragments that meet the tag conditions.

[0053] The matching unit is configured to perform semantic vector encoding on each resource text in the candidate knowledge fragment set to obtain a candidate vector; based on the semantic encoding, map the user question and semantic extension question into a query vector, calculate the similarity between the query vector and each candidate vector, and select fragments with similarity greater than a similarity threshold as knowledge supplementary content;

[0054] The output unit is configured to concatenate all the aforementioned supplementary knowledge content to generate a resource-recommended answer.

[0055] It is understandable that the resource recommendation methods and systems based on hybrid retrieval RAG described above have the same beneficial effects, and will not be elaborated further here. Attached Figure Description

[0056] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0057] Figure 1 A flowchart illustrating a resource recommendation method based on hybrid retrieval RAG provided in an embodiment of the present invention;

[0058] Figure 2 This is a functional block diagram of a resource recommendation system based on hybrid retrieval RAG provided in an embodiment of the present invention. Detailed Implementation

[0059] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the disclosure to those skilled in the art. It should be noted that, unless otherwise specified, embodiments and features in the embodiments of the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0060] Existing retrieval-enhanced generation systems suffer from insufficient multimodal data fusion and a simplistic retrieval logic in their resource recommendation processes. Recall mechanisms based solely on semantic vectors fail to effectively integrate structured tag information, resulting in missing key field constraints in resource filtering conditions and the inclusion of a large amount of redundant text irrelevant to user needs in the returned results. User query intent recognition relies solely on a single question expression, failing to cover diverse expressions through semantic expansion, leading to missed detections of potentially relevant resources. The lack of an automated conversion mechanism between query statements and database fields prevents structured tags from participating in precise condition filtering, directly impacting the recall accuracy of candidate knowledge fragments. The lack of dynamic adjustment capabilities in semantic similarity threshold settings allows low-quality text to infiltrate the generation context, affecting the information density and accuracy of the final recommendation results.

[0061] For example, in the government's intelligent recommendation platform for science and technology innovation resources, when users submit complex query requests with multiple constraints, the system only uses vector space calculation to match similar text fragments. When a user queries "policy on sharing R&D equipment in the biomedical field," the existing method cannot identify the resource category tag corresponding to "biomedical," nor can it map "sharing R&D equipment" to the policy type field for structured filtering. This results in the returned results containing a large number of irrelevant policy documents such as those related to medical talent recruitment and drug approval processes. The lack of a semantic expansion mechanism prevents the system from generating relevant statements such as "Guidelines for Open Use of Biomedical Laboratory Instruments," further reducing the recall rate of potentially high-value resources. The lack of tag condition filtering in vector similarity calculation causes non-policy texts such as equipment maintenance procedures to be misjudged as valid results due to semantic similarity.

[0062] If the above problems are not addressed, the system will face a long-term bottleneck in both retrieval accuracy and recall. The idle nature of structured tags prevents the activation of database field-level filtering capabilities, resulting in a loss of precise control over the resource recommendation process. Insufficient semantic expansion leads to incomplete coverage of user intent, and the missed detection rate of key resources increases exponentially with query complexity. The inclusion of low-quality text in the generation context reduces content credibility, requiring users to spend extra time on manual screening and verification. Ultimately, this significantly reduces the practical value of the resource recommendation system, making it difficult to meet the demands for high-precision, high-efficiency professional domain services.

[0063] For this, please refer to Figure 1 As shown, this application proposes a resource recommendation method based on hybrid retrieval RAG, including:

[0064] S100: Collect resource text data and construct vector library and tag library based on the resource text data.

[0065] S200: Collect user questions, expand user questions based on language models, and obtain several semantically extended questions.

[0066] S300: Perform intent recognition on all semantic extension questions, determine whether the user's question is a resource recommendation question, and if so, determine the target category type to which the resource belongs.

[0067] S400: Transforms user questions and semantic extension questions into structured query statements, filters data based on field definitions in the tag library, and obtains a set of candidate knowledge fragments that meet the tag conditions.

[0068] S500: Semantic vector encoding is performed on each resource text in the candidate knowledge fragment set to obtain candidate vectors. Based on semantic encoding, user questions and semantic extension questions are mapped to query vectors. The similarity between the query vector and each candidate vector is calculated, and fragments with similarity greater than a similarity threshold are selected as knowledge supplementary content.

[0069] S600: Combines all supplementary knowledge content to generate resource-recommended answers.

[0070] Specifically, Hybrid Retrieval Aggregation (RAG) refers to a retrieval enhancement generation method that combines vector retrieval and structured tag retrieval. It can be implemented using semantic vector matching and structured query language to improve the accuracy and contextual relevance of resource recommendations. A vector library is a database storing semantic vectors of resource text. It can be implemented using a semantic embedding model to encode chunked text, supporting semantic similarity calculation. A tag library is a database storing structured tag information of resources. It can be implemented using keyword extraction and rule template annotation, supporting structured conditional filtering. Semantic expansion questions refer to diverse question expressions generated through a language model. It can be implemented using prompt word templates to guide generation, expanding the scope of user intent. Intent recognition determines whether a user's question belongs to a resource recommendation request. It can be implemented using predefined intent classification and majority voting mechanisms to determine whether to trigger the recommendation process. Structured query statements convert user questions into executable filtering conditions in the database. It can be implemented using semantic matching tag fields to generate SQL statements, accurately filtering candidate resources. Semantic vector encoding converts text into a high-dimensional vector representation. It can be implemented using an embedding model consistent with the vector library, used to calculate semantic similarity. Similarity threshold refers to the minimum matching standard for filtering candidate vectors. It can be achieved through cosine similarity calculation and threshold setting, and is used to filter low-quality text. Resource splicing refers to integrating the filtered knowledge fragments into a coherent answer. It can be achieved through sorting splicing and post-processing strategies, and is used to generate structured recommendation results.

[0071] This application solves the problems of insufficient accuracy and redundancy caused by a single retrieval mode in existing RAGs in resource recommendation scenarios by integrating semantic vector matching and structured tag filtering through a hybrid retrieval mechanism, combined with intent recognition and question expansion technology, thereby achieving resource recommendation with high accuracy and high coverage.

[0072] The working process and principle of this application are as follows: Resource text data is collected, and a vector library and a tag library are constructed based on this data. The vector library stores the semantic vector representation of the resource text, and the tag library stores the structured tag information of the resources. User questions are collected, and semantic expansion is performed on these questions based on a language model to generate multiple semantically similar but differently expressed extended questions. This step aims to increase question coverage and improve the recall rate of relevant resources. Then, intent recognition is performed on all semantically expanded questions to determine whether the user question is a resource recommendation question. If it is a resource recommendation question, the target category of the resource is further determined. This step can filter out non-recommendation queries, improving efficiency. User questions and semantically expanded questions are transformed into structured query statements, and the data is filtered according to the field definitions in the tag library to obtain a set of candidate knowledge fragments that meet the tag conditions. This step uses structured tag information for precise filtering, narrowing the candidate range. Each resource text in the candidate knowledge fragment set is semantically vector-encoded to obtain a candidate vector. Simultaneously, user questions and semantically expanded questions are mapped to query vectors, the similarity between the query vector and each candidate vector is calculated, and fragments with similarity greater than a threshold are selected as supplementary knowledge content. This step combines vector similarity matching to further improve retrieval accuracy. Finally, all supplementary knowledge is concatenated to generate the final resource recommendation answer. The entire process combines structured tag filtering and semantic vector matching to realize a hybrid retrieval RAG resource recommendation method.

[0073] As a preferred embodiment, the solution of this application is specifically implemented as follows:

[0074] Collect textual data from resources, including policy documents, research reports, and patent literature. Preprocess the collected text data, including removing HTML tags, standardizing punctuation, and normalizing spaces. Then, construct a vector library and a tag library. During vector library construction, the BERT model is used to encode the text in blocks, generating a 768-dimensional vector for each text block. During tag library construction, keywords and entities, such as policy type, applicable field, and publication time, are extracted from the text to form structured tags.

[0075] After collecting user questions, the T5 model is used to rewrite the questions, generating 5-10 semantically similar extended questions. For example, the original question "What are the equipment sharing policies in the biopharmaceutical field?" could be expanded to include "What are the regulations for the open use of instruments and equipment in the biopharmaceutical industry?"

[0076] The extended question undergoes intent recognition, using a pre-trained BERT classifier to determine if it is a resource recommendation question. If so, the target resource type is further identified, such as "policy document".

[0077] Transform the user's question into an SQL query, such as "SELECT * FROM policy WHERE field=Biomedicine AND type=Equipment sharing". Execute the SQL query to obtain candidate knowledge fragments.

[0078] Candidate segments are vectorized using the same BERT model as the vector library. The cosine similarity between the user's question vector and the candidate vectors is calculated, and segments with a similarity greater than 0.8 are selected as supplementary knowledge content.

[0079] Finally, the selected knowledge content is sorted and concatenated according to similarity to generate a recommended answer. The answer includes information such as the policy name, a summary of the main content, and the scope of application.

[0080] Through the above scheme, this application achieves hybrid retrieval of structured tags and semantic vectors, improving the accuracy of resource recommendation. The generation of semantic expansion questions increases the recall rate of potentially relevant resources. The intent recognition mechanism avoids invalid retrieval for non-recommendation questions. Automatic generation of structured query statements enables precise tag condition filtering. The setting of semantic similarity thresholds ensures the quality of supplementary knowledge content. Overall, this scheme improves the performance of resource recommendation in complex query scenarios.

[0081] In some of the solutions mentioned above in this application, a scheme to construct a vector library and a tag library is proposed to support resource recommendation. However, in this process, the resource text data may have problems such as disordered format, noise interference, and unreasonable semantic segmentation, which leads to insufficient accuracy of subsequent vector encoding and tag extraction. At the same time, the tag library is not structured enough, making it difficult to support accurate conditional filtering.

[0082] This application further proposes a scheme for collecting resource text data and constructing vector and tag libraries based on the resource text data, including: preprocessing the resource text data, including text segmentation, format regularization, and noise removal. When constructing the vector library, the preprocessed resource text data is segmented into blocks according to semantic logic, including segmentation by title level and parent-child semantic structure. Each segmented text block is vectorized and encoded using a semantic embedding model to obtain semantic vectors. A mapping relationship is established between the semantic vectors and the original resource text data, and the mapping relationship is stored in a database to obtain the vector library. When constructing the tag library, keywords and entities related to application conditions are extracted from the preprocessed resource text data to construct tag names and tag values. Resource content is tagged based on rule templates to form tag expressions. Several tag expressions are combined according to business logic to establish a tag condition tree, and the tag content is bound to the corresponding resource fragment. The bound tag information is organized into a structured table format and stored in a database to obtain the tag library.

[0083] The preprocessing steps involve segmenting long texts into logical paragraphs, standardizing text encoding and layout, and removing noise by filtering out non-textual symbols and irrelevant content. The segmentation uses heading-level segmentation to identify chapter structure and parent-child semantic structure segmentation to identify logical nesting relationships between paragraphs. Vectorization encoding uses a pre-trained semantic embedding model, such as BERT or Sentence-BERT, to map text blocks into 768-dimensional vectors. Tag names and values ​​are extracted using named entity recognition and keyword extraction algorithms, such as a BERT-CRF-based entity recognition model. The tag condition tree combines tag expressions using logical operators, such as nested combinations of AND, OR, and NOT.

[0084] Specifically, the preprocessing stage first cleans the original resource text, such as removing HTML tags, standardizing full-width and half-width punctuation, and merging redundant spaces, ensuring that subsequent processing is based on standardized text. In the block segmentation stage, for policy documents or technical documents, text blocks are divided according to chapter headings, for example, treating "Chapter 1" to "Chapter 3" as independent blocks. Simultaneously, parent-child relationships between paragraphs are identified, such as merging a definition paragraph with its explanatory paragraph into the same block. During vector library construction, each text block is encoded using a semantic embedding model, indexed and mapped to the original text, and stored in a vector database such as Milvus or FAISS. During tag library construction, conditional fields in the resource content are matched using rule templates. For example, "Applicable to: Enterprises" is labeled as the tag name "Applicable to" and the tag value "Enterprise." Multiple tag expressions are combined using a condition tree, such as "(tag A = value 1 AND tag B = value 2) OR tag C = value 3." Finally, tag names, tag values, and associated resource fragment IDs are stored in a structured table. Therefore, the vector library supports semantic similarity retrieval, and the tag library supports structured conditional filtering. The two work together to improve the accuracy and efficiency of resource retrieval.

[0085] As a preferred embodiment, the solution of this application is specifically implemented as follows:

[0086] When collecting resource text data and constructing vector and tag libraries based on the resource text data, the following steps are included:

[0087] The resource text data undergoes preprocessing. Preprocessing includes text segmentation, formatting, and noise removal. Specifically, text segmentation can be done using natural paragraphs or fixed-length segments. Formatting includes standardizing font, font size, and line spacing. Noise removal includes removing special characters and HTML tags.

[0088] Build a vector library. The preprocessed resource text data is segmented into blocks according to semantic logic. Block segmentation includes segmentation by heading level and segmentation based on parent-child semantic structure. For example, segmentation can be based on document heading level, or on parent-child structure based on semantic relationships between paragraphs.

[0089] Each text block is vectorized and encoded using a semantic embedding model to obtain a semantic vector. Pre-trained language models such as BERT and RoBERTa can be used as semantic embedding models to map text into fixed-dimensional vector representations.

[0090] A vector library is created by mapping semantic vectors to the original resource text data and storing the mapping relationship in a database. This can be done using key-value pairs, with the vector as the key and the original text as the value.

[0091] Build a tag library. Extract keywords and entities related to the application conditions from the preprocessed resource text data to construct tag names and tag values. Named entity recognition technology can be used to extract key entities, such as time, location, and organization name.

[0092] Resource content is tagged and labeled using rule templates to form tagged representations. Rule templates may include "if-then" structures, matching corresponding tags based on text content features.

[0093] Based on business logic, several tag expressions are combined to build a tag condition tree, and the tag content is bound to the corresponding resource fragment. A decision tree structure can be used to organize multiple tag conditions.

[0094] The bound tag information is organized into a structured table format and stored in a database to obtain a tag library. A relational database can be used for storage, with fields such as resource ID, tag name, and tag value.

[0095] Through the above technical solutions, this application achieves efficient preprocessing and structured storage of resource text data. The construction of the vector library supports subsequent semantic similarity retrieval, while the tag library provides a foundation for precise conditional filtering. This hybrid indexing method improves the accuracy and efficiency of resource retrieval, effectively solving the limitations of a single retrieval method. Simultaneously, tagging makes resource content easier to manage and classify, providing reliable data support for subsequent personalized recommendations.

[0096] In some of the solutions mentioned above in this application, the user's question expression is too simple, resulting in insufficient semantic expansion, which fails to cover diverse query intents and affects the recall and accuracy of subsequent searches.

[0097] This application further proposes a method for collecting user questions, expanding user questions based on a language model, and obtaining several semantically expanded questions, including: rewriting user questions based on the language model to generate multiple semantically different but query-meaningful questions. Figure 1 The question expression format is optimized. Before question rewriting, a prompt word template is constructed to guide the language model in generating diverse questions. The prompt word template includes instruction information, language style requirements, and context memory loading fields. During question rewriting, the current session history information is incorporated.

[0098] The process includes several key elements: Problem rewriting, which uses a language model to generate extended questions that are semantically equivalent to the original but expressed differently (e.g., rewriting "How to apply for research funding" as "What are the steps in the research funding application process?" or "What materials are needed to obtain research funding?"); Prompt word templates constrain the generation direction through preset instruction information (e.g., requiring the generation of interrogative or declarative sentences containing keywords); and language style requirements specifying the formality or colloquialism of the generated questions. Context memory loading fields are used to inject domain terminology or business rules. Conversation history information is cached to retrieve content already interacted in the current dialogue; for example, using the preceding question "Application conditions for science and technology innovation projects" as context, the extended question can be associated with related content such as the "List of Application Materials".

[0099] Specifically, after receiving the user's original question, the language model first loads the instruction information from the prompt word template, such as "Generate three different expressions of the question, which must include keywords, application conditions, and a list of materials." The model adjusts the generated sentence structure according to the language style requirements in the template, for example, using the interrogative sentence "What conditions must be met to apply for a science and technology innovation project?" or the declarative sentence "Please explain the eligibility requirements for applying for a science and technology innovation project." The context memory loading field uses the "project type is national level" mentioned in the current session as a constraint to ensure that the generated extended questions are limited to a specific category. During question rewriting, the model combines the cached previous dialogue content, such as associating "application process" with "material list," to generate extended questions such as "the specific steps and required documents for a national-level science and technology innovation project from application to approval." The generated semantic extended questions undergo majority voting by the subsequent intent recognition module to filter out effective queries that match the intent of resource recommendation, thereby improving the coverage of search conditions and the matching accuracy of candidate knowledge fragments.

[0100] As a preferred embodiment, the solution of this application is specifically implemented as follows:

[0101] After collecting user questions, the questions are rewritten based on a language model. The question rewriting process uses the GPT-3.5 model to generate multiple questions with different semantic expressions but different query meanings. Figure 1The prompt word template guides the language model to generate diverse questions before question rewriting. The prompt word template includes instruction information, language style requirements, and a context memory loading field. The instruction information is "Please rewrite the following questions to generate 5 semantically similar but differently expressed questions." The language style requirement is "Use concise and colloquial expressions." The context memory loading field contains the user's historical question-and-answer records. During question rewriting, the current session history is used to ensure that the generated questions are consistent with the user's query intent.

[0102] For example, after rewriting the user question "What policy support is suitable for college students to start businesses?", the following semantically expanded question might be generated:

[0103] 1. What preferential policies are available for college students starting their own businesses?

[0104] 2. What support measures are available for college students starting their own businesses?

[0105] 3. How can the government support college students in starting their own businesses?

[0106] 4. What policy benefits can college student entrepreneurs enjoy?

[0107] 5. What kind of support can college students receive for starting their own businesses?

[0108] Through the above technical solution, this application can generate multiple semantically similar but differently expressed questions, expanding the semantic coverage of the questions. By introducing prompt word templates and contextual information, the generated questions are closer to the user's actual query intent. This method improves the accuracy and comprehensiveness of subsequent resource recommendations and effectively solves the problem of insufficient resource matching that may be caused by a single question expression.

[0109] In some of the solutions mentioned above in this application, the intent recognition of user questions relies on a single question input, which is prone to misjudgment due to the ambiguity or diversity of user expressions. This can lead to inaccurate screening of target resource categories or incorrect triggering of rejection, affecting the accuracy of recommendations and user experience.

[0110] This application further proposes constructing a predefined list of intent categories, including resource recommendation and non-resource-related inquiries. All semantic expansion questions are input, and intent classification results are generated. A majority vote is performed on the identification results of multiple semantic expansion questions. If the result belongs to the resource recommendation intent, the target classification type of the resource is determined based on semantic keywords. Otherwise, the recommendation process is terminated and a rejection is triggered.

[0111] The intent category list explicitly distinguishes between resource recommendations and non-resource inquiries through predefined methods, covering the core needs of users. Semantic extension questions are input into the intent classification model in diverse forms, enhancing the input diversity for intent recognition. A majority voting mechanism reduces the probability of misclassification for a single question by statistically analyzing the classification results of multiple extension questions. Semantic keyword matching, based on a keyword library within the resource classification system and combined with the intent results output by the classification model, determines the specific target classification type.

[0112] Specifically, after semantically expanding a user's question to generate multiple variants, each variant is input into a pre-trained intent classification model. This model outputs the intent category probability distribution for each question based on a list of classification labels. The classification results of multiple questions are statistically analyzed using majority voting. If the proportion of intent related to resource recommendations exceeds a preset threshold, the user's need is determined to belong to the resource recommendation category. Further, for user questions determined to be resource recommendation type, keywords related to resource classification, such as "policy," "scientific research services," and "scientific and technological innovation resources," are extracted from the original question and variant questions. These keywords are then matched with preset classification labels to determine the target category of the resource. If the majority voting result indicates a non-resource intent, the recommendation process is terminated and a rejection message is returned. This process reduces the risk of misjudgment based on a single question through multiple question inputs and majority voting, and improves classification accuracy by combining keyword matching, ensuring that resource selection criteria align with the user's actual needs.

[0113] As a preferred embodiment, the solution of this application is specifically implemented as follows:

[0114] Construct a predefined list of intent categories, including resource recommendations and non-resource inquiries. Input all semantic expansion questions and generate intent classification results. Perform a majority vote on the identification results of multiple semantic expansion questions. If the result belongs to the resource recommendation intent, determine the target classification type of the resource based on semantic keywords. Otherwise, terminate the recommendation process and trigger a rejection.

[0115] Specifically, the predefined list of intent categories includes multiple categories such as "resource recommendation," "policy consultation," and "business processing." Each semantic extension question is processed by an intent classification model to obtain the corresponding intent category. For example, for the extension question "What technology projects are suitable for startups to receive funding?", the intent classification result is "resource recommendation." And for "How to apply for a business license?", the intent classification result is "business processing."

[0116] Furthermore, the intent classification results of all extended questions are statistically analyzed, and a simple majority voting method is used to determine the final intent. If more than half of the extended questions are classified as "resource recommendation," then the user's question is determined to be in the resource recommendation category. Therefore, keywords such as "startup" and "technology project" are extracted from the extended questions and matched with a pre-defined resource classification system to determine the target category as "technology innovation funding."

[0117] If the final intent does not fall under the category of resource recommendation, the subsequent resource retrieval and recommendation process will be terminated. Alternatively, a rejection mechanism will be triggered, explaining to the user that the current question is not applicable to resource recommendation, and guiding the user to ask a new question or be transferred to another service module.

[0118] Through the above technical solutions, this application achieves accurate identification and classification of user question intent. The voting mechanism using multiple semantically expanded questions improves the robustness of intent judgment and avoids misjudgments that might arise from a single question expression. Simultaneously, for non-resource recommendation questions, unnecessary retrieval processes are terminated promptly, improving response efficiency. Furthermore, the determination of the target category lays the foundation for subsequent accurate resource matching, helping to improve the relevance and targeting of recommendation results.

[0119] In some of the solutions mentioned above in this application, user questions and semantic extension questions are transformed into structured query statements to filter tag library data. However, when generating structured query statements, the complex field definitions of the tag library or the semantic understanding deviation may cause the statement to fail to be parsed or the query to fail, resulting in inaccurate filtering conditions or failure to execute.

[0120] This application further proposes extracting table structure information from the tag library, including field names, field types, field comments, tag names, and an enumerated set of tag values. Based on the user's question and semantic extension question, the field descriptions and comments in the tag library are semantically matched to generate a structured SQL statement that satisfies the semantic intent. The structured SQL statement includes a WHERE clause to specify the matching conditions between tag fields and tag values. If the structured SQL statement cannot be parsed or the query fails, a fallback strategy rule is used to match the mapping between keywords and tag names, manually constructing the SQL query conditions. Based on the structured SQL statement, a query operation is executed in the tag library to obtain a set of candidate knowledge fragments that match the semantic conditions of the user's question.

[0121] When extracting table structure information, the process involves parsing the metadata of the tag library to obtain field names, types, and comments, and combining this with the enumerated set of tag values ​​to form a complete set of field definitions. When generating structured SQL statements, natural language understanding technology is used to map semantic elements in the user's question to tag library fields, automatically combining WHERE condition expressions. A fallback strategy rule is based on a preset tag name and keyword mapping table; when automatic generation fails, query conditions are manually constructed through keyword matching to ensure the filtering logic is executable.

[0122] Specifically, extracting table structure information provides field constraints for structured queries, avoiding syntax errors caused by mismatched field types or value ranges. When semantic matching generates SQL statements, it dynamically combines query conditions by associating field descriptions with keywords in the user's question, improving filtering accuracy. A fallback strategy is triggered when automatic generation fails, using keyword mapping rules to construct query conditions and ensure the robustness of the filtering process. For example, when a user's question contains "research funding support," semantic matching maps it to the "funding type" field in the tag library, generating an SQL statement containing "WHERE funding type = research funding." If field mapping fails, it matches tag names based on the keyword "research funding" to construct a conditional expression. Through these steps, it ensures that the filtering conditions of the candidate knowledge fragment set align with the user's intent. Figure 1 This will improve the efficiency and accuracy of subsequent vector retrieval.

[0123] As a preferred embodiment, the solution of this application is specifically implemented as follows:

[0124] Extract the table structure information of the tag library, including field names, field types, field comments, tag names, and an enumerated set of tag values. For example, for a policy resource tag library, the table structure may contain fields such as "policy type," "applicable objects," and "release time," each with a corresponding data type, comment description, and possible value range.

[0125] Based on the user's question and the semantic extension question, the semantic matching tag library contains field descriptions and annotations, generating a structured SQL statement that satisfies the semantic intent. The SQL statement includes a WHERE clause to specify the matching conditions between tag fields and tag values. For example, for the user question "What science and technology innovation policies were released in 2023?", the following SQL query might be generated:

[0126] SELECT * FROM policy_table

[0127] WHERE policy_type = Technological Innovation

[0128] AND release_year = 2023;

[0129] If the structured SQL statement cannot be parsed or the query fails, a fallback strategy rule is used to match the keyword and tag name mapping rules, manually constructing the SQL query conditions. For example, if the tag "Technological Innovation" cannot be directly matched, fuzzy matching rules may be used to expand the query conditions.

[0130] SELECT * FROM policy_table

[0131] WHERE policy_type LIKE %Technology% OR policy_type LIKE %Innovation%

[0132] AND release_year = 2023;

[0133] The system executes queries on a tag library using structured SQL statements to obtain a set of candidate knowledge fragments that match the semantic conditions of the user's question. These candidate knowledge fragments will then serve as input for subsequent semantic vector matching, further improving retrieval accuracy.

[0134] Through the above technical solution, this application achieves automatic conversion of user questions into structured queries, improving the accuracy of resource retrieval. By combining data filtering with field definitions in the tag library, interference from irrelevant resources is reduced, improving retrieval efficiency. Simultaneously, by setting a fallback strategy, relevant results are still returned even in complex query scenarios. This method effectively solves the limitations of traditional RAG systems in handling resource queries in highly semantically complex domains, providing users with more accurate and relevant resource recommendations.

[0135] In some of the solutions described above in this application, the resource texts in the candidate knowledge fragment set have problems such as format differences, redundant noise, and length redundancy, which leads to inconsistencies in the semantic vector encoding process, affects the accuracy of vector similarity calculation, and thus reduces the relevance of the recall results.

[0136] This application further proposes semantic vector encoding for each resource text in the candidate knowledge fragment set. The process of obtaining candidate vectors includes: normalizing each resource text in the candidate knowledge fragment set, which includes removing HTML tags, standardizing punctuation, normalizing spaces, and pruning the text length. A semantic embedding model consistent with the vector library construction phase is used to encode the candidate knowledge fragments. The semantic embedding model maps each candidate knowledge fragment to a fixed-dimensional semantic vector to obtain candidate vectors. These candidate vectors are obtained using mean pooling or CLStoken pooling strategies during the encoding process.

[0137] The normalization process comprises four layers: an HTML tag removal layer that uses regular expressions to match and remove Hypertext Markup Language (HTML) elements from the document; a punctuation normalization layer that converts full-width characters to half-width characters and standardizes different punctuation expressions; a whitespace normalization layer that merges consecutive spaces and eliminates tabs; and a length pruning layer that sets a maximum character limit for the text and truncates any excess characters. The semantic embedding model uses the same pre-trained model parameters as the vector library construction phase, ensuring that candidate vectors and semantic vectors in the vector library are in the same spatial coordinate system. The pooling strategy is selected based on the model type: CLS token pooling is preferred for Transformer-based models, while mean pooling is used for BiLSTM-based models.

[0138] Specifically, before candidate knowledge fragments enter the encoding stage, the HTML tag removal layer identifies the text... <tag>Format string and perform deletion operation, for example, change " Policy Interpretation The text is converted from "" to "policy interpretation". The punctuation standardization layer converts full-width commas "," to half-width commas "", and unifies tildes "~" to dashes "—". The space normalization layer compresses consecutive spaces into single spaces, eliminating tab-based whitespace. The length pruning layer sets a maximum character count of 512, truncating any excess characters at the end. The normalized text is input into a semantic embedding model. After the model outputs the hidden vector for each token, the pooling method is selected based on the model architecture: if the model contains a CLS token, the vector corresponding to the CLS position is extracted as a candidate vector. If the model is a recurrent neural network, the mean of all token vectors is calculated as the candidate vector. This process ensures that the candidate vectors maintain dimensionality and spatial consistency with the original encodings in the vector library, avoiding semantic shifts caused by preprocessing differences.

[0139] As a preferred embodiment, the solution of this application is specifically implemented as follows:

[0140] Each resource text in the candidate knowledge fragment set is normalized. Normalization includes removing HTML tags, standardizing punctuation, normalizing spaces, and pruning the text. Specifically, regular expressions are first used to match and remove HTML tags, such as... , Then, standardize the Chinese and English punctuation marks to the correct format, such as converting full-width commas "," to half-width commas ,". Next, standardize the spaces in the text, removing extra spaces and adding appropriate spaces after punctuation marks. Finally, trim the text length, keeping only the first 512 characters.

[0141] Furthermore, the candidate knowledge fragments are encoded using a semantic embedding model consistent with the vector library construction phase. This semantic embedding model employs a pre-trained BERT model, taking normalized text as input and outputting a 768-dimensional vector representation.

[0142] Therefore, the semantic embedding model maps each selected knowledge fragment to a fixed-dimensional semantic vector to obtain candidate vectors. These candidate vectors are obtained during the encoding process using either mean pooling or CLS token pooling strategies. Specifically, the mean pooling strategy averages the hidden states of all tokens in the last layer of BERT, while the CLS token strategy directly uses the [CLS] label to represent the corresponding hidden state as the representation of the entire sequence.

[0143] Through the above technical solutions, this application achieves the normalization and vectorization of candidate knowledge fragments. Normalization improves text quality, eliminates format inconsistencies, and lays the foundation for subsequent vectorization. Using a semantic embedding model consistent with the vector library ensures the consistency of the vector space, which is beneficial to the accuracy of subsequent similarity calculations. Employing mean pooling or CLS token strategies to obtain fixed-dimensional vector representations captures the semantic information of the entire text and facilitates subsequent vector operations. This series of processes improves the representation quality of candidate knowledge fragments, providing a reliable foundation for subsequent semantic matching and similarity calculations, thereby enhancing the accuracy and relevance of resource recommendations.

[0144] In some of the solutions described above in this application, when recalling knowledge fragments based on semantic vector similarity, filtering by a single similarity threshold may result in the simultaneous selection of fragments with high similarity but duplicate content, or the existence of multiple fragments with similarity but scattered semantics, affecting the accuracy and information density of the final recommended content.

[0145] This application further proposes encoding user questions based on a semantic embedding model, mapping user questions and semantically extended questions to query vectors. The query vectors and candidate vectors have the same dimension and semantic space. Cosine similarity is calculated between each candidate vector and the query vector to obtain the similarity score for each candidate vector. All similarities are then compared with a similarity threshold to obtain supplementary knowledge content.

[0146] The semantic embedding model employs the same model architecture and parameter configuration as the vector library construction phase, ensuring that query vectors and candidate vectors reside in the same semantic space. Cosine similarity is calculated using the ratio of vector dot product to modulus, with matrix operations used for acceleration. The similarity threshold is dynamically adjusted based on historical data recall accuracy, ranging from 0.7 to 0.9. When multiple resource texts meet the threshold condition, a secondary ranking strategy based on semantic relevance is adopted. This involves calculating the semantic similarity matrix between candidate segments to remove redundant content while preserving diversity.

[0147] Specifically, the user's question is encoded into a 768-dimensional query vector using a semantic embedding model, and each text in the candidate knowledge fragment set is encoded into a 768-dimensional candidate vector using the same model. The cosine similarity between the query vector and each candidate vector is calculated, forming a similarity score list. This score list is compared to a preset similarity threshold of 0.8, filtering out candidate fragments with scores higher than the threshold. When multiple candidate fragments meet the criteria, the semantic similarity between each fragment is further calculated. If the similarity between two fragments exceeds 0.95, they are considered duplicate content, and only the fragment with the highest similarity is retained. Finally, the fragments are sorted in descending order of similarity score, and the top three non-duplicate fragments are selected as supplementary knowledge content. This process improves content quality through a dual filtering mechanism: threshold filtering eliminates low-relevance content, and semantic deduplication and sorting optimize information density, ensuring that the recommendation results simultaneously meet the requirements of accuracy and diversity.

[0148] As a preferred embodiment, the solution of this application is specifically implemented as follows:

[0149] User questions are encoded using a semantic embedding model, mapping both the user question and its semantic extension to query vectors. Query vectors and candidate vectors share the same dimension and semantic space. Cosine similarity is calculated between each candidate vector and the query vector to obtain a similarity score. All similarities are then compared to a similarity threshold to obtain supplementary knowledge.

[0150] Specifically, a pre-trained BERT model is first used as the semantic embedding model. The user's question is input into the BERT model, and the hidden states corresponding to the last layer [CLS] labels are extracted as a 768-dimensional query vector. For each candidate vector, its cosine similarity to the query vector is calculated. A similarity threshold of 0.7 is set, and the resource texts corresponding to candidate vectors with similarity greater than 0.7 are used as supplementary knowledge content. If multiple resource texts meet the condition, semantic relevance analysis is performed, and they are sorted from high to low similarity and high to low relevance. The top three are selected as the final supplementary knowledge content.

[0151] Through the above technical solution, this application achieves accurate knowledge matching based on semantic similarity. This improves the accuracy and relevance of resource recommendations and avoids the introduction of irrelevant or low-quality text. Furthermore, by setting similarity thresholds and semantic relevance ranking, the quality and diversity of recommended content are ensured, enhancing the user experience.

[0152] In some of the solutions described above in this application, when there are multiple resource texts with similarity greater than or equal to the threshold in the candidate knowledge fragment set, directly selecting all fragments that meet the conditions may lead to content redundancy or inconsistent quality, affecting the accuracy of the recommendation results and user satisfaction.

[0153] This application further proposes that when the similarity is greater than or equal to the similarity threshold, the resource text corresponding to that similarity is used as supplementary knowledge content. If multiple resource texts meet the conditions, semantic relevance analysis is performed on the multiple resource texts, and they are sorted from high to low similarity and from high to low relevance, and the top n texts are selected as supplementary knowledge content.

[0154] Semantic relevance analysis is achieved by calculating the semantic distance between candidate segments, employing a sentence-to-sentence relation discrimination method based on a pre-trained model. The number of selected top n segments is dynamically adjusted according to the preset recommendation result length, with n ranging from 3 to 10. Similarity and relevance are weighted using a scoring mechanism, with the similarity weight coefficient set to 0.7 and the relevance weight coefficient set to 0.3. A sliding window strategy is used during the ranking process to perform secondary verification on candidate segments with adjacent scores.

[0155] Specifically, after obtaining candidate segments that meet the similarity threshold, they are first sorted in descending order based on their cosine similarity values ​​to form an initial queue. Then, semantic relationships between each segment and other segments in the queue are modeled, and content redundancy is assessed by calculating the Euclidean distance between sentence vectors, eliminating redundant segments with a redundancy score higher than 0.9. The remaining segments are then reordered according to a weighted sum of their similarity and relevance scores, calculated as: Total Score = Similarity × 0.7 + (1 - Redundancy) × 0.3. Finally, the top 5 segments with the highest total scores are selected as supplementary knowledge content. This number automatically adapts to the length of the user's question, automatically expanding to 8 segments when the question contains more than 3 entities. This dual sorting mechanism ensures that highly matched segments are presented first, while semantically overlapping content is eliminated through relevance filtering, resulting in recommendation results that are both accurate and diverse.

[0156] As a preferred embodiment, the specific implementation of this application is as follows: After calculating the similarity between the candidate vector and the query vector, all resource texts in the candidate knowledge fragment set with a similarity reaching a preset threshold of 0.85 are selected as a preliminary candidate set. For the case where the preliminary candidate set contains three resource texts, the semantic relevance score between each text and the user's question is calculated: the title keywords of the first resource text match the user's question by 92%, the contextual logical coherence score of the second resource text is 0.78, and the entity coverage rate of the third resource text is 85%. After sorting by similarity in descending order, the similarity of the first resource text is 0.91, the second is 0.88, and the third is 0.86. The sorting is further adjusted according to the semantic relevance score, elevating the second resource text to the first position. Finally, the first two resource texts are selected as supplementary knowledge content, and a recommended answer containing both texts is generated.

[0157] Through the above technical solution, this application solves the problem in existing technologies where low-quality text is recalled due to reliance solely on vector similarity. It effectively improves the relevance and accuracy of recommended content through a dual ranking mechanism. Specifically, when multiple resource texts meet the similarity threshold, a secondary ranking is performed using semantic relevance analysis. Content with more complete semantic logic and more comprehensive keyword coverage is prioritized, avoiding semantic bias caused by relying solely on vector space distance. This ensures that the final recommendation result simultaneously meets the requirements of similarity and semantic coherence.

[0158] In some of the solutions described above in this application, directly splicing supplementary knowledge content during the resource recommendation answer generation process may result in redundant answers, inconsistent terminology, or chaotic structure, affecting the accuracy and readability of the recommendation results.

[0159] This application further proposes a resource concatenation process for generating resource-recommended answers by combining all supplementary knowledge content. This includes: sorting the selected supplementary knowledge content according to its similarity to the query vector from highest to lowest; concatenating the sorted content into a context input; inserting separators between each piece of content during the concatenation process; and embedding the original user question and semantic expansion question into the concatenation context. The generated answer output is then post-processed, including removing invalid statements, standardizing terminology, and deduplicating and classifying entries to obtain the resource-recommended answer.

[0160] The sorting process prioritizes highly relevant content based on similarity metrics. Inserting delimiters prevents semantic confusion between different knowledge fragments, maintaining a clear text structure. Embedding original and extended questions ensures contextual coherence and prevents generated content from deviating from user intent. The post-processing stage employs a sentence filtering mechanism to remove duplicate or meaningless text, standardizes professional terminology through a terminology mapping table, and merges similar information based on semantic clustering.

[0161] Specifically, when generating resource recommendation answers, candidate knowledge fragments are first sorted in descending order of similarity to form a priority queue. A vertical bar is used as a separator between adjacent content, and the original user question and its extended questions are embedded at the beginning of the concatenated text as generation prompts. For example, when processing 5 candidate items, the top 3 with the highest similarity are selected and concatenated sequentially, with each item separated by a "|" separator. After generating the initial answer, regular expressions are used to remove sentences containing uncertain words such as "maybe" or "probably," a domain terminology dictionary is used to uniformly replace "LLM" with "large language model," and content involving the same policy clauses is merged. The final output is a recommendation answer with complete paragraph structure, standardized terminology, and no redundant information.

[0162] As a preferred embodiment, the solution of this application is specifically implemented as follows:

[0163] After candidate knowledge fragments are sorted from highest to lowest similarity, the first five fragments are concatenated using horizontal separators from Markdown syntax to form the context input. During concatenation, a metadata identifier of "Resource ID - Similarity Value" is added to the header of each knowledge fragment, and the original user question and three semantically extended questions are inserted at the beginning of the concatenated text. The generated context input is fed into a pre-trained language model for text generation. The output undergoes the following post-processing steps: Regular expression matching is used to remove statements containing "may contain errors" or "for reference only." Terminology in policy documents is standardized to the expressions defined in the "Standardized Format Specifications of the State Council." Only the version with the highest similarity is retained for repeated entries of the same resource.

[0164] Through the above technical solutions, this application effectively solves the problems of content redundancy, loose structure, and inconsistent terminology in resource recommendation answers. By standardizing the sorting and delimiters, highly relevant knowledge fragments are prioritized in the generation process, improving the logical coherence of the answers. Metadata identification and question embedding mechanisms enhance the generative model's ability to capture contextual relevance. Deduplication and terminology standardization operations in the post-processing workflow reduce the probability of interference from invalid information, ensuring that the output content conforms to the expression norms of the professional domain.

[0165] In some of the solutions described above in this application, the existing technology cannot effectively utilize the structured tag information in the resource data for accurate condition filtering, resulting in redundant returned content or matching deviations. At the same time, user questions are difficult to automatically convert into operable structured retrieval logic, and simply relying on vector space recall is prone to introducing irrelevant or low-quality text.

[0166] The above embodiments overcome the recall redundancy and matching bias problems caused by relying solely on semantic similarity retrieval in existing RAG systems by integrating structured tag filtering and semantic vector fine-tuning mechanisms. Dual modeling is applied to resource text: on the one hand, a semantic vector library is constructed to capture deep semantic information of resource content; on the other hand, a tag library is constructed to extract conditional attribute information of policies or resources, supporting precise matching based on field rules. Language models are used to rewrite and semantically expand user questions in various ways, improving recall coverage. Simultaneously, intent recognition and resource type classification ensure the accuracy of recommendation logic. User intent is automatically converted into SQL form to achieve conditional filtering of tag fields and obtain highly relevant candidate knowledge fragments. Semantic encoding and similarity calculation are performed on candidate content, thresholds are set, and fine-tuning is conducted. Contextual concatenation generates well-structured and semantically targeted recommended answers. This improves the accuracy of resource recommendations, context matching, and the controllability of generated content.

[0167] In another preferred embodiment based on the above embodiments, see [reference] Figure 2 As shown, this embodiment provides a resource recommendation system based on hybrid retrieval RAG, used to apply the above-described resource recommendation method based on hybrid retrieval RAG, including:

[0168] The acquisition unit is configured to acquire resource text data and construct a vector library and a tag library based on the resource text data.

[0169] The processing unit is configured to collect user questions, expand the user questions based on the language model, and obtain several semantically extended questions.

[0170] The identification unit is configured to perform intent recognition on all semantically extended questions, determine whether the user's question is a resource recommendation question, and if so, determine the target category type to which the resource belongs.

[0171] The filtering unit is configured to transform user questions and semantic extension questions into structured query statements, filter the data according to the field definitions in the tag library, and obtain a set of candidate knowledge fragments that meet the tag conditions.

[0172] The matching unit is configured to perform semantic vector encoding on each resource text in the candidate knowledge fragment set to obtain candidate vectors. Based on the semantic encoding, user questions and semantic extension questions are mapped to query vectors. The similarity between the query vector and each candidate vector is calculated, and fragments with similarity greater than a similarity threshold are selected as knowledge supplementary content.

[0173] The output unit is configured to combine all supplementary knowledge content to generate resource-recommended answers.

[0174] Specifically, the acquisition unit preprocesses resource text data, segments and encodes it according to semantic logic to generate a vector library, and extracts keywords and entities to construct a tag library. The tag library stores the binding relationship between tag condition trees and resource fragments in a structured table. The processing unit rewrites user questions in various ways based on a language model to generate semantically expanded questions, guiding the generation process with prompt word templates and conversation history information. The identification unit classifies multiple semantically expanded questions using a predefined intent category list, determines the intent type through a majority voting mechanism, and identifies the target category if it belongs to the resource recommendation category. The filtering unit extracts the tag library table structure information, converts user questions into structured SQL statements for tag field matching queries, and uses rule matching to construct query conditions if the query fails. The matching unit normalizes candidate knowledge fragments, encodes them using the same semantic embedding model as the vector library to generate candidate vectors, calculates the cosine similarity between the user question query vector and the candidate vectors, and selects supplementary knowledge content by filtering and sorting according to a threshold. The output unit sorts and concatenates the supplementary knowledge content by similarity, inserts delimiters and embeds it into the user question, removes invalid statements through post-processing, and unifies terminology style to generate the final answer.

[0175] Specifically, the system implements a hybrid retrieval mechanism by constructing a structured tag library and a semantic vector library. The processing unit generates diverse semantically extended questions to improve the robustness of intent recognition, while the recognition unit reduces the probability of false positives through majority voting. The filtering unit converts natural language questions into structured query statements and uses field definitions from the tag library for precise conditional filtering, reducing redundant data. The matching unit combines semantic vector similarity calculation with threshold filtering to ensure the relevance of the recalled content. The output unit sorts, concatenates, and post-processes multi-source content to improve the coherence and accuracy of the responses. These units work collaboratively, combining structured tag retrieval with semantic vector matching to address the problems of insufficient conditional filtering, intent recognition bias, and low-quality recall in existing technologies, achieving high-precision resource recommendation.

[0176] As a preferred embodiment, the solution of this application is implemented as follows: The resource recommendation system comprises six collaboratively operating modules. The acquisition unit obtains policy documents from the government data platform through a distributed crawler program, and uses a text cleaning tool to segment the raw data, dividing the cleaned text into semantically coherent paragraphs. The paragraph segmentation adopts a three-level division strategy based on heading level. The processing unit calls a pre-trained language model interface to transform the user's input "How to apply for high-tech enterprise subsidies" into five extended questions with different sentence structures. The expansion process incorporates the user's previous inquiries about R&D investment in the current session. The identification unit uses an ensemble classification model to perform intent analysis on the extended questions. When three extended questions are determined to be qualification certification inquiries, the resource recommendation process is triggered. The filtering unit converts the user's question into SQL query conditions containing "Policy Type = Technology Support" and "Applicable Object = Enterprise," and filters out knowledge fragments containing tax incentive clauses and application procedures from the tag library. The matching unit uses a BERT-base model to vectorize the candidate text, calculates the cosine similarity with the user's question vector, and selects text fragments with a similarity exceeding 0.82. The output unit sorts the selected policy clauses according to their level of effectiveness, inserts page breaks between clauses during the splicing process, and finally generates a recommended response that includes the policy basis, application conditions, and processing channels.

[0177] Through the above technical solutions, this application effectively addresses the shortcomings of traditional RAG systems in terms of structured data utilization and intent recognition. By constructing a dual retrieval mechanism of tag and vector libraries, a hybrid recall based on field condition filtering and semantic similarity matching is achieved, reducing the false recall rate of irrelevant policy clauses. A combined strategy of multi-question expansion and integrated intent classification improves the accuracy of resource recommendation question identification from 78% in single-question analysis to 92% in multi-question voting mechanisms. The automatic generation mechanism of structured query statements enables the system to accurately match key fields such as the applicable objects and validity period of policy documents, overcoming the technical bottleneck of traditional semantic retrieval's inability to handle precise condition filtering. The dual filtering and ranking mechanism of knowledge fragments effectively avoids the inclusion of low-quality text, ensuring that the final recommended content is both compliant with policy norms and semantically relevant.

[0178] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program goods. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program goods embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0179] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program goods according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0180] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0181] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0182] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the protection scope of the claims of the present invention. < / tag>

Claims

1. A resource recommendation method based on hybrid retrieval RAG, characterized in that, include: Collect resource text data, and construct a vector library and a tag library based on the resource text data; Collect user questions, expand the user questions based on a language model, and obtain several semantically extended questions; For all the semantic expansion questions, perform intent recognition to determine whether the user question is a resource recommendation question; if so, determine the target category type to which the resource belongs. The user questions and semantic expansion questions are transformed into structured query statements. The data is then filtered according to the field definitions in the tag library to obtain a set of candidate knowledge fragments that meet the tag conditions. Semantic vector encoding is performed on each resource text in the candidate knowledge fragment set to obtain candidate vectors; based on semantic encoding, the user question and semantic extension question are mapped to query vectors, the similarity between the query vector and each candidate vector is calculated, and fragments with similarity greater than a similarity threshold are selected as knowledge supplementary content; All the aforementioned supplementary knowledge is combined to generate resource-recommended answers.

2. The resource recommendation method based on hybrid retrieval RAG according to claim 1, characterized in that, When collecting resource text data and constructing vector and tag libraries based on the resource text data, the process includes: The resource text data is preprocessed, including text segmentation, format regularization, and noise removal. Constructing the vector library: The preprocessed resource text data is segmented into blocks according to semantic logic. The segmentation includes segmentation by title level and segmentation by parent-child semantic structure. Each text block is vectorized and encoded using a semantic embedding model to obtain a semantic vector. The semantic vectors are mapped to the original resource text data, and the mapping relationship is stored in the database to obtain the vector library; Building a tag library: Extracting keywords and entities related to application conditions from preprocessed resource text data, and constructing tag names and tag values; Resource content is tagged and labeled based on rule templates to form a tag expression; Based on business logic, several tag expressions are combined to build a tag condition tree, and the tag content is bound to the corresponding resource fragment; The bound tag information is organized into a structured table format and stored in a database to obtain the tag library.

3. The resource recommendation method based on hybrid retrieval RAG according to claim 1, characterized in that, When collecting user questions and expanding the user questions based on a language model to obtain several semantically expanded questions, including: Based on the language model, the user's question is rewritten to generate multiple question expressions with different semantic expressions but the same query intent. Before rewriting the question, the language model is guided to generate diverse questions by constructing a prompt word template, which includes instruction information, language style requirements and context memory loading fields. During the question rewriting process, the current session history information is taken into account.

4. The resource recommendation method based on hybrid retrieval RAG according to claim 1, characterized in that, When performing intent recognition on all the aforementioned semantic expansion questions to determine whether the user question is a resource recommendation question, the process includes: Construct a predefined list of intent categories, including resource recommendations and non-resource inquiries. Input all the semantic expansion questions and generate intent classification results. A majority vote is conducted on the identification results of multiple semantic expansion questions. If the result belongs to the resource recommendation intent, the target classification type of the resource is determined based on the semantic keywords; otherwise, the recommendation process is terminated and a rejection is triggered.

5. The resource recommendation method based on hybrid retrieval RAG according to claim 1, characterized in that, When transforming the user questions and semantic expansion questions into structured query statements, and filtering the data according to the field definitions in the tag library to obtain a set of candidate knowledge fragments that meet the tag conditions, the process includes: Extract the table structure information of the tag library, which includes: field name, field type, field comment, tag name and tag value enumeration set; Based on the user question and the semantic extension question, the field descriptions and annotations in the tag library are semantically matched to generate a structured SQL statement that satisfies the semantic intent. The structured SQL statement includes a WHERE statement to specify the matching conditions between the tag field and the tag value. If the structured SQL statement cannot be parsed or the query fails, a fallback strategy rule is used to match the rules for mapping keywords and tag names, and the SQL query conditions are manually constructed. Based on the structured SQL statement, a query operation is performed in the tag library to obtain a set of candidate knowledge fragments that match the semantic conditions of the user's question.

6. The resource recommendation method based on hybrid retrieval RAG according to claim 1, characterized in that, Semantic vector encoding is performed on each resource text in the candidate knowledge fragment set to obtain candidate vectors, including: Each resource text in the candidate knowledge fragment set is normalized, and the normalization process includes removing HTML tags, standardizing punctuation, standardizing spaces, and pruning the length of the text. Candidate knowledge fragments are encoded using a semantic embedding model consistent with the vector library construction phase; The semantic embedding model maps each selected knowledge fragment to a fixed-dimensional semantic vector to obtain the candidate vector. The candidate vector is obtained during the encoding process using mean pooling or CLS token pooling strategies.

7. The resource recommendation method based on hybrid retrieval RAG according to claim 6, characterized in that, Based on semantic encoding, the user question and semantic expansion question are mapped to query vectors. The similarity between the query vector and each candidate vector is calculated. When selecting segments with similarity greater than a similarity threshold as supplementary knowledge content, the process includes: The user question and the semantic extension question are encoded based on the semantic embedding model, and the user question is mapped to a query vector. The query vector and the candidate vector have the same dimension and semantic space. For each candidate vector, a cosine similarity is calculated with the query vector to obtain the similarity of each candidate vector; All the aforementioned similarities are compared with a similarity threshold to obtain the supplementary knowledge content.

8. The resource recommendation method based on hybrid retrieval RAG according to claim 7, characterized in that, When comparing all the aforementioned similarities with a similarity threshold to obtain the supplementary knowledge content, the process includes: When the similarity is greater than or equal to the similarity threshold, the resource text corresponding to the similarity is used as the knowledge supplement content. If multiple resource texts meet the condition, semantic relevance analysis is performed on the multiple resource texts, and they are sorted from high to low similarity and from high to low relevance. The top n texts are selected as the knowledge supplement content.

9. The resource recommendation method based on hybrid retrieval RAG according to claim 1, characterized in that, When combining all the aforementioned supplementary knowledge to generate a resource-recommended answer, the following should be included: The selected knowledge supplements are sorted from high to low according to their similarity to the query vector, and the sorted content is concatenated into the context input. During the concatenation process, a separator is inserted between each content, and the original user question and semantic expansion question are embedded in the concatenation context. The generated answer output is post-processed, including removing invalid statements, standardizing terminology style, deduplicating and classifying, to obtain the recommended answer for the resource.

10. A resource recommendation system based on hybrid retrieval RAG, used to apply the resource recommendation method based on hybrid retrieval RAG as described in any one of claims 1-9, characterized in that, include: The acquisition unit is configured to acquire resource text data and construct a vector library and a tag library based on the resource text data. The processing unit is configured to collect user questions, expand the user questions based on a language model, and obtain several semantically extended questions; The identification unit is configured to perform intent recognition on all the semantic extension questions, determine whether the user question is a resource recommendation question, and if so, determine the target category type to which the resource belongs. The filtering unit is configured to transform the user questions and semantic extension questions into structured query statements, filter the data according to the field definitions in the tag library, and obtain a set of candidate knowledge fragments that meet the tag conditions. The matching unit is configured to perform semantic vector encoding on each resource text in the candidate knowledge fragment set to obtain a candidate vector; based on the semantic encoding, map the user question and semantic extension question into a query vector, calculate the similarity between the query vector and each candidate vector, and select fragments with similarity greater than a similarity threshold as knowledge supplementary content; The output unit is configured to concatenate all the aforementioned supplementary knowledge content to generate a resource-recommended answer.

Images