Large language model-oriented efficient data retrieval method and system

By converting the query text entered by the user into a set of intent vectors, combining it with an index database and multi-dimensional feature evaluation, the problem of insufficient understanding of user intent in existing data retrieval methods is solved, and efficient and accurate data retrieval is achieved, significantly improving the response speed and user experience, especially in high-frequency query scenarios.

CN120705305APending Publication Date: 2025-09-26DADAO CLOUD TECHNOLOGY DEVELOPMENT (DONGGUAN CITY) CO LTD

Patent Information

Application Number
CN202510852788.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-06-24
Publication Date
2025-09-26

AI Technical Summary

Technical Problem

Existing data retrieval methods have difficulty in deeply understanding user intentions when faced with complex user needs, resulting in low relevance of retrieval results and a lack of comprehensive consideration of the multidimensional characteristics of data such as source credibility and timeliness, which is particularly inadequate when processing big data.

Method used

By converting the query text entered by the user into a set of intent vectors, combining it with the index database to accurately locate candidate text fragments, and performing multi-dimensional feature evaluation, a comprehensive scoring matrix is ​​constructed to screen out selected text fragments, and combined with a caching mechanism to improve information processing efficiency.

Benefits of technology

It improves the relevance and accuracy of search results and significantly improves information processing efficiency. In particular, it can quickly provide query results that meet user needs in high-frequency query scenarios, thereby enhancing the overall satisfaction of user experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120705305A_ABST
    Figure CN120705305A_ABST
Patent Text Reader

Abstract

The invention provides an efficient data retrieval method and system oriented to a large language model. The method comprises the following steps: determining an intention vector set matched with a user intention; determining a specified number of candidate text fragments with the highest correlation with the user intention in an index database; obtaining multi-dimensional feature evaluation data of each candidate text fragment, and constructing a multi-dimensional feature numerical matrix of each candidate text fragment according to the multi-dimensional feature evaluation data; according to the multi-dimensional feature numerical value matrix, selecting selected text fragments from the candidate text fragments; sorting the selected text fragments, and sequentially performing scene matching on the selected text fragments and a pre-cached high-frequency query scene according to the sorting, so as to extract cache content matched with the target high-frequency query scene from the cache as a query result when a matching result represents that the current query scene accords with the target high-frequency query scene. The invention provides an efficient data retrieval method, and the quality and precision of data retrieval are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of big data processing, and more specifically, to an efficient data retrieval method and system for large language models. Background Art

[0002] Data retrieval technology, as a key bridge connecting user needs with massive amounts of information, is of irreplaceable importance. It directly affects the efficiency of information acquisition and the quality of decision-making, and is the core support for the development of intelligent applications. However, current data retrieval methods often expose obvious shortcomings when faced with complex user needs. Many solutions rely solely on simple keyword matching, which makes it difficult to deeply understand user intent, resulting in low relevance of retrieval results. At the same time, there is a lack of comprehensive consideration of the multidimensional characteristics of data, such as source credibility and timeliness, and the ranking results are often not accurate enough. This problem is particularly prominent in the field of big data processing. In the context of big data processing, with the rapid growth of data volume, how to effectively retrieve data has become an urgent problem to be solved. Traditional retrieval methods seem to be unable to cope with big data processing.

[0003] Therefore, the present application provides an efficient data retrieval method and system for large language models to solve one of the above technical problems. Summary of the Invention

[0004] The purpose of this application is to provide an efficient data retrieval method and system for large language models, which can solve at least one of the technical problems mentioned above. The specific solution is as follows: According to the specific implementation of the present application, in a first aspect, the present application provides an efficient data retrieval method for a large language model, comprising: Based on the query text input by the user, a set of intention vectors that match the user intention is determined; based on the intention vector set, a specified number of candidate text segments with the highest correlation with the user intention are determined in the index database; multidimensional feature evaluation data of each candidate text segment is obtained, and a multidimensional feature value matrix of each candidate text segment is constructed based on the multidimensional feature evaluation data; based on the multidimensional feature value matrix, a comprehensive score of each candidate text segment is calculated respectively, and selected text segments are screened out from the candidate text segments based on the comprehensive score; the selected text segments are sorted, and the selected text segments are matched with pre-cached high-frequency query scenarios in turn according to the sorting, so that when the matching result indicates that the current query scenario meets the target high-frequency query scenario, the cache content matching the target high-frequency query scenario is extracted from the cache as the query result.

[0005] In one embodiment, determining the intention vector set that matches the user intention based on the user input text includes: extracting at least one core word from the query text through a pre-established semantic dictionary database, and semantically annotating the core word to obtain a semantic tag set; mapping the semantic tag set to a high-dimensional space based on the semantic tag set to obtain a first semantic vector set; for a first semantic vector whose dimension value is lower than a preset dimension threshold, supplementing the first semantic vector with a dimension to obtain a second semantic vector set; comparing the semantic association between each vector in the second semantic vector set, obtaining at least one vector combination with a correlation higher than a preset correlation threshold, and constructing a semantic vector subset that matches the user intention with each vector combination; for the semantic vector subset, grouping the vectors in the semantic vector subset, and extracting vectors related to the complexity of the query text as intention vectors from the grouping results to obtain the intention vector set.

[0006] In one embodiment, the method determines a specified number of candidate text segments with the highest relevance to the user intention in an index database based on the intention vector set, including: obtaining an initial segment set related to the intention vector set in the index database, comparing the proximity between each initial segment and the intention vector using a vector distance calculation method, and constructing a text segment list with the initial segments whose proximity meets preset requirements; comparing the semantic relevance between each text segment and the intention vector, and constructing a candidate text segment set with each text segment whose relevance is higher than a preset relevance threshold; semantically decomposing the candidate text segments to obtain core semantic units, screening target core semantic units that are consistent with the contextual logic of the user intention, and constructing a verified candidate text segment list with the candidate text segments corresponding to each target core semantic unit; arranging the candidate text segments in the verified candidate text segment list, and intercepting a specified number of candidate text segments according to the order; wherein the semantic relevance and contextual logic of the candidate text segments are respectively positively correlated with the order priority of the candidate text segments.

[0007] In one embodiment, the multidimensional feature evaluation data includes a source identifier and a release time; the multidimensional feature numerical matrix of each candidate text fragment is constructed based on the multidimensional feature evaluation data, including: comparing the source identifiers of each candidate text fragment respectively, calculating the timeliness weight based on the difference between the release time and the current time, and obtaining preliminary scoring data for credibility and timeliness of each candidate text fragment respectively; standardizing the preliminary scoring data, and using different types of preliminary scoring data as different matrix elements to construct a multidimensional feature numerical matrix.

[0008] In one embodiment, the comprehensive score of each candidate text fragment is calculated respectively according to the multidimensional feature value matrix, and selected text fragments are screened out from the candidate text fragments according to the comprehensive score, including: according to a pre-established feature weight table, each matrix element in the multidimensional feature value matrix is ​​weighted to obtain a weighted feature value set of the multidimensional feature value matrix; each weighted feature value in the weighted feature value set is summed to obtain a comprehensive score of the candidate text fragment; for each candidate text fragment, the candidate text fragment whose comprehensive score is higher than the preset score threshold is screened out, and the candidate text fragment whose fragment logic does not meet the evaluation criteria is screened out to obtain the selected text fragment.

[0009] In one embodiment, the method further includes: obtaining click-through rate data and dwell time data from user interaction records, classifying and summarizing the click-through rate data and the dwell time data to obtain feedback index values ​​corresponding to each feature weight in the feature weight table; for feature weights whose feedback index values ​​are lower than preset feedback standards, determining a weight parameter range that the feature weight supports adjustment; mining historical user interaction data to obtain potential influencing factors related to the feedback index values; for the weight parameter range, determining a weight adjustment range that is suitable for the current scenario and conforms to the weight parameter range based on the potential influencing factors; and generating an updated weight parameter table by applying the weight adjustment range to the weight parameter.

[0010] In one embodiment, the method further includes: extracting text features of each of the selected text fragments and converting them into a first feature vector to generate a second feature vector corresponding to the user intention; respectively determining the cosine similarity between each of the first feature vectors and the second feature vector, and screening out the selected text fragments whose cosine similarity is lower than a preset similarity threshold.

[0011] In one implementation, the sorting of the selected text segments includes sorting the selected text segments according to the cosine similarity.

[0012] According to a specific embodiment of the present application, in a second aspect, the present application provides an efficient data retrieval system for a large language model, comprising: A determination unit is used to determine a set of intention vectors that match the user's intention based on the query text input by the user; and is used to determine a specified number of candidate text segments with the highest correlation with the user's intention in the index database based on the intention vector set; a creation unit is used to obtain multidimensional feature evaluation data of each candidate text segment, and construct a multidimensional feature value matrix of each candidate text segment based on the multidimensional feature evaluation data; a processing unit is used to calculate the comprehensive score of each candidate text segment based on the multidimensional feature value matrix, and screen out selected text segments from the candidate text segments based on the comprehensive score; and is used to sort the selected text segments, and match the selected text segments with pre-cached high-frequency query scenarios in sequence according to the sorting, so that when the matching result indicates that the current query scenario meets the target high-frequency query scenario, the cache content matching the target high-frequency query scenario is extracted from the cache as the query result.

[0013] Compared with the prior art, the above-mentioned scheme of the embodiment of the present application has at least the following beneficial effects: the present application provides an efficient data retrieval method for a large language model. By converting the query text input by the user into a set of intent vectors, it can not only deeply understand the actual needs of the user, but also accurately locate the candidate text fragments that are highly relevant to the user's intent in the index database. This method fundamentally improves the relevance and accuracy of the retrieval results and solves the problem of insufficient semantic understanding in the traditional keyword matching retrieval method. Furthermore, by performing multi-dimensional feature evaluation on each candidate text fragment and constructing a numerical matrix, multiple dimensions such as the credibility and timeliness of the data source can be comprehensively considered, making the comprehensive scoring more scientific and reasonable, thereby screening out the selected text fragments that best meet the user's needs. The practice of sorting the selected text fragments and combining them with a caching mechanism significantly improves the efficiency of information processing, especially when dealing with high-frequency query scenarios, it can quickly provide query results that meet the user's needs, greatly shorten the response time, and enhance the overall satisfaction of the user experience. In this way, the entire retrieval process is not only more intelligent and efficient, but also ensures the quality and pertinence of the output content, providing users with truly valuable information. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 A flowchart of an efficient data retrieval method for large language models is shown; Figure 2 A flow chart of a method for determining a set of intent vectors matching a user's intent is shown; Figure 3 A flow chart of a method for determining candidate text segments is shown; Figure 4 A flow chart of a method for constructing a multidimensional feature value matrix is ​​shown; Figure 5 A flow chart of a method for selecting selected text segments from candidate text segments is shown; Figure 6 A flow chart of a method for updating a weight parameter table is shown; Figure 7 A flow chart of a method for further screening selected text snippets is shown; Figure 8 A unit block diagram of an efficient data retrieval system for large language models is shown. DETAILED DESCRIPTION

[0015] To make the objectives, technical solutions, and advantages of this application more clear, this application will be further described in detail below with reference to the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of this application.

[0016] The terms used in the examples of this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms "a," "the," and "the" used in the examples of this application and the appended claims are also intended to include plural forms, and unless the context clearly indicates otherwise, "a plurality" generally includes at least two.

[0017] It should be understood that the term "and / or" as used herein is merely a description of the relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " in this document generally indicates that the associated objects are in an "or" relationship.

[0018] It should be understood that although the terms first, second, third, etc. may be used to describe in the embodiments of the present application, these descriptions should not be limited to these terms. These terms are only used to distinguish the descriptions. For example, without departing from the scope of the embodiments of the present application, the first may also be referred to as the second, and similarly, the second may also be referred to as the first.

[0019] As used herein, the words "if" and "if" may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to the determination" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)," depending on the context.

[0020] It should also be noted that the terms "include," "comprises," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a product or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such product or system. In the absence of further limitations, an element defined by the phrase "comprises a..." does not exclude the presence of other identical elements in the product or system comprising the element.

[0021] It should be noted in particular that any symbols and / or numbers in the specification that are not marked in the accompanying drawings are not drawing marks.

[0022] The optional embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0023] The embodiment provided in this application is an embodiment of an efficient data retrieval method for a large language model.

[0024] The following combination Figure 1 The embodiments of the present application are described in detail.

[0025] Figure 1 A flowchart of an efficient data retrieval method for large language models is shown. Figure 1 As shown, the following steps are included.

[0026] Step S101: Determine a set of intent vectors that match the user's intent based on the query text input by the user.

[0027] Step S102 : determining a specified number of candidate text segments that are most relevant to the user's intention in the index database based on the intention vector set.

[0028] Step S103: obtaining multidimensional feature evaluation data of each candidate text segment, and constructing a multidimensional feature value matrix of each candidate text segment according to the multidimensional feature evaluation data.

[0029] Step S104 , calculating the comprehensive score of each candidate text segment according to the multidimensional feature value matrix, and selecting selected text segments from the candidate text segments according to the comprehensive score.

[0030] Step S105, sort the selected text fragments, and match the selected text fragments with the pre-cached high-frequency query scenarios in sequence according to the sorting, so that when the matching result indicates that the current query scenario meets the target high-frequency query scenario, the cache content matching the target high-frequency query scenario is extracted from the cache as the query result.

[0031] The method provided in this application, by converting the query text input by the user into a set of intent vectors, can not only deeply understand the actual needs of the user, but also accurately locate the candidate text fragments that are highly relevant to the user's intent in the index database. This method fundamentally improves the relevance and accuracy of the retrieval results and solves the problem of insufficient semantic understanding in the traditional keyword matching retrieval method. Furthermore, by performing multi-dimensional feature evaluation on each candidate text fragment and constructing a numerical matrix, multiple dimensions such as the credibility and timeliness of the data source can be comprehensively considered, making the comprehensive scoring more scientific and reasonable, thereby screening out the selected text fragments that best meet the user's needs. The practice of sorting the selected text fragments and combining them with a caching mechanism significantly improves the efficiency of information processing, especially when dealing with high-frequency query scenarios, and can quickly provide query results that meet the user's needs, greatly shortening the response time and enhancing the overall satisfaction of the user experience. In this way, the entire retrieval process is not only more intelligent and efficient, but also ensures the quality and pertinence of the output content, providing users with truly valuable information.

[0032] In some embodiments, high-frequency query scenarios are monitored in real time based on a pre-established query record database. Query frequency data is obtained from historical query logs. If it is detected that the frequency of a certain query scenario exceeds a preset dimension threshold, it is marked as a high-frequency scenario and a corresponding classification identifier is generated. The query content of the high-frequency scenario is pre-stored through a cache mechanism, and a distributed memory database is used to store the query result data. Relevant features are extracted from the classification identifier to determine the mapping relationship between the pre-stored results and the query scenario. When a new query request is received, the system performs feature comparison on the query scenario and obtains the corresponding pre-stored results from the cached mapping relationship. If the comparison result shows that the current query belongs to a high-frequency scenario, the data in the cache is directly extracted to determine the final retrieval output content.

[0033] As a feasible embodiment, in a real-time monitoring scenario based on a query record database, identifying and processing high-frequency queries is a core step. For example, in a query system for an online knowledge base, the system records the frequency data of users searching for specific topics. Assuming that the query frequency for a topic such as "Basic Data Analysis Tutorial" has reached 100 times per hour in the past 24 hours, exceeding the preset threshold of 50 times per hour, the system will mark it as a high-frequency scenario and generate a classification identifier, such as "Data Analysis - Basics."

[0034] For example, with the application of caching mechanisms, a distributed in-memory database can be used to pre-store query results for high-frequency scenarios. For example, after the system detects the high-frequency scenario of "Basic Data Analysis Tutorial," it will pre-store related popular articles, video tutorials, and other content in the in-memory database to form a result set. The system also extracts features from classification identifiers, such as the topic keyword "Data Analysis" and the difficulty tag "Basics," to establish a mapping relationship between the query scenario and the pre-stored results. This approach can significantly reduce the overhead of repeated calculations and database access.

[0035] In one specific embodiment, when a new query is received, the system performs a feature comparison. For example, if a user enters "Getting Started with Data Analysis," the system parses the query, extracts the keywords "data analysis" and "getting started," and matches them against the cached category identifier "Data Analysis - Basics." If a match is found, the system directly retrieves the pre-stored list of articles and videos from the cache without revisiting the main database. This rapid response improves the user experience, especially during peak query volumes.

[0036] For example, when formatting search output, the system organizes the cached data through the content presentation module. For example, if the retrieved results include three articles and two videos, the system adjusts the presentation based on the user's device type, for example, prioritizing video thumbnails on mobile devices and highlighting article titles and summaries on desktops. The processed data is then delivered to the user through a pre-defined output interface, ensuring the final presentation is clear and easy to read.

[0037] For example, a distributed in-memory database can use a sharded storage strategy to disperse the result data of high-frequency scenarios across multiple nodes. For example, data for the "Data Analysis - Basics" scenario might be stored on Node A and Node B. The system dynamically selects the access node based on load. This design effectively distributes query pressure and improves overall system stability.

[0038] In one specific embodiment, a dynamic update mechanism can be introduced to establish and manage mapping relationships. For example, if the system detects that the query popularity of "Basic Data Analysis Tutorial" has decreased, while "Advanced Data Modeling" has gradually become a new high-frequency scenario, the system will promptly adjust the cache content, freeing up space for old data to store new data. This flexibility ensures efficient use of cache resources while also adapting to changing user needs and maintaining the relevance of query results.

[0039] Figure 2 A flow chart of a method for determining a set of intent vectors that match user intent is shown. Figure 2 As shown, the following steps are included.

[0040] Step S201 : extracting at least one core word from the query text through a pre-established semantic dictionary database, and semantically annotating the core word to obtain a semantic tag set.

[0041] Step S202 : Mapping the semantic tag set to a high-dimensional space according to the semantic tag set to obtain a first semantic vector set.

[0042] Step S203 : for the first semantic vector whose dimension value is lower than the preset dimension threshold, the dimension of the first semantic vector is supplemented to obtain a second semantic vector set.

[0043] Step S204 , comparing the semantic associations between the vectors in the second semantic vector set, obtaining at least one vector combination having an association higher than a preset association threshold, and constructing a semantic vector subset matching the user intention using each vector combination.

[0044] Step S205 , for the semantic vector subset, grouping the vectors in the semantic vector subset, and extracting vectors related to the complexity of the query text from the grouping results as intention vectors to obtain an intention vector set.

[0045] In some embodiments, at least one core word is extracted from the query text input by the user through a pre-established semantic dictionary database, and these core words are preliminarily semantically annotated to obtain a set of semantic tags. This process ensures that the extracted core words can accurately reflect the user's focus, laying a solid foundation for subsequent semantic understanding. According to the obtained semantic tag set, it is mapped to a high-dimensional space to generate a first semantic vector representation. If it is found that the dimension value of the first semantic vector representation is lower than the preset dimension threshold, the dimension is supplemented and adjusted to ensure the integrity of the vector representation, thereby improving the accuracy of subsequent semantic relevance calculations.

[0046] As a feasible embodiment, after completing the dimension supplementation, the semantic relevance between each vector in the second semantic vector set is compared. For example, the Euclidean distance method is used to measure the distance between the two vectors "weather" and "environment". On this basis, since the relationship between semantic relevance and Euclidean distance is mappable, the present application can determine the Euclidean distance range that meets the screening requirements according to the semantic relevance range that meets the requirements, and then set the corresponding screening threshold for the semantic relevance range or Euclidean distance range. Among them, one way is to establish a mapping relationship between Euclidean distance and semantic relevance and set a semantic relevance threshold, and then map it to semantic relevance when determining the Euclidean distance between two vectors, and compare the size relationship between it and the semantic relevance threshold, so that the two vectors are grouped together when the semantic relevance is higher than the preset relevance threshold. Another way is to directly set the Euclidean distance threshold according to the Euclidean distance range, and then compare the size relationship between it and the Euclidean distance threshold when determining the Euclidean distance between two vectors, so that the two vectors are grouped together when their Euclidean distance is higher than the Euclidean distance threshold. Finally, the system can construct a subset of semantic vectors that matches the user's intent based on the grouped vectors. This method effectively filters out the semantic information most relevant to the user's intent, avoids the interference of irrelevant information, and improves the relevance of the retrieval results.

[0047] As a specific embodiment, vectors are grouped for a subset of semantic vectors. Assume that the grouping results show that "weather"-related vectors are grouped together and correlate with the complexity of the query text "What's the weather like today?" The system extracts vectors related to the query text complexity as intent vectors, forming the final set of intent vectors. This process enables the system to focus on the core needs of the user's query, significantly improving the accuracy of intent recognition.

[0048] For example, the implementation methods for each of the above technical topics are closely centered around semantic understanding of user queries. The combination of semantic annotation and vector mapping not only converts natural language into structured information that can be understood by machines, but also further optimizes the semantic matching effect through dimensionality adjustment and distance calculation. Clustering processing extracts the parts that best match user intent from a complex vector set. The synergistic effect of these steps not only enhances the system's ability to understand user intent, but also significantly improves response efficiency and accuracy, providing strong support for subsequent conversational interactions. Through this technical process, the system can continuously optimize the user experience in a single scenario, ensuring the coherence and domain relevance of the conversation.

[0049] Figure 3 A flow chart of a method for determining candidate text segments is shown, Figure 3 As shown, the following steps are included.

[0050] Step S301: obtain an initial segment set related to an intention vector set in an index database, compare the proximity between each initial segment and the intention vector using a vector distance calculation method, and construct a text segment list based on the initial segments whose proximity meets preset requirements.

[0051] In step S302 , the semantic relevance between each text segment and the intention vector is compared respectively, and a candidate text segment set is constructed based on each text segment having a relevance higher than a preset relevance threshold.

[0052] Step S303 , decompose the semantic structure of the candidate text segments to obtain core semantic units, screen the target core semantic units that are consistent with the context logic of the user's intention, and construct a verified candidate text segment list with the candidate text segments corresponding to each target core semantic unit.

[0053] Step S304 : For the verified candidate text segment list, the candidate text segments in the list are sorted, and a specified number of candidate text segments are intercepted according to the sorting.

[0054] Among them, the semantic relevance and context logic of the candidate text fragments are positively correlated with the ranking priority of the candidate text fragments.

[0055] In some embodiments, a pre-established index database is used to retrieve a set of initial segments related to a set of intent vectors from a large amount of data. A vector distance calculation is used to compare the proximity between each initial segment and the intent vector. A list of text segments is constructed based on initial segments that meet pre-defined proximity requirements. This method can quickly identify content that is highly relevant to the user's query intent, significantly reducing the amount of data to be processed later.

[0056] In this embodiment, the comparison calculation process first involves converting the user's query text and the initial segments in the index database into vector representations. Specifically, a pre-trained large language model is used to convert the query text into an intent vector, and a corresponding segment vector is generated for each initial segment in the database. Next, cosine similarity is used as a distance metric to calculate the similarity between the intent vector and each segment vector. For example, if the user's query is about "environmental technology," the system converts this query into a vector in a high-dimensional space and calculates the cosine of the angle between each relevant segment vector in the database and the intent vector. The closer the cosine value is to 1, the more consistent the direction of the two vectors, meaning they are semantically similar. Correspondingly, the preset requirement refers to a threshold set during the screening process that determines which initial segments are deemed sufficiently relevant to the user's query intent to be included in the final list of text segments. For example, if the preset requirement is a cosine similarity greater than 0.8, only initial segments with a calculated similarity score exceeding 0.8 will be retained. This threshold is not fixed but is a best practice value determined based on the application scenario, data characteristics, and experimental adjustments. It ensures that the selected text snippets are not only of moderate quantity but also of high quality, and can most effectively meet the user's query needs. In this way, the system can efficiently locate content that highly matches the user's query intent in massive amounts of data.

[0057] As a feasible embodiment, for a list of initially matched text segments, the semantic relevance between each segment and the intent vector is compared. If the calculated similarity exceeds a preset relevance threshold, the segment is identified as a candidate, thereby constructing a set of candidate segments that are highly relevant to the user's query intent. In this way, the text segments most relevant to the user's query intent can be more accurately captured, improving the relevance and accuracy of the search results.

[0058] For example, when semantically decomposing candidate text segments, the system breaks them down into core semantic units, such as "location," "activity," and "evaluation." For example, if a segment reads "The seaside resorts in Sanya, Hainan have beautiful environments," the system extracts "Sanya, Hainan" as the location unit and "seaside resorts" as the activity unit, and determines whether these core semantic units align with the contextual logic of the user's intent. Ultimately, candidate text segments corresponding to core semantic units that align with the contextual logic are included in the verified candidate text segment list. This process helps ensure the logic and integrity of the information.

[0059] As a specific embodiment, for the verified candidate text fragment list, each candidate text fragment in the list is arranged. The sorting basis includes semantic relevance and contextual logical connection, and these two are positively correlated with the sorting priority of the candidate text fragment. For example, there is a fragment about "Sanya Seaside Resort" in the list, which is ranked first because it describes "price" and "facilities" in detail, while another fragment that only mentions "seaside" is ranked last, and finally an ordered list of candidate text fragments is formed. This sorting mechanism ensures that the content that best meets user needs can be presented first, thereby improving the efficiency and accuracy of information acquisition.

[0060] For example, from an overall perspective, the aforementioned steps, from initial screening to final sorting, form a coarse-to-fine semantic processing chain. This chain not only quickly identifies content highly relevant to user intent within massive amounts of data, but also significantly improves the accuracy of information screening through layered processing. Furthermore, it effectively addresses the semantic understanding needs of complex query scenarios, providing users with a more targeted list of text snippets. This approach makes the entire retrieval process more intelligent and efficient, significantly improving the overall user experience.

[0061] In some embodiments, the multi-dimensional feature evaluation data includes a source identifier and a release time.

[0062] Figure 4 A flow chart of a method for constructing a multidimensional feature value matrix is ​​shown in FIG. Figure 4 As shown, the following steps are included.

[0063] In step S401 , the source identifiers of the candidate text segments are compared respectively, and the timeliness weight is calculated based on the difference between the release time and the current time, so as to obtain preliminary scoring data of each candidate text segment for credibility and timeliness.

[0064] Step S402 : Standardize the preliminary scoring data, and use different types of preliminary scoring data as different matrix elements to construct a multi-dimensional feature value matrix.

[0065] In some embodiments, based on a list of candidate text segments, the source identifier and publication time of each segment are obtained and compared. A timeliness weight is calculated based on the difference between the publication time and the current time, thereby obtaining preliminary scores for the credibility and timeliness of each candidate text segment. This method effectively assesses the reliability and freshness of each segment, ensuring that users receive both accurate and timely information.

[0066] As a feasible embodiment, when processing the preliminary scoring data, it is standardized. Specifically, in addition to credibility and timeliness, the preliminary scoring data may also include multiple dimensions such as the completeness of the fragment content and semantic depth. For example, a fragment is scored as 0.7 for content completeness and 0.6 for semantic depth. These different types of scoring data will be integrated into a multidimensional feature value matrix. By placing the data of each dimension under the same dimension, it is convenient for subsequent comparison and weighted processing, thereby more comprehensively evaluating the fit between the fragment and the user's intention.

[0067] In a specific scenario, assuming a candidate text snippet originates from a widely recognized industry information platform, its credibility score might reach 0.9, while another snippet from an unknown source might only receive a score of 0.3. Furthermore, considering the difference between the publication time and the current time, if a snippet was published within 24 hours, it might receive a timeliness weight of 0.8. However, if it was published a month ago, its timeliness weight might drop to 0.2. This comprehensive consideration ensures that the preliminary rating data reflects not only the reliability of the snippet but also its freshness, helping to select the content that best meets user needs.

[0068] Furthermore, by using different types of preliminary rating data as different elements of the matrix, this information can be systematically organized and managed, providing a solid foundation for subsequent comprehensive rating calculations. For example, the scores of a certain fragment in terms of credibility, timeliness, content completeness, and semantic depth are 0.9, 0.8, 0.7, and 0.6, respectively. These scores are integrated into a unified numerical matrix for subsequent analysis and processing. This process not only improves the efficiency of information processing, but also provides users with a more accurate and personalized search result. Through such a technical process, the system can quickly locate and present the text fragments that best meet the user's intentions in massive data, significantly improving the overall satisfaction of the user experience.

[0069] Figure 5 A flow chart of a method for screening selected text segments from candidate text segments is shown, as shown in FIG. Figure 5 As shown, the following steps are included.

[0070] Step S501 : performing weighting processing on each matrix element in the multidimensional feature value matrix according to a pre-established feature weight table to obtain a weighted feature value set of the multidimensional feature value matrix.

[0071] Step S502 : summing up the weighted feature values ​​in the weighted feature value set to obtain a comprehensive score of the candidate text segment.

[0072] Step S503 , for each candidate text segment, screening candidate text segments with comprehensive scores higher than a preset score threshold, and screening out candidate text segments whose segment logic does not meet the evaluation criteria, to obtain selected text segments.

[0073] In some embodiments, a pre-established feature weight table is used to perform weighted processing on each matrix element in the multidimensional feature value matrix to obtain a weighted feature value set for each candidate text segment. Based on this weighted feature value set, a weighted summation method is used to calculate the comprehensive score of each segment. By comparing the comprehensive score with a preset score threshold, candidate text segments with a comprehensive score higher than the threshold are screened out, and those segments that logically do not meet the evaluation criteria are further screened out, and finally a set of selected text segments is determined for subsequent content processing procedures.

[0074] As a feasible embodiment, when processing a list of candidate text fragments, one can start with the acquisition of multidimensional feature data to deeply understand the specific characteristics of each fragment. For example, in the field of industry information screening, a candidate fragment may be a summary of multiple industry articles, and each fragment contains multidimensional features such as source credibility, content depth, and update time. For the feature of source credibility, its value can be quantified by analyzing the background and historical records of the publishing platform. For example, a fragment comes from a well-known industry media and its credibility score is 0.9. Another fragment comes from an unknown source and has a score of only 0.3. The extraction of this multidimensional feature lays the foundation for subsequent weighted processing.

[0075] In one possible implementation, assume that in the feature weighting table, source credibility has a weight of 0.5, content depth has a weight of 0.3, and update time has a weight of 0.2. For a clip with a source credibility score of 0.9, a content depth score of 0.7, and an update time of 0.6, the weighted feature values ​​obtained after weighting are 0.45, 0.21, and 0.12, respectively. These weighted feature values ​​reflect the contribution of each feature to the overall score, ensuring that important features dominate the score.

[0076] As a specific embodiment, when calculating the score, a weighted summation method is used to integrate the various evaluation values ​​to intuitively reflect the overall performance of the segment. For example, if a segment has weighted feature values ​​of 0.45, 0.21, and 0.12, its overall score is the sum of these three values, 0.78. If the preset scoring threshold is 0.7, the segment is marked as retained because its score exceeds the threshold. This method can quickly screen out segments that meet the requirements, significantly improving processing efficiency.

[0077] For example, during the integration process of the snippet collection after preliminary screening, evaluation logic can be incorporated to further optimize the results. Assuming that the initial screening yields 10 snippets, the integration process can be grouped based on thematic relevance between the snippets, eliminating duplicate content. For example, if two snippets report on the same event but have significantly different source credibility scores, the snippet with the higher score will be retained. This integration approach ensures that the final collection of selected text snippets is more representative.

[0078] Furthermore, the final set of featured snippets can be adjusted based on user feedback. For example, if historical data indicates that users prefer snippets with higher content depth, snippets with lower content depth scores can be rescreened during integration, even if their overall scores are above the threshold. This adjustment can ensure that the final results are more aligned with actual needs and improve user satisfaction.

[0079] For example, in subsequent content processing, a collection of featured snippets can serve as core material for generating industry reports or pushing content. For example, if a collection of featured snippets contains five highly rated snippets covering industry trends and the latest developments, subsequent processes can conduct in-depth analysis around these snippets. This approach not only ensures the quality and relevance of the output content, but also provides users with more valuable information, improving overall user experience satisfaction.

[0080] Figure 6 A flow chart of a method for updating the weight parameter table is shown in FIG. Figure 6 As shown, the following steps are included.

[0081] Step S601: Obtain click rate data and dwell time data from user interaction records, classify and summarize the click rate data and dwell time data, and obtain feedback indicator values ​​corresponding to each feature weight in the feature weight table.

[0082] Step S602 : for feature weights whose feedback index values ​​are lower than a preset feedback standard, determine a weight parameter range that supports adjustment of the feature weights.

[0083] Step S603: mining historical user interaction data to obtain potential influencing factors related to the feedback indicator value.

[0084] Step S604 : For the weight parameter range, determine a weight adjustment range that is suitable for the current scenario and within the weight parameter range according to potential influencing factors.

[0085] Step S605 : generating an updated weight parameter table by applying the weight adjustment amplitude to the weight parameters.

[0086] In some embodiments, click-through rate data and dwell time data are obtained from user interaction records, and these data are classified and summarized to obtain the feedback index value corresponding to each feature weight. By comparing the feedback index value with the preset feedback standard, if the feedback index of a certain feature weight is lower than the preset feedback standard, an adjustment signal is generated to determine the weight parameter range that needs to be dynamically adjusted. Then, the historical user interaction data is deeply mined to identify the potential influencing factors of the feedback index, and based on this, the weight adjustment range suitable for the current scenario is determined. Finally, the weight adjustment range is applied to the weight parameter to generate an updated weight parameter table.

[0087] As a feasible embodiment, in the field of analysis and ranking optimization of user interaction data, the collection and processing of click-through rate data and dwell time data can deeply explore potential needs from the perspective of user behavior. For example, suppose a retrieval system processes 100,000 queries per day, and the top three positions of each query result usually receive 80% of the clicks, while the average dwell time of users on these results is 30 seconds. Further, these data are classified and summarized by query type. For example, the click-through rate of the results for the search keyword "technology news" may reach 5%, and the average dwell time is 40 seconds, which is significantly higher than other categories. This shows that users have a high interest in this type of content, and the weights of relevant features should be given priority.

[0088] For example, in a specific embodiment, assume that the preset feedback criteria are a click-through rate of not less than 3% and a dwell time of not less than 25 seconds. If the feedback indicator corresponding to a certain feature weight shows a click-through rate of 2.5% and a dwell time of 20 seconds, the system will generate an adjustment signal. The specific implementation method is that the system automatically identifies the correlation between the feature weight and the feedback indicator, and finds that its impact is mainly concentrated in certain specific query scenarios, such as "product review" type searches. Subsequently, the system determines that the weight parameter range that needs to be adjusted is between 0.1 and 0.3 for precise regulation.

[0089] For example, dynamic adjustments to the weight parameter range can be used to deeply mine historical user interaction data from the past 30 days. Suppose the system analysis finds that in searches for "product reviews," users are more likely to click on result pages containing images and videos, while pure text results have a lower click-through rate. Therefore, the system determines that the appropriate weight adjustment for the current scenario is to increase the weight of multimedia content by 0.2 and decrease the weight of pure text content by 0.1. This adjustment method can better match user preferences and significantly improve the feedback indicator value.

[0090] For example, in the process of generating new optimized configuration data and updating it to the retrieval output module, the adjusted weight parameters can be saved in real time. For example, the multimedia content weight of the "product review" search is increased from the original 0.5 to 0.7, and synchronized to the front-end retrieval module through the system background update mechanism. The adjusted data shows that the user click rate increased from 2.5% to 3.2%, and the dwell time also increased from 20 seconds to 28 seconds, which shows that the optimized configuration effectively improved the user experience. This method not only ensures the accuracy and relevance of the retrieval results, but also improves the overall user satisfaction. Through continuous iterative optimization, the system can significantly improve the quality of retrieval results and user experience without increasing additional resource investment.

[0091] Figure 7 A flow chart of a method for further screening selected text fragments is shown, Figure 7 As shown, the following steps are included.

[0092] Step S701 : extract text features of each selected text segment and convert them into a first feature vector, thereby generating a second feature vector corresponding to the user intention.

[0093] Step S702 : determining the cosine similarity between each first feature vector and the second feature vector respectively, and filtering out selected text segments whose cosine similarity is lower than a preset similarity threshold.

[0094] In some embodiments, the selected text snippets may be ranked according to cosine similarity.

[0095] In some embodiments, for a set of selected text fragments, the text features of each fragment are extracted and converted into a corresponding first feature vector. At the same time, a second feature vector corresponding to the user query is generated according to the user intent, thereby obtaining two sets of vector data for subsequent comparison. The angle distance between each fragment vector and the intention vector is compared one by one, the similarity value of the text fragment is obtained, and the degree of match between the fragment and the user intent is determined. If the similarity value is lower than a preset threshold, the text fragment is removed from the set, and the fragment content that is highly relevant to the user intent is retained. Finally, the remaining fragments are sorted from high to low according to their similarity values ​​to form a final fragment sequence after correlation verification for subsequent processing.

[0096] As a feasible embodiment, when processing the task of screening and sorting text fragments, we can start from text feature extraction and gradually go deep into the whole process of similarity calculation and final sorting, and conduct analysis around the core area of ​​content relevance verification. For example, suppose 10 text fragments are extracted from a news article, and each fragment contains a title and summary content. The text content of each fragment is converted into a high-dimensional first feature vector, such as converting the fragments related to "science and technology news" into a vector representation containing semantic information. The core of this conversion process is to map the vocabulary in the text into the vector space for subsequent numerical comparison and analysis.

[0097] For example, the process of generating a second feature vector based on user intent can be understood as converting the user's search keyword or query into a vector. For example, if the user enters the query "latest technology products," this is converted into a second feature vector with the same dimensions as the segment vector. This step is crucial in capturing the core semantics of the user's intent, ensuring that subsequent comparisons accurately reflect the relevance between the user's needs and the segment content.

[0098] Among them, it should be noted that although the intention vector involved in the above embodiment and the second feature vector involved in this embodiment are both vectors used to represent semantic information, their essential uses and generation backgrounds are different. The intention vector is obtained by constructing a semantic understanding model based on deep learning, and converting the semantic vector of the input text according to the complexity of the user query, in order to accurately capture the user's query needs. The second feature vector is a feature vector corresponding to the user query generated according to the user's intention in the secondary verification stage of the relevance of the retrieval results. It is mainly used to compare with the first feature vector of the candidate text segment to calculate the cosine similarity between the two, so as to evaluate the relevance between the text segment and the user's query intention. Although both are vectors used to express semantic information, the intention vector focuses more on the understanding and representation of the user's query intention, while the second feature vector is mainly used to compare with the candidate text segments to quantify the degree of match between these segments and the user's query intention.

[0099] For example, cosine similarity is calculated to measure the angular distance between the first eigenvector and the second eigenvector. Assuming that the angle between the vector of a certain segment and the user intent vector is small, the calculated similarity value is 0.85, and the preset dimension threshold is 0.6, then the segment is judged to be highly relevant and retained. Conversely, if the similarity of another segment is only 0.3, it will be eliminated. In this way, content segments with a high degree of match with user intent can be effectively screened out. The advantage of this method is that it can intuitively reflect the correlation in a quantitative manner, which is convenient for subsequent processing.

[0100] For example, in the sorting phase of selected text snippets after filtering, assuming the similarity values ​​of the remaining five snippets are 0.85, 0.78, 0.72, 0.65, and 0.62, respectively, they are arranged in descending order to form the final snippet sequence. The significance of this sorting logic is to prioritize the content that best meets user intent, improving information acquisition efficiency. This approach is particularly effective in content recommendations or search result presentation scenarios, allowing users to access the most valuable information more quickly.

[0101] For example, from a business perspective, the screening and sorting of text snippets stems from users' urgent need for accurate information, while the subsequent process is to meet this need through technological means. For example, on news aggregation platforms, users want to quickly access the latest reports related to "technology products." The aforementioned process ensures that the presented snippets are both accurate and organized. This approach brings significant benefits, not only improving the user experience but also increasing the platform's content appeal.

[0102] In some other embodiments, a comprehensive ranking can be performed based on the similarity calculation and the timeliness of the clips. For example, if a clip has a similarity of 0.78 but was published within the last 24 hours, its sorting priority can be appropriately increased. This approach further enriches the screening logic and makes the results more relevant to practical application scenarios. Through multi-dimensional considerations, the resulting clip sequence can better serve user needs, demonstrating the practical value of technology in content processing.

[0103] The present application also provides a system embodiment that is consistent with the above embodiment, which is used to implement the method steps of the above embodiment. The explanation based on the same name meaning is the same as the above embodiment, and has the same technical effect as the above embodiment, which will not be repeated here.

[0104] like Figure 8 As shown, the present application provides an efficient data retrieval system 800 for a large language model, comprising: The determination unit 801 is configured to determine a set of intention vectors matching the user's intention based on the query text input by the user, and to determine a specified number of candidate text segments with the highest relevance to the user's intention in the index database based on the set of intention vectors.

[0105] The creation unit 802 is configured to obtain multidimensional feature evaluation data of each candidate text segment and construct a multidimensional feature value matrix of each candidate text segment based on the multidimensional feature evaluation data.

[0106] Processing unit 803 is configured to calculate a comprehensive score for each candidate text segment based on the multidimensional feature value matrix, and select selected text segments from the candidate text segments based on the comprehensive score. Furthermore, processing unit 803 is configured to sort the selected text segments and, in order of the sorting, match the selected text segments with pre-cached high-frequency query scenarios. When the matching result indicates that the current query scenario matches the target high-frequency query scenario, cached content matching the target high-frequency query scenario is extracted from the cache as the query result.

[0107] Regarding the system in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0108] Although operations are described in a particular order in the drawings, this should not be understood as requiring that the operations be performed in the particular order shown or in serial order, or that all shown operations be performed to obtain the desired results. In certain circumstances, multitasking and parallel processing may be advantageous.

[0109] The methods and systems of the present application can be implemented using standard programming techniques, using rule-based logic or other logic to implement the various method steps. It should also be noted that the terms "system" and "module" as used herein and in the claims are intended to include implementations using one or more lines of software code and / or hardware implementations and / or devices for receiving input.

[0110] Any steps, operations or procedures described herein may be performed or implemented using one or more hardware or software modules, either alone or in combination with other devices. In one embodiment, the software modules are implemented using a computer program product comprising a computer-readable medium containing computer program code, which can be executed by a computer processor to perform any or all of the steps, operations or procedures described.

[0111] The foregoing description of the implementation of the present application has been provided for purposes of illustration and description. The foregoing description is not intended to be exhaustive or to limit the present application to the precise form disclosed, and various variations and modifications are possible in accordance with the above teachings or may result from the practice of the present application. These embodiments have been selected and described in order to illustrate the principles of the present application and its practical application, so as to enable those skilled in the art to utilize the present application in various embodiments and modifications as appropriate for the particular use contemplated.

[0112] Regarding the system in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0113] It is further understood that, unless otherwise specified, “connection” includes a direct connection where there are no other components between the two elements, and also includes an indirect connection where there are other elements between the two elements.

[0114] It should be further understood that although operations are described in a particular order in the drawings in the embodiments of the present application, this should not be construed as requiring that these operations be performed in the particular order shown or in a serial order, or that all of the illustrated operations be performed to obtain the desired results. In certain circumstances, multitasking and parallel processing may be advantageous.

[0115] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. This application is intended to encompass any variations, uses, or adaptations of the present application that follow the general principles of the present application and include common knowledge or customary techniques in the field of the present application that are not disclosed herein. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the scope of claims below.

[0116] It should be understood that the present application is not limited to the precise structures described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the scope of the appended claims.

[0117] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An efficient data retrieval method for large language models, characterized by: include: Determine a set of intent vectors that match the user's intent based on the query text entered by the user; Determining, in an index database based on the set of intention vectors, a specified number of candidate text segments that are most relevant to the user intention; Acquire multidimensional feature evaluation data of each candidate text segment, and construct a multidimensional feature value matrix of each candidate text segment based on the multidimensional feature evaluation data; Calculating the comprehensive score of each candidate text segment according to the multidimensional feature value matrix, and selecting selected text segments from the candidate text segments according to the comprehensive score; The selected text fragments are sorted, and the selected text fragments are matched with the pre-cached high-frequency query scenarios in sequence according to the sorting, so that when the matching result indicates that the current query scenario meets the target high-frequency query scenario, the cache content matching the target high-frequency query scenario is extracted from the cache as the query result.

2. The method according to claim 1, characterized in that The step of determining, based on the user input text, a set of intention vectors that matches the user intention includes: Extracting at least one core word from the query text using a pre-established semantic dictionary database, and semantically annotating the core word to obtain a semantic tag set; According to the semantic tag set, the semantic tag set is mapped to a high-dimensional space to obtain a first semantic vector set; For a first semantic vector whose dimension value is lower than a preset dimension threshold, supplement the dimension of the first semantic vector to obtain a second semantic vector set; Comparing the semantic associations between the vectors in the second semantic vector set, obtaining at least one vector combination having an association higher than a preset association threshold, and constructing a semantic vector subset matching the user's intention using each of the vector combinations; For the semantic vector subset, the vectors in the semantic vector subset are grouped, and vectors related to the complexity of the query text are extracted from the grouping results as intention vectors to obtain the intention vector set.

3. The method according to claim 1, characterized in that Determining, in an index database based on the set of intention vectors, a specified number of candidate text segments that are most relevant to the user intention includes: Obtaining a set of initial segments related to the set of intention vectors in the index database, comparing the proximity between each of the initial segments and the intention vector using a vector distance calculation method, and constructing a text segment list based on the initial segments whose proximity meets preset requirements; Comparing the semantic relevance between each of the text segments and the intention vector respectively, and constructing a candidate text segment set based on the text segments whose relevance is higher than a preset relevance threshold; Decomposing the candidate text segments into semantic structures to obtain core semantic units, screening target core semantic units that are consistent with the contextual logic of the user's intention, and constructing a verified candidate text segment list with the candidate text segments corresponding to each target core semantic unit; For the verified candidate text segment list, sort the candidate text segments in the list and extract a specified number of candidate text segments according to the sorting; The semantic relevance and contextual logic of the candidate text segments are respectively positively correlated with the sorting priority of the candidate text segments.

4. The method according to claim 1, wherein The multi-dimensional feature evaluation data includes source identification and release time; The step of constructing a multidimensional feature value matrix for each candidate text segment based on the multidimensional feature evaluation data includes: Comparing the source identifiers of the candidate text segments, respectively, and calculating the timeliness weight based on the difference between the release time and the current time, thereby obtaining preliminary scoring data for the credibility and timeliness of the candidate text segments; The preliminary scoring data are standardized, and different types of preliminary scoring data are used as different matrix elements to construct a multidimensional feature value matrix.

5. The method according to claim 1, wherein The step of calculating the comprehensive score of each candidate text segment according to the multidimensional feature value matrix, and selecting selected text segments from the candidate text segments according to the comprehensive score, includes: According to a pre-established feature weight table, weighting is performed on each matrix element in the multidimensional feature value matrix to obtain a weighted feature value set of the multidimensional feature value matrix; Summing each weighted feature value in the weighted feature value set to obtain a comprehensive score for the candidate text segment; For each of the candidate text segments, the candidate text segments whose comprehensive scores are higher than a preset score threshold are screened, and the candidate text segments whose segment logic does not meet the evaluation criteria are screened out to obtain the selected text segments.

6. The method according to claim 5, characterized in that The method further comprises: Obtaining click-through rate data and dwell time data from user interaction records, classifying and summarizing the click-through rate data and the dwell time data to obtain a feedback indicator value corresponding to each feature weight in the feature weight table; For feature weights whose feedback indicator values ​​are lower than preset feedback standards, determining a weight parameter range that supports adjustment of the feature weights; Mining historical user interaction data to obtain potential influencing factors related to the feedback indicator value; For the weight parameter range, determine a weight adjustment range that is suitable for the current scenario and within the weight parameter range based on the potential influencing factors; An updated weight parameter table is generated by applying the weight adjustment amplitude to the weight parameter.

7. The method according to claim 1, characterized in that The method further comprises: Extracting text features of each of the selected text segments and converting them into a first feature vector, thereby generating a second feature vector corresponding to the user intention; The cosine similarity between each of the first feature vectors and the second feature vector is determined respectively, and the selected text segments whose cosine similarity is lower than a preset similarity threshold are filtered out.

8. The method according to claim 6, characterized in that The ranking of the selected text snippets includes: The selected text snippets are ranked according to cosine similarity.

9. An efficient data retrieval system for large language models, characterized by: include: A determination unit, configured to determine a set of intent vectors matching the user's intent based on the query text input by the user; for determining, in an index database based on the set of intention vectors, a specified number of candidate text segments that are most relevant to the user intention; A creation unit, configured to obtain multidimensional feature evaluation data of each candidate text segment, and construct a multidimensional feature value matrix of each candidate text segment based on the multidimensional feature evaluation data; A processing unit is used to calculate the comprehensive score of each candidate text fragment according to the multidimensional feature value matrix, and to screen out selected text fragments from the candidate text fragments according to the comprehensive score; and to sort the selected text fragments, and match the selected text fragments with pre-cached high-frequency query scenarios in sequence according to the sorting, so that when the matching result indicates that the current query scenario meets the target high-frequency query scenario, the cache content matching the target high-frequency query scenario is extracted from the cache as the query result.

Citation Information

Patent Citations

  • Training method and device of intention recognition model and intention recognition method and device

    CA3180493A1

  • Data query method and device based on text processing model

    CN118708704A

  • Data retrieval method based on large language model

    CN118820405A

  • Semantic modeling for search

    US10891673B1

Cited By

  • Retrieval enhancement generation method and device, equipment, storage medium and program product

    CN121501967A

  • Unified streaming processing method, system and device for multi-mode AI interactive content, medium and program product

    CN121705057A