A data retrieval method based on large language model

Abstract

The present invention relates to the field of data retrieval technology, and specifically discloses a data retrieval method based on a large language model, comprising the following steps: Step 1: obtaining a query statement input by a user; Step 2: inputting the query statement into the large language model to obtain multiple candidate results related to the query statement; Step 3: calculating the weight value of each candidate result according to a preset weight formula; Step 4: sorting the candidate results according to the weight value; Step 5: selecting the top N candidate results with the highest weight value as the retrieval result; Step 6: returning the retrieval result to the user. The present invention can improve retrieval accuracy, enhance flexibility, consider multi-factor weights, adapt to dynamic changes, enhance user experience, and promote technological development. These effects make the method have broad application prospects and value in various fields.

Description

Technical Field

[0001] The present invention relates to the technical field of data retrieval, and in particular to a data retrieval method based on a large language model. Background Art

[0002] With the rapid development of information technology, the generation and accumulation of various data are getting faster and faster, and the amount of data is showing an explosive growth trend. This includes various types of data such as text, images, audio, video, etc. In this context, how to effectively manage and utilize these massive data has become an urgent problem to be solved.

[0003] Traditional data retrieval methods are mainly based on keyword matching. This method returns relevant search results by matching the keywords entered by the user with the documents in the database. However, this method has some obvious limitations.

[0004] First, keyword matching cannot understand semantics. It simply judges relevance based on the appearance of keywords, but cannot understand the semantic relationship and contextual information between words. This may lead to inaccurate retrieval results and miss some documents that are relevant to the query intent but do not contain specific keywords.

[0005] Secondly, the traditional method lacks flexibility. It usually requires users to enter keywords accurately. For some complex query intents or unfamiliar fields, users may find it difficult to accurately express their needs. In addition, keyword matching is also difficult to handle fuzzy queries or queries with similar semantics.

[0006] The emergence of big language models provides new ideas and solutions for data retrieval. The big language model is a natural language processing model based on deep learning technology, which has powerful language understanding and generation capabilities. By learning from a large amount of text data, the big language model can understand the semantics and grammatical rules of natural language and can generate natural and fluent language text. This enables it to better understand the user's query intention, not just limited to keyword matching, but to capture the semantic information and contextual relationships in the query statement. In addition, the big language model is also flexible and adaptable. It can handle various types of natural language queries, including complex questions, vague descriptions, etc. It can perform intelligent analysis and reasoning based on user input to provide more accurate and comprehensive retrieval results.

[0007] In summary, the emergence of large language models has brought new opportunities and challenges to data retrieval. It can make up for the limitations of traditional data retrieval methods, improve the accuracy and flexibility of retrieval, and provide users with better services and experience. In view of this, a data retrieval method based on large language models is proposed. Summary of the invention

[0008] The purpose of the present invention is to provide a data retrieval method based on a large language model to solve the problems raised in the above background technology.

[0009] To achieve the above object, the present invention provides the following technical solution: a data retrieval method based on a large language model, comprising the following steps:

[0010] Step 1: Get the query statement entered by the user;

[0011] Step 2: Input the query statement into the large language model to obtain multiple candidate results related to the query statement;

[0012] Step 3: Calculate the weight value of each candidate result according to the preset weight formula. The weight formula is: ;in, Indicates The weight of each candidate result; Indicates The similarity between the candidate results and the query statement; Indicates The relevance of the candidate results; Indicates The timeliness of the candidate results; , , is the preset weight coefficient;

[0013] Similarity The calculation formula is: ;in, Indicates the number of words in the query statement; Indicates The words in The weight of each candidate result; Indicates The word and The similarity of the candidate results;

[0014] Relevance The calculation formula is: ;in, Indicates the number of documents in the candidate results; Indicates The document in The weight of each candidate result; Indicates The relevance of the document to the query statement;

[0015] Timeliness The calculation formula is: ;in, Indicates the current time; Indicates The release time of the candidate results; is the preset time attenuation coefficient;

[0016] Step 4: Sort the candidate results according to the weight value;

[0017] Step 5: Select the top N candidate results with the highest weight values ​​as the search results;

[0018] Step 6: Return the search results to the user.

[0019] Preferably, in step 2, a pre-trained language model based on the Transformer architecture is used as the large language model.

[0020] Preferably, in step three, the weight value of each candidate result is calculated through data preprocessing and embedding representation, query intent analysis, context-related document screening, refined retrieval ranking, and dynamic feedback and query optimization.

[0021] Preferably, data preprocessing and embedding representation are used to preprocess query statements and candidate results, and the formula is: ;in, Indicates The embedding vector of the text; Indicates using the BERT model for Text The result of encoding; Represents the parameters of the BERT model, which are obtained during the model training process and are used to convert text into high-dimensional vectors rich in semantic information; Indicates TF-IDF weighted vector of texts; Represents the adjustment coefficient, which is used to control the contribution of the TF-IDF vector in the final embedding vector.

[0022] Preferably, query intent analysis is used to analyze the intent of the query statement and determine the user's needs and concerns. The formula is: ;in, Embedding vector representing the query statement; Represents a predefined set of intent categories; Representing a collection Middle intention category The embedding vector of Represents the dot product of the query sentence embedding vector and the intent category embedding vector; Indicates in the collection Find the function that maximizes the value .

[0023] Preferably, context-related document screening is used to screen out documents related to the context according to the query intent, and the formula is: ;in, Represents the filtered set of related documents; Represents the first documents; Intent vector representing the query Indicates The intent vector of each document; is a similarity function used to calculate the similarity between two intent vectors. Is a similarity threshold used to determine whether a document is relevant to the query.

[0024] Preferably, refined search sorting is used to sort the filtered documents more finely, and the formula is: ;in, Representation Document Rank score; Indicates documents; From the document The set of all key sentences or paragraphs extracted from Representing a collection An element in , i.e., an index of a key sentence or paragraph; Indicates a key sentence or paragraph The weight of represents the query vector; Indicates a key sentence or paragraph The embedding vector of : represents the query vector With key sentence or paragraph vector The cosine similarity between ; Represents the query vector calculated based on the BM25 algorithm With key sentences or paragraphs The correlation score between Represents the query vector With key sentences or paragraphs The semantic similarity score between them.

[0025] Preferably, dynamic feedback and query optimization are used to continuously optimize query results based on user feedback and query history, and the formula is: ;in, represents the updated query vector; represents the query vector; represents the adjustment coefficient; Represents the first The difference between the documents; Represents the first The weight of a document; Represents the size of the document set, that is, the first documents.

[0026] The present invention proposes a data retrieval method based on a large language model, which has the following beneficial effects:

[0027] 1. Improve retrieval accuracy: The data retrieval method based on the large language model can better understand the user's query intention. Through in-depth understanding and analysis of natural language, it can accurately capture the user's needs and provide more accurate retrieval results. Users can find the information they really need faster, improving work efficiency and satisfaction.

[0028] 2. Enhanced flexibility: It can process various types of query statements, including natural language queries; this allows users to express their needs in a more natural and intuitive way without having to follow a specific syntax or format; the system can understand and parse the user's intent, providing more flexible retrieval services suitable for various application scenarios.

[0029] 3. Considering multiple factors: Through the preset weight formula, the similarity, relevance and timeliness of the candidate results are comprehensively considered; this makes the search results more comprehensive and reasonable, considering not only the similarity with the query, but also the relevance and timeliness of the results; it can ensure that users obtain the most relevant and latest information, improving the quality and value of the search.

[0030] 4. Adapt to dynamic changes: It can dynamically adjust and optimize according to user feedback and data updates; user feedback can help the system continuously improve and optimize the retrieval algorithm to provide better services; at the same time, data updates can also be reflected in the retrieval results in a timely manner to ensure that users obtain the latest information; this adaptability can improve the stability and adaptability of the retrieval effect and meet the ever-changing needs of users.

[0031] 5. Improve user experience: Provide users with faster, more accurate and personalized search services; fast search speed can save users' time, accurate results can meet users' needs, and personalized services can provide customized search results based on users' preferences and historical behaviors; all of these can improve user experience and satisfaction, and increase users' trust in the system and willingness to use it.

[0032] In summary, the data retrieval method based on the large language model has many beneficial effects. It can improve retrieval accuracy, enhance flexibility, consider multi-factor weights, adapt to dynamic changes, enhance user experience, and promote technological development. These effects make this method have broad application prospects and value in various fields. DETAILED DESCRIPTION

[0033] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0034] Embodiment, the present invention provides a technical solution: a data retrieval method based on a large language model, comprising the following steps:

[0035] Step 1: Get the query statement entered by the user;

[0036] The user enters a query statement through the system interface or API interface, which is a keyword or phrase that the user wants to retrieve information;

[0037] Step 2: Input the query statement into the large language model to obtain multiple candidate results related to the query statement;

[0038] The query statement entered by the user is input into the pre-trained large language model; the large language model uses its powerful natural language processing capabilities and extensive knowledge base to generate multiple candidate results related to the query statement;

[0039] Step 3: Calculation of candidate result weights:

[0040] Calculate a comprehensive weight value for each candidate result , which is determined by the similarity , Correlation and timeliness The three factors are weighted to arrive at;

[0041] Similarity : Measures the similarity between candidate results and each word in the query, calculated by the weight and similarity of the words in the candidate results;

[0042] Relevance :Evaluate the overall relevance of the documents in the candidate results to the query statement, based on the document weight and the relevance score between the document and the query statement;

[0043] Timeliness : Consider the impact of the release time of the candidate results on the weight, calculated through the time decay function, to ensure that the latest information gets a higher weight;

[0044] According to the preset weight formula, the weight value of each candidate result is calculated. The weight formula is: ;in, Indicates The weight of each candidate result; Indicates The similarity between the candidate results and the query statement; Indicates The relevance of the candidate results; Indicates The timeliness of the candidate results; , , is the preset weight coefficient;

[0045] This formula is used to calculate the overall weight of each candidate result, combining the three dimensions of similarity, relevance, and timeliness, and balancing the contributions of these dimensions through a preset weight coefficient; the weight coefficient can be adjusted according to actual needs to optimize the search results;

[0046] Similarity The calculation formula is: ;in, Indicates the number of words in the query statement; Indicates The words in The weight of each candidate result; Indicates The word and The similarity of the candidate results;

[0047] Relevance The calculation formula is: ;in, Indicates the number of documents in the candidate results; Indicates The document in The weight of each candidate result; Indicates The relevance of the document to the query statement;

[0048] Timeliness The calculation formula is: ;in, Indicates the current time; Indicates The release time of the candidate results; is the preset time decay coefficient, which is used to control the decay rate of the time factor; this formula is based on the sigmoid function of logistic regression to ensure that when the candidate result is very new (i.e. Very large), is close to 1; and when the candidate result is very old, Close to 0;

[0049] Step 4: Sort the candidate results according to the weight value;

[0050] Step 5: Select the top N candidate results with the highest weight values ​​as the search results;

[0051] The candidate results are sorted according to the weight values ​​calculated by the above weight formula; then, the first N candidate results with the highest weight values ​​are selected as the final search results, where N is a preset value, indicating the number of search results the user wants to see; according to the calculated weight values , sort the candidate results in descending order, and the results with higher weight values ​​rank higher;

[0052] Step 6: Return the search results to the user;

[0053] The top N search results are returned to the user in a user-friendly manner (such as lists, charts, etc.), and the user can further browse or perform other operations based on these results.

[0054] In step 2, a pre-trained language model based on the Transformer architecture is used as the large language model.

[0055] In step three, the weight value of each candidate result is calculated through data preprocessing and embedding representation, query intent analysis, context-related document screening, refined retrieval ranking, and dynamic feedback and query optimization.

[0056] Data preprocessing and embedding representation are used to preprocess query statements and candidate results. The formula is: ;in, Indicates The embedding vector of the text; Indicates using the BERT model for Text The result of encoding; Represents the parameters of the BERT model, which are obtained during the model training process and are used to convert text into high-dimensional vectors rich in semantic information;

[0057] Indicates TF-IDF weighted vector of a text; TF-IDF is a statistical method used to assess the importance of a word to a document set or one of the documents in a corpus; it is calculated by multiplying the frequency (TF) of the word with the inverse document frequency (IDF); in this formula, it is used to enhance the representation of text, especially those words that may not be so significant in traditional methods; Represents the adjustment coefficient, which is used to control the contribution of the TF-IDF vector in the final embedding vector; by adjusting The value of can balance the impact of BERT model output and TF-IDF vector to achieve better text representation effect.

[0058] Query intent analysis is used to analyze the intent of the query statement and determine the user's needs and concerns. The formula is: ;in, Embedding vector representing the query statement; Represents a predefined set of intent categories; Representing a collection Middle intention category The embedding vector of Represents the dot product of the query sentence embedding vector and the intent category embedding vector; Indicates in the collection Find the function that maximizes the value .

[0059] Detailed explanation of the formula:

[0060] Calculate the dot product: For a set Each intent category in , calculate its embedding vector Embedded vector with query The dot product of ;

[0061] Find the maximum dot product: Find the maximum value among all the calculated dot product values;

[0062] Determine the intent category: the intent category corresponding to the maximum value This is the intent of the query.

[0063] Context-related document filtering is used to filter out documents related to the context based on the query intent. The formula is: ;in, Represents the filtered set of related documents; Represents the first documents; Intent vector representing the query Indicates The intent vector of each document; is a similarity function used to calculate the similarity between two intent vectors. Is a similarity threshold used to determine whether a document is relevant to the query.

[0064] Detailed explanation of the formula: It means that according to the query intent vector and the intent vector for each document The similarity between them is used to filter out document sets that are similar to the query intent. ;

[0065] First, for each document in the document collection , both calculate their intention vectors and query intent vector The similarity between

[0066] Then, the calculated similarity is compared with the preset similarity threshold Make comparisons;

[0067] If the similarity is greater than or equal to , the document is considered relevant to the query intent and is added to the collection middle;

[0068] final, All documents related to the query intent will be included to form a filtered document set.

[0069] Refined search sorting is used to sort the filtered documents more finely. The formula is: ;in, Representation Document Rank score; Indicates documents; From the document The set of all key sentences or paragraphs extracted from Representing a collection An element in , i.e., an index of a key sentence or paragraph; Indicates a key sentence or paragraph The weight of represents the query vector; Indicates a key sentence or paragraph The embedding vector of Represents the query vector With key sentence or paragraph vector The cosine similarity between ; Represents the query vector calculated based on the BM25 algorithm With key sentences or paragraphs The correlation score between Represents the query vector With key sentences or paragraphs The semantic similarity score between them.

[0070] Indicates the document All key sentences or paragraphs in Each element in the collection ) to traverse and accumulate the calculation results corresponding to each element; each key sentence or paragraph l has a weight , this weight reflects the importance of the key sentence or paragraph in the document; the specific calculation method of the weight is based on many factors, such as the position of the key sentence or paragraph in the document, its length, the frequency of containing specific words, etc.; Represents the query vector With key sentence or paragraph vector The cosine similarity is a measure of the degree of similarity between two vectors in a certain direction. The range of the cosine similarity is [−1, 1]. Here, the closer the value is to 1, the more similar it is, that is, the more relevant the key sentence or paragraph is to the query. It takes into account multiple factors such as the frequency of the query words in the document, the length of the document, and the frequency of the query words in the document collection; the higher the BM25 score, the stronger the relevance of the query to the key sentence or paragraph; Semantic similarity scores are able to capture deeper meaning connections between queries and documents than just word-based matching.

[0071] Dynamic feedback and query optimization are used to continuously optimize query results based on user feedback and query history. The formula is: ;in, represents the updated query vector; represents the query vector; represents the adjustment coefficient; Represents the first The difference between the documents; Represents the first The weight of a document; Represents the size of the document set, that is, the first documents.

[0072] Updated query vector This is the query vector adjusted by a dynamic feedback mechanism. It aims to more accurately reflect the user's query intent by taking into account the difference between the query and the retrieved relevant documents; adjustment coefficient (learning rate) This is a hyperparameter that controls the magnitude of the query vector update; the difference function This function computes the raw query vector After sorting of the documents Document vector The difference between the two; the calculation method of the difference can be selected according to the specific application scenario, such as using the negative of the cosine similarity, Euclidean distance, etc.; the difference reflects the degree of mismatch between the query and the document, and is the key basis for adjusting the query vector; document weight The weight of each document indicates the importance of the document in updating the query vector; the weight can be determined based on the document's relevance score, ranking position, or other factors; This expression calculates the sum of the products of the differences and weights of all considered documents; it combines the adjustments suggested by all relevant documents to the query vector and weights them according to the importance of the documents; the result of the weighted sum is a vector indicating the direction and magnitude of the adjustment required for the query vector; finally Multiply the original query vector by the result of the weighted sum (adjusted by the coefficient The result vector is then added to the original query vector to obtain an updated query vector. This step implements dynamic adjustment of the query vector to make it closer to the user's actual query intent.

[0073] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A data retrieval method based on a large language model, characterized in that: The following steps are involved: Step 1: Get the query statement entered by the user; Step 2: Input the query statement into the large language model to obtain multiple candidate results related to the query statement; Step 3: Calculate the weight value of each candidate result according to the preset weight formula. The weight formula is: ;in, Indicates The weight of each candidate result; Indicates The similarity between the candidate results and the query statement; Indicates The relevance of the candidate results; Indicates The timeliness of the candidate results; , , is the preset weight coefficient; Similarity The calculation formula is: ;in, Indicates the number of words in the query statement; Indicates The words in The weight of each candidate result; Indicates The word and The similarity of the candidate results; Relevance The calculation formula is: ;in, Indicates the number of documents in the candidate results; Indicates The document in The weight of each candidate result; Indicates The relevance of the document to the query statement; Timeliness The calculation formula is: ;in, Indicates the current time; Indicates The release time of the candidate results; is the preset time attenuation coefficient; Step 4: Sort the candidate results according to the weight value; Step 5: Select the top N candidate results with the highest weight values ​​as the search results; Step 6: Return the search results to the user; In step three, the weight value of each candidate result is calculated through data preprocessing and embedding representation, query intent analysis, context-related document screening, refined retrieval ranking, dynamic feedback and query optimization; Data preprocessing and embedding representation are used to preprocess query statements and candidate results. The formula is: ;in, Indicates The embedding vector of the text; Indicates using the BERT model for Text The result of encoding; Represents the parameters of the BERT model, which are obtained during the model training process and are used to convert text into high-dimensional vectors rich in semantic information; Indicates TF-IDF weighted vector of texts; Represents the adjustment coefficient, which is used to control the contribution of the TF-IDF vector in the final embedding vector; Query intent analysis is used to analyze the intent of the query statement and determine the user's needs and concerns. The formula is: ;in, Embedding vector representing the query statement; Represents a predefined set of intent categories; Representing a collection Middle intention category The embedding vector of Represents the dot product of the query sentence embedding vector and the intent category embedding vector; Indicates in the collection Find the function that maximizes the value ; Context-related document filtering is used to filter out documents related to the context based on the query intent. The formula is: ;in, Represents the filtered set of related documents; Represents the first documents; Intent vector representing the query Indicates The intent vector of each document; is a similarity function used to calculate the similarity between two intent vectors. is a similarity threshold used to determine whether a document is relevant to the query; Refined search sorting is used to sort the filtered documents more finely. The formula is: ;in, Representation Document Rank score; Indicates documents; From the document The set of all key sentences or paragraphs extracted from Representing a collection An element in , i.e., an index of a key sentence or paragraph; Indicates a key sentence or paragraph The weight of represents the query vector; Indicates a key sentence or paragraph The embedding vector of : represents the query vector With key sentence or paragraph vector The cosine similarity between ; Represents the query vector calculated based on the BM25 algorithm With key sentences or paragraphs The correlation score between Represents the query vector With key sentences or paragraphs The semantic similarity score between them; Dynamic feedback and query optimization are used to continuously optimize query results based on user feedback and query history. The formula is: ;in, represents the updated query vector; represents the query vector; represents the adjustment coefficient; Represents the first The difference between the documents; Represents the first The weight of a document; Represents the size of the document collection, that is, the first documents.

2. A data retrieval method based on a large language model according to claim 1, characterized in that: In step 2, a pre-trained language model based on the Transformer architecture is used as the large language model.

Images