Information retrieval method and device, electronic equipment, storage medium and computer program product

Abstract

The invention discloses an information retrieval method and device, electronic equipment, a storage medium and a computer program product, and the method comprises the steps: receiving a first query statement input by a user; judging whether a first list contains the first query statement or not to obtain a judgment result; the first list comprises one or more query statements, and each query statement in the first list is associated with zero or one or more answers in a set database; when the judgment result represents that the first list contains the first query statement and the first query statement is associated with one or more answers in the set database, randomly selecting a first answer from the one or more answers, and calculating a first score based on user interaction information of the first answer; and when the first score is greater than a first score threshold value, outputting the first answer as a retrieval result of the first query statement.

Description

Technical Field

[0001] The present application relates to the field of information technology, and in particular to information retrieval methods, devices, electronic devices, storage media and computer program products. Background Art

[0002] With the continuous advancement of information retrieval methods, people are increasingly concerned about how to improve the accuracy and exhaustiveness of the output results of the retrieval system. In related methods, the main focus is on general knowledge retrieval systems. However, when it is necessary to retrieve knowledge in a specific field or vertical field, the retrieval results of the system are less relevant to the query statement, resulting in poor output of the answer in the system. Summary of the invention

[0003] To solve related technical problems, the embodiments of the present application provide an information retrieval method, device, electronic device, storage medium and computer program product.

[0004] The technical solution of the embodiment of the present application is implemented as follows:

[0005] The present application provides an information retrieval method, including:

[0006] Receive a first query statement input by a user;

[0007] Determine whether the first query statement is included in the first list, and obtain a determination result; the first list includes one or more query statements, and each query statement in the first list is associated with zero or one or more answers in the database;

[0008] When the judgment result indicates that the first list includes the first query statement, and the first query statement is associated with one or more answers in the setting database, randomly selecting a first answer from the one or more answers, and calculating a first score based on user interaction information of the first answer;

[0009] When the first score is greater than a first score threshold, the first answer is output as a retrieval result of the first query statement.

[0010] In the above scheme, the method further comprises:

[0011] When the first score is less than or equal to the first score threshold, or when the judgment result indicates that the first list does not include the first query statement, or when the judgment result indicates that the first list includes the first query statement and the first query statement corresponds to zero answers in the set database, generating one or more answers to the first query statement through a retrieval-augmented generation (RAG) system;

[0012] One or more answers to the first query statement are associated with the first query statement and written into the setting database.

[0013] In the above scheme, the method further comprises:

[0014] When the first score is less than or equal to the first score threshold, the first answer is deleted from the setting data.

[0015] In the above solution, after outputting the first answer as the search result of the first query statement, the method further includes:

[0016] Update user interaction information of the first answer; the user interaction information includes user like information for the first answer and / or exposure information of the first answer.

[0017] In the above solution, the generating one or more answers to the first query statement by the RAG system includes:

[0018] The first query statement is searched based on a search strategy set to obtain first resource data; the search strategy set includes a first search strategy and / or a second search strategy, wherein the first search strategy is used to search after finding the intersection of terms in the first query statement, and the second search strategy is used to search a query vector generated based on the first query statement;

[0019] The language big model is called to process the first resource data to obtain one or more answers to the first query statement.

[0020] In the above solution, the first query statement is searched based on the search strategy set to obtain the first resource data, including:

[0021] In the retrieval strategy set, one or more retrieval strategies with the highest priority are used to search the first query statement to obtain the first resource data; wherein each retrieval strategy in the retrieval strategy set has a priority;

[0022] The calling language big model processes the first resource data to obtain one or more answers to the first query statement, including:

[0023] When the second score is greater than the second score threshold, the language macro model is called to process the first resource data to obtain one or more answers to the first query statement; wherein,

[0024] The second score represents the relevance between the first resource data and the first query statement.

[0025] In the above solution, after obtaining the first resource data, the method further includes:

[0026] When the second score is less than or equal to the second score threshold, in the search strategy set, one or more search strategies with lower priorities are sequentially used to search the first query statement to obtain second resource data;

[0027] When the third score is greater than the second score threshold, the language macro model is called to process the second resource data to obtain one or more answers to the first query statement; wherein,

[0028] The third score represents the relevance between the second resource data and the first query statement.

[0029] In the above solution, the calling language big model processes the first resource data to obtain one or more answers to the first query statement, including:

[0030] In response to the marking information input by the user, extracting third resource data from the first resource data, wherein the third resource data represents all or part of the first resource data marked by the marking information;

[0031] The language big model is called to process the third resource data to obtain one or more answers to the first query statement.

[0032] The present application also provides an information retrieval device, including:

[0033] A receiving unit, configured to receive a first query statement input by a user;

[0034] A first judgment unit is used to judge whether the first query statement is included in the first list, and obtain a judgment result; the first list includes one or more query statements, and each query statement in the first list is associated with zero or one or more answers in the database;

[0035] a calculation unit, configured to randomly select a first answer from the one or more answers, and calculate a first score based on user interaction information of the first answer, if the judgment result indicates that the first list includes the first query statement and the first query statement is associated with one or more answers in the setting database;

[0036] The second judgment unit is configured to output the first answer as a retrieval result of the first query statement when the first score is greater than a first score threshold.

[0037] An embodiment of the present application also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program that can be executed on the processor, and the processor is used to execute any of the above-mentioned information retrieval methods.

[0038] An embodiment of the present application further provides a storage medium on which a computer program is stored, wherein the computer program implements the steps of any of the above methods when executed by a processor.

[0039] An embodiment of the present application further provides a computer program product, including a computer program, characterized in that when the computer program is executed by a processor, the steps of any of the above methods are implemented.

[0040] In the information retrieval method, device, electronic device, storage medium and computer program product provided by the embodiments of the present application, first, it is determined whether the first query statement input by the user is included in the first list; second, when the first query statement is included in the first list, and the first query statement is associated with one or more answers in the set database, the first answer is randomly selected from the one or more answers; finally, the first score is calculated based on the user interaction information of the first answer, and when the first score is greater than the first score threshold, the first answer is output as the retrieval result of the first query statement. In actual application, the set database can be a database of a specific field or a vertical field, thereby improving the accuracy and relevance of the answers in the specific field or vertical field to the user's query statement, and can also improve the efficiency of information retrieval and reduce the time delay of information retrieval. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A schematic diagram of a flow chart of an information retrieval method according to an embodiment of the present application;

[0042] Figure 2 A flowchart of another information retrieval method according to an embodiment of the present application;

[0043] Figure 3 This is a flow chart of the third information retrieval method according to an embodiment of the present application;

[0044] Figure 4 This is a flow chart of the fourth information retrieval method according to an embodiment of the present application;

[0045] Figure 5 This is a schematic diagram of the structure of an information retrieval device according to an embodiment of the present application;

[0046] Figure 6 A schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0047] With the continuous advancement of information retrieval methods, people are increasingly concerned about how to improve the accuracy and exhaustiveness of the output results of the retrieval system. Among them, how to improve the relevance of search results and query statements has also become a key direction for search system optimization. Currently, more search systems on the market are built for general fields. These systems all call general language models, and the content of their answers cannot meet the needs of users in specific fields or vertical fields. Specifically, in specific fields or vertical fields, the answers output by these systems are often less relevant to the user's query statements, and the output answers are not rich enough. Among them, the specific field or vertical field includes but is not limited to the tourism field, education field, medical field or legal field.

[0048] In the information retrieval method, device, electronic device, storage medium and computer program product provided by the embodiments of the present application, first, it is determined whether the first query statement input by the user is included in the first list; second, when the first query statement is included in the first list, and the first query statement is associated with one or more answers in the set database, the first answer is randomly selected from the one or more answers; finally, the first score is calculated based on the user interaction information of the first answer, and when the first score is greater than the first score threshold, the first answer is output as the retrieval result of the first query statement. In actual application, the set database can be a database of a specific field or a vertical field, thereby improving the accuracy and relevance of the answers in the specific field or vertical field to the user's query statement, and can also improve the efficiency of information retrieval and reduce the time delay of information retrieval.

[0049] The present application is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0050] The present application embodiment provides an information retrieval method, referring to Figure 1 , the method comprising:

[0051] Step 101: receiving a first query statement input by a user;

[0052] Step 102: Determine whether the first list contains the first query statement, and obtain a determination result;

[0053] The first list includes one or more query statements, and each query statement in the first list is associated with zero or one or more answers in the database;

[0054] Step 103: When the judgment result indicates that the first list includes the first query statement and the first query statement is associated with one or more answers in the set database, randomly select a first answer from the one or more answers, and calculate a first score based on the user interaction information of the first answer;

[0055] Step 104: When the first score is greater than a first score threshold, the first answer is output as a search result of the first query statement.

[0056] In the embodiment of the present application, the first query statement is the query statement input by the user, which can also be referred to as a query in actual application. According to the number of Chinese word segments in the query statement, the query statement can be divided into a long query statement and a short query statement. For example, a query statement with a word number less than or equal to 2 is a short query statement, and a query statement with a word number greater than 2 is a long query statement.

[0057] In an embodiment of the present application, the first list may be a list of popular query statements filtered out based on the user's historical search data. The screening method may be an analysis method using big data or manual intervention, and according to the set extraction rules, some or all of the query statements in the user's historical search data are extracted as a list of popular query statements. For each query statement in the first list, it may correspond to zero or one or more answers in the set database, that is, each query statement in the first list may be associated with one or more answers in the set database, or there may be no associated answers. The first list may be stored in the set database or in other databases. In actual application, the database may be referred to as database or db. In the case of a large number of users performing queries, there will be a large number of repeated, highly popular, identical or similar query statements, which will cause huge traffic and high concurrency. Therefore, by filtering out a list of popular query statements, and caching the corresponding answers of these query statement lists in the set database, when repeated query statements appear, they can be directly searched from the database, avoiding the waste of huge computing power by regenerating answers through a large model each time.

[0058] In the embodiment of the present application, it is determined whether the first list contains the first query statement to obtain a determination result. If the first list contains the first query statement and the first query statement is associated with one or more answers in the set database, the first answer can be randomly selected from the one or more answers. Specifically, if the first query statement is associated with only one answer in the set database, then this answer is selected as the first answer.

[0059] In the embodiment of the present application, the user interaction information of the first answer includes one or more of the like information, dislike information, and exposure information of the first answer. The first score can be calculated through the user interaction information of the first answer. Specifically, when the number of likes increases, the score becomes higher; when the number of dislikes increases, the score decreases; when the number of exposures is large but the number of likes or dislikes is small, the score decreases.

[0060] In the embodiment of the present application, when the first score is greater than the first score threshold, it indicates that the user is relatively satisfied with the first answer, and it can be considered that the quality of the first answer is high. At this time, the first answer is output as the retrieval result of the first query statement.

[0061] In one embodiment, the method further comprises:

[0062] When the first score is less than or equal to the first score threshold, the first answer is deleted from the setting database.

[0063] In an embodiment of the present application, when the first score is less than or equal to the first score threshold, it indicates that the quality of the first answer is low or the relevance is poor. At this time, the first answer can be deleted from the setting database to avoid calling the first answer again or avoiding outputting the first answer as a search result.

[0064] In one embodiment, after outputting the first answer as a search result of the first query statement, the method further includes:

[0065] Update the user interaction information of the first answer; the user interaction information includes the user's like information on the first answer and / or the exposure information of the first answer.

[0066] In an embodiment of the present application, after the first answer is output as the search result of the first query statement, the first answer is displayed on the user interface, and the user can give a thumbs-up or thumbs-down to the first answer. The first answer can also continuously update the number of exposures, thereby updating the user interaction information of the first answer. The user interaction information of the first answer includes one or more of the thumbs-up information, thumbs-down information, and exposure information of the first answer. In actual application, the user interaction information of the first answer may also not include thumbs-down information. By updating the user interaction information of the first answer, the first score of the first answer can be updated when the first answer is selected later. When the first answer does not reach the first score threshold, the first answer can be not selected as the search result of the first query statement, thereby realizing the dynamic update of the search result output and ensuring the output quality of the search result.

[0067] In one embodiment, the method further comprises:

[0068] When the first score is less than or equal to the first score threshold, or when the judgment result indicates that the first list does not include the first query statement, or when the judgment result indicates that the first list includes the first query statement and the first query statement corresponds to zero answers in the set database, one or more answers to the first query statement are generated by the RAG system;

[0069] One or more answers to the first query statement are associated with the first query statement and written into the setting database.

[0070] In an embodiment of the present application, in any one of situations one to three, one or more answers to the first query statement are generated through the RAG system. Among them, situation one refers to when the first score is less than or equal to the first score threshold, which indicates that the quality of the first answer is poor; situation two refers to the judgment result that the first list does not contain the first query statement, indicating that the first query statement is less popular; situation three refers to the judgment result that the first list contains the first query statement, and the first query statement corresponds to zero answers in the set database, indicating that the set database does not store the answer corresponding to the first query statement. In these three situations, in order to obtain high-quality answers, it is necessary to generate one or more answers to the first query statement through the RAG system, associate the one or more answers to the first query statement with the first query statement and write them into the set database.

[0071] In the embodiment of the present application, by generating one or more answers to the first query statement through the RAG system in the above three situations, associating the one or more answers to the first query statement with the first query statement and writing them into the setting database, the defect of no answer corresponding to the first query statement or poor quality of the corresponding answer can be compensated, and the one or more answers generated for the first query statement can be written into the setting database to facilitate the rapid output of the retrieval results of the first query statement. When a query statement identical to the first query statement appears subsequently, the one or more answers generated above can also be directly called in the setting database, which can effectively improve the response delay and throughput of the system.

[0072] In one embodiment, generating one or more answers to the first query statement by the RAG system includes:

[0073] The first query statement is searched based on a search strategy set to obtain first resource data; the search strategy set includes a first search strategy and / or a second search strategy, wherein the first search strategy is used to search after finding the intersection of terms in the first query statement, and the second search strategy is used to search a query vector generated based on the first query statement;

[0074] The language big model is called to process the first resource data to obtain one or more answers to the first query statement.

[0075] In the embodiment of the present application, the retrieval strategy set includes one or more retrieval strategies, that is, the first query statement can be searched based on one or more retrieval strategies in the retrieval strategy set to obtain the first resource data.

[0076] In the embodiment of the present application, the retrieval strategy set includes the first retrieval strategy and / or the second retrieval strategy. The first retrieval strategy is used to search after the intersection of the words in the first query statement. For example, when the first query statement is "How to play in Chengdu tourism", the segmentation results are "Chengdu", "tourism" and "how to play", then the first retrieval strategy is "Chengdu & Tourism & How to Play", where "&" represents the intersection, that is, all corresponding segmentations need to appear in the retrieved document. Further, the first retrieval strategy can also be used to search after the intersection of the core words in the first query statement. For example, when the first query statement is "How to play in Chengdu tourism", the segmentation results are "Chengdu", "tourism" and "how to play", where the core words are "Chengdu" and "tourism", then the first retrieval strategy is "Chengdu & Tourism &? How to play", that is, the corresponding core words all need to appear in the retrieved document, where "&?" indicates that the non-core word "how to play" does not necessarily need to appear in the retrieved document, and if it appears, it will increase the relevance of the result. The retrieval strategy set may include one or more first retrieval strategies.

[0077] In an embodiment of the present application, the first search strategy and / or the second search strategy are included in the retrieval strategy set. The second search strategy is used to retrieve the query vector generated based on the first query statement. Exemplarily, when the first query statement is "How to play Chengdu tourism", each word of the first query statement can be mapped to a high-dimensional vector space, and the vector is normalized, and the query vector generated by the first query statement is retrieved by the vector similarity algorithm. Further, the second search strategy can also be used to generate a query statement similar to the semantics of the first query statement, and retrieve based on the query statement similar to the semantics. Exemplarily, when the first query statement is "How to play Chengdu tourism", a query statement of "Chengdu travel guide", "Chengdu travel route" or "Sichuan provincial capital city travel guide" similar to the semantics of the first query statement can be generated, and the query vector generated based on the query statement similar to the above semantics is retrieved. Similarly, one or more second search strategies can also be included in the retrieval strategy set.

[0078] In the embodiment of the present application, the first query statement is searched based on a set of search strategies to obtain first resource data, and the language macro model is called to process the first resource data to obtain one or more answers to the first query statement. The use of different search strategies, including text search in the first search strategy, vector search in the second search strategy, or hybrid search shared by the first search strategy and the second search strategy, can improve the accuracy and relevance of the language macro model's answer, thereby improving the quality of one or more answers to the first query statement.

[0079] In one embodiment, searching the first query statement based on the search strategy set to obtain the first resource data includes:

[0080] In the retrieval strategy set, use one or more retrieval strategies with the highest priority to search the first query statement to obtain the first resource data; wherein each retrieval strategy in the retrieval strategy set has a priority;

[0081] The calling language big model processes the first resource data to obtain one or more answers to the first query statement, including:

[0082] When the second score is greater than the second score threshold, the language macro model is called to process the first resource data to obtain one or more answers to the first query statement; wherein,

[0083] The second score represents the relevance between the first resource data and the first query statement.

[0084] In the embodiment of the present application, a priority can be assigned to each search strategy in the search strategy set, and the smaller the value, the higher the priority. For example, if the priority of search strategy A is 1 and the priority of search strategy B is 2, then the priority of search strategy A is higher than the priority of search strategy B, wherein search strategy A can be to search after finding the intersection of the words in the first query statement, and search strategy B can be to search the query vector generated based on the first query statement. Exemplarily, the retrieval strategy set may include retrieval strategy C, retrieval strategy D, retrieval strategy E and retrieval strategy F, the priority of retrieval strategy C and retrieval strategy D is 1, and the priority of retrieval strategy E and retrieval strategy F is 2, then retrieval strategies C and D can be performed simultaneously, retrieval strategies E and F can be performed simultaneously, and the priority of retrieval strategies C and D is higher than the priority of retrieval strategies E and F, wherein retrieval strategy C can be to search after intersecting the words in the first query statement, retrieval strategy D can be to search the query vector generated based on the first query statement, retrieval strategy E can be to search after intersecting the core words in the first query statement, and retrieval strategy F can be to generate a query statement semantically similar to the first query statement, and search based on the semantically similar query statement.

[0085] In an embodiment of the present application, in the retrieval strategy set, one or more retrieval strategies with the highest priority are used to search the first query statement to obtain the first resource data. After obtaining the first resource data, the relevance between the first resource data and the first query statement is scored to obtain a second score. When the second score is greater than the second score threshold, the language model is called to process the first resource data to obtain one or more answers to the first query statement. Among them, when using retrieval strategies related to text retrieval, the relevance is scored using the BM25 similarity algorithm; when using retrieval strategies related to vector retrieval, the relevance is scored using the vector similarity score.

[0086] In an embodiment of the present application, by using one or more retrieval strategies with the highest priority in a retrieval strategy set to search for a first query statement, first resource data is obtained, and when the second score is greater than the second score threshold, the language macro model is called to process the first resource to obtain one or more answers to the first query statement. This not only ensures that the retrieval strategy with the highest priority is used first, but also ensures that the language macro model will be called to process the first resource data only when the first resource data obtained by the retrieval strategy with the highest priority has a high correlation with the first query statement.

[0087] In one embodiment, after obtaining the first resource data, the method further includes:

[0088] When the second score is less than or equal to the second score threshold, in the search strategy set, one or more search strategies with lower priorities are sequentially used to search the first query statement to obtain second resource data;

[0089] When the third score is greater than the second score threshold, the language model is called to process the second resource data to obtain one or more answers to the first query statement; wherein,

[0090] The third score represents the relevance between the second resource data and the first query statement.

[0091] In the embodiment of the present application, after obtaining the first resource data using the highest priority search strategy, the first resource data is scored to obtain a second score, and when the second score is less than or equal to the second score threshold, it is proved that the correlation between the first resource data and the first query statement is not high at this time, and it is necessary to adopt the method of expanded recall, that is, in the set of search strategies, one or more search strategies with lower priority are used in turn to retrieve the first query statement to obtain the second resource data, and the correlation between the second resource data and the first query statement is scored to obtain a third score. When the third score is greater than the second score threshold, it means that the second resource data has a high correlation with the first query statement, that is, the second resource data can be expanded and recalled to facilitate calling the language large model to process the second resource data and obtain one or more answers to the first query statement.

[0092] In an embodiment of the present application, when the correlation between the first resource data and the first query statement is not high, the first query statement is retrieved using one or more retrieval strategies with lower priority by adopting an expanded recall method to obtain the second resource data, thereby avoiding the situation where no resource data is available.

[0093] In one embodiment, calling a language macro model to process the first resource data to obtain one or more answers to the first query statement includes:

[0094] In response to the marking information input by the user, extracting the third resource data from the first resource data, the third resource data representing all or part of the first resource data marked by the marking information;

[0095] The language big model is called to process the third resource data to obtain one or more answers to the first query statement.

[0096] In an embodiment of the present application, after the user inputs the first query statement, the first resource data is obtained based on one or more retrieval strategies in the retrieval strategy set, and then the first resource data is processed by calling the language large model to obtain one or more answers to the first query statement. After the one or more answers are output, the user can not only see the answers, but also see what data the adopted first resource data includes. When the user is not satisfied with the result, the user can select and mark the first resource data, that is, extract the third resource data from the first resource data, and the third resource data represents all or part of the first resource data marked by the marking information. Exemplarily, the first resource data includes document 1, document 2 and document 3. If the user does not want to use document 3, document 1 and document 2 are extracted as the third resource data. Afterwards, the language large model can be called to process the third resource data to obtain one or more answers to the first query statement again.

[0097] The present application is further described in detail below in conjunction with application examples.

[0098] The present application proposes another embodiment of the information retrieval method, such as Figure 2 As shown, in this embodiment, answers to popular query statements can be prefabricated and further optimized. Specifically, this is achieved through the following steps:

[0099] Step 1: Use the user's historical search records, after big data analysis, extract the top-ranked popular queries, and filter them with manual intervention and other means to retain the final popular query list. When the user's query statement appears in the popular query list, the user's query statement is determined to be a popular query;

[0100] Step 2: Determine whether the user's input query is in the hot query list. If the query is in the hot query list, go to step 4. If the query is not in the hot query list, go to step 3.

[0101] Step 3: Enter the vertical search RAG system, search for the query and input it into the big model to generate the final answer. The vertical search RAG system represents a system built based on a specific vertical field, and the documents in the knowledge base of the system are mainly related documents in the field. Furthermore, the big model can also be fine-tuned in combination with the knowledge base in the system;

[0102] Step 4: Determine whether the answer exists in the cached database (db). If so, proceed to step 5. If not, proceed to step 6.

[0103] Step 5: First, select an answer for the user (if there are multiple answers, select one randomly), and calculate the score of the answer based on the data of the number of likes, dislikes, and exposures. If it is a high-scoring answer, proceed to step 7; if it is a low-scoring answer, delete the answer and proceed to step 6; the score calculation formula is:

[0104]

[0105] Among them, w 1 is the like weight, c 1 is the number of likes, w 2 is the weight of stepping on, c 2 is the number of thumbs down, exposure is the number of exposures, P is the experience value (for example, 5%), w 3 To suppress the weight. When the ratio of likes + dislikes to exposure is high, 1 、c 2 The score is calculated using a weighted approach. On the contrary, if the answer has a lot of exposure but few active user interactions (active interactions include likes and dislikes), it needs to be suppressed to a certain extent, that is, the suppression score calculated by subtracting the number of exposures.

[0106] Step 6: Generate answers based on RAG retrieval, and perform multiple comparative tests based on ABTest to generate multiple sets of answers for the same question (the temperature parameter of the large model is not 0 and has a certain degree of randomness), and write the answers to the db, and then proceed to step 7;

[0107] Step 7: Display the answer and record the user's click / dislike behavior, and update the like / dislike / exposure data (where likes and dislikes are user behaviors, and exposure is the number of times the result is displayed, usually each display of the answer is recorded as one exposure).

[0108] In the above embodiment, the answers generated by popular queries are mainly cached in the db. When repeated popular queries appear, they are directly searched from the db, thereby avoiding the waste of a lot of computing power by regenerating answers through a large model each time; at the same time, a manual intervention interface is provided to support the modification of answers to specific queries and write them directly into the db, thereby avoiding the problems of poor quality, poor relevance or uncontrollable answers generated by some large models. In addition, the quality score of the answer is dynamically calculated based on the user's exposure, clicks, and thumbs-down visits to the answer, and the pre-made answers corresponding to the popular query are updated according to the quality score. When the established strategy is met, the answer is directly displayed, which can effectively improve the system's response latency and throughput.

[0109] The present application proposes a third embodiment of the information retrieval method, such as Figure 3As shown, in this embodiment, when there is no pre-made answer (that is, when there is no answer corresponding to the query statement in the cached db), retrieval can be performed using retrieval strategies of different priorities. Specifically, this is achieved by the following steps:

[0110] Step 1: Perform Chinese word segmentation on the query;

[0111] Step 2: Determine the number of words. When the number of words is <= 2, proceed to steps 3-1 and 3-2. When the number of words is > 2, proceed to step 4.

[0112] Step 3-1: When the number of words is less than or equal to 2, generate a text search strategy strategy1, and assign the query strategy a priority of priority=1 (wherein, the smaller the number, the higher the priority. When multiple priority strategies are generated, the strategy with the highest priority is implemented first; when the search strategy with the highest priority has a result, the strategy with the lowest priority is no longer implemented. This strategy is to ensure that high-quality results are given priority, and at the same time, if there is no result, a slightly lower-quality result is used as a backup), and configure the search strategy to search after the intersection of words, for example, the word segmentation result is AB, and the query statement is A&B;

[0113] Step 3-2: When the number of words is less than or equal to 2, generate a vector search strategy strategy2, assign the priority of the search strategy to priority=2, and configure the search strategy to generate a vector for the query statement before searching;

[0114] Step 4: When the number of words is > 2, use multiple search strategies for search. When a high-priority strategy exists, a low-priority strategy will not be executed. Only when the high-priority strategy has no results or the score does not meet the standard, the low-priority strategy will have a chance to be executed. When the priorities are the same, the execution is parallel. For example, the search strategies A and B with priority = 1 are generated, and the search strategies C and D with priority = 2 are generated. A and B are executed in parallel first. When there are results and the score requirements are met, C and D are no longer executed. When A and B have no results, C and D are executed in parallel. Step 4 includes vector retrieval in steps 4-1 and 4-2, and text retrieval in steps 4-3 to 4-5.

[0115] Step 4-1: Generate vector search query strategy 3, use the original query for search, and assign priority = 1;

[0116] Step 4-2: By interacting with the big model, the original query is expanded to generate a semantically similar query query', a vector retrieval query strategy strategy4 is generated, query' is used for querying, and priority = 2 is assigned;

[0117] Step 4-3: Extract query core words;

[0118] Step 4-4: Generate text search strategy strategy5, the search strategy is to find the intersection of all words (for example, query = how to travel in Chengdu, the word segmentation results are Chengdu, travel, how to travel, and the generated search strategy is Chengdu & travel & how to travel, that is, all corresponding words need to appear in the searched document), and assign priority = 1;

[0119] Step 4-5: Generate text search strategy strategy6. The search strategy is to find the intersection of all core words, and non-core words are optional words to participate in the search (they can be omitted in the search system, and the hit will increase the relevance score). At the same time, priority = 2 is assigned (here is an example: for example, query = how to play in Chengdu tourism, the word segmentation results are Chengdu, tourism, how to play, and the core words are Chengdu, tourism. The generated search strategy is Chengdu & Tourism &? How to play, that is, the corresponding core words must appear in the searched document, and non-core words do not necessarily need to appear. If they appear, the relevance of the result will be improved);

[0120] Step 4-6: Merge the strategies generated in steps 4-1 to 4-5 for retrieval;

[0121] Step 5: Search and query the vector database (if there is a vector search strategy) and the text search database (if there is a text search strategy) according to the search strategy priority. Specifically, first search for the strategy with priority=1, and if the result does not meet the requirements, search for the strategy with priority=2.

[0122] Step 6: When there are search results, score the result set. The calculation formula is:

[0123]

[0124] Among them, score_i is the relevance score returned by each result during retrieval (each retrieval result has a relevance score with the original query. Text retrieval is generally calculated using BM25, and vector retrieval is generally calculated using vector similarity score. When there are multiple recalls, algorithms such as RRF are generally used to normalize the result scores of different recall types), and weight is the weight of the retrieval strategy priority corresponding to the result.

[0125] Step 7: After determining whether the number and score of the result set meet the requirements, input them into the large model for answer input; when the result combination does not meet the requirements (including the number of results and the score), return to step 5 to query again, and expand the search results by performing multiple rounds of expansion and recall to meet the retrieval requirements under different scenarios and queries, thereby improving the relevance of the answer output of the large model and avoiding the situation where no results are available. Among them, expansion and recall means that high-priority results are executed first. If the results are not satisfied, low-priority results will have the opportunity to be executed, ensuring that high-quality results are given priority, and recalling slightly lower-quality results when there are no high-quality results.

[0126] In the above embodiment, the correlation between the length of the query and the search effect is mainly considered, and the three search strategies of text search, vector search, and hybrid search are used according to the length of the query. Multiple rounds of recall are used for the search. When the optimal search strategy has no results, the candidate strategy is supplemented with recall, which can optimize the search results and improve the answer quality of the large model answer.

[0127] Currently, all kinds of RAG systems in the industry are black boxes for users, and the correlation between search results and large model output cannot be controlled. Considering that the results of the retrieval system cannot meet all the needs of users, there are problems such as poor sorting effect and inaccurate recall. This application proposes an embodiment of the fourth information retrieval method, such as Figure 4 As shown, in this embodiment, the search result list can be displayed at the same time as the result is displayed. If the user is not satisfied with the generated answer, the user can manually select which search results to generate the desired answer from. Specifically, this is achieved by the following steps:

[0128] Step 1: Search based on user query;

[0129] Step 2: The large model generates answers based on the search results and the user query;

[0130] Step 3: Display the answer given by the big model and the search results list in the process, including the search source, title, summary and other information;

[0131] Step 4: Prompt the user whether they are satisfied. The user can choose whether to regenerate. If satisfied, the process ends directly. Otherwise, proceed to step 5.

[0132] Step 5: The user manually selects the document that he wants the big model to refer to and submits it to the big model to regenerate the answer.

[0133] In the above embodiment, by allowing users to independently select documents for reference in the large model, the needs of different users can be met, and the satisfaction and relevance of search results can be improved.

[0134] In order to implement the information retrieval method of the embodiment of the present application, the embodiment of the present application also provides an information retrieval device, such as Figure 5 Said device comprises:

[0135] The receiving unit 501 is used to receive a first query statement input by a user;

[0136] A first determination unit 502 is used to determine whether the first query statement is included in the first list, and obtain a determination result; the first list includes one or more query statements, and each query statement in the first list is associated with zero or one or more answers in the database;

[0137] A calculation unit 503 is configured to randomly select a first answer from the one or more answers, and calculate a first score based on user interaction information of the first answer, if the judgment result indicates that the first list includes the first query statement and the first query statement is associated with one or more answers in the setting database;

[0138] The second judgment unit 504 is configured to output the first answer as a search result of the first query statement when the first score is greater than a first score threshold.

[0139] In one embodiment, the device further comprises:

[0140] A generating unit, configured to generate one or more answers to the first query statement by using a retrieval enhancement generation RAG system when the first score is less than or equal to the first score threshold, or when the judgment result indicates that the first list does not include the first query statement, or when the judgment result indicates that the first list includes the first query statement and the first query statement corresponds to zero answers in the set database;

[0141] The writing unit is used to associate one or more answers of the first query statement with the first query statement and write the answers into the setting database.

[0142] In one embodiment, the device further comprises:

[0143] A deleting unit is used to delete the first answer in the setting database when the first score is less than or equal to the first score threshold.

[0144] In one embodiment, the device further comprises:

[0145] An updating unit is used to update user interaction information of the first answer after the first answer is output as a retrieval result of the first query statement; the user interaction information includes user like information of the first answer and / or exposure information of the first answer.

[0146] In one embodiment, the generating one or more answers to the first query statement by the RAG system includes:

[0147] The first query statement is searched based on a search strategy set to obtain first resource data; the search strategy set includes a first search strategy and / or a second search strategy, wherein the first search strategy is used to search after finding the intersection of terms in the first query statement, and the second search strategy is used to search a query vector generated based on the first query statement;

[0148] The language big model is called to process the first resource data to obtain one or more answers to the first query statement.

[0149] In one embodiment, the first query statement is searched based on the search strategy set to obtain the first resource data, including:

[0150] In the retrieval strategy set, use one or more retrieval strategies with the highest priority to search the first query statement to obtain the first resource data; wherein each retrieval strategy in the retrieval strategy set has a priority;

[0151] The calling language big model processes the first resource data to obtain one or more answers to the first query statement, including:

[0152] When the second score is greater than the second score threshold, the language macro model is called to process the first resource data to obtain one or more answers to the first query statement; wherein,

[0153] The second score represents the relevance between the first resource data and the first query statement.

[0154] In one embodiment, the device further comprises:

[0155] A retrieval unit, configured to, after obtaining the first resource data, in a case where the second score is less than or equal to the second score threshold, sequentially use one or more retrieval strategies with lower priorities in the retrieval strategy set to search the first query statement to obtain the second resource data;

[0156] A calling unit is used to call the language model to process the second resource data to obtain one or more answers to the first query statement when the third score is greater than the second score threshold; wherein the third score represents the correlation between the second resource data and the first query statement.

[0157] In one embodiment, the calling language macro model processes the first resource data to obtain one or more answers to the first query statement, including:

[0158] In response to the marking information input by the user, extracting the third resource data from the first resource data, the third resource data representing all or part of the first resource data marked by the marking information;

[0159] The language big model is called to process the third resource data to obtain one or more answers to the first query statement.

[0160] In practical application, the above-mentioned units can be implemented by a processor in the information retrieval device.

[0161] It should be noted that the information retrieval device provided in the above embodiment is only illustrated by the division of the above program modules. In actual applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device is divided into different program modules to complete all or part of the above-described processing. In addition, the information retrieval device provided in the above embodiment and the information retrieval method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0162] Based on the hardware implementation of the above program modules, the embodiment of the present application also provides an electronic device, such as Figure 6 As shown, the electronic device 600 includes:

[0163] Communication interface 601, capable of exchanging information with other network nodes;

[0164] The processor 602 is connected to the communication interface 601 to implement information exchange with other network nodes, and is used to execute the method provided by one or more technical solutions when running a computer program. The computer program is stored in the memory 603.

[0165] When the processor 602 runs the computer program to execute the above one or more information retrieval methods, specifically:

[0166] The processor 602 is configured to:

[0167] Receive a first query statement input by a user;

[0168] Determine whether the first query statement is included in the first list, and obtain a determination result; the first list includes one or more query statements, and each query statement in the first list is associated with zero or one or more answers in the database;

[0169] If the judgment result indicates that the first list includes the first query statement, and the first query statement is associated with one or more answers in the setting database, randomly select a first answer from the one or more answers, and calculate a first score based on user interaction information of the first answer;

[0170] When the first score is greater than a first score threshold, the first answer is output as a retrieval result of the first query statement.

[0171] In one embodiment, the processor 602 is further configured to:

[0172] When the first score is less than or equal to the first score threshold, or when the judgment result indicates that the first list does not include the first query statement, or when the judgment result indicates that the first list includes the first query statement and the first query statement corresponds to zero answers in the set database, one or more answers to the first query statement are generated by the retrieval enhancement generation RAG system;

[0173] One or more answers to the first query statement are associated with the first query statement and written into the setting database.

[0174] In one embodiment, the processor 602 is further configured to:

[0175] When the first score is less than or equal to the first score threshold, the first answer is deleted from the setting database.

[0176] In one embodiment, after outputting the first answer as a search result of the first query statement, the processor 602 is further configured to:

[0177] Update the user interaction information of the first answer; the user interaction information includes the user's like information on the first answer and / or the exposure information of the first answer.

[0178] In one embodiment, generating one or more answers to the first query statement by the RAG system includes:

[0179] The first query statement is searched based on a search strategy set to obtain first resource data; the search strategy set includes a first search strategy and / or a second search strategy, wherein the first search strategy is used to search after finding the intersection of terms in the first query statement, and the second search strategy is used to search a query vector generated based on the first query statement;

[0180] The language big model is called to process the first resource data to obtain one or more answers to the first query statement.

[0181] In one embodiment, searching the first query statement based on the search strategy set to obtain the first resource data includes:

[0182] In the retrieval strategy set, use one or more retrieval strategies with the highest priority to search the first query statement to obtain the first resource data; wherein each retrieval strategy in the retrieval strategy set has a priority;

[0183] The calling language big model processes the first resource data to obtain one or more answers to the first query statement, including:

[0184] When the second score is greater than the second score threshold, the language macro model is called to process the first resource data to obtain one or more answers to the first query statement; wherein,

[0185] The second score represents the relevance between the first resource data and the first query statement.

[0186] In one embodiment, the processor 602 is further configured to:

[0187] After obtaining the first resource data, when the second score is less than or equal to the second score threshold, in the retrieval strategy set, one or more retrieval strategies with lower priorities are sequentially used to search the first query statement to obtain the second resource data;

[0188] When the third score is greater than the second score threshold, the language model is called to process the second resource data to obtain one or more answers to the first query statement; wherein,

[0189] The third score represents the relevance between the second resource data and the first query statement.

[0190] In one embodiment, the calling language macro model processes the first resource data to obtain one or more answers to the first query statement, including:

[0191] In response to the marking information input by the user, extracting the third resource data from the first resource data, the third resource data representing all or part of the first resource data marked by the marking information;

[0192] The language big model is called to process the third resource data to obtain one or more answers to the first query statement.

[0193] It should be noted that the specific processing process of the processor 602 and the communication interface 601 can be understood by referring to the above method.

[0194] Of course, in actual application, the various components in the electronic device 600 are coupled together through the bus system 604. It can be understood that the bus system 604 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 604 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Figure 6 Various buses are labeled as bus system 604 .

[0195] The memory 603 in the embodiment of the present application is used to store various types of data to support the operation of the electronic device 600. Examples of such data include: any computer program used to operate on the electronic device 600.

[0196] The method disclosed in the above embodiment of the present application can be applied to the processor 602, or implemented by the processor 602. The processor 602 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 602. The above processor 602 may be a general-purpose processor, a digital signal processor (DSP, Digital Signal Processor), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 602 can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiment of the present application, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in the memory 603, and the processor 602 reads the information in the memory 603 and completes the steps of the above method in combination with its hardware.

[0197] In an exemplary embodiment, the electronic device 600 can be implemented by one or more application specific integrated circuits (ASIC), DSP, programmable logic device (PLD), complex programmable logic device (CPLD), field programmable gate array (FPGA), general processor, controller, microcontroller (MCU), microprocessor, or other electronic components to execute the aforementioned method.

[0198] It can be understood that the memory 603 of the embodiment of the present application can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM, SyncLink Dynamic Random Access Memory), and direct RAMbus random access memory (DRRAM, Direct Rambus Random Access Memory).The memories described in the embodiments of the present application are intended to include, but are not limited to, these and any other suitable types of memories.

[0199] In an exemplary embodiment, the present application also provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, for example, a memory 603 storing a computer program, and the computer program can be executed by a processor 602 of an electronic device 600 to complete the steps of the aforementioned information retrieval method. The computer-readable storage medium can be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface storage, optical disk, or CD-ROM.

[0200] Illustratively, an embodiment of the present application further provides a computer program product, including a computer program, which can be executed by the processor 602 of the electronic device 600 to complete the steps of the aforementioned information retrieval method.

[0201] It should be noted that: "first", "second", etc. are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0202] The term "one or more" herein refers to any combination of at least two of any one or more of a plurality, for example, including at least one of A, B, and C may refer to any one or more elements selected from the set consisting of A, B, and C. The term "one or more" herein refers to any one of a plurality or any combination of at least two of a plurality, for example, including at least one of A, B, and C may refer to any one or more elements selected from the set consisting of A, B, and C.

[0203] In addition, the technical solutions described in the embodiments of the present application can be combined arbitrarily without conflict.

[0204] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application.

Claims

1. An information retrieval method, characterized in that: The method comprises: Receive a first query statement input by a user; Determine whether the first query statement is included in the first list, and obtain a determination result; the first list includes one or more query statements, and each query statement in the first list is associated with zero or one or more answers in the database; When the judgment result indicates that the first list includes the first query statement, and the first query statement is associated with one or more answers in the setting database, randomly selecting a first answer from the one or more answers, and calculating a first score based on user interaction information of the first answer; When the first score is greater than a first score threshold, the first answer is output as a retrieval result of the first query statement.

2. The information retrieval method according to claim 1, characterized in that: The method further comprises: When the first score is less than or equal to the first score threshold, or when the judgment result indicates that the first list does not include the first query statement, or when the judgment result indicates that the first list includes the first query statement and the first query statement corresponds to zero answers in the setting database, one or more answers to the first query statement are generated by the retrieval enhancement generation RAG system; One or more answers to the first query statement are associated with the first query statement and written into the setting database.

3. The information retrieval method according to claim 2, characterized in that: The method further comprises: When the first score is less than or equal to the first score threshold, the first answer is deleted from the setting database.

4. The information retrieval method according to claim 1, characterized in that: After outputting the first answer as a search result of the first query statement, the method further includes: Update user interaction information of the first answer; the user interaction information includes user like information for the first answer and / or exposure information of the first answer.

5. The information retrieval method according to claim 2, characterized in that: Generating one or more answers to the first query statement by the RAG system includes: The first query statement is searched based on a search strategy set to obtain first resource data; the search strategy set includes a first search strategy and / or a second search strategy, wherein the first search strategy is used to search after finding the intersection of terms in the first query statement, and the second search strategy is used to search a query vector generated based on the first query statement; The language big model is called to process the first resource data to obtain one or more answers to the first query statement.

6. The information retrieval method according to claim 5, characterized in that: The step of searching the first query statement based on the search strategy set to obtain the first resource data includes: In the retrieval strategy set, one or more retrieval strategies with the highest priority are used to search the first query statement to obtain the first resource data; wherein each retrieval strategy in the retrieval strategy set has a priority; The calling language big model processes the first resource data to obtain one or more answers to the first query statement, including: When the second score is greater than the second score threshold, the language macro model is called to process the first resource data to obtain one or more answers to the first query statement; wherein, The second score represents the relevance between the first resource data and the first query statement.

7. The information retrieval method according to claim 6, characterized in that: After obtaining the first resource data, the method further includes: When the second score is less than or equal to the second score threshold, in the search strategy set, one or more search strategies with lower priorities are sequentially used to search the first query statement to obtain second resource data; When the third score is greater than the second score threshold, the language macro model is called to process the second resource data to obtain one or more answers to the first query statement; wherein, The third score represents the relevance between the second resource data and the first query statement.

8. The information retrieval method according to claim 5, characterized in that: The calling language big model processes the first resource data to obtain one or more answers to the first query statement, including: In response to the marking information input by the user, extracting third resource data from the first resource data, wherein the third resource data represents all or part of the first resource data marked by the marking information; The language big model is called to process the third resource data to obtain one or more answers to the first query statement.

9. An information retrieval device, characterized in that: include: A receiving unit, configured to receive a first query statement input by a user; A first judgment unit, used to judge whether the first query statement is included in the first list, and obtain a judgment result; The first list includes one or more query statements, and each query statement in the first list is associated with zero or one or more answers in the database; a calculation unit, configured to randomly select a first answer from the one or more answers, and calculate a first score based on user interaction information of the first answer, if the judgment result indicates that the first list includes the first query statement and the first query statement is associated with one or more answers in the setting database; The second judgment unit is configured to output the first answer as a retrieval result of the first query statement when the first score is greater than a first score threshold.

10. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program executable on the processor, and the processor is used to execute the method for information retrieval according to any one of claims 1 to 8.

11. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images