Query statement rewriting method and device and program product

Abstract

The invention discloses a query statement rewriting method and device and a program product, and relates to the technical field of search. And taking the original query statement, a plurality of query rewriting statements generated by the original query statement and the correlation score between the original query statement and each query rewriting statement as training data, training to obtain a rewriter, and rewriting the original query statement by the rewriter to obtain a target query statement. As the rewriter learns the relevance score between the original query statement and each query rewriting statement in the training stage, the rewriter can be directly used for query rewriting, and when the rewriter is used for query rewriting, the target query statement with the higher relevance score with the original query statement can be output, so that the query rewriting efficiency is improved. Compared with the existing query rewriting technology, the method has the advantage that the rewriting efficiency and the rewriting quality of the query statement can be improved.

Description

Technical Field

[0001] This application relates to the field of search technology, and in particular, to a query statement rewriting method, a query statement rewriting device, and a computer program product. Background Art

[0002] In open-domain question answering and other retrieval-based tasks, the ability to rewrite queries is crucial for improving the performance of the system. The purpose of query rewriting is to improve the accuracy of the original query expression to avoid ambiguity, adapt to the rules of different search systems or databases, and can improve the recall rate.

[0003] However, in existing query rewriting technologies, such as using keywords for keyword expansion, the disadvantage is that continuous collection of domain keywords is required to improve the effect; or using the method of large model prompts for rewriting, the advantage is that it is convenient to implement and no additional training is required, but the disadvantage is that the large model has high requirements for its own rewriting and term decomposition capabilities, and using the large model for rewriting will significantly increase the time and hardware consumption of the overall retrieval system.

[0004] The above content is only used to assist in understanding the technical solution of this application, and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main purpose of this application is to provide a query statement rewriting method, a query statement rewriting device, and a computer program product, aiming to solve the technical problems of low efficiency and poor quality in rewriting current query statements.

[0006] To achieve the above object, this application proposes a query statement rewriting method, and the query statement rewriting method includes:

[0007] Obtain an original query statement, and generate multiple query rewritten statements of the original query statement;

[0008] Determine the correlation score between the original query statement and each of the query rewritten statements;

[0009] Use the original query statement, each of the query rewritten statements, and the correlation score as training data to train a rewrite engine, so as to rewrite the original query statement through the rewrite engine to obtain a target query statement.

[0010] In one embodiment, the step of determining the correlation score between the original query statement and each of the query rewritten statements includes:

[0011] For any one of the query rewritten statements, retrieve one or more recall results in a preset knowledge base;

[0012] Based on the recall results, determine the correlation scores between the original query statement and each of the query rewritten statements.

[0013] In one embodiment, the step of determining the correlation scores between the original query statement and each of the query rewritten statements based on the recall results includes:

[0014] Through an expert scoring large model, determine the scoring scores between the original query statement and the recall results of each of the query rewritten statements, and calculate the average scoring score according to the number of the recall results;

[0015] Take the average scoring score as the correlation score between the original query statement and the query rewritten statement.

[0016] In one embodiment, before the step of determining the scoring scores between the original query statement and each of the recall results through an expert scoring large model and calculating the average scoring score according to the number of the recall results, further includes:

[0017] Obtain the historical recall results of historical original query statements and corresponding historical query rewritten statements;

[0018] Based on the historical recall results, perform scoring annotation on the rewriting quality of the historical query rewritten statements to obtain a scoring result;

[0019] Take the historical original query statements, the historical recall results of the corresponding historical query rewritten statements, and the scoring results as training data to train an expert scoring large model.

[0020] In one embodiment, the step of performing scoring annotation on the rewriting quality of the historical query rewritten statements based on the historical recall results to obtain a scoring result includes:

[0021] Determine the hit results between the historical original query statement and the historical recall results of the corresponding historical query rewritten statement, and perform scoring annotation on the rewriting quality of the historical query rewritten statement according to the hit results to obtain a scoring result; wherein, the higher the hit rate of the historical recall results of the historical query rewritten statement hitting the historical recall results of the historical original query statement, the higher the marked score.

[0022] In one embodiment, before the step of determining the scoring scores between the original query statement and each of the recall results through an expert scoring large model and calculating the average scoring score according to the number of the recall results, further includes:

[0023] Obtain training data, wherein the negative sample data in the training data includes question-and-answer pairs with low scores in user feedback, single-round question-and-answer, and / or multi-round question-and-answer during the historical question-and-answer process. The low user feedback score includes a low confidence score between the user's query request and the historical returned answer, a too long user stay time, a too high frequency of secondary inquiries, and / or the existence of negative feedback operations. The negative feedback operations include explicit bad review clicks and implicit cross-platform verification behaviors. The low single-round question-and-answer score means that the relevance score between the recall result of the single-round question-and-answer and the query statement is too low. The low multi-round question-and-answer score means that there are semantic conflicts and / or answer copy search behaviors between consecutive multi-round queries;

[0024] Based on the training data, train an expert scoring large model.

[0025] In one embodiment, before the step of determining the scoring scores between the original query statement and each of the recall results through the expert scoring large model and calculating the average scoring score according to the number of recall results, the following steps are further included:

[0026] Obtain the private domain data of the field where the query statement is located, and fine-tune the expert scoring large model based on the private domain data of the field where the query statement is located.

[0027] In one embodiment, after the step of using the original query statement, each of the query rewritten statements, and the relevance score as training data to train a rewrite device, the following steps are further included:

[0028] Obtain the first retrieval result of the original query statement and the second retrieval result of the target query statement;

[0029] Re-rank and refine the first retrieval result and the second retrieval result, and use the first preset number of retrieval results as the final retrieval results.

[0030] In addition, to achieve the above object, the present application also proposes a query statement rewriting device, which includes:

[0031] A preparation module, configured to obtain an original query statement and generate multiple query rewritten statements of the original query statement;

[0032] A relevance module, configured to determine the relevance scores between the original query statement and each of the query rewritten statements;

[0033] A training module, configured to use the original query statement, each of the query rewritten statements, and the relevance score as training data to train a rewrite device, so as to rewrite the original query statement through the rewrite device to obtain a target query statement.

[0034] In addition, to achieve the above object, the present application further provides a query statement rewriting device, which includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the query statement rewriting method as described above.

[0035] In addition, to achieve the above object, the present application further provides a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium, and when the computer program is executed by a processor, it implements the steps of the query statement rewriting method as described above.

[0036] In addition, to achieve the above object, the present application further provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps of the query statement rewriting method as described above.

[0037] One or more technical solutions proposed by the present application have at least the following technical effects:

[0038] In the present application, a new query statement rewriting method is proposed. The original query statement, multiple query rewritten statements generated from the original query statement, and the correlation scores between the original query statement and each query rewritten statement are used as training data to train a rewrite engine, and the rewrite engine rewrites the original query statement to obtain a target query statement.

[0039] Since the rewrite engine learns the correlation scores between the original query statement and each query rewritten statement during the training phase, it can directly use the rewrite engine for query rewriting, and when using the rewrite engine for query rewriting, it can output a target query statement with a high correlation score with the original query statement. Therefore, compared with the current existing query rewriting technologies, the rewriting efficiency and quality of the query statement can be improved. In the present application, a score-based reinforcement learning reward mechanism is designed to directly guide the rewrite engine to favor highly relevant rewrites. By introducing multiple rewritten statements and their correlation scores, the rewrite engine during training can learn multiple possible rewrite paths simultaneously, and use the correlation scores as weight signals to automatically identify high-value rewrite patterns. By automatically generating training data including low-score rewrites, that is, multiple query rewritten statements have different high and low correlation scores, the rewrite engine during training can explicitly learn to avoid generating invalid rewrites that deviate semantically or are redundant (such as wrongly rewriting "high-cost performance Bluetooth headsets" as "cheap headsets" and ignoring the implicit requirement of "sound quality"). BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The accompanying drawings herein are incorporated into the specification and form a part of the specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.

[0041] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0042] Figure 1 It is a schematic flowchart provided for the first embodiment of the query statement rewriting method of the present application;

[0043] Figure 2 It is an application schematic diagram of an application scenario of the query statement rewriting method of the present application;

[0044] Figure 3 It is a schematic flowchart provided for the second embodiment of the query statement rewriting method of the present application;

[0045] Figure 4 It is a schematic module structure diagram of the query statement rewriting device in the embodiment of the present application;

[0046] Figure 5 It is a schematic device structure diagram of the hardware operating environment involved in the query statement rewriting method in the embodiment of the present application.

[0047] The realization of the purpose, functional features and advantages of the present application will be further described with reference to the embodiments and the drawings. Specific Embodiments

[0048] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0049] The embodiment of the present application provides a query statement rewriting method, referring to Figure 1 , Figure 1 It is a schematic flowchart of the first embodiment of the query statement rewriting method of the present application.

[0050] In this embodiment, the query statement rewriting method includes steps S10 to S30:

[0051] Step S10, obtain the original query statement and generate multiple query rewritten statements of the original query statement;

[0052] In query rewriting, the original query statement refers to the statement form of the query request initially proposed by the user or the system. For example, in the search engine scenario, it is the statement expressing the user's initial demand information, or in the database query, it is the database query statement containing the initial query demand.

[0053] In this embodiment, after obtaining the original query statement, the language generation and analysis capabilities of large language models (LLMs) can be utilized to generate multiple query rewritten statements for the original query statement using the LLMs. Among them, the generated query rewritten statements can be subsequently used to train a rewrite engine and a human scoring large model. In this embodiment, the method for generating query rewritten statements for the original query statement is not limited.

[0054] Step S20: Determine the correlation scores between the original query statement and each query rewritten statement.

[0055] In this embodiment, the correlation score between the two can be determined by whether they are relevant at the semantic level, purpose level, lexical level, etc. between the original query statement and the rewritten query rewritten statement.

[0056] 1. At the semantic level: Whether the core concepts are retained. For example, if the original query statement is "The application of artificial intelligence in the medical field", and the rewritten query rewritten statement is "The use of artificial intelligence in the medical field", the two core concepts of "artificial intelligence" and "medical field" are both retained, indicating a high semantic correlation between the two, as they are both elaborating on the same theme; Whether the semantic logic is consistent. For example, if the original query statement is "Because it rained, the ground is wet", and the rewritten query rewritten statement is "The ground is wet because of the rain". In these two statements, the semantic logic of the causal relationship is the same. Although the sentence structures are different, the expressed logical relationship is the same, reflecting a high correlation.

[0057] 2. At the purpose level: Whether the information needs are the same. Suppose the original query statement is "Find methods to improve the battery life of electric vehicles", and the rewritten query rewritten statement is "How to improve the battery life of electric vehicles". The purpose of both statements is to obtain information about improving the battery life of electric vehicles, so there is a strong correlation between them, both pointing to the same information need.

[0058] 3. At the lexical level: Whether the keywords are similar. For example, if the original query statement is "Recommend popular tourist attractions", and the rewritten query rewritten statement is "Recommend popular tourist locations", the keyword "tourism" exists in both statements, "scenic spots" and "locations" are similar concepts, and "popular" and "recommend" are also retained. These similar keywords indicate a high correlation between the two statements.

[0059] In this embodiment, the specific implementation method for determining the correlation score between the original query statement and the rewritten query rewritten statement by whether they are relevant at the semantic level, purpose level, lexical level, etc. is not limited.

[0060] Step S30: Use the original query statement, each query rewritten statement, and the relevance score as training data to train a rewrite engine, so as to rewrite the original query statement through the rewrite engine to obtain the target query statement.

[0061] After preparing the original query statement, multiple query rewritten statements generated from the original query statement, and the relevance scores between the original query statement and each query rewritten statement, use the prepared data as training data to train a rewrite engine. This rewrite engine is different from the large language models (LLMs) mentioned above that are used to generate multiple query rewritten statements for the original query statement. It learns the relevance scores between the original query statement and each query rewritten statement during the training phase. Therefore, during the deployment and inference phase of the rewrite engine, it can output a target query statement with a relatively high relevance score to the original query statement without the need to use large language models (LLMs) with high inference costs, and ensures the rewrite efficiency and quality of the query statement. In one embodiment, the original model of the rewrite engine is usually a model that relies on natural language processing (NLP) technology, such as large language models (LLMs) or a specific rewrite model designed specifically for the rewrite task, such as RewriteLM.

[0062] In a feasible implementation manner, before training the rewrite engine, the original model of the rewrite engine can be preliminarily fine-tuned using a general open-source dataset (such as financial news, financial reports, etc.) and business annotation data (such as scenario data in legal regulations Q&A, customer service conversations, etc.) in a specific domain scenario, such as the financial scenario.

[0063] In an application scenario, after training the rewrite engine, the rewrite engine can be added to a RAG (Retrieval-Augmented Generation) system to obtain the query rewriting ability, so as to better handle problems such as unclear semantic expressions of users, text representation differences, and broad keywords, thereby improving the overall performance and user experience of the RAG system.

[0064] In a feasible implementation manner, after the step S30, it further includes:

[0065] Obtain the first retrieval result of the original query statement and the second retrieval result of the target query statement;

[0066] Re-rank and refine the first retrieval result and the second retrieval result, and use the first preset number of retrieval results as the final retrieval result.

[0067] Among them, the re-ranking and refinement (Re-ranking) of retrieval results refers to the process of finely sorting candidate items on the basis of preliminary recall and rough ranking in the multi-stage ranking process of information retrieval and recommendation systems.

[0068] After the training of the rewriter is completed, in one of the application scenarios of the rewriter, the obtained rewriter can be incorporated into the retrieval process. In one embodiment, after the original query statement is obtained, it is sent to the rewriter for rewriting to obtain the target query statement. Then, the original query statement and the target query statement are simultaneously put into the retriever to obtain the first retrieval result and the second retrieval result respectively corresponding to each of them. Next, the first retrieval result and the second retrieval result are re-ranked and refined, and the first preset number of retrieval results are used as the final retrieval results. In this way, the diversity and novelty of the rewriting results are further improved.

[0069] In one application scenario, referring to Figure 3 , the rewriter is used as a query rewriting module, and the rewritten query statement and the original statement are simultaneously put into retrievers such as a dense retriever and a BM25 (Best Matching 25) retriever. After obtaining multiple retrieval results, they are re-ranked and refined, and the first K documents obtained after refinement are used as the final recall results.

[0070] Based on the first embodiment of the present application, in the second embodiment of the present application, for the same or similar content as in the above-mentioned first embodiment, reference can be made to the above introduction and will not be repeated hereinafter. On this basis, please refer to Figure 3 , step S20 includes steps S201 to S202:

[0071] Step S201: For any query rewriting statement, retrieve one or more recall results in the preset knowledge base;

[0072] Step S202: Based on the recall results, determine the correlation scores between the original query statement and each query rewriting statement.

[0073] In this embodiment, a method for determining the correlation scores between the original query statement and each query rewriting statement different from that in the first embodiment is provided. Since the query statement is used to retrieve recall results in the preset knowledge base, therefore, in addition to directly determining the correlation scores between the two, the correlation scores between the two can also be indirectly determined by the recall results retrieved by the query statement in the preset knowledge base.

[0074] In a feasible implementation manner, step S202 includes steps S202A to S202B:

[0075] Step S202A: Through an expert scoring large model, determine the scoring scores between the original query statement and the recall results of each query rewriting statement, and calculate the average scoring score according to the number of recall results;

[0076] Step S202B: Use the average scoring score as the relevance score between the original query statement and the query rewritten statement.

[0077] In this embodiment, through the pre-trained expert scoring large model M r , determine the scoring scores between the original query statement and the recall results of each query rewritten statement, and use the average scoring score calculated according to the number of recall results as the relevance score S(q, q′) between the original query statement and the query rewritten statement:

[0078]

[0079] where q is the original query statement, q′ is the query rewritten statement, D′ is the recall result retrieved by putting q′ into the retriever, that is, D’ = {d1, d2, …, d k} is the recall result of the query rewritten statement q′, and |D′| is the number of recall results.

[0080] Among them, first obtain the scoring score M r (q, d′) between the original query statement and the recall result of each query rewritten statement. Then, determine the total scoring score ∑ d∈D′ M r (q, d′) between the original query statement and the recall results of all query rewritten statements. Finally, use the average scoring score to evaluate the overall quality of the rewrite, and use this average scoring score as the relevance score between the original query statement and the query rewritten statement.

[0081] In a feasible implementation manner, when using the expert scoring large model to determine the relevance score between the original query statement and the query rewritten statement, and further using the original query statement, each query rewritten statement, and the relevance score as training data, knowledge distillation is performed through reinforcement learning technology to train a rewrite engine. In this way, the rewrite engine can learn the scoring strategy of the expert scoring large model, thereby improving the rewrite quality of the rewrite engine and making the relevance score between the rewritten statement output by the rewrite engine and the original statement higher.

[0082] In one embodiment, the rewrite engine is trained through the reinforcement learning algorithm DPO (Direct Preference Optimization), so that the rewrite engine can optimize the query rewrite strategy according to the scoring of the scorer, that is, the expert scoring large model:

[0083]

[0084] This formula is the loss function of direct preference optimization (DPO), which is used to adjust the model generation strategy by comparing good and bad samples. Among them, is the loss function, represents the expectation of sampling triplets (query statement q, good sample q f from

[0085] the data distribution T, bad sample q g , β is the temperature parameter of DPO, which is used to control the intensity of contrast, M b ), M θ (q′ b |q) is the conditional probability that the model to be optimized generates the bad sample q′ b , and M ref (q′ b |q) is the conditional probability that the reference model (such as the pre-trained model) generates the bad sample. During the process of reinforcement learning, M θ can update the parameters according to the feedback of the quality of the rewritten text provided by the M ref model and continuously improve. Its core is to suppress the model from generating bad samples. By calculating the logarithm of the probability ratio of the bad samples generated by the model and the reference model and adjusting its weight β. If the probability that the current model M θ generates bad samples is lower than that of the reference model (i.e., the ratio < 1), the log result is negative, and σ(β·negative value) approaches 0. At this time, logσ(·) is negative, and after taking the negative sign, the loss is positive, and the model parameters are updated to further reduce the probability of bad samples. That is to say, this loss function adjusts the model to reduce the bad output by comparing the probabilities of the bad samples generated by the current model and the reference model. Finally, the rewritten text generator after reinforcement learning training can generate higher-quality query rewrites, significantly improving the retrieval performance.

[0086] In a feasible implementation manner, before the step S202A, it further includes:

[0087] Obtaining the historical original query statements and the historical recall results of the corresponding historical query rewritten statements;

[0088] Based on the historical recall results, scoring and annotating the rewriting quality of the historical query rewritten statements to obtain a scoring result;

[0089] Using the historical original query statements, the historical recall results of the corresponding historical query rewritten statements, and the scoring results as training data to train and obtain an expert scoring large model.

[0090] Before using the expert scoring large model, it is necessary to pre-train and obtain the expert scoring large model.

[0091] First, obtain the historical original query statement and the historical recall results corresponding to the historical query rewritten statement. Then, use the recall results to score and annotate the rewriting quality to obtain a scoring result, so as to use the labeled private domain data to match the relevance of the text blocks of the query statement and the recall results.

[0092] In a feasible implementation manner, the step of scoring and annotating the rewriting quality of the historical query rewritten statement based on the historical recall results to obtain a scoring result includes:

[0093] Determine the hit result between the historical original query statement and the historical recall results corresponding to the historical query rewritten statement, and score and annotate the rewriting quality of the historical query rewritten statement according to the hit result to obtain a scoring result; among them, the higher the hit rate of the historical recall results of the historical query rewritten statement hitting the historical recall results of the historical original query statement, the higher the annotated score.

[0094] In one embodiment, if the rewritten query statement can hit the text block of the recall result of the original query statement, it is annotated as 1, otherwise it is 0. In another embodiment, the labeled data is divided into multiple levels based on the hit result. For example, the text block of the recall result of the rewritten query statement that is most relevant to the text block of the recall result of the original query statement is scored 5, and the text block of the recall result that is least relevant is 0. In this way, it is ensured that the expert scoring large model to be trained can learn to distinguish relevant and irrelevant text content. Finally, use the historical original query statement, the historical recall results corresponding to the historical query rewritten statement, and the scoring result as training data to train an expert scoring large model.

[0095] In a feasible implementation manner, before the step S202A, it further includes:

[0096] Obtain training data, where the negative sample data in the training data includes question-and-answer pairs with too low scores in user feedback, single-round question-and-answer, and / or multi-round question-and-answer during the historical question-and-answer process. User feedback with too low scores includes low confidence scores between the user's query request and the historical returned answer, too long user stay time, too high frequency of secondary questions, and / or negative feedback operations. Negative feedback operations include explicit bad review clicks and implicit cross-platform verification behaviors. Too low score for single-round question-and-answer means too low correlation score between the recall result of the single-round question-and-answer and the query statement. Too low score for multi-round question-and-answer means there are semantic conflicts and / or answer copying and searching behaviors between consecutive multi-round queries;

[0097] Based on the training data, train an expert scoring large model.

[0098] Before training the expert scoring large model, negative sample data in the training data is constructed by continuously collecting low-quality question-and-answer pairs in the historical question-and-answer process, and dynamic data training is carried out with this. These dynamic data can help continuously update and improve the model to ensure its adaptation to the changing user needs and business scenarios. During the collection process of dynamic data, special attention is paid to those negative data with extremely low scores and extremely poor quality, which will be used in the subsequent query rewriting stage to improve the performance and quality of the rewrite engine.

[0099] Among them, the steps of collecting dynamic data include: 1. Through the constructed multi-dimensional user behavior buried point collection system, collect the question-and-answer pairs with too low user feedback scores in the historical question-and-answer process. Through real-time buried points, record the confidence score between the user query request and the historical returned answer, the user stay time, the frequency of secondary follow-up questions, and the negative feedback operation, and the negative feedback operation includes explicit bad review clicks and implicit cross-platform verification behaviors; 2. Through single-round low-quality question-and-answer data research and judgment, collect the question-and-answer pairs with too low single-round question-and-answer scores in the historical question-and-answer process. The average value of the correlation scores between the recall results and the query statements can be given by the online retrieval and fine-ranking method as the evaluation score of the retrieval quality. For the question-and-answer historical data with an evaluation score ≤ 0.4, a low-quality question-and-answer warning is automatically triggered; 3. Through multi-round low-quality question-and-answer data research and judgment, collect the question-and-answer pairs with too low multi-round question-and-answer scores in the historical question-and-answer process. When using the large model to judge whether there is a semantic conflict between consecutive queries (such as the first question "account opening process" and the second question "way to purchase without opening an account") or answer copy search behavior, a low-quality answer warning is automatically triggered.

[0100] In a feasible implementation manner, before the step S202A, it also includes:

[0101] Obtain the private domain data of the field where the query statement is located, and fine-tune the expert scoring large model based on the private domain data of the field where the query statement is located.

[0102] In addition, the rewriting effects of existing query rewriting technologies generally fail to meet the expectations when facing the terms and professional knowledge in the private domain data.

[0103] Therefore, in this embodiment, in the training stage of the expert scoring large model, the method of large model instruction fine-tuning SFT (Supervised Fine-Tuning) can also be adopted to fine-tune the expert scoring large model by using the private domain data of the field where the query statement is located. For example, a small amount of financial private domain data is used to fine-tune the expert scoring large model to meet the needs of the financial field. And a linear layer is added after the output layer of the expert scoring large model to perform score prediction. Thus, train the expert scoring large model to score the rewriting quality by using the semantic relevance between the query statement and the recall result.

[0104] In an application scenario of this application, the fine-tuned expert scoring large model is used as an expert to score the recall results, and this score is used to evaluate the quality of the rewritten query statements and as the basis for training the rewritter. First, the large language model (LLMs) is used to generate multiple query rewrite statements, and these rewrite statements are scored by the expert scoring large model to obtain their relevance to the original query statement. Then, by sorting these scoring results, the optimal query rewrite is selected. Next, these sorting results are used as training data, and knowledge distillation is performed on a smaller rewritter model through reinforcement learning technology to improve its query rewrite ability.

[0105] Therefore, an innovative query statement rewriting mechanism is proposed. This mechanism can effectively improve semantic understanding and retrieval performance in the face of the deficiencies of existing retrieval channels when processing private domain data, especially in the face of interference factors such as long and short text representations and professional term abbreviations. Specifically, with only a small amount of labeled corpus, a text scoring and evaluation model, that is, the expert scoring large model, is formed by fine-tuning the large model, and a rewritter is trained through knowledge distillation in combination with reinforcement learning technology. This method not only has a low labeling cost and small hardware requirements, but also can significantly improve the retrieval performance, especially for the retrieval effect of private domain data. It significantly improves the quality of query rewriting and retrieval performance, and ultimately improves the retrieval effect in specific fields such as the financial field.

[0106] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the query statement rewriting method of this application. Based on this technical concept, more forms of simple transformations are within the protection scope of this application.

[0107] The embodiment of this application also provides a query statement rewriting device. Please refer to Figure 4 The query statement rewriting device includes:

[0108] A preparation module 10, configured to obtain an original query statement and generate multiple query rewrite statements of the original query statement;

[0109] A relevance module 20, configured to determine the relevance scores between the original query statement and each query rewrite statement;

[0110] A training module 30, configured to use the original query statement, each query rewrite statement, and the relevance scores as training data to train a rewritter, so as to rewrite the original query statement through the rewritter to obtain a target query statement.

[0111] In an embodiment, the relevance module 20 is further configured to:

[0112] For any query rewrite statement, retrieve one or more recall results in a preset knowledge base;

[0113] Based on the recall results, determine the correlation scores between the original query statement and each query rewritten statement.

[0114] In one embodiment, the correlation module 20 is further configured to:

[0115] Through an expert scoring large model, determine the scoring scores between the original query statement and the recall results of each query rewritten statement, and calculate the average scoring score according to the number of recall results;

[0116] Use the average scoring score as the correlation score between the original query statement and the query rewritten statement.

[0117] In one embodiment, the correlation module 20 is further configured to:

[0118] Before the step of determining the scoring scores between the original query statement and each recall result through the expert scoring large model and calculating the average scoring score according to the number of recall results:

[0119] Obtain the historical recall results of the historical original query statements and the corresponding historical query rewritten statements;

[0120] Based on the historical recall results, score and label the rewriting quality of the historical query rewritten statements to obtain a scoring result;

[0121] Use the historical original query statements, the historical recall results of the corresponding historical query rewritten statements, and the scoring results as training data to train an expert scoring large model.

[0122] In one embodiment, the correlation module 20 is further configured to:

[0123] Determine the hit results between the historical original query statements and the historical recall results of the corresponding historical query rewritten statements, and score and label the rewriting quality of the historical query rewritten statements according to the hit results to obtain a scoring result; wherein, the higher the hit rate of the historical recall results of the historical query rewritten statements hitting the historical recall results of the historical original query statements, the higher the labeled score.

[0124] In one embodiment, the correlation module 20 is further configured to:

[0125] Before the step of determining the scoring scores between the original query statement and each recall result through the expert scoring large model and calculating the average scoring score according to the number of recall results:

[0126] Obtain training data, where the negative sample data in the training data includes question-and-answer pairs with low scores in user feedback, single-round question-and-answer, and / or multi-round question-and-answer during the historical question-and-answer process. Low user feedback scores include low confidence scores between the user's query request and the historical returned answer, long user stay times, high frequencies of secondary inquiries, and / or negative feedback operations. Negative feedback operations include explicit negative review clicks and implicit cross-platform verification behaviors. Low single-round question-and-answer scores mean that the relevance score between the recall result of the single-round question-and-answer and the query statement is too low. Low multi-round question-and-answer scores mean that there are semantic conflicts between consecutive multi-round queries and / or there are answer copying and searching behaviors.

[0127] Based on the training data, train an expert scoring large model.

[0128] In one embodiment, the relevance module 20 is further configured to:

[0129] Before the step of determining the scoring scores between the original query statement and each recall result through the expert scoring large model and calculating the average scoring score according to the number of recall results:

[0130] Obtain the private domain data of the field where the query statement is located, and fine-tune the expert scoring large model based on the private domain data of the field where the query statement is located.

[0131] In one embodiment, the query statement rewriting device further includes an application module, configured to:

[0132] After the step of using the original query statement, each query rewritten statement, and the relevance score as training data to train a rewrite device:

[0133] Obtain the first retrieval result of the original query statement and the second retrieval result of the target query statement;

[0134] Re-rank and refine the first retrieval result and the second retrieval result, and use the first preset number of retrieval results as the final retrieval results.

[0135] The query statement rewriting device provided by the present application adopts the query statement rewriting method in the above embodiment, and can solve the technical problems of low rewriting efficiency and poor rewriting quality of the current query statement. Compared with the prior art, the beneficial effects of the query statement rewriting device provided by the present application are the same as those of the query statement rewriting method provided by the above embodiment, and other technical features in the query statement rewriting device are the same as those disclosed in the above embodiment method, and will not be elaborated here.

[0136] It should be noted that the execution subject of each method embodiment of the present application can be a computing service device with data processing, network communication, and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device, a query statement rewriting device, etc. that can implement the above functions.

[0137] An embodiment of the present application provides a query statement rewriting device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the query statement rewriting method in Embodiment 1 above.

[0138] Reference is made below to Figure 5 , which shows a schematic structural diagram of a query statement rewriting device suitable for implementing the embodiments of the present application. The query statement rewriting device in the embodiments of the present application may include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), in-vehicle terminals (such as in-vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The query statement rewriting device shown is only an example and should not impose any limitations on the functions and usage scope of the embodiments of the present application.

[0139] As Figure 5As shown, the query statement rewriting device may include a processing device 1001 (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM: Read Only Memory) 1002 or the program loaded from the storage device 1003 into the random access memory (RAM: Random Access Memory) 1004. In the RAM 1004, various programs and data required for the operation of the query statement rewriting device are also stored. The processing device 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. The input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the query statement rewriting device to communicate with other devices wirelessly or wiredly to exchange data. Although the figure shows a query statement rewriting device with various systems, it should be understood that it is not required to implement or have all the shown systems. More or fewer systems can be implemented or had alternatively.

[0140] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device, or installed from the storage device 1003, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above functions defined in the methods of the embodiments disclosed in the present application are executed.

[0141] The query statement rewriting device provided by the present application adopts the query statement rewriting method in the above embodiments, and can solve the technical problems of low rewriting efficiency and poor rewriting quality of the current query statements. Compared with the prior art, the beneficial effects of the query statement rewriting device provided by the present application are the same as those of the query statement rewriting method provided by the above embodiments, and the other technical features in the query statement rewriting device are the same as those disclosed in the method of the previous embodiment, and will not be elaborated here.

[0142] It should be understood that each part disclosed in this application can be implemented by hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in a suitable manner in any one or more embodiments or examples.

[0143] As described above, the above is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art within the technical scope disclosed in this application can easily think of changes or substitutions, which should all be covered within the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

[0144] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., computer programs) stored thereon, and the computer-readable program instructions are used to execute the query statement rewriting method in the above embodiments.

[0145] The computer-readable storage medium provided by this application can be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) or flash memory, optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this embodiment, the computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or combined with an instruction execution system or device. The program code contained on the computer-readable storage medium can be transmitted by any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0146] The above computer-readable storage medium can be included in the query statement rewriting device; it can also exist separately without being assembled into the query statement rewriting device.

[0147] The above computer-readable storage medium carries one or more programs, which, when executed by a query statement rewriting device, cause the query statement rewriting device to: obtain an original query statement and generate multiple query rewrite statements for the original query statement; determine the correlation scores between the original query statement and each query rewrite statement; use the original query statement, each query rewrite statement, and the correlation scores as training data to train a rewrite device, so as to rewrite the original query statement through the rewrite device to obtain a target query statement.

[0148] Computer program code for performing the operations of the present application can be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, and C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, by using an Internet service provider to connect through the Internet).

[0149] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0150] The modules described in the embodiments of the present application can be implemented in software or in hardware. Among them, the name of the module does not constitute a limitation on the unit itself in some cases.

[0151] The readable storage medium provided by this application is a computer-readable storage medium. The computer-readable storage medium stores computer-readable program instructions (i.e., computer programs) for executing the above query statement rewriting method, which can solve the technical problems of low rewriting efficiency and poor rewriting quality of the current query statement. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by this application are the same as those of the query statement rewriting method provided by the above embodiments, and will not be elaborated here.

[0152] This application also provides a computer program product, including a computer program, and the steps of the above query statement rewriting method are implemented when the computer program is executed by a processor.

[0153] The computer program product provided by this application can solve the technical problems of low rewriting efficiency and poor rewriting quality of the current query statement. Compared with the prior art, the beneficial effects of the computer program product provided by this application are the same as those of the query statement rewriting method provided by the above embodiments, and will not be elaborated here.

[0154] The above are only partial embodiments of this application, and thus do not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application by using the content of the specification and drawings of this application, or directly / indirectly applied in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A query statement rewriting method, characterized in that: The query statement rewriting method comprises: Obtaining an original query statement, and generating a plurality of query rewriting statements of the original query statement; Determining a correlation score between the original query statement and each of the query rewrite statements; The original query statement, each of the query rewriting statements and the relevance score are used as training data to train a rewriter, so as to rewrite the original query statement through the rewriter to obtain a target query statement.

2. The query statement rewriting method according to claim 1, characterized in that: The step of determining the correlation score between the original query statement and each of the query rewriting statements comprises: For any of the query rewriting statements, one or more recall results are retrieved from a preset knowledge base; Based on the recall result, a correlation score between the original query statement and each of the query rewriting statements is determined.

3. The query statement rewriting method according to claim 2, characterized in that: The step of determining the relevance score between the original query statement and each of the query rewriting statements based on the recall result comprises: Determine the score between the original query statement and the recall results of each of the query rewriting statements through the expert scoring model, and calculate the average score according to the number of the recall results; The average rating score is used as the correlation score between the original query statement and the rewritten query statement.

4. The query statement rewriting method according to claim 3, characterized in that: Before the step of determining the score between the original query statement and each of the recalled results by using the expert scoring model and calculating the average score according to the number of the recalled results, the step further includes: Obtain the historical original query statements and the historical recall results of the corresponding historical query rewrite statements; Based on the historical recall result, scoring and marking the rewriting quality of the historical query rewriting statement to obtain a scoring result; The historical original query statements, the historical recall results of the corresponding historical query rewritten statements and the scoring results are used as training data to train a large expert scoring model.

5. The query statement rewriting method according to claim 4, characterized in that: The step of scoring and marking the rewriting quality of the historical query rewriting statement based on the historical recall result to obtain the scoring result comprises: Determine the hit result between the historical original query statement and the historical recall result of the corresponding historical query rewrite statement, score and annotate the rewrite quality of the historical query rewrite statement according to the hit result, and obtain a scoring result; wherein, the higher the hit rate of the historical recall result of the historical query rewrite statement to the historical recall result of the historical original query statement, the higher the annotated score.

6. The query statement rewriting method according to claim 3, characterized in that: Before the step of determining the score between the original query statement and each of the recalled results by using the expert scoring model and calculating the average score according to the number of the recalled results, the step further includes: Acquire training data, wherein the negative sample data in the training data includes question-answer pairs with too low user feedback, single-round question-answering, and / or multi-round question-answering scores in the historical question-answering process, wherein the user feedback score is too low, including the confidence score between the user query request and the historical returned answer is too low, the user stay time is too long, the frequency of secondary questioning is too high, and / or there is a negative feedback operation, wherein the negative feedback operation includes explicit negative review clicks and implicit cross-platform verification behaviors, the single-round question-answering score is too low, and the correlation score between the recall result of the single-round question-answering and the query statement is too low, and the multi-round question-answering score is too low, and there is a semantic conflict between multiple rounds of continuous queries and / or there is a copy answer search behavior; Based on the training data, a large expert scoring model is trained.

7. The query statement rewriting method according to claim 3, characterized in that: The step of determining the score between the original query statement and each of the recalled results by using the expert scoring model, and calculating the average score according to the number of the recalled results, further includes: Obtain private domain data in the field where the query statement is located, and fine-tune the expert scoring model based on the private domain data in the field where the query statement is located.

8. The query statement rewriting method according to claim 1, characterized in that: After the step of using the original query statement, each of the query rewriting statements and the relevance score as training data to train a rewriter, the following step further comprises: Obtaining a first search result of the original query statement and a second search result of the target query statement; The first search result and the second search result are re-ranked and the first preset number of search results are used as the final search results.

9. A query statement rewriting device, characterized in that: The query statement rewriting device comprises: A preparation module, used to obtain an original query statement and generate a plurality of query rewriting statements of the original query statement; A relevance module, configured to determine a relevance score between the original query statement and each of the query rewrite statements; A training module is used to use the original query statement, each of the query rewriting statements and the relevance score as training data to train a rewriter, so as to rewrite the original query statement through the rewriter to obtain a target query statement.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the query statement rewriting method according to any one of claims 1 to 6 are implemented.

Images