Query extension method, computer equipment, storage medium and program product

Abstract

The invention relates to the technical field of natural language processing, and discloses a query expansion method, computer equipment, a storage medium and a program product. Performing question-answer extension, title extension and keyword extension on the initial query question to generate a question-answer extension result, a title extension result and a keyword extension result; and generating an abstract based on the question and answer extension result, the title extension result and the keyword extension result to determine a target query extension question. According to the method, query expansion is abstracted into three sub-tasks with different granularities, the three sub-tasks are respectively used for acquiring document-level, sentence-level and word-level key information, finally, the three results are integrated, a final target query expansion problem is output, the final target query expansion problem can be used as expansion and supplement of an initial query problem, and the original intention of a user is reflected more accurately.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a query expansion method, a computer device, a storage medium and a program product. Background Art

[0002] Large Language Model (LLM) is a key technology in the field of natural language processing. Its base model is a deep learning model pre-trained on large-scale text data. In recent years, with the rapid development of deep learning technology, large language models are applicable to a variety of natural language processing tasks such as text generation, question-answering systems, and sentiment analysis. With the development of deep learning technology, especially the widespread application of large pre-trained models, query expansion technology has also undergone significant changes. Now, generative query expansion technology based on large models has become one of the mainstream trends. This type of technology can not only use a rich corpus to generate high-quality expansion words, but also better understand and adapt to the user's query intent.

[0003] Current query expansion technology mainly relies on pre-trained large language models and single-step large model output for generative expansion. This method simplifies the query expansion process, but the quality of query expansion is largely limited by the performance of the base large model in specific functional areas. Summary of the invention

[0004] In view of this, the present invention provides a query expansion method, a computer device, a storage medium and a program product to solve the problem that the current expanded query method is limited by the field.

[0005] In a first aspect, the present invention provides a query expansion method, the method comprising: Get the initial query question; Performing question and answer expansion, title expansion and keyword expansion on the initial query question respectively to generate question and answer expansion results, title expansion results and keyword expansion results, wherein the question and answer expansion results at least include answers to questions related to the initial query question, the title expansion results at least include query titles related to the initial query question, and the keyword expansion results at least include keywords related to the initial query question; A summary is generated based on the question and answer expansion result, the title expansion result and the keyword expansion result to determine a target query expansion question.

[0006] In an optional implementation, the question and answer expansion, title expansion and keyword expansion are performed on the initial query question to generate a question and answer expansion result, a title expansion result and a keyword expansion result, including: Preprocessing the initial query question to generate a query sequence; Inputting the query sequence into the question and answer expansion submodel to determine a question and answer expansion result based on the question and answer expansion submodel; inputting the query sequence into a title expansion submodel to determine a title expansion result based on the title expansion submodel; The query sequence is input into a keyword expansion sub-model to determine a keyword expansion result based on the keyword expansion sub-model.

[0007] In an optional implementation, the preprocessing of the initial query question to generate a query sequence includes: Performing word segmentation processing on the initial query question to obtain an initial vocabulary; The initial vocabulary is filtered to remove meaningless words in the initial vocabulary and generate a query sequence.

[0008] In an optional implementation, inputting the query sequence into the question and answer expansion submodel to determine the question and answer expansion result based on the question and answer expansion submodel includes: Inputting the query sequence and the preset question and answer expansion prefix into the question and answer expansion sub-model, wherein the question and answer expansion sub-model includes a question and answer expansion output layer; Autoregressive decoding is performed based on the question and answer expansion output layer to output the question and answer expansion result.

[0009] In an optional implementation, the parameters of the question and answer expansion output layer and the embedding layer of the preset question and answer expansion prefix constitute a fine-tuning parameter set.

[0010] In an optional embodiment, the method further includes: A first loss function is determined based on the question and answer expansion result, a preset question reference answer sequence, and the fine-tuning parameter set, so as to adjust the fine-tuning parameter set based on the first loss function.

[0011] In an optional implementation, inputting the query sequence into a title expansion submodel to determine a title expansion result based on the title expansion submodel includes: The query sequence and a preset title extension prefix are input into the title extension sub-model, and the title extension sub-model outputs the title extension result based on autoregressive decoding.

[0012] In an optional embodiment, the method further includes: Determine a second loss function based on the title expansion result, the query sequence, and a preset reference title; The preset title extension prefix and the output layer parameters of the title extension sub-model are optimized according to the second loss function.

[0013] In an optional implementation, inputting the query sequence into a keyword expansion submodel to determine a keyword expansion result based on the keyword expansion submodel includes: The query sequence and a preset keyword expansion prefix are input into the keyword expansion sub-model, and the keyword expansion sub-model outputs the keyword expansion result based on autoregressive decoding.

[0014] In an optional embodiment, the method further includes: Determine a third loss function based on the keywords in the keyword expansion result, the query sequence, and a preset reference keyword sequence, wherein the third loss function is a cross entropy loss; The preset keyword extension prefix and the output layer parameters of the keyword extension sub-model are optimized according to the third loss function.

[0015] In an optional implementation, the generating a summary based on the question and answer expansion result, the title expansion result and the keyword expansion result to determine the target query expansion question includes: Performing structured processing on the question and answer expansion results, the title expansion results and the keyword expansion results respectively; Based on the summary generation model, summary generation processing is performed on the result after the structured processing, and a target query expansion question is output.

[0016] In an optional embodiment, the method further includes: Obtain annotated reference query expansion documents; Generate a training data set based on the annotated reference query document, the structured processing result of the question and answer expansion result, the structured processing result of the title expansion result, and the structured processing result of the keyword expansion result; A cross entropy loss function is determined based on the training data set, and the summary generation model is optimized based on the cross entropy loss function.

[0017] The query expansion method provided in this embodiment includes obtaining an initial query question; performing question and answer expansion, title expansion and keyword expansion on the initial query question to generate question and answer expansion results, title expansion results and keyword expansion results, wherein the question and answer expansion results at least include answers to questions related to the initial query question, the title expansion results at least include query titles related to the initial query question, and the keyword expansion results at least include keywords related to the initial query question; and performing summary generation based on the question and answer expansion results, the title expansion results and the keyword expansion results to determine a target query expansion question. This method abstracts query expansion into three subtasks of different granularities, wherein question and answer expansion, title expansion and keyword expansion are used to obtain key information at the document level, sentence level and word level respectively, and finally the three results are integrated to output the final target query expansion question, which can be used as an extension and supplement to the initial query question and more accurately reflect the original intention of the user.

[0018] In a second aspect, the present invention provides a query expansion device, comprising: A question acquisition module is used to obtain the initial query question; A question expansion module, used to perform question and answer expansion, title expansion and keyword expansion on the initial query question, so as to generate a question and answer expansion result, a title expansion result and a keyword expansion result, wherein the question and answer expansion result at least includes the answer to the question related to the initial query question, the title expansion result at least includes the query title related to the initial query question, and the keyword expansion result at least includes the keyword related to the initial query question; A summary generation module is used to generate a summary based on the question and answer expansion result, the title expansion result and the keyword expansion result to determine the target query expansion question.

[0019] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the query expansion method of the above-mentioned first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0020] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the query expansion method of the first aspect or any corresponding embodiment thereof.

[0021] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions, wherein the computer instructions are used to enable a computer to execute the query expansion method of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0023] Figure 1 is a flow chart of a query expansion method according to an embodiment of the present invention; Figure 2 is a schematic diagram of a processing flow of a question-answer expansion sub-model according to an embodiment of the present invention; Figure 3 is a schematic diagram of a processing flow of a title extension sub-model according to an embodiment of the present invention; Figure 4 is a schematic diagram of a processing flow of a keyword expansion sub-model according to an embodiment of the present invention; Figure 5 is a schematic diagram of a processing flow of a summary generation model according to an embodiment of the present invention; Figure 6 is a schematic diagram of a query expansion method according to an embodiment of the present invention; Figure 7 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0025] Large language models are a key technology in the field of natural language processing. At present, generative query expansion technology based on large models has become a mainstream trend. This type of technology can generate high-quality extension words using rich corpora and can better understand and adapt to the user's query intent. Generative query expansion technology uses large pre-trained models to generate new query words or phrases to expand the original query. In the retrieval generation enhancement framework, generative query expansion technology can significantly enhance the performance of the retrieval stage. By using large models to generate relevant extension words in the pre-retrieval stage, the retrieval scope can be expanded and the relevance and diversity of retrieval results can be improved. The retrieval generation enhancement framework includes dynamic context expansion, iterative query expansion, and a generation and retrieval cycle. Dynamic context expansion is used to dynamically generate extension words based on the specific content and context information of the user's query; iterative query expansion is used to iteratively update the query based on user feedback or new information in the interaction process to obtain more accurate retrieval results; the generation and retrieval cycle means that the generated extended query can be used to further retrieve documents, and then a new query is generated again based on the retrieval results, forming a cyclic optimization process.

[0026] Existing query expansion methods usually use a single output of a large language model as the expansion result. This method relies on the performance of the model, which directly affects the quality of the expansion result. If the model has insufficient knowledge or inaccurate understanding of a specific field, the generated expansion words or phrases may not match the user's intent, resulting in a decrease in the accuracy of the search results. Based on this, an embodiment of the present invention provides a query expansion method. According to an embodiment of the present invention, a query expansion method embodiment is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0027] In this embodiment, a query expansion method is provided, which can be used in terminals such as computers and tablet computers. Figure 1 is a flowchart of a query expansion method according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps: Step S101, obtaining an initial query question.

[0028] First, the initial query question input by the user is obtained. The initial query question can be a sequence of multiple content words. Content words are words in a language that can express exact meanings and have real meanings. They are the basic units of a sentence and are used to describe the specific characteristics, properties and relationships of things, such as "processor, memory, hard disk, motherboard". The initial query question can also be a complete sentence, such as "What are the components of desktop computer hardware?"

[0029] In some optional implementations, the initial query question input by the user may also include special symbols, meaningless words, etc. in addition to content words.

[0030] Step S102, performing question and answer expansion, title expansion and keyword expansion on the initial query question to generate question and answer expansion results, title expansion results and keyword expansion results.

[0031] The initial query question is respectively subjected to question and answer expansion, title expansion and keyword expansion. Specifically, the initial query question can be processed based on the corresponding language sub-model, so as to obtain question and answer expansion results, title expansion results and keyword expansion results respectively. Among them, the question and answer expansion results at least include answers to questions related to the initial query question. The expected generation of the answer to the question is a long text of no less than a preset number of words, which contains as much detailed information related to the input query as possible and belongs to the document level. Before processing, a sub-model for performing the question and answer expansion task can be pre-defined, and the sub-model can be trained using pre-labeled questions and reference answers as a training data set. After executing this question and answer expansion, the sub-model can continue to be optimized based on the question and answer expansion results output this time.

[0032] The title expansion result at least includes the query title related to the initial query question. Specifically, a sentence related to the initial query question that is condensed within a preset number of words can be generated, which belongs to the sentence level. Before processing, a sub-model for performing the title expansion task can be pre-defined, and the sub-model can be trained using a combination of pre-annotated descriptions and reference titles as a training data set. After executing this question and answer, the sub-model can continue to be optimized based on the title expansion result output this time.

[0033] The keyword expansion results at least include keywords related to the initial query question. Specifically, a discrete keyword sequence can be generated, which contains one or more keywords and belongs to the word level. In the processing, a sub-model for performing the keyword expansion task can be pre-defined, and the pre-annotated query question and reference keyword combination can be used as a training data set to train the sub-model. After executing this question and answer, the sub-model can continue to be optimized based on the keyword expansion results output this time.

[0034] Step S103, generating a summary based on the question and answer expansion results, the title expansion results and the keyword expansion results to determine the target query expansion question.

[0035] A predefined summary generation model is used to generate summaries for question and answer expansion results, title expansion results, and keyword expansion results. The three results are structured and spliced ​​through prompt words and tags, and are input into the summary generation model as a multi-granularity structured information. The summary generation model performs the summary generation task and outputs the target query expansion question. The target query expansion question is an extended document that integrates the three information granularities of question and answer expansion results, title expansion results, and keyword expansion results. It can be used as an extension and supplement to the initial query question and more accurately reflects the user's original intention.

[0036] Optionally, when generating summaries, context-aware extended queries can be added to record historical query questions previously entered by the user, and the final target query extended questions can be generated by combining the historical query questions. The user's historical query questions may contain data related to the current query, so combining the historical query questions can improve the ability to understand the user's intentions to a certain extent, making the final generated results more in line with the user's expectations and needs.

[0037] The query expansion method provided in this embodiment includes obtaining an initial query question; performing question and answer expansion, title expansion and keyword expansion on the initial query question to generate question and answer expansion results, title expansion results and keyword expansion results, wherein the question and answer expansion results at least include answers to questions related to the initial query question, the title expansion results at least include query titles related to the initial query question, and the keyword expansion results at least include keywords related to the initial query question; and performing summary generation based on the question and answer expansion results, the title expansion results and the keyword expansion results to determine a target query expansion question. This method abstracts query expansion into three subtasks of different granularities, wherein question and answer expansion, title expansion and keyword expansion are used to obtain key information at the document level, sentence level and word level respectively, and finally the three results are integrated to output the final target query expansion question, which can be used as an extension and supplement to the initial query question and more accurately reflect the original intention of the user.

[0038] In this embodiment, a query expansion method is provided, which includes the following steps: Step S201, obtaining an initial query question.

[0039] For details, please see Figure 1 Step S101 of the illustrated embodiment will not be described in detail here.

[0040] Step S202, performing question and answer expansion, title expansion and keyword expansion on the initial query question to generate question and answer expansion results, title expansion results and keyword expansion results.

[0041] Specifically, step S202 includes: Step S2021, pre-processing the initial query question to generate a query sequence.

[0042] Before the formal query expansion process, the initial query problem needs to be preprocessed to generate a query sequence that can be used in subsequent steps.

[0043] In some optional implementations, the above step S2021 includes: Step S301, performing word segmentation processing on the initial query question to obtain an initial vocabulary; Step S302: Filter the initial vocabulary to remove meaningless words in the initial vocabulary and generate a query sequence.

[0044] Since there is no guarantee that the initial query questions entered by the user are all the expected noise-free text, word segmentation and filtering are first required to improve the signal-to-noise ratio of the input data.

[0045] First, the initial query question text is segmented into independent vocabulary units based on word segmentation technology to obtain one or more initial vocabulary. Since there may be meaningless words in the text, including special symbols, letter combinations, etc. Specifically, special symbols without meaning such as "#%&*" and meaningless letter combinations such as "aaaa, asxasdq2dadw" can be filtered out through string operations to remove meaningless words in the initial vocabulary. The remaining content words after filtering constitute the query sequence, and the query sequence only contains content words.

[0046] The remaining content words constitute the query sequence. This step can reduce noise and improve the signal-to-noise ratio of subsequent model input data.

[0047] Step S2022, inputting the query sequence into the question and answer expansion sub-model to determine the question and answer expansion result based on the question and answer expansion sub-model; Step S2023, inputting the query sequence into the title expansion sub-model to determine the title expansion result based on the title expansion sub-model; Step S2024: input the query sequence into the keyword expansion sub-model to determine the keyword expansion result based on the keyword expansion sub-model.

[0048] The question and answer expansion submodel, title expansion submodel and keyword expansion submodel can be used as three subtask modules. Each submodel has corresponding preset parameters. The preset parameters are predefined prefixes, which can be considered as predefined vectors, which can be used to guide the model to generate key components related to the input. Before running the entire query expansion method, the parameters of each subtask model can be adjusted separately when optimizing each subtask model, and the parameters of the unique layer of each subtask model can be saved. At runtime, the base model is first loaded, and then the parameters and prompt words of the corresponding module are loaded. The processed query sequence is input into the corresponding submodel respectively, and the results generated by each submodel are obtained, including question and answer expansion results, title expansion results and keyword expansion results.

[0049] Step S203, generating a summary based on the question and answer expansion results, the title expansion results and the keyword expansion results to determine the target query expansion question.

[0050] For details, please see Figure 1 Step S103 of the illustrated embodiment will not be described in detail here.

[0051] In some optional embodiments, the above step S2022 includes: inputting the query sequence and the preset question and answer expansion prefix into the question and answer expansion sub-model; wherein the question and answer expansion sub-model includes a question and answer expansion output layer; and performing autoregressive decoding based on the question and answer expansion output layer to output the question and answer expansion result.

[0052] The processing flow of the question-answer expansion sub-model is as follows: Figure 2 As shown, the preset question-answer expansion prefix is ​​a predefined vector, that is, the question-answer expansion prefix Prompt in the figure, and the input question q is the query sequence. Tokenizer is a tool that divides text data into smaller units, which can convert continuous character sequences in natural language into discrete units that the model can understand and process.

[0053] The query sequence is input into the tokenizer of the Q&A expansion sub-model to obtain the input vector of the Q&A expansion task. The input vector of the Q&A expansion task is used as the input of the base model. At the same time, the preset Q&A expansion prefix vector is input into the base model as the initial state (past_key_values ​​attribute). The Q&A expansion sub-model includes a Q&A expansion output layer. The base model uses a trainable Q&A expansion output layer when outputting. The Q&A expansion output layer is a 2-layer linear network with Softmax activation. The model outputs the Q&A expansion result according to the input question through autoregressive decoding. The past_key_values ​​attribute is the state of the key and value synchronized before when processing sequence data. The Tokenizer only processes the filtered query text, maps it to the input word id sequence, and then inputs the id sequence into the base model. The first layer of the base model is a nn.Embedding layer (embedding layer) that maps the id sequence to a vector as the input_embed (input embedding) of the base model.

[0054] In some optional implementations, the parameters of the question and answer expansion output layer and the embedding layer of the preset question and answer expansion prefix constitute a fine-tuning parameter set. Further, the method also includes: determining a first loss function based on the question and answer expansion result, the preset question reference answer sequence and the fine-tuning parameter set, so as to adjust the fine-tuning parameter set based on the first loss function.

[0055] When training and optimizing the question-answering extension sub-model, define the question-answering dataset related to the question-answering extension sub-model, including pre-labeled questions and reference answer combinations, for example, ,in Represents a pre-annotated question and reference answer combination. After obtaining the question and answer expansion results, the question answer in the question and answer expansion results is and reference answers to preset questions According to the autoregressive method, the negative log-likelihood loss is calculated. The first loss function as follows:

[0056] in, represents the decoded output of the step model at time t, Represents a sequence of reference answers to preset questions. Represents a set of fine-tuning parameters, Represents a query sequence. The fine-tuning parameter set includes parameters that need to be updated by gradients, which are specifically composed of the embedding layer of the preset question and answer expansion prefix (i.e., the embedding corresponding to the question and answer expansion prefix Prompt) and the parameters of the question and answer expansion output layer. The base model parameters are not affected.

[0057] That is, when optimizing the model, only the parameters of the embedding layer and the question and answer expansion output layer with the preset question and answer expansion prefix are adjusted.

[0058] In some optional implementations, the above step S2023 includes: inputting the query sequence and the preset title extension prefix into the title extension sub-model, and the title extension sub-model outputs the title extension result based on autoregressive decoding.

[0059] The processing flow of the title extension sub-model is as follows Figure 3 As shown in the figure, the preset title extension prefix is ​​a predefined vector, that is, the title extension prefix Prompt in the figure, and the input query c is the query sequence. When the Tokenizer processes the embedding (embedding layer) corresponding to the title extension prefix Prompt, it will map it to a trainable embedding. The query sequence is input to the tokenizer of the title extension sub-model to obtain the input vector of the title extension task, and the input vector of the title extension task is used as the input of the base model. The title extension output layer of the model outputs the title extension result through the autoregressive decoding method.

[0060] In some optional embodiments, the method further includes: determining a second loss function based on the title expansion result, the query sequence, and the preset reference title; and optimizing the output layer parameters of the preset title extension prefix and the title extension sub-model according to the second loss function.

[0061] When training and optimizing the title expansion sub-model, define a question-answering dataset for the title expansion sub-model, including a combination of pre-labeled descriptions and reference titles, for example, ,in, Represents a pre-annotated combination of description and reference title.

[0062] After getting the title expansion results, expand the results according to the title , query sequence And the preset reference title Calculate the negative log-likelihood loss, the second loss function as follows:

[0063] in, represents the decoded output of the step model at time t, Represents a set of fine-tuning parameters. The fine-tuning parameter set consists of the embedding layer of the title question extension prefix (i.e., the embedding corresponding to the title extension prefix Prompt) and the output layer parameters of the title extension sub-model. When optimizing the model, update the fine-tuning parameter set while fixing the base model.

[0064] In some optional implementations, step S2024 includes: inputting the query sequence and a preset keyword expansion prefix into a keyword expansion sub-model, and the keyword expansion sub-model outputs a keyword expansion result based on autoregressive decoding.

[0065] The processing flow of the keyword expansion sub-model is as follows Figure 4 As shown in the figure, the preset keyword expansion prefix is ​​a predefined vector, that is, the keyword expansion prefix Prompt in the figure, and the input query q is the query sequence. The query sequence is input to the tokenizer of the keyword expansion sub-model to obtain the input vector of the keyword expansion task, and the input vector of the keyword expansion task is used as the input of the base model. The keyword expansion output layer of the model outputs the keyword expansion result through autoregressive decoding.

[0066] In some optional embodiments, the method also includes: determining a third loss function based on the keywords, query sequence and preset reference keyword sequence in the keyword expansion results, the third loss function being the cross entropy loss; and optimizing the preset keyword expansion prefix and the output layer parameters of the keyword expansion sub-model according to the third loss function.

[0067] When training and optimizing the keyword expansion sub-model, define the dataset for the keyword expansion sub-model, including pre-annotated queries and reference keywords A combination of, for example, ,in, Represents a pre-annotated query and reference keywords combination.

[0068] After obtaining the keyword expansion results, the third loss function is determined according to the keywords, query sequence and preset reference keyword sequence in the keyword expansion results. Due to the disorder of keyword combinations, the third loss function can adopt bag-of-words cross entropy loss. The third loss function as follows:

[0069] in, Represents any keyword in the keyword expansion results. represents the query sequence, Represents a preset reference keyword sequence, Represents a set of fine-tuning parameters, which consists of the preset keyword expansion prefix and the output layer parameters of the keyword expansion sub-model. When optimizing the model, the fine-tuning parameter set is updated while the base model is fixed.

[0070] In some optional implementations, step S103 includes the following steps: S301, structurally processing the question and answer expansion results, the title expansion results, and the keyword expansion results respectively; S302, performing summary generation processing on the result after structured processing based on the summary generation model, and outputting the target query expansion question.

[0071] The processing flow of the summary generation model is as follows Figure 5 As shown in the figure, the question and answer expansion results, title expansion results and keyword expansion results are structured respectively, and these results are structured according to the set specifications, and combined with prompt words and tags to obtain the structured results, which is a multi-granular structured information. It is used as the input of the summary generation model to generate the target query expansion question.

[0072] As an example, for the question and answer expansion results , Title expansion results And the keyword expansion results And the summary generation instructions are organized according to the following structure: {"Question and Answer Extension":{ “Question and Answer Extended Documentation”: , "Description": "Question-based extended documents contain long text information related to the query, which covers multiple perspectives and details of the query at a wider level. This document mainly contains document-level information related to the query."}, "Title Extensions": { "Header Extension Document": , "Description": "The title expansion document contains a summary description of the query title expansion, mainly containing sentence-level information related to the query." }, "Keyword Expansion": { "Keyword Expansion Document": , "Description": "Keyword expansion includes various keyword sequences related to the query, which mainly contain key information at the word level related to the query." }, "System Instructions": "You are a writer with a wide range of interests and profound professional knowledge. Please extract key related information and generate an extended query based on the information and instructions of different granularities in the following ``Question and Answer Extension Document``, ``Title Extension Document`` and ``Keyword Extension Document``. The extended query should appropriately supplement the query-related detailed information while containing the important information of the query, so that the extended query can match the data related to the query semantics as much as possible."} In some optional embodiments, the method further comprises: Step S401, obtaining annotated reference query expansion documents; Step S402, generating a training data set based on the annotated reference query document, the structured processing results of the question and answer expansion results, the structured processing results of the title expansion results, and the structured processing results of the keyword expansion results; Step S403: determining a cross entropy loss function based on the training data set, and optimizing the summary generation model based on the cross entropy loss function.

[0073] In order to make the summary generation model have better performance, a data set can be constructed according to the above implementation method, and then summary annotation is performed to obtain the reference query expansion document s. The data set is as follows:

[0074] in, Indicates the question and answer expansion result. Represents the title expansion result, Represents a keyword expansion result.

[0075] After obtaining the data set, fine-tune using the cross entropy loss to obtain the cross entropy loss function :

[0076] in, represents the cross entropy loss function, N represents the length of the fine-tuning dataset, Indicates The length of sample data, Indicates the question and answer expansion result. Represents the title expansion result, Represents the keyword expansion result. represents the target word of the tth step of the i-th sample, Represents the sequence of target words before the current time step t. Based on the cross entropy loss function The parameters of the summary generation model are adjusted to optimize the summary generation model.

[0077] In this embodiment, a query expansion method is provided, such as Figure 6 As shown in the figure. The left sub-model is the question-answer expansion sub-model, the middle is the title expansion sub-model, and the right is the keyword expansion sub-model. In the processing flow of each sub-task, the pre-processed query sequence and the expansion prefix corresponding to each sub-model are used as the input of each sub-model, and the sub-model outputs its own expansion result. The expansion results are integrated, spliced ​​with the summary generation prefix, and input into the summary generation model, and finally the target query expansion question is output.

[0078] The question and answer expansion result is obtained by concatenating the filtered original query with the question and answer prefix Prompt and inputting it into the base model and the output layer of the corresponding task for autoregressive answer text generation.

[0079] Before running the entire system, it is necessary to fine-tune each subtask module based on the base system and save the parameters of the corresponding module's unique layer. When the system is running, first load the base model and then load the parameters and prompt words of the corresponding module, and input them into the operation logic of the corresponding module according to the input query. Finally, the outputs of the three modules of question and answer expansion, title expansion and keyword expansion are structured and spliced, and input into the final summary extraction module to generate an extended document that integrates the three information granularities.

[0080] The embodiment of the present invention provides a multi-granularity query expansion method based on a large language model, which uses prompt words with different functions to enable the base model to perform question and answer expansion, title expansion and keyword expansion respectively with the original query as the core under a given context, and expands from the document level, sentence level to the keyword level respectively, and then formats the three parts of the output and uses the summary function of the base model to comprehensively generate the final expanded query document. The query expansion is abstracted into three subtasks of different granularities. The question and answer expansion is used to collect as much and long document-level information as possible; the title expansion aims to obtain key information at the sentence level; the keyword expansion is used to obtain keyword sequence information related to the query. All sub-models share the base language model, and the fine-tuning process only affects the parameters of the output layer unique to the sub-model. Model fine-tuning only needs to provide a small amount of labeled data as required and then fine-tune according to the specified fine-tuning loss function, thereby improving processing efficiency.

[0081] In this embodiment, a query expansion device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0082] This embodiment provides a query expansion device, including: A question acquisition module is used to obtain the initial query question; A question expansion module, used to perform question and answer expansion, title expansion and keyword expansion on the initial query question, so as to generate a question and answer expansion result, a title expansion result and a keyword expansion result, wherein the question and answer expansion result at least includes the answer to the question related to the initial query question, the title expansion result at least includes the query title related to the initial query question, and the keyword expansion result at least includes the keyword related to the initial query question; A summary generation module is used to generate a summary based on the question and answer expansion result, the title expansion result and the keyword expansion result to determine the target query expansion question.

[0083] In some optional implementations, the question expansion module includes: A sequence generation unit, used to pre-process the initial query question and generate a query sequence; A question and answer expansion unit, configured to input the query sequence into a question and answer expansion sub-model to determine a question and answer expansion result based on the question and answer expansion sub-model; a title expansion unit, configured to input the query sequence into a title expansion sub-model to determine a title expansion result based on the title expansion sub-model; The keyword expansion unit is used to input the query sequence into the keyword expansion sub-model to determine a keyword expansion result based on the keyword expansion sub-model.

[0084] In some optional implementations, the sequence generation unit includes: A word segmentation processing subunit, used to perform word segmentation processing on the initial query question to obtain an initial vocabulary; The query sequence subunit is used to filter the initial vocabulary to remove meaningless words in the initial vocabulary and generate a query sequence.

[0085] In some optional implementations, the question and answer expansion unit includes: A question and answer expansion subunit, configured to input the query sequence and a preset question and answer expansion prefix into the question and answer expansion submodel, wherein the question and answer expansion submodel includes a question and answer expansion output layer; The first output subunit is used to perform autoregressive decoding based on the question and answer expansion output layer to output the question and answer expansion result.

[0086] In some optional implementations, the parameters of the question and answer expansion output layer and the embedding layer of the preset question and answer expansion prefix constitute a fine-tuning parameter set.

[0087] In some optional embodiments, the device further comprises: The first adjustment module is used to determine a first loss function based on the question and answer expansion result, a preset question reference answer sequence and the fine-tuning parameter set, so as to adjust the fine-tuning parameter set based on the first loss function.

[0088] In some optional implementations, the title extension unit includes: The title extension subunit is used to input the query sequence and the preset title extension prefix into the title extension submodel, and the title extension submodel outputs the title extension result based on autoregressive decoding.

[0089] In some optional embodiments, the device further comprises: A second loss function determination module, configured to determine a second loss function based on the title expansion result, the query sequence, and a preset reference title; The second adjustment module is used to optimize the preset title extension prefix and the output layer parameters of the title extension sub-model according to the second loss function.

[0090] In some optional implementations, the keyword expansion unit includes: The keyword expansion subunit is used to input the query sequence and a preset keyword expansion prefix into the keyword expansion submodel, and the keyword expansion submodel outputs the keyword expansion result based on autoregressive decoding.

[0091] In some optional embodiments, the device further comprises: A third loss function determination module, configured to determine a third loss function based on the keywords in the keyword expansion result, the query sequence, and a preset reference keyword sequence, wherein the third loss function is a cross entropy loss; The third adjustment module is used to optimize the preset keyword extension prefix and the output layer parameters of the keyword extension sub-model according to the third loss function.

[0092] In some optional implementations, the summary generation module includes: A structured processing unit, used for performing structured processing on the question and answer expansion result, the title expansion result and the keyword expansion result respectively; The question output unit is used to perform summary generation processing on the result after the structured processing based on the summary generation model, and output the target query expansion question.

[0093] In some optional embodiments, the device further comprises: A document acquisition module, used to acquire annotated reference query expansion documents; A data set generation module, used to generate a training data set based on the annotated reference query document, the structured processing result of the question and answer expansion result, the structured processing result of the title expansion result and the structured processing result of the keyword expansion result; A model optimization module is used to determine a cross entropy loss function based on the training data set, and optimize the summary generation model based on the cross entropy loss function.

[0094] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0095] The query expansion device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0096] An embodiment of the present invention further provides a computer device having the above-mentioned query expansion device.

[0097] See also Figure 7 , Figure 7 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 7 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 7 A processor 10 is taken as an example.

[0098] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0099] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0100] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0101] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0102] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0103] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0104] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention. Such modifications and variations are all within the scope defined by the present invention.

Claims

1. A query expansion method, characterized in that: The method comprises: Get the initial query question; Performing question and answer expansion, title expansion and keyword expansion on the initial query question respectively to generate question and answer expansion results, title expansion results and keyword expansion results, wherein the question and answer expansion results at least include answers to questions related to the initial query question, the title expansion results at least include query titles related to the initial query question, and the keyword expansion results at least include keywords related to the initial query question; A summary is generated based on the question and answer expansion result, the title expansion result and the keyword expansion result to determine a target query expansion question.

2. The method according to claim 1, characterized in that The question and answer expansion, title expansion and keyword expansion are performed on the initial query question to generate question and answer expansion results, title expansion results and keyword expansion results, including: Preprocessing the initial query question to generate a query sequence; Inputting the query sequence into the question and answer expansion submodel to determine a question and answer expansion result based on the question and answer expansion submodel; inputting the query sequence into a title expansion submodel to determine a title expansion result based on the title expansion submodel; The query sequence is input into a keyword expansion sub-model to determine a keyword expansion result based on the keyword expansion sub-model.

3. The method according to claim 2, characterized in that The preprocessing of the initial query question to generate a query sequence includes: Performing word segmentation processing on the initial query question to obtain an initial vocabulary; The initial vocabulary is filtered to remove meaningless words in the initial vocabulary and generate a query sequence.

4. The method according to claim 2, characterized in that: The step of inputting the query sequence into the question and answer expansion sub-model to determine the question and answer expansion result based on the question and answer expansion sub-model includes: Inputting the query sequence and the preset question and answer expansion prefix into the question and answer expansion sub-model, wherein the question and answer expansion sub-model includes a question and answer expansion output layer; Autoregressive decoding is performed based on the question and answer expansion output layer to output the question and answer expansion result.

5. The method according to claim 4, characterized in that The parameters of the question and answer expansion output layer and the embedding layer of the preset question and answer expansion prefix constitute a fine-tuning parameter set.

6. The method according to claim 5, characterized in that The method further comprises: A first loss function is determined based on the question and answer expansion result, a preset question reference answer sequence, and the fine-tuning parameter set, so as to adjust the fine-tuning parameter set based on the first loss function.

7. The method according to claim 2, characterized in that The step of inputting the query sequence into a title expansion sub-model to determine a title expansion result based on the title expansion sub-model comprises: The query sequence and a preset title extension prefix are input into the title extension sub-model, and the title extension sub-model outputs the title extension result based on autoregressive decoding.

8. The method according to claim 7, characterized in that The method further comprises: Determine a second loss function based on the title expansion result, the query sequence, and a preset reference title; The preset title extension prefix and the output layer parameters of the title extension sub-model are optimized according to the second loss function.

9. The method according to claim 2, characterized in that: The step of inputting the query sequence into a keyword expansion sub-model to determine a keyword expansion result based on the keyword expansion sub-model includes: The query sequence and a preset keyword expansion prefix are input into the keyword expansion sub-model, and the keyword expansion sub-model outputs the keyword expansion result based on autoregressive decoding.

10. The method according to claim 9, characterized in that The method further comprises: Determine a third loss function based on the keywords in the keyword expansion result, the query sequence, and a preset reference keyword sequence, wherein the third loss function is a cross entropy loss; The preset keyword extension prefix and the output layer parameters of the keyword extension sub-model are optimized according to the third loss function.

11. The method according to claim 1, characterized in that: The generating a summary based on the question and answer expansion result, the title expansion result and the keyword expansion result to determine the target query expansion question includes: Performing structured processing on the question and answer expansion results, the title expansion results and the keyword expansion results respectively; Based on the summary generation model, summary generation processing is performed on the result after the structured processing, and a target query expansion question is output.

12. The method according to claim 11, characterized in that The method further comprises: Obtain annotated reference query expansion documents; Generate a training data set based on the annotated reference query document, the structured processing result of the question and answer expansion result, the structured processing result of the title expansion result, and the structured processing result of the keyword expansion result; A cross entropy loss function is determined based on the training data set, and the summary generation model is optimized based on the cross entropy loss function.

13. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the query expansion method according to any one of claims 1 to 12 by executing the computer instructions.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the query expansion method according to any one of claims 1 to 12.

15. A computer program product, characterized in that The method comprises computer instructions, wherein the computer instructions are used to cause a computer to execute the query expansion method according to any one of claims 1 to 12.

Images