Intelligent dialogue-oriented multi-task vector retrieval method and device

Abstract

The invention provides an intelligent dialogue-oriented multi-task vector retrieval method and device, and relates to the technical field of artificial intelligence, and the method comprises the steps: firstly converting a current dialogue language input by a user into a current dialogue vector through an Embedded model; then obtaining a first vector used for representing the identity of a current user through a trained plug-in vector alignment model, obtaining a second vector used for representing the intention of the user through a fine-tuning vector generation model based on the current dialogue language input by the user, and finally obtaining a second vector used for representing the identity of the user through a generative pre-training converter. And outputting a target dialogue language corresponding to the current dialogue language. According to the intelligent dialogue-oriented multi-task vector retrieval method provided by the invention, the target dialogue language can be accurately and quickly output, and high-quality intelligent dialogue is realized.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a multi-task vector retrieval method and device for intelligent dialogue. Background Art

[0002] In recent years, with the development of large-scale pre-trained language models (LLMs), natural language generation tasks based on deep learning have made great progress. At the same time, intelligent dialogue systems based on artificial intelligence have also emerged and applied to various industries, such as online intelligent customer service, offline smart furniture, and voice assistants.

[0003] However, existing intelligent dialogue systems rely solely on language models such as Generative Pre-trained Transformer (GPT) to generate dialogues. The dialogue content generated by GPT has problems with static knowledge and insufficient generalization, which leads to inaccurate responses and slow response speeds, making it difficult to provide high-quality dialogue output in dynamic, domain-specific or information-intensive scenarios. Summary of the invention

[0004] The present invention provides a multi-task vector retrieval method and device for intelligent dialogue, which are used to solve the technical problems of inaccurate dialogue content and slow dialogue response speed in the prior art of intelligent dialogue.

[0005] The present invention provides a multi-task vector retrieval method for intelligent dialogue, comprising the following steps: Get the current dialog language entered by the user; Based on the current conversation language, generate a current conversation vector through an Embed model; Based on the current conversation vector, a first vector is obtained through a trained insertion vector alignment model, and based on the current conversation language, a second vector is obtained through a trained fine-tuning vector generation model; the first vector is used to characterize the current user identity, and the second vector is used to characterize the current user intention; the insertion vector alignment model is obtained through training based on the first historical sample data; the fine-tuning vector generation model is obtained through training based on the second historical sample data; Based on the first vector and the second vector, a target dialogue language corresponding to the current dialogue language is obtained through a generative pre-trained transformer.

[0006] According to a multi-task vector retrieval method for intelligent dialogue provided by the present invention, the training step of the plug-in vector alignment model includes: Obtaining a first historical conversation record and a historical user identity corresponding to the first historical conversation record; Based on the first historical conversation record, a first historical conversation vector is generated by using an Embed model, and based on the historical user identity, a first historical target vector is generated by using an Embed model; Using the first historical conversation vector as a first historical training sample and the first historical target vector as a first historical sample label, constructing first historical sample data; Based on the first historical sample data, supervised training is performed on the plug-in vector alignment model to obtain a trained plug-in vector alignment model.

[0007] According to a multi-task vector retrieval method for intelligent dialogue provided by the present invention, the plug-in vector alignment model includes any one of a multi-layer perceptron, an error back propagation network model and a radial basis function neural network model.

[0008] According to a multi-task vector retrieval method for intelligent dialogue provided by the present invention, the training step of the fine-tuning vector generation model includes: Get the second historical conversation record; Based on the second historical conversation record, obtaining the historical user intention through coreference resolution; Based on the historical user intention, generating a second historical target vector through an Embed model; Using the second historical conversation record as a second historical training sample and the second historical target vector as a second historical sample label, constructing second historical sample data; Based on the second historical sample data, supervised training is performed on the BERT model to obtain a trained BERT model; Use the trained BERT model as a fine-tuned vector generation model.

[0009] According to a multi-task vector retrieval method for intelligent dialogue provided by the present invention, the loss function of the plug-in vector alignment model includes any one of a mean square error loss function and a cosine similarity loss function.

[0010] According to a multi-task vector retrieval method for intelligent dialogue provided by the present invention, the loss function of the fine-tuning vector generation model includes any one of a mean square error loss function and a cosine similarity loss function.

[0011] The present invention also provides a multi-task vector retrieval device for intelligent dialogue, comprising the following modules: The acquisition module is used to obtain the current dialogue language input by the user; A first generating module, configured to generate a current dialogue vector through an Embed model based on the current dialogue language; A second generation module is used to obtain a first vector based on the current dialogue vector through a trained insertion vector alignment model, and to obtain a second vector based on the current dialogue language through a trained fine-tuning vector generation model; the first vector is used to characterize the current user identity, and the second vector is used to characterize the current user intention; the insertion vector alignment model is obtained by training based on the first historical sample data; the fine-tuning vector generation model is obtained by training based on the second historical sample data; A dialogue module is used to obtain a target dialogue language corresponding to the current dialogue language through a generative pre-trained transformer based on the first vector and the second vector.

[0012] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein when the processor executes the computer program, it implements any of the multi-task vector retrieval methods for intelligent dialogue as described above.

[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the multi-task vector retrieval methods for intelligent dialogue described above.

[0014] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements any of the multi-task vector retrieval methods for intelligent dialogue described above.

[0015] The multi-task vector retrieval method for intelligent dialogue provided by the present invention first converts the current dialogue language input by the user into the current dialogue vector through the Embed model; then obtains the first vector for characterizing the current user identity through the trained insertion-type vector alignment model, so that in the task scenario of obtaining the user identity, the user input is directly converted into a vector, and then the vector retrieval can be performed through the vector alignment method, thereby improving the accuracy and response speed of the vector retrieval process, so as to accurately and quickly obtain the user's identity information; and based on the current dialogue language input by the user, a second vector for characterizing the user's intention is obtained through a fine-tuning vector generation model, so that in the task scenario of obtaining the user's intention, key information is directly extracted according to the current dialogue language input by the user, the accuracy of vector retrieval is improved, and the user's intention is accurately and quickly obtained; finally, the target dialogue language corresponding to the current dialogue language is output through the generative pre-trained transformer, because the generative pre-trained transformer receives the first vector and the second vector after processing the current dialogue language input by the user, and no language processing and large-scale vector retrieval are required, so the target dialogue language can be accurately and quickly output, and high-quality intelligent dialogue is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 It is a flowchart of a multi-task vector retrieval method for intelligent dialogue provided by the present invention.

[0018] Figure 2 It is a schematic diagram of a flow chart of vector retrieval using an insertable vector alignment model provided by the present invention.

[0019] Figure 3 It is a schematic diagram of the process of performing vector retrieval using a fine-tuning vector generation model provided by the present invention.

[0020] Figure 4 It is a schematic diagram of the training process of an inserted vector alignment model provided by the present invention.

[0021] Figure 5 It is a structural schematic diagram of a multi-task vector retrieval device for intelligent dialogue provided by the present invention.

[0022] Figure 6 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] Existing intelligent dialogue systems only rely on language models such as Generative Pre-trained Transformer (GPT) to generate dialogues.

[0025] In the process of generating dialogues, keyword retrieval, vector retrieval, and hybrid retrieval can be performed through the Retrieval-Augmented Generation (RAG) technology. These methods are fast and easy to implement, and are suitable for static documents or task scenarios with clear keywords, but they lack the understanding of deep semantics and have difficulty processing synonyms or semantically complex queries. Therefore, the dialogue content generated by GPT has the problems of static knowledge and insufficient generalization ability, which leads to inaccurate response content and slow response speed, making it difficult to provide high-quality dialogue output in dynamic, domain-specific, or information-intensive scenarios.

[0026] In order to solve the above problems, the present invention proposes a multi-task vector retrieval method and device for intelligent dialogue. Figures 1 to 6 The present invention describes a multi-task vector retrieval method and device for intelligent dialogue.

[0027] Figure 1 FIG. 1 is a flow chart of a multi-task vector retrieval method for intelligent dialogue provided by the present invention. Figure 1 As shown, the method comprises the following steps: Step 101: Obtain the current dialogue language input by the user.

[0028] Specifically, the specific content of the current dialogue language input by the user may be asking questions, making suggestions, and raising demands, etc.

[0029] Step 102: Generate a current dialogue vector through an Embed model based on the current dialogue language.

[0030] Specifically, the Embed model is an existing model that maps high-dimensional data to a low-dimensional space. The Embed model can map the data to a low-dimensional vector space by learning the intrinsic structure and semantic information of the data, so that similar data points are close in position in the vector space.

[0031] In the embodiment of the present invention, the Embed model generates an embedding vector corresponding to the current conversation language, that is, the current conversation vector, by mapping high-dimensional data (that is, the current conversation language) to a low-dimensional space, and uses the current conversation vector as the input of the inserted vector alignment model.

[0032] Step 103: Based on the current conversation vector, a first vector is obtained through a trained insertion vector alignment model, and based on the current conversation language, a second vector is obtained through a trained fine-tuning vector generation model; the first vector is used to characterize the current user identity, and the second vector is used to characterize the current user intention; the insertion vector alignment model is trained based on the first historical sample data; and the fine-tuning vector generation model is trained based on the second historical sample data.

[0033] Specifically, in intelligent dialogue scenarios, in order to improve the quality of dialogue (for example, accurately and quickly responding to user needs, etc.), it is necessary to accurately and quickly obtain the user identity and user intention based on the current dialogue language input by the user.

[0034] In the task scenario of obtaining the user's identity, the current conversation vector is input into the trained plug-in vector alignment model, and vector retrieval is performed through vector alignment to obtain the user's identity information, thereby accurately and quickly outputting the first vector used to characterize the current user's identity.

[0035] The inserted vector alignment model is trained based on the first historical sample data. During the training process, the inserted vector alignment model can accurately match the user's actual identity according to the conversation language by learning the correlation characteristics between the first historical conversation record and the corresponding historical user identity.

[0036] Optionally, the loss function of the plug-in vector alignment model includes any one of a mean square error loss function and a cosine similarity loss function.

[0037] Figure 2 FIG. 1 is a flow chart of a plug-in vector alignment model for vector retrieval provided by the present invention. Figure 2 shown.

[0038] After the current conversation language input by the user is converted into the current conversation vector through the Embed model, the current conversation vector is fitted and aligned with the vector in the vector database through the retrieval enhancement generation technology, and the similarity between the current conversation vector and the vector in the vector database is calculated through the preset loss function (such as mean square error or cosine similarity, etc.), so as to output a vector with a similarity higher than the preset threshold, which matches the user identity. Among them, the vector database is constructed by converting a variety of languages ​​representing different user identities prepared in advance into corresponding vectors through the Embed model.

[0039] In the process of vector alignment, only the similarity between vectors needs to be calculated to achieve vector fitting. There is no need to understand semantic relationships or interact between natural language and vectors. Therefore, the amount of calculation is greatly reduced, the vector retrieval process is simplified, the response speed and accuracy are improved, and the first vector used to characterize the current user identity is accurately and quickly output.

[0040] In the task scenario of obtaining user intention, the current conversation language input by the user is input into the trained fine-tuning vector generation model, key semantic information is extracted according to the current conversation language input by the user, and vector retrieval is performed, so as to accurately and quickly output the second vector used to represent the current user intention.

[0041] The fine-tuning vector generation model is trained based on the second historical sample data. During the training process, the fine-tuning vector generation model can accurately extract the user's actual intention based on the conversation language by learning the correlation characteristics between the second historical conversation records and the corresponding historical user intentions.

[0042] Optionally, the loss function of the fine-tuning vector generation model includes any one of a mean square error loss function and a cosine similarity loss function.

[0043] Figure 3 FIG. 1 is a flow chart of a fine-tuning vector generation model for vector retrieval provided by the present invention. Figure 3 shown.

[0044] The current conversation language input by the user is input into the fine-tuned vector generation model, key semantic information is extracted through coreference resolution, and the text data is encoded into dense vectors. These vectors contain the semantic and contextual information of the text. The similarity between these vectors can be calculated through a preset loss function (such as mean square error or cosine similarity, etc.), achieving more accurate information retrieval and accurately extracting user intent, thereby accurately and quickly outputting a second vector used to represent the current user intent.

[0045] In an embodiment of the present invention, the task of obtaining the user identity is performed by a pre-trained plug-in vector alignment model, and the task of obtaining the user intent is performed by a pre-trained fine-tuning vector generation model. According to the current dialogue language input by the user, different tasks are performed in parallel by the two models, thereby improving the processing efficiency in multi-task scenarios. At the same time, through efficient vector retrieval, the user identity and user intent are accurately and quickly obtained, thereby providing a basis for subsequent high-quality dialogues.

[0046] Step 104: Based on the first vector and the second vector, a target dialogue language corresponding to the current dialogue language is obtained through a generative pre-trained transformer.

[0047] Specifically, the first vector and the second vector are input into a generative pre-trained transformer together, and the generative pre-trained transformer can quickly output a target dialogue language corresponding to the current dialogue language to achieve high-quality intelligent dialogue.

[0048] Among them, Generative Pre-trained Transformer (GPT) is an existing language model based on artificial intelligence technology. GPT learns the statistical laws of language through pre-training on a large-scale corpus and can generate coherent and natural text. GPT technology is based on the Transformer architecture in deep learning. It is pre-trained through unsupervised learning and fine-tuned on specific tasks to achieve efficient and accurate language processing.

[0049] However, during intelligent conversations, GPT cannot accurately grasp the user's identity and intentions, and the generated conversation content suffers from static knowledge and insufficient generalization capabilities, which in turn leads to inaccurate responses and slow response speeds.

[0050] The embodiment of the present invention makes improvements from the input end of GPT, performs the task of obtaining user identity through an inserted vector alignment model, and performs the task of obtaining user intention through a fine-tuning vector generation model, so that GPT can accurately and timely grasp the changing user identity and user intention in dynamic dialogue tasks. In addition, since the input received by GPT is a first vector representing the user identity and a second vector representing the user intention, there is no need to perform language processing and large-scale vector retrieval, so that the target dialogue language corresponding to the current dialogue language can be accurately and quickly output, and high-quality intelligent dialogue can be carried out with the user.

[0051] The multi-task vector retrieval method for intelligent dialogue provided by the present invention first converts the current dialogue language input by the user into the current dialogue vector through the Embed model; then obtains the first vector for characterizing the current user identity through the trained insertion-type vector alignment model, so that in the task scenario of obtaining the user identity, the user input is directly converted into a vector, and then the vector retrieval can be performed through the vector alignment method, thereby improving the accuracy and response speed of the vector retrieval process, so as to accurately and quickly obtain the user's identity information; and based on the current dialogue language input by the user, a second vector for characterizing the user's intention is obtained through a fine-tuning vector generation model, so that in the task scenario of obtaining the user's intention, key information is directly extracted according to the current dialogue language input by the user, the accuracy of vector retrieval is improved, and the user's intention is accurately and quickly obtained; finally, the target dialogue language corresponding to the current dialogue language is output through the generative pre-trained transformer, because the generative pre-trained transformer receives the first vector and the second vector after processing the current dialogue language input by the user, and no language processing and large-scale vector retrieval are required, so the target dialogue language can be accurately and quickly output, and high-quality intelligent dialogue is achieved.

[0052] Optionally, the training step of the plug-in vector alignment model includes: Obtaining a first historical conversation record and a historical user identity corresponding to the first historical conversation record; Based on the first historical conversation record, a first historical conversation vector is generated by using an Embed model, and based on the historical user identity, a first historical target vector is generated by using an Embed model; Using the first historical conversation vector as a first historical training sample and the first historical target vector as a first historical sample label, constructing first historical sample data; Based on the first historical sample data, supervised training is performed on the plug-in vector alignment model to obtain a trained plug-in vector alignment model.

[0053] Specifically, Figure 4 is a schematic diagram of the training process of an insertable vector alignment model provided by the present invention, such as Figure 4 shown.

[0054] First, obtain the first historical conversation record and the historical user identity corresponding to the first historical conversation record, that is, obtain the historical conversation records of a large number of users and the user identities corresponding to these historical conversation records. In this process, historical conversation records of users with different identities on different chat platforms can be counted, such as a large number of conversation records of users with different identities such as primary and secondary school students, university students, primary and secondary school teachers, university teachers, engineers in various industries, individual business owners, farmers, and workers on public chat platforms (such as Weibo comment areas and film and television comment areas, etc.).

[0055] Then, the first historical conversation record is converted into a first historical conversation vector through the Embed model, and the corresponding historical user identity is converted into a first historical target vector through the Embed model.

[0056] Then, first historical sample data is constructed by taking all first historical conversation vectors as first historical training samples and taking all first historical target vectors as first historical sample labels.

[0057] Finally, the first historical sample data is input into the plug-in vector alignment model for supervised training. Through repeated iterations, the model parameters are updated to obtain a trained plug-in vector alignment model.

[0058] In an embodiment of the present invention, all first historical conversation vectors are used as first historical training samples, and all first historical target vectors are used as first historical sample labels, so that there is no natural language in the training data, and only vectors are included, so that the trained plug-in vector alignment model can directly calculate the similarity between vectors and vectors through vector fitting alignment to achieve vector fitting, without the need to understand semantic relationships or the interaction between natural language and vectors, thereby reducing the amount of calculation, simplifying the vector retrieval process, and improving the response speed and accuracy.

[0059] Optionally, the plug-in vector alignment model includes any one of a multilayer perceptron, an error back propagation network model, and a radial basis function neural network model.

[0060] Specifically, since the plug-in vector alignment model does not require interaction between natural language and vectors, nor does it need to understand complex semantic relationships, it only needs to perform alignment fitting between vectors through similarity calculation, so it can be a multi-layer feedforward neural network with a simple structure and a small number of parameters to perform the task of obtaining user intentions, such as any of the multi-layer perceptron, error back propagation network model, and radial basis function neural network model.

[0061] The plug-in vector alignment model in the embodiment of the present invention not only reduces the computational complexity of the model and improves the compatibility and adaptability to the deployment environment through a lightweight model design, but also further improves the response speed. At the same time, by simplifying the vector retrieval process and performing vector alignment, the accuracy of vector retrieval is improved.

[0062] Optionally, the training step of the fine-tuning vector generation model includes: Get the second historical conversation record; Based on the second historical conversation record, obtaining the historical user intention through coreference resolution; Based on the historical user intention, generating a second historical target vector through an Embed model; Using the second historical conversation record as a second historical training sample and the second historical target vector as a second historical sample label, constructing second historical sample data; Based on the second historical sample data, supervised training is performed on the BERT model to obtain a trained BERT model; Use the trained BERT model as a fine-tuned vector generation model.

[0063] Specifically, when training the fine-tuning vector generation model, it is first necessary to obtain the second historical conversation record, that is, to obtain a large number of historical conversation records of different users. In this process, historical conversation records of users with different identities on different chat platforms can be counted, such as primary and secondary school students, university students, primary and secondary school teachers, university teachers, engineers in various industries, individual business owners, farmers, and workers, etc., a large number of conversation records of users with different identities on public chat platforms (such as Weibo comment areas and film and television comment areas, etc.).

[0064] Then, based on the second historical conversation record, coreference resolution is used to accurately understand the entity reference in the user's question and obtain the historical user intention.

[0065] For example, in some long conversations, in order to avoid repetition, pronouns, titles, and abbreviations are often used to refer to the full names of the entities mentioned earlier. For example, at the beginning, "XX University" is written, and then it is shortened to "XX University". In subsequent conversations, "this university", "it", etc. are used to refer to the previous "XX University". This phenomenon is called co-reference. Through co-reference resolution, different descriptions of the same entity can be merged together, key information can be extracted from long conversations, user intent can be captured, and long conversations can be converted into more concise language.

[0066] Then, the historical user intent is converted into a second historical target vector through the Embed model, that is, the concise language corresponding to the user intent is converted into the corresponding vector representation.

[0067] Then, second history sample data is constructed by taking all second history conversation records as second history training samples and taking all second history target vectors as second history sample labels.

[0068] Then, the second historical sample data is input into the BERT model for supervised training, and the model parameters are updated through repeated iterations to obtain a trained BERT model; Finally, the trained BERT model is used as the fine-tuned vector generation model.

[0069] In an embodiment of the present invention, the bidirectional encoder representations from transformers (BERT) model can simultaneously utilize the contextual information of words before and after through bidirectional encoding to improve semantic understanding ability. The BERT model is trained using the second historical sample data so that the model learns the correlation characteristics between the second historical conversation record and the corresponding historical user intention, and the trained BERT model is used as a fine-tuning vector generation model, so that the fine-tuning vector generation model can extract key semantic information according to the conversation language input by the user, accurately extract the user's actual intention, and accurately and quickly output the vector representation used to characterize the user's intention, thereby improving the accuracy of vector retrieval.

[0070] The multi-task vector retrieval device for intelligent dialogue provided by the present invention is described below. The multi-task vector retrieval device for intelligent dialogue described below and the multi-task vector retrieval method for intelligent dialogue described above can be referenced to each other.

[0071] Based on any of the above embodiments, Figure 5 is a schematic diagram of the structure of a multi-task vector retrieval device for intelligent dialogue provided by the present invention, such as Figure 5 The embodiment of the present invention provides a multi-task vector retrieval device for intelligent dialogue, including an acquisition module 501, a first generation module 502, a second generation module 503 and a dialogue module 504, wherein: The acquisition module 501 is used to acquire the current dialogue language input by the user; the first generation module 502 is used to generate the current dialogue vector through the Embed model based on the current dialogue language; the second generation module 503 is used to obtain a first vector based on the current dialogue vector through a trained insertion vector alignment model, and to obtain a second vector based on the current dialogue language through a trained fine-tuning vector generation model; the first vector is used to characterize the current user identity, and the second vector is used to characterize the current user intention; the insertion vector alignment model is obtained by training based on the first historical sample data; the fine-tuning vector generation model is obtained by training based on the second historical sample data; the dialogue module 504 is used to obtain a target dialogue language corresponding to the current dialogue language through a generative pre-trained transformer based on the first vector and the second vector.

[0072] The multi-task vector retrieval device for intelligent dialogue provided by the present invention first converts the current dialogue language input by the user into the current dialogue vector through the Embed model; then obtains the first vector for characterizing the current user identity through the trained insertion-type vector alignment model, so that in the task scenario of obtaining the user identity, the user input is directly converted into a vector, and then the vector retrieval can be performed through the vector alignment method, thereby improving the accuracy and response speed of the vector retrieval process, so as to accurately and quickly obtain the user's identity information; and based on the current dialogue language input by the user, a second vector for characterizing the user's intention is obtained through a fine-tuning vector generation model, so that in the task scenario of obtaining the user's intention, key information is directly extracted according to the current dialogue language input by the user, the accuracy of vector retrieval is improved, and the user's intention is accurately and quickly obtained; finally, the target dialogue language corresponding to the current dialogue language is output through the generative pre-trained transformer, because the generative pre-trained transformer receives the first vector and the second vector after processing the current dialogue language input by the user, and no language processing and large-scale vector retrieval are required, so the target dialogue language can be accurately and quickly output, and high-quality intelligent dialogue is achieved.

[0073] Figure 6 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 6 As shown, the electronic device may include: a processor 610, a communications interface 620, a memory 630 and a communication bus 640, wherein the processor 610, the communications interface 620 and the memory 630 communicate with each other through the communication bus 640. The processor 610 may call the logic instructions in the memory 630 to execute the multi-task vector retrieval method for intelligent dialogue, and the method includes: Get the current dialog language entered by the user; Based on the current conversation language, generate a current conversation vector through an Embed model; Based on the current conversation vector, a first vector is obtained through a trained insertion vector alignment model, and based on the current conversation language, a second vector is obtained through a trained fine-tuning vector generation model; the first vector is used to characterize the current user identity, and the second vector is used to characterize the current user intention; the insertion vector alignment model is obtained through training based on the first historical sample data; the fine-tuning vector generation model is obtained through training based on the second historical sample data; Based on the first vector and the second vector, a target dialogue language corresponding to the current dialogue language is obtained through a generative pre-trained transformer.

[0074] In addition, the logic instructions in the above-mentioned memory 630 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0075] On the other hand, the present invention further provides a computer program product, the computer program product comprising a computer program, the computer program can be stored in a non-transitory computer-readable storage medium, and when the computer program is executed by a processor, the computer can execute the multi-task vector retrieval method for intelligent dialogue provided by the above methods, the method comprising: Get the current dialog language entered by the user; Based on the current conversation language, generate a current conversation vector through an Embed model; Based on the current conversation vector, a first vector is obtained through a trained insertion vector alignment model, and based on the current conversation language, a second vector is obtained through a trained fine-tuning vector generation model; the first vector is used to characterize the current user identity, and the second vector is used to characterize the current user intention; the insertion vector alignment model is obtained through training based on the first historical sample data; the fine-tuning vector generation model is obtained through training based on the second historical sample data; Based on the first vector and the second vector, a target dialogue language corresponding to the current dialogue language is obtained through a generative pre-trained transformer.

[0076] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the multi-task vector retrieval method for intelligent dialogue provided by the above methods, the method comprising: Get the current dialog language entered by the user; Based on the current conversation language, generate a current conversation vector through an Embed model; Based on the current conversation vector, a first vector is obtained through a trained insertion vector alignment model, and based on the current conversation language, a second vector is obtained through a trained fine-tuning vector generation model; the first vector is used to characterize the current user identity, and the second vector is used to characterize the current user intention; the insertion vector alignment model is obtained through training based on the first historical sample data; the fine-tuning vector generation model is obtained through training based on the second historical sample data; Based on the first vector and the second vector, a target dialogue language corresponding to the current dialogue language is obtained through a generative pre-trained transformer.

[0077] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0078] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0079] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0080] It should also be noted that the terms "target", "first", "second", etc. in the present invention are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of the same type, and the number of objects is not limited. For example, the first object can be one or more.

[0081] "Determine B based on A" in the embodiments of the present application means that the factor A should be considered when determining B. It is not limited to "B can be determined based on A alone", but should also include: "Determine B based on A and C", "Determine B based on A, C and E", "Determine C based on A, and further determine B based on C", etc. In addition, it can also include taking A as a condition for determining B, for example, "When A meets the first condition, use the first method to determine B"; for another example, "When A meets the second condition, determine B", etc.; for another example, "When A meets the third condition, determine B based on the first parameter", etc. Of course, it can also be a condition that takes A as a factor for determining B, for example, "When A meets the first condition, use the first method to determine C, and further determine B based on C", etc.

[0082] In the present invention, the term "plurality" refers to two or more than two, and other quantifiers are similar to them.

[0083] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-task vector retrieval method for intelligent dialogue, characterized in that: include: Get the current dialog language entered by the user; Based on the current conversation language, generate a current conversation vector through an Embed model; Based on the current conversation vector, a first vector is obtained by using a trained insertion vector alignment model, and based on the current conversation language, a second vector is obtained by using a trained fine-tuning vector generation model; the first vector is used to represent the current user identity, and the second vector is used to represent the current user intention; The insertion-type vector alignment model is obtained by training based on the first historical sample data; the fine-tuning vector generation model is obtained by training based on the second historical sample data; Based on the first vector and the second vector, a target dialogue language corresponding to the current dialogue language is obtained through a generative pre-trained transformer.

2. The multi-task vector retrieval method for intelligent dialogue according to claim 1, characterized in that: The training steps of the plug-in vector alignment model include: Obtaining a first historical conversation record and a historical user identity corresponding to the first historical conversation record; Based on the first historical conversation record, a first historical conversation vector is generated by using an Embed model, and based on the historical user identity, a first historical target vector is generated by using an Embed model; Using the first historical conversation vector as a first historical training sample and the first historical target vector as a first historical sample label, constructing first historical sample data; Based on the first historical sample data, supervised training is performed on the plug-in vector alignment model to obtain a trained plug-in vector alignment model.

3. The multi-task vector retrieval method for intelligent dialogue according to claim 2, characterized in that: The plug-in vector alignment model includes any one of a multi-layer perceptron, an error back propagation network model, and a radial basis function neural network model.

4. The multi-task vector retrieval method for intelligent dialogue according to claim 1, characterized in that: The training steps of the fine-tuning vector generation model include: Get the second historical conversation record; Based on the second historical conversation record, obtaining the historical user intention through coreference resolution; Based on the historical user intention, generating a second historical target vector through an Embed model; Using the second historical conversation record as a second historical training sample and the second historical target vector as a second historical sample label, constructing second historical sample data; Based on the second historical sample data, supervised training is performed on the BERT model to obtain a trained BERT model; Use the trained BERT model as a fine-tuned vector generation model.

5. The multi-task vector retrieval method for intelligent dialogue according to claim 1, characterized in that: The loss function of the plug-in vector alignment model includes any one of a mean square error loss function and a cosine similarity loss function.

6. The multi-task vector retrieval method for intelligent dialogue according to claim 1, characterized in that: The loss function of the fine-tuning vector generation model includes any one of a mean square error loss function and a cosine similarity loss function.

7. A multi-task vector retrieval device for intelligent dialogue, characterized in that: include: The acquisition module is used to obtain the current dialogue language input by the user; A first generating module, configured to generate a current dialogue vector through an Embed model based on the current dialogue language; A second generation module is configured to obtain a first vector based on the current dialogue vector by using a trained insertion-type vector alignment model, and to obtain a second vector based on the current dialogue language by using a trained fine-tuning vector generation model; the first vector is used to represent the current user identity, and the second vector is used to represent the current user intention; The insertion-type vector alignment model is obtained by training based on the first historical sample data; the fine-tuning vector generation model is obtained by training based on the second historical sample data; A dialogue module is used to obtain a target dialogue language corresponding to the current dialogue language through a generative pre-trained transformer based on the first vector and the second vector.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the multi-task vector retrieval method for intelligent dialogue according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the multi-task vector retrieval method for intelligent dialogue as claimed in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the multi-task vector retrieval method for intelligent dialogue as claimed in any one of claims 1 to 6 is implemented.

Images