AI-based long dialogue completion methods, devices, terminal equipment, and media

Abstract

This application relates to the field of artificial intelligence technology, and particularly to a long dialogue completion method, apparatus, terminal device, and medium based on artificial intelligence. The method is applied to insurance sales, using the same encoder to encode features of the current question information and historical dialogue information from each round, obtaining corresponding first and second encoding results. All second and first encoding results are concatenated to obtain a feature concatenation result. This feature concatenation result is then decoded and used as the encoder's output data. This solves the problem of output length limitations in the encoding model, improves the accuracy of encoded features in long dialogues, and uses the current question information of the current round of dialogue as the start label in the autoregressive mechanism to guide the decoder in decoding the encoded features, generating a decoding result. This improves the generation accuracy of the target task, thereby improving the accuracy of the information to be completed in the decoding result.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a long dialogue completion method, apparatus, terminal device and medium based on artificial intelligence. Background Technology

[0002] When promoting financial products or other services, financial institutions such as banks, securities firms, and insurance companies leverage voice assistants to enhance the personalized user experience. This involves extracting customers' geographical and social data from long conversations to provide personalized services, such as customizing specific insurance policies and coverage. During the conversation, to better understand the user's meaning, it's necessary to capture semantic information from the current dialogue. Due to the naturalness of conversational language, users tend to use pronouns to refer to nouns mentioned earlier and may omit shared information already known to the other party. A thorough understanding of user utterances is crucial in voice assistant applications; only by fully comprehending the user's speech can better service be provided. Long conversations also present another challenge: if the current utterance is incomplete, it's necessary to identify and complete the relevant background discussion within the long conversation to ensure a more comprehensive understanding of the user's message.

[0003] In existing technologies, when completing the preceding background discussion fragments in a long dialogue, the preceding background discussion fragments are generally spliced ​​into a long dialogue, and features from the long dialogue are extracted to complete the current utterance. However, due to the limitation of the output length in deep learning models, the accuracy of the completed current utterance is low. Therefore, how to improve the completion accuracy in long dialogue completion has become an urgent problem to be solved. Summary of the Invention

[0004] In view of this, embodiments of this application provide a long dialogue completion method, apparatus, terminal device and medium based on artificial intelligence to solve the problem of low completion accuracy in long dialogue completion.

[0005] In a first aspect, embodiments of this application provide a long dialogue completion method based on artificial intelligence, the long dialogue completion method comprising:

[0006] Obtain historical dialogue information of at least one round of dialogue within a historical time period, and current question information of the current round of dialogue at the current time. The historical dialogue information includes at least one question and at least one answer.

[0007] For any round of historical dialogue information, the question and answer information in the historical dialogue information are concatenated in the order of question and answer to obtain the information concatenation result;

[0008] The same encoder is used to encode the current dialogue information and the information concatenation result respectively to obtain a first encoding result corresponding to the current dialogue information and a second encoding result corresponding to the information concatenation result;

[0009] All the second encoding results and the first encoding results are concatenated to obtain the feature concatenation result, and the position information of the first encoding result in the feature concatenation result is determined;

[0010] Based on the autoregressive mechanism, the feature concatenation result is decoded to obtain the feature decoding result. According to the location information, the result after the location information is determined from the decoded feature result as the target result, and the target result is determined as the answer result of the current question information.

[0011] Secondly, embodiments of this application provide an artificial intelligence-based long dialogue completion device, the long dialogue completion device comprising:

[0012] The acquisition module is used to acquire historical dialogue information of at least one round of dialogue within a historical time period, as well as the current question information of the current round of dialogue at the current time. The historical dialogue information includes at least one question and at least one answer.

[0013] The splicing module is used to splice the question and answer information in the historical dialogue information according to the question and answer order for any round of historical dialogue information to obtain the information splicing result;

[0014] The feature encoding module is used to encode the current dialogue information and the information concatenation result using the same encoder to obtain a first encoding result corresponding to the current dialogue information and a second encoding result corresponding to the information concatenation result;

[0015] The determination module is used to concatenate all the second encoding results and the first encoding results to obtain a feature concatenation result, and determine the position information of the first encoding result in the feature concatenation result;

[0016] The decoding module is used to decode the feature concatenation result based on an autoregressive mechanism to obtain the feature decoding result. Based on the location information, the module determines the result after the location information from the decoded feature result as the target result and determines the target result as the answer result of the current question information.

[0017] Thirdly, embodiments of this application provide a terminal device, the terminal device including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the long dialogue completion method based on artificial intelligence as described in the first aspect.

[0018] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the long dialogue completion method based on artificial intelligence as described in the first aspect.

[0019] The beneficial effects of the embodiments in this application compared with the prior art are:

[0020] Acquire historical dialogue information from at least one round of dialogue within a historical time period, as well as the current question information for the current round of dialogue at the current time. The historical dialogue information includes at least one question and at least one answer. For any round of historical dialogue information, concatenate the question and answer information in the historical dialogue information according to the question-and-answer order to obtain the concatenated information result. Use the same encoder to perform feature encoding on the current question information and the concatenated information result respectively to obtain the first encoding result corresponding to the current dialogue information and the second encoding result corresponding to the concatenated information result. Concatenate all the second encoding results and the first encoding results to obtain the feature concatenation result. Determine the position information of the first encoding result in the feature concatenation result. Based on the autoregressive mechanism, decode the feature concatenation result to obtain the feature decoding result. Based on the position information, determine the result after determining the position information from the decoded feature result as the target result. The target result is determined to be the answer result of the current question information. In this application, each round of a long dialogue is encoded separately, and the encoded features are concatenated, which solves the problem of the output length limitation of the encoding model and improves the accuracy of the encoded features in long dialogues. The current question information of the current round of dialogue at the current time is used as the start label in the autoregressive mechanism to guide the decoder to decode the encoded features and generate the decoding result, which improves the generation accuracy of the target task and thus improves the accuracy of the information to be completed in the decoding result. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of an application environment for a long dialogue completion method based on artificial intelligence provided in an embodiment of this application;

[0023] Figure 2 This is a flowchart illustrating a long dialogue completion method based on artificial intelligence provided in an embodiment of this application;

[0024] Figure 3This is a schematic diagram of the structure of a long dialogue completion device based on artificial intelligence provided in an embodiment of this application;

[0025] Figure 4 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application. Detailed Implementation

[0026] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0027] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0028] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0029] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0030] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0031] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0032] The embodiments of this invention can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that utilize digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0033] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0034] It should be understood that the sequence number of each step in the following embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0035] To illustrate the technical solution of this application, specific embodiments are described below.

[0036] This application applies to the financial insurance industry. In insurance sales, chatbots typically engage in dialogue with users to obtain relevant information and automatically sell corresponding insurance products. During automated sales, due to the naturalness of conversational language, users tend to use pronouns to refer to nouns mentioned in the dialogue and may omit shared information already obtained from the other party. For example, when selling different insurance products to a user, the chatbot recommends three products: A, B, and C, and introduces their advantages and target audiences. When asked which product the user prefers, if the user responds that they prefer the second product, the current question in this round of dialogue is "Which insurance product do you prefer?" The dialogue preceding this question is considered historical dialogue information within a specific time period. For any round of historical dialogue information, the question and answer information are concatenated according to the question-and-answer order to obtain the concatenated information. The same encoder is used to concatenate the information. The result is used for feature encoding, and the current question information "Which insurance product do you like?" is also used for feature encoding. This yields a first encoded result corresponding to the current dialogue information and a second encoded result corresponding to the concatenated information. All the second encoded results and the first encoded results are concatenated to obtain the feature concatenation result. The position information of the first encoded result in the feature concatenation result is determined. Based on the autoregressive mechanism, the feature concatenation result is decoded to obtain the feature decoding result. Based on the position information, the result after determining the position information from the decoded feature result is the target result. If the decoded result of "Which insurance product do you like?" is Insurance Product B, then Insurance Product B is the answer result of the current question information.

[0037] This application provides an AI-based long dialogue completion method, which can be applied to applications such as... Figure 1 In this application environment, the client and server communicate with each other. Clients include, but are not limited to, smart TVs, PDAs, desktop computers, laptops, ultra-mobile personal computers (UMPCs), netbooks, cloud terminal devices, and personal digital assistants (PDAs). The server can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0038] See Figure 2 This is a flowchart illustrating an artificial intelligence-based long dialogue completion method provided in an embodiment of this application. The method is applied to the server, where the user uploads the long dialogue from the client to the server. Figure 2 As shown, this long dialogue completion method may include the following steps:

[0039] S201: Obtain historical dialogue information for at least one round of dialogue within a historical time period, as well as the current question information for the current round of dialogue at the current time.

[0040] In step S201, the historical dialogue information includes at least one question and at least one answer. The current round of dialogue at the current time has related information with the dialogues in the historical time period. That is, the current question in the current round of dialogue at the current time corresponds to the same topic as the historical dialogue information of at least one round of dialogue in the historical time period.

[0041] In this embodiment, dialogue information with users is obtained from the insurance sales system. Historical dialogue information of at least one round of dialogue within a historical time period is obtained. The historical dialogue information includes at least one question and at least one answer. When the historical dialogue information includes long dialogues, the historical dialogue information is sorted in chronological order.

[0042] S202: For any round of historical dialogue information, concatenate the question and answer information in the historical dialogue information according to the question and answer order to obtain the information concatenation result.

[0043] In step S202, the question and answer information in the obtained historical dialogue information is spliced ​​together in the order of question and answer.

[0044] In this embodiment, when concatenating the question and answer information in any round of historical dialogue information according to the question-and-answer order, a separator is used for concatenation. For example, for a question-and-answer pair: the question information is "When was the Vat formally opened?", and the corresponding answer information is "Formally established in 1475", then the concatenated result is "When was the Vat formally opened?Formally established in 1475", where is the separator.

[0045] It should be noted that the type of delimiter can be different in different application scenarios. Any symbol used to split statements can be used as the delimiter in this embodiment of the invention.

[0046] S203: Use the same encoder to perform feature encoding on the current dialogue information and the information concatenation result respectively, to obtain the first encoding result corresponding to the current dialogue information and the second encoding result corresponding to the information concatenation result.

[0047] In step S203, the same encoder is used to encode the current dialogue information and the concatenated information result, respectively, to obtain a first encoding result corresponding to the current dialogue information and a second encoding result corresponding to the concatenated information result. Using the same encoder allows for simultaneous learning of the current dialogue information and the concatenated information result, facilitating the completion of the current dialogue information based on historical information.

[0048] In this embodiment, the encoder in the text-to-text task model is used to perform feature encoding on the current question information and the information concatenation result to obtain the first encoding result corresponding to the current question information and the second encoding result corresponding to the information concatenation result.

[0049] It's important to note that when using the encoder in a text-to-text task model for feature encoding, fine-tuning of the text-to-text task model is necessary. This involves adding perturbation noise to the text-to-text task model to obtain a perturbed text-to-text task model. The concatenation result is then used to train this perturbed model, resulting in a trained text-to-text task model. During fine-tuning, we used noisytune to narrow the gap between the downstream task and the text-to-text task model. Noisytune adds noise to the parameters of the text-to-text task model during fine-tuning, helping it to better fine-tune the downstream task. More specifically, noisytune is a matrix perturbation method that adds different uniform noise to different parameter matrices based on their standard deviations. This allows us to consider the different characteristics of different types of parameters in the text-to-text task model. In our model, we use this method to add perturbations to the text-to-text task model, improving the fine-tuning effect and obtaining a trained text-to-text task model.

[0050] Optionally, the same encoder is used to encode the current dialogue information and the concatenated information result separately, to obtain a first encoding result corresponding to the current dialogue information and a second encoding result corresponding to the concatenated information result, including:

[0051] The current question information and the information concatenation result are processed by word segmentation to obtain the first word segmentation result corresponding to the current question information and the second word segmentation result corresponding to the information concatenation result;

[0052] The first word segmentation result and the second word segmentation result are respectively input into the same encoder, and the first encoding result of the first word segmentation result and the second encoding result of the corresponding second word segmentation result are output.

[0053] In this real-time example, the current question information and the information concatenation result are processed separately for word segmentation. For English text, word segmentation can be performed based on spaces; for Chinese text, it compares the strings in the text content with a word segmentation dictionary. If the dictionary contains the string, the string matches the word, and the string is segmented at both ends; otherwise, it is not segmented. This method generally relies on a very large word segmentation dictionary. If the dictionary is insufficient or erroneous, the word segmentation effect will be greatly reduced. Statistical word segmentation algorithms mainly include N-gram models and Hidden Markov Models (HMMs). These work based on the frequency of adjacent characters in a sentence, determining whether two characters are the same word based on their frequency. This method can segment all characters in the text, belonging to the full segmentation method. This embodiment uses a combination of both methods to process the text features for word segmentation. The first word segmentation result corresponding to the current question information and the second word segmentation result corresponding to the information concatenation result are compared.

[0054] After word segmentation, a pre-defined word vector calculation tool is used to determine the corresponding word vectors. Word2vec, a natural language processing tool, is used to convert words in natural language into word vectors that computers can understand. Traditional word vectors are prone to the curse of dimensionality, and any two words are isolated, failing to reflect the relationships between words. Therefore, this embodiment uses word2vec to obtain word vectors, which can reflect the similarity between words by calculating the distance between vectors. Word2vec mainly uses two models: Skip-Gram and CBOW. This embodiment uses Skip-Gram to implement the word vector conversion. The Skip-Gram model mainly predicts the surrounding words through the center word.

[0055] It's important to note that to ensure the positional information of each word is also represented, the position of each word needs to be encoded, resulting in a position vector for each word. To ensure the input carries sentence segmentation information, we also encode the sentence to which each word belongs, resulting in a corresponding segment vector. The word vector, position vector, and segment vector of the same word are then fused or superimposed to obtain the vector representation of that word, which carries the semantic information of the semantic unit. All the vector representations corresponding to each word are arranged row-wise to form an input matrix or input sequence.

[0056] The input matrix or sequence corresponding to the first word segmentation result and the input matrix or sequence corresponding to the second word segmentation result are input into the same encoder for feature encoding, resulting in a first encoded result for the first word segmentation result and a second encoded result for the corresponding second word segmentation result. This same encoder can be a bidirectional encoder of a Transformer, extracting text information from all layers (up, down, left, and right) to achieve bidirectional text representation. Because it extracts context, words, and sentences in detail, the resulting word vectors are dynamic, ensuring that the word vectors for the same word differ in different linguistic environments. This better expresses the relationships between words, sentences, and context, generating corresponding encoding results.

[0057] It should be noted that when the length of the information concatenation results in different rounds is different, the number of word segments obtained may be different, and the length of the encoded results may be different. In order to obtain a fixed-length encoded result, the information concatenation results are processed by a variable-length sequence. The shorter parts of the information concatenation results are padded using a mask mechanism, and the shorter text in the information concatenation results is padded with 0.

[0058] S204: Concatenate all the second encoding results and the first encoding results to obtain the feature concatenation result, and determine the position information of the first encoding result in the feature concatenation result.

[0059] In step S204, the second encoding result corresponding to the long dialogue history information within the historical time period is concatenated to obtain the feature encoding result corresponding to the long dialogue history information within the historical time period. The feature encoding result contains the features in the long dialogue history information within the historical time period, so as to decode the corresponding information based on the dialogue information within the historical time period. The feature encoding result is concatenated with the first encoding result to obtain the feature concatenation result, so as to decode the current question information of the current round of dialogue at the current time.

[0060] In this embodiment, all the second encoding results and the first encoding results are concatenated to obtain the feature concatenation result. During concatenation, all the second encoding results are first concatenated to obtain the encoding results in the long dialogue history information within the historical time period, and then concatenated with the first encoding results to obtain the feature concatenation result. This facilitates decoding based on the current question information of the current round of dialogue at the current time.

[0061] S205: Based on the autoregressive mechanism, the feature splicing result is decoded to obtain the feature decoding result. Based on the location information, the result after determining the location information from the decoded feature result is the target result. The target result is determined as the answer result of the current question information.

[0062] In step S205, an autoregressive decoder is used for decoding. The autoregressive mechanism can be understood as using a layer of decoding and encoding attention mechanism to decode the target content hidden layer vector group, and to decode the feature concatenation result, thereby obtaining the decoding result related to the first encoding result. Based on the position information, the result after determining the position information from the decoded feature result is the target result, and the target result is determined as the answer result of the current question information.

[0063] In this embodiment, an autoregressive decoder is used to decode the code and obtain the decoding results related to the first encoding result in sequence. Through the autoregressive mechanism, each slot segment is decoded sequentially until the ending [EOS] identifier is predicted.

[0064] The autoregressive decoder can include a masked multi-head attention layer, a first residual layer, a first normalization layer, a feedforward layer, a second residual layer, a second normalization layer, and an activation layer. Utilizing the established linguistic logic patterns of the autoregressive decoder, the probability of the next character appearing at the current position can be predicted. The masked multi-head attention layer in the autoregressive decoder constructs the contextual relationship between the first encoding result of the current input position and all second encoding results, obtaining a contextual feature vector. The obtained contextual feature vector and the feature vector with positional information are input into the first residual layer, which aids in convergence. The vector processed by the first residual layer is input into the first normalization layer. The normalization layer performs mean and variance processing on the input vector processed by the first residual layer, preventing the output feature vector sequence from being too large or too small, which could lead to gradient instability. Next, the data processed by the first normalization layer is input into the feedforward layer to reconstruct the contextual feature vector, resulting in a reconstructed feature vector sequence. This reconstructed feature vector sequence is then input into the second residual layer and the second normalization layer. After processing by multiple autoregressive decoders, the decoding result is obtained.

[0065] Optionally, the feature concatenation result is used as input data to a preset decoder, the decoding result is output, the decoding result is concatenated and fused with the input data to obtain a fused feature, the fused feature is used as input data to perform the step of inputting input data to a preset decoder, until the end label is output to obtain the final feature decoding result.

[0066] In this embodiment, the autoregressive decoder performs sequential decoding, using the decoding result as input for the next word. Each decoding step involves inputting the obtained context feature vector sequence and a feature vector sequence with positional information into the first residual layer to obtain the first residual layer processed feature vector sequence. This first residual layer processed feature vector sequence is then input into the first normalization layer to obtain the first normalized layer processed feature vector sequence. This first normalized layer processed feature vector sequence is then input into the feedforward layer to obtain the reconstructed feature vector sequence. The reconstructed feature vector sequence and the first normalized layer processed feature vector sequence are then input into the second residual layer to obtain the second residual layer processed feature vector sequence. This second residual layer processed feature vector sequence is then input into the second normalization layer to obtain the second normalized layer processed feature vector sequence. Finally, this second normalized layer processed feature vector sequence is input into the activation layer to obtain the decoding result.

[0067] In this embodiment, the first encoding result with location information is input into a masked multi-head attention layer to establish a contextual relationship between the first encoding result and all second encoding results, resulting in a contextual feature vector sequence. The contextual feature vector sequence and the feature vector sequence with location information are then input into a first residual layer to obtain a first residual layer processed feature vector sequence. This first residual layer processing aids in convergence. Finally, the first residual layer processed feature vector sequence is input into a first normalization layer. Mean and variance processing of the input first residual layer processed feature vector sequence prevents gradient instability caused by excessively large or small feature vector sequences.

[0068] It should be noted that the decoder of the autoregressive mechanism decodes based on the last element of the encoder. For example, all the second encoding results are concatenated with the first encoding results, and the first encoding result is used as the last element of the encoder output. The first decoding result is predicted based on the first encoding result. The first decoding result is concatenated with the feature concatenation result, and the first decoding result is used as the last element of the encoder output. The decoding and prediction are performed again based on the first decoding result to obtain the final feature decoding result.

[0069] Optionally, the feature concatenation result is used as input data to a preset decoder, and the decoding result is output, including:

[0070] The feature concatenation result is input into the preset decoder. The multi-head attention layer in the preset decoder performs multi-head attention calculation on the feature concatenation result to obtain attention features.

[0071] Based on the cross-attention mechanism, the attention features are determined as query vectors, the cross-attention value of the query vector in the second encoding result is calculated, and the cross-attention features are obtained based on the cross-attention value.

[0072] Based on the cross-attention features, the cross-attention features are decoded to obtain the decoding result.

[0073] In this embodiment, the feature concatenation result is input into a preset decoder. The multi-head attention layer in the preset decoder performs multi-head attention calculation on the feature concatenation result to obtain attention features. The multi-head attention layer is a multi-head attention layer with a mask. Based on the cross-attention mechanism, the attention features are determined as query vectors. The cross-attention value of the query vector in the second encoding result is calculated. Based on the cross-attention value, the contextual relationship between the first encoding result at the current position and all second encoding results is determined to obtain the context feature vector.

[0074] When determining the context feature vector, the attention feature is determined as the query vector through the cross-attention mechanism. The cross-attention value of the query vector in the second encoding result is calculated. Based on the cross-attention value, the context feature vector is determined. Based on the context feature vector, the cross-attention feature is decoded to obtain the decoding result.

[0075] Using a multi-head attention layer, the attention values ​​of the query vector in the second encoded feature are weighted and accumulated, fusing multiple local feature vectors to obtain a global feature vector. The multi-head attention matrix helps determine the attention relationships between local features, enabling targeted fusion and improving decoding and prediction results.

[0076] Optionally, based on the cross-attention mechanism, the attention features are determined as query vectors, the cross-attention value of the query vector in the second encoding result is calculated, and the cross-attention features are obtained based on the cross-attention value, including:

[0077] Based on the cross-attention mechanism, the key vector and value vector are determined from the second encoding result;

[0078] Based on the query vector, key vector, and value vector, calculate the cross-attention value of the query vector in the second encoding result, and obtain the cross-attention feature based on the cross-attention value.

[0079] In this embodiment, based on the cross-attention mechanism, the key-value vector and value vector are determined from the second encoding result. The query vector and value vector can be the feature vector corresponding to the second encoding result, or the feature vector after processing the second encoding result. Based on the query vector, key-value vector, and value vector, the cross-attention value of the query vector in the second encoding result is calculated to determine the correlation between the decoded result and each word segment in the second encoding result, thereby determining the correlation between each word segment in the first encoding result and the second encoding result, and improving the decoding prediction result.

[0080] Optionally, based on the cross-attention features, the cross-attention features are decoded to obtain the decoding result, including:

[0081] The cross-attention features are reconstructed by a pre-set feedforward layer to obtain the reconstructed features;

[0082] The reconstructed features are normalized by a preset normalization layer to obtain the normalization result;

[0083] The normalization result is input into the activation layer, and the corresponding decoding result is output.

[0084] In this embodiment, the cross-attention features are reconstructed through a preset feedforward layer to obtain reconstructed features. Based on the reconstructed features, the reconstructed feature vector sequence corresponding to the reconstructed features is obtained. The reconstructed feature vector sequence is normalized through a normalization layer. The normalization process can be mean and variance processing to obtain the normalization result. The normalization result is input into the activation layer, and the decoding result corresponding to the normalization result is output. The activation function in the activation layer can be the ReLU activation function.

[0085] Acquire historical dialogue information from at least one round of dialogue within a historical time period, as well as the current question information for the current round of dialogue at the current time. The historical dialogue information includes at least one question and at least one answer. For any round of historical dialogue information, concatenate the question and answer information in the historical dialogue information according to the question-and-answer order to obtain the concatenated information result. Use the same encoder to perform feature encoding on the current question information and the concatenated information result respectively to obtain the first encoding result corresponding to the current dialogue information and the second encoding result corresponding to the concatenated information result. Concatenate all the second encoding results and the first encoding results to obtain the feature concatenation result. Determine the position information of the first encoding result in the feature concatenation result. Based on the autoregressive mechanism, decode the feature concatenation result to obtain the feature decoding result. Based on the position information, determine the result after determining the position information from the decoded feature result as the target result. The target result is determined to be the answer result of the current question information. In this application, each round of a long dialogue is encoded separately, and the encoded features are concatenated, which solves the problem of the output length limitation of the encoding model and improves the accuracy of the encoded features in long dialogues. The current question information of the current round of dialogue at the current time is used as the start label in the autoregressive mechanism to guide the decoder to decode the encoded features and generate the decoding result, which improves the generation accuracy of the target task and thus improves the accuracy of the information to be completed in the decoding result.

[0086] See Figure 3 , Figure 3A structural block diagram of an AI-based long dialogue completion device provided in an embodiment of this application is shown. This AI-based long dialogue completion device is applied to the aforementioned server. For ease of explanation, only the parts relevant to the embodiments of this application are shown.

[0087] See Figure 3 The long dialogue completion device 30 includes: an acquisition module 31, a splicing module 32, a feature encoding module 33, a determination module 34, and a decoding module 35.

[0088] The acquisition module 31 is used to acquire historical dialogue information of at least one round of dialogue within a historical time period, as well as the current question information of the current round of dialogue at the current time. The historical dialogue information includes at least one question and at least one answer.

[0089] The splicing module 32 is used to splice the question and answer information in the historical dialogue information according to the question and answer order for any round of historical dialogue information to obtain the information splicing result.

[0090] The feature encoding module 33 is used to encode the current dialogue information and the information concatenation result using the same encoder to obtain the first encoding result corresponding to the current dialogue information and the second encoding result corresponding to the information concatenation result.

[0091] The determination module 34 is used to concatenate all the second encoding results and the first encoding results to obtain the feature concatenation result and determine the position information of the first encoding result in the feature concatenation result.

[0092] The decoding module 35 is used to decode the feature splicing result based on the autoregressive mechanism to obtain the feature decoding result. Based on the location information, the result after determining the location information from the decoded feature result is the target result, and the target result is the answer result of the current question information.

[0093] Optionally, the feature encoding module 33 includes:

[0094] Word segmentation is used to segment the current question information and the information concatenation result separately, to obtain the first word segmentation result corresponding to the current question information and the second word segmentation result corresponding to the information concatenation result.

[0095] The encoding unit is used to input the first word segmentation result and the second word segmentation result into the same encoder, and output the first encoding result of the first word segmentation result and the second encoding result of the corresponding second word segmentation result.

[0096] Optionally, the decoding module 35 includes:

[0097] The loop unit is used to input the feature concatenation result as input data into the preset decoder, output the decoding result, concatenate and fuse the decoding result with the input data to obtain the fused feature, and input the fused feature as input data into the preset decoder again for decoding until the end label is output to obtain the final feature decoding result.

[0098] Optionally, the above-mentioned loop unit includes:

[0099] The computational subunit is used to input the feature concatenation result into the preset decoder. Through the multi-head attention layer in the preset decoder, multi-head attention calculation is performed on the feature concatenation result to obtain attention features.

[0100] The cross-attention calculation subunit is used to determine the attention features as query vectors based on the cross-attention mechanism, calculate the cross-attention value of the query vector in the second encoding result, and decode the cross-attention features according to the cross-attention value to obtain the decoding result.

[0101] Optionally, the above-mentioned computational subunit includes:

[0102] Determine the sub-units used to determine the key vector and value vector from the second encoding result based on the cross-attention mechanism.

[0103] The attention calculation subunit is used to calculate the cross-attention value of the query vector in the second encoding result based on the query vector, the key vector, and the value vector.

[0104] Optionally, the above-mentioned computational subunit includes:

[0105] The reconstruction subunit is used to reconstruct the cross-attention features through a preset feedforward layer to obtain the reconstructed features.

[0106] The normalization subunit is used to normalize the reconstructed features through a preset normalization layer to obtain the normalization result.

[0107] The activation subunit is used to input the normalization result into the activation layer and output the decoding result corresponding to the normalization result.

[0108] It should be noted that the information interaction and execution process between the above modules are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0109] Figure 4 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application. Figure 4 As shown, the terminal device of this embodiment includes: at least one processor ( Figure 4Only one is shown in the diagram), a memory, and a computer program stored in the memory and executable on at least one processor, which, when executed by the processor, implements the steps in any of the above embodiments of the AI-based long dialogue completion method.

[0110] The terminal device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that... Figure 4 This is merely an example of a terminal device and does not constitute a limitation on the terminal device. A terminal device may include more or fewer components than shown in the figure, or a combination of certain components, or different components, such as network interfaces, displays, and input devices.

[0111] The processor referred to can be a CPU, but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0112] The memory includes readable storage media, internal memory, etc., wherein the internal memory can be the main memory of the terminal device, and the internal memory provides an environment for the operation of the operating system and computer-readable instructions stored in the readable storage media. The readable storage media can be the hard drive of the terminal device, or in some embodiments, it can be an external storage device of the terminal device, such as a plug-in hard drive, a Smart Media Card (SMC), a Secure Digital Card (SD), or a Flash Card. Furthermore, the memory can include both internal storage units and external storage devices of the terminal device. The memory is used to store the operating system, applications, bootloader, data, and other programs, such as program code for computer programs. The memory can also be used to temporarily store data that has been output or will be output.

[0113] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above device can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the above method embodiments. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium can include at least: any entity or device capable of carrying computer program code, a recording medium, a computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0114] The implementation of all or part of the processes in the methods of the above embodiments can also be accomplished by a computer program product. When the computer program product is run on a terminal device, the terminal device executes the steps in the above method embodiments.

[0115] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0116] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0117] In the embodiments provided in this application, it should be understood that the disclosed apparatus / terminal devices and methods can be implemented in other ways. For example, the apparatus / terminal device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0118] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0119] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. An artificial intelligence-based long dialogue completion method, characterized by, The long dialogue completion method comprises: obtaining historical dialogue information of at least one round of dialogue in a historical time period and current question information of the current round of dialogue at a current time, wherein the historical dialogue information comprises at least one question information and at least one answer information; for any round of historical dialogue information, the question information and the answer information in the historical dialogue information are spliced in the order of question and answer to obtain an information splicing result; using the same encoder to respectively encode the current question information and the information splicing result to obtain a first encoding result corresponding to the current question information and a second encoding result corresponding to the information splicing result; splicing all the second encoding results and the first encoding result to obtain a feature splicing result, and determining the position information of the first encoding result in the feature splicing result; based on the autoregressive mechanism, the feature splicing result is decoded to obtain a feature decoding result, and according to the position information, the result after the position information in the decoding feature result is determined as a target result, and the target result is determined as the answer result of the current question information; the feature splicing result is decoded based on the autoregressive mechanism to obtain a feature decoding result, comprising: splicing all the second encoding results and the first encoding result, the first encoding result as the last element of the encoder output, decoding and predicting the first decoding result according to the first encoding result, splicing the first decoding result to the feature splicing result, taking the first decoding result as the last element of the encoder output, decoding and predicting again according to the first decoding result to obtain the final feature decoding result.

2. The long- conversation completion method of claim 1, wherein, the current question information and the information splicing result are respectively encoded using the same encoder to obtain a first encoding result corresponding to the current question information and a second encoding result corresponding to the information splicing result, comprising: performing word segmentation processing on the current question information and the information splicing result respectively to obtain a first word segmentation processing result corresponding to the current question information and a second word segmentation processing result corresponding to the information splicing result; inputting the first word segmentation processing result and the second word segmentation processing result into the same encoder respectively to output the first encoding result of the first word segmentation processing result and the second encoding result corresponding to the second word segmentation processing result.

3. The long- conversation completion method of claim 1, wherein, the feature splicing result is decoded based on the autoregressive mechanism to obtain a feature decoding result, comprising: inputting the feature splicing result as input data into a preset decoder to output a decoding result, splicing and fusing the decoding result and the input data to obtain a fusion feature, and inputting the input data into the preset decoder until an output end label is output to obtain a final feature decoding result.

4. The long- conversation completion method of claim 3, wherein, the feature splicing result is decoded based on the autoregressive mechanism to obtain a feature decoding result, comprising: inputting the feature splicing result into a preset decoder, performing multi-head attention calculation on the feature splicing result through a multi-head attention layer in the preset decoder to obtain attention features; The attention feature is determined as a query vector based on a cross-attention mechanism, a cross-attention value of the query vector in the second encoding result is calculated, and a cross-attention feature is obtained according to the cross-attention value; The cross-attention feature is decoded according to the cross-attention feature to obtain a decoding result.

5. The long- conversation completion method of claim 4, wherein, The attention feature is determined as a query vector based on a cross-attention mechanism, a cross-attention value of the query vector in the second encoding result is calculated, and a cross-attention feature is obtained according to the cross-attention value, including: The key-value vector and the value vector are determined from the second encoding result based on a cross-attention mechanism; The cross-attention value of the query vector in the second encoding result is calculated according to the query vector, the key-value vector and the value vector, and a cross-attention feature is obtained according to the cross-attention value.

6. The long- conversation completion method of claim 5, wherein, The cross-attention feature is decoded according to the cross-attention feature to obtain a decoding result, including: The cross-attention feature is reconstructed by a pre-set feedforward layer to obtain a reconstructed feature; The reconstructed feature is normalized by a pre-set normalization layer to obtain a normalized processing result; The normalized processing result is input into an activation layer, and the decoding result corresponding to the normalized result is output.

7. An artificial intelligence-based long dialogue completion device, characterized by, The long dialogue completion device includes: An acquisition module is configured to acquire historical dialogue information of at least one round of dialogue in a historical time period and current question information of a current round of dialogue at a current time, the historical dialogue information including at least one question information and at least one answer information; A splicing module is configured to splice the question information and the answer information in the historical dialogue information according to a question and answer order to obtain an information splicing result for any round of historical dialogue information; A feature encoding module is configured to use the same encoder to respectively encode the current question information and the information splicing result to obtain a first encoding result corresponding to the current question information and a second encoding result corresponding to the information splicing result; A determination module is configured to splice all the second encoding results and the first encoding result to obtain a feature splicing result, and determine position information of the first encoding result in the feature splicing result; A decoding module is configured to decode the feature splicing result based on a self-recurrent mechanism to obtain a feature decoding result, determine a result after the position information in the decoding feature result as a target result, and determine the target result as an answer result of the current question information. The feature encoding module includes: ​ 8. The long- conversation completion apparatus according to claim 7, wherein ​ The word segmentation processing is used for performing word segmentation processing on the current question information and the information splicing result respectively to obtain a first word segmentation processing result corresponding to the current question information and a second word segmentation processing result corresponding to the information splicing result. The encoding unit is configured to input the first word segmentation processing result and the second word segmentation processing result into the same encoder respectively, and output a first encoding result of the first word segmentation processing result and a second encoding result corresponding to the second word segmentation processing result.

9. A terminal device, comprising: The terminal device comprises a processor, a memory, and a computer program stored in the memory and executable on the processor, and the processor implements the long dialogue completion method according to any one of claims 1 to 6 when executing the computer program.

10. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 9. The computer program is executed by the processor to implement the long dialogue completion method according to any one of claims 1 to 6.

Images