Intelligent dialogue method, related device, equipment and storage medium

Abstract

The invention discloses an intelligent dialogue method, a related device, equipment and a storage medium, and the intelligent dialogue method comprises the steps: carrying out the conversion based on a first statement represented by a natural language, and obtaining a first feature sequence represented by a machine language; processing the first feature sequence based on an intelligent dialogue model to obtain a second feature sequence represented by a machine language; wherein the second feature sequence comprises feature information used for replying the first statement; and based on the second feature sequence, performing conversion to obtain a second statement which is expressed in a natural language and used for replying the first statement. According to the scheme, the accuracy and efficiency of intelligent dialogues can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of natural language processing, and in particular to an intelligent dialogue method and related devices, equipment and storage media. Background Art

[0002] At present, in the process of intelligent dialogue models realizing intelligent dialogue tasks, the input and output are natural languages ​​such as Chinese and English.

[0003] Due to the limitations of natural language expression and thinking order, natural language is not suitable for efficient expression and calculation by computers, and is not concise enough. Therefore, under the limitations of natural language, the intelligent dialogue model is confined to the original logic of natural language text and is difficult to expand. The efficiency and accuracy of intelligent dialogue are not high.

[0004] Therefore, how to improve the accuracy and efficiency of intelligent dialogue has become an urgent problem to be solved. Summary of the invention

[0005] The main technical problem solved by the present application is to provide an intelligent dialogue method and related devices, equipment and storage media, which can improve the accuracy and efficiency of intelligent dialogue.

[0006] In order to solve the above technical problems, the first aspect of the present application provides an intelligent dialogue method, which includes: based on a first sentence expressed in natural language, converting to obtain a first feature sequence expressed in machine language; processing the first feature sequence based on an intelligent dialogue model to obtain a second feature sequence expressed in machine language; wherein the second feature sequence includes feature information for replying to the first sentence; based on the second feature sequence, converting to obtain a second sentence expressed in natural language and used to reply to the first sentence.

[0007] In order to solve the above technical problems, the second aspect of the present application provides an intelligent dialogue device, which includes a first conversion module, a processing module and a second conversion module; the first conversion module is used to convert a first feature sequence expressed in a machine language based on a first sentence expressed in a natural language; the processing module is used to process the first feature sequence based on an intelligent dialogue model to obtain a second feature sequence expressed in a machine language; wherein the second feature sequence includes feature information for replying to the first sentence; the second conversion module is used to convert a second sentence expressed in a natural language and used to reply to the first sentence based on the second feature sequence.

[0008] In order to solve the above technical problems, the third aspect of the present application provides an electronic device, including a memory and a processor coupled to each other, and the processor is used to execute program instructions stored in the memory to implement the intelligent dialogue method in the above first aspect.

[0009] In order to solve the above technical problems, the fourth aspect of the present application provides a computer-readable storage medium on which program instructions are stored. When the program instructions are executed by a processor, the intelligent dialogue method in the above first aspect is implemented.

[0010] In the above scheme, a first sentence expressed in natural language is converted to obtain a first feature sequence expressed in machine language, and the first feature sequence is processed based on the intelligent dialogue model to obtain a second feature sequence expressed in machine language, wherein the second feature sequence includes feature information for replying to the first sentence, and then based on the second feature sequence, a second sentence expressed in natural language and used to reply to the first sentence is converted. Since natural language requires a specific word order and specific words to express and transmit information, it is greatly affected by its own expression method and logic, while machine language expresses and transmits information in computer language, which is more suitable for the execution environment of the intelligent dialogue model. Therefore, before the intelligent dialogue model executes the intelligent dialogue task of the sentence, the sentence in natural language is converted into a feature sequence expressed in machine language. The intelligent dialogue model realizes the intelligent dialogue task by processing the feature sequence that is more suitable for it and replying with the feature sequence. That is to say, the intelligent dialogue model changes from a natural language to a natural language working mode to a feature sequence to a feature sequence working mode, that is, the intelligent dialogue model processes data in machine language throughout the process, so it can improve the accuracy and efficiency of intelligent dialogue. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 This is a flow chart of an embodiment of the intelligent dialogue method of the present application;

[0012] Figure 2 This is a schematic diagram of a framework of an embodiment of a language conversion model of the present application;

[0013] Figure 3 for Figure 2 A structural diagram of an embodiment of a language conversion model;

[0014] Figure 4 for Figure 2 The framework diagram of the language conversion model in different conversion modes;

[0015] Figure 5 This is a schematic diagram of the framework of an embodiment of the intelligent dialogue system of the present application;

[0016] Figure 6 A schematic diagram of a framework of an embodiment of an intelligent dialogue device of the present application;

[0017] Figure 7 A schematic diagram of a framework of an embodiment of an electronic device of the present application;

[0018] Figure 8A schematic diagram of a framework of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0019] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.

[0020] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0021] The term "multiple" herein means two or more than two. In addition, the term "at least one" herein means any combination of at least two of any one or more of a plurality of, for example, including at least one of A, B, and C, and may mean including any one or more elements selected from the set consisting of A, B, and C.

[0022] See also Figure 1 , Figure 1 This is a flowchart of an embodiment of the intelligent dialogue method of the present application. In the embodiment of the present disclosure, natural language is used to express and transmit information through sentences. The language of natural language may include but is not limited to: Chinese, English, Japanese, etc. The sentence expressed in natural language is composed of one or more words or strings, and expresses a specific meaning through a specific grammatical structure. The sentence expressed in natural language may include but is not limited to the first sentence, second sentence, first sample sentence, second sample sentence, third sample sentence, etc. mentioned below. The definitions of these sentences will not be repeated below. Specifically, the intelligent dialogue method of the embodiment of the present disclosure includes the following steps:

[0023] Step S11: Based on the first sentence expressed in natural language, convert and obtain a first feature sequence expressed in machine language.

[0024] In one implementation scenario, in order to convert the first sentence into a first feature sequence, the first embedding representation of the first sentence can be extracted, and the initial feature sequence can be obtained, and then cyclic decoding can be performed based on the first embedding representation and the initial feature sequence to obtain the decoding features of each round of decoding output, until the decoding feature representation is converted from natural language to machine language, so that the first feature sequence can be obtained through the decoding features of each round of decoding output. Among them, the initial feature sequence includes initial features expressed in machine language, and the initial features are used to indicate the start of conversion from natural language to machine language. Therefore, by indicating the start of conversion from natural language to machine language with the initial features expressed in machine language, and combining the initial feature sequence with the first embedding representation of the first sentence for cyclic decoding, an automated process from natural language to machine language can be achieved.

[0025] In a specific implementation scenario, the first embedding representation of the first sentence can be obtained by performing vector mapping on the first sentence through a word embedding algorithm (embedding). For example, the first embedding representation can be in the form of a one-hot vector.

[0026] In a specific implementation scenario, the feature sequence represented in machine language can be a vector sequence of indefinite length. For example, the vector dimension of each feature in the feature sequence is C, the sequence length is L, and the feature sequence is an L*C floating point tensor. In this case, the initial feature in the initial feature sequence can be an all-0 vector or other vector representing blank information.

[0027] In a specific implementation scenario, the feature dimension of the first embedding representation is the same as the feature dimension of the initial feature sequence.

[0028] In a specific implementation scenario, the decoding feature representing the end of the conversion from natural language to machine language may be an all-0 vector or other vector representing blank information.

[0029] In a specific implementation scenario, the initial feature sequence used for each round of cyclic decoding will change according to the different cyclic rounds. Specifically, in the next round of the current round, the initial feature sequence is obtained by fusing the decoding features of the current round with the initial feature sequence in the current round. In order to obtain the decoding features of the current round, the first fused features of the current round can be obtained by fusing the first embedded representation and the initial feature sequence, and decoding is performed according to the first fused features of the current round to obtain the decoding features of the current round. Therefore, the initial feature sequence used for each round of cyclic decoding is related to the decoding features outputted in the previous round. By gradually adding decoding features to the initial feature sequence and performing cyclic decoding using the first embedded representation and the initial feature sequence containing more and more feature information, the prediction accuracy of the decoding features can be gradually improved.

[0030] As a possible implementation, the first embedding representation is a vector sequence with a feature dimension of C and a sequence length of S. The initial feature sequence includes initial features with a feature dimension of C, and the sequence length of the initial feature sequence is 1. The first embedding representation and the initial feature sequence can be fused through the concat function to obtain a first fused feature. The first fused feature is a vector sequence with a feature dimension of C and a sequence length of S+1. After decoding according to the first fused feature of the first round to obtain a decoded feature, the decoded feature of the first round is fused with the initial feature sequence in the current round to obtain an initial feature sequence with a sequence length of 2, and the fused initial feature sequence is fused again with the first embedding representation through the concat function to obtain the first fused feature of the next round with a sequence length of S+2, and decoding is performed based on the first fused feature of the next round until the decoding feature representation is converted from natural language to machine language.

[0031] As another possible implementation, the decoding features of the current round can be directly fused with the first fusion features of the next round to obtain the first fusion features of the next round, so as to perform decoding based on the first fusion features of the next round until the decoding feature representation is converted from natural language to machine language.

[0032] In a specific implementation scenario, after decoding is performed according to the first fusion feature of the current round as described above to obtain the decoding features of the current round, and before the initial feature sequence in the next round of the current round is updated, since the feature dimensions of the decoding features output by the current round may be different from the feature dimensions of the initial feature sequence, when the feature dimensions are different, the decoding features of the current round can be dimensionally transformed before the initial feature sequence in the next round of the current round is updated.

[0033] In a specific implementation scenario, in the first round of cyclic decoding, in addition to the initial features, the initial feature sequence may also include a number of pre-set reference features. The reference features can provide more feature information in the process of converting natural language to machine language, thereby improving the accuracy of converting natural language to machine language.

[0034] In another implementation scenario, the first sentence may be converted into a first feature sequence in combination with the large language model. Specifically, a prompt is constructed by the first sentence and the first conversion requirement, and the first conversion requirement may include a sequence length, a dimension, etc. of a feature sequence represented in a machine language for converting the first sentence, thereby instructing the large language model to convert the first sentence into the first feature sequence according to the first conversion requirement.

[0035] In another implementation scenario, the first embedded representation of the first sentence can also be directly decoded in a loop to obtain decoding features of each round of decoding output until the decoding feature representation is completely converted from natural language to machine language, thereby obtaining a first feature sequence through the decoding features of each round of decoding output.

[0036] In one implementation scenario, a first sentence is converted into a first feature sequence by a language conversion model, wherein the language conversion model can at least realize the conversion of natural language into machine language, and the language conversion model is trained by a first sample sentence expressed in natural language. In order to train the language conversion model, the first sample sentence can be converted by the language conversion model to obtain a first prediction feature sequence expressed in machine language, and the first prediction feature sequence can be converted by the language conversion model to obtain a first prediction sentence expressed in natural language, and the network parameters of the language conversion model are adjusted based on at least a first loss measured by the difference between the first sample sentence and the first prediction sentence. In this training method, the language conversion model not only has the ability to convert natural language into machine language, but also has the ability to convert machine language into natural language. Therefore, by adjusting the network parameters of the language conversion model based on the difference between the first sample sentence initially input into the language conversion model and the first prediction sentence finally output by the language conversion model, the language conversion model can be trained to restore the natural language after converting the natural language into machine language, thereby improving the accuracy of the language conversion model in converting between natural language and machine language.

[0037] The specific process of converting the first sample sentence by using the language conversion model to obtain the first prediction feature sequence expressed in machine language can refer to the process of converting the first sentence expressed in natural language to obtain the first feature sequence expressed in machine language in the aforementioned embodiment. The specific process of converting the first prediction feature sequence by using the language conversion model to obtain the first prediction sentence expressed in natural language can refer to the process of converting the second feature sequence to obtain the second sentence expressed in natural language and used to reply to the first sentence in the subsequent embodiment, which will not be repeated here. It should be noted that in the process of obtaining the first feature sequence in the aforementioned embodiment, the cyclic decoding is stopped according to the decoding feature of the decoding output representation that ends the conversion from natural language to machine language, and in the process of training the language conversion model to obtain the first prediction feature sequence, the cyclic decoding is performed according to the preset rounds, and the cyclic decoding is stopped after the preset rounds are reached, and in the process of obtaining the first prediction sentence, the cyclic decoding of the same preset rounds as the first prediction feature sequence is required, and the cyclic decoding is stopped after the preset rounds are reached.

[0038] In a specific implementation scenario, the first loss can be obtained through a loss function such as cross entropy loss, and the specific process of adjusting the network parameters of the language conversion model can refer to related technologies such as gradient descent that use training loss to adjust network parameters, which will not be repeated here.

[0039] In a specific implementation scenario, when the number of characters in the first sample sentence is less than a preset number of characters (e.g., 1 character, 2 characters, 3 characters, etc.), the first sample sentence can be used as short text training data for training the language conversion model. Training with short text training data can improve the accuracy of the language conversion model in learning and restoring specific concept vocabulary in natural language.

[0040] In a specific implementation scenario, when the number of characters in the first sample sentence is greater than the preset number of characters (e.g., 50 characters, 100 characters, 200 characters, etc.), the first sample sentence can be used as long text training data for training the language conversion model. In this case, in addition to adjusting the network parameters of the language conversion model by the first loss measured by the difference between the first sample sentence and the first predicted sentence, the network parameters of the language conversion model can also be adjusted in combination with the first loss and the second loss. The second loss includes at least one of the first sub-loss and the second sub-loss, the first sub-loss is positively correlated with the sum of the absolute values ​​of each feature element in the first predicted feature sequence and negatively correlated with the first total number of feature elements, the second sub-loss is positively correlated with the second total number of valid feature elements in the first predicted feature sequence and negatively correlated with the first total number, and the valid feature elements represent feature elements in the first predicted feature sequence that are greater than a preset threshold. Therefore, training with long text training data and designing a loss function based on the first predicted feature sequence can promote the language conversion model to learn to restore natural language information while learning the ability to compress natural language information. As a possible example, when the second loss includes the first sub-loss and the second sub-loss, the sum of the first loss (C), the first sub-loss (P) and the second sub-loss (Q) is used as the total loss Loss for adjusting the network parameters of the language conversion model, Loss = C + Q + P. In addition, in order to train different capabilities of the language conversion model in a targeted manner, different weight factors can be set for the first loss and the second loss in different training scenarios, and then the total loss for adjusting the network parameters of the language conversion model is obtained by weighted averaging.

[0041] As a possible implementation, by taking the L1 norm of all feature elements in the first prediction feature sequence, the sum L of the absolute values ​​of each feature element is obtained, and the first total number n is obtained by counting the number of feature elements, and then the first sub-loss P=k1*(L / n), where k1 is the first positive proportional coefficient, and the value of k1 can be 1, 2, 3, etc. Therefore, the absolute value mean of the feature elements in the first prediction feature sequence is used as the first sub-loss, which can avoid the increase of the L1 norm caused by the length of the first prediction feature sequence as much as possible, and thus can improve the language conversion model's ability to balance data.

[0042] As another possible implementation, the first total number n is obtained by counting the number of feature elements in the first prediction feature sequence, and the second total number m is obtained by counting the number of valid feature elements in the first prediction feature sequence, and the second sub-loss Q=k2*(m / n) is obtained, wherein k2 is the second positive proportional coefficient, and the value of k2 can be 1, 2, 3, and so on. Therefore, the number m of valid feature elements in the first prediction feature sequence is positively correlated with the second sub-loss, that is, the fewer the number of valid feature elements, the more natural language information each valid feature element can represent, thereby improving the compression capability of the language conversion model for natural language information. In addition, feature elements that are less than or equal to a preset threshold can be further regarded as invalid feature elements, and the invalid feature elements can be replaced with all-0 vectors, for example, through the relu function v=relu(vector 2 -ε0, vector represents the eigenvalue of the feature element, ε represents the preset threshold, and when the eigenvalue vector1 is less than or equal to the preset threshold, v=0, then vector1 is replaced with a vector of all 0s.

[0043] In one implementation scenario, a first sentence is converted into a first feature sequence by a language conversion model, wherein the language conversion model can at least realize the conversion of natural language into machine language, and the language conversion model is trained by a second sample sentence expressed in natural language, and the second sample sentence is annotated with a target feature sequence expressed in machine language. In order to train the language conversion model, the second sample sentence can be converted by the language conversion model to obtain a second predicted feature sequence expressed in machine language, and the network parameters of the language conversion model are adjusted based on the difference between the second predicted feature sequence and the target feature sequence. Therefore, taking the annotated target feature sequence as the training target, the first sentence is converted into the first feature sequence by the language conversion model, which can improve the accuracy and efficiency of the conversion from natural language to machine language. It should be noted that the specific process of converting the second sample sentence using the language conversion model to obtain the second predicted feature sequence expressed in machine language can refer to the process of converting the first sentence expressed in natural language to obtain the first feature sequence expressed in machine language in the aforementioned embodiment, which will not be repeated here.

[0044] In a specific implementation scenario, the target feature sequence can be predicted by the language conversion model trained by the first sample sentence in the aforementioned embodiment. For the sake of distinction, the language conversion model in the aforementioned embodiment is described as the "first language conversion model", and the language conversion model trained based on the second sample sentence in this embodiment is described as the "second language conversion model". Then, the second sample sentence is converted by the first language conversion model to obtain the target feature sequence of the second sample sentence. In addition, in order to improve the language refinement of the second language conversion model in converting natural language into machine language, after the second sample sentence is converted into a feature sequence represented by machine language using the first language conversion model, the zero vector in the feature sequence can be deleted to obtain a streamlined feature sequence, and a full zero vector is added to the end of the streamlined feature sequence to indicate the termination of the sequence, thereby obtaining the target feature sequence. Therefore, the second language conversion model uses the refined target feature sequence as the training target, which can improve the language refinement of the second language conversion model in converting natural language into machine language.

[0045] In a specific implementation scenario, the second sample sentences may include a number of long text sentences and short text sentences, thereby improving the compatibility of the second language conversion model with texts of different lengths.

[0046] Step S12: Processing the first feature sequence based on the intelligent dialogue model to obtain a second feature sequence expressed in machine language; wherein the second feature sequence includes feature information for replying to the first sentence.

[0047] In one implementation scenario, the intelligent dialogue tasks that the intelligent dialogue model can handle may include but are not limited to: knowledge question-and-answer tasks, text translation tasks, etc. As one possible example, the intelligent dialogue model is used to implement the knowledge question-and-answer task, the first sentence is a question text in a certain field, after the first sentence is converted into a first feature sequence, the second feature sequence obtained by processing the first feature sequence by the intelligent dialogue model may include feature information of the answer text used to reply to the first sentence. As another possible example, the intelligent dialogue model is used to implement the text translation task, the first sentence is a Chinese text, after the first sentence is converted into a first feature sequence, the second feature sequence obtained by processing the first feature sequence by the intelligent dialogue model may include feature information of the translated text in other languages ​​corresponding to the first sentence.

[0048] In one implementation scenario, the intelligent dialogue model may include but is not limited to: a large language model, a seq2seq model, etc.

[0049] In a specific implementation scenario, the large language model may include but is not limited to open source large models such as LLAMA and Bloom, and of course other large models. The structure of the large language model is not specifically limited here.

[0050] In one implementation scenario, at least one of the conversion from natural language to machine language and the conversion from machine language to natural language is implemented based on a language conversion model, and after the language conversion model training converges, the intelligent dialogue model is trained based on a second sample feature sequence represented in machine language, the second sample feature sequence is obtained by converting a third sample sentence represented in natural language based on the language conversion model, and the second sample feature sequence is annotated with a label feature sequence represented in machine language, and the label feature sequence contains feature information for replying to the third sample sentence. Therefore, in combination with the language conversion model trained and converged in the aforementioned embodiment, assisting the intelligent dialogue model in obtaining input data represented in machine language and / or assisting the intelligent dialogue model in restoring the output machine language data to the form of sentences can improve the accuracy and efficiency of the intelligent dialogue model in processing intelligent dialogues.

[0051] In a specific implementation scenario, in order to train an intelligent dialogue model, the second sample feature sequence can be processed by the intelligent dialogue model to obtain a third predicted feature sequence represented in machine language, and the network parameters of the intelligent dialogue model can be adjusted based on the difference between the third predicted feature sequence and the label feature sequence. Therefore, since the label feature sequence contains feature information for replying to the third sample sentence, the label feature sequence is used as the training target, which can train the intelligent dialogue model to have the ability to process machine language and the ability to achieve intelligent dialogue tasks.

[0052] Step S13: Based on the second feature sequence, convert and obtain a second sentence expressed in natural language and used to reply to the first sentence.

[0053] In one implementation scenario, in order to convert the second feature sequence into the second sentence, the second embedding representation of the initial character can be extracted, wherein the initial character is used to indicate the start of the conversion from machine language to natural language, and the second embedding representation and the second feature sequence are used for cyclic decoding to obtain the decoded characters output by each round of decoding, until the decoded character representation ends the conversion from machine language to natural language, and then based on the decoded characters output by each round of decoding, the second sentence can be obtained. Therefore, by indicating the start of the conversion from machine language to natural language with the initial character represented in natural language, and combining the initial character with the second feature sequence for cyclic decoding, an automated process from machine language to natural language can be achieved.

[0054] In a specific implementation scenario, the initial character may be a null character or other characters representing blank information. In addition, the extraction process of the second embedded representation of the initial character may refer to the extraction process of the first embedded representation, which will not be described in detail here.

[0055] In a specific implementation scenario, the feature dimension of the second embedding representation is the same as the feature dimension of the second feature sequence.

[0056] In a specific implementation scenario, the second embedded representation used for each round of cyclic decoding will change according to the different cyclic rounds. Specifically, the second embedded representation in the next round of the current round is obtained by updating the second embedded representation in the current round by the embedded representation of the decoded character in the current round. In order to obtain the decoded character of the current round, the second fused feature of the current round can be obtained by fusing the second embedded representation and the second feature sequence, and decoding is performed according to the second fused feature of the current round to obtain the decoded character of the current round. Therefore, the second embedded representation used for each round of cyclic decoding is related to the decoded character output in its previous round. By gradually adding the embedded representation of the decoded character to the second embedded representation, and performing cyclic decoding using the second feature sequence and the second embedded representation containing more and more feature information, the prediction accuracy of the decoded character can be gradually improved.

[0057] As a possible implementation, the second feature sequence is a vector sequence with a feature dimension of C and a sequence length of L, and the sequence length of the second embedded representation of the initial character is 1. The second embedded representation and the second feature sequence can be fused through the concat function to obtain a second fused feature. The second fused feature is a vector sequence with a feature dimension of C and a sequence length of L+1. After decoding according to the second fused feature of the first round to obtain the decoded character, the second embedded representation of the decoded character of the first round is fused with the second embedded representation of the initial character in the current round to obtain a second embedded representation with a sequence length of 2, and the fused second embedded representation is fused again with the second feature sequence through the concat function to obtain the second fused feature of the next round with a sequence length of L+2, and decoding is performed based on the second fused feature of the next round until the decoded character representation is converted from natural language to machine language.

[0058] As another possible implementation, the second embedded representation of the decoded characters of the current round can be directly fused with the second fused features of the next round to obtain the second fused features of the next round, so as to perform decoding based on the second fused features of the next round until the decoded character representation is converted from natural language to machine language.

[0059] In a specific implementation scenario, in the first round of cyclic decoding, in addition to the initial characters, several pre-set reference characters can also be included, and the second embedded representation of the first round is obtained based on the character string including the initial characters and the reference characters. The reference characters can provide more feature information in the process of converting from machine language to natural language, thereby improving the accuracy of converting from machine language to natural language.

[0060] In another implementation scenario, the second feature sequence can be converted into a second sentence in combination with the large language model. Specifically, a prompt is constructed by the second feature sequence and the second conversion requirement, and the second conversion requirement can include the number of characters, language, etc. represented by the second feature sequence converted into a machine language, thereby instructing the large language model to convert the second feature sequence into a second sentence according to the second conversion requirement.

[0061] In another implementation scenario, the second feature sequence can be directly decoded in a loop to obtain decoded characters output by each round of decoding until the decoded character representation is completely converted from natural language to machine language, thereby obtaining the second sentence through the decoded characters output by each round of decoding.

[0062] In one implementation scenario, the second sentence is predicted by a language conversion model, the language conversion model can realize the conversion from machine language to natural language, the language conversion model is trained by the first sample feature sequence represented by machine language, and the first sample feature sequence is annotated with a real sentence represented by natural language. In order to distinguish it from the first language conversion model and the second language conversion model in the aforementioned embodiment, the language conversion model obtained by training the first sample feature sequence can be expressed as a "third language conversion model". In order to train the third language conversion model, the first sample feature sequence can be converted by the third language conversion model to obtain a second predicted sentence represented by natural language, and the network parameters of the language conversion model are adjusted according to the difference between the second predicted sentence and the real sentence. Therefore, the third language conversion model takes the real sentence corresponding to the first sample feature sequence as the training target, and converts the second feature sequence into the second sentence through the trained third language conversion model, which can improve the accuracy and efficiency of the conversion from machine language to natural language. It should be noted that the specific process of converting the first sample feature sequence by using the language conversion model to obtain the second predicted sentence can refer to the process of converting the second feature sequence represented by machine language to obtain the second sentence represented by natural language in the aforementioned embodiment, which will not be repeated here.

[0063] In one implementation scenario, see Figure 2 , Figure 2 This is a framework diagram of an embodiment of the language conversion model of the present application. The aforementioned language conversion model (first language conversion model, second language conversion model, third language conversion model) includes a first input network, a second input network, a decoding prediction network, a first output network, and a second output network, and the input of the decoding prediction network is respectively connected to the first input network and the second input network, and the output of the decoding prediction network is respectively connected to the first output network and the second output network; wherein the first input network is used to obtain the embedded representation of the input string, the second input network is used to encode the input feature sequence, the decoding prediction network is used to predict the decoding features or the decoding characters, the first output network is used to output the output string represented in natural language after the input feature sequence is converted based on the decoding characters, and the second output network is used to output the output feature sequence represented in machine language after the input string is converted based on the decoding features. Therefore, under the model framework of the language conversion model, language conversion models for different purposes can be trained through different training methods, which can improve the flexibility of language conversion model training. As one possible example, when the input string is a first embedded representation and the input feature sequence is an initial feature sequence, the decoding prediction network is used to predict the decoding features; as another possible example, when the input string is a second embedded representation of initial characters and the input feature sequence is a second feature sequence, the decoding prediction network is used to predict the decoding characters.

[0064] In a specific implementation scenario, please refer to Figure 2 , Figure 3 , Figure 3 for Figure 2 A structural schematic diagram of an embodiment of a language conversion model in the embodiment of the invention, wherein the first input network includes a word embedding layer (embedding) for obtaining an embedding representation of an input string; the second input network includes a first linear layer linear(c*n) for converting a feature dimension of an input feature sequence represented in a machine language into a feature map with a feature dimension of n, so as to be sent to a decoding prediction network for processing; the decoding prediction network includes a large language model body (LLM) for predicting decoding features or decoding characters; the first output network includes a second linear layer linear(n*V) and a normalization layer softmax for further outputting an output string represented in a natural language after conversion from the input feature sequence according to the decoded characters output by the large language model body; the second output network includes a third linear layer linear(n*c) for converting the decoding features output by the large language model from feature dimension n to feature dimension c, and further outputting an output feature sequence represented in a machine language after conversion from the input string according to the decoding features of the feature dimension c. In addition, the language conversion model may also include a concat fusion layer, the input of the concat fusion layer is respectively connected to the word embedding layer and the first linear layer, the output of the concat fusion layer is connected to the input of the large language model body (LLM), the concat fusion layer is used to fuse the embedding representation of the input string with the input feature sequence after the feature dimension transformation, and output the fused features to the large language model body (LLM), and the output of the large language model body (LLM) is respectively connected to the second linear layer and the third linear layer. As another possible implementation, the decoding prediction network may also include a seq2seq model (Sequence to Sequence, sequence to sequence model).

[0065] In a specific implementation scenario, see Figure 4 , Figure 4 for Figure 2 The framework diagram of the language conversion model in different conversion modes, the language conversion model can realize the first conversion mode from natural language to machine language ( Figure 4 A mode in the left figure), and a second conversion mode ( Figure 4 In the right figure (mode B). Under the same language conversion model framework, the first conversion mode can be implemented by the first input network, the second input network, the decoding prediction network and the second output network, and the second conversion mode can be implemented by the first input network, the second input network, the decoding prediction network and the first output network.

[0066] In one implementation scenario, see Figure 5 , Figure 5 This is a schematic diagram of a framework of an embodiment of an intelligent dialogue system of the present application. The intelligent dialogue system includes a language conversion model A in a first conversion mode, an intelligent dialogue model and a language conversion model B in a second conversion mode, and the intelligent dialogue model is connected between the language conversion model A and the language conversion model B. Among them, the language conversion model A is used to convert natural language into machine language, the intelligent dialogue model is used to implement intelligent dialogue tasks based on machine language, the language conversion model B is used to convert machine language into natural language, the language conversion model A may include the first language conversion model or the second language conversion model in the aforementioned embodiment, and the language conversion model B may include the first language conversion model or the third language conversion model in the aforementioned embodiment.

[0067] In the above scheme, a first sentence expressed in natural language is converted to obtain a first feature sequence expressed in machine language, and the first feature sequence is processed based on the intelligent dialogue model to obtain a second feature sequence expressed in machine language, wherein the second feature sequence includes feature information for replying to the first sentence, and then based on the second feature sequence, a second sentence expressed in natural language and used to reply to the first sentence is converted. Since natural language requires a specific word order and specific words to express and transmit information, it is greatly affected by its own expression method and logic, while machine language expresses and transmits information in computer language, which is more suitable for the execution environment of the intelligent dialogue model. Therefore, before the intelligent dialogue model executes the intelligent dialogue task of the sentence, the sentence in natural language is converted into a feature sequence expressed in machine language. The intelligent dialogue model realizes the intelligent dialogue task by processing the feature sequence that is more suitable for it and replying with the feature sequence. That is to say, the intelligent dialogue model changes from a natural language to a natural language working mode to a feature sequence to a feature sequence working mode, that is, the intelligent dialogue model processes data in machine language throughout the process, so it can improve the accuracy and efficiency of intelligent dialogue.

[0068] See also Figure 6 , Figure 6 This is a framework diagram of an embodiment of an intelligent dialogue device of the present application, wherein the intelligent dialogue device 60 includes a first conversion module 61, a processing module 62 and a second conversion module 63; the first conversion module 61 is used to convert a first feature sequence expressed in a machine language based on a first sentence expressed in a natural language; the processing module 62 is used to process the first feature sequence based on an intelligent dialogue model to obtain a second feature sequence expressed in a machine language; wherein the second feature sequence includes feature information for replying to the first sentence; the second conversion module 63 is used to convert a second sentence expressed in a natural language and used for replying to the first sentence based on the second feature sequence.

[0069] In the above scheme, the intelligent dialogue device 60 converts the first sentence expressed in natural language into a first feature sequence expressed in machine language, and processes the first feature sequence based on the intelligent dialogue model to obtain a second feature sequence expressed in machine language, wherein the second feature sequence includes feature information for replying the first sentence, and then converts the second sentence expressed in natural language and used for replying the first sentence based on the second feature sequence. Since natural language requires a specific word order and specific words to express and transmit information, it is greatly affected by its own expression method and logic, while machine language expresses and transmits information in computer language, which is more suitable for the execution environment of the intelligent dialogue model. Therefore, before the intelligent dialogue model executes the intelligent dialogue task of the sentence, the sentence in natural language is converted into a feature sequence expressed in machine language. The intelligent dialogue model realizes the intelligent dialogue task by processing the feature sequence that is more suitable for it and replying with the feature sequence. That is to say, the intelligent dialogue model changes from the working mode of natural language to natural language to the working mode of feature sequence to feature sequence, that is, the intelligent dialogue model processes data in machine language throughout the process, so the accuracy and efficiency of the intelligent dialogue can be improved.

[0070] In some disclosed embodiments, the first conversion module 61 is specifically used to: extract a first embedded representation of the first sentence and obtain an initial feature sequence; wherein the initial feature sequence includes initial features expressed in machine language, and the initial features are used to indicate the start of conversion from natural language to machine language; perform cyclic decoding based on the first embedded representation and the initial feature sequence to obtain decoding features of each round of decoding output until the decoding feature representation is converted from natural language to machine language; obtain a first feature sequence based on the decoding features of each round of decoding output.

[0071] Therefore, by starting the conversion from natural language to machine language with an initial feature indication represented in machine language, and performing cyclic decoding in combination with the initial feature sequence and the first embedded representation of the first sentence, an automated process from natural language to machine language can be achieved.

[0072] In some disclosed embodiments, when the first conversion module 61 is used to perform cyclic decoding based on the first embedded representation and the initial feature sequence to obtain decoding features output in each round of decoding, the first conversion module 61 is also specifically used to: perform fusion based on the first embedded representation and the initial feature sequence to obtain the first fused features of the current round; perform decoding based on the first fused features of the current round to obtain the decoding features of the current round; wherein the initial feature sequence in the next round of the current round is obtained by fusing the decoding features of the current round with the initial feature sequence in the current round.

[0073] Therefore, the initial feature sequence used for each round of cyclic decoding is related to the decoding features outputted in the previous round. By gradually adding decoding features to the initial feature sequence and performing cyclic decoding using the first embedded representation and the initial feature sequence containing more and more feature information, the prediction accuracy of the decoding features can be gradually improved.

[0074] In some disclosed embodiments, the second conversion module 63 is specifically used to: extract a second embedded representation of an initial character; wherein the initial character is used to indicate the start of conversion from machine language to natural language; perform cyclic decoding based on the second embedded representation and the second feature sequence to obtain decoded characters output by each round of decoding, until the decoded character representation is converted from machine language to natural language; obtain a second sentence based on the decoded characters output by each round of decoding.

[0075] Therefore, by starting the conversion from machine language to natural language with an initial character indication represented in natural language, and performing cyclic decoding in combination with the initial character and the second feature sequence, an automated process from machine language to natural language can be achieved.

[0076] In some disclosed embodiments, when the second conversion module 63 is used to perform cyclic decoding based on the second embedded representation and the second feature sequence to obtain decoded characters outputted in each round of decoding, the second conversion module 63 is also specifically used to: fuse based on the second embedded representation and the second feature sequence to obtain the second fused features of the current round; decode based on the second fused features of the current round to obtain the decoded characters of the current round; wherein the second embedded representation in the next round of the current round is obtained by updating the second embedded representation in the current round by the embedded representation of the decoded characters in the current round.

[0077] Therefore, the second embedded representation used for each round of cyclic decoding is related to the decoded character output in the previous round. By gradually adding the embedded representation of the decoded character to the second embedded representation and performing cyclic decoding using the second feature sequence and the second embedded representation containing more and more feature information, the prediction accuracy of the decoded character can be gradually improved.

[0078] In some disclosed embodiments, at least one of converting from natural language to machine language and converting from machine language to natural language is implemented based on a language conversion model.

[0079] Therefore, by implementing the language conversion model to convert natural language into machine language and / or machine language into natural language, the efficiency of language conversion can be improved.

[0080] In some disclosed embodiments, conversion from natural language to machine language and from machine language to natural language are both implemented based on a language conversion model. The intelligent dialogue device 60 also includes a first training module, which is used to train a language conversion model based on a first sample sentence expressed in natural language. The steps of training the language conversion model by the first training module include: converting the first sample sentence based on the language conversion model to obtain a first prediction feature sequence expressed in machine language; converting the first prediction feature sequence based on the language conversion model to obtain a first prediction sentence expressed in natural language; and adjusting the network parameters of the language conversion model based on at least a first loss measured by the difference between the first sample sentence and the first prediction sentence.

[0081] Therefore, by adjusting the network parameters of the language conversion model based on the difference between the first sample sentence initially input into the language conversion model and the first predicted sentence finally output by the language conversion model, the language conversion model can be trained to be able to restore natural language after converting it into machine language, thereby improving the accuracy of the language conversion model in converting between natural language and machine language.

[0082] In some disclosed embodiments, when the number of characters in the first sample sentence is greater than a preset number of characters, the first training module is further used to obtain a second loss based on at least one of a first sub-loss and a second sub-loss before adjusting the network parameters of the language conversion model based at least on the first loss; wherein the first sub-loss is positively correlated with the sum of the absolute values ​​of each feature element in the first prediction feature sequence and negatively correlated with the first total number of feature elements, and the second sub-loss is positively correlated with the second total number of valid feature elements in the first prediction feature sequence and negatively correlated with the first total number, and the valid feature elements represent feature elements in the first prediction feature sequence that are greater than a preset threshold; and the network parameters of the language conversion model are adjusted based on the first loss and the second loss.

[0083] Therefore, when the number of characters in the first sample sentence is greater than the preset number of characters, training is performed using long text training data, and a loss function is designed based on the first prediction feature sequence. This can promote the language conversion model to learn to restore natural language information while learning the ability to compress natural language information.

[0084] In some disclosed embodiments, at least the conversion from natural language to machine language is implemented based on a language conversion model, and the intelligent dialogue device 60 also includes a second training module, which is used to train a language conversion model based on a second sample sentence expressed in natural language, and the second sample sentence is annotated with a target feature sequence expressed in machine language. The steps of the second training module for training the language conversion model include: converting the second sample sentence based on the language conversion model to obtain a second predicted feature sequence expressed in machine language; and adjusting the network parameters of the language conversion model based on the difference between the second predicted feature sequence and the target feature sequence.

[0085] Therefore, taking the labeled target feature sequence as the training target and converting the first sentence into the first feature sequence through the language conversion model can improve the accuracy and efficiency of converting natural language into machine language.

[0086] In some disclosed embodiments, at least the conversion from machine language to natural language is implemented based on a language conversion model, and the intelligent dialogue device 60 also includes a third training module, which is used to train a language conversion model based on a first sample feature sequence expressed in machine language, and the first sample feature sequence is annotated with a real sentence expressed in natural language. The third training module is used to train the language conversion model in the following steps: converting the first sample feature sequence based on the language conversion model to obtain a second predicted sentence expressed in natural language; and adjusting the network parameters of the language conversion model based on the difference between the second predicted sentence and the real sentence.

[0087] Therefore, the third language conversion model takes the real sentence corresponding to the first sample feature sequence as the training target, and converts the second feature sequence into the second sentence through the trained third language conversion model, which can improve the accuracy and efficiency of converting machine language into natural language.

[0088] In some disclosed embodiments, the language conversion model includes a first input network, a second input network, a decoding prediction network, a first output network, and a second output network, and the input of the decoding prediction network is respectively connected to the first input network and the second input network, and the output of the decoding prediction network is respectively connected to the first output network and the second output network; wherein the first input network is used to obtain an embedded representation of an input string, the second input network is used to encode an input feature sequence, the decoding prediction network is used to predict decoding features or decoding characters, the first output network is used to output an output string represented in a natural language after conversion of the input feature sequence based on the decoding characters, and the second output network is used to output an output feature sequence represented in a machine language after conversion of the input string based on the decoding features.

[0089] Therefore, under the model framework of the language conversion model, language conversion models for different purposes can be trained through different training methods, which can improve the flexibility of language conversion model training.

[0090] In some disclosed embodiments, at least one of conversion from natural language to machine language and conversion from machine language to natural language is implemented based on a language conversion model, and after the language conversion model training converges, the intelligent dialogue model is trained based on a second sample feature sequence represented in machine language, the second sample feature sequence is obtained by converting a third sample sentence represented in natural language based on the language conversion model, the second sample feature sequence is annotated with a label feature sequence represented in machine language, and the label feature sequence includes feature information for replying to the third sample sentence.

[0091] Therefore, combined with the language conversion model that has converged in training, assisting the intelligent dialogue model in obtaining input data represented in machine language and / or assisting the intelligent dialogue model in restoring the output machine language data to the form of sentences can improve the accuracy and efficiency of the intelligent dialogue model in processing intelligent dialogues.

[0092] In some disclosed embodiments, the intelligent dialogue device 60 also includes a fourth training module, which is used to train an intelligent dialogue model. The training steps of the fourth training module for training the intelligent dialogue model include: processing the second sample feature sequence based on the intelligent dialogue model to obtain a third predicted feature sequence expressed in machine language; and adjusting the network parameters of the intelligent dialogue model based on the difference between the third predicted feature sequence and the label feature sequence.

[0093] Therefore, since the label feature sequence contains feature information for replying to the third sample sentence, using the label feature sequence as the training target can train the intelligent dialogue model to have the ability to process machine language while training the intelligent dialogue model to have the ability to perform intelligent dialogue tasks.

[0094] See also Figure 7 , Figure 7 Schematic diagram of the framework of an embodiment of an electronic device 70 of the present application. The electronic device 70 includes a memory 71 and a processor 72 coupled to each other, and the processor 72 is used to execute program instructions stored in the memory 71 to implement the steps in any of the above-mentioned intelligent dialogue method embodiments. Specifically, the electronic device 70 may include but is not limited to: a microcomputer, a server, a laptop computer, a tablet computer, a smart phone, etc., which are not limited here.

[0095] Specifically, the processor 72 is used to control itself and the memory 71 to implement the steps of any of the above-mentioned intelligent dialogue method embodiments. The processor 72 can also be called a CPU (Central Processing Unit). The processor 72 may be an integrated circuit chip with signal processing capabilities. The processor 72 can also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 72 can be implemented by an integrated circuit chip.

[0096] In the above scheme, the electronic device 70 converts the first sentence expressed in natural language into a first feature sequence expressed in machine language, and processes the first feature sequence based on the intelligent dialogue model to obtain a second feature sequence expressed in machine language, wherein the second feature sequence includes feature information for replying to the first sentence, and then converts the second sentence expressed in natural language and used for replying to the first sentence based on the second feature sequence. Since natural language requires a specific word order and specific words to express and transmit information, it is greatly affected by its own expression method and logic, while machine language expresses and transmits information in computer language, which is more suitable for the execution environment of the intelligent dialogue model. Therefore, before the intelligent dialogue model executes the intelligent dialogue task of the sentence, the sentence in natural language is converted into a feature sequence expressed in machine language. The intelligent dialogue model realizes the intelligent dialogue task by processing the feature sequence that is more suitable for it and replying with the feature sequence. That is to say, the intelligent dialogue model changes from the working mode of natural language to natural language to the working mode of feature sequence to feature sequence, that is, the intelligent dialogue model processes data in machine language throughout the process, so it can improve the accuracy and efficiency of intelligent dialogue.

[0097] See also Figure 8 , Figure 8 This is a schematic diagram of a framework of an embodiment of a computer-readable storage medium 80 of the present application. The computer-readable storage medium 80 stores program instructions 81 that can be executed by a processor, and the program instructions 81 are used to implement the steps of the above-mentioned intelligent dialogue method embodiment.

[0098] In the above scheme, the computer-readable storage medium 80 converts the first sentence expressed in natural language into a first feature sequence expressed in machine language, and processes the first feature sequence based on the intelligent dialogue model to obtain a second feature sequence expressed in machine language, wherein the second feature sequence includes feature information for replying to the first sentence, and then converts the second sentence expressed in natural language and used for replying to the first sentence based on the second feature sequence. Since natural language requires a specific word order and specific words to express and transmit information, it is greatly affected by its own expression method and logic, while machine language expresses and transmits information in computer language, which is more suitable for the execution environment of the intelligent dialogue model. Therefore, before the intelligent dialogue model executes the intelligent dialogue task of the sentence, the sentence in natural language is converted into a feature sequence expressed in machine language. The intelligent dialogue model realizes the intelligent dialogue task by processing the feature sequence that is more suitable for it and replying with the feature sequence. That is to say, the intelligent dialogue model changes from the working mode of natural language to natural language to the working mode of feature sequence to feature sequence, that is, the intelligent dialogue model performs data processing in machine language throughout the process, so it can improve the accuracy and efficiency of intelligent dialogue.

[0099] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0100] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

[0101] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0102] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0103] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0104] If the technical solution of this application involves personal information, the product using the technical solution of this application has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using the technical solution of this application has obtained the individual's separate consent before processing the sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, clear and prominent signs are set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that he or she agrees to the collection of his or her personal information; or on the device that processes personal information, the personal information processing rules are notified by obvious signs / information, and the individual's authorization is obtained through pop-up information or by asking the individual to upload his or her personal information; among them, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

Claims

1. An intelligent dialogue method, characterized in that: include: Based on the first sentence expressed in the natural language, converting to obtain a first feature sequence expressed in the machine language; Processing the first feature sequence based on the intelligent dialogue model to obtain a second feature sequence represented in machine language; wherein the second feature sequence includes feature information for replying to the first sentence; Based on the second feature sequence, a second sentence represented in a natural language and used to reply to the first sentence is converted.

2. The method according to claim 1, characterized in that The converting, based on the first sentence expressed in the natural language, to obtain a first feature sequence expressed in the machine language comprises: Extracting a first embedding representation of the first sentence and obtaining an initial feature sequence; wherein the initial feature sequence includes initial features represented in machine language, and the initial features are used to indicate the start of conversion from natural language to machine language; Performing cyclic decoding based on the first embedded representation and the initial feature sequence to obtain decoding features outputted by each round of decoding, until the decoding feature representation is completely converted from natural language to machine language; The first feature sequence is obtained based on the decoding features output by each round of decoding.

3. The method according to claim 2, characterized in that The step of performing cyclic decoding based on the first embedded representation and the initial feature sequence to obtain decoding features outputted by each round of decoding includes: Based on the first embedding representation and the initial feature sequence, a fusion is performed to obtain a first fused feature of the current round; Decoding is performed based on the first fusion feature of the current round to obtain a decoding feature of the current round; The initial feature sequence in the next round of the current round is obtained by fusing the decoding features of the current round with the initial feature sequence in the current round.

4. The method according to claim 1, characterized in that: The converting, based on the second feature sequence, into a second sentence expressed in a natural language and used to reply to the first sentence comprises: Extracting a second embedded representation of an initial character; wherein the initial character is used to indicate the start of conversion from machine language to natural language; Performing cyclic decoding based on the second embedded representation and the second feature sequence to obtain decoded characters output by each round of decoding, until the decoded character representation is converted from machine language to natural language; The second sentence is obtained based on the decoded characters outputted from each round of decoding.

5. The method according to claim 4, characterized in that The step of performing cyclic decoding based on the second embedded representation and the second feature sequence to obtain decoded characters outputted by each round of decoding includes: Fusing the second embedding representation and the second feature sequence to obtain a second fused feature of the current round; Decoding is performed based on the second fused feature of the current round to obtain a decoded character of the current round; The second embedded representation in the next round of the current round is obtained by updating the second embedded representation in the current round with the embedded representation of the decoded character in the current round.

6. The method according to claim 1, characterized in that At least one of converting from natural language to machine language and converting from machine language to natural language is implemented based on a language conversion model.

7. The method according to claim 6, characterized in that The conversion from natural language to machine language and from machine language to natural language are both implemented based on the language conversion model. The language conversion model is trained by a first sample sentence expressed in natural language. The training steps of the language conversion model include: Converting the first sample sentence based on the language conversion model to obtain a first prediction feature sequence represented in machine language; Converting the first prediction feature sequence based on the language conversion model to obtain a first prediction sentence expressed in natural language; The network parameters of the language conversion model are adjusted based on at least a first loss measured by the difference between the first sample sentence and the first predicted sentence.

8. The method according to claim 7, characterized in that When the number of characters in the first sample sentence is greater than a preset number of characters, before adjusting the network parameters of the language conversion model based at least on the first loss, the method further includes: A second loss is obtained based on at least one of the first sub-loss and the second sub-loss; wherein the first sub-loss is positively correlated with the sum of the absolute values ​​of the feature elements in the first prediction feature sequence and negatively correlated with the first total number of the feature elements, and the second sub-loss is positively correlated with the second total number of valid feature elements in the first prediction feature sequence and negatively correlated with the first total number, and the valid feature elements represent feature elements in the first prediction feature sequence that are greater than a preset threshold; The adjusting the network parameters of the language conversion model based at least on the first loss includes: Based on the first loss and the second loss, a network parameter of the language conversion model is adjusted.

9. The method according to claim 6, characterized in that At least the conversion from natural language to machine language is implemented based on the language conversion model, the language conversion model is trained by a second sample sentence expressed in natural language, and the second sample sentence is annotated with a target feature sequence expressed in machine language, and the training steps of the language conversion model include: Converting the second sample sentence based on the language conversion model to obtain a second prediction feature sequence represented in machine language; Based on the difference between the second predicted feature sequence and the target feature sequence, the network parameters of the language conversion model are adjusted.

10. The method according to claim 6, characterized in that At least the conversion from machine language to natural language is implemented based on the language conversion model, the language conversion model is trained by a first sample feature sequence represented in machine language, and the first sample feature sequence is annotated with a real sentence represented in natural language, and the training steps of the language conversion model include: Converting the first sample feature sequence based on the language conversion model to obtain a second prediction sentence expressed in natural language; Based on the difference between the second predicted sentence and the true sentence, the network parameters of the language conversion model are adjusted.

11. The method according to claim 6, characterized in that The language conversion model includes a first input network, a second input network, a decoding prediction network, a first output network, and a second output network, and the input of the decoding prediction network is connected to the first input network and the second input network respectively, and the output of the decoding prediction network is connected to the first output network and the second output network respectively; Among them, the first input network is used to obtain the embedded representation of the input string, the second input network is used to encode the input feature sequence, the decoding prediction network is used to predict the decoding features or decoding characters, the first output network is used to output the output string represented in natural language after the input feature sequence is converted based on the decoding characters, and the second output network is used to output the output feature sequence represented in machine language after the input string is converted based on the decoding features.

12. The method according to claim 1, characterized in that At least one of conversion from natural language to machine language and conversion from machine language to natural language is implemented based on a language conversion model, and after the training of the language conversion model converges, the intelligent dialogue model is trained based on a second sample feature sequence expressed in machine language, the second sample feature sequence is obtained by converting a third sample sentence expressed in natural language based on the language conversion model, the second sample feature sequence is annotated with a label feature sequence expressed in machine language, and the label feature sequence contains feature information for replying to the third sample sentence.

13. The method according to claim 12, characterized in that The training steps of the intelligent dialogue model include: Processing the second sample feature sequence based on the intelligent dialogue model to obtain a third predicted feature sequence represented in machine language; Based on the difference between the third predicted feature sequence and the label feature sequence, the network parameters of the intelligent dialogue model are adjusted.

14. An intelligent dialogue device, characterized in that: include: A first conversion module, configured to convert a first sentence expressed in a natural language into a first feature sequence expressed in a machine language; A processing module, configured to process the first feature sequence based on an intelligent dialogue model to obtain a second feature sequence represented in a machine language; wherein the second feature sequence includes feature information for replying to the first sentence; The second conversion module is used to convert, based on the second feature sequence, a second sentence expressed in natural language and used to reply to the first sentence.

15. An electronic device, characterized in that: It comprises a memory and a processor coupled to each other, wherein the processor is used to execute program instructions stored in the memory to implement the intelligent dialogue method according to any one of claims 1 to 13.

16. A computer-readable storage medium having program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the intelligent dialogue method according to any one of claims 1 to 13 is implemented.

Images