A method and apparatus for generating statement information

By using a sequence-to-sequence model encoder in the banking robot service, combined with randomness and custom sentence information to generate diverse responses, the problem of banking robots providing only negative responses is solved, thus improving customer experience and communication efficiency.

CN118277528BActive Publication Date: 2026-11-10INDUSTRIAL AND COMMERCIAL BANK OF CHINA
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202410393594.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-04-02
Publication Date
2026-11-10
Estimated Expiration
2044-04-02

AI Technical Summary

Technical Problem

Bank robots tend to provide one-dimensional and negative responses in customer service, resulting in poor customer experience, increased demand for human service and labor costs, and reduced communication efficiency.

Method used

An encoder from a sequence-to-sequence model is used, combined with randomness and custom sentence information. By adding random values ​​to sentence space variables, a greedy search method is used to generate diverse response statements. The model is then trained to improve response diversity and customer interest.

Benefits of technology

It improved the efficiency of communication services between banks and customers, enhanced customer experience, reduced the need for manual services, and lowered labor costs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118277528B_ABST
    Figure CN118277528B_ABST
Patent Text Reader

Abstract

This invention provides a method and apparatus for generating statement information, relating to the field of artificial intelligence technology, and applicable to the financial sector or other technical fields. The method includes: acquiring text information and sentence function information from a user dialogue; processing the text information and sentence function information based on a pre-trained model to obtain sentence space variables; adding random values ​​to the sentence space variables to obtain initial input variables for decoding; and processing the initial input variables based on a decoder in a sequence-to-sequence model to obtain response statement information for replying to the text information. The apparatus executes the above method. The statement information generation method and apparatus provided by this invention can improve the efficiency of communication services between banks and customers.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to a method and apparatus for generating statement information. Background Technology

[0002] In the field of text generation, the most widely used model is Seq2Seq, which stands for Sequence to Sequence. It's a method that can generate another sequence from a given sequence using specific generation techniques, and these two sequences can be of unequal length. For example, in human-computer dialogue, if a user asks a robot, "Who are you?", the robot will return the answer, "I am [user's name]".

[0003] However, in scenarios where banks use robots to serve customers, robots tend to give the same one-dimensional answers to the same questions, and sometimes their responses contain negative emotions and question-like phrases, leading to a poor customer experience. This results in a poor customer experience with the bank's services, which is not conducive to the bank's customer maintenance efforts. In addition, after a poor customer experience, customers will turn to human service, increasing labor costs and reducing the efficiency of communication and service between the bank and its customers. Summary of the Invention

[0004] To address the problems in the prior art, embodiments of the present invention provide a method and apparatus for generating statement information, which can at least partially solve the problems existing in the prior art.

[0005] On the one hand, this invention proposes a method for generating statement information, including:

[0006] Obtain text information and sentence function information from user dialogues, and process the text information and sentence function information based on a pre-trained model to obtain sentence space variables;

[0007] The model is an encoder in a sequence-to-sequence model, and the sentence functional information is random sentence information and / or custom sentence information used to increase the diversity of response statements;

[0008] Random values ​​are added to the sentence space variables to obtain the initial input variables for decoding;

[0009] The random value is a value randomly generated within a preset iteration period before the model training is completed, with the upper limit being the average Euclidean distance between the first spatial variable corresponding to the original first encoder used for sequence transformation and the second spatial variable corresponding to the second encoder that receives the original response statement.

[0010] The initial input variables are processed by the decoder in the sequence-to-sequence model to obtain response statement information for replying to the text information.

[0011] The step of adding random values ​​to the sentence space variables to obtain the initial input variables for decoding includes:

[0012] The initial input variables are obtained by adding random values ​​based on a normal distribution to each dimension of the sentence space variables.

[0013] The step of processing the initial input variables based on the decoder in the sequence-to-sequence model to obtain response statement information for replying to the text information includes:

[0014] Based on the decoder in the sequence-to-sequence model, and by using a greedy search method to process the initial input variables, the response statement information is obtained.

[0015] The statement information generation method further includes:

[0016] The model is trained using the first encoder and its corresponding first decoder, and the second encoder and its corresponding second decoder.

[0017] The step of training the model using the first encoder and its corresponding first decoder, and the second encoder and its corresponding second decoder, includes:

[0018] The first spatial variable is obtained by processing the text sample information and sentence function sample information based on the first encoder; the second spatial variable is obtained by processing the original response statement based on the second encoder.

[0019] Add the first latent variable corresponding to the first spatial variable to the first spatial variable and output the first response statement information through the first decoder; add the second latent variable corresponding to the second spatial variable to the second spatial variable and output the second response statement information through the second decoder.

[0020] The first encoder is trained based on the first response statement information, the second response statement information, and a preset loss function to obtain the encoder in the sequence-to-sequence model.

[0021] The construction of the preset loss function includes:

[0022] Based on the first spatial variable, the first decoder, and the first response statement information, a first preset loss function based on information entropy is constructed.

[0023] Based on the second spatial variable, the second decoder, and the second response statement information, a second preset loss function based on information entropy is constructed.

[0024] Based on the Euclidean distance between the first spatial variable and the second spatial variable, a third preset loss function is constructed;

[0025] The triplet information is obtained based on the second spatial variable, and a fourth preset loss function is constructed based on the triplet loss method and the triplet information.

[0026] Based on the KL divergence of the first latent variable and the second latent variable, a fifth preset loss function is constructed;

[0027] Based on the second latent variable, the second response statement information, and the sentence function sample information, a sixth preset loss function based on cross-entropy classification loss is constructed;

[0028] The preset loss function is constructed based on the first preset loss function, the second preset loss function, the third preset loss function, the fourth preset loss function, the fifth preset loss function, and the sixth preset loss function.

[0029] The step of training the first encoder based on the first response statement information, the second response statement information, and a preset loss function to obtain the encoder in the sequence-to-sequence model includes:

[0030] Based on the first response statement information, the second response statement information, and the preset loss function, and by training the first encoder using a planned sampling mechanism, the encoder in the sequence-to-sequence model is obtained.

[0031] On one hand, the present invention proposes a statement information generation device, comprising:

[0032] The acquisition unit is used to acquire text information and sentence function information of user dialogue, and process the text information and sentence function information based on a pre-trained model to obtain sentence space variables;

[0033] The model is an encoder in a sequence-to-sequence model, and the sentence functional information is random sentence information and / or custom sentence information used to increase the diversity of response statements;

[0034] An addition unit is used to add random values ​​to the sentence space variables to obtain the initial input variables for decoding;

[0035] The random value is a value randomly generated within a preset iteration period before the model training is completed, with the upper limit being the average Euclidean distance between the first spatial variable corresponding to the original first encoder used for sequence transformation and the second spatial variable corresponding to the second encoder that receives the original response statement.

[0036] The generation unit is used to process the initial input variables based on the decoder in the sequence-to-sequence model to obtain response statement information for replying to the text information.

[0037] In another aspect, embodiments of the present invention provide an electronic device, including: a processor, a memory, and a bus, wherein,

[0038] The processor and the memory communicate with each other via the bus;

[0039] The memory stores program instructions that can be executed by the processor, and the processor can execute the following methods by calling the program instructions:

[0040] Obtain text information and sentence function information from user dialogues, and process the text information and sentence function information based on a pre-trained model to obtain sentence space variables;

[0041] The model is an encoder in a sequence-to-sequence model, and the sentence functional information is random sentence information and / or custom sentence information used to increase the diversity of response statements;

[0042] Random values ​​are added to the sentence space variables to obtain the initial input variables for decoding;

[0043] The random value is a value randomly generated within a preset iteration period before the model training is completed, with the upper limit being the average Euclidean distance between the first spatial variable corresponding to the original first encoder used for sequence transformation and the second spatial variable corresponding to the second encoder that receives the original response statement.

[0044] The initial input variables are processed by the decoder in the sequence-to-sequence model to obtain response statement information for replying to the text information.

[0045] This invention provides a non-transitory computer-readable storage medium, comprising:

[0046] The non-transitory computer-readable storage medium stores computer instructions that cause the computer to perform the following methods:

[0047] Obtain text information and sentence function information from user dialogues, and process the text information and sentence function information based on a pre-trained model to obtain sentence space variables;

[0048] The model is an encoder in a sequence-to-sequence model, and the sentence functional information is random sentence information and / or custom sentence information used to increase the diversity of response statements;

[0049] Random values ​​are added to the sentence space variables to obtain the initial input variables for decoding;

[0050] The random value is a value randomly generated within a preset iteration period before the model training is completed, with the upper limit being the average Euclidean distance between the first spatial variable corresponding to the original first encoder used for sequence transformation and the second spatial variable corresponding to the second encoder that receives the original response statement.

[0051] The initial input variables are processed by the decoder in the sequence-to-sequence model to obtain response statement information for replying to the text information.

[0052] The sentence information generation method and apparatus provided in this invention acquire text information and sentence function information of user dialogue, process the text information and sentence function information based on a pre-trained model to obtain sentence space variables; wherein, the model is an encoder in a sequence-to-sequence model, and the sentence function information is random sentence information and / or custom sentence information used to increase the diversity of response sentences; random values ​​are added to the sentence space variables to obtain initial input variables for decoding; wherein, the random values ​​are values ​​randomly generated within a preset iteration period before the model training is completed, with the upper limit being the average Euclidean distance between the first space variable corresponding to the original first encoder used for sequence conversion and the second space variable corresponding to the second encoder receiving the original response sentence; the decoder in the sequence-to-sequence model processes the initial input variables to obtain response sentence information for replying to the text information. By generating diverse response sentences, the communication service efficiency between banks and customers can be improved, which is beneficial for banks to carry out customer maintenance work. Attached Figure Description

[0053] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:

[0054] Figure 1 This is a flowchart illustrating a statement information generation method provided in an embodiment of the present invention.

[0055] Figure 2 This is a flowchart illustrating a statement information generation method provided in another embodiment of the present invention.

[0056] Figure 3 This is a flowchart illustrating a statement information generation method provided in another embodiment of the present invention.

[0057] Figure 4 This is a schematic diagram of the structure of a statement information generation device provided in an embodiment of the present invention.

[0058] Figure 5 This is a schematic diagram of the physical structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. Here, the illustrative embodiments and descriptions of the present invention are used to explain the present invention, but are not intended to limit the present invention. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be arbitrarily combined with each other.

[0060] Figure 1 This is a flowchart illustrating a statement information generation method according to an embodiment of the present invention, as shown below. Figure 1 As shown, the statement information generation method provided in this embodiment of the invention includes:

[0061] Step S1: Obtain text information and sentence function information of user dialogue, process the text information and sentence function information based on the pre-trained model, and obtain sentence space variables;

[0062] The model is a sequence-to-sequence encoder, and the sentence functional information is randomized sentence information and / or custom sentence information used to increase the diversity of response statements.

[0063] Step S2: Add random values ​​to the sentence space variables to obtain the initial input variables for decoding;

[0064] The random value is a value randomly generated within a preset iteration period before the model training is completed, with the upper limit being the average Euclidean distance between the first spatial variable corresponding to the original first encoder used for sequence transformation and the second spatial variable corresponding to the second encoder that receives the original response statement.

[0065] Step S3: Process the initial input variables based on the decoder in the sequence-to-sequence model to obtain response statement information for replying to the text information.

[0066] In step S1 above, the device acquires text information and sentence function information of the user's dialogue, processes the text information and sentence function information based on a pre-trained model, and obtains sentence space variables.

[0067] The model is a sequence-to-sequence encoder, and the sentence function information consists of randomized sentence information and / or custom sentence information used to increase the diversity of response statements. The device can be a computer device executing the method, for example, it may include a server, specifically a server for a control robot that interacts with customers. It should be noted that the information collected in this embodiment of the invention is information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure, and application of related data all comply with the relevant laws, regulations, and standards of relevant countries and regions, necessary confidentiality measures have been taken, public order and good morals are not violated, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0068] Provide users with corresponding operation entry points, allowing them to choose to agree to or reject the automated decision results; if the user chooses to reject, the process will proceed to the expert decision-making process.

[0069] The application scenario of the method of the present invention can be in a bank service hall. While customers are queuing to handle business, the robot can serve customers by chatting with them. On the one hand, customers can learn about the information they need to prepare for handling business (including taking a number, preparing bank cards and ID numbers, etc.) through conversation with the robot, which helps to speed up the subsequent business handling process. On the other hand, customers can obtain some interesting content by talking with the robot, which can help pass the boring waiting time. Therefore, the application scenario of the method of the present invention can be understood as an on-site business handling scenario that assists in handling business.

[0070] The text information of user dialogue can be understood as the text information of the dialogue between customers who come to the bank service hall to conduct business and the robot.

[0071] Sentence functional information can include random sentence information, such as one or several sentences randomly obtained from a humorous corpus.

[0072] The sentence function information can also include custom sentence information, such as sentences that do not contain interrogative sentences or negative emotional sentences, as set by relevant bank staff. It can also include sentences that warn customers about business processing risks, such as reminding customers not to believe anyone's claims of low-interest loans or high-yield financial management information provided by any platform. It can also include sentences that assist in business processing, such as reminding customers to bring their ID cards with their funds and to keep their queue number receipt.

[0073] like Figure 2 As shown, textual information and sentence function information are input into the encoder in the sequence model, and the output of the encoder in the sequence model is used as the sentence space variable.

[0074] In step S2 above, the device adds random values ​​to the sentence space variables to obtain the initial input variables for decoding;

[0075] The random value is a value randomly generated within a preset iteration period before the model training is completed, with the upper limit being the average Euclidean distance between the first spatial variable corresponding to the original first encoder used for sequence transformation and the second spatial variable corresponding to the second encoder receiving the original response statement. The preset iteration period can be set independently according to the actual situation. If the model training has undergone a total of 500 iterations, the period from the 400th iteration to the 500th iteration can be determined as the preset iteration period.

[0076] Each iteration within the aforementioned preset iteration period calculates the Euclidean distance between the first spatial variable and the second spatial variable, thereby calculating the average Euclidean distance over the entire preset iteration period.

[0077] By generating random values ​​with the mean Euclidean distance as the upper limit, it is possible to ensure that the variables have small deviations and that the generated vectors are still in the same cluster in space.

[0078] The original first encoder used for sequence conversion was Figure 3 The Encoder1 shown is the original second encoder used for sequence conversion. Figure 3 The Encoder2 shown.

[0079] The step of adding random values ​​to the sentence space variables to obtain the initial input variables for decoding includes:

[0080] The initial input variable is obtained by adding random values ​​based on a normal distribution to each dimension of the sentence space variable. The initial input variable Z′ can be expressed as follows:

[0081] Z′=Z′ s2s +r

[0082] Among them, Z′ s2s Let r be a sentence space variable, and r be a random value that can conform to a normal distribution.

[0083] In step S3 above, the device processes the initial input variables based on the decoder in the sequence-to-sequence model to obtain response statement information for replying to the text information. For example... Figure 2 As shown, the initial input variables can be input into the decoder in the sequence model, and the output of the decoder in the sequence model can be used as the response statement information to reply to the text information.

[0084] The decoder in the sequence-to-sequence model processes the initial input variables to obtain response statement information for replying to the text information, including:

[0085] Based on the decoder in the sequence-to-sequence model, and by processing the initial input variables using a greedy search method, the response statement information is obtained. That is, a greedy search is used, where the word with the highest probability is selected and output at each step, ultimately forming the entire sentence output.

[0086] The statement information generation method further includes:

[0087] The model is trained using the first encoder and its corresponding first decoder, and the second encoder and its corresponding second decoder.

[0088] Training the model using the first encoder and its corresponding first decoder, and the second encoder and its corresponding second decoder, includes:

[0089] The first spatial variable is obtained by processing the text sample information and sentence function sample information based on the first encoder; the second spatial variable is obtained by processing the original response statement based on the second encoder.

[0090] Add the first latent variable corresponding to the first spatial variable to the first spatial variable and output the first response statement information through the first decoder; add the second latent variable corresponding to the second spatial variable to the second spatial variable and output the second response statement information through the second decoder.

[0091] The first encoder is trained based on the first response statement information, the second response statement information, and a preset loss function to obtain the encoder in the sequence-to-sequence model.

[0092] Constructing the preset loss function includes:

[0093] Based on the first spatial variable, the first decoder, and the first response statement information, a first preset loss function based on information entropy is constructed.

[0094] Based on the second spatial variable, the second decoder, and the second response statement information, a second preset loss function based on information entropy is constructed.

[0095] Based on the Euclidean distance between the first spatial variable and the second spatial variable, a third preset loss function is constructed;

[0096] The triplet information is obtained based on the second spatial variable, and a fourth preset loss function is constructed based on the triplet loss method and the triplet information.

[0097] Based on the KL divergence of the first latent variable and the second latent variable, a fifth preset loss function is constructed;

[0098] Based on the second latent variable, the second response statement information, and the sentence function sample information, a sixth preset loss function based on cross-entropy classification loss is constructed;

[0099] The preset loss function is constructed based on the first preset loss function, the second preset loss function, the third preset loss function, the fourth preset loss function, the fifth preset loss function, and the sixth preset loss function.

[0100] The step of training the first encoder based on the first response statement information, the second response statement information, and a preset loss function to obtain the encoder in the sequence-to-sequence model includes:

[0101] Based on the first response statement information, the second response statement information, and the preset loss function, and by training the first encoder using a planned sampling mechanism, the encoder in the sequence-to-sequence model is obtained.

[0102] The explanation is as follows:

[0103] First, the problem to be solved is described in a formulaic way:

[0104]

[0105] Where X is the input sequence, Y is the output sequence, and y1, y2, ..., yn are each element in the output sequence. According to this formula, the goal of the model is to calculate P(Y|X,f), which is the probability of generating the output sequence Y from the input sequence X and a specified function factor. The model takes the output sequence with the highest probability as the result.

[0106] like Figure 3As shown, a traditional Seq2seq model is combined with its autoencoder model and trained together. Triplet loss and Euclidean distance loss are introduced as additional supervision criteria to fit the model, thereby enabling the corresponding variables of different functional sentences to form different clusters in the space, and ensuring that two sentences that are very close in the space have similar content.

[0107] Based on this characteristic, the model can extract semantic variables from small-distance spaces in different directions and decode them into various functional sentences. To achieve this, the model needs an encoder to map each preceding sentence to a meaningful variable, and a decoder to transform the variables in the space into different sentences. For example... Figure 3 The model structure is shown, and its basic framework consists of a Seq2seq model and an autoencoder, both of which share parameters using the same decoder part.

[0108] During the training process, the encoder part of the Seq2seq model encodes the context and functional information into the first spatial variable Z. s2s This part (the encoder part of the Seq2seq model) includes a bidirectional GRU (BIGRU) recurrent neural network, a fully connected network, and a noise layer.

[0109] Furthermore, the encoder part of the autoencoder represents the original response statement as a second-space variable Z. AE This part (the encoder part of the autoencoder) includes a bidirectional GRU (BIGRU) recurrent neural network layer plus a noise layer.

[0110] In addition, a two-layer fully connected network was built to obtain deep-level latent variables in the sentence, denoted as the first latent variable L. s2s With the second latent variable L AE To extract more information about the features and content.

[0111] The shared decoder portion of the model consists of a single-layer GRU neural network and a Softmax layer. During the decoding phase, Z... s2s and L s2s , and Z AE and L AE These are combined as the initial state for the decoder part, and the latent variables are used as additional inputs to gradually obtain the target response. The details of the model can be described as follows:

[0112] The first and second spatial variables were generated by a bidirectional GRU (BIGRU) recurrent neural network:

[0113]

[0114] Extracting latent variable parameters from a sentence using a fully connected MLP network:

[0115] [μ,σ 2 ] = MLP latent (Z)

[0116] Among them, MLP latent (Z) indicates that Z (representing the first and second spatial variables respectively) is input into the fully connected MLP network used to obtain the latent variable parameters.

[0117] The latent variable L is randomly obtained from the first sample set N according to the sample function:

[0118] L=sample(N(μ,σ 2 ))

[0119] Where L represents the first latent variable and the second latent variable, respectively.

[0120] When generating a response, the state at the previous time step (t-1) is taken into account, and the latent variable L is used as an additional input:

[0121] s t =GRU(s) t-1 ,embed(y t-1 ),s0=Z+L

[0122] Among them, s t Let y be the state variable at time t. t-1 The response statement for the output at time t-1, embed(y t-1 This indicates that the response statement to the output at time t-1 is embedded, which can be implemented using conventional techniques.

[0123] Generate the response statement at time t using a softmax layer:

[0124] y t =softmax(W*(s) t +L))

[0125] Where W represents the weight.

[0126] After the above process, the model will simultaneously generate the first response statement information. s2s The second response statement information AE The generation loss of the decoder part (first preset loss function and second preset loss function) is as follows:

[0127]

[0128]

[0129] To ensure the interpretability of variables encoded by Seq2seq and autoencoder models in space, and to maintain as much semantic information as possible while being spatially close, Euclidean distance is introduced as an evaluation criterion. The loss (third preset loss function) is as follows:

[0130]

[0131] Where n represents the number of samples in a training batch (batch data) of the second sample set, d represents the dimension of the variable, and dist represents the method of calculating the Euclidean distance. This part uses standardization to avoid the loss ratio varying with the size of the batch (batch data).

[0132] Experiments have shown that Euclidean distance outperforms cosine distance and Manhattan distance. Furthermore, Tripletloss is introduced as an additional metric into the model. Based on the function label of each response in the training data, different variables will form different groups in space, and each group will be as far away from other groups as possible. Details of the specific loss (fourth preset loss function) are as follows:

[0133]

[0134] in, Let be the second spatial variable corresponding to the i-th sample. To and These are positive examples of the same category. To and Negative examples that are not from the same class. var is a constant describing the boundary distance between different clusters; here, Euclidean distance is also used to calculate the differences between variables.

[0135] This model draws on the ideas of recognition networks and prior networks, and uses a two-layer fully connected network to obtain the latent variable L. s2s With L AE This variable captures the functional and content information in the preceding and responding text. To ensure that the corresponding latent variables belong to the same distribution, the KL divergence is used as the loss function (the fifth preset loss function), described as follows:

[0136] loss KL =KL(L AE ||L s2s )

[0137] In addition, the model uses a classifier to evaluate the latent variable L during the training process. AECan it capture function-related information? This classifier consists of two fully connected layers and a softmax layer, and the result uses cross-entropy as the classification loss. This approach forces L... AE The sentence capture function enhances the spatial meaning of deep latent variables. The specific loss function (sixth preset loss function) is shown below:

[0138]

[0139] Where f represents the sentence function sample information, and Y represents the second response statement information.

[0140] The loss function for the entire module is a linear combination of the six losses mentioned above. Different weights are used to balance the distance loss and generation loss. Simultaneously, the KL divergence annealing algorithm is employed to address the vanishing KL divergence problem. The overall loss function (preset loss function) is as follows:

[0141] loss = α(loss s2s +loss AE )+β(loss EUC +loss T )+δloss KL +γloss dis

[0142] Here, α, β, δ, and γ are weights, and their sum equals 1.

[0143] The Seq2seq structure and autoencoder are trained together to optimize the loss function mentioned above. During training, the model employs a Scheduled Sampling mechanism, which effectively prevents overfitting and results in more fluent sentences.

[0144] The sentence information generation method provided in this invention adds sentence functional information to the traditional Seq2Seq model, proposing a new sentence information generation model based on functional factors. This model generates different sentences by extracting corresponding functional variables from the space, increasing the diversity and interest of responses, enhancing customer interest, improving the efficiency of communication services between banks and customers, and facilitating banks' customer maintenance efforts.

[0145] The sentence information generation method provided in this invention obtains text information and sentence function information of user dialogue, processes the text information and sentence function information based on a pre-trained model to obtain sentence space variables; wherein, the model is an encoder in a sequence-to-sequence model, and the sentence function information is random sentence information and / or custom sentence information used to increase the diversity of response sentences; random values ​​are added to the sentence space variables to obtain initial input variables for decoding; wherein, the random values ​​are values ​​randomly generated within a preset iteration period before the model training is completed, with the upper limit being the average Euclidean distance between the first space variable corresponding to the original first encoder used for sequence transformation and the second space variable corresponding to the second encoder receiving the original response sentence; the decoder in the sequence-to-sequence model processes the initial input variables to obtain response sentence information for replying to the text information. By generating diverse response sentences, the efficiency of communication services between banks and customers can be improved, which is beneficial for banks to carry out customer maintenance work.

[0146] Further, the step of adding random values ​​to the sentence space variables to obtain the initial input variables for decoding includes:

[0147] The initial input variables are obtained by adding random values ​​based on a normal distribution to each dimension of the sentence space variables. This can be referred to the above embodiment for further explanation, and will not be repeated here.

[0148] The statement information generation method provided in this embodiment of the invention can obtain initial input variables with small data fluctuations.

[0149] Further, the process of processing the initial input variables based on the decoder in the sequence-to-sequence model to obtain response statement information for replying to the text information includes:

[0150] Based on the decoder in the sequence-to-sequence model, and by processing the initial input variables using a greedy search method, the response statement information is obtained. This can be referred to the above embodiments for further explanation, and will not be repeated here.

[0151] The statement information generation method provided in this embodiment of the invention can efficiently obtain reply statement information.

[0152] Furthermore, the statement information generation method also includes:

[0153] The model is trained using the first encoder and its corresponding first decoder, and the second encoder and its corresponding second decoder. This can be referred to the above embodiments for further explanation, and will not be repeated here.

[0154] The statement information generation method provided in this embodiment of the invention can facilitate model training on the basis of existing architecture.

[0155] Further, training the model using the first encoder and its corresponding first decoder, and the second encoder and its corresponding second decoder, includes:

[0156] The first spatial variable is obtained by processing the text sample information and sentence function sample information based on the first encoder; the second spatial variable is obtained by processing the original response statement based on the second encoder; the above embodiments can be referred to for explanation, and will not be repeated here.

[0157] The first latent variable corresponding to the first spatial variable is added to the first spatial variable, and the first response statement information is output through the first decoder. The second latent variable corresponding to the second spatial variable is added to the second spatial variable, and the second response statement information is output through the second decoder. The above embodiments can be referred to for explanation, and will not be repeated here.

[0158] The first encoder is trained based on the first response statement information, the second response statement information, and a preset loss function to obtain the encoder in the sequence-to-sequence model. This can be referred to the above embodiments for further explanation and will not be repeated here.

[0159] The statement information generation method provided in this embodiment of the invention can guarantee the accuracy of model training.

[0160] Furthermore, the preset loss function is constructed, including:

[0161] Based on the first spatial variable, the first decoder, and the first response statement information, a first preset loss function based on information entropy is constructed; this can be referred to the above embodiments for explanation, and will not be repeated here.

[0162] Based on the second spatial variable, the second decoder, and the second response statement information, a second preset loss function based on information entropy is constructed; this can be referred to the above embodiments for explanation, and will not be repeated here.

[0163] A third preset loss function is constructed based on the Euclidean distance between the first spatial variable and the second spatial variable; the above embodiments can be referred to for explanation, and will not be repeated here.

[0164] The triplet information is obtained based on the second spatial variable. A fourth preset loss function is constructed based on the triplet loss method and the triplet information. The above embodiments can be referred to for explanation, and will not be repeated here.

[0165] Based on the KL divergence of the first latent variable and the second latent variable, a fifth preset loss function is constructed; the above embodiments can be referred to for explanation, and will not be repeated here.

[0166] Based on the second latent variable, the second response statement information, and the sentence function sample information, a sixth preset loss function based on cross-entropy classification loss is constructed; the above embodiments can be referred to for explanation, and will not be repeated here.

[0167] The preset loss function is constructed based on the first preset loss function, the second preset loss function, the third preset loss function, the fourth preset loss function, the fifth preset loss function, and the sixth preset loss function. This can be referred to the above embodiments for explanation, and will not be repeated here.

[0168] The statement information generation method provided in this embodiment of the invention can further improve the accuracy of model training.

[0169] Further, the step of training the first encoder based on the first response statement information, the second response statement information, and a preset loss function to obtain the encoder in the sequence-to-sequence model includes:

[0170] Based on the first response statement information, the second response statement information, and the preset loss function, and by training the first encoder using a planned sampling mechanism, the encoder in the sequence-to-sequence model is obtained. This can be referred to the above embodiments for further explanation, and will not be repeated here.

[0171] The statement information generation method provided in this embodiment of the invention can effectively prevent the overfitting problem of the model.

[0172] It should be noted that the statement information generation method provided in this embodiment of the invention can be used in the financial field, or in any technical field other than the financial field. This embodiment of the invention does not limit the application field of the statement information generation method.

[0173] Figure 4 This is a schematic diagram of the structure of a statement information generation device provided in an embodiment of the present invention, as shown below. Figure 4 As shown, the statement information generation device provided in this embodiment of the invention includes an acquisition unit 401, an addition unit 402, and a generation unit 403, wherein:

[0174] The acquisition unit 401 is used to acquire text information and sentence function information of user dialogue, and process the text information and sentence function information based on a pre-trained model to obtain sentence space variables; wherein, the model is an encoder in a sequence-to-sequence model, and the sentence function information is random sentence information and / or custom sentence information used to increase the diversity of response statements; the adding unit 402 is used to add random values ​​to the sentence space variables to obtain initial input variables for decoding; wherein, the random values ​​are values ​​randomly generated within a preset iteration period before the model training is completed, with the upper limit being the average Euclidean distance between the first space variable corresponding to the original first encoder used for sequence conversion and the second space variable corresponding to the second encoder receiving the original response statement; the generation unit 403 is used to process the initial input variables based on the decoder in the sequence-to-sequence model to obtain response statement information for replying to the text information.

[0175] Specifically, the acquisition unit 401 in the device is used to acquire text information and sentence function information of the user dialogue, and process the text information and sentence function information based on a pre-trained model to obtain sentence space variables; wherein, the model is an encoder in a sequence-to-sequence model, and the sentence function information is random sentence information and / or custom sentence information used to increase the diversity of response statements; the adding unit 402 is used to add random values ​​to the sentence space variables to obtain initial input variables for decoding; wherein, the random values ​​are values ​​randomly generated within a preset iteration period before the model training is completed, with the upper limit being the average Euclidean distance between the first space variable corresponding to the original first encoder used for sequence conversion and the second space variable corresponding to the second encoder receiving the original response statement; the generation unit 403 is used to process the initial input variables based on the decoder in the sequence-to-sequence model to obtain response statement information for replying to the text information.

[0176] The sentence information generation device provided in this embodiment of the invention acquires text information and sentence function information of user dialogue, processes the text information and sentence function information based on a pre-trained model to obtain sentence space variables; wherein, the model is an encoder in a sequence-to-sequence model, and the sentence function information is random sentence information and / or custom sentence information used to increase the diversity of response sentences; random values ​​are added to the sentence space variables to obtain initial input variables for decoding; wherein, the random values ​​are values ​​randomly generated within a preset iteration period before the model training is completed, with the upper limit being the average Euclidean distance between the first space variable corresponding to the original first encoder used for sequence conversion and the second space variable corresponding to the second encoder receiving the original response sentence; the decoder in the sequence-to-sequence model processes the initial input variables to obtain response sentence information for replying to the text information. By generating diverse response sentences, the communication service efficiency between banks and customers can be improved, which is beneficial for banks to carry out customer maintenance work.

[0177] Furthermore, the adding unit 402 is specifically used for:

[0178] The initial input variables are obtained by adding random values ​​based on a normal distribution to each dimension of the sentence space variables.

[0179] The statement information generation device provided in this embodiment of the invention can obtain initial input variables with small data fluctuations.

[0180] Furthermore, the generation unit 403 is specifically used for:

[0181] Based on the decoder in the sequence-to-sequence model, and by using a greedy search method to process the initial input variables, the response statement information is obtained.

[0182] The statement information generation device provided in this embodiment of the invention can efficiently obtain reply statement information.

[0183] Furthermore, the statement information generation device is also used for:

[0184] The model is trained using the first encoder and its corresponding first decoder, and the second encoder and its corresponding second decoder.

[0185] The statement information generation device provided in this embodiment of the invention can easily enable model training on the basis of existing architecture.

[0186] Furthermore, the statement information generation device is also specifically used for:

[0187] The first spatial variable is obtained by processing the text sample information and sentence function sample information based on the first encoder; the second spatial variable is obtained by processing the original response statement based on the second encoder.

[0188] Add the first latent variable corresponding to the first spatial variable to the first spatial variable and output the first response statement information through the first decoder; add the second latent variable corresponding to the second spatial variable to the second spatial variable and output the second response statement information through the second decoder.

[0189] The first encoder is trained based on the first response statement information, the second response statement information, and a preset loss function to obtain the encoder in the sequence-to-sequence model.

[0190] The statement information generation device provided in this embodiment of the invention can ensure the accuracy of model training.

[0191] Furthermore, the statement information generation device is also specifically used for:

[0192] Based on the first spatial variable, the first decoder, and the first response statement information, a first preset loss function based on information entropy is constructed.

[0193] Based on the second spatial variable, the second decoder, and the second response statement information, a second preset loss function based on information entropy is constructed.

[0194] Based on the Euclidean distance between the first spatial variable and the second spatial variable, a third preset loss function is constructed;

[0195] The triplet information is obtained based on the second spatial variable, and a fourth preset loss function is constructed based on the triplet loss method and the triplet information.

[0196] Based on the KL divergence of the first latent variable and the second latent variable, a fifth preset loss function is constructed;

[0197] Based on the second latent variable, the second response statement information, and the sentence function sample information, a sixth preset loss function based on cross-entropy classification loss is constructed;

[0198] The preset loss function is constructed based on the first preset loss function, the second preset loss function, the third preset loss function, the fourth preset loss function, the fifth preset loss function, and the sixth preset loss function.

[0199] The statement information generation device provided in this embodiment of the invention can further improve the accuracy of model training.

[0200] Furthermore, the statement information generation device is also specifically used for:

[0201] Based on the first response statement information, the second response statement information, and the preset loss function, and by training the first encoder using a planned sampling mechanism, the encoder in the sequence-to-sequence model is obtained.

[0202] The statement information generation device provided in this embodiment of the invention can effectively prevent the overfitting problem of the model.

[0203] The embodiments of the present invention provide a statement information generation device that can be used to execute the processing flow of the above method embodiments. Its functions will not be repeated here, but can be referred to the detailed description of the above method embodiments.

[0204] Figure 5 This is a schematic diagram of the physical structure of an electronic device provided in an embodiment of the present invention, such as... Figure 5 As shown, the electronic device includes: a processor 501, a memory 502, and a bus 503;

[0205] The processor 501 and the memory 502 communicate with each other via the bus 503.

[0206] The processor 501 is used to call program instructions in the memory 502 to execute the methods provided in the above-described method embodiments, including, for example:

[0207] Obtain text information and sentence function information from user dialogues, and process the text information and sentence function information based on a pre-trained model to obtain sentence space variables;

[0208] The model is an encoder in a sequence-to-sequence model, and the sentence functional information is random sentence information and / or custom sentence information used to increase the diversity of response statements;

[0209] Random values ​​are added to the sentence space variables to obtain the initial input variables for decoding;

[0210] The random value is a value randomly generated within a preset iteration period before the model training is completed, with the upper limit being the average Euclidean distance between the first spatial variable corresponding to the original first encoder used for sequence transformation and the second spatial variable corresponding to the second encoder that receives the original response statement.

[0211] The initial input variables are processed by the decoder in the sequence-to-sequence model to obtain response statement information for replying to the text information.

[0212] This embodiment discloses a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, and when the program instructions are executed by a computer, the computer can perform the methods provided in the above-described method embodiments, such as:

[0213] Obtain text information and sentence function information from user dialogues, and process the text information and sentence function information based on a pre-trained model to obtain sentence space variables;

[0214] The model is an encoder in a sequence-to-sequence model, and the sentence functional information is random sentence information and / or custom sentence information used to increase the diversity of response statements;

[0215] Random values ​​are added to the sentence space variables to obtain the initial input variables for decoding;

[0216] The random value is a value randomly generated within a preset iteration period before the model training is completed, with the upper limit being the average Euclidean distance between the first spatial variable corresponding to the original first encoder used for sequence transformation and the second spatial variable corresponding to the second encoder that receives the original response statement.

[0217] The initial input variables are processed by the decoder in the sequence-to-sequence model to obtain response statement information for replying to the text information.

[0218] This embodiment provides a computer-readable storage medium storing a computer program that causes the computer to execute the methods provided in the above-described method embodiments, including, for example:

[0219] Obtain text information and sentence function information from user dialogues, and process the text information and sentence function information based on a pre-trained model to obtain sentence space variables;

[0220] The model is an encoder in a sequence-to-sequence model, and the sentence functional information is random sentence information and / or custom sentence information used to increase the diversity of response statements;

[0221] Random values ​​are added to the sentence space variables to obtain the initial input variables for decoding;

[0222] The random value is a value randomly generated within a preset iteration period before the model training is completed, with the upper limit being the average Euclidean distance between the first spatial variable corresponding to the original first encoder used for sequence transformation and the second spatial variable corresponding to the second encoder that receives the original response statement.

[0223] The initial input variables are processed by the decoder in the sequence-to-sequence model to obtain response statement information for replying to the text information.

[0224] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0225] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0226] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0227] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0228] In the description of this specification, the references to terms such as "an embodiment," "a specific embodiment," "some embodiments," "for example," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0229] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for generating statement information, characterized in that, include: Obtain text information and sentence function information from user dialogues, and process the text information and sentence function information based on a pre-trained model to obtain sentence space variables; The model is an encoder in a sequence-to-sequence model, and the sentence functional information is random sentence information and / or custom sentence information used to increase the diversity of response statements; Random values ​​are added to the sentence space variables to obtain the initial input variables for decoding; The random value is a value randomly generated within a preset iteration period before the model training is completed, with the upper limit being the average Euclidean distance between the first spatial variable corresponding to the original first encoder used for sequence transformation and the second spatial variable corresponding to the second encoder that receives the original response statement. The initial input variables are processed by the decoder in the sequence-to-sequence model to obtain response statement information for replying to the text information.

2. The statement information generation method according to claim 1, characterized in that, The step of adding random values ​​to the sentence space variables to obtain the initial input variables for decoding includes: The initial input variables are obtained by adding random values ​​based on a normal distribution to each dimension of the sentence space variables.

3. The statement information generation method according to claim 1, characterized in that, The decoder in the sequence-to-sequence model processes the initial input variables to obtain response statement information for replying to the text information, including: Based on the decoder in the sequence-to-sequence model, and by using a greedy search method to process the initial input variables, the response statement information is obtained.

4. The statement information generation method according to any one of claims 1 to 3, characterized in that, The statement information generation method further includes: The model is trained using the first encoder and its corresponding first decoder, and the second encoder and its corresponding second decoder.

5. The statement information generation method according to claim 4, characterized in that, Training the model using the first encoder and its corresponding first decoder, and the second encoder and its corresponding second decoder, includes: The first spatial variable is obtained by processing the text sample information and sentence function sample information based on the first encoder; the second spatial variable is obtained by processing the original response statement based on the second encoder. Add the first latent variable corresponding to the first spatial variable to the first spatial variable and output the first response statement information through the first decoder; add the second latent variable corresponding to the second spatial variable to the second spatial variable and output the second response statement information through the second decoder. The first encoder is trained based on the first response statement information, the second response statement information, and a preset loss function to obtain the encoder in the sequence-to-sequence model.

6. The statement information generation method according to claim 5, characterized in that, Constructing the preset loss function includes: Based on the first spatial variable, the first decoder, and the first response statement information, a first preset loss function based on information entropy is constructed. Based on the second spatial variable, the second decoder, and the second response statement information, a second preset loss function based on information entropy is constructed. Based on the Euclidean distance between the first spatial variable and the second spatial variable, a third preset loss function is constructed; The triplet information is obtained based on the second spatial variable, and a fourth preset loss function is constructed based on the triplet loss method and the triplet information. Based on the KL divergence of the first latent variable and the second latent variable, a fifth preset loss function is constructed; Based on the second latent variable, the second response statement information, and the sentence function sample information, a sixth preset loss function based on cross-entropy classification loss is constructed; The preset loss function is constructed based on the first preset loss function, the second preset loss function, the third preset loss function, the fourth preset loss function, the fifth preset loss function, and the sixth preset loss function.

7. The statement information generation method according to claim 5, characterized in that, The step of training the first encoder based on the first response statement information, the second response statement information, and a preset loss function to obtain the encoder in the sequence-to-sequence model includes: Based on the first response statement information, the second response statement information, and the preset loss function, and by training the first encoder using a planned sampling mechanism, the encoder in the sequence-to-sequence model is obtained.

8. A statement information generation device, characterized in that, include: The acquisition unit is used to acquire text information and sentence function information of user dialogue, and process the text information and sentence function information based on a pre-trained model to obtain sentence space variables; The model is an encoder in a sequence-to-sequence model, and the sentence functional information is random sentence information and / or custom sentence information used to increase the diversity of response statements; An addition unit is used to add random values ​​to the sentence space variables to obtain the initial input variables for decoding; The random value is a value randomly generated within a preset iteration period before the model training is completed, with the upper limit being the average Euclidean distance between the first spatial variable corresponding to the original first encoder used for sequence transformation and the second spatial variable corresponding to the second encoder that receives the original response statement. The generation unit is used to process the initial input variables based on the decoder in the sequence-to-sequence model to obtain response statement information for replying to the text information.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Citation Information

Patent Citations

  • Conversation generation method and device, storage medium and electronic equipment

    CN110347792A

  • Question classification model, classification method, training method and device

    CN115577097A