Method, electronic device and computer program product for chat robot

Abstract

The embodiment of the invention relates to a method for a chatting robot, electronic equipment and a computer program product. The method includes determining a first representation associated with a query based on the query input by a user to a chat robot. The method further includes generating a second representation based on the first representation and a domain to which the query belongs, where the dimension of the second representation is smaller than that of the first representation. The method further includes generating, by a decoder corresponding to the domain, a response to the query based on the second representation. Through the embodiment of the invention, the quality of the generated response aiming at the query and the consistency of the response can be improved, and the universality and the specificity of the response are balanced.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computers, and more particularly, to methods, electronic devices, and computer program products for chatbots. Background Art

[0002] Domain-specific chatbots are conversational agents that provide information or services related to a specific domain. However, most existing chatbot models are not constrained by a specific knowledge base, which may result in irrelevant or inaccurate responses that harm user experience and trust. Therefore, when building a domain-specific chatbot, it is important to ensure that it can provide relevant, accurate, and reliable information and services. Summary of the invention

[0003] Embodiments of the present disclosure provide a method, an electronic device, and a computer program product for a chatbot.

[0004] According to a first aspect of the present disclosure, a method for a chatbot is provided. The method includes determining a first representation associated with a query input by a user to the chatbot. The method also includes generating a second representation based on the first representation and a domain to which the query belongs, wherein the dimension of the second representation is smaller than the first representation. The method also includes generating a response to the query based on the second representation by a decoder corresponding to the domain.

[0005] According to a second aspect of the present disclosure, an electronic device is also provided. The electronic device includes a processor and a memory coupled to the processor, the memory having instructions stored therein, and the instructions, when executed by the processor, cause the device to perform an action. The action includes determining a first representation associated with a query based on a query input by a user to a chatbot. The action also includes generating a second representation based on the first representation and the domain to which the query belongs, wherein the dimension of the second representation is smaller than the first representation. The action also includes generating a response to the query based on the second representation by a decoder corresponding to the domain.

[0006] According to a third aspect of the present disclosure, there is provided a computer program product, which is tangibly stored on a computer-readable medium and comprises computer-executable instructions, which when executed by a device cause the device to perform the method according to the first aspect.

[0007] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:

[0009] Figure 1A A schematic diagram schematically illustrates an example environment in which embodiments of the present disclosure can be implemented;

[0010] Figure 1B A schematic diagram schematically illustrates the overall scheme of the present disclosure;

[0011] Figure 2 A schematic diagram schematically shows the overall architecture of a method for a chatbot according to an exemplary implementation of the present disclosure;

[0012] Figure 3 A flowchart of a method for a chatbot according to an exemplary implementation of the present disclosure is schematically shown;

[0013] Figure 4 A block diagram schematically illustrates the workflow of a chatbot model according to an exemplary implementation of the present disclosure;

[0014] Figure 5 A block diagram schematically illustrates a workflow of a comparison head according to an exemplary implementation of the present disclosure;

[0015] Figure 6 A block diagram schematically illustrates the workflow of a multi-task learning module according to an exemplary implementation of the present disclosure; and

[0016] Figure 7 A block diagram of a device for a chatbot according to an exemplary implementation of the present disclosure is schematically shown.

[0017] Throughout the drawings, the same or similar reference numbers denote the same or similar elements. DETAILED DESCRIPTION

[0018] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0019] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc. may refer to different or the same objects. Other explicit and implicit definitions may also be included below. In addition, all specific numerical values ​​in this document are examples, which are only for the purpose of helping understanding, and are not intended to limit the scope.

[0020] As discussed in the background technology above, a domain-specific chatbot is a conversational agent that provides information or services related to a specific domain. However, most existing chatbot models are not constrained by a specific knowledge base, which may lead to irrelevant or inaccurate responses, thereby damaging user experience and trust. To address this issue, some traditional technologies propose to integrate domain knowledge into chatbot models using various methods, such as knowledge graphs, memory networks, or reinforcement learning. However, these methods either require a large amount of expensive and scarce annotated data, or rely on hand-crafted rules or heuristics, which are not scalable or robust.

[0021] Contrastive learning is a technique for learning similar representations of positive samples and dissimilar representations of negative samples. It has been widely used in self-supervised learning, especially in computer vision, where data augmentation is used to create positive and negative sample pairs. However, data augmentation is usually domain-specific and requires prior knowledge about the domain. For example, image cropping and rotation may not be suitable for speech or tabular data. Therefore, some traditional schemes propose domain-independent methods for contrastive learning, such as using mixed (mixup) noise or random projection. However, these methods do not consider the domain relevance of the pairs, which may lead to suboptimal representations for domain-specific tasks.

[0022] To address the above shortcomings, embodiments of the present disclosure provide a solution for a chatbot. The solution uses contrastive learning and multi-task learning to integrate domain knowledge into the working of the chatbot model. In some embodiments, the solution uses contrastive learning to combine the chatbot's responses with a domain knowledge base and distinguish them from responses in other domains. In some embodiments, multi-task learning is used to learn common features and across different tasks or domains. In some embodiments, the solution provides a lightweight contrast head that can be easily added to any existing chatbot model, and uses a contrastive loss function to take into account the semantic similarity and domain relevance of responses.

[0023] Figure 1AA schematic diagram of an example environment 100A in which embodiments of the present disclosure can be implemented is shown. As shown in FIG. 1 , the example environment 100 may include a computing device 102. The computing device 102 may be, for example, a computing system or a server. A chatbot may be installed on the computing device 102, or the computing device 102 itself may be a chatbot. The computing device 102 receives a user's input as a query 104. The input may, for example, include a question from the user, such as, for a customer service chatbot, what is the current status of my order, or what are the return conditions.

[0024] After receiving the query 104, the chatbot model 120 of the chatbot can obtain relevant content in the historical dialogue 108 or the knowledge base 110 of the user. The encoder 122 in the chatbot model 120 can generate a representation 112 (also referred to as a first representation) based on one or more of the query 104, the historical dialogue 108, and the knowledge base 110. The representation 112 can be input to the comparison head 124. The comparison head 124 generates a representation 114 (also referred to as a second representation) in combination with a specific domain. The representation 114 contains information about the domain to which the query 104 belongs, and is therefore more accurate.

[0025] The representation 114 can be input into a decoder specific to the domain to which the query 104 belongs, such as decoder 126-1. In the chatbot model 120, other decoders are also included, such as decoder 126-2 (decoder 126-1, decoder 126-2, etc. can be collectively or individually referred to as decoder 126). Decoder 126-1 is specific to the domain to which the query 104 belongs, and therefore can decode more accurately. After decoder 126-1 decodes the representation 114, it generates a response 106 corresponding to the query 104. For example, if the query 104 is a question: What is the current status of my order, then the response 106 can be the answer: It has been shipped today. It can be understood that the chatbot model 120 may include multiple decoders, each of which is specific to a different domain, and is not limited to Figure 1A Decoder 126-1 and decoder 126-2 are shown.

[0026] It should be understood that the architecture and functions in the example environment 100A are described for exemplary purposes only, and do not imply any limitation on the scope of the present disclosure. The embodiments of the present disclosure may also be applied to other environments with different structures and / or functions.

[0027] Figure 1B The schematic diagram of the overall scheme 100B of the present disclosure is schematically shown. Figure 1BAs shown, box 130 represents input, and the input can be text in text form. It can also be speech that can be converted into text. The input can also include an external knowledge base (such as word, pdf, etc.). Based on the traditional chatbot model, this solution adds a decoder head 134 (also referred to as a comparison head in this article), where the comparison head is trained based on a specially designed comparison loss function 136. Therefore, the newly added lightweight comparison head can be easily added to any existing chatbot model without a lot of modification or retraining. A comparison loss function is provided, which takes into account the semantic similarity and domain relevance of the response 132, which can balance the trade-off between generality and specificity.

[0028] Before describing the technical solution of the present invention, it is necessary to introduce the concepts related to the technical solution of the present invention, which is helpful for a better understanding of the technical solution of the present invention. For a chatbot model, it can generate responses for user queries in a specific domain. Assume that the chatbot model can access a domain knowledge base (KB), which is a collection of facts or information related to the domain. For example, the knowledge base of the retail domain may contain information about products, prices, and reviews.

[0029] A domain knowledge base can be represented as a set of triples of the form (e1, r, e2), where e1 and e2 are entities and r is the relationship between them. For example, (A phone, price, $999) is a triple that means that the price of A phone is $999. Chatbot models can use multi-task learning to handle multiple tasks or domains at the same time. Multi-task learning is a technique that trains models on multiple related tasks or domains using shared representations. For example, a chatbot model can be trained in both the retail and banking domains. Consider a chatbot model that can generate responses for user queries in a specific domain. For example, a chatbot model can be trained in both the retail and banking domains, using a common encoder and a separate decoder for each domain.

[0030] The user query can be represented as q and the chatbot response as r. The hidden state of the chatbot model is represented as h, which is calculated by the encoder based on the query q. The contrastive representation of the response is represented as z, which is calculated by the contrastive head through the hidden state h. Denote the set of all possible responses as R and the set of all possible entities as E. Denote the set of positive responses for a given query as R+ and the set of negative responses as R-. Denote the set of relevant entities for a given query as E+ and the set of irrelevant entities as E-. Denote the similarity function between two vectors as s(·,·) and the relevance function between a vector and an entity as r(·,·). A common encoder and separate decoder can be used for each domain.

[0031] Therefore, the technical solution proposed in the present disclosure faces many challenges, such as the chatbot's response is not only semantically similar to the user query, but also related to the domain knowledge base. Specifically, how to create positive and negative pairs for comparative learning from user queries and chatbot responses, how to combine the comparative representation of chatbot responses with the domain knowledge base, how to distinguish the comparative representation of chatbot responses from responses in other domains, how to balance the trade-off between the generality and specificity of chatbot responses, and how to design a lightweight comparative head that can be easily added to any existing chatbot model. One or more of these challenges are worth considering, and the overall architecture of the solution of the present disclosure is therefore proposed.

[0032] Figure 2 A schematic diagram of an overall architecture 200 of a method for a chatbot according to an exemplary implementation of the present disclosure is schematically shown. The overall architecture 200 includes a chatbot model 222, a contrast head 224, and a multi-task learning module 226. By implementing the overall architecture 200, a contrast representation z of each chatbot response r can be learned, wherein the contrast representation z is similar to the hidden state h of the user query q, that is, s(z,h) is high. The contrast representation z is related to the entities in the domain knowledge base E+, that is, for all e∈E+, r(z,e) is high. The contrast representation z is different from the hidden state of the negative response R-, that is, for all h' corresponding to r'∈R-, s(z,h') is low. The contrast representation z is independent of entities in other domains E-, that is, r(z,e) is low for all e∈E-.

[0033] The chatbot model 222 generates natural language responses to user questions. The chatbot model can be any existing pre-trained language model. The chatbot model takes user utterances, conversation history, and domain knowledge paragraphs retrieved from the knowledge base as input. The chatbot model outputs responses related to the user's questions and domain knowledge.

[0034] Contrastive Head 224 is a lightweight module that can be easily added to a chatbot model without extensive modifications or retraining. Contrastive Head 224 learns similar representations for positive pairs and dissimilar representations for negative pairs. Positive pairs consist of responses and domain knowledge passages that are related to each other. Negative pairs consist of responses and domain knowledge passages that are unrelated to each other. Contrastive Head 224 uses a contrastive loss function that considers both semantic similarity and domain relevance of responses. Contrastive Head 224 aims to align the chatbot's responses with the domain knowledge base and distinguish them from responses from other domains.

[0035] The multi-task learning module 226 is a module that trains the chatbot model on multiple related tasks or domains simultaneously. The multi-task learning module 226 uses a shared encoder and a task-specific decoder for each task or domain. The multi-task learning module 226 learns common features and transfers knowledge across different tasks or domains. The multi-task learning module 226 is intended to improve the robustness and adaptability of the chatbot model.

[0036] The following will combine Figures 3 to 6 The process according to the embodiment of the present disclosure is described in detail. For ease of understanding, the specific data mentioned in the following description are exemplary and are not intended to limit the scope of protection of the present disclosure. It is understood that the embodiments described below may also include additional actions not shown and / or the actions shown may be omitted, and the scope of the present disclosure is not limited in this respect.

[0037] Figure 3 The flowchart of a method 300 for a chatbot according to an exemplary implementation of the present disclosure is schematically shown. At block 302, based on a query input by a user to the chatbot, a first representation associated with the query is determined. For example, the chatbot model 120 receives the query 104 and determines the representation 112. Now, in conjunction with Figure 4 Let’s introduce the details and workflow of the chatbot model 120. Figure 4 A block diagram schematically shows a workflow 400 of a chatbot model according to an exemplary implementation of the present disclosure.

[0038] In general, the workflow 400 of the chatbot model includes the chatbot 402 obtaining input 404, processing input 406, and generating response 408. Specifically, processing input 406 may also include a chatbot workflow 410 and a comparison workflow 412. When pre-training the chatbot model, a multi-task learning workflow 414 may also be included.

[0039] like Figure 4 As shown, the chatbot model 400 generates natural language responses for user queries in a specific domain. In general, the chatbot model 400 can be any existing or future developed generative pre-trained language model. The chatbot model 400 takes user utterances q, dialogue histories H, and paragraphs of domain knowledge K retrieved from a knowledge base as input. The chatbot model 400 outputs responses r related to user queries and domain knowledge.

[0040] The chatbot model 400 includes an encoder and a decoder. The encoder is a Transformer-based neural network that encodes the input into a hidden state sequence h = (h1, h2, ..., hn), where n is the length of the input. The decoder is also a Transformer-based neural network that generates response tokens one by one, conditioned on the encoder's hidden state and previously generated tokens. The decoder uses an attention mechanism to focus on relevant parts of the encoder's hidden state and the decoder's hidden state. The chatbot model 400 is trained using a cross-entropy loss function that measures the difference between the generated response and the true response. The cross-entropy loss function (also known as the first loss function) is defined as formula (1)

[0041]

[0042] Where m represents the length of the response, p(r i |q,H,K) represents the probability of generating the i-th token of the response given the input.

[0043] Return now Figure 3 At block 304, a second representation is generated based on the first representation and the domain to which the query belongs, wherein the dimension of the second representation is smaller than that of the first representation. For example, the comparison head 124 generates the representation 114 based on the representation 112 and the domain to which the representation 112 belongs. Figure 5 Let's describe the contrast head in detail. Figure 5 A block diagram schematically illustrates a workflow 500 of a comparison head according to an exemplary implementation of the present disclosure.

[0044] In general, the workflow 500 of the comparison head includes obtaining responses 502, domain knowledge passages or graphs 504, a comparison head 506, and a comparison loss function 508. As discussed above, the comparison head is a lightweight module that can be easily added to the chatbot model without extensive modification or retraining. The comparison head learns similar representations for positive pairs and different representations for negative pairs. Positive pairs consist of responses and domain knowledge passages that are related to each other. Negative pairs consist of responses and domain knowledge passages that are unrelated to each other. The comparison head uses a comparison loss function that takes into account the similarity and domain relevance of semantic responses. The purpose of the comparison head is to align the chatbot's responses with the domain knowledge base and distinguish them from responses from other domains that are unrelated to each other.

[0045] The contrast head takes the hidden state h of the chatbot model as input and outputs a contrast representation z (i.e., the second representation) for each response r. The contrast representation z is calculated by applying a linear transformation and then normalizing the hidden state h. The linear transformation projects the hidden state h into a low-dimensional space, and the normalization ensures that the contrast representation z has unit length. The contrast head can be defined as formula (2):

[0046]

[0047] Where W (also known as the first parameter) and b (also known as the second parameter) represent the learnable parameters of the linear transformation. Wh+b is also known as the third representation.

[0048] The contrast head is trained using a contrastive loss function that measures the difference between the contrastive representations of positive and negative pairs. The contrastive loss function (also known as the second loss function) is defined as formula (3):

[0049]

[0050] Wherein, N represents the number of positive pairs, K represents the number of negative pairs, τ represents the temperature parameter, and s(·,·) represents the similarity function between two vectors. In some embodiments, cosine similarity is used as the similarity function, and the similarity function can be defined as formula (4):

[0051]

[0052] Where x and y represent arbitrary vectors.

[0053] The contrast loss function encourages the contrast representation zi to be more similar to the hidden state hi+ of the positive response than to the hidden state hj- of the negative response. However, in order to prevent the contrast loss function from only considering the semantic similarity between responses without considering the domain relevance of the responses, an additional term can be introduced into the contrast loss function to measure the relevance between the contrast representation z and the entities in the domain knowledge base E. The relevance term (also known as the third loss function) can be defined as:

[0054]

[0055] where r(·,·) represents the relevance function between the vector and the entity. A lookup table can be used as the relevance function, which assigns a score to each pair of vector and entity based on the co-occurrence frequency in the training data. The lookup table is initialized with random values ​​and updated during training. The relevance term encourages comparison between the representation zi and the entities in the domain knowledge base. Compared with entities in other fields More relevant.

[0056] The final loss function of method 300 may be a weighted combination of cross entropy loss, contrast loss, and correlation loss, that is, formula (6)

[0057] L=L CE +αL CL +βL RL (6)

[0058] where α and β represent hyperparameters that control the trade-off between generality and specificity of the chatbot’s responses.

[0059] The comparison head designed in this way can reduce the complexity and computational overhead of the chatbot model. The comparison head only needs a linear transformation to calculate the normalization of the comparison representation of each response, which is computationally efficient and does not affect the performance of the chatbot model. This can make method 300 more flexible and compatible with different chatbot models and platforms.

[0060] Return now Figure 3 At block 306, a decoder corresponding to the domain generates a response to the query based on the second representation. For example, the domain to which query 104 belongs corresponds to decoder 126-1, and the decoder is used to generate response 106 to query 104. In some embodiments, the decoder for a specific domain is obtained by a multi-task learning module. Figure 6 Let’s describe the multi-task learning module in detail.

[0061] Figure 6 A block diagram of a workflow 600 of a multi-task learning module according to an exemplary implementation of the present disclosure is schematically shown. In general, the workflow 600 of the multi-task learning module includes a shared encoder 602, a task-specific decoder 604-1, a task-specific decoder 604-2 (which can also be collectively referred to as multiple decoders 602), a multi-task learning module 606, and an improved chatbot model 608. The multi-task learning module is a module that trains a chatbot model in multiple related tasks or domains at the same time. The multi-task learning module uses a shared encoder and a task-specific decoder for each task or domain. The multi-task learning module learns common features and transfers knowledge across different tasks or domains. The multi-task learning module is intended to improve the robustness and adaptability of the chatbot model.

[0062] Specifically, the multi-task learning module takes as input the user utterance q, the dialogue history H, and the paragraphs of domain knowledge K retrieved from the knowledge base for each task or domain. The multi-task learning module outputs the response r for each task or domain using the corresponding decoder. The shared encoder is the same as the one used in the chatbot model, which encodes the input into a sequence of hidden states h = (h1, h2, ..., hn). The task-specific decoders are also similar to the decoders used in the chatbot model, but they have different parameters for each task or domain.

[0063] The multi-task learning module is trained using a weighted sum of the loss functions for each task or domain. The loss function for each task or domain is the same as the loss function used in method 300, which is a weighted combination of cross entropy loss, contrast loss, and relevance loss. The weighted sum of the loss functions is defined as formula (7):

[0064]

[0065] Where k represents the number of tasks or domains, λ i represents the hyperparameter that controls the importance of each task or domain, and Li represents the loss function of the i-th task or domain.

[0066] The multi-task learning module can benefit from sharing information and knowledge across different tasks or domains, especially when the data for some tasks or domains is limited or noisy. The multi-task learning module can also adapt to new tasks or domains by fine-tuning the shared encoder and adding new decoders. The multi-task learning module uses a shared encoder and task-specific decoders for each task or domain, which can share information and knowledge across different tasks or domains. This can benefit from data augmentation and regularization effects, especially when the data for some tasks or domains is limited or noisy. This can also enable the chatbot model to adapt to new tasks or domains by fine-tuning the shared encoder and adding new decoders.

[0067] Thus, by implementing an embodiment as provided by method 300, contrastive learning and multi-task learning can be used to incorporate domain knowledge into a chatbot model. In some embodiments, a lightweight contrast head can be easily added to any existing chatbot model, and a contrastive loss function that considers semantic similarity and domain relevance of responses can be used. In some embodiments, multiple tasks or domains can be processed simultaneously, and the quality and consistency of chatbot responses in different domains can be improved. Therefore, user experience and trust can be enhanced, and sales conversion rates can be increased and manual customer service costs can be reduced.

[0068] Furthermore, by implementing method 300, domain knowledge can be used to generate relevant and accurate responses to provide support for satisfying user queries in different areas, such as product information, technical information, order status, etc. This can improve customer satisfaction and loyalty, and reduce customer churn and complaints.

[0069] Figure 7 1 shows a schematic block diagram of a device 700 that can be used to implement an embodiment of the present disclosure. The device 700 may be a device or apparatus described in an embodiment of the present disclosure. Figure 7 As shown, the device 700 includes a central processing unit / graphics processing unit (CPU / GPU) 701, which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) 702 or loaded from a storage unit 708 to a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The CPU / GPU 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704. Although not shown in FIG. Figure 7 As shown in FIG. 7 , device 700 may further include a coprocessor.

[0070] A number of components in the device 700 are connected to the I / O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a disk, an optical disk, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0071] The various methods or processes described above may be performed by the CPU / GPU 701. For example, in some embodiments, the methods may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into the RAM 703 and executed by the CPU / GPU 701, one or more steps or actions in the methods or processes described above may be performed.

[0072] In some embodiments, the methods and processes described above may be implemented as a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present disclosure.

[0073] Computer readable storage medium can be a tangible device that can hold and store instructions used by an instruction execution device. Computer readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a convex structure in a groove on which instructions are stored, and any suitable combination thereof. The computer readable storage medium used here is not interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated by a waveguide or other transmission medium (for example, a light pulse by an optical fiber cable), or an electrical signal transmitted by a wire.

[0074] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0075] The computer program instructions for performing the disclosed operation may be assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, programming languages ​​including object-oriented programming languages, and conventional procedural programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, executed as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., utilizing an Internet service provider to connect via the Internet). In certain embodiments, by utilizing the state information of a computer-readable program instruction to customize an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), the electronic circuit may execute a computer-readable program instruction, thereby realizing various aspects of the present disclosure.

[0076] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0077] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0078] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the equipment, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each frame in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the frame can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous frames can actually be executed substantially in parallel, and they can also be executed in the opposite order sometimes, depending on the functions involved. It should also be noted that each frame in the block diagram and / or flow chart, and the combination of frames in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0079] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The choice of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for a chatbot, comprising: Based on a query input by a user to the chatbot, determining a first representation associated with the query; generating a second representation based on the first representation and the domain to which the query belongs, wherein the dimension of the second representation is smaller than that of the first representation; as well as A response to the query is generated by a decoder corresponding to the domain based on the second representation.

2. The method of claim 1 , wherein based on a query input by a user to the chatbot, determining a first representation associated with the query comprises: The first representation associated with the query is determined based on at least one of a sentence input by the user, a historical dialogue between the user and the chatbot, and a knowledge base of the domain.

3. The method according to claim 2, wherein determining the first representation associated with the query based on at least one of a statement input by the user, a historical conversation between the user and the chatbot, and a knowledge base of the domain comprises: Obtaining at least one of the statement, the historical conversation, and the knowledge base as the query; as well as Based on the query, a latent vector is generated as the first representation using a shared encoder of the chatbot, wherein the shared encoder is capable of encoding queries belonging to any domain.

4. The method of claim 1 , wherein based on the first representation and the domain to which the query belongs, generating the second representation comprises: performing a linear transformation on the first representation to determine a third representation; as well as The third representation is normalized to determine the second representation.

5. The method of claim 4, wherein the chatbot comprises a contrast head for generating the second representation, and the first and second parameters associated with the linear transformation are determined by pre-training.

6. The method according to claim 5, wherein the pre-training comprises: Acquire a reference query set and a reference response set as positive samples, a reference query set and a reference response set as negative samples, and a training query set for the pre-training; Determining a set of training second representations corresponding to the training query set; as well as The first parameter and the second parameter are adjusted so that the training second representation set is close to the first distance set of the positive samples, and the training second representation set is far away from the second distance set of the negative samples.

7. The method according to claim 6, wherein the comparison head is trained based on the following loss function: a first loss function associated with the first representation; a second loss function associated with the first distance and the second distance; and A third loss function is associated with the second set of representations and a plurality of sets of entities in a knowledge base.

8. A method according to claim 7, wherein the third loss function is associated with a lookup table that maps the set of training second representations to corresponding entities in the multiple entities, and the lookup table assigns a score to each training second representation and corresponding entity in the set of training second representations based on a co-occurrence frequency in a training query set, and the co-occurrence frequency indicates the probability that the training second representation correctly matches the entity in the corresponding domain.

9. The method of claim 1, wherein the decoder corresponding to the domain comprises an encoder having specific parameters corresponding to the domain.

10. The method according to claim 9, wherein the decoder corresponding to a specific domain is trained using multiple first loss functions, multiple second loss functions, and multiple third loss functions of multiple different domains, and multiple weights associated with the multiple different domains.

11. An electronic device, comprising: processor; as well as a memory coupled to the processor, the memory having instructions stored therein, the instructions, when executed by the processor, causing the device to perform actions, the actions comprising: Based on a query input by a user to the chatbot, determining a first representation associated with the query; generating a second representation based on the first representation and the domain to which the query belongs, wherein the dimension of the second representation is smaller than that of the first representation; and A response to the query is generated by a decoder corresponding to the domain based on the second representation.

12. The electronic device of claim 11, wherein based on a query input by a user to the chatbot, determining a first representation associated with the query comprises: The first representation associated with the query is determined based on at least one of a sentence input by the user, a historical dialogue between the user and the chatbot, and a knowledge base of the domain.

13. The electronic device according to claim 12, wherein determining the first representation associated with the query based on at least one of a statement input by the user, a historical conversation between the user and the chatbot, and a knowledge base of the domain comprises: Obtaining at least one of the statement, the historical conversation, and the knowledge base as the query; as well as Based on the query, a latent vector is generated as the first representation using a shared encoder of the chatbot, wherein the shared encoder is capable of encoding queries belonging to any domain.

14. The electronic device of claim 11, wherein based on the first representation and the domain to which the query belongs, generating the second representation comprises: performing a linear transformation on the first representation to determine a third representation; as well as The third representation is normalized to determine the second representation.

15. The electronic device of claim 14, wherein the chatbot comprises a contrast head for generating the second representation, and the first parameter and the second parameter associated with the linear transformation are determined by pre-training.

16. The electronic device according to claim 15, wherein the pre-training comprises: Acquire a reference query set and a reference response set as positive samples, a reference query set and a reference response set as negative samples, and a training query set for the pre-training; Determining a set of training second representations corresponding to the training query set; as well as The first parameter and the second parameter are adjusted so that the training second representation set is close to the first distance set of the positive samples, and the training second representation set is far away from the second distance set of the negative samples.

17. The electronic device according to claim 16, wherein the contrast head is trained based on the following loss function: a first loss function associated with the first representation; a second loss function associated with the first distance and the second distance; and A third loss function is associated with the second set of representations and a plurality of sets of entities in a knowledge base.

18. An electronic device according to claim 17, wherein the third loss function is associated with a lookup table that maps the set of training second representations to corresponding entities among the multiple entities, and the lookup table assigns a score to each training second representation and corresponding entity in the set of training second representations based on a co-occurrence frequency in a training query set, and the co-occurrence frequency indicates the probability that the training second representation correctly matches the entity in the corresponding field.

19. An electronic device according to claim 11, wherein the decoder corresponding to the domain includes an encoder having specific parameters corresponding to the domain, wherein the decoder corresponding to the specific domain is trained using multiple first loss functions, multiple second loss functions, and multiple third loss functions of multiple different domains, and multiple weights associated with the multiple different domains.

20. A computer program product tangibly stored on a computer readable medium and comprising computer executable instructions which, when executed by a device, cause the device to: Based on a query input by a user to the chatbot, determining a first representation associated with the query; generating a second representation based on the first representation and the domain to which the query belongs, wherein the dimension of the second representation is smaller than that of the first representation; as well as A response to the query is generated by a decoder corresponding to the domain based on the second representation.

Images