Conversational management method and apparatus

Abstract

A dialogue management method and apparatus are disclosed. The dialogue management method includes: sequentially determining an application domain for providing services to a user, a function among the functions of the application domain that is suitable for the user's intent, and at least one slot for performing the function; adaptively determining an expression scheme for a dialogue management interface according to the progress of the sequentially determined steps; and displaying the dialogue management interface.

Description

[0001] This application claims the benefit of Korean Patent Application No. 10-2019-0094048, filed on August 1, 2019, with the Korean Intellectual Property Office, the entire disclosure of which is incorporated herein by reference for all purposes. Technical Field

[0002] The following description relates to a dialogue management approach based on a dialogue management framework, as well as devices that include interactive interfaces and natural language understanding (NLU). Background Technology

[0003] Chatbot technology, designed to overcome the limitations of voice or speech recognition, is evolving into an interactive interface technology. A chatbot is a compound word for chat and robot, and refers to a computer program configured to perform a dialogue in response to audio or text input from a user. For example, chatbots can be used in customer support or business settings, such as for 24-hour customer service.

[0004] NLU is a technology used to understand natural language spoken by humans. NLU can automatically analyze the intent corresponding to user utterances from audio or text input. Summary of the Invention

[0005] This summary is provided to introduce, in a simplified form, the selection of concepts further described in the following detailed description. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to help determine the scope of the claimed subject matter.

[0006] In one general aspect, a dialogue management method is provided, the dialogue management method comprising: sequentially determining, based on user interactions including user utterances, an application domain for providing services to the user, a function related to the user's intent among the functions of the application domain, and at least one slot for performing the function; adaptively determining an expression scheme of a dialogue management interface based on the application domain, the function, and the at least one slot, according to the progress of the step of determining the application domain; and displaying the dialogue management interface based on the expression scheme.

[0007] In another general aspect, a dialogue management device is provided, the dialogue management device comprising: a memory in which a program is stored; and a processor configured to execute the program, wherein the program is configured to perform the following steps: based on user interaction including user utterances, sequentially determining an application domain for providing services to the user, a function related to the user's intent among the functions of the application domain, and at least one slot for performing the function; adaptively determining an expression scheme of a dialogue management interface based on the application domain, the function, and the at least one slot, according to the progress of the step of determining the application domain; and displaying the dialogue management interface based on the expression scheme.

[0008] In one general aspect, a dialogue management method includes: based on user interactions including user utterances, sequentially determining an application domain for providing services to the user, a function among a plurality of functions of the application domain that is appropriate to the user's intent, and at least one slot required to perform the function; adaptively determining an expression scheme for a dialogue management interface based on the application domain, the function, and the at least one slot, according to the progress of the sequentially determined steps; and displaying the dialogue management interface based on the adaptively determined expression scheme.

[0009] The adaptive determination step may include: when the progress corresponds to a request for the at least one slot, determining an expression scheme such that the requested slot does not include the value of the requested slot.

[0010] The dialogue management method may further include: before determining the application domain, displaying multiple interface objects in the dialogue management interface that indicate multiple application domains installed on the user's terminal.

[0011] The sequential determination steps may include: determining the application domain based on input from an interface object that indicates one of the plurality of interface objects through an input method other than utterance.

[0012] The multiple interface objects can be displayed as moving objects within the dialog management interface, rather than being fixed.

[0013] The dialogue management method may further include: parameterizing information included in the at least one slot; and invoking the application programming interface (API) of the function in the application domain based on the parameterized information.

[0014] The sequential decision-making steps may include at least one of the following: selecting an application domain corresponding to the content of the discourse from a plurality of application domains installed on the user's terminal; and selecting a function corresponding to the content of the discourse from the plurality of functions.

[0015] The sequential determination steps may include at least one of the following: requesting an application domain for providing services in response to determining that there is no application domain corresponding to the content of the utterance among a plurality of application domains installed on the user's terminal; and querying an application domain for providing services among the plurality of candidate application domains in response to determining that there are a plurality of candidate application domains corresponding to the content of the utterance among a plurality of application domains installed on the user's terminal.

[0016] The sequential determination steps may include at least one of the following: in response to determining that there is no function among the plurality of functions that corresponds to the content of the utterance, requesting a function that is suitable for the user's intent; and in response to determining that there are a plurality of candidate functions among the plurality of functions that correspond to the content of the utterance, querying among the plurality of candidate functions a function that is suitable for the user's intent.

[0017] The sequential determination steps may include at least one of the following: requesting a first slot in response to a utterance that does not contain content sufficient to determine a first slot among the at least one slots; and querying a second slot among the plurality of candidate values ​​in response to the utterance containing a plurality of candidate values ​​corresponding to a second slot among the at least one slots.

[0018] The adaptive determination steps may include at least one of the following: determining an expression scheme that highlights the application domain among a plurality of application domains installed on the user's terminal; and determining an expression scheme that highlights the function among the plurality of functions that is appropriate to the user's intent.

[0019] The displayed steps may include at least one of the following: in response to the application domain being determined, displaying multiple interface objects in the dialog management interface that indicate multiple functions of the application domain; and in response to the function being determined, displaying at least one interface object in the dialog management interface that indicates at least one slot.

[0020] The adaptive determination steps may include: determining an expression scheme that visualizes the hierarchical structure between the application domain, the functionality, and the at least one slot.

[0021] In response to receiving any one or any combination of inputs indicating an interface object after the application domain has been determined, input indicating an interface object after the function has been determined, and input indicating an interface object indicating a determined slot after at least one of the at least one slots has been determined, the sequential determination step may include rolling back the sequential determination for at least one of the following: a component indicated by an interface object corresponding to the received input, the component corresponding to any one or any combination of the application domain, the function, and the at least one slot; and a component in the hierarchy corresponding to a lower level of the component.

[0022] The adaptive determination steps may include: rolling back the determined expression scheme for at least one of the following: a component indicated by an interface object corresponding to the received input, the component corresponding to any one or any combination of the application domain, the function, and the at least one slot; and a component in the hierarchy corresponding to a lower level of the component.

[0023] The dialogue management method may further include: storing parameterized information in a storage space; displaying the parameterized information in an interface indicating the storage space in the dialogue management interface in response to input selecting the storage space; and receiving drag-and-drop input from the interface indicating the storage space to drag the parameterized information and place the parameterized information on one of the at least one slot, wherein the sequential determination step may include: determining the slot such that it corresponds to the parameterized information.

[0024] In another general aspect, a dialogue management device includes: a memory in which a program is stored; and a processor configured to execute the program, wherein the program is configured to perform the following steps: based on user interaction including user utterances, sequentially determining an application domain for providing services to the user, a function among a plurality of functions of the application domain that is appropriate to the user's intent, and at least one slot required to execute the function; adaptively determining an expression scheme for a dialogue management interface based on the application domain, the function, and the at least one slot, according to the progress of the sequentially determined steps; and displaying the dialogue management interface based on the adaptively determined expression scheme.

[0025] Other features and aspects will become clear from the following detailed description, drawings, and claims. Attached Figure Description

[0026] Figure 1 An example of a dialogue management method is shown.

[0027] Figure 2 This shows an example of the default screen for the conversation management framework.

[0028] Figure 3 This illustrates an example of determining the application domain used to provide services to users within a dialogue management framework.

[0029] Figure 4 This illustrates an example of determining, within a defined application domain, the functionality applicable to the user's intent within a dialogue management framework.

[0030] Figure 5 This example illustrates at least one slot required for the function to execute a decision within a dialogue management framework.

[0031] Figure 6 This illustrates an example of at least one slot required for the function of requesting the execution of a decision in a dialogue management framework.

[0032] Figure 7 This shows an example of the result of rolling back the decision to apply the domain, the domain's functionality, and at least one slot.

[0033] Figure 8 An example is shown that uses information previously parameterized on the user terminal to determine at least one slot.

[0034] Figure 9 This illustrates an example of sending a request to the user to confirm the results of a decision regarding the application domain, the domain's functionality, and at least one slot.

[0035] Figure 10 An example of a dialogue management method is shown.

[0036] Figure 11 This shows an example of the configuration of the dialogue management framework.

[0037] Throughout the accompanying drawings and detailed embodiments, unless otherwise described or provided, the same reference numerals will be understood to denote the same elements, features, and structures. The drawings may not be to scale, and for clarity, illustration, and convenience, the relative dimensions, scale, and depiction of elements in the drawings may be exaggerated. Detailed Implementation

[0038] The following detailed embodiments are provided to aid the reader in gaining a comprehensive understanding of the methods, apparatus, and / or systems described herein. However, various changes, modifications, and equivalents of the methods, apparatus, and / or systems described herein will become apparent upon understanding this disclosure. For example, the order of operations described herein is merely illustrative and is not limited to those orders set forth herein, but may be altered as will become clear upon understanding this disclosure, except for operations that must occur in a specific order.

[0039] The features described herein may be implemented in different forms and should not be construed as being limited to the examples described herein. Rather, the examples described herein have been provided only to illustrate some of the many feasible ways of implementing the methods, apparatus, and / or systems described herein that will become clear upon understanding the disclosure of this application.

[0040] While terms such as “first,” “second,” and “third” may be used herein to describe various components, assemblies, regions, layers, or parts, these components, assemblies, regions, layers, or parts should not be limited by these terms. Rather, these terms are used only to distinguish one component, assembly, region, layer, or part from another. Thus, without departing from the teaching of the examples described herein, the first component, assembly, region, layer, or part referred to as the second component, assembly, region, layer, or part may also be referred to as the second component, assembly, region, layer, or part.

[0041] It should be noted that if a component is described as "connected", "combined", or "linked" to another component, then while the first component may be directly connected, combined, or linked to the second component, a third component may be "connected", "combined", or "linked" between the first and second components.

[0042] Unless the context clearly indicates otherwise, the singular form is intended to include the plural form as well. It will also be understood that when the terms “comprising” and / or “including” are used herein, they indicate the presence of the stated features, wholes, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or groups thereof.

[0043] The examples described below can be applied to a variety of devices that support voice recognition or speech recognition (such as, for example, smartphones, mobile phones, wearable devices (such as rings, watches, glasses, eyeglasses, bracelets, anklets, belts, necklaces, earrings, headbands, helmets, devices embedded in clothing, or eyeglass displays (EGD)), computing devices (such as servers, laptops, notebook computers, minicomputers, netbooks, tablet PCs, tablet phones, mobile internet devices (MIDs), personal digital assistants (PDAs), enterprise digital assistants (EDAs), ultra-mobile personal computers (UMPCs), portable laptop computers (PCs), and electronic products (such as...). Robots, digital cameras, digital camcorders, portable game consoles, MP3 players, portable / personal multimedia players (PMPs), handheld e-readers, Global Positioning System (GPS) navigators, personal navigation devices, portable navigation devices (PNDs), handheld game consoles, e-readers, televisions (TVs), high-definition televisions (HDTVs), smart TVs, smart home appliances, smart home devices or security devices for door control, walking aids, smart speakers, robots), various Internet of Things (IoT) devices, smart cars, autonomous vehicles, autonomous driving systems or autonomous driving systems, intelligent vehicles, advanced driver assistance systems (ADAS), head-up displays (HUDs) and augmented reality head-up displays (AR HUDs), and can be executed by applications, middleware or operating systems installed on user devices, or by programs on servers that interact with the corresponding applications.

[0044] Figure 1 An example of a dialogue management method is shown. Although Figure 1 The operations can be performed in the order and manner shown, but without departing from the spirit and scope of the illustrative examples described, the order of some operations may be changed, or some operations may be omitted. Figure 1 Many of the operations shown can be performed in parallel or simultaneously. Figure 1 One or more blocks, and combinations thereof, may be implemented by a computer and apparatus (such as a processor) based on dedicated hardware that performs a specific function, or by a combination of dedicated hardware and computer instructions.

[0045] Reference Figure 1 The dialogue management method includes a preparation operation 110, a domain decision operation 120, an intent decision operation 130, and a slot request & value / argument decision operation 140. In one example, the dialogue management method may also include a confirmation operation 150 and an application programming interface (API) call operation 160.

[0046] In preparation operation 110, the dialogue management framework waits for user input after outputting the default screen. User input includes verbal input and input via methods other than verbal input. Input methods other than verbal input include, for example, on-screen clicks, on-screen touches, and input that tracks the user's gaze.

[0047] In one example, the user's speech is input as audio via a microphone. In another example, the user's speech is received via an input device such as a keyboard or touchpad. On-screen clicks are received via an input device such as a mouse, and on-screen touches are received via an input device such as a touchscreen. The user's gaze is tracked using techniques that capture images of the user in real-time, contact lenses with embedded mirrors or detection coils, and sensors attached around the user's eyes. Furthermore, user input includes input via gestures and input via actions.

[0048] In one example, when a user's speech is input as audio, the dialogue management framework converts the audio to text. In this example, the dialogue management framework uses a speech-to-text engine to convert the audio to text. When the user's speech is received as text via an input device (such as a keyboard or touchpad), the dialogue management framework uses the received speech instead of converting it to text.

[0049] In preparation operation 110, the default screen display output from the dialog management framework indicates at least a portion of multiple interface objects from multiple application domains installed on the user's terminal.

[0050] An interface object is an information component that provides information to a user. Information components provide information to the user through various means, such as visual, audio, tactile, olfactory, and gustatory information delivery. In one example, the interface object includes at least one of images and icons that provide visual information to the user.

[0051] An interface object is an input control that receives user input. Input controls receive user input through various input methods, such as mouse input, keyboard input, gesture input, action input, and voice input. For example, an interface object includes at least one of a button, checkbox, radio button, and toggle switch.

[0052] An interface object is an element that combines information components and input controls. For example, an interface object may include at least one of optional images, optional lists, and optional icons.

[0053] In domain decision operation 120, when user input about the interface object to be displayed is received, the dialog management framework selects the application domain indicated by the interface object that received the user input.

[0054] The application domain to be displayed on the default screen is selected from multiple application domains based on various criteria. For example, for all application domains installed on the terminal, the interface object of the application domain is indicated to be displayed on the default screen. In one example, the interface object of the most recently installed application domain is indicated to be displayed on the default screen. In another example, the interface object of the most recently used domain is indicated to be displayed on the default screen. In yet another example, the interface object of a domain individually configured by the user is indicated to be displayed on the default screen. Besides the examples above, the application domain to be displayed on the default screen can be selected in various other ways.

[0055] The following will refer to Figure 2 Further description of preparation operation 110.

[0056] In domain determination operation 120, the dialogue management framework determines the application domain for providing services to the user based on interactions with the user. In intent determination operation 130 and slot request & value / parameter determination operation 140, the dialogue management framework determines an intent or at least one slot based on interactions with the user, which will be described later. In one example, interactions with the user include input received by the dialogue management framework from the user and output sent from the dialogue management framework to the user. In one example, input received by the dialogue management framework from the user includes the user's utterances, clicks on interface objects, touches on the location of displayed interface objects, and tracking the user's gaze. Output sent from the dialogue management framework to the user includes requests for the application domain, intent, and at least one slot; queries for the application domain, intent, and at least one slot; and confirmation requests for the determined application domain, the determined intent, and the determined at least one slot.

[0057] The dialogue management framework sends a request when there is no application domain corresponding to the content of the utterance, or when there is no function corresponding to the content of the utterance, or when the utterance does not include content sufficient to determine the first slot in at least one slot.

[0058] The dialogue management framework sends a query when there are multiple candidate application domains corresponding to the content of the utterance, or when there are multiple candidate functions corresponding to the content of the utterance, or when the content of the utterance includes multiple candidate values ​​corresponding to a second slot in at least one slot.

[0059] In domain determination operation 120, the dialogue management framework displays multiple interface objects in the dialogue management interface that indicate multiple application domains installed on the user's terminal. The user selects the application domain from the multiple interface objects indicating the multiple application domains to be determined as the application domain for providing services to the user. The user selects the application domain by at least one of the following: utterance; and at least one of the following input methods other than utterance. In this example, the dialogue management framework determines the application domain for providing services to the user based on the input received from the user. For example, when the input received from the user corresponds to a click or touch, the dialogue management framework determines the application domain of the interface object that receives the click or touch input as the application domain for providing services to the user.

[0060] When input received from a user corresponds to a user's utterance, the dialogue management framework selects the application domain corresponding to the content of the received utterance as the application domain for providing services to the user. To select the application domain corresponding to the content of the received utterance, the dialogue management framework compares keywords included in the utterance content with the name of the application domain. In one example, the dialogue management framework uses a pre-trained machine learning model to select the application domain corresponding to the content of the received utterance. In another example, without departing from the spirit and scope of the illustrative example described, the dialogue management framework may use various other methods to select the application domain corresponding to the content of the received utterance.

[0061] In response to determining that no application domain corresponds to the input received from the user, or that multiple application domains correspond to the input received from the user, the dialogue management framework requests or queries the application domain used to provide services to the user. In one example, in response to determining that no application domain corresponds to the received input, the dialogue management framework requests the application domain used to provide services. When multiple candidate application domains correspond to the received input, the dialogue management framework queries the application domain used to provide services. In one example, when the input received from the user corresponds to the user's utterance, it is more likely to be misunderstood and misinterpreted compared to input via means other than utterance. Therefore, the dialogue management framework requests or queries the application domain. In one example, this request or query is directed to the user.

[0062] In response to determining that no application domain corresponds to the input received from the user, or that multiple application domains correspond to the input received from the user, the dialogue management framework may wait to receive new input from the user instead of immediately sending a request or query to determine the application domain to provide services to the user. In one example, when the input received from the user corresponds to a user click on an interface object or a touch on the location of the displayed interface object, there is a possibility that the click or touch may be accidentally performed when the user moves the terminal or performs another action associated with the terminal. Therefore, the dialogue management framework may wait to receive new input from the user. In one example, if no new input is received from the user within a threshold time, the dialogue management framework requests or queries the application domain.

[0063] When the application domain used to provide services to users is determined, the dialogue management framework displays multiple interface objects in the dialogue management interface that indicate the functions of the determined application domain, which will be further described below for intent determination operation 130.

[0064] The following will refer to Figure 3 Further description of the domain determines operation 120.

[0065] In intent determination operation 130, the dialogue management framework determines, based on its interaction with the user, a function relevant to the user's intent (e.g., a function applicable to the user's intent) from the functions of the application domain used to provide services to the user. The dialogue management framework displays multiple interface objects in the dialogue management interface that indicate the functions of the application domain determined in domain determination operation 120. The user performs input to an interface object that indicates a function applicable to the user's intent. In this example, based on the input received from the user, the dialogue management framework determines the function applicable to the user's intent from the functions of the determined application domain. In one example, the dialogue management framework determines the function applicable to the user's intent as indicated by the interface object that received the input.

[0066] When the input received from the user corresponds to the user's utterance, the dialogue management framework selects the functionality corresponding to the content of the received utterance as the appropriate functionality for the user's intent. In one example, to select the functionality corresponding to the content of the received utterance, the dialogue management framework compares keywords included in the content of the utterance with the names of the functionality in the selected application domain. In another example, the dialogue management framework uses a pre-trained machine learning model to select the functionality corresponding to the content of the received utterance. Furthermore, without departing from the spirit and scope of the illustrative examples described, the dialogue management framework may use various existing techniques to select the functionality corresponding to the content of the received utterance.

[0067] Similar to when determining an application domain, if it is determined that no function corresponds to the input received from the user, or if multiple functions correspond to the input received from the user, the dialogue management framework requests or queries for functions suitable for the user's intent. When it is determined that no function corresponds to the received input, the dialogue management framework requests functions suitable for the user's intent. When it is determined that multiple candidate functions correspond to the received input, the dialogue management framework queries for functions suitable for the user's intent. In one example, when the input received from the user corresponds to the user's utterance, the dialogue management framework requests or queries for functions suitable for the user's intent. In one example, this request or query is directed at the user.

[0068] When it is determined that no function corresponds to the input received from the user, or that multiple functions correspond to the input received from the user, the dialogue management framework may wait to receive new input from the user. In one example, when the input received from the user corresponds to a user click on an interface object or a touch on the location of the displayed interface object, the dialogue management framework may wait for a predetermined threshold time to receive new input from the user.

[0069] When the function applicable to the user's intent is determined, the dialogue management framework displays at least one interface object in the dialogue management interface that indicates at least one slot for performing the decision, which will be further described below for slot request & value / parameter decision operation 140.

[0070] The following will refer to Figure 4 Further description of the intent determines operation 130.

[0071] In the slot request & value / parameter decision operation 140, the dialogue management framework displays at least one slot in the dialogue management interface for performing the function determined in the intent decision operation 130. In this example, the dialogue management framework displays at least one slot in the dialogue management interface by displaying at least one interface object indicating at least one slot for performing the function.

[0072] Regarding the first slot among at least one slot, the dialogue management framework determines the first slot to correspond to at least a portion of the content of the utterance when the utterance includes content sufficient to determine the first slot. For example, when the first slot corresponds to a slot associated with "message content," and the content of the utterance includes content sufficient to determine the message content, the dialogue management framework determines the first slot to correspond to the message content included in the utterance.

[0073] The dialogue management framework determines the first slot by deciding its value. The value of the first slot can include its parameters. The parameters of the first slot correspond to the data provided as input to functions used to call the application programming interface (API).

[0074] In one example, the dialogue management framework simultaneously displays at least one slot for functions used to perform decisions within the dialogue management interface. Alternatively, in another example, the dialogue management framework sequentially displays at least one slot for functions used to perform decisions within the dialogue management interface.

[0075] The dialogue management framework receives responses from the user to requests or queries sent to the user in domain decision operation 120, intent decision operation 130, and slot request & value / parameter determination operation 140. In this example, user interaction includes responses from the user. When a response is received from the user to a request or query sent to the user in domain decision operation 120, the dialogue management framework determines the application domain for providing services to the user based on the response received from the user. When a response is received from the user to a request or query sent to the user in intent decision operation 130, the dialogue management framework determines the functionality applicable to the user's intent based on the response received from the user. When a response is received from the user to a request or query sent to the user in slot request & value / parameter determination operation 140, the dialogue management framework determines at least one slot based on the response received from the user.

[0076] When the domain decision operation 120, intent decision operation 130, and slot request & value / parameter decision operation 140 are executed sequentially, the dialogue management framework adaptively determines the representation scheme of the dialogue management interface based on the application domain used to provide services to the user, the functions within the application domain used to provide services to the user that are suitable for the user's intent, and at least one slot for executing the functions suitable for the user's intent. The dialogue management framework determines the representation scheme to visualize the hierarchical structure between the application domain, the functions within the application domain that are suitable for the user's intent, and at least one slot for executing the functions suitable for the user's intent. In this example, the dialogue management framework visually presents the application domain, the functions within the application domain that are suitable for the user's intent, and at least one slot for executing the functions suitable for the user's intent, forming a hierarchical structure sequentially.

[0077] The dialogue management framework determines the representation of the dialogue management interface to visually present the hierarchical structure. For example, the dialogue management framework uses a tree structure to determine the representation of the dialogue management interface. In the tree structure, the application domain is at the top, branches from the application domain indicate the functions of the application domain, and branches from the functions of the application domain indicate at least one slot.

[0078] The dialogue management framework determines the representation scheme of the dialogue management interface to highlight the application domain used to provide services to the user (the application domain determined in domain decision operation 120) and the functions applicable to the user's intent (the functions determined in intent decision operation 130). For example, when the application domain used to provide services to the user is determined in domain decision operation 120, the dialogue management framework determines the representation scheme of the dialogue management interface to increase the size of the interface object indicating the determined application domain among multiple interface objects indicating multiple application domains installed on the user's terminal, so as not to display the interface objects indicating the remaining domains from the dialogue management interface.

[0079] In one example, the dialogue management framework determines the presentation scheme of the dialogue management interface to highlight, in real time, the slot associated with user input among at least one slot used to perform a function appropriate to the user's intent. For example, when the dialogue management framework needs to receive input associated with "message content" from the user to perform a "message sending" function, the slot associated with "message content" is highlighted in real time.

[0080] When at least one slot is requested, the dialog management framework determines the representation scheme of the dialog management interface to display the requested slot, excluding the value of the requested slot. For example, when a slot is requested because the value of the slot associated with "Message Content" is empty, the slot associated with "Message Content" is displayed in the dialog management interface. Furthermore, by design, the slot to be requested can be highlighted in the dialog management interface.

[0081] For ease of description, the operations for determining the expression scheme of the dialogue management interface are described separately from the domain determination operation 120, intent determination operation 130, and slot request & value / parameter determination operation 140. However, the operations for determining the expression scheme of the dialogue management interface may not be performed separately from operations 120 to 140. As operations 120 to 140 are executed sequentially, the expression scheme of the dialogue management interface is adaptively determined. For example, the dialogue management framework progressively determines the expression scheme of the dialogue management interface based on the progress of operations 120 to 140.

[0082] The following will refer to Figures 2 to 8 Further describe the presentation scheme of the dialogue management interface.

[0083] In confirmation operation 150, the dialogue management framework confirms whether the application domain for providing services to the user was correctly determined in domain determination operation 120, whether the function applicable to the user's intent was correctly determined in intent determination operation 130, and whether at least one slot for performing the function was correctly determined in slot request & value / parameter determination operation 140. Specifically, the dialogue management framework queries the user to confirm whether the application domain, function, and at least one slot were correctly determined. The dialogue management framework displays the message in the dialogue management interface confirming the correct determination of the application domain, function, and at least one slot.

[0084] The dialogue management framework displays the dialogue management interface based on an adaptively determined expression scheme. The dialogue management framework displays the dialogue management interface in a way that makes the results of determining the application domain, function, and at least one slot visually easily identifiable. The dialogue management framework displays the dialogue management interface after performing domain determination operation 120, intent determination operation 130, and slot request & value / parameter determination operation 140. The dialogue management framework displays the dialogue management interface after the user confirms in confirmation operation 150 whether the application domain, function, and at least one slot have been correctly determined.

[0085] For ease of description, the confirmation operation 150 is described separately from the domain decision operation 120, intent decision operation 130, and slot request & value / parameter decision operation 140. It is not necessary to perform the tasks of confirming whether the application domain for providing services to the user was correctly determined in the domain decision operation 120, whether the function applicable to the user's intent was correctly determined in the intent decision operation 130, and whether at least one slot for performing that function was correctly determined in the slot request & value / parameter decision operation 140 through separate operations. In one example, the dialogue management framework confirms whether the application domain for providing services to the user was correctly determined in the domain decision operation 120, whether the function applicable to the user's intent was correctly determined in the intent decision operation 130, and whether at least one slot for performing that function was correctly determined in the slot request & value / parameter decision operation 140.

[0086] The following will refer to Figure 8 Further description confirms operation 150.

[0087] In API call operation 160, the dialogue management framework invokes the API based on information included in at least one slot determined in slot request & value / parameter determination operation 140. The dialogue management framework parameterizes the first slot based on the value of the first slot. In one example, the dialogue management framework immediately parameterizes the first slot based on its parameters.

[0088] Based on parameterized information, the dialogue management framework invokes the API of the function determined in intent-determination operation 130. Within the application domain determined in domain-determination operation 120, the API of the function determined in intent-determination operation 130 is invoked.

[0089] Based on the invoked API, operations appropriate to the user's intent are executed on the user's terminal. For example, the dialogue management framework determines the application domain used to provide services to the user as the "message domain," the function appropriate to the user's intent as "message sending," the slot "recipient" as "John," and the slot "message content" as a picture of noodles. In this example, when the dialogue management framework invokes the "message sending" API within the "message domain," the user's terminal sends a picture of noodles to "John."

[0090] When the dialogue management method is executed, the user-selectable commands are effectively displayed on the default screen output by the dialogue management framework. Furthermore, the dialogue management interface effectively displays information on whether the commands executed by the dialogue management framework have been appropriately entered as intended by the user. Additionally, the dialogue management interface effectively displays information necessary for the session between the dialogue management framework and the user, as well as the task progress. Moreover, the input content can be easily edited.

[0091] The following will refer to Figure 7 Further describe an example of editing input content.

[0092] Figure 2 This shows an example of the default screen for the conversation management framework.

[0093] Reference Figure 2 In the absence of user input, the dialogue management framework outputs a default screen and awaits input from the user (210). The default screen displays at least a portion of multiple interface objects indicating multiple application domains installed on the user's terminal. By displaying interface objects indicating application domains on the default screen, the dialogue management framework visually displays the application domains installed on the terminal to the user. When input is received from the user for the displayed interface objects, the application domain used to provide services to the user is determined.

[0094] In one example, multiple interface objects are shown moving within the dialog management interface, rather than being fixed. For instance, the multiple interface objects move irregularly, as if floating on water. Other types of movement of multiple interface objects (such as random movement, movement created by gradient patterns, or movement through a fixed grid) may be used without departing from the spirit and scope of the illustrative example described.

[0095] In one example, a portion of multiple interface objects moving within the dialog management interface can be moved off-screen and disappear. Furthermore, when a new interface object moves from off-screen to on-screen, the new interface object can be displayed. In this example, more interface objects are displayed on the default screen compared to an example where only pre-defined interface objects are displayed on the default screen.

[0096] The dialogue management framework displays messages in the dialogue management interface that enable users to perform input on interface objects. In one example, the dialogue management framework displays a message stating "How can I help you?" in the dialogue management interface, thus notifying the user that the dialogue management framework is ready to receive user input.

[0097] exist Figure 1 In the preparation operation 110, refer to the following. Figure 2 The described operation.

[0098] Figure 3 This illustrates an example of determining the application domain used to provide services to users within a dialogue management framework.

[0099] Reference Figure 3 The dialogue management framework determines the application domain to provide services to the user based on the user's interaction. Specifically, the framework displays the results of receiving user input in the dialogue management interface. It also displays a message stating the name of the domain corresponding to the user input. For example, when a user touches the interface object corresponding to the domain "Telephone" displayed on the default screen, the framework displays a message stating "Telephone" in the dialogue management interface.

[0100] When the input received from the user corresponds to the user's utterance, the dialogue management framework displays a message containing the content of the utterance in the dialogue management interface. For example, when the user receives the utterance stating "phone text message", the dialogue management framework displays a message stating "phone text message" in the dialogue management interface (310).

[0101] The dialogue management framework displays the results of user input in the dialogue management interface. When determining the application domain for providing services to the user based on user input, the dialogue management framework determines the presentation scheme of the dialogue management interface to highlight the determined application domain.

[0102] The dialogue management framework highlights the display of interface objects corresponding to the determined application domain. For example, when the application domain used to provide services to users is determined to be "telephone," the dialogue management framework will display the interface object corresponding to the application domain "telephone" as larger than other interface objects.

[0103] The dialogue management framework displays a message in the dialogue management interface stating the name of the decided application domain to highlight the decided application domain. For example, when the application domain used to provide services to users is determined to be "telephone," the dialogue management framework displays a message in the dialogue management interface stating "Select Telephone Application."

[0104] When multiple application domains are identified as corresponding to input received from the user, the dialogue management framework queries the application domain used to provide services to the user. For example, when the user receives the statement "phone SMS", the dialogue management framework displays the message "Select application, [phone][message]" in the dialogue management interface (320).

[0105] When multiple application domains are identified as corresponding to input received from the user, the dialogue management framework determines the representation scheme of the dialogue management interface to highlight the multiple application domains identified as corresponding to the input received from the user. For example, when the user receives the statement "phone / text message," the dialogue management framework will display the interface objects corresponding to the application domains "phone" and "message" as larger than other interface objects.

[0106] exist Figure 1 The domain determines the execution reference in operation 120. Figure 3 The described operation.

[0107] Figure 4 This illustrates an example of determining the appropriate user intent from the functions of the determined application domain within a dialogue management framework.

[0108] Reference Figure 4 The dialogue management framework, based on user interaction, determines the appropriate functionality for the user's intent from the functions of the application domains used to provide services to the user. The framework highlights the determined application domains and displays their functions in the dialogue management interface. The functions of the application domains are displayed as interface objects, which include images, photos, icons, and dialog boxes.

[0109] The dialogue management framework determines the representation scheme of the dialogue management interface to visually present the hierarchical structure between the determined application domain and the functions of the determined application domain. For example, as shown in the dialogue management interface (420), the dialogue management framework determines the representation scheme of the dialogue management interface such that the determined application domain "telephone" is on the left, and the branches coming out of the application domain indicate the functions of the application domain "telephone" (e.g., "Enable Wi-Fi Calling", "Wi-Fi Calling", "Calling", and "Auto-Answer Settings").

[0110] Similar to when determining the application domain, the dialogue management framework displays the result of receiving user input in the dialogue management interface when determining the function applicable to the user's intent. When the user touches the interface object corresponding to the function "Open Wi-Fi Call", the dialogue management framework displays the message "Open Wi-Fi Call" in the dialogue management interface. As another example, when the dialogue management framework receives the statement "Call Wi-Fi" from the user, it displays the message "Call Wi-Fi" in the dialogue management interface (410).

[0111] When determining the appropriate function based on user input, the dialogue management framework defines the presentation of the dialogue management interface to highlight the determined function. The interface will display the interface object corresponding to the determined function as a highlight. For example, when the appropriate function is determined to be "Turn on Wi-Fi calling," the interface object corresponding to "Turn on Wi-Fi calling" will be displayed larger than the interface objects corresponding to other functions. In one example, the dialogue management framework also displays a message indicating the name of the determined function in the dialogue management interface.

[0112] When multiple functions are identified to correspond to input received from the user, in one example, the dialogue management framework queries the functions applicable to the user's intent. For example, when the user receives the statement "Call Wi-Fi", the dialogue management framework displays the message "Select command, [Open Wi-Fi Call][Wi-Fi Call]" (420) in the dialogue management interface.

[0113] When multiple functions are identified as corresponding to input received from the user, the dialogue management framework determines the presentation scheme of the dialogue management interface to highlight the functions identified as corresponding to the user input. This highlighting can be done in various ways. For example, when the user receives the statement "Call Wi-Fi," the dialogue management framework will display the interface objects corresponding to the functions "Turn on Wi-Fi Call" and "Wi-Fi Call" as larger than the interface objects corresponding to other functions, and will also display the interface objects corresponding to "Turn on Wi-Fi Call" and "Wi-Fi Call" with thicker borders than the interface objects corresponding to other functions.

[0114] exist Figure 1 The intention determines the execution reference in operation 130. Figure 4 The described operation.

[0115] Figure 5 This illustrates an example of at least one slot in a dialogue management framework that determines the function used to execute a decision.

[0116] Reference Figure 5The dialogue management framework displays at least one slot in the dialogue management interface for performing functions applicable to the user's intent. For example, when the user receives the statement "Send a message to John" (510), the dialogue management framework determines the application domain for providing services to the user as "Message", determines the function applicable to the user's intent as "Message Sending" (520) among the functions of the application domain "Message", and displays the slots "Recipient" and "Message Content" (530) in the dialogue management interface for performing the function "Message Sending".

[0117] Regarding the first slot among at least one slot, when the utterance includes content sufficient to determine the first slot, the dialogue management framework determines the first slot to correspond to at least a portion of the utterance's content. For example, since the utterance stating "send a message to John" includes the content "John" for determining the slot "recipient," and the content "John" is sufficient to determine the recipient, the dialogue management framework determines the "recipient" to be "John."

[0118] In another example, the dialogue management framework requests the first slot when the utterance does not include content sufficient to determine the first slot. For instance, because the utterance “send a message to John” does not include content necessary to determine the slot “message content”, the dialogue management framework requests the slot “message content”.

[0119] When a utterance includes multiple candidate values ​​for determining a second slot among at least one slot, the dialogue management framework queries the second slot among the multiple candidate values. For example, since the utterance “send a message to John and Tom” includes multiple candidate values ​​“John” and “Tom” for determining the slot “recipient”, the dialogue management framework queries the slot “recipient”.

[0120] In one example, a request sent from the dialogue management framework to the user is displayed as a message in the dialogue management interface. For instance, the dialogue management framework displays a message stating "Input content" in the dialogue management interface to request the slot "Message content".

[0121] exist Figure 1 The slot request and value / parameter determine the execution reference in operation 140. Figure 5 The described operation.

[0122] Figure 6 This illustrates an example of requesting at least one slot for functionality to perform a decision within a dialogue management framework.

[0123] Reference Figure 6Regarding the first slot in at least one slot, when the utterance does not include content sufficient to determine the first slot (e.g., when the content of the utterance is too unclear to determine the first slot), the dialogue management framework requests the first slot. For example, when a user receives the utterance “Send a message to John”, the received utterance includes “John”, which is sufficient to determine the slot “Recipient”. However, if the contacts stored in the terminal include “John (010-1111-1111)” and “Brother John (010-1111-2222)”, the content of the received utterance is too unclear to determine the slot “Recipient”. Therefore, the dialogue management framework requests the slot “Recipient”. In this example, the dialogue management framework displays the message “Select number, [John (010-1111-1111)][Brother John (010-1111-2222)]” in the dialogue management interface to determine the slot “Recipient” (630).

[0124] When the content of the utterance is unclear enough to determine the first slot, multiple candidates for the first slot are displayed in the dialog management interface. For example, when the utterance “Send a message to John” is received, the dialog management framework displays an additional dialog box near the slot “Recipient” to show the candidates “John (010-1111-1111)” and “Brother John (010-1111-2222)”. In this example, user input for a request includes the utterance as well as input methods other than the utterance (e.g., clicks and touches on the additionally displayed dialog box) (this also applies to user input for a query). In another example, user input is the movement of the user's eye toward the additionally displayed dialog box that the user expects to select.

[0125] If the same API is invoked when any of the candidates to be decided as the first slot is selected, the dialogue management framework will not request the first slot even when the content of the utterance is unclear enough to determine the first slot. For example, when the contacts stored in the terminal include "John (010-1111-1111)" and "Brother John (010-1111-1111)", the message will be sent to the same number even though the content of the received utterance "Send a message to John" is unclear enough to determine the first slot "recipient". In this example, the dialogue management framework determines the first slot by selecting one of several candidates rather than requesting the first slot.

[0126] If the same API value (not shown) is invoked in response to the selection of any candidate value, the dialogue management framework does not query the second slot, even if the content of the utterance includes multiple candidate values ​​corresponding to a second slot in at least one slot. For example, when the received utterance “Send a message to John and Jane” includes multiple candidate values ​​“John” and “Jane” for determining the slot “recipient”, and the contacts stored in the terminal include “John (010-1111-1111)” and “Jane (010-1111-1111)”, the message will be sent to the same number regardless of which is determined as the slot “recipient”. In this example, the dialogue management framework determines the second slot by selecting any one of the candidate values ​​instead of querying the second slot.

[0127] exist Figure 1 The slot request and value / parameter determine the execution reference in operation 140. Figure 6 The described operation.

[0128] Figure 7 This shows an example of the result of rolling back the decision to apply the domain, the domain's functionality, and at least one slot.

[0129] Reference Figure 7 The dialogue management framework receives input selecting any one or any combination of the following: an interface object indicating the determined application domain, an interface object indicating the determined function, and an interface object indicating at least one slot. In one example, the dialogue management framework rolls back the decision of the order for at least one of the following: a component indicated by the interface object corresponding to the received input, which corresponds to any one or any combination of the determined application domain, the determined function, and at least one slot; and a component corresponding to a lower level of the component indicated by the interface object corresponding to the received input. For example, if a user receives the utterance “Send a message to John”, the application domain for providing services to the user is determined to be “Messages”, the function applicable to the user’s intent is determined to be “Message Sending”, and the slot “Recipient” for performing that function is determined to be “John”. In this example, when the user receives input (710) touching the interface object corresponding to the function “Message Sending”, the dialogue management framework rolls back the decision of the order for at least one slot corresponding to that function and the lower-level function. In this example, the application domain for providing services to the user remains “Messages”. However, the decision associated with the function applicable to the user's intent and at least one slot for performing that function was cancelled (720).

[0130] When input is received for at least one of the interface objects indicating the application domain of the decision, the interface object indicating the function of the decision, and the interface object indicating at least one slot of the decision, the dialogue management framework rolls back the representation scheme associated with at least one of the following: a component indicated by the interface object corresponding to the received input, which corresponds to any one or any combination of the decided application domain, the decided function, and the decided at least one slot; and a component corresponding to a lower level of the component indicated by the interface object corresponding to the received input. In this example, when the component indicated by the interface object corresponding to the received input and the component corresponding to a lower level of the component indicated by the interface object corresponding to the received input are determined sequentially, the representation scheme of the associated dialogue management interface is determined again.

[0131] Figure 8 An example is shown that uses information previously parameterized on the user terminal to determine at least one slot.

[0132] Reference Figure 8 The dialogue management framework stores at least a portion of the parameterized information from the user terminal in the storage space. For example, when the message “Do you remember what we ate last time?” is sent together with a photo of the food from the user terminal to “John (010-1111-2222)”, the dialogue management framework stores the parameterized information for the corresponding operation in the storage space.

[0133] When input indicating the selection of storage space is received (810), the dialogue management framework displays parameterized information stored in the storage space in the storage space indicator interface within the dialogue management interface (820). Input to the storage space indicator interface includes clicks and touches on the storage space indicator interface. For example, when the storage space indicator interface is located at the bottom of the terminal, when the bottom of the terminal is clicked, the dialogue management framework displays at least a portion of the recently parameterized information on the terminal in the storage space indicator interface.

[0134] Regarding the result of displaying parameterized information in the interface indicating storage space, when input is received that at least a portion of the information to be displayed is dragged and dropped onto one of at least one slot, the dialogue management framework determines the slot to correspond to the dragged and dropped information. For example, when input is received that a food photo from the information to be displayed is dragged and dropped onto the slot "Content" (820), the dialogue management framework determines the slot "Content" to be the dragged and dropped food photo (830).

[0135] Figure 9 This illustrates an example of sending a request to the user to confirm the results of a decision regarding the application domain, the domain's functionality, and at least one slot.

[0136] Reference Figure 9The dialogue management framework queries the user to determine whether the application domain used to provide services to the user has been correctly determined, whether the function applicable to the user's intent has been correctly determined, and whether at least one slot used to perform that function has been correctly determined.

[0137] The dialogue management framework displays a message in the dialogue management interface to query whether the application domain, function, and at least one slot have been correctly determined. For example, when the function applicable to the user's intent is "message sending," the dialogue management framework queries the user whether the application domain, function, and at least one slot have been correctly determined by displaying a message in the dialogue management interface stating "Do you want to send this message?" (910).

[0138] exist Figure 1 The confirmation operation 150 is executed in reference. Figure 9 The described operation.

[0139] Figure 10 An example of a dialogue management method is shown. Although Figure 10 The operations in the description can be performed in the order and manner shown, but without departing from the spirit and scope of the illustrative examples described, the order of some operations may be changed, or some operations may be omitted. Figure 10 Many of the operations shown can be performed in parallel or simultaneously. Figure 10 One or more blocks, and combinations thereof, may be implemented by a computer and apparatus (such as a processor) based on dedicated hardware that performs a specific function, or by a combination of dedicated hardware and computer instructions. In addition to the following... Figure 10 In addition to the description, Figures 1 to 9 The description also applies to Figure 10 And is included here by reference. Therefore, the above description will not be repeated here.

[0140] Reference Figure 10 In operation 1010, the dialogue management framework, based on user interactions including user utterances, sequentially determines the application domain for providing services to the user, the functions within the application domain that are appropriate to the user's intent, and at least one slot for executing the functions appropriate to the user's intent. User interactions include input received by the dialogue management framework from the user and output sent from the dialogue management framework to the user. Input received by the dialogue management framework from the user includes user utterances and input via methods other than utterances. The dialogue management framework requests or queries the application domain for providing services to the user, the functions appropriate to the user's intent, and at least one slot.

[0141] In Operation 1020, the dialogue management framework adaptively determines the presentation scheme of the dialogue management interface based on a sequentially determined schedule, the application domain, the functionality applicable to the user's intent, and at least one slot. The dialogue management framework determines the presentation scheme of the dialogue management interface to visualize the hierarchical structure between the application domain, the functionality applicable to the user's intent, and at least one slot.

[0142] In operation 1030, the dialogue management framework displays the dialogue management interface based on an adaptively determined expression scheme. The dialogue management framework displays the dialogue management interface to visually represent the functions that determine the application domain, the intent applicable to the user, and the result of at least one slot.

[0143] Figure 11 This shows an example of the configuration of the dialogue management framework.

[0144] Reference Figure 11 The dialogue management framework 1100 includes a processor 1110, a memory 1120, and an input / output interface 1130.

[0145] Processor 1110 includes reference Figures 1 to 10 At least one device described, or performing the reference Figures 1 to 10 At least one method is described. Processor 1110 is a data processing device implemented in hardware, which includes circuitry having a physical structure for performing desired operations. For example, the desired operations include instructions or code included in a program. For example, the hardware-implemented data processing device includes a microprocessor, a central processing unit (CPU), a processor core, a multi-core processor, a reconfigurable processor, a multiprocessor, an application-specific integrated circuit (ASIC) and a field-programmable gate array (FPGA), a graphics processing unit (GPU), or any other type of multiprocessor configuration or single-processor configuration. Further details about processor 1110 are provided below.

[0146] Memory 1120 stores at least a portion of the parameterized information of the user terminal, or stores programs that implement the operating methods of the dialogue management framework 1100. Memory 1120 is volatile or non-volatile memory. Memory 1120 includes a high-capacity storage medium (such as a hard disk) for storing various types of data. Further details about memory 1120 are provided below.

[0147] Processor 1110 executes programs and controls the dialogue management framework 1100. The program code executed by processor 1110 is stored in memory 1120. Dialogue management framework 1100 is connected to an external device (e.g., a personal computer or network) via an input / output device (not shown) and exchanges data with it. Dialogue management framework 1100 interacts with the user through input / output interface 1130.

[0148] In one example, the dialogue management framework 1100 displays a dialogue management interface on the input / output interface 1130 to visually present the results that determine the application domain, the functions applicable to the user's intent, and at least one slot.

[0149] In one example, the input / output interface 1130 may be a display that receives input from a user or provides output. In one example, the input / output interface 1130 may be used as an input device and receive input from the user via conventional input methods (e.g., keyboard and mouse) and new input methods (e.g., touch input, voice input, and image input). Therefore, the input / output interface 1130 may include, for example, a keyboard, mouse, touchscreen, microphone, and other devices capable of detecting input from the user and sending the detected input to the dialogue management framework 1100.

[0150] In one example, the input / output interface 1130 may be used as an output device and provide the output of the dialogue management framework 1100 to the user via a visual channel, an auditory channel, or a tactile channel. The input / output interface 1130 may include, for example, a display, a touch screen, a vibration generator, and other devices that can provide output to the user.

[0151] However, the input / output interface 1130 is not limited to the examples described above, and any other display (such as a computer monitor and glasses display (EGD) operatively connected to the dialogue management framework 1100 may be used without departing from the spirit and scope of the illustrative examples described. In one example, the input / output interface 1130 is a physical structure comprising one or more hardware components that provide the ability to present a user interface, a display, and / or receive user input.

[0152] As described above, when executing the dialogue management method, the commands available to the user can be effectively displayed on the default screen output from the dialogue management framework. Furthermore, the information required for the dialogue management framework to execute commands can be effectively displayed in the dialogue management interface, as well as the task progress, as well as the information required for the dialogue between the dialogue management framework and the user. Additionally, the input content can be easily edited.

[0153] Dialogue Management Framework 1100 and its application here Figures 1 to 11Other devices, units, modules, apparatuses, and other components described are implemented through hardware components. Examples of hardware components that can be used to perform the operations described in this application include, where appropriate, controllers, sensors, generators, drivers, memories, comparators, arithmetic logic units, adders, subtractors, multipliers, dividers, integrators, and any other electronic components configured to perform the operations described in this application. In other examples, one or more of the hardware components performing the operations described in this application are implemented through computing hardware (e.g., through one or more processors or computers). A processor or computer can be implemented through one or more processing elements, such as logic gate arrays, controllers and arithmetic logic units, digital signal processors, microcomputers, programmable logic controllers, field-programmable gate arrays, programmable logic arrays, microprocessors, or any other means or combination of means configured to respond to and execute instructions in a defined manner to achieve a desired result. In one example, the processor or computer includes or is connected to one or more memories storing instructions or software executed by the processor or computer. The hardware components implemented by the processor or computer can execute instructions or software (such as an operating system (OS) and one or more software applications running on the OS) for performing the operations described in this application. Hardware components can also access, manipulate, process, create, and store data in response to the execution of instructions or software. For simplicity, the singular terms "processor" or "computer" are used in the description of the examples described in this application; however, in other examples, multiple processors or computers may be used, or a processor or computer may include multiple processing elements or multiple types of processing elements or both. For example, a single hardware component or two or more hardware components may be implemented by a single processor, or two or more processors, or a processor and a controller. One or more hardware components may be implemented by one or more processors, or a processor and a controller, and one or more other hardware components may be implemented by one or more other processors, or additional processors and additional controllers. One or more processors, or processors and controllers, may implement a single hardware component or two or more hardware components. Hardware components may have any one or more different processing configurations, examples of which include: a single processor, a standalone processor, a parallel processor, single-instruction single-data (SISD) multiple processing, single-instruction multiple-data (SIMD) multiple processing, multiple-instruction single-data (MISD) multiple processing, and multiple-instruction multiple-data (MIMD) multiple processing.

[0154] Performing the operations described in this application Figures 1 to 11The illustrated methods are executed by computing hardware (e.g., by one or more processors or a computer), which is implemented to execute instructions or software as described above to perform the operations performed by the methods described in this application. For example, a single operation or two or more operations may be executed by a single processor or two or more processors, or a processor and a controller. One or more operations may be executed by one or more processors, or a processor and a controller, and one or more other operations may be executed by one or more other processors, or additional processors and additional controllers. One or more processors, or a processor and a controller, may execute a single operation or two or more operations.

[0155] Instructions or software for controlling a processor or computer to implement hardware components and perform the methods described above are written as computer programs, code segments, instructions, or any combination thereof to individually or collectively instruct or configure the processor or computer, such as a machine or special-purpose computer, to perform the operations performed by the hardware components and methods described above. In one example, the instructions or software include at least one of a applet storing a session management method, a dynamic link library (DLL), middleware, firmware, a device driver, and an application. In one example, the instructions or software include machine code (such as machine code generated by a compiler) that is directly executed by the processor or computer. In another example, the instructions or software include high-level code that is executed by the processor or computer using an interpreter. Programmers skilled in the art can readily write instructions or software based on the block diagrams and flowcharts shown in the accompanying drawings and the corresponding descriptions in the specification, which disclose algorithms for performing the operations performed by the hardware components and methods described above.

[0156] Instructions or software for controlling computing hardware (e.g., one or more processors or computers) to implement hardware components and perform the methods described above, as well as any associated data, data files, and data structures, may be recorded, stored, or fixed in one or more non-transitory computer-readable storage media, or may be recorded, stored, or fixed on one or more non-transitory computer-readable storage media. Examples of non-transitory computer-readable storage media include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disc storage, hard disk drive (HDD), solid-state drive (SSD), card storage (such as multimedia cards, secure digital (SD) cards, or extreme digital (XD) cards), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid-state drive, and any other device configured to store instructions or software and any associated data, data files, and data structures in a non-transitory manner, and to provide instructions or software and any associated data, data files, and data structures to a processor or computer such that the processor or computer can execute the instructions. In one example, the instructions or software and any associated data, data files, and data structures are distributed across a network-connected computer system, causing one or more processors or computers to store, access, and execute the instructions, software, and any associated data, data files, and data structures in a distributed manner.

[0157] While this disclosure includes specific examples, it will be clear upon understanding this disclosure that various changes in form and detail may be made to these examples without departing from the spirit and scope of the claims and their equivalents. The examples described herein should be considered descriptive only and not for limiting purposes. The description of features or aspects in each example should be considered applicable to similar features or aspects in other examples. Suitable results may be achieved if the described techniques are performed in a different order, and / or if components in the described system, architecture, apparatus, or circuit are combined in a different manner, and / or replaced or supplemented by other components or their equivalents. Therefore, the scope of the disclosure is not limited by the specific embodiments but by the claims and their equivalents, and all variations within the scope of the claims and their equivalents should be interpreted as included in the disclosure.

Claims

1. A dialogue management method, comprising: Based on user interactions including user speech, an application domain for providing services to the user, a function related to the user's intent among the functions of the application domain, and at least one slot for performing the function related to the user's intent are determined sequentially. Based on the sequentially determined progress of the steps, and taking into account the application domain, the functions related to the user's intent, and the at least one slot, the expression scheme of the dialogue management interface is adaptively determined. as well as The dialogue management interface is displayed based on the expression scheme. The dialogue management method further includes: in response to receiving any one or any combination of inputs indicating an interface object that indicates the application domain after the application domain is determined, inputs indicating an interface object that indicates a function related to the user's intent after the function related to the user's intent is determined, and inputs indicating an interface object that indicates the at least one slot after the at least one slot is determined, canceling the decision associated with the corresponding component and components at a lower level than the corresponding component, wherein the corresponding component is indicated by the interface object corresponding to the received input and corresponds to the application domain, the function related to the user's intent, and any one or any combination of the at least one slot. The decision-making steps include: when a user's utterance includes multiple candidate values ​​corresponding to a slot in the at least one slot, if the same application interface is invoked in response to the selection of any of the multiple candidate values, then the slot is determined by selecting any one of the multiple candidate values ​​instead of requesting or querying the slot, and The decision-making process includes the following steps: In response to a user interaction that determines either the absence of an application domain or the existence of multiple application domains based on the user's utterances, the system waits for new input from the user within a predetermined threshold time period instead of sending a request or query for a decision. In response to an interaction with the user, including the user's utterances, that determines either that there is no function related to the user's intent or that there are multiple functions related to the user's intent, the system waits for a predetermined threshold time to receive new input from the user instead of sending a request or query for the decision.

2. The dialogue managing method according to claim 1, wherein The determining steps include: in response to the progress corresponding to a request for the at least one slot, determining an expression scheme to display the requested slot that does not include a value.

3. The dialogue management method according to claim 1 further includes: Before determining the application domain, multiple interface objects indicating multiple application domains installed on the user's terminal are displayed in the dialog management interface.

4. The dialogue managing method according to claim 3, wherein The decision-making steps include: selecting an interface object from the plurality of interface objects through an input method other than speech.

5. The dialogue managing method according to claim 3, wherein The multiple interface objects are displayed as moving within the dialog management interface, rather than being fixed.

6. The dialogue managing method according to claim 3, wherein The multiple interface objects float on the dialog management interface.

7. The dialogue management method according to claim 5, wherein, Some of the multiple interface objects are removed from the dialog management interface and disappear.

8. The dialogue management method according to claim 5, wherein, The new interface object has been moved from outside the dialog management interface to inside the dialog management interface.

9. The dialogue management method according to any one of claims 1 to 8, further comprising: Parameterize the information included in the at least one slot; and Application programming interfaces (APIs) that invoke user intent-related functions within the application domain based on parameterized information.

10. The dialogue managing method according to any one of claims 1 to 8, wherein The decision-making process includes at least one of the following: Select the application domain corresponding to the content of the speech from multiple application domains installed on the user's terminal; and Select the function corresponding to the content of the discourse from the functions of the application domain.

11. The dialogue managing method according to any one of claims 1 to 8, wherein The decision-making process includes at least one of the following: In response to the determination that there is no application domain corresponding to the utterance among the multiple application domains installed on the user's terminal, a request is made for the application domain used to provide the service. as well as In response to determining that a candidate application domain corresponding to the utterance is installed on the user's terminal, the application domain used to provide the service is queried from the candidate application domains.

12. The dialogue managing method according to claim 1, wherein The decision-making process includes: in response to determining that there is no application domain corresponding to the utterance in the application domains installed on the user's terminal, waiting for a period of time to receive another utterance from the user.

13. The dialogue managing method according to claim 1, wherein The decision-making process includes selecting a function that is relevant to the user's intent by comparing keywords in the user's utterance with the names of functions in the application domain.

14. The dialogue managing method according to any one of claims 1 to 8 and 12 to 13, wherein, The decision-making process includes at least one of the following: In response to determining that there is no function corresponding to the utterance among the functions of the application domain, a function related to the user's intent is requested; as well as In response to determining that there are candidate functions corresponding to the utterance among the functions in the application domain, a function related to the user's intent is queried from the candidate functions.

15. The dialogue managing method according to any one of claims 1 to 8 and 12 to 13, wherein, The decision-making process includes at least one of the following: In response to the utterance being insufficient to determine the first slot among the at least one slots, a request is made for the first slot; and In response to a utterance including a candidate value corresponding to a second slot among the at least one slots, the second slot is queried from the candidate values.

16. The dialogue managing method according to claim 1, wherein The determined steps include at least one of the following: Determine a representation scheme to highlight the application domain among the multiple application domains installed on the user's terminal; as well as Determine a presentation scheme to highlight the functions of the application domain that are relevant to the user's intent.

17. The dialogue management method according to claim 1, wherein, The steps shown include at least one of the following: In response to the determination of the application domain, an interface object indicating the function of the application domain is displayed in the dialog management interface; as well as In response to the determination of a function related to the user's intent, at least one interface object indicating the at least one slot is displayed in the dialog management interface.

18. The dialogue management method according to claim 1, wherein, The at least one slot includes two or more slots, and the display step includes displaying the two or more slots simultaneously.

19. The dialogue managing method according to claim 1, wherein The at least one slot includes two or more slots, and the display step includes displaying the two or more slots sequentially.

20. The dialogue managing method according to any one of claims 1 to 8, 12 to 13, and 16 to 19, wherein, The determined steps include: determining an expression scheme to visualize the application domain, the functions related to the user's intent, and the hierarchical structure between the at least one slot.

21. The dialogue managing method according to claim 20, wherein The determined steps include: rolling back the determined expression scheme for at least one of the following: The component indicated by the interface object corresponding to the received input; and The component in the hierarchy that corresponds to a lower level of the component.

22. The dialogue management method according to claim 9 further includes: The parameterized information is stored in the storage space; In response to the input of selecting a storage space, parameterized information is displayed in the interface indicating the storage space in the dialog management interface; as well as In response to receiving drag-and-drop input, parameterized information is dragged from the interface indicating storage space and placed on the at least one slot. The decision-making steps include: determining the at least one slot to correspond to the parameterized information.

23. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform the dialogue management method of claim 1.

24. A dialogue management device, comprising: Memory; the program is recorded in memory. as well as The processor is configured to execute the program. The program is configured to perform the following steps: Based on user interactions including user speech, an application domain for providing services to the user, a function related to the user's intent among the functions of the application domain, and at least one slot for performing the function related to the user's intent are determined sequentially. Based on the sequentially determined steps, and taking into account the application domain, user intent-related functions, and at least one slot, the expression scheme of the dialogue management interface is adaptively determined; and The dialogue management interface is displayed based on the expression scheme. The program is further configured to perform the following steps: in response to receiving any one or any combination of input indicating an interface object for the application domain after the application domain is determined, input indicating an interface object for a function related to the user's intent after the function related to the user's intent is determined, and input indicating an interface object for the at least one slot after the at least one slot is determined, canceling the decision associated with the corresponding component and components at a lower level than the corresponding component, wherein the corresponding component is indicated by the interface object corresponding to the received input and corresponds to the application domain, the function related to the user's intent, and any one or any combination of the at least one slot. The decision-making steps include: when a user's utterance includes multiple candidate values ​​corresponding to a slot in the at least one slot, if the same application interface is invoked in response to the selection of any of the multiple candidate values, then the slot is determined by selecting any one of the multiple candidate values ​​instead of requesting or querying the slot, and The decision-making process includes the following steps: In response to a user interaction that determines either the absence of an application domain or the existence of multiple application domains based on the user's utterances, the system waits for new input from the user within a predetermined threshold time period instead of sending a request or query for a decision. In response to an interaction with the user, including the user's utterances, that determines either that there is no function related to the user's intent or that there are multiple functions related to the user's intent, the system waits for a predetermined threshold time to receive new input from the user instead of sending a request or query for the decision.

25. The dialogue management device according to claim 24, wherein, The determining steps include: in response to the progress corresponding to a request for the at least one slot, determining an expression scheme to display the requested slot that does not include a value.

26. The dialogue managing apparatus according to claim 24, wherein The program is also configured to perform the following steps: before determining the application domain, displaying multiple interface objects in the dialog management interface that indicate multiple application domains installed on the user's terminal.

27. The dialogue managing apparatus according to claim 26, wherein The decision-making steps include: selecting an interface object from the plurality of interface objects through an input method other than speech.

28. The dialogue managing apparatus according to claim 26, wherein The multiple interface objects are displayed as moving within the dialog management interface, rather than being fixed.

29. The dialogue managing apparatus according to any one of claims 24 to 28, wherein The program is also configured to perform the following steps: Parameterizing the information included in the at least one slot; and Application programming interfaces (APIs) that invoke user intent-related functions within the application domain based on parameterized information.

30. The dialogue managing apparatus according to any one of claims 24 to 28, wherein The decision-making process includes at least one of the following: Select the application domain corresponding to the content of the speech from multiple application domains installed on the user's terminal; and Select the function corresponding to the content of the discourse from the functions of the application domain.

31. The dialogue management device according to any one of claims 24 to 28, wherein, The decision-making process includes at least one of the following: In response to the determination that there is no application domain corresponding to the utterance among the multiple application domains installed on the user's terminal, a request is made for the application domain used to provide the service. as well as In response to determining that a candidate application domain corresponding to the utterance is installed on the user's terminal, the application domain used to provide the service is queried from the candidate application domains.

32. The dialogue management device according to any one of claims 24 to 28, wherein, The decision-making process includes at least one of the following: In response to determining that there is no function corresponding to the utterance among the functions of the application domain, a function related to the user's intent is requested; as well as In response to determining that there are candidate functions corresponding to the utterance among the functions in the application domain, a function related to the user's intent is queried among the candidate functions.

33. The dialogue managing apparatus according to any one of claims 24 to 28, wherein The decision-making process includes at least one of the following: In response to the utterance being insufficient to determine the first slot among the at least one slots, a request is made for the first slot; and In response to a utterance including a candidate value corresponding to a second slot among the at least one slots, the second slot is queried among the candidate values.

34. The dialogue managing apparatus according to claim 24, wherein The determined steps include at least one of the following: Determine a representation scheme to highlight the application domain among the multiple application domains installed on the user's terminal; as well as Determine a presentation scheme to highlight the functions of the application domain that are relevant to the user's intent.

35. The dialogue managing apparatus according to claim 24, wherein The steps shown include at least one of the following: In response to the determination of the application domain, an interface object indicating the function of the application domain is displayed in the dialog management interface; as well as In response to the determination of a function related to the user's intent, at least one interface object indicating the at least one slot is displayed in the dialog management interface.

36. The dialogue managing apparatus according to any one of claims 24 to 28 and 34 to 35, wherein The determined steps include: determining an expression scheme to visualize the application domain, the functions related to the user's intent, and the hierarchical structure between the at least one slot.

37. The dialogue managing apparatus according to claim 36, wherein The determined steps include: rolling back the determined expression scheme for at least one of the following: The component indicated by the interface object corresponding to the received input; and The component in the hierarchy that corresponds to a lower level of the component.

38. The dialogue managing apparatus according to claim 29, wherein The program is also configured to perform the following steps: The parameterized information is stored in the storage space; In response to the input of selecting a storage space, parameterized information is displayed in the interface indicating the storage space in the dialog management interface; as well as In response to receiving drag-and-drop input, parameterized information is dragged from the interface indicating storage space and placed on the at least one slot. The decision-making steps include: determining the at least one slot to correspond to the parameterized information.

Images