Structured dialog segmentation and state tracking

Abstract

Systems and methods for open domain dialog segmentation and state tracking are provided. Specifically, a computing device may acquire and analyze a dialog in near real-time, generate a structured cue template for a state prediction model based on the dialog, and generate a structured output using the state prediction model based on the structured cue template. The structured output includes a round summary and a state tag for each round of conversation.

Description

BACKGROUND

[0001] Task-oriented dialog systems are designed to assist users in achieving specific goals. To this end, traditional dialog systems often use dialog state tracking (DST) to track user preferences and intents by filling in a number of predefined slots during a conversation to complete a task. For example, a website for an airline company can utilize a chatbot that will track a conversation with a user to attempt and understand the user’s intent, such as booking a flight, canceling a flight, changing a seat assignment, etc. Accurately tracking state is fundamental to a successful dialog system, as dialog state information can help such systems to appropriately route backend skills, improve task detection and completion rates, and infer user interests for better personalization. These dialog systems also track user preferences and intents by filling in a number of predefined slots during a conversation to complete a task.

[0002] However, a new type of dialog system has emerged in the era of large language models (LLMs). LLMs are capable of switching on an arbitrarily large set of topics and can integrate with a wide variety of task-oriented plugins in addition to supporting social (e.g., taskless) conversations. In LLM-driven conversations, traditional DST is not available because the scope of the conversation is too broad and often switches from one topic to another. Additionally, real-world conversations often present long-form utterances that span multiple conversation turns in order to more fully examine or explore different topics. This extended nature of conversations makes tracking contextual consistency very challenging.

[0003] It is with respect to these considerations and others that the disclosed aspects were made. Additionally, it should be appreciated that the examples can apply in any specific context, and are not limited to solving the specific problems identified in the background. SUMMARY

[0004] According to at least one example of the present disclosure, a method for open-domain dialog segmentation and state tracking is provided. The method includes obtaining and analyzing a dialog in near real-time, the dialog being an open-domain dialog; generating a structured prompt template for a state prediction model based on the dialog; and generating a structured output using the state prediction model based on the structured prompt template, the structured output including a turn summary and a state label for each conversation turn.

[0005] According to at least one example of this disclosure, a computing device for open-domain dialogue segmentation and state tracking is provided. The computing device may include a processor and a memory having a plurality of instructions stored thereon, which, when executed by the processor, cause the computing device to: acquire and analyze a dialogue, which is an open-domain dialogue, in near real-time; generate a structured cue template for a state prediction model based on the dialogue; and generate structured output using the state prediction model based on the structured cue template. The structured output includes a turn summary and state labels for each dialogue turn, and the state labels for each dialogue turn include segment boundary labels, user intent labels, and dialogue domain labels for each dialogue turn.

[0006] According to at least one example of this disclosure, a non-transitory computer-readable medium is provided that stores instructions for open-domain dialogue segmentation and state tracking. When executed by one or more processors of a computing device, the instructions cause the computing device to: acquire and analyze a dialogue, which is an open-domain dialogue, in near real-time; generate a structured cue template for a state prediction model based on the dialogue; and use the state prediction model based on the structured cue template to generate structured output. The structured output includes a turn summary and state labels for each dialogue turn. The state labels for each dialogue turn include segment boundary labels, user intent labels, and dialogue domain labels for each dialogue turn. The structured cue template includes, in a structured representation format, marked instructions, a structured list of valid states, and a turn-by-turn structured dialogue.

[0007] This synopsis is provided to introduce some concepts in a simplified form, which will be further described in the detailed description below. This synopsis is not intended to identify key or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. Additional aspects, features, and / or advantages of the examples will be set forth in part in the description which follows, and in part will be apparent from the description, or may be learned by practice of this disclosure. Attached Figure Description

[0008] Refer to the following figures to describe non-restrictive and non-exhaustive examples.

[0009] Figure 1 A block diagram depicts an example of an operating environment in which a conversation tracking tool can be implemented, according to the examples of this disclosure; Figure 2A and Figure 2B A flowchart depicts an example method for tracking the state of a dialogue in an open-domain dialogue, according to an example of this disclosure; Figure 2C and Figure 2D A flowchart is provided illustrating an example method for segmenting open-domain dialogues and determining dialogue state based on dialogue segments, according to an example of this disclosure. Figure 3 The example open-domain, open-ended, LLM-driven dialogue between a user and an artificial intelligence (AI) agent, according to the present disclosure, is shown, which extends over multiple dialogue rounds to discuss a variety of topics; Figure 4 This is a block diagram illustrating an exemplary overview of a dialogue state in an open-domain conversation between a user and an AI agent, according to an example of this disclosure. Figures 5A-5F These are experimental results of a structured prompting method for open-domain dialogue segmentation and state tracking, based on examples of this disclosure; Figure 6A and Figure 6B An overview of example generative machine learning models that can be used based on the examples in this disclosure is shown; Figure 7 This is a block diagram illustrating example physical components of a computing device that can be used to practice various aspects of the present disclosure; Figure 8 A simplified block diagram of a computing device that can be used to implement various aspects of this disclosure; and Figure 9 This is a simplified block diagram of a distributed computing system in which the various aspects of this disclosure can be implemented. Detailed Implementation

[0010] In the following detailed description, reference is made to the accompanying drawings, which form a part of the description, and specific aspects or examples are illustrated therein. These aspects may be combined, other aspects may be utilized, and structural changes may be made without departing from this disclosure. The aspects may be practiced as methods, systems, or devices. Therefore, the aspects may take the form of hardware implementations, entirely software implementations, or implementations combining software and hardware aspects. Accordingly, the following detailed description should not be considered limiting, and the scope of this disclosure is defined by the appended claims and their equivalents.

[0011] As mentioned above, task-oriented dialogue systems are designed to help users achieve specific goals. To this end, traditional task-oriented dialogue systems typically use Dialogue State Tracking (DST), which tracks user preferences and intentions by filling in multiple predefined positions during the conversation to complete the task.

[0012] However, emerging dialogue systems in the era of Large Language Models (LLM) have created chat or dialogue systems based on open-domain LLM, such as ChatGPT and Bing Chat. LLM-based chat systems can conduct conversations on arbitrarily large sets of topics and, in addition to supporting social (e.g., taskless) conversations, can integrate with a wide variety of task-oriented plugins. For example, LLM-based chat systems can perform user tasks that previously required dedicated systems, such as code generation, article writing, and question answering, out of the box. Therefore, in LLM-driven conversations, the use of traditional DSTs and predefined positions is not designed to capture the breadth of domains and intentions that might be explored in an LLM-driven conversation.

[0013] Based on examples in this disclosure, a dialogue tracking tool is described that provides an analysis and labeling framework for open-domain dialogues using a structured cueing approach. Open-domain dialogues typically involve numerous back-and-forth exchanges between parties (e.g., clarification, negotiation, etc.) to pursue a single intent or topic, and the context may switch multiple times within a single dialogue between unrelated intents and / or topics. Therefore, the dialogue tracking tool is configured to track states (e.g., fragment boundary labels, user intent labels, and dialogue domain labels) at a turn-by-turn level in open-domain, multi-intent dialogues. To this end, the dialogue tracking tool generates structured cueing templates for state prediction models to determine state labels for each fragment and accurately track long dialogue contexts at a turn-by-turn level. The structured cueing templates include labeling instructions, a structured list of valid states, and a turn-by-turn structured dialogue. It should be understood that the structured cueing templates do not include any example input-output pairs (e.g., zero samples).

[0014] According to examples in this disclosure, the dialogue tracking tool generates tagging instructions that include segmentation instructions and pre-analysis re-collection (PAR) instructions. Segmentation instructions are generated to instruct the state prediction model to divide the dialogue into one or more context-related segments. In other words, each dialogue segment is a topic-related (e.g., related to a single intent or topic) sequence of consecutive utterances. Additionally, segmentation instructions are generated to instruct the state prediction model to use the same user intent and dialogue domain for dialogue turns within the same dialogue segment. Without segmentation instructions, the state prediction model may over-rely on the content of dialogue turns without considering the more comprehensive prior context, potentially leading to conflicting intent and domain label predictions between dialogue turns within coherent single topical segments of the dialogue.

[0015] According to examples in this disclosure, a PAR instruction is generated to instruct the state prediction model to summarize each dialogue turn before determining the state label for the corresponding dialogue turn. Additionally, a PAR instruction is generated to instruct the state prediction model to return to previous context fragments when determining the state label. In other words, making each output state prediction based on the content of the corresponding dialogue turn and / or previous context fragments allows the state prediction model to accurately track long dialogue contexts without forgetting or hallucinations.

[0016] According to examples in this disclosure, a structured list of valid states is generated by formatting a structured representation (e.g., a hierarchical Extensible Markup Language (XML)-structured format) based on valid state values ​​associated with the dialogue. For example, valid state values ​​include one or more valid fragment boundary labels, one or more valid intent labels, and one or more valid domain labels associated with the dialogue. Furthermore, a turn-by-turn structured dialogue is generated by converting the dialogue into a turn-level structured representation of the dialogue. In other words, a turn-by-turn structured representation of a dialogue includes a structured representation of each dialogue turn.

[0017] Figure 1 A block diagram depicts an example of an operating environment 100 in which a conversation tracking tool may be implemented, according to an example of this disclosure. For this purpose, the operating environment 100 includes a computing device 120 associated with a user 110. The operating environment 100 may also include one or more remote devices, such as a productivity platform server 140, communicatively coupled to the computing device 120 via a network 130. The network 130 may include any kind of computing network, including but not limited to wired or wireless local area networks (LANs), wired or wireless wide area networks (WANs), and / or the Internet.

[0018] Computing device 120 includes a processor 122, memory 124, and a communication interface 126. In some embodiments, a conversation tracking tool 150 may execute on computing device 120. Additionally, computing device 120 may be, but is not limited to, a computer, notebook computer, laptop computer, mobile device, smartphone, tablet computer, portable device, wearable device, or any other suitable computing device capable of communicating with server 140. Server 140 includes a processor 142, memory 144, and conversation tracking tool 150. Server 140 may be any suitable computing device capable of executing conversation tracking tool 150.

[0019] The dialogue tracking tool 150 is configured to use a state prediction model to track the dialogue state in open-domain dialogues. It should be understood that the dialogue tracking tool 150 allows for near real-time tracking of dialogue state labels. Near real-time means that the dialogue tracking tool 150 acquires or receives dialogue utterances occurring in real time at the fastest speed allowed by the network 130. To this end, the dialogue tracking tool 150 also includes a dialogue monitor 152, a structured input generator 156, and a structured output generator 158.

[0020] The dialogue monitor 152 is configured to monitor or otherwise acquire dialogues between at least two parties. A dialogue is a conversation between two or more parties. The parties can be humans or artificial intelligence (AI) agents. For example, a dialogue can be an open-domain, open-ended conversation.

[0021] The structured input generator 154 is configured to generate structured cue templates for the state prediction model. It should be understood that the state prediction model is one or more Large Language Models (LLMs) (e.g., GPT4, KOSMOS). The structured cue templates include marked instructions, a structured list of valid states, and turn-by-turn structured dialogue.

[0022] The structured input generator 154 is configured to generate labeling instructions for the state prediction model. These labeling instructions include segmentation instructions and pre-analysis re-collection (PAR) instructions. To this end, the structured input generator 154 is configured to generate segmentation instructions that instruct the state prediction model to divide a dialogue into one or more context-related segments. Each dialogue segment is a topic-related (e.g., related to a single intent or topic) sequence of consecutive utterances. The structured input generator 154 is configured to generate segmentation instructions that instruct the state prediction model to identify segment boundaries when no topical relationship can be identified between a dialogue turn and its previous context. Additionally, the structured input generator 154 is configured to generate segmentation instructions that instruct the state prediction model to use the same user intent and dialogue domain for dialogue turns within the same dialogue segment. Without segmentation instructions, the state prediction model may over-rely on the content of dialogue turns without considering the more comprehensive previous context, potentially leading to conflicting intent and domain label predictions between dialogue turns within a coherent single topic segment of the dialogue.

[0023] Additionally, the structured input generator 154 is configured to generate PAR instructions to instruct the state prediction model to summarize each dialogue turn before determining the state label for the corresponding dialogue turn. The structured input generator 154 is configured to generate PAR instructions to instruct the state prediction model to return to previous context segments when determining the state label. Instructing the state prediction model to base each output state prediction on the content of the corresponding dialogue turn and / or previous context fragments will allow the state prediction model to accurately track long dialogue contexts without forgetting or illusion.

[0024] The structured input generator 154 is also configured to generate a structured valid state list based on valid state values associated with a conversation. For example, the valid state values include one or more valid segment boundary tags, one or more valid intent tags, and one or more valid domain tags related to the conversation. The structured input generator 154 is configured to format the valid state values into a structured representation (e.g., a hierarchical extensible markup language (XML)-structured format).

[0025] The structured input generator 154 is configured to generate a turn-by-turn structured conversation by converting the conversation into a turn-level conversation structured representation. In other words, the turn-by-turn structured representation of the conversation includes the structured representation of each conversation turn of the conversation. For example, as Figure 4 shown, an exemplary conversation between a user and an AI agent is converted into a structured representation in a hierarchical XML-structured format, where the conversation turns are labeled with turn id numbers <T{id}>…< / T{id}> numbered from 1 to N, and each conversation turn includes nested user and agent turns labeled with appropriate tags (e.g., <user>...< / user> and <agent>...< / agent>). It should be understood that the conversation can be between multiple users or between one or more users and one or more AI agents.

[0026] The structured input generator 154 is configured to generate a structured prompt template for a state prediction model based on tagging instructions, a structured valid state list, and a turn-by-turn structured conversation. It should be understood that the structured prompt template does not include any example input-output pairs (e.g., zero-shot). For example, as Figure 4 shown, the structured prompt template can be generated by appending the turn-by-turn structured conversation to the structured valid state list and the tagging instructions. In the Figure 4 example shown, the structured prompt template is in a hierarchical XML-structured format, which is human-readable and flexible while still being highly structured. The structured prompt template is input into a state prediction model (e.g., GPT4) that provides a structured output representation. The structured input and output representations help provide coherence and consistency to the input and output, allowing the state prediction model to accurately determine and label the session state.

[0027] The structured output generator 156 is configured to generate structured output based on a structured cue template using a state prediction model. The structured output generator 156 is configured to generate structured output using the same structured format (e.g., hierarchical XML structured format) as the structured cue template, employing the state prediction model. The structured output includes a turn summary and state labels for each dialogue turn (e.g., fragment boundaries, user intent, and dialogue domain labels). For example, fragment boundary labels can be binary labels, and intent and domain labels can be categorical labels. For example, as... Figure 4 As shown, the structured output is generated in a hierarchical XML structured format, where each dialogue round from 1 to N includes an XML tree.<T{id}> …< / T{id}> and its nested XML tags. The tags of nested tags (e.g.,<precoding_topical_relation> …< / precoding_topical_relation> , <intent> ...< / intent> and <domain> ...< / domain> The `()` indicates the segment boundary and the location of interest, and each value between the start and end labels represents one or more predicted values. It should be understood that structured output generated in a bounded, well-defined structured format is more likely to align with markup instructions and is easier to parse than free-form text, which reduces post-processing requirements.

[0028] Now for reference Figure 2A and Figure 2B A method 200 is provided for tracking the state of a dialogue in an open-domain dialogue. Figure 2A and Figure 2B The diagram illustrates the general sequence of steps for method 200. Typically, method 200 begins at 202 and ends at 224. Method 200 may include more than Figure 2A and Figure 2B More or fewer steps are shown, or can be arranged differently. Figure 2A and Figure 2B These steps are arranged in the order shown. In illustrative terms, method 200 is performed by a computing device (e.g., server 140). However, it should be understood that one or more steps of method 200 may be performed by another device of user 110 (e.g., user device 120).

[0029] Specifically, in some aspects, method 200 can be executed by a dialogue tracking tool (e.g., 150) running on server 140. For example, server 140 can be any suitable computing device capable of executing a dialogue tracking tool (e.g., 150). For example, computing device 120 can be, but is not limited to, a computer, notebook, laptop, mobile device, smartphone, tablet, portable device, wearable device, or any other suitable computing device capable of communicating with server (e.g., 160). Method 200 can be executed as a set of computer-executable instructions executed by a computer system and encoded or stored on a computer-readable medium. Furthermore, method 200 can be executed by gates or circuits associated with a processor, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), system-on-a-chip (SOC), or other hardware device. References will be incorporated herein by reference. Figure 1 and Figures 7-9 Method 200 is explained by describing the system, components, modules, software, data structures, user interface, etc.

[0030] Method 200 begins at operation 202, where the process can proceed to 204. At operation 204, the dialogue tracking tool 150 monitors and analyzes the dialogue in near real-time. A dialogue is a conversation between two or more parties. The parties can be humans or artificial intelligence (AI) agents. For example, the dialogue can be open-domain, open-ended, LLM-driven. As mentioned above, a true open-domain dialogue typically involves a large amount of back-and-forth communication (e.g., clarification, negotiation, etc.) between parties in pursuit of a single intent or topic, and the context can shift multiple times within a single dialogue between unrelated intents and / or topics. For example, in an open-domain dialogue, a single intent can span several rounds, and a single session can contain multiple intents. As mentioned above, near real-time means that the dialogue tracking tool 150 acquires or receives utterances of the dialogue occurring in real time with the network 130.

[0031] At operation 206, the dialogue tracking tool 150 generates a structured cue template based on the dialogue for use in a state prediction model. It should be understood that the state prediction model is one or more Large Language Models (LLMs) (e.g., GPT4, KOSMOS). The structured cue template includes tagged instructions, a structured list of valid states, and a turn-by-turn structured representation of the dialogue.

[0032] To this end, at operation 208, the dialogue tracking tool 150 generates labeling instructions, which include segmentation instructions and pre-analysis re-collection (PAR) instructions. As described above, the segmentation instructions instruct the state prediction model to divide the dialogue into one or more context-related segments. In other words, each dialogue segment is a topic-related (e.g., related to a single intent or topic) sequence of consecutive utterances. The segmentation instructions instruct the state prediction model to identify segment boundaries when no topical relationship can be identified between a dialogue turn and its previous context. Additionally, the segmentation instructions also instruct the state prediction model to use the same user intent and dialogue domain for dialogue turns within the same dialogue segment. Without the segmentation instructions, the state prediction model may over-rely on the content of dialogue turns without considering the more comprehensive previous context, which could lead to conflicting intent and domain label predictions between dialogue turns within a coherent single topic segment of the dialogue.

[0033] The PAR directive further provides continuity in state alignment. The PAR directive instructs the state prediction model to summarize each dialogue turn before determining the state label for the corresponding dialogue turn. Additionally, when determining the state label, the PAR directive also instructs the state prediction model to return to previous context fragments. In other words, making each output state prediction based on the content of the corresponding dialogue turn and / or previous context fragments allows the state prediction model to accurately track long dialogue contexts without forgetting or illusions.

[0034] At operation 210, the dialogue tracking tool 150 generates a structured list of valid states based on valid state values ​​associated with the dialogue. For example, valid state values ​​include one or more valid fragment boundary labels, one or more valid intent labels, and one or more valid domain labels associated with the dialogue. To this end, the valid state values ​​are formatted as a structured representation (e.g., hierarchical Extensible Markup Language (XML) - structured format).

[0035] Additionally, at operation 212, the dialogue tracking tool 150 generates a turn-by-turn structured dialogue by converting the dialogue into a turn-level structured representation. In other words, the turn-by-turn structured representation of the dialogue includes a structured representation of each turn of the dialogue. For example, as... Figure 4 As shown, an exemplary dialogue between a user and an AI agent is transformed into a structured representation in a hierarchical XML format, where dialogue turns are labeled with turn IDs numbered from 1 to N.<T{id}> …< / T{id}> And each dialogue round includes nested user and agent rounds tagged with appropriate labels (e.g., <user>... and...). <agent> ...< / agent> It should be understood that dialogues can take place between multiple users or between one or more users and one or more AI agents.

[0036] Subsequently, at operation 214, the dialogue tracking tool 150 generates a structured cue template for the state prediction model based on the tagged instructions, the structured list of valid states, and the turn-by-turn structured dialogue. It should be understood that the structured cue template does not include any example input-output pairs (e.g., zero samples). For example, as... Figure 4 As shown, a structured prompt template can be generated by attaching a round-by-round structured dialogue to a list of structured valid states and flag instructions. Figure 4 In the example shown, the structured cue template is a hierarchical XML structured format that is human-readable and flexible, yet still highly structured. The structured cue template is input into a state prediction model (e.g., GPT4) that provides a structured output representation. This structured input and output representation helps provide coherence and consistency between the input and output, allowing the state prediction model to accurately determine and label session states.

[0037] exist Figure 2B At operation 216, the dialogue tracking tool 150 uses a state prediction model based on a structured cue template to generate structured output. The structured output is generated in the same structured format as the structured cue template (e.g., a hierarchical XML structured format) and includes a turn summary and state labels for each dialogue turn (e.g., segment boundaries, user intent, and dialogue domain labels). For example, segment boundary labels are binary labels, and intent and domain labels are category labels.

[0038] For example, such as Figure 4 As shown, the structured output is generated in a hierarchical XML structured format, where each dialogue round from 1 to N includes an XML tree.<T{id}> …< / T{id}> and its nested XML tags. The tags of nested tags (e.g.,<precoding_topical_relation> …< / precoding_topical_relation> , <intent> ...< / intent> and <domain> ...< / domain> The ) represents the fragment boundary and the location of interest, and each value between the start label and the end label represents one or more predicted values.

[0039] It should be understood that structured output generated in a bounded, well-defined structured format is more likely to align with markup instructions and is easier to parse than free-form text, which reduces post-processing requirements.

[0040] Subsequently, at operation 218, the dialogue tracking tool 150 acquires and analyzes subsequent dialogue turns in the same dialogue in near real-time.

[0041] At operation 220, the dialogue tracking tool 150 updates the structured cue template based on subsequent dialogue turns. For example, it generates or updates turn-by-turn structured dialogues to include subsequent dialogue turns.

[0042] At operation 222, the dialogue tracking tool 150 generates subsequent structured output using a state prediction model based on the updated structured cue template. In some implementations, previous structured output may also be considered when generating subsequent structured output using the state prediction model. Subsequently, method 200 may end at operation 224.

[0043] Now for reference Figure 2C and Figure 2D A method 1000 is provided for segmenting open-domain dialogue and determining the dialogue state based on dialogue segments. The general sequence of the steps in method 1000 is as follows: Figure 2C As shown in the diagram. Typically, method 1000 begins at 1002 and ends at 1028. Method 200 may contain more than... Figure 2C More or fewer steps can be shown, or you can press the button. Figure 2C The steps are arranged in different orders as shown. In illustrative terms, method 1000 is performed by a computing device (e.g., server 140). However, it should be understood that one or more steps of method 1000 may be performed by another device of user 110 (e.g., user device 120).

[0044] Specifically, in some aspects, method 1000 can be executed by a dialogue tracking tool (e.g., 150) running on server 140. For example, server 140 can be any suitable computing device capable of executing a dialogue tracking tool (e.g., 150). Method 1000 can be executed as a set of computer-executable instructions executed by a computer system and encoded or stored on a computer-readable medium. Furthermore, method 1000 can be executed by gates or circuits associated with a processor, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), system-on-a-chip (SOC), or other hardware device. References will be incorporated herein by reference. Figure 1 and Figures 7-9 The system, components, modules, software, data structures, user interface, etc., are described to explain method 1000.

[0045] Method 1000 begins with operation 1002, where the process can proceed to 1004. At operation 1004, the dialogue tracking tool 150 monitors and analyzes the dialogue in near real-time. A dialogue is a conversation between two or more parties. These parties can be humans or artificial intelligence (AI) agents. For example, the dialogue can be an open-domain, open-ended, LLM-driven conversation. As mentioned above, a true open-domain dialogue typically involves a large amount of back-and-forth communication (e.g., clarification, negotiation, etc.) between parties to pursue a single intent or topic, and the context can shift multiple times within a single dialogue between unrelated intents and / or topics. For example, in an open-domain dialogue, a single intent can span several rounds, and a single conversation can contain multiple intents. As mentioned above, near real-time means that the dialogue tracking tool 150 acquires or receives utterances of the dialogue occurring in real-time with the network 130.

[0046] At operation 1006, the dialogue tracking tool 150 uses a state prediction model to determine segmentation predictions. Segmentation predictions include segmentation boundaries indicating each dialogue segment, as well as the user intent and dialogue domain of the dialogue segment. To this end, at operation 1008, the dialogue tracking tool 150 divides the dialogue into one or more topic-related segments. As described above, the dialogue tracking tool 150 instructs the state prediction model to divide the dialogue into one or more context-related segments. In other words, each dialogue segment is a topic-related (e.g., related to a single intent or topic) sequence of consecutive utterances. For example, when no topic relationship can be identified between a dialogue turn and its previous context, the dialogue tracking tool 150 instructs the state prediction model to identify segment boundaries.

[0047] Additionally, at operation 1010, a state prediction model is used to determine the user intent and dialogue domain for each dialogue segment. As further described below, the state prediction model is configured to apply the same user intent and dialogue domain to one or more dialogue turns within each dialogue segment. Without dialogue segmentation, the state prediction model may over-rely on the content of dialogue turns without considering the more comprehensive prior context, which could lead to conflicting intent and domain label predictions between dialogue turns within a coherent single topic segment of the dialogue. At operation 1012, the dialogue tracking tool 150 stores the segmentation predictions.

[0048] Subsequently, at operation 1014, the dialogue tracking tool 150 uses a state prediction model to generate structured output based on segmented prediction. To this end, at operation 1016, the dialogue tracking tool 150 generates a turn summary for each turn of the dialogue. Additionally, at operation 1018, the dialogue tracking tool 150 determines the state label for each turn based on segmented prediction. For example, the state label for each turn includes segment boundaries, user intent, and dialogue domain labels. To this end, the state prediction model determines, based on segmented prediction, whether the corresponding turn belongs to the same dialogue segment as a previous turn. In other words, the state prediction model determines, based on segmented prediction, whether the current turn is related to the contextual topic of the previous turn. If the current turn is related to the contextual topic of the previous turn, the same user intent and dialogue domain as the previous turn are applied to the current turn (i.e., the same user intent and the same dialogue domain are applied to all turns within the same dialogue segment). If the current conversation turn is not related to the context of previous conversation turns, the user intent and conversation domain for the current conversation turn are predicted based on the context of one or more previous segments of the conversation.

[0049] Subsequently, at operation 1020, the dialogue tracking tool 150 further acquires and analyzes subsequent dialogue turns within the same dialogue in near real-time. At operation 1022, the dialogue tracking tool 150 determines whether a subsequent dialogue turn belongs to the same dialogue segment as a previous dialogue turn based on segmented prediction. At operation 1024, the dialogue tracking tool 150 instructs the state prediction model to generate a turn summary for the subsequent turn and determines a state label for the subsequent turn based on the determination at operation 1022. For example, in response to determining that a subsequent dialogue turn belongs to the same dialogue segment as a previous dialogue, the same state label as the previous dialogue turn is applied to the subsequent dialogue turn. However, in response to determining that a subsequent dialogue turn does not belong to the same dialogue segment as a previous dialogue, the state label for the subsequent dialogue turn is determined based on the context of one or more previous segments of the dialogue and the structured output of one or more previous dialogue turns. Subsequently, at operation 1026, the dialogue tracking tool 150 updates the structured output based on the subsequent dialogue turn. Method 200 may then terminate at operation 1028.

[0050] Now for reference Figure 3, an exemplary dialogue 300 is synthesized to illustrate that in an open-domain dialogue, a single user intention can span several dialogue turns, and a single dialogue can contain multiple user intentions. Exemplary dialogue 300 is an open-domain, open-ended, LLM-driven dialogue between a user and an artificial intelligence (AI) agent, which extends over multiple dialogue turns to discuss diverse topics. For example, the user intention for the first set of dialogue turns 302 is to create an annotated bibliography, the user intention for the second set of dialogue turns 304 is to social chat, and the user intention for the third set of dialogue turns 306 is to check the weather.

[0051] Figure 4 An overview 400 of tracking the dialogue state in an open-domain dialogue between a user and an AI agent is shown. As described above, the structured prompt template includes a list of structured valid states, tagging instructions, and a turn-by-turn structured representation of the dialogue. As Figure 4 shown, the structured prompt template is generated in a hierarchical XML structured format. The list of structured valid states is generated based on the valid state values associated with the dialogue. For example, the valid state values include one or more valid segment boundary tags, one or more valid intention tags, and one or more valid domain tags. For this purpose, the valid state values are formatted in an XML structured format. Additionally, the original dialogue is converted into a structured representation in XML structured format, where the dialogue turns are tagged with turn id numbers <T{id}>…< / T{id}> numbered from 1 to N, and each dialogue turn includes nested user turns and agent turns, which are tagged with appropriate tags (e.g., <user>…< / user> and <agent> …< / agent> ).

[0052] Additionally, the structured output is generated in a hierarchical XML structured format, where each dialogue turn from 1 to N includes an XML tree <T{id}>…< / T{id}> and nested XML tags within it. The tags of the nested tags (e.g., <precoding_topical_relation>…< / precoding_topical_relation>, <intent> ...< / intent> and <domain> ...< / domain> ) represent the segment boundaries and the locations of interest, and each value between the start tag and the end tag represents one or more predicted values.

[0053] Figures 5A-5E is an illustration of the experimental results showing that the structured prompt method for open-domain dialogue state tracking (DST) (also known as "S3-DST" in Figure 5A -E) achieves a significant performance improvement compared to similar zero-shot prompts. As described above, the dialogue tracking tool (e.g., 150) uses the structured prompt method (S3-DST) for open-domain DST in a zero-shot setting to perform dialogue tagging.

[0054] Human-LLM Conversation Dataset Construction: To evaluate the S3-DST method across a wide range of human-LLM conversations spanning numerous topics and intents, logs from Microsoft's Bing chat system, an LLM chat interface powered by the Bing search engine, were collected. Specifically, 6K anonymized English conversations were randomly sampled from Bing chat. Since S3-DST operates under the zero-shot hypothesis, no training set was collected. Instead, only a development set for prompting iterations and a test set for evaluation were collected. Two complementary sampling methods were employed for the development and test sets. First, 75 conversations were randomly sampled from an initial pool of 6K English conversations for each set. Next, this pool was filtered to conversations with 5 or more rounds to represent “challenging” conversations more likely to contain intents and domain shifts. Sampling was then performed on 75 conversations from each set of this filtered challenging set. After removing any overlap between the development and test sets, 150 development conversations and 145 test conversations remained, corresponding to 882 development rounds and 923 test rounds. Figure 5A The table shows the statistics for evaluating the test set.

[0055] Notes: To obtain ground truth labels for evaluation, human annotations for segments and states were obtained from annotators with high technical expertise and familiarity with the Bing chat system. For each round, annotators were instructed to provide binary IsSegmentBoundary labels, categorical SegmentIntent labels, and categorical SegmentDomain labels. When no topical relationship could be identified between a round and its previous context, annotators were instructed to label the segment boundaries. For intents and domains, classifications developed for the Bing chat system, which consists of 4 intents and 49 domains, were used. Due to the large number of domains, annotators were provided with four candidate domain values ​​and other options for each round. To ensure consistency among annotators before labeling the entire dataset, annotations were first collected on a set of 68 rounds, and the Frisska Pa coefficient was calculated for each label type. For IsSegmentBoundary, the Frisska Pa coefficient κ=0.83, for SegmentIntent, κ=0.74, and for SegmentDomain, κ=0.88, all of which were considered highly consistent according to the Frisska Pa coefficient criterion.

[0056] Public Benchmarks: Currently, there are no readily available public dialogue benchmarks that cover the breadth of domains and intentions reflected in Bing chat data. Therefore, three datasets (MWOZ 2.1, MWOZ 2.4, and DialSeg711) were used to perform separate DST and segmented evaluations of public benchmarks.

[0057] The Multi-WOZ (MWOZ) multi-domain dialogue dataset is currently one of the most common DST benchmarks. MWOZ is a task-oriented dataset consisting of 1,000 test dialogues and 7.3,000 test rounds. Figure 5A As shown, two updated versions of MWOZ, 2.1 and 2.4, are used. The latter is considered the "cleanest" version of MWOZ, while the former has been used more frequently in the literature.

[0058] The DialSeg711 benchmark has been frequently used in recent dialogue segmentation research. It is an English dataset in which 711 multi-segment dialogues are constructed by incorporating dialogues from an existing task-oriented dialogue corpus.

[0059] Baselines: Zero-sample baselines for TBT-DST, IC-DST, S3-DST (without PAR), S3-DST (unstructured hints), and S3-DST have been considered. GPT4 was used as the LLM backbone for all baselines on the Bing chat dataset.

[0060] The IC-DST baseline is a zero-sample version of the cueing strategy, deeply adapted to open-domain dialogue settings.

[0061] The TBT-DST baseline is a version of S3-DST that does not include segmentation instructions and uses S3-DST prompts to configure round-by-round acquisition of intent labels and domain labels. Additionally, S3-DST offers two ablation mechanisms: No PAR (S3-DST prompts without PAR instructions) and Unstructured prompts (S3-DST prompts that use plain text instead of XML to organize all instructions and dialogue).

[0062] On MWOZ, the codex-175B was used to reproduce the results for the number of IC-DSTs. IC-DSTs were then rerun using GPT4. Additionally, the zero-sample ChatGPT performance on MWOZ 2.1 was reproduced.

[0063] Unsupervised TextTidiling, CSM, and DialStart methods were considered. All numbers were reproduced. Segment labels were generated using the IC-DST baseline model with hints in the same SQL output format as the original IC-DST.

[0064] Metrics: For state tracking, Joint Objective Accuracy (JGA) is considered, which measures the proportion of rounds in which all state values ​​are correctly inferred. Intent and domain accuracy on Bing Chat can illustrate the current capabilities and limitations of LLM on open-domain session data. For segmentation, Pκ and WindowDiff are considered, both of which are error metrics (i.e., lower values ​​are better) that use an adjustable sliding window to quantify the difference between predicted segment boundaries and ground truth segment boundaries.

[0065] like Figure 5B As shown, S3-DST hints achieve the highest performance in cross-round intent, domain, and JGA (Joint Intent and Accuracy) predictions.

[0066] Additionally, the TBT-DST baseline, which does not perform segmentation, has so far had the lowest performance. As mentioned above, without instructing the LLM to use the same intent and domain within a dialogue segment, the LLM tends to over-rely on the content of the turn without considering the more comprehensive prior context, which can lead to conflicting intent and domain labels between turns within a coherent single-topic dialogue.

[0067] IC-DST is a very strong baseline. However, while IC-DST utilizes structured outputs, it does not have a corresponding structured input representation, which may lead to the illusion of higher non-existent rounds compared to S3-DST.

[0068] Therefore, both ablation methods of S3-DST performed poorly compared to S3-DST, thus confirming the importance of LLM's ability to review PAR and structured input during generation. Figure 5F The relationship between dialogue length and performance was plotted, showing that S3-DST avoids a sharp decline in performance without PAR as dialogues become longer. These results support the necessity of PAR for long dialogues of 10 rounds and beyond.

[0069] Figure 5C and 5D The overall results for MWOZ and the number of domains are provided. Compared to the powerful LLM, S3-DST achieves a significant lead over zero-sample JGA. Even the strongest zero-sample baseline, IC-DST (GPT4), shows a nearly 5-point absolute JGA performance gap on MWOZ 2.1 and a 7-point absolute JGA performance gap on MWOZ 2.4. In almost all individual domains, S3-DST outperforms IC-DST (GPT4), and the gap is significant in some domains, such as a JGA improvement of over 13 points in the train domain.

[0070] Figure 5EThe performance on DialSeg711 is shown. S3-DST achieves near-zero error on this dataset. Specifically, by combining dialogues on very different topics to construct DialSeg711, this results in extremely artificial and abrupt context transitions between segments. However, IC-DST hints at a baseline error significantly higher than S3-DST. When LLM fails to track the context of multiple dialogues in the dataset, it leads to the forgetting of the original dialogue context. These results highlight the importance of PAR and dialogue context tracking for successful segmentation.

[0071] Figure 6A and Figure 6B An overview of example generative machine learning models that can be used based on the aspects described in this article is shown. First, refer to... Figure 6A Conceptual diagram 600 depicts an overview of a pre-trained generative model package 604 according to aspects described herein, which processes input 602 to generate model outputs for storing entries in generative model output 606 (e.g., structured output) and / or retrieving information from generative model output 606 (e.g., structured output).

[0072] In the example, the generative model package 604 is pre-trained on a variety of inputs (e.g., various human languages, various programming languages, and / or various content types), and therefore does not require fine-tuning or training for a specific scenario. Instead, the generative model package 604 can be pre-trained in a more general way, such that the input 602 includes prompts that are generated, selected, or otherwise engineered to guide the generative model package 604 to produce certain generative model outputs 606. It should be understood that the input 602 and the generative model output 606 can each include any type of content from various content types, including but not limited to text output, image output, audio output, video output, programming output, and / or binary output. In the example, the input 602 and the generative model output 606 can have different content types, which may occur when the generative model package 604 includes a generative multimodal machine learning model.

[0073] Therefore, generative model package 604 can be used in any scenario across a wide range of scenarios, and furthermore, different generative model packages can be used instead of generative model package 604 without substantially modifying other related aspects (e.g., similar to those discussed in this paper). Figures 1-3 (Those aspects described). Therefore, the generative model package 604 operates as a tool for performing machine learning processing, wherein certain inputs 602 to the generative model package 604 are generated programmatically or otherwise determined, thereby enabling the generative model package 604 to produce model outputs 606, which can then be used for further processing.

[0074] Generative model package 604 can be provided or otherwise used according to any of the various paradigms. For example, generative model package 604 can be used on computing devices (e.g., Figure 1 The computing device 120 in the middle can be used locally, or it can be obtained from a machine learning service (e.g., Figure 1 The generative model package 604 can be accessed remotely via server 140. In other examples, aspects of the generative model package 604 are distributed across multiple computing devices. In some cases, the generative model package 604 may be accessed via an application programming interface (API), such as by the operating system of the computing device and / or by machine learning services.

[0075] Referring now to aspects of generative model package 604, generative model package 604 includes input tokenization 608, input embedding 610, model layer 612, output layer 614, and output decoding 616. In the example, input tokenization 608 processes input 602 to generate input embedding 610, which includes a sequence of symbol representations corresponding to input 602. Therefore, input embedding 610 is processed by model layer 612, output layer 614, and output decoding 616 to produce model output 606. Figure 6B The example architecture corresponding to Generative Model Package 604 is depicted below, which will be discussed in further detail. Even so, it should be understood that the architecture shown and described herein should not be considered limiting, and any of the various other architectures may be used in other examples.

[0076] Figure 6B This is a conceptual diagram depicting an example architecture 650 of a pre-trained generative machine learning model that can be used according to the aspects described herein. As mentioned above, any of the various alternative architectures and corresponding ML models can be used in other examples without departing from the aspects described herein.

[0077] As shown in the figure, architecture 650 processes input 602 to produce generative model output 606, aspects of which are discussed above. Figure 6A Discussion. Architecture 650 is depicted as a transformer model comprising encoder 652 and decoder 654. Encoder 652 processes input embedding 658 (its aspects may be similar to...). Figure 6A The input embedding (610) includes a sequence of symbolic representations corresponding to input 656. In the example, input 656 includes content data 602 corresponding to a content item.

[0078] Furthermore, positional encoding 660 can incorporate information about the relative and / or absolute positions of the lexical units in the input embedding 658. Similarly, the output embedding 674 includes a sequence of symbolic representations corresponding to the output 672, and positional encoding 676 can similarly incorporate information about the relative and / or absolute positions of the lexical units in the output embedding 674.

[0079] As shown in the figure, encoder 652 includes example layer 670. It should be understood that any number of such layers can be used, and the architecture depicted is simplified for illustrative purposes. Example layer 670 includes two sub-layers: a multi-head attention layer 662 and a feedforward layer 666. In the example, residual connections are included around each layer 662 and layer 666, followed by normalization layers 664 and 668, respectively.

[0080] Decoder 654 includes example layer 690. Similar to encoder 652, any number of such layers may be used in other instances, and the depicted architecture of decoder 654 is simplified for illustrative purposes. As shown, example layer 690 includes three sublayers: a masked multi-head attention layer 678, a multi-head attention layer 682, and a feedforward layer 686. Aspects of multi-head attention layer 682 and feedforward layer 686 may be similar to those discussed above regarding multi-head attention layer 662 and feedforward layer 666, respectively. Additionally, masked multi-head attention layer 678 performs multi-head attention on the output of encoder 652 (e.g., output 672). In the example, masked multi-head attention layer 678 prevents each position from focusing on subsequent positions. This masking, combined with embedding offsets (e.g., offsetting by one position, as shown in multi-head attention layer 682), ensures that the prediction for a given position depends on the known outputs of one or more positions smaller than the given position. As shown in the figure, residual connections are included around layers 678, 682 and 686, followed by normalized layers 680, 684 and 688, respectively.

[0081] Multi-head attention layers 662, 678, and 682 can each use a set of linear projections to linearly project the query, key, and value onto their corresponding dimensions. Each linear projection can be processed using an attention function (e.g., dot product attention or summation attention), thus producing an n-dimensional output value for each linear projection. The resulting values ​​can be concatenated and projected again, such that subsequent... Figure 6B The values ​​are processed as shown (e.g., by the corresponding normalization layer 664, 680, or 684).

[0082] Feedforward layers 666 and 686 can each be fully connected feedforward networks, applicable to each location. In the example, feedforward layers 666 and 686 each include multiple linear transformations with rectified linear unit activation between them. In the example, each linear transformation is identical at different locations, while different parameters can be used compared to other linear transformations of the feedforward network.

[0083] Furthermore, aspects of the linear transformation 692 can be analogous to the linear transformations discussed above with respect to the multi-head attention layers 662, 678, and 682, and the feedforward layers 666 and 686. Softmax 694 can further transform the output of the linear transformation 692 into the predicted next lexical probability, as indicated by the output probability 696. It should be understood that the architecture shown is provided as an example, and in other examples, any of various other model architectures can be used based on the disclosed aspects.

[0084] Therefore, according to the aspects described herein, the output probability 696 can thus form a generative model output 606, such that the output of the generative ML model (e.g., which may include one or more state labels) is used as input for determining the action according to the aspects described herein. In other examples, the generative model output 606 is provided as a generated structured output.

[0085] Figures 7-9 The associated description provides a discussion of various operating environments in which the aspects of this disclosure can be practiced. However, regarding Figures 7-9 The devices and systems shown and discussed are for illustrative purposes only and are not intended to limit the wide range of computing device configurations that may be used to practice the aspects of this disclosure described herein.

[0086] Figure 7 This is a block diagram illustrating the physical components (e.g., hardware) of a computing device 700 that can be used to practice various aspects of this disclosure. The computing device components described below can be adapted to the aforementioned computing device, including one or more devices associated with machine learning services (e.g., production platform server 140), and the components mentioned above. Figure 1 The computing device 140 under discussion. In a basic configuration, the computing device 700 may include at least one processing unit 702 and system memory 704. Depending on the configuration and type of the computing device, the system memory 704 may include, but is not limited to, volatile memory (e.g., random access memory), non-volatile memory (e.g., read-only memory), flash memory, or any combination of such memory.

[0087] System memory 704 may include an operating system 705 and one or more program modules 706 suitable for running software applications 720, such as one or more components supported by the system described herein. As an example, system memory 704 may store a dialogue tracking tool 722, including a dialogue monitor 724, a structured input generator 726, and a structured output generator 728. Operating system 705 may, for example, be suitable for controlling the operation of computing device 700.

[0088] Furthermore, aspects of this disclosure can be practiced in conjunction with graphics libraries, other operating systems, or any other applications, and are not limited to any particular application or system. This basic configuration is... Figure 7 The components within the dashed line 708 are shown. The computing device 700 may have additional features or functions. For example, the computing device 700 may also include additional data storage devices (removable and / or non-removable), such as disks, optical discs, or magnetic tapes. Such additional storage... Figure 7 The image is shown by a removable storage device 709 and a non-removable storage device 710.

[0089] As described above, multiple program modules and data files can be stored in system memory 704. When executed on processing unit 702, program module 706 (e.g., application 720) can perform processes including, but not limited to, the aspects described herein. Other program modules that can be used according to aspects of this disclosure may include email and contact applications, word processing applications, spreadsheet applications, database applications, PowerPoint presentation applications, drawing or computer-aided applications, etc.

[0090] Furthermore, aspects of this disclosure can be implemented on circuits including discrete electronic components, packages or integrated electronic chips containing logic gates, circuits utilizing microprocessors, or single chips containing electronic components or microprocessors. For example, aspects of this disclosure can be implemented via a system-on-a-chip (SOC), wherein... Figure 7 Each or more of the components shown can be integrated onto a single integrated circuit. Such a SoC device may include one or more processing units, graphics units, communication units, system virtualization units, and various application functions, all integrated (or “burned in”) as a single integrated circuit onto a chip substrate. When operating via the SoC, the capabilities described herein regarding the client switching protocol can be operated via dedicated logic integrated with other components of the computing device 700 on the single integrated circuit (chip). Aspects of this disclosure can also be practiced using other techniques capable of performing logical operations (such as, for example, AND, OR, and NOT), including but not limited to mechanical, optical, fluid, and quantum technologies. Furthermore, aspects of this disclosure can be practiced within a general-purpose computer or in any other circuit or system.

[0091] The computing device 700 may also have one or more input devices 712, such as a keyboard, mouse, pen, voice or speech input device, touch or swipe input device, etc. It may also include output devices (multiples) 714, such as a display, speaker, printer, etc. The above devices are examples, and other devices may be used. The computing device 700 may include one or more communication connections 716 that allow communication with other computing devices 750. Examples of suitable communication connections 716 include, but are not limited to, radio frequency (RF) transmitters, receivers, and / or transceiver circuitry; universal serial bus (USB), parallel and / or serial ports.

[0092] As used herein, the term computer-readable medium can include computer storage media. Computer storage media can include volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, or program modules. System memory 704, removable storage device 709, and non-removable storage device 710 are examples of computer storage media (e.g., memory storage). Computer storage media can include RAM, ROM, electrically erasable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical storage, magnetic tape cassette, magnetic tape, disk storage or other magnetic storage devices, or any other article of manufacture that can be used to store information and can be accessed by computing device 700. Any such computer storage medium may be part of computing device 500. Computer storage media does not include carrier waves or other propagated or modulated data signals.

[0093] Communication media can be embodied in computer-readable instructions, data structures, program modules, or other data in modulated data signals (such as carrier waves or other transmission mechanisms), and include any information transmission medium. The term "modulated data signal" can describe a signal having one or more characteristics set or altered in a manner that encodes information in the signal. By way of example and not limitation, communication media can include wired media such as wired networks or direct wired connections, and wireless media such as acoustic, radio frequency (RF), infrared, and other wireless media.

[0094] Figure 8The diagram illustrates a system 800, which can be, for example, a mobile computing device such as a mobile phone, smartphone, wearable computer (such as a smartwatch), tablet computer, laptop computer, etc., using which various aspects of this disclosure can be implemented. In one example, the system 800 is implemented as a "smartphone" capable of running one or more applications (e.g., a browser, email, calendar, contact manager, messaging client, game, and media client / player). In some aspects, the system 800 is integrated as a computing device, such as integrating a personal digital assistant (PDA) and a wireless telephone.

[0095] In a basic configuration, such a mobile computing device is a handheld computer with both input and output elements. System 800 typically includes a display 805 and one or more input buttons that allow the user to input information into system 800. The display 805 can also be used as an input device (e.g., a touchscreen display).

[0096] If included, optional side input elements allow for additional user input. For example, a side input element could be a rotary switch, a button, or any other type of manual input element. Alternatively, system 800 can incorporate more or fewer input elements. For example, in some aspects, display 805 may not be a touchscreen. In another example, an optional keyboard 835 may also be included, which could be a physical keyboard or a “soft” keyboard generated on the touchscreen display.

[0097] In various aspects, output elements include a display 805 for displaying a graphical user interface (GUI), a visual indicator (e.g., a light-emitting diode 820), and / or an audio transducer 825 (e.g., a speaker). In some aspects, a vibration transducer is included to provide tactile feedback to the user. In yet another aspect, input and / or output ports are included, such as audio inputs (e.g., a microphone jack), audio outputs (e.g., a headphone jack), and video outputs (e.g., an HDMI port) for sending signals to or receiving signals from external devices.

[0098] One or more applications 866 may be loaded into memory 862 and run on or associated with operating system 864. Examples of applications include telephone dialers, email programs, personal information management (PIM) programs, word processing programs, spreadsheet programs, internet browser programs, messaging programs, etc. System 800 also includes a non-volatile storage area 868 within memory 862. The non-volatile storage area 868 may be used to store persistent information that should not be lost if system 800 is powered off. Applications 866 may use and store information in the non-volatile storage area 868, such as emails or other messages used by email applications. A synchronization application (not shown) also resides on system 800 and is programmed to interact with a corresponding synchronization application residing on a host computer to keep the information stored in the non-volatile storage area 868 synchronized with the corresponding information stored on the host computer. It should be understood that other applications may be loaded into memory 862 and run on system 800 as described herein (e.g., content capture manager, content retrieval manager, etc.).

[0099] System 800 has a power supply 870, which can be implemented as one or more batteries. The power supply 870 may also include an external power source, such as an AC adapter or power docking bracket for replenishing or recharging the batteries.

[0100] System 800 may also include a radio interface layer 872 that performs functions of transmitting and receiving radio frequency communications. Radio interface layer 872 facilitates wireless connectivity between system 800 and the "outside world" via a communications operator or service provider. Transmissions to and from radio interface layer 872 are conducted under the control of operating system 864. In other words, communications received by radio interface layer 872 can be propagated to application 866 via operating system 864, and vice versa.

[0101] A visual indicator 820 can be used to provide visual notifications, and / or an audio interface 874 can be used to generate auditory notifications via an audio transducer 825. In the illustrated example, the visual indicator 820 is a light-emitting diode (LED), and the audio transducer 825 is a speaker. These devices can be directly coupled to a power supply 870 such that when activated, they remain on for the duration specified by the notification mechanism, even if the processor 860 and other components may be turned off to conserve battery power. The LED can be programmed to remain on indefinitely until the user takes action to indicate the device's power-on status. The audio interface 874 is used to provide and receive audible signals to and from the user. For example, in addition to being coupled to the audio transducer 825, the audio interface 874 can also be coupled to a microphone to receive audible input, such as to facilitate telephone conversations. According to various aspects of this disclosure, the microphone can also be used as an audio sensor to facilitate control of notifications, as described below. The system 800 may also include a video interface 876, which enables the operation of the onboard camera 830 to record still images, video streams, etc.

[0102] It should be understood that system 800 may have additional features or functions. For example, system 800 may also include additional data storage devices (removable and / or non-removable), such as disks, optical discs, or magnetic tapes. Such additional storage... Figure 8 The non-volatile storage region 868 is shown in the middle.

[0103] As described above, data / information generated, captured, and stored via system 800 can be stored locally, or the data can be stored on any number of storage media accessible by the device via radio interface layer 872 or via a wired connection between system 800 and a separate computing device associated with system 800 (e.g., a server computer in a distributed computing network such as the Internet). It should be understood that such data / information can be accessed via radio interface layer 872 or via a distributed computing network. Similarly, such data / information can be easily transferred between computing devices for storage and use according to any of various data / information transmission and storage means (including email and collaborative data / information sharing systems).

[0104] Figure 9 An aspect of the architecture of a system for processing data received at a computing system from a remote source (such as a personal computer 904, a tablet computing device 906, or a mobile computing device 908), as described above, is illustrated. Content displayed at server device 902 can be stored in different communication channels or other storage types. For example, various documents can be stored using a directory service 924, a portal website 925, an email service 926, an instant messaging repository 928, or a social networking site 930.

[0105] Application 920 (e.g., similar to application 520) can be employed by a client communicating with server device 902. Additionally or alternatively, a dialogue tracking tool 909, including a dialogue monitor 910, a structured input generator 911, and a structured output generator 912, can be employed by server device 902. Server device 902 can provide data to and from client computing devices such as personal computer 904, tablet computing device 906, and / or mobile computing device 908 (e.g., smartphone) via network 915. As an example, the aforementioned computer system can be embodied in personal computer 904, tablet computing device 906, and / or mobile computing device 908 (e.g., smartphone). In addition to receiving graphical data that can be preprocessed at the graphics initiating system or post-processed at the receiving computing system, any of these examples of computing devices can obtain content from storage 916.

[0106] It should be understood that the aspects and functions described herein can operate on distributed systems (e.g., cloud-based computing systems), where application functions, memory, data storage and retrieval, and various processing functions can remotely operate on each other via distributed computing networks (such as the Internet or intranets). Various types of user interfaces and information can be displayed via onboard computing device displays or via remote display units associated with one or more computing devices. For example, various types of user interfaces and information can be displayed and interacted with on a wall surface on which various types of user interfaces and information are projected. Interaction with multiple computing systems on which aspects of this disclosure can be practiced includes keystroke input, touchscreen input, voice or other audio input, gesture input, wherein the associated computing device is equipped with detection (e.g., camera) functions for capturing and interpreting user gestures used to control the functions of the computing device, etc.

[0107] For example, the foregoing references to block diagrams and / or operational illustrations of methods, systems, and computer program products according to various aspects of this disclosure describe aspects of the disclosure. Functions / actions mentioned in the blocks may not occur in the order shown in any flowchart. For example, depending on the functions / actions involved, two blocks shown consecutively may actually be executed substantially simultaneously, or these blocks may sometimes be executed in reverse order.

[0108] The description and illustrations of one or more aspects provided in this application are not intended to limit or restrict the scope of the claimed disclosure in any way. The aspects, examples, and details provided in this application are considered sufficient to convey ownership and enable others to make and use the claimed aspects of this disclosure. The claimed disclosure should not be construed as limited to any aspect, example, or detail provided in this application. Various features (structural and methodological features) are intended to be selectively included or omitted, whether shown and described in combination or separately, to produce aspects having a particular set of features. Given the descriptions and illustrations provided in this application, those skilled in the art can contemplate variations, modifications, and alternatives falling within the spirit of the broader aspects of the overall inventive concept embodied in this application without departing from the broader scope of the claimed disclosure.

[0109] Furthermore, the aspects and functions described herein can operate on distributed systems (e.g., cloud-based computing systems), where application functions, memory, data storage and retrieval, and various processing functions can remotely operate on each other via distributed computing networks (such as the Internet or intranets). Various types of user interfaces and information can be displayed via onboard computing device displays or via remote display units associated with one or more computing devices. For example, various types of user interfaces and information can be displayed and interacted with on a wall surface on which various types of user interfaces and information are projected. Interaction with multiple computing systems on which aspects of this disclosure can be practiced includes keystroke input, touchscreen input, voice or other audio input, gesture input, wherein the associated computing device is equipped with detection (e.g., camera) functions for capturing and interpreting user gestures used to control the functions of the computing device, etc.

[0110] The phrases “at least one,” “one or more,” “or,” and “and / or” are open-ended expressions that possess both conjunction and disjunctive properties in their use. For example, each of the expressions “at least one of A, B, and C,” “at least one of A, B, or C,” “one or more of A, B, and C,” “one or more of A, B, or C,” “A, B, and / or C,” and “A, B, or C” indicates a single A, a single B, a single C, A and B together, A and C together, B and C together, or A, B, and C together.

[0111] The term "a" or "an" entity refers to one or more of the entities. Therefore, the terms "a" (or "an"), "one or more," and "at least one" are used interchangeably herein. It should also be noted that the terms "comprising," "including," and "having" are used interchangeably.

[0112] As used herein, the term "automatic" and its variations refer to any process or operation that is generally continuous or semi-continuous, performed without substantial human input. However, a process or operation can be automatic even if its execution uses substantial or non-substantial human input, if input is received prior to its execution. Human input is considered substantial if it influences how the process or operation will be performed. Human input consenting to the execution of a process or operation is not considered "substantial."

[0113] Any steps, functions, and operations discussed in this article can be performed continuously and automatically.

[0114] Example systems and methods of this disclosure have been described with respect to computing devices. However, to avoid unnecessarily obscuring this disclosure, several known structures and devices have been omitted from the foregoing description. Such omissions should not be construed as limiting. Specific details have been set forth to provide an understanding of this disclosure. However, it should be understood that this disclosure can be practiced in various ways beyond the specific details set forth herein.

[0115] Furthermore, while the examples shown herein illustrate various components of a co-located system, some components of the system may be located remotely in distant parts of a distributed network (such as a LAN and / or the Internet), or within a dedicated system. Therefore, it should be understood that system components may be combined into one or more devices, such as servers, communication equipment, or co-located on specific nodes of a distributed network, such as analog and / or digital telecommunications networks, packet-switched networks, or circuit-switched networks. As will be understood from the foregoing description, and for computational efficiency reasons, system components may be placed anywhere within the distributed network of components without affecting the operation of the system.

[0116] Furthermore, it should be understood that the various links of the connecting element can be wired or wireless links, or any combination thereof, or any other known or later-developed element(s) capable of providing data to and / or transmitting data from the connecting element. These wired or wireless links can also be secure links and capable of transmitting encrypted information. For example, the transmission medium used as the link can be any suitable carrier for electrical signals, including coaxial cables, copper wires, and optical fibers, and can take the form of sound waves or light waves, such as those generated during radio wave and infrared data communication.

[0117] Although flowcharts have been discussed and explained regarding specific event sequences, it should be understood that the sequence can be changed, added to, or omitted without substantially affecting the operation of the disclosed configuration and aspects.

[0118] Several variations and modifications of this disclosure may be used. Some features of this disclosure may be provided without providing others.

[0119] In another configuration, the systems and methods of this disclosure may be implemented in combination with a dedicated computer, a programmable microprocessor or microcontroller and (multiple) peripheral integrated circuit elements, an ASIC or other integrated circuit, a digital signal processor, hardwired electronic or logic circuitry (such as discrete component circuitry), a programmable logic device or gate array (such as a PLD, PLA, FPGA, PAL), a dedicated computer, any similar means, etc. Generally, any (multiple) means or apparatuses capable of implementing the methods described herein can be used to implement various aspects of this disclosure. Example hardware that may be used in this disclosure includes computers, handheld devices, telephones (e.g., cellular, internet-enabled, digital, analog, hybrid, etc.), and other hardware known in the art. Some of these devices include processors (e.g., single or multiple microprocessors), memory, non-volatile storage devices, input devices, and output devices. Furthermore, alternative software implementations, including but not limited to distributed processing or component / object distributed processing, parallel processing, or virtual machine processing, may also be constructed to implement the methods described herein.

[0120] In another configuration, the disclosed method can be readily implemented using software that provides portable source code for object-oriented or object-based software development environments usable on various computer or workstation platforms. Alternatively, the disclosed system can be implemented partially or entirely in hardware using standard logic circuitry or VLSI designs. Whether to implement the system according to this disclosure using software or hardware depends on the system's speed and / or efficiency requirements, specific functions, and the particular software or hardware system or microprocessor or microcomputer system utilized.

[0121] In another configuration, the disclosed methods can be implemented in part in software, which can be stored on a storage medium and executed on a programmed general-purpose computer in cooperation with a controller and memory, a dedicated computer, a microprocessor, etc. In these cases, the systems and methods of this disclosure can be implemented as programs embedded in a personal computer, such as applets, JAVA®, or CGI scripts; as resources residing on a server or computer workstation; or as routines embedded in a dedicated measurement system, system components, etc. The system can also be implemented by physically integrating the system and / or methods into a software and / or hardware system.

[0122] As described herein, this disclosure is not limited to standards and protocols. Other similar standards and protocols not mentioned herein exist and are included in this disclosure. Furthermore, the standards and protocols mentioned herein, as well as other similar standards and protocols not mentioned herein, are periodically replaced by faster or more efficient equivalents having substantially the same function. Such replacement standards and protocols having the same function are considered equivalents included in this disclosure.

[0123] According to at least one example of this disclosure, a method for open-domain dialogue segmentation and state tracking is provided. The method includes: acquiring and analyzing a dialogue, which is an open-domain dialogue, in near real-time; generating a structured cue template for a state prediction model based on the dialogue; and using the state prediction model based on the structured cue template to generate a structured output, which includes a turn summary and state labels for each dialogue turn.

[0124] According to at least one aspect of the above method, the method may further include: wherein the state label for each dialogue turn includes a fragment boundary label, a user intent label, and a dialogue domain label for each dialogue turn.

[0125] According to at least one aspect of the above method, the method may further include: wherein the structured prompt template includes, in a structured representation format, a marked instruction, a structured list of valid states, and a round-by-round structured dialogue.

[0126] According to at least one aspect of the above method, the method may further include: wherein generating the structured prompt template for the state prediction model based on the dialogue includes: generating the marking instruction, wherein the marking instruction includes a segmentation instruction and a pre-analysis recollection (PAR) instruction.

[0127] According to at least one aspect of the above method, the method may further include: wherein the segmentation instruction is designed to instruct the state prediction model to divide the dialogue into one or more topic-related segments, wherein each of the one or more dialogue segments is a topic-related continuous sequence of utterances.

[0128] According to at least one aspect of the above method, the method may further include: wherein the segmentation instruction is designed to instruct the state prediction model to identify segment boundaries when there is no topic relationship between the dialogue turn and its previous context that can be identified.

[0129] According to at least one aspect of the above method, the method may further include: wherein the segmentation instruction is designed to instruct the state prediction model to use the same user intent and dialogue domain for dialogue turns within the same dialogue segment.

[0130] According to at least one aspect of the above method, the method may further include: wherein the PAR instruction is designed to instruct the state prediction model to summarize the state for each dialogue turn before determining the state label for that dialogue turn.

[0131] According to at least one aspect of the above method, the method may further include: wherein the PAR instruction is designed to instruct the state prediction model to review previous context fragments when determining the state label corresponding to the dialogue turn.

[0132] According to at least one aspect of the above method, the method may further include: wherein generating the structured cue template for the state prediction model based on the dialogue includes: generating the structured list of valid states by formatting one or more valid state values ​​associated with the dialogue into a structured representation.

[0133] According to at least one aspect of the above method, the method may further include: wherein generating the structured cue template for the state prediction model based on the dialogue includes: generating the round-by-round structured dialogue by converting the dialogue into a round-level structured representation.

[0134] According to at least one aspect of the above method, the method may further include: wherein the structured representation adopts a hierarchical extensible markup language (XML) structured format.

[0135] According to at least one aspect of the above method, the method may further include: wherein the state prediction model is a generative large language model (LLM) or a multimodal large language model (MLLM).

[0136] According to at least one example of this disclosure, a computing device for open-domain dialogue segmentation and state tracking is provided. The computing device may include a processor and a memory having a plurality of instructions stored thereon, the instructions, when executed by the processor, causing the computing device to: acquire and analyze a dialogue, which is an open-domain dialogue, in near real-time; generate a structured cue template for a state prediction model based on the dialogue; and use the state prediction model based on the structured cue template to generate a structured output, the structured output including a turn summary and state labels for each dialogue turn, and the state labels for each dialogue turn including a segment boundary label, a user intent label, and a dialogue domain label for each dialogue turn.

[0137] According to at least one aspect of the computing device described above, the computing device may include: wherein the structured prompt template includes, in a structured representation format, a marked instruction, a structured list of valid states, and a turn-by-turn structured dialogue.

[0138] According to at least one aspect of the computing device described above, the computing device may include: wherein generating the structured prompt template for the state prediction model based on the dialogue includes: generating the marking instruction, wherein the marking instruction includes a segmentation instruction and a pre-analysis recollection (PAR) instruction.

[0139] According to at least one aspect of the computing device described above, the computing device may include: wherein segmentation instructions are designed to instruct a state prediction model (1) to divide a dialogue into one or more topic-related segments, wherein each dialogue segment in the one or more segments is a topic-related sequence of consecutive utterances, (2) to identify segment boundaries when no topic relationship can be identified between a dialogue turn and its previous context, and (3) to use the same user intent and dialogue domain for dialogue turns within the same dialogue segment.

[0140] According to at least one aspect of the computing device described above, the computing device may include: wherein the PAR instruction is designed to instruct the state prediction model (1) to summarize the state label for each dialogue turn before determining the state label for that dialogue turn, and (2) to review previous context fragments when determining the state label for that dialogue turn.

[0141] According to at least one example of this disclosure, a non-transitory computer-readable medium is provided that stores instructions for open-domain dialogue segmentation and state tracking. When executed by one or more processors of a computing device, the instructions cause the computing device to: acquire and analyze a dialogue, which is an open-domain dialogue, in near real-time; generate a structured cue template for a state prediction model based on the dialogue; and use the state prediction model based on the structured cue template to generate structured output. The structured output includes a turn summary and state labels for each dialogue turn. The state labels for each dialogue turn include segment boundary labels, user intent labels, and dialogue domain labels for each dialogue turn. The structured cue template includes, in a structured representation format, marking instructions, a structured list of valid states, and a turn-by-turn structured dialogue. According to at least one aspect of the aforementioned non-transitory computer-readable medium, the instructions, when executed by one or more processors, also cause a computing device to perform a method, wherein generating the structured prompt template for the state prediction model based on the dialogue includes: generating the marking instructions, wherein the marking instructions include segmentation instructions and pre-analysis recollection (PAR) instructions.

[0142] This disclosure, in various configurations and aspects, includes components, methods, processes, systems, and / or apparatuses substantially as shown and described herein, including various combinations, sub-combinations, and subsets thereof. Those skilled in the art, upon understanding this disclosure, will know how to manufacture and use the systems and methods disclosed herein. This disclosure, in various configurations and aspects, also covers implementing related apparatuses and processes without the presence of components not illustrated and / or described herein, including omitting such components that might be used in conventional apparatuses or processes, for example, to improve performance, simplify implementation, and / or reduce implementation costs.

Claims

1. A method for open-domain dialogue segmentation and state tracking, the method comprising: Near real-time acquisition and analysis of dialogues, wherein the dialogues are open-domain dialogues; Based on the dialogue, a structured prompt template is generated for the state prediction model; as well as Based on the structured prompt template, the state prediction model is used to generate structured output, which includes a turn summary and state labels for each dialogue turn.

2. The method of claim 1, wherein the state label for each dialogue turn includes a segment boundary label, a user intent label, and a dialogue domain label for each dialogue turn.

3. The method according to claim 1, wherein the structured prompt template includes, in a structured representation format, a marker instruction, a structured list of valid states, and a round-by-round structured dialogue.

4. The method of claim 3, wherein generating the structured prompt template for the state prediction model based on the dialogue comprises: Generate the marking instructions, wherein the marking instructions include segmentation instructions and pre-analysis recollection (PAR) instructions.

5. The method of claim 4, wherein the segmentation instruction is designed to instruct the state prediction model to divide the dialogue into one or more topic-related segments, wherein each of the one or more dialogue segments is a topic-related continuous sequence of utterances.

6. The method of claim 4, wherein the segmentation instruction is designed to instruct the state prediction model to identify segment boundaries when no topic relationship can be identified between the dialogue turn and its previous context.

7. The method of claim 4, wherein the segmentation instruction is designed to instruct the state prediction model to use the same user intent and dialogue domain for dialogue turns within the same dialogue segment.

8. The method of claim 4, wherein the PAR instruction is designed to instruct the state prediction model to summarize the corresponding dialogue turn before determining the state label for each dialogue turn.

9. The method of claim 4, wherein the PAR instruction is designed to instruct the state prediction model to review previous context fragments when determining the state label corresponding to the dialogue turn.

10. The method of claim 3, wherein generating the structured prompt template for the state prediction model based on the dialogue comprises: The structured list of valid states is generated by formatting one or more valid state values ​​associated with the dialogue into a structured representation.

11. The method of claim 3, wherein generating the structured prompt template for the state prediction model based on the dialogue comprises: The round-by-round structured dialogue is generated by converting the dialogue into a round-level structured representation.

12. The method of claim 11, wherein the structured representation is in a hierarchical extensible markup language (XML) structured format.

13. The method according to claim 1, wherein the state prediction model is a generative large language model (LLM) or a multimodal large language model (MLLM).

14. A method for open-domain dialogue segmentation and state tracking, the method comprising: Acquire and analyze the dialogue, which is an open-domain dialogue; By dividing the dialogue into one or more topic-related segments and determining the user intent and dialogue domain for each segment, a state prediction model is used to determine segment predictions, where each segment includes a continuous subsequence of one or more topic-related dialogue turns; as well as The state prediction model is used to generate structured output based on the segmented prediction, the structured output including a turn summary and state labels for each dialogue turn.

15. The method of claim 14, wherein the status label for each dialogue turn includes a fragment boundary label, a user intent label, and a dialogue domain label for the corresponding dialogue turn, and wherein the fragment boundary label indicates whether a topic relationship exists between the context of the corresponding dialogue turn and the context of a previous dialogue turn.

16. The method of claim 15, further comprising: The segmented prediction is stored, including the one or more segments, the user intent for each segment, and the dialogue domain.

17. The method of claim 14, wherein generating the structured output using the state prediction model based on the segmented prediction comprises: The same user intent and dialogue domain are applied to each dialogue turn within the same dialogue segment.

18. The method of claim 14, further comprising: Obtain the subsequent dialogue rounds of the given dialogue; Based on the segmentation prediction, it is determined whether the subsequent dialogue rounds belong to the same dialogue segment as the previous dialogue rounds; as well as Update the structured output to include a round summary and status labels for the subsequent dialogue rounds.

19. The method of claim 18, wherein determining whether the subsequent dialogue turn belongs to the same dialogue segment as the previous dialogue turn based on the segmentation prediction comprises: The segmented prediction is used to determine whether the subsequent dialogue rounds are related to the contextual topics of the previous dialogue rounds.

20. The method of claim 18, wherein updating the structured output to include a turn summary and status labels for the subsequent dialogue turns comprises: In response to determining that the subsequent dialogue turn belongs to the same dialogue segment as the previous dialogue, the same status label as the previous dialogue turn is applied to the subsequent dialogue turn; as well as In response to determining that the subsequent dialogue turn does not belong to the same dialogue segment as the previous dialogue, a state label for the subsequent dialogue turn is determined based on the context of one or more previous segments of the dialogue and the structured output of one or more previous dialogue turns.

Images