Conversation state tracking method and device and conversation state tracking model
By dividing the dialogue system into skill units and identifying relevance based on importance scores and fine-grained averaging strategies, the catastrophic forgetting problem in task-oriented dialogue systems is solved, achieving effective knowledge transfer and performance improvement.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-02
- Publication Date
- 2026-03-10
AI Technical Summary
Existing technologies in task-oriented dialogue systems face catastrophic forgetting problems and difficulties in knowledge transfer between tasks, resulting in the system's inability to effectively retain the accuracy of old knowledge when learning new tasks.
By identifying skill units in the dialogue system based on a preset grouping strategy, and by recognizing the relevance of each skill unit to the current and previous dialogue tasks based on the importance score of the target dialogue task and a fine-grained averaging strategy, effective knowledge transfer across dialogue tasks is achieved, reducing catastrophic forgetting.
It improves the performance of dialogue state tracking, maintains a balance between knowledge from previous dialogue tasks and completion of the current dialogue task, reduces catastrophic forgetting, and facilitates the differentiation between task-specific and task-shared skill units.
Smart Images
Figure CN121636642A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of artificial intelligence, and in particular to a dialogue state tracking method, device and dialogue state tracking model. Background Technology
[0002] With the continuous development of artificial intelligence, dialogue systems are undergoing profound transformations and significant improvements. Dialogue systems enable computers to understand and generate human language, allowing them to engage in natural, fluent, and meaningful communication with users. Moreover, dialogue systems are being applied in fields such as intelligent customer service, voice assistants, and intelligent chatbots.
[0003] In task-oriented dialogue systems, dialogue state tracking (DST) enables the maintenance and updating of the understanding of user intent between dialogues. However, the introduction of new tasks often leads to a decline in the system's accuracy in previous tasks, a phenomenon known as catastrophic forgetting. Related technologies primarily rely on continuous learning (CL) to continuously learn new knowledge from new data while retaining the memory of old knowledge, thus reducing the catastrophic forgetting of performance on old tasks when learning new ones. Methods such as memory replay and regularization can be employed.
[0004] However, memory replay requires significant memory and computational overhead. Regularization techniques typically fail to fundamentally address the catastrophic forgetting problem and may even hinder the dialogue system's ability to effectively learn new tasks. In other words, current DST (Discussion-Based Learning) methods face difficulties in addressing catastrophic forgetting and inter-task knowledge transfer (KT). Therefore, a method is urgently needed to reduce the risk of catastrophic forgetting and achieve effective knowledge transfer. Summary of the Invention
[0005] This application provides a dialogue state tracking method, apparatus, and dialogue state tracking model, which can solve the problems of catastrophic forgetting and knowledge transfer between tasks.
[0006] Firstly, a dialogue state tracking method is provided, including:
[0007] Based on a preset grouping strategy, skill units in the dialogue system are determined. Each skill unit is a subset of model parameters in the dialogue system, and each skill unit includes one or more model parameters.
[0008] In the target dialogue task, an importance score is determined, where the target dialogue task is used to indicate the current or previous dialogue task in the dialogue.
[0009] Based on the importance score of the target dialogue task and the fine-grained averaging strategy, the relevance of each skill unit to the current dialogue task and previous dialogue tasks is determined, so as to obtain the dialogue state corresponding to the current dialogue task.
[0010] This dialogue state tracking method determines the relevance of each skill unit to the current and previous dialogue tasks by using importance scores of the target dialogue task and a fine-grained averaging strategy. It provides effective knowledge transfer across dialogue tasks and improves the performance of continuous dialogue state tracking. In other words, it distinguishes between task-specific and task-shared skill units through skill localization and skill consolidation, thereby reducing catastrophic forgetting and maintaining a balance between knowledge from previous dialogue tasks and the ability to complete the current dialogue task.
[0011] In one possible implementation, determining the relevance of each skill unit to the current dialogue task and previous dialogue tasks includes:
[0012] Based on the importance scores of each previous dialogue task, the importance distribution up to the first dialogue task is determined. The first dialogue task is the dialogue task that is adjacent to the current dialogue task in time sequence and precedes the current dialogue task among the previous dialogue tasks.
[0013] Based on a fine-grained averaging strategy, the importance distribution up to the first dialogue task and the importance score of the current dialogue task are analyzed to obtain the correlation between each skill unit and the current dialogue task and previous dialogue tasks.
[0014] In one possible implementation, a fine-grained averaging strategy is used to analyze the importance distribution up to the first dialogue task and the importance score of the current dialogue task, yielding the correlation between each skill unit and the current and previous dialogue tasks, including:
[0015] Based on a preset threshold strategy, the importance thresholds for the current dialogue task, previous dialogue tasks, and cumulative dialogue tasks are determined. The preset threshold strategy is used to dynamically determine the importance thresholds for the current dialogue task and previous dialogue tasks. The cumulative dialogue tasks include all previous dialogue tasks up to the first dialogue task.
[0016] The importance of a skill unit in the current dialogue task is determined based on the importance threshold of the current dialogue task; and the importance of a skill unit in the cumulative dialogue task is determined based on the importance threshold of the cumulative dialogue task.
[0017] Based on the importance of each skill unit in the current dialogue task and the importance of each skill unit in the cumulative dialogue task, the correlation between each skill unit and the current dialogue task and previous dialogue tasks is obtained.
[0018] In one possible implementation, the importance distribution up to the first dialogue task satisfies the following formula:
[0019]
[0020] In the formula, Let β represent the importance distribution up to the first dialogue task, β∈[0,1] represent the hyperparameters, and Norm(·) represent the normalization function. The score indicates the importance of the first dialogue task. This indicates the importance distribution of a dialogue task whose deadlines precede those of the first dialogue task.
[0021] In one possible implementation, the relevance of each skill unit to the current dialogue task and previous dialogue tasks satisfies the following formula:
[0022]
[0023] In the formula, This represents the relevance of each skill unit to the current dialogue task and up to the previous dialogue task, where γ∈[0,1] represents a hyperparameter. This indicates that the skill unit is important for the previous dialogue mission. This indicates that the skill unit is not important to the previous dialogue quest. This indicates that the skill unit is important for the current dialogue task. This indicates that the skill unit is not important to the current dialogue task.
[0024] In one possible implementation, determining the importance score of the target dialogue task includes:
[0025] Determine the importance function and importance score of each technical unit in the target dialogue task, satisfying the following formula:
[0026]
[0027] In the formula, Represents skill unit Importance score, w ij The model parameters are represented by d1×d2, which represent skill units. The total number of model parameters, S(·) is the objective importance function of a single model parameter, n = 1, ..., N;
[0028] The objective importance function for a single model parameter satisfies the following formula:
[0029]
[0030] In the formula, S(wij ) represents the model parameter w ij The objective importance function, where t represents the number of iterations, and S (t) (w ij ) represents the model parameters w when the number of iterations is t. ij The objective importance function, This represents the smoothing sensitivity obtained through exponential moving average. Indicated by C (t) and The uncertainty term quantified by local changes between them.
[0031] In a second aspect, a dialogue state tracking device is provided, comprising a processor and a memory, wherein the memory stores a computer program and the processor retrieves and runs the computer program from the memory, causing the dialogue state tracking device to execute any of the dialogue state tracking methods in the first aspect.
[0032] Thirdly, a dialogue state tracking apparatus is provided, including a unit for performing any of the dialogue state tracking methods in the first aspect. This apparatus may be a terminal device or a chip within the terminal device.
[0033] Fourthly, a computer-readable storage medium is provided, which stores a computer program that, when run by a dialogue state tracking device, causes a processor to execute any of the dialogue state tracking methods in the first aspect.
[0034] Fifthly, a dialogue state tracking model is provided. This model is used to execute any of the dialogue state tracking methods described in the first aspect. The dialogue state tracking model includes a skill localization component and a skill consolidation component; wherein,
[0035] The skill localization component is used to determine the importance score of a target dialogue task based on skill units, where skill units are determined through a preset grouping strategy; the target dialogue task is used to indicate the current or previous dialogue task in the dialogue.
[0036] The skill consolidation component is used to consolidate skill units in a target dialogue task based on a fine-grained averaging strategy.
[0037] In a sixth aspect, a computer program product is provided, comprising a computer program that, when executed by a processor, causes the processor to execute either the dialogue state tracking method of the first aspect or the dialogue state tracking model of the fifth aspect. Attached Figure Description
[0038] Figure 1This is a schematic diagram of the dialogue state tracking method provided in the embodiments of this application;
[0039] Figure 2 This is a schematic diagram illustrating the relevance of dialogue tasks provided in an embodiment of this application;
[0040] Figure 3 This is a schematic diagram of another dialogue state tracking method provided in the embodiments of this application;
[0041] Figure 4 This is a schematic diagram of another dialogue state tracking method provided in the embodiments of this application;
[0042] Figure 5 This is a schematic diagram of another dialogue state tracking method provided in the embodiments of this application;
[0043] Figure 6 This is a schematic diagram of the structure of the dialogue state tracking model provided in the embodiments of this application;
[0044] Figure 7 This is a schematic diagram of comparative experimental data between the dialogue state tracking model provided in this application embodiment and other models;
[0045] Figure 8 This is a schematic diagram of the dialogue state tracking device provided in the embodiments of this application;
[0046] Figure 9 A schematic diagram of the structure of an electronic device provided in this application is shown. Detailed Implementation
[0047] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; "and / or" in this text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, in the description of the embodiments of this application, "multiple" refers to two or more than two.
[0048] Hereinafter, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first," "second," or "third" may explicitly or implicitly include one or more of that feature.
[0049] Specific details, such as particular system architectures and techniques, are set forth for illustrative purposes and not for limitation, to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application can be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted to avoid unnecessary detail that could obscure the description of this application.
[0050] For ease of understanding, some concepts involved in the embodiments of this application are introduced and explained below:
[0051] (a) Dialogue system
[0052] A dialogue system is a computer program that can simulate human conversational behavior and interact with users using natural language. Dialogue systems are one of the important applications in the field of artificial intelligence and can be widely used in areas such as customer service robots, intelligent assistants, and smart homes.
[0053] (II) Memory Replay
[0054] Memory replay is a common method to prevent forgetting by storing and replaying training samples from previous tasks. These samples can then be used for review when learning new tasks to reinforce old knowledge.
[0055] (III) Regularization
[0056] Regularization techniques can be used to constrain the learning process of a model to reduce overfitting to new tasks while preventing the forgetting of old tasks. For example, L2 regularization can prevent overfitting by penalizing the model parameters.
[0057] (iv) Catastrophic Forgetting
[0058] Catastrophic forgetting refers to a model's severe forgetting of previously learned knowledge or patterns when learning new tasks or data.
[0059] (v) Knowledge Transfer (KT)
[0060] Knowledge transfer refers to the process of transferring and sharing knowledge between different entities, systems, or individuals.
[0061] Forward transfer typically refers to transferring existing knowledge, experience, or skills from one source (such as senior employees, mature systems, or existing projects) to a new target (such as new employees, new systems, or new projects). For example, in a company, senior employees passing on their work skills and experience to newly hired employees is an example of forward transfer. Similarly, applying knowledge and methods accumulated in a successful project to a newly launched, similar project is also a manifestation of forward transfer.
[0062] Backward transfer, on the other hand, refers to the application of new knowledge and experience gained from a new goal or environment back to its original source. For example, a new employee discovers a new method or process at work and provides feedback to senior employees or the existing work system, thereby supplementing and improving existing knowledge. Or, an innovative model explored in a new project is applied to similar projects in the past, achieving backward transfer of knowledge.
[0063] By effectively combining forward and backward transfer, continuous knowledge updating and optimization can be promoted, improving the overall performance and innovation capabilities of an organization or system. For example, in a technology company, the R&D team forward transfers technical knowledge from old products to the development of new products, while technical problems and solutions discovered during the development of new products can be backward transferred to improve the performance and maintenance strategies of old products.
[0064] (vi) Continuous learning (CL)
[0065] Continuous learning aims to enable models to retain existing knowledge while learning new knowledge, thus avoiding catastrophic forgetting.
[0066] In task-oriented dialogue systems, dialogue state tracking enables the maintenance and updating of the understanding of user intent between dialogues. However, the introduction of new tasks often leads to a decline in the system's accuracy in previously learned tasks; this phenomenon is known as catastrophic forgetting. Related technologies primarily rely on continuous learning to acquire new knowledge from new data while retaining the memory of old knowledge, thus reducing the catastrophic forgetting of performance on older tasks when learning new ones. For example, methods such as memory replay and regularization can be employed.
[0067] However, memory replay requires a large amount of memory and computational overhead. Regularization techniques usually cannot fundamentally solve the problem of catastrophic forgetting and may also hinder the ability of dialogue systems to effectively learn new tasks. In other words, DST has difficulties in solving the problem of catastrophic forgetting and knowledge transfer between tasks.
[0068] In view of this, embodiments of this application propose a dialogue state tracking method, apparatus, and dialogue state tracking model. The method determines skill units in the dialogue system based on a preset grouping strategy; determines the importance score of the target dialogue task in the target dialogue task; and determines the relevance of each skill unit to the current dialogue task and previous dialogue tasks based on the importance score of the target dialogue task and a fine-grained averaging strategy, so as to obtain the dialogue state corresponding to the current dialogue task; provides effective knowledge transfer across dialogue tasks, distinguishes task-specific and task-shared skill units through skill localization and skill consolidation, reduces catastrophic forgetting, and maintains a balance between knowledge in previous dialogue tasks and the ability to complete the current dialogue task.
[0069] It should be understood that in continuous dialogue state tracking, the aim is to train a model f: X×T→Y, where X represents the dialogue between the dialogue system and the user, T represents the dialogue task (domain) involved in the dialogue process, and Y represents the prediction target, i.e., the state of the current dialogue task.
[0070] This involves a series of dialogue tasks T1, ..., T K For example, in a given task T k The dialogue system and the user have M rounds of interactive dialogue, denoted as X. M ={(A1,U1),(A2,U2),…,(A M U M )}, where A m U represents the response of the dialogue system in the m-th round of interactive dialogue. m This represents the user's input in the m-th round of interactive dialogue.
[0071] For each dialogue task T k Each has a set of associated predefined slots S = {S1, ..., S2}. J}, where J is the total number of slots.
[0072] In other words, the prediction target Y of Dialogue State Tracking (DST) is based on the dialogue context X. M Predicting Dialogue State B M The predicted dialogue state is a set of slot-value pairs (slot, slot value), represented as... in, It is slot S in the m-th round of interactive dialogue. J The value (slot value).
[0073] Therefore, DST actually involves training the model. in, This indicates simple text concatenation.
[0074] It should be understood that each dialogue task T k Both have dataset Dk It should be understood that a dataset can be a single dataset or multiple datasets, and can be transformed between different datasets. For example, a schema-guided dialog dataset (SGD) can be used for a dataset.
[0075] The following is combined with Figures 1 to 5 The dialogue state tracking method provided in the embodiments of this application will be described in detail.
[0076] Figure 1 This is a schematic diagram of the dialogue state tracking method provided in the embodiments of this application, such as... Figure 1 As shown, the dialogue state tracking method includes the following steps:
[0077] S110. Based on a preset grouping strategy, determine the skill units in the dialogue system.
[0078] In this context, a skill unit is a subset of model parameters in a dialogue system. A skill unit includes one or more model parameters that contain specific functional capabilities or knowledge related to the task.
[0079] Skill units can be understood as the space of model parameters, rather than the classification of slots in the dataset, which can overcome the low accuracy caused by dataset noise.
[0080] It should be understood that model parameters refer to numerical values used to define and describe the structure and behavior of a model, such as weights. In dialogue systems, the number of model parameters depends on the architecture and design of the model in the dialogue system; complex models typically have more model parameters.
[0081] It should also be understood that the preset grouping strategy is used to divide the model parameters in the dialogue system into multiple skill units, wherein some skill units may include one model parameter while other skill units include multiple model parameters; or each skill unit may include multiple model parameters.
[0082] The division of skill units can vary depending on the structure of the model in the dialogue system and the number of model parameters. For example, in a Transformer-based model, a skill unit might be a specific matrix in the self-attention mechanism, such as a Query, Key, or Value matrix.
[0083] For example, a LoRA (Low-Rank Adaptation) adapter was added to the attention layer of the Large Language Model Meta AI (LLaMA). Each layer consists of 8 different skill units. Given that LLaMA-7B consists of 32 layers, a total of 256 skill units can be identified.
[0084] As shown in Table 1 below, after adding the LoRA (Low-Rank Adaptation) adapter to the attention layer of the Large Language Model Meta AI (LLaMA), there are 8 skill units in one layer.
[0085] Table 1 Examples of Skill Units
[0086]
[0087] For a model f with N skill units for dialogue state tracking: u = {u1, ..., u2} N}, where u1,…,u N Used to represent each skill unit.
[0088] By dividing the model into skill units, it is possible to effectively distinguish between model parameters that are crucial for a specific dialogue task and those that are common across multiple tasks. Identifying these differences allows for targeted knowledge updates to improve performance on the current dialogue task while retaining knowledge from previous tasks.
[0089] S120. In the target dialogue task, the importance score of the target dialogue task is determined based on the importance score of each skill unit.
[0090] The target dialogue task is used to indicate the current or previous dialogue task in the dialogue. It should be understood that in a dialogue system, there may be a series of dialogue tasks, including the current dialogue task and the previous dialogue task. For each ongoing dialogue task (i.e. the target dialogue task), the importance score of each skill unit can be determined.
[0091] It should be understood that the previous dialogue task is the dialogue task that precedes the current dialogue task, that is, the previous dialogue task is the dialogue task that precedes the current dialogue task in time sequence.
[0092] It should also be understood that, in terms of chronological order, if there are dialogue tasks preceding the ongoing dialogue task, these preceding dialogue tasks are called previous dialogue tasks.
[0093] It should be understood that for each current dialogue task, the importance score of each skill unit needs to be calculated first, and then the importance score of the current dialogue task needs to be determined. The importance score of each skill unit is used to measure the overall importance of all model parameters within each skill unit.
[0094] In other words, when the target dialogue task is a previous dialogue task, the importance score of each skill unit in the previous dialogue task is calculated first, and then the importance score of the previous dialogue task is determined; when the target dialogue task is the current dialogue task, the importance score of each skill unit in the current dialogue task is calculated first, and then the importance score of the current dialogue task is determined.
[0095] For each current dialogue task T k Each skill unit The importance score satisfies the following formula (1):
[0096]
[0097] In the formula, Represents skill unit Importance score, w ij The model parameters are represented by d1×d2, which represent skill units. The total number of model parameters, S(·) is the objective importance function of a single model parameter, n = 1, ..., N.
[0098] In some embodiments, a single model parameter w ij Target importance function S(w ij It is determined through the following process:
[0099] First, determine the initial importance function for each model parameter.
[0100] The initial importance function of each model parameter is determined by the model parameter and its gradient.
[0101] For model parameters w ij The initial importance function C(w) ij ), satisfying the following formula (2):
[0102]
[0103] In the formula, C(w) ij ) represents the model parameter w ij The initial importance function, Represents model parameters w ij The gradient.
[0104] This approximates the change in loss when a model parameter is set to zero. If removing a model parameter has a significant impact, then the model is sensitive to it, and we should retain that model parameter.
[0105] If the initial importance function is determined solely by the absolute gradient, the performance of the corresponding dialogue state tracking model will be reduced. To improve performance, the initial importance function can be obtained by multiplying the model parameters and the gradient and then taking the absolute value, which can enhance the performance of the dialogue state tracking model.
[0106] The initial importance function is derived from the dialogue task T. k Dataset D k Partial model parameters w from the middle sampling ij The calculated metrics are affected by changes due to random sampling and training dynamics, which may introduce uncertainty when determining the initial importance function. To mitigate this, the initial importance function can be processed by applying smoothing and uncertainty quantification.
[0107] Secondly, based on the initial importance function, the smoothing sensitivity and uncertainty term are determined, and then the target importance function for each model parameter is determined. The target importance function for each model parameter is determined by the product of the smoothing sensitivity and the uncertainty term.
[0108] For model parameters w ij Target importance function S(w ij ), satisfying the following formula (5):
[0109]
[0110] In the formula, S(w ij ) represents the model parameter w ij The objective importance function, where t represents the number of iterations, and S (t) (w ij ) represents the model parameters w when the number of iterations is t. ij The objective importance function, This represents the smoothing sensitivity obtained through exponential moving average. Indicated by C (t) and The uncertainty term quantified by local changes between them.
[0111] Among them, smoothing sensitivity The following formula (3) is satisfied, and the uncertainty term is... The following formula (4) must be satisfied:
[0112]
[0113] In the formula, α1 and α2 ∈ [0,1] represent smoothing factors, which are hyperparameters; t is the number of iterations; This represents the smoothing sensitivity obtained through exponential moving average. Indicated by C (t) and The uncertainty term quantified by local changes between them.
[0114] By using the above process to determine the target importance function, the performance of the dialogue state tracking model can be improved compared to directly using the initial importance function. At the same time, it highlights the importance of accurate skill localization and the value of skill localization in improving model performance.
[0115] S130. Based on the importance score of the target dialogue task and the fine-grained averaging strategy, determine the correlation between each skill unit and the current dialogue task and previous dialogue tasks, so as to obtain the dialogue state corresponding to the current dialogue task.
[0116] The fine-grained averaging strategy analyzes skill units and distinguishes between task-sharing and task-specific strategies. Task-sharing means that the corresponding skill unit can be shared between the current dialogue task and the previous dialogue task, that is, the skill unit is important for both the current and previous dialogue tasks. Task-specific means that the corresponding skill unit is important in the current dialogue task but not in the previous dialogue task, or the skill unit is important in the previous dialogue task but not in the current dialogue task.
[0117] It should be understood that the relevance of each skill unit to the current dialogue task and previous dialogue tasks means that the relevance of each skill unit is different for different dialogue tasks.
[0118] For example, skill unit A is highly relevant to the current dialogue task, but weakly relevant to previous dialogue tasks. In other words, the model parameters contained in skill unit A are important model parameters for the current dialogue task, but not important model parameters for previous dialogue tasks.
[0119] For example, skill unit B is highly relevant to the current dialogue task and also highly relevant to previous dialogue tasks. In other words, the model parameters contained in skill unit B are important model parameters for both the current and previous dialogue tasks.
[0120] Figure 2 This is a schematic diagram illustrating the relevance of dialogue tasks provided in an embodiment of this application, such as... Figure 2As shown, taking dialogue tasks related to "hotels," "restaurants," and "hailing a taxi" as examples, the dialogue tasks related to "hotels" and "restaurants" can share similar slots semantically, such as "area" and "reservation date." These shared slots are crucial for knowledge transfer. However, task-specific slots, such as "food" in the context of "restaurants," can introduce unique information and disrupt previously acquired knowledge. Therefore, by introducing skill units and determining the relevance of each skill unit to the current dialogue task and previous dialogue tasks, task-specific and task-shared parameters can be integrated to promote effective knowledge transfer and reduce forgetting.
[0121] pass Figure 2 The content introduced introduces skill localization and skill consolidation for dialogue tasks, aiming to improve knowledge transfer between dialogue tasks without relying on memory replay. Skill localization is achieved by identifying and integrating the importance distribution of skill units (including model parameters) across tasks. Skill consolidation, on the other hand, achieves effective knowledge transfer by integrating the relevance of each skill unit to the current dialogue task and previous dialogue tasks, while minimizing forgetting.
[0122] This application embodiment determines the correlation between each skill unit and the current dialogue task and previous dialogue tasks by using an importance score of the target dialogue task and a fine-grained averaging strategy. Thus, for a model composed of different skill units used for dialogue state tracking, the dialogue state corresponding to the current dialogue task can be obtained.
[0123] By integrating skills at a fine-grained level, updates ensure that they do not overwrite important, task-specific knowledge. This selective integration maintains high performance on previously learned dialogue tasks while adapting to new ones, reducing catastrophic forgetting.
[0124] The above embodiments focus on how the dialogue state tracking method determines the relevance to the current dialogue task and previous dialogue tasks through skill units. The following will explain... Figure 3 The illustrated embodiment describes in detail how to determine the relevance of each skill unit to the current dialogue task and previous dialogue tasks based on the importance score of the target dialogue task and a fine-grained averaging strategy. Figure 3 This is a schematic diagram of another dialogue state tracking method provided in the embodiments of this application, such as... Figure 3 As shown, determining the relevance of each skill unit to the current dialogue task and previous dialogue tasks includes the following steps:
[0125] S310. Based on the importance scores of each previous dialogue task, determine the importance distribution up to the first dialogue task.
[0126] The first dialogue task is the dialogue task that is adjacent to and precedes the current dialogue task in the time sequence among the previous dialogue tasks.
[0127] like Figure 2 As shown, the current dialogue task is about "hailing a taxi". Previous dialogue tasks include dialogue tasks about "hotels" and dialogue tasks about "restaurants". The dialogue task about "restaurants" is the first dialogue task.
[0128] To avoid the inefficiency of storing the importance scores of each previous dialogue task, the importance scores of previous dialogue tasks can be aggregated into the first dialogue task to obtain the cumulative importance score, which is the importance distribution up to the first dialogue task.
[0129] It should be understood that the importance distribution up to the first dialogue task refers to the importance distribution of all previous dialogue tasks.
[0130] like Figure 2 As shown, by aggregating the importance scores of the dialogue task about "hotel" and the importance scores of the dialogue task about "restaurant", the importance distribution of the dialogue task up to "restaurant" can be obtained.
[0131] In some embodiments, the importance distribution up to the first dialogue task can be represented by... It is expressed as follows, and satisfies the following formula (6):
[0132]
[0133] In the formula, Let β represent the importance distribution up to the first dialogue task, β∈[0,1] represent the hyperparameters, and Norm(·) represent the normalization function. The score indicates the importance of the first dialogue task. This indicates the importance distribution of dialogue tasks whose deadlines precede those of the first dialogue task.
[0134] It should be understood that different dialogue tasks have different distributions of importance scores. After normalization, the importance distribution between each dialogue task is better. Therefore, by normalizing the importance score of the first dialogue task and the importance distribution of all previous dialogue tasks in chronological order before the first dialogue task to the range of [0, 1] using a normalization function, the differences between different dialogue tasks can be resolved.
[0135] S320. Based on the fine-grained averaging strategy, the importance distribution of the first dialogue task and the importance score of the current dialogue task are analyzed to obtain the correlation between each skill unit and the current dialogue task and previous dialogue tasks.
[0136] It should be understood that the importance score of the current dialogue task can also be used to construct the importance distribution of the current dialogue task.
[0137] Furthermore, by analyzing the importance distribution of the first dialogue task and the importance distribution (or importance score) of the current dialogue task for each skill unit using a fine-grained averaging strategy, the correlation between each skill unit and the current dialogue task and previous dialogue tasks can be obtained, which can then be used to distinguish between task-shared and task-specific tasks.
[0138] The fine-grained averaging strategy consolidates each skill unit based on its importance in different dialogue tasks, and adjusts the model parameters within each skill unit using the hyperparameters determined by the fine-grained averaging strategy, thereby obtaining the correlation between each skill unit and the current dialogue task and previous dialogue tasks.
[0139] This application introduces skill units. By identifying and separating these skill units, it is possible to more accurately determine which parts of the model are crucial for performing specific tasks, thereby promoting knowledge transfer in a targeted manner and reducing the risk of catastrophic forgetting.
[0140] To enhance the importance of each skill unit in different dialogue tasks and strengthen each skill unit, an importance threshold can be introduced to analyze the corresponding dialogue tasks for each skill unit. This allows for selective strengthening of skill units and prevents knowledge of specific dialogue tasks from being overwritten during updates. The following will demonstrate... Figure 4 The illustrated embodiment describes in detail how to obtain the correlation between each skill unit and the current dialogue task and previous dialogue tasks. Figure 4 This is a schematic diagram of another dialogue state tracking method provided in the embodiments of this application, such as... Figure 4 As shown, step 320, obtaining the correlation between each skill unit and the current dialogue task and previous dialogue tasks, includes the following steps:
[0141] S321. Based on a preset threshold strategy, determine the importance threshold of the current dialogue task, the importance threshold of the previous dialogue task, and the importance threshold up to the first dialogue task.
[0142] The preset threshold strategy is used to dynamically determine the importance threshold of the current dialogue task and the importance threshold of the previous dialogue tasks. It should be understood that for different dialogue tasks, the role of each skill unit in the dialogue task is different, that is, for different dialogue tasks, the importance of each skill unit in the dialogue task is different.
[0143] It should be understood that the preset threshold strategy is used to indicate the importance threshold of the current dialogue task and the importance threshold of the previous dialogue task. The importance threshold of the current dialogue task and the importance threshold of the previous dialogue task are dynamically determined. That is, for the preset threshold strategy, the importance threshold of each dialogue task can be dynamically determined according to the preset quantile corresponding to each dialogue task; or the importance threshold of each dialogue task can be dynamically determined according to the dynamic distribution of each dialogue task.
[0144] It should also be understood that the preset threshold strategy can also determine the importance threshold of all dialogue tasks through a unified preset quantile, and this preset quantile is dynamic.
[0145] This application employs a dynamic update mechanism, adjusting the learning rate and update range based on task relevance and grouping importance indicators. The key to effectively implementing selective updates is the ability to quickly adapt to new tasks without sacrificing performance on existing tasks.
[0146] For example, for each dialogue task, the preset quantile can be either the top 20% or the top 15% of importance scores. For another example, for dialogue task C, the skill units with the top 20% importance scores are selected using the preset quantile of 20%; for dialogue task D, the skill units with the top 23% importance scores are selected using the preset quantile of 23%.
[0147] It should also be understood that cumulative dialogue tasks include all previous dialogue tasks up to the first dialogue task.
[0148] In some embodiments, the importance threshold of a cumulative dialogue task can be determined by the importance threshold of each previous dialogue task.
[0149] In some embodiments, the importance threshold of the cumulative dialogue task can also be determined directly through a preset threshold strategy.
[0150] S322. Determine the importance of skill units in the current dialogue task based on the importance threshold of the current dialogue task.
[0151] S323. Determine the importance of skill units in cumulative dialogue tasks based on the importance threshold of the cumulative dialogue task.
[0152] For the current dialogue task T k If skill unit Importance score Meets the importance threshold (e.g., importance score). Equal to or higher than the importance threshold δ k If so, then the skill unit is considered important, and can be represented as If skill unit Importance score The importance threshold (e.g., importance score) is not met. Below the importance threshold δ k If the value is less than or equal to 0, then the skill unit is considered unimportant and can be represented as:
[0153] Similarly, for cumulative dialogue tasks (including T1…T…) k-1 If the skill unit Importance score Meets the importance threshold (e.g., importance score). Equal to or above the importance threshold If a skill unit is considered important, it can be represented as follows: If skill unit Importance score The importance threshold (e.g., importance score) is not met. Below the importance threshold If the value is less than or equal to 0, then the skill unit is considered unimportant and can be represented as:
[0154] S324. Based on the importance of each skill unit in the current dialogue task and the importance of each skill unit in the cumulative dialogue task, the correlation between each skill unit and the current dialogue task and the previous dialogue task is obtained.
[0155] The analysis of the importance of each skill unit in the current dialogue task and the importance of each skill unit in the cumulative dialogue task can be divided into the following four cases:
[0156] The first scenario is that the skill unit is important in both the current dialogue task and the cumulative dialogue tasks. In other words, the skill unit is important in both the current dialogue task and each previous dialogue task. Such a skill unit is a task-shared skill unit. By integrating newly acquired knowledge into such task-shared skill units, reverse knowledge transfer can be achieved.
[0157] The second scenario: Skill units are only important in the cumulative dialogue task. Such skill units are task-specific. By keeping the knowledge of the skill unit unchanged in the cumulative dialogue task, we can prevent historical knowledge from being contaminated by information unrelated to the cumulative dialogue task (knowledge in the current dialogue task).
[0158] The third scenario: Skill units are only important in the current dialogue task. These are task-specific skill units. By keeping the knowledge of this skill unit unchanged in the current dialogue task, the integrity of the model parameters within the task-specific skill unit of the current dialogue task is ensured, thus preserving the necessary knowledge for excellent performance in the current dialogue task.
[0159] Due to model fk Is The training is performed on the initialization of the data, so the knowledge learned in the past is used to enhance the performance of the current dialogue task, thereby achieving positive knowledge transfer.
[0160] The fourth scenario: If a skill unit is not important in either the current dialogue task or the cumulative dialogue task, the old and new knowledge in that skill unit can be retained, or some of the new knowledge and some of the old knowledge in that skill unit can be retained.
[0161] like Figure 2 As shown, the first region 221 and the second region 222 correspond to the regions of task-specific skill units, and the third region 210 corresponds to the regions of task-shared skill units.
[0162] The correlation between each skill unit and the current dialogue task and previous dialogue tasks can satisfy the following formula (7):
[0163]
[0164] In the formula, This represents the relevance of each skill unit to the current dialogue task and up to the previous dialogue task, where γ∈[0,1] represents a hyperparameter. This indicates that the skill unit is important for the previous dialogue mission. This indicates that the skill unit is not important to the previous dialogue quest. This indicates that the skill unit is important for the current dialogue task. This indicates that the skill unit is not important to the current dialogue task.
[0165] In the above formula, skill unit u n When both the current dialogue task and the cumulative dialogue task are important, the influence of each dialogue task can be controlled by the hyperparameter γ.
[0166] And, for skill unit u n For units that are not relevant to any dialogue task, a simple averaging method can be used.
[0167] It should be understood that, for the first and fourth cases, the determination of the relevance of the skill unit to the current dialogue task and the previous dialogue task is not limited to the algorithm in the above formula.
[0168] Figure 5 This is a schematic diagram of another dialogue state tracking method provided in the embodiments of this application, such as... Figure 5 As shown, small square 401 represents a skill unit, where, during fine-tuning, the current dialogue task T is determined. k The importance scores of skill units; and, based on a fine-grained averaging strategy, the model was merged through skill consolidation. and model f k ,Model Having accumulated knowledge from all previous tasks, the model f for the current dialogue task... k The importance scores of each skill unit in the current dialogue task are integrated. Then, we update the scores up to the current dialogue task T using formula (6). k Importance scores for all skill units, and the model
[0169] It should be understood that the model It consists of all the skill units from all previous dialogue quests. Composition, model f k It consists of all skill units in the current dialogue task. Composition, Model All skill units from all dialogue tasks (including all previous dialogue tasks and current dialogue tasks). constitute.
[0170] It should be understood that Figure 5 This only applies to the current dialogue task T. k For example, if the current dialogue task is T... k-1 T k+1 The parameters can be adjusted according to the corresponding dialogue task.
[0171] The above embodiments accurately locate task-specific skill units and task-shared skill units, addressing difficulties in catastrophic forgetting and knowledge transfer between tasks. Specifically, task-specific skill units are used to locate specific model parameters in a dialogue task, separating them from model parameters common to multiple dialogue tasks. This helps minimize interference between the learned knowledge of the current dialogue task and the retention of knowledge from previous tasks.
[0172] The above text combined Figures 1 to 5 The present application describes in detail the dialogue state tracking method according to its embodiments. The following will combine... Figure 6 and Figure 7 The present application describes the model embodiments in detail. It should be understood that the dialogue state tracking model in the embodiments of the present application can be used to perform various dialogue state tracking methods described in the foregoing embodiments of the present application.
[0173] Figure 6 This is a schematic diagram of the structure of the dialogue state tracking model provided in the embodiments of this application, as shown below. Figure 6 As shown, the dialogue state tracking model 500 includes a skill localization component 510 and a skill consolidation component 520.
[0174] The skill localization component 510 is used to determine the importance score of a target dialogue task based on skill units, wherein the skill units are determined through a preset grouping strategy; the target dialogue task is used to indicate the current or previous dialogue task in the dialogue. In other words, a new grouping importance index is used to accurately identify the distribution of parameter importance across tasks.
[0175] The Skill Consolidation Component 520 is used to consolidate skill units in the target dialogue task based on a fine-grained averaging strategy. That is, a fine-grained averaging strategy is used to integrate model parameters (mainly model weights) from current and past tasks to achieve effective knowledge transfer.
[0176] The dialogue state tracking model provided in this application addresses the problem of catastrophic forgetting. After learning a new dialogue task, the model can maintain proficiency in older dialogue tasks. Furthermore, it should facilitate cross-domain knowledge transfer to improve the performance of new dialogue tasks. This knowledge transfer includes forward knowledge transfer and backward knowledge transfer. The former utilizes knowledge from previous dialogue tasks to improve the performance of the current dialogue task, while the latter improves the performance of previous dialogue tasks after learning the current dialogue task, achieving a balance between retaining knowledge from previous dialogue tasks and completing the current dialogue task.
[0177] In some embodiments, the dialogue state tracking model can be analyzed using performance metrics such as average joint goal accuracy (JGA), forward transfer (FWT), and backward transfer (BWT).
[0178] Here, the average JGA represents the average value of the current dialogue task T. k The average JGA across all dialogue tasks after training. Forward Transfer (FWT) evaluates the model's generalization ability by measuring the average zero-shot performance. Backward Transfer (BWT) evaluates the impact of learning on subsequent tasks following previous ones.
[0179] Figure 7 This is a schematic diagram of comparative experimental data between the dialogue state tracking model provided in this application embodiment and other models, such as... Figure 7 As shown in the comparative experiments, it can be found that the dialogue state tracking model demonstrates excellent continuous learning performance in terms of effective knowledge transfer, and has made substantial improvements in average JGA, increasing from 44.1% to 62.1%, and has also made significant progress in both FWT and BWT.
[0180] The dialogue state tracking model not only surpasses CPT, which relies on memory replay, but also achieves average JGA (i.e., Figure 7The average JGA (Avg. JGA) improved from 61.2% to 62.1%, achieving a significant increase from 59.0% to 62.1%, and improved the BWT (Browser WT) from -17.9% to -9.1%, exceeding 8%, demonstrating strong reverse knowledge transfer capabilities. Furthermore, the dialogue state tracking model achieved high FWT scores under various conditions.
[0181] Notably, without relying on memory replay, the dialogue state tracking model matches the memory replay performance of DST-EGQA, particularly in BWT, where the difference is minimal at 3.2%. Furthermore, the average JGA approaches the upper limit of 64% of the performance set by the CPT multitasking policy, highlighting the effectiveness of fine-grained averaging policies for both task-sharing and task-specific skill units (model parameters).
[0182] It should be understood that the above are illustrative examples of application scenarios and do not limit the application scenarios of this application in any way.
[0183] It should be understood that the above examples are provided to help those skilled in the art understand the embodiments of this application, and are not intended to limit the embodiments of this application to the specific values or scenarios illustrated. Those skilled in the art can obviously make various equivalent modifications or changes based on the above examples, and such modifications or changes also fall within the scope of the embodiments of this application.
[0184] The above text combined Figures 1 to 5 The present application describes in detail the dialogue state tracking method according to its embodiments. The following will combine... Figure 8 This document describes in detail the device embodiments of this application. It should be understood that the dialogue state tracking device in the embodiments of this application can execute the various dialogue state tracking methods described in the foregoing embodiments of this application. That is, the specific working processes of the various products described below can be referred to the corresponding processes in the foregoing method embodiments.
[0185] Figure 8 This is a schematic diagram of the dialogue state tracking device provided in the embodiments of this application, such as... Figure 8 As shown,
[0186] It should be understood that the dialogue state tracking device 600 can perform... Figures 1 to 5 The dialogue state tracking method shown; the dialogue state tracking device 600 includes a grouping unit 610, a skill positioning unit 620 and a skill consolidation unit 630.
[0187] Grouping unit 610 is used to determine skill units in the dialogue system based on a preset grouping strategy, wherein the skill unit is a subset of model parameters in the dialogue system, and each skill unit includes one or more model parameters.
[0188] Skill positioning unit 620 is used to determine the importance score of the target dialogue task in the target dialogue task, wherein the target dialogue task is used to indicate the current dialogue task or the previous dialogue task in the dialogue.
[0189] Skill consolidation unit 630 is used to determine the relevance of each skill unit to the current dialogue task and previous dialogue tasks based on the importance score of the target dialogue task and a fine-grained averaging strategy, so as to obtain the dialogue state corresponding to the current dialogue task.
[0190] It should be noted that the aforementioned dialogue state tracking device 600 is embodied in the form of a functional unit. The term "unit" here can be implemented in software and / or hardware, without specific limitations.
[0191] For example, a "unit" can be a software program, hardware circuitry, or a combination of both that implements the above-described functions. Hardware circuitry may include application-specific integrated circuits (ASICs), electronic circuitry, a processor (e.g., a shared processor, a proprietary processor, or a group processor) and memory for executing one or more software or firmware programs, integrated logic circuitry, and / or other suitable components that support the described functions.
[0192] Therefore, the units of the various examples described in the embodiments of this application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0193] Figure 9 A schematic diagram of the structure of an electronic device provided in this application is shown. Figure 9 The dashed lines in the diagram indicate that the unit or module is optional. The electronic device 700 can be used to implement the dialogue state tracking method described in the above method embodiments.
[0194] The electronic device 700 includes one or more processors 701, which support the implementation of the dialogue state tracking method in the method embodiments of the electronic device 700. The processor 701 can be a general-purpose processor or a special-purpose processor. For example, the processor 701 can be a central processing unit (CPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, such as discrete gates, transistor logic devices, or discrete hardware components.
[0195] The processor 701 can be used to control the electronic device 700, execute software programs, and process data from the software programs. The electronic device 700 may also include a communication unit 705 for inputting (receiving) and outputting (transmitting) signals.
[0196] For example, electronic device 700 may be a chip, communication unit 705 may be the input and / or output circuit of the chip, or communication unit 705 may be the communication interface of the chip, and the chip may be a component of terminal device or other electronic device.
[0197] For example, electronic device 700 can be a terminal device, communication unit 705 can be the transceiver of the terminal device, or communication unit 705 can be the transceiver circuit of the terminal device.
[0198] The electronic device 700 may include one or more memories 702, which store a program 704. The program 704 can be executed by the processor 701 to generate instructions 703, causing the processor 701 to execute the dialogue state tracking method described in the above method embodiments according to the instructions 703.
[0199] Optionally, the memory 702 may also store data. Optionally, the processor 701 may also read the data stored in the memory 702, which may be stored at the same memory address as the program 704, or the data may be stored at a different memory address than the program 704.
[0200] The processor 701 and memory 702 can be configured separately or integrated together; for example, integrated on the system on chip (SOC) of the terminal device.
[0201] For example, the memory 702 can be used to store the related program 704 of the dialogue state tracking method provided in the embodiments of this application, and the processor 701 can be used to call the related program 704 of the dialogue state tracking method stored in the memory 702 when performing image restoration on the terminal device, and execute the dialogue state tracking method of the embodiments of this application.
[0202] This application also provides a computer program product that, when executed by processor 701, implements the dialogue state tracking method of any method embodiment of this application, or the dialogue state tracking model of any embodiment of this application.
[0203] The computer program product can be stored in memory 702, for example, program 704. Program 704 is finally converted into an executable object file that can be executed by processor 701 after processing such as preprocessing, compilation, assembly and linking.
[0204] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a computer, implements the dialogue state tracking method of any method embodiment in this application. The computer program may be a high-level language program or an executable object program.
[0205] The computer-readable storage medium is, for example, memory 702. Memory 702 can be volatile memory or non-volatile memory, or memory 702 can include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).
[0206] In this application, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.
[0207] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.
[0208] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0209] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.
[0210] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for example, the division of units is merely a logical functional division, and there may be other division methods in actual implementation; for example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.
[0211] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0212] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.
[0213] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A dialogue state tracking method, characterized by, The method comprises the following steps: determining skill units in a dialogue system based on a preset grouping strategy, wherein the skill units are subsets of model parameters in the dialogue system, and each skill unit comprises one or more model parameters; determining an importance score of a target dialogue task in the target dialogue task, wherein the target dialogue task is used to indicate a current dialogue task or a previous dialogue task in a dialogue; determining the relevance of each skill unit to the current dialogue task and the previous dialogue task based on the importance score of the target dialogue task and a fine-grained average strategy, to obtain a dialogue state corresponding to the current dialogue task. 2.The dialogue state tracking method of claim 1, wherein, The determination of the relevance of each skill unit to the current dialogue task and the previous dialogue task comprises: determining an importance distribution of dialogue tasks up to a first dialogue task based on the importance scores of the previous dialogue tasks, wherein the first dialogue task is a dialogue task adjacent to and prior to the current dialogue task in a time sequence among the previous dialogue tasks; analyzing the importance distribution of dialogue tasks up to the first dialogue task and the importance score of the current dialogue task based on the fine-grained average strategy, to obtain the relevance of each skill unit to the current dialogue task and the previous dialogue task. 3.The dialog state tracking method of claim 2, wherein, The analysis of the importance distribution of dialogue tasks up to the first dialogue task and the importance score of the current dialogue task based on the fine-grained average strategy to obtain the relevance of each skill unit to the current dialogue task and the previous dialogue task comprises: determining an importance threshold of the current dialogue task, an importance threshold of the previous dialogue task, and an importance threshold of cumulative dialogue tasks based on a preset threshold strategy, wherein the preset threshold strategy is used to dynamically determine the importance threshold of the current dialogue task and the importance threshold of the previous dialogue task; the cumulative dialogue tasks comprise all previous dialogue tasks up to the first dialogue task; determining the importance of each skill unit in the current dialogue task based on the importance threshold of the current dialogue task, and determining the importance of each skill unit in the cumulative dialogue tasks based on the importance threshold of the cumulative dialogue tasks; obtaining the relevance of each skill unit to the current dialogue task and the previous dialogue task based on the importance of each skill unit in the current dialogue task and the importance of each skill unit in the cumulative dialogue tasks. 4.The dialogue state tracking method of claim 2, wherein, The importance distribution of dialogue tasks up to the first dialogue task satisfies the following formula: In the formula, denotes the importance distribution of the first dialogue task, β ∈ [0, 1] denotes a hyperparameter, and Norm(·) denotes a normalization function, denotes the importance score of the first dialogue task, denotes the importance distribution of a dialogue task before the first dialogue task in the time sequence.
5. The dialog state tracking method of claim 3, wherein, The relevance of each skill unit to the current dialogue task and the previous dialogue task satisfies the following formula: In the formula, represents the relevance of each skill unit to the current dialogue task and the previous dialogue task, and γ∈[0,1] represents a hyperparameter, represents that the skill unit is important for the previous dialogue task, represents that the skill unit is not important for the previous dialogue task, represents that the skill unit is important for the current dialogue task, represents that the skill unit is not important for the current dialogue task. 6.The dialogue state tracking method of claim 1, wherein, The determination of the importance score of the target dialogue task in the target dialogue task comprises: determining an importance function of each skill unit in the target dialogue task, and the importance score of each skill unit satisfies the following formula: wherein denotes the importance score of a skill unit w ij denotes the model parameters, d1xd2denotes the number of total model parameters in a skill unit S(·) is a target importance function of an individual model parameter, n = 1,..., N; wherein the target importance function of a single model parameter satisfies the following formula: where S(w ij ) denotes a target importance function of model parameters w ij , t denotes an iteration number, S (t) (w ij ) denotes a target importance function of model parameters w ij at the iteration number t, denotes a smoothed sensitivity obtained by an exponential moving average, denotes an uncertainty term quantified by a local change between C (t) and .
7. A dialogue state tracking apparatus characterized by comprising: The dialogue state tracking device comprises a processor and a memory, the memory is used for storing a computer program, the processor is used for calling and running the computer program from the memory, so that the dialogue state tracking device executes the dialogue state tracking method in any one of claims 1 to 6.
8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, when the computer program is executed by the processor, the processor executes the dialogue state tracking method in any one of claims 1 to 6.
9. A dialogue state tracking model, the method comprising: The dialogue state tracking model is used to execute the dialogue state tracking method in any one of claims 1 to 6, the dialogue state tracking model comprises a skill positioning component and a skill consolidation component; wherein, The skill positioning component is used to determine the importance score of the target dialogue task based on the skill unit, wherein the skill unit is determined by a preset grouping strategy; the target dialogue task is used to indicate the current dialogue task or the previous dialogue task in the dialogue: The skill consolidation component is used to consolidate the skill unit in the target dialogue task based on a fine-grained average strategy.
10. A computer program product, characterised in that, The computer program product comprises a computer program, when the computer program is executed by the processor, the processor executes the method in any one of claims 1 to 6, or the processor executes the model in claim 9.