A training method for Korean dialogue state tracking model

Abstract

The invention discloses a training method for a Korean dialogue state tracking model, and relates to the technical field of electronic digital data processing. First, user dialogue interaction feature data in a Korean dialogue interaction platform within a tracking supervision period is sensed, so that user needs and behavior patterns can be better understood, and the model optimization correction proportional coefficient of the Korean dialogue interaction platform is analyzed to improve the optimization accuracy of the training data volume. At the same time, the model performance data and user feedback data in the Korean dialogue interaction platform within the tracking supervision period are analyzed to comprehensively evaluate the actual performance of the model, and then the model training set of the Korean dialogue interaction platform is analyzed to update and inject a specified number of times, so that the training strategy can be adjusted and the training effect of the model can be optimized. Finally, the training set is updated and injected for training, which can improve the accuracy and response quality of the model.

Description

Technical Field

[0001] The present invention relates to the technical field of electronic digital data processing, and in particular to a training method for a Korean dialogue state tracking model. Background Art

[0002] In the fields of intelligent customer service systems and virtual assistant applications, the state tracking model can help the system understand the user's Korean conversation requests, intentions, and context, thereby providing more accurate and timely services and answers. The training data of Korean conversations will directly affect the performance of the state tracking model, resulting in the system possibly being unable to provide accurate answers, which in turn affects the practicality of the state tracking model.

[0003] Prior art, such as the invention patent publication CN117149987B, discloses a method and apparatus for training a multilingual dialogue state tracking model. This method includes obtaining source language data; performing code conversion on multiple dialogue contexts based on a multilingual dictionary to obtain code-converted dialogue context information; and performing multi-task joint training on the multilingual training model using the code-converted dialogue context information as training samples and a first function as a training loss function to obtain a multilingual dialogue state tracking model. By designing multiple fine-tuning tasks, this method constructs an effective multilingual generative dialogue state tracking model, enabling the model to understand dialogue contexts in different languages ​​and improving multilingual dialogue state tracking performance.

[0004] Existing technology, such as the invention patent publication number CN117150023B, discloses a method for tracking conversation states for intelligent dialogue systems. This method involves deep learning and natural language processing technologies, including obtaining and preprocessing conversation sentences and conversation states in the intelligent dialogue system; obtaining a word vector feature representation matrix based on the preprocessed data in conjunction with a BERT pre-trained model; performing a first domain classification on the feature vectors in the word vector feature representation matrix; performing a second state classification on the slot context vectors containing domain information, determining a new slot word corresponding to the slot based on the identifier of the classified operation state, and completing conversation state tracking for the intelligent dialogue system based on the new slot word. This invention improves the accuracy of relevant public datasets, reduces the amount of data, and highly refines and preserves key information from historical conversations, demonstrating excellent practicality in intelligent dialogue systems.

[0005] Combining the above solutions, we found that currently in the field of dialogue state tracking, only a single analysis of contextual information or data volume is usually performed, which cannot reduce the impact of the training data set on the performance of the state tracking model, and may affect the accuracy of dialogue state tracking. Summary of the Invention

[0006] In view of the deficiencies in the prior art, the present invention provides a training method for a Korean dialogue state tracking model, which can effectively solve the problems involved in the above-mentioned background technology.

[0007] To achieve the above objectives, the present invention is implemented through the following technical solutions: a training method for a Korean dialogue state tracking model, comprising marking the application platform of the Korean dialogue state tracking model as a Korean dialogue interaction platform, and perceiving user dialogues in the Korean dialogue interaction platform within a preset tracking supervision period, and obtaining user dialogue interaction feature data in the Korean dialogue interaction platform within the tracking supervision period.

[0008] Based on the user conversation interaction feature data in the Korean conversation interaction platform during the tracking and supervision cycle, the model optimization correction proportional coefficient of the Korean conversation interaction platform is obtained through processing and transmitted to the integrated data processor.

[0009] Statistically track the model performance data and user feedback data in the Korean conversation interaction platform during the supervision cycle, obtain the comprehensive model performance representation value of the Korean conversation interaction platform through processing, and transmit it to the integrated data processor.

[0010] The integrated data processor receives the model optimization correction ratio coefficient and the model comprehensive performance representation value of the Korean dialogue interaction platform, and obtains the model training set of the Korean dialogue interaction platform through centralized processing to update the specified number of injections.

[0011] The Korean dialogue state tracking model is updated and injected with a specified number of training sets using the model training set to perform training.

[0012] Furthermore, the user dialogue interaction feature data in the Korean dialogue interaction platform during the tracking and supervision period includes the speech speed of each Korean dialogue of each user, the input length of each Korean dialogue of each user, and the Korean dialogue turns of each user.

[0013] Furthermore, the model optimization correction proportional coefficient of the Korean dialogue interaction platform is obtained, and the specific analysis process is: during the tracking and supervision cycle, the speech speed of each Korean dialogue of each user and the input length of each Korean dialogue of each user are counted.

[0014] The Korean conversation turns of each user in the tracking supervision cycle are extracted, and the model optimization correction coefficient of the Korean conversation interaction platform is obtained through comprehensive analysis based on the speech speed and input length of each user's Korean conversation.

[0015] Furthermore, the statistical tracking of model performance data and user feedback data in the Korean dialogue interaction platform during the supervision period includes statistical tracking of model performance data in the Korean dialogue interaction platform during the supervision period and statistical tracking of user feedback data in the Korean dialogue interaction platform during the supervision period.

[0016] Furthermore, the statistical tracking of model performance data in the Korean dialogue interaction platform during the supervision period is specifically carried out as follows: the model performance data in the Korean dialogue interaction platform during the supervision period includes the model generation response time in the Korean dialogue interaction platform and the user's Korean dialogue start time, the number of repeated inquiries of the same question by each user, and the total number of inquiries by each user.

[0017] The difference between the model generation response time and the user's Korean conversation start time in the Korean conversation interaction platform during the tracking supervision cycle is recorded as the response time, and the average value is taken to obtain the average response time of the model in the Korean conversation interaction platform.

[0018] The average of the ratios of the number of times each user repeatedly asks the same question to the total number of times each user asks a question is taken to obtain the average repeated question rate of each Korean conversation of each user.

[0019] Based on the average response time of the model in the Korean conversation interaction platform and the average repeated inquiry rate of each Korean conversation of each user, the model performance index value in the Korean conversation interaction platform was analyzed.

[0020] Furthermore, the statistical tracking of user feedback data in the Korean dialogue interaction platform during the supervision period is specifically carried out as follows: the user feedback data in the Korean dialogue interaction platform during the supervision period includes the number of repeated questions asked by each user for the same question, the total number of questions asked by each user, and the frequency of occurrence of negative feedback words in the feedback information of each user.

[0021] The time interval from the start time to the end time of each user's Korean conversation during the monitoring cycle is statistically tracked to obtain the duration of each user's Korean conversation.

[0022] The average of the ratios of the number of times each user repeatedly asks the same question to the total number of times each user asks a question is taken to obtain the average repeated question rate of each Korean conversation of each user.

[0023] The frequency of negative feedback words in the feedback information of each user in the Korean dialogue interaction platform is extracted. Combined with the duration of each Korean conversation of each user, the user interaction quality index value in the Korean dialogue interaction platform is analyzed.

[0024] Furthermore, the processing obtains the model comprehensive performance representation value of the Korean dialogue interaction platform. The specific analysis process is: based on the model performance index value in the Korean dialogue interaction platform and the user interaction quality index value in the Korean dialogue interaction platform, a comprehensive analysis is performed to obtain the model comprehensive performance representation value of the Korean dialogue interaction platform.

[0025] Furthermore, the model training set of the Korean dialogue interaction platform is updated and injected with a specified number through centralized processing. The specific process is: the integrated data processor receives the model optimization correction ratio coefficient and the model comprehensive performance representation value of the Korean dialogue interaction platform, analyzes and obtains the model optimization index value of the Korean dialogue interaction platform, and then matches and obtains the model training set of the Korean dialogue interaction platform. The specified number of injections are updated.

[0026] Furthermore, the model comprehensive performance representation value of the Korean dialogue interaction platform is specifically analyzed under the following conditions:

[0027] Where, Represents the comprehensive performance representation value of the model of the Korean dialogue interaction platform, Represents the model performance index value in the Korean dialogue interaction platform, Indicates the weight factor corresponding to the set model performance index value, Indicates the user interaction quality index value in the Korean dialogue interaction platform. It represents the weight factor corresponding to the set user interaction quality index value, and e represents a natural constant.

[0028] Furthermore, the model optimization index value of the Korean dialogue interaction platform is specifically analyzed under the following conditions:

[0029] Where, represents the model optimization index value of the Korean dialogue interaction platform, Represents the comprehensive performance representation value of the model of the Korean dialogue interaction platform, Represents the model optimization correction scale coefficient of the Korean dialogue interaction platform.

[0030] The present invention has the following beneficial effects:

[0031] (1) The present invention provides a training method for a Korean conversation state tracking model. First, the user conversation interaction feature data in the Korean conversation interaction platform is analyzed to identify common patterns and problems in the conversation. Then, the model performance data and user feedback data are analyzed to understand the advantages and disadvantages of the model in practical applications. The analysis obtains a specified number of updates to the model training set, and the training set update injection training is performed, which helps to improve the model's ability to understand user needs, thereby improving the quality of conversation.

[0032] (2) By analyzing the user dialogue interaction feature data in the Korean dialogue interaction platform, the present invention can understand the common patterns and problems in user dialogues, and then analyze and obtain the model optimization correction coefficient, which can improve the fit between the training data and the actual dialogue scenario, thereby improving the generalization ability of the model.

[0033] (3) By analyzing the model performance data and user feedback data in the Korean dialogue interaction platform, the present invention can discover the deficiencies of the model, thereby providing data support for updating the model training set. Analyzing the model performance data and user feedback data can help adjust the model training set, thereby improving the overall user experience and satisfaction.

[0034] (4) The present invention performs update injection training of the Korean dialogue state tracking model by injecting a specified number of training sets into the model training set, which helps the model learn new dialogue patterns and user behaviors, thereby improving the accuracy of dialogue state prediction and further improving the flexibility of model dialogue. The updated model can more accurately understand and process user input, thereby improving the user's interactive experience.

[0035] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 Schematic diagram of the method of the present invention.

[0037] Figure 2 This is an example diagram of the model's comprehensive performance representation value for the Korean conversation interaction platform. DETAILED DESCRIPTION

[0038] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0039] See also Figure 1 As shown, an embodiment of the present invention provides a technical solution: a training method for a Korean dialogue state tracking model, comprising marking an application platform of the Korean dialogue state tracking model as a Korean dialogue interaction platform, and perceiving user dialogues in the Korean dialogue interaction platform within a preset tracking supervision period, and obtaining user dialogue interaction feature data in the Korean dialogue interaction platform within the tracking supervision period.

[0040] Based on the user conversation interaction feature data in the Korean conversation interaction platform during the tracking and supervision cycle, the model optimization correction proportional coefficient of the Korean conversation interaction platform is obtained through processing and transmitted to the integrated data processor.

[0041] It should be added that the integrated data processor is a software tool used to integrate and analyze data from multiple sources to optimize model training and updating. It is mainly responsible for receiving and processing model optimization correction coefficients and model comprehensive performance representation values ​​from the Korean dialogue interaction platform.

[0042] Statistically track the model performance data and user feedback data in the Korean conversation interaction platform during the supervision cycle, obtain the comprehensive model performance representation value of the Korean conversation interaction platform through processing, and transmit it to the integrated data processor.

[0043] The integrated data processor receives the model optimization correction ratio coefficient and the model comprehensive performance representation value of the Korean dialogue interaction platform, and obtains the model training set of the Korean dialogue interaction platform through centralized processing to update the specified number of injections.

[0044] The Korean dialogue state tracking model is updated and injected with a specified number of training sets using the model training set to perform training.

[0045] It should be added that the Korean dialogue state tracking model is updated and injected with a specified number of training sets using the model training set update. The specific process is to select a model training set from the existing model training set and inject a specified number of training sets for updating, merge the updated model training set with the existing training set to form a new training set, perform model training on the new training set, and deploy the trained model to the Korean dialogue interaction platform.

[0046] It should be added that the model training set usually includes conversation history (including user input and model response), conversation state (such as user intent and slot information), conversation tags (such as annotations of conversation state), and possible entity and relationship annotation data. This data helps the model understand the context and state of the conversation, thereby optimizing conversation management and state prediction.

[0047] It should be noted that a specified number of training sets are injected into the updated model training set from the existing model training set, and newly injected training set data is prepared at the same time. The newly injected training set data is added to the specified number of training sets injected into the updated model training set to obtain an updated model training set. The updated model training set is merged with the existing training set to form a new training set. This can be achieved through a simple splicing operation. For example, if the existing training set contains 10,000 samples and the newly injected training set contains 1,000 samples, then the merged new training set will contain 11,000 samples.

[0048] It is important to note that the model should be trained on a new training set, deployed to the Korean conversational interaction platform, trained on the new training set, and the model's hyperparameters (such as learning rate and batch size) should be adjusted to optimize performance. The trained model should be exported to a suitable format to ensure compatibility with the Korean conversational interaction platform. If the platform provides an API interface, the trained model should be integrated into the platform's API to load and infer the model and configure the API call parameters. For example, to develop a Korean customer service conversation system, a basic Korean conversation model should be established. 1,000 new customer conversations, including new topics such as order tracking and return policies, should be collected. These 1,000 data points should be merged with the existing 5,000 data points, resulting in a total of 6,000 data points. The merged data should be used to retrain the BERT model, hyperparameters should be adjusted to optimize performance, the trained model should be exported as a .pt file, and integrated into the customer service platform's API.

[0049] Specifically, the user dialogue interaction feature data in the Korean dialogue interaction platform during the tracking supervision period includes the speech speed of each Korean dialogue of each user, the input length of each Korean dialogue of each user, and the Korean dialogue turns of each user.

[0050] Specifically, the model optimization correction coefficient of the Korean dialogue interaction platform is obtained. The specific analysis process is: during the tracking and supervision cycle, the speech speed of each Korean dialogue of each user and the input length of each Korean dialogue of each user are counted.

[0051] It should be noted that the Korean conversation rounds of each user refer to the number of communication rounds between the user and the model in a conversation. Each time the user speaks and the platform responds is counted as a round. The conversation records of each user are extracted from the conversation log, and each conversation record is traversed to calculate the number of interaction rounds between the user and the platform. For example, if the platform responds after the user speaks, and then the user speaks again and receives a response, then the conversation has two rounds. The total number of conversation rounds for each user is counted to obtain the Korean conversation rounds of each user.

[0052] It should be noted that the speech rate of each Korean conversation of each user refers to the speaking speed, usually expressed in words per minute. The speech of the Korean conversation is transcribed into text using speech recognition technology, and the length of the text and the total duration of the speech are calculated. The value of the speech rate of the Korean conversation is the length of the text divided by the total duration of the speech.

[0053] It should be noted that the input length of each Korean conversation of each user refers to the length of each speech of the user in the Korean conversation, which is usually expressed in the number of words or characters. The number of characters or words can be calculated using a programming language or text analysis tool.

[0054] The Korean conversation turns of each user in the tracking supervision cycle are extracted, and the model optimization correction coefficient of the Korean conversation interaction platform is obtained through comprehensive analysis based on the speech speed and input length of each user's Korean conversation.

[0055] It should be noted that the model optimization correction coefficient of the Korean dialogue interaction platform is obtained through comprehensive analysis. The specific process is to compare the speech rate of each Korean dialogue of each user and the input length of each Korean dialogue of each user with the set thresholds corresponding to the speech rate and the set thresholds corresponding to the input length, respectively. If the speech rate of a Korean dialogue of a certain user is higher than or equal to the set threshold corresponding to the speech rate and the input length of the Korean dialogue of the user is higher than or equal to the set threshold corresponding to the input length, then the model optimization correction coefficient of the Korean dialogue interaction platform can be obtained by combining the Korean dialogue turns of each user, the speech rate of each Korean dialogue of each user, and the input length of each Korean dialogue of each user. If the speech rate of a Korean dialogue of a certain user is lower than the set threshold corresponding to the speech rate or the input length of the Korean dialogue of the user is lower than the set threshold corresponding to the input length, then only the Korean dialogue turns of each user need to be considered to obtain the model optimization correction coefficient of the Korean dialogue interaction platform.

[0056] It should be noted that the model optimization correction coefficient of the Korean conversation interaction platform is a numerical quantitative result obtained by analyzing the user conversation interaction feature data in the Korean conversation interaction platform. It is used to comprehensively quantify the impact of Korean conversation turns on the model optimization correction coefficient of the Korean conversation interaction platform. It can be obtained through the following analysis method. The specific analysis conditions are as follows:

[0057] ;

[0058] Where, Indicates the model optimization correction coefficient of the Korean dialogue interaction platform, represents the Korean conversation turn of the i-th user, Indicates the impact factor corresponding to the set Korean dialogue round, represents the speech speed of the jth Korean conversation of the i-th user, Indicates the threshold corresponding to the set speech speed. Indicates the impact factor corresponding to the set speech speed. represents the input length of the jth Korean conversation of the i-th user, Indicates the threshold corresponding to the set input length, Indicates the impact factor corresponding to the set input length, Indicates the number of each user, , represents the total number of users, j represents the number of each Korean conversation, , Indicates the total number of Korean conversations.

[0059] It should be noted that, in this embodiment, the impact factors corresponding to the preset Korean dialogue turns, the impact factors corresponding to the speech rate, and the impact factors corresponding to the input length are obtained from the Korean dialogue tracking database. The impact factors corresponding to the Korean dialogue turns, the impact factors corresponding to the speech rate, and the impact factors corresponding to the input length are all numbers between 0 and 1.

[0060] It should be noted that the influence factors corresponding to the preset Korean conversation turns obtained in the Korean conversation tracking database represent the numerical values ​​of the degree of influence of the Korean conversation turns on the model optimization correction proportional coefficient of the Korean conversation interaction platform, and the corresponding relationship is a pre-set mapping relationship. For example, the Korean conversation turns and the influence factors corresponding to the preset Korean conversation turns in the Korean conversation tracking database form a mapping set, and the real-time Korean conversation turns of a certain user are input into the mapping set to obtain the influence factors corresponding to the Korean conversation turns of the user.

[0061] It should be noted that the influence factor corresponding to the preset speech rate obtained in the Korean conversation tracking database represents the numerical value of the influence of the speech rate of the Korean conversation on the model optimization correction proportional coefficient of the Korean conversation interaction platform, and the corresponding relationship is a pre-set mapping relationship. For example, the speech rate of the Korean conversation and the influence factor corresponding to the preset speech rate in the Korean conversation tracking database form a mapping set, and the real-time speech rate of a Korean conversation of a certain user is input into the mapping set to obtain the influence factor corresponding to the speech rate of the Korean conversation of the user.

[0062] It should be noted that the influence factor corresponding to the preset input length obtained in the Korean conversation tracking database represents the numerical value of the influence of the input length of the Korean conversation on the model optimization correction proportional coefficient of the Korean conversation interaction platform, and the corresponding relationship is a pre-set mapping relationship. For example, the input length of the Korean conversation and the influence factor corresponding to the preset input length in the Korean conversation tracking database form a mapping set, and the real-time input length of a Korean conversation of a certain user is input into the mapping set to obtain the influence factor corresponding to the input length of the Korean conversation of the user.

[0063] It should be added that the more rounds of Korean conversations each user has, the more the model cannot quickly solve the user's problem, and the more the model needs to be optimized. If the speech speed of the Korean conversation is higher than or equal to the set threshold corresponding to the speech speed, it means that the speech speed of the Korean conversation exceeds the range that the model can adapt to, and it is necessary to consider the impact of the speech speed on the model and then optimize the model. If the input length of the Korean conversation is higher than or equal to the set threshold corresponding to the input length, it means that the input length of the Korean conversation exceeds the range that the model can adapt to, and it is necessary to consider the impact of the input length on the model and then optimize the model.

[0064] It should be explained that the analysis method of this embodiment, by combining the Korean conversation turns of each user, the speech speed of each Korean conversation of each user, and the input length of each Korean conversation of each user for analysis, obtains the model optimization correction proportional coefficient of the Korean conversation interaction platform, which helps to evaluate the model's response performance to the user's complex conversation environment, and can adjust the model more accurately according to user needs. Complex conversation environments include fast speech speed and long conversation length, which may affect the model's understanding accuracy of Korean conversations.

[0065] It should be noted that the number of Korean conversation turns for each user, the speech rate of each Korean conversation for each user, and the input length of each Korean conversation for each user in this formula are correlated and do not exist independently. For example, long Korean conversation turns are usually accompanied by more complex conversation content, which may lead to large variations in the speech rate of Korean conversations. The variation in speech rate in multiple rounds of Korean conversations increases the processing difficulty. Therefore, more training data is needed to adapt to this complexity. The quality of model training can be ensured by adjusting the amount of data to avoid over- or under-training.

[0066] In this implementation scheme, analyzing the Korean conversation turns of each user helps to evaluate the adaptability of the Korean conversation interaction platform to the input length of long Korean conversations and interactions with multiple Korean conversation turns. Analyzing the speech rate of each user's Korean conversations can help adjust the processing speed and response time of the model. Analyzing the input length of each user's Korean conversations can evaluate the Korean conversation interaction platform's processing capabilities for different input lengths. The model optimization correction proportional coefficient of the Korean conversation interaction platform obtained through analysis can improve the representativeness and comprehensiveness of the training set, thereby optimizing the performance and reliability of the model in practical applications.

[0067] Specifically, the statistical tracking of model performance data and user feedback data in the Korean dialogue interaction platform during the regulatory period includes statistical tracking of model performance data in the Korean dialogue interaction platform during the regulatory period and statistical tracking of user feedback data in the Korean dialogue interaction platform during the regulatory period.

[0068] Specifically, the model performance data in the Korean dialogue interaction platform during the tracking and supervision period are statistically tracked, and the specific process is: the model performance data in the Korean dialogue interaction platform during the tracking and supervision period includes the model generation response time in the Korean dialogue interaction platform and the user's Korean dialogue start time, the number of repeated inquiries of the same question by each user, and the total number of inquiries by each user.

[0069] The difference between the model generation response time and the user's Korean conversation start time in the Korean conversation interaction platform during the tracking supervision cycle is recorded as the response time, and the average value is taken to obtain the average response time of the model in the Korean conversation interaction platform.

[0070] The average of the ratios of the number of times each user repeatedly asks the same question to the total number of times each user asks a question is taken to obtain the average repeated question rate of each Korean conversation of each user.

[0071] It should be explained that the total number of inquiries from each user refers to the total number of inquiries from the same user during the tracking and supervision cycle.

[0072] It should be noted that the average model response time in the Korean conversational interaction platform refers to the average time from the platform receiving user input to generating and sending a response. The timestamps of each user input and model response are recorded, and data analysis tools (such as Python scripts, SQL queries, etc.) are used to process the timestamp data to calculate the response time.

[0073] It should be noted that the average repeated question rate of each Korean conversation of each user refers to the frequency with which each user repeatedly asks the same or similar questions in the conversation. The average repeated question rate can be obtained by extracting question texts from the conversation log and performing statistics and analysis.

[0074] Based on the average response time of the model in the Korean conversation interaction platform and the average repeated inquiry rate of each Korean conversation of each user, the model performance index value in the Korean conversation interaction platform was analyzed.

[0075] It should be noted that the model performance index value in the Korean conversational interaction platform represents a numerical quantitative result obtained by analyzing the model performance data in the Korean conversational interaction platform. It is used to comprehensively quantify the impact of the average response time of the model on the model performance index value in the Korean conversational interaction platform. It can be obtained through the following analysis method. The specific analysis conditions are as follows:

[0076] ;

[0077] Where, Represents the model performance index value in the Korean dialogue interaction platform, Represents the average response time of the model in the Korean conversation interaction platform Indicates the set reference response time, Indicates the impact factor corresponding to the set response time, represents the average repeated inquiry rate of the jth Korean conversation of the i-th user, Indicates the set reference re-query rate, Indicates the impact factor corresponding to the set repeated query rate, Indicates the number of each user, , represents the total number of users, j represents the number of each Korean conversation, , represents the total number of Korean conversations, and e represents a natural constant.

[0078] It should be noted that, in this embodiment, the preset impact factors corresponding to the response time and the impact factors corresponding to the repeated inquiry rate are obtained from the Korean dialogue tracking database, and the impact factors corresponding to the response time and the impact factors corresponding to the repeated inquiry rate are both numbers between 0 and 1.

[0079] It should be noted that the impact factor corresponding to the preset response time obtained in the Korean conversation tracking database represents the numerical value of the degree of influence of the average response time of the model in the Korean conversation interaction platform on the model performance index value in the Korean conversation interaction platform, and the corresponding relationship is a pre-set mapping relationship. For example, the average response time of the model in the Korean conversation interaction platform and the impact factor corresponding to the preset response time in the Korean conversation tracking database form a mapping set, and the average response time of the model in the real-time Korean conversation interaction platform is input into the mapping set to obtain the impact factor corresponding to the response time.

[0080] It should be noted that the influence factor corresponding to the preset repeated inquiry rate obtained in the Korean conversation tracking database represents the numerical value of the degree of influence of the average repeated inquiry rate of Korean conversations on the model performance indicator value in the Korean conversation interaction platform, and the corresponding relationship is a pre-set mapping relationship. For example, the average repeated inquiry rate of Korean conversations and the influence factor corresponding to the preset repeated inquiry rate in the Korean conversation tracking database form a mapping set, and the real-time average repeated inquiry rate of a Korean conversation of a certain user is input into the mapping set to obtain the influence factor corresponding to the repeated inquiry rate of the Korean conversation of the user.

[0081] It should be explained that the analysis method of this embodiment combines the average response time of the model in the Korean conversation interaction platform and the average repeated inquiry rate of each Korean conversation of each user for analysis to obtain the model performance index value in the Korean conversation interaction platform, which can more comprehensively evaluate the performance of the model and understand the performance of the model in actual conversations.

[0082] It should be noted that the average response time of the model in the Korean conversation interaction platform in this formula is correlated with the average repeated inquiry rate of each Korean conversation of each user, and does not exist independently. For example, a shorter response time can reduce the user's waiting time, thereby reducing repeated inquiries caused by long waits or unclear responses. When the model responds quickly and accurately, users usually get the information they need, thereby reducing the need for repeated inquiries. Comprehensive analysis can more accurately evaluate the model performance indicators in the Korean conversation interaction platform.

[0083] It should be noted that a shorter average model response time usually means that the model processes user requests more efficiently. Analyzing the average model response time helps improve the accuracy of model performance indicator evaluation. A higher repeat inquiry rate usually indicates that users are dissatisfied with the model's feedback. Users repeat inquiries because the model fails to accurately understand the user's intentions. Analyzing the repeat inquiry rate can more accurately evaluate model performance indicators in the Korean conversation interaction platform.

[0084] In this implementation, analyzing the average response time of the model in the Korean conversation interaction platform helps to evaluate the fluency of the overall model conversation. Analyzing the average repeated inquiry rate of each user's Korean conversation can show the user's difficulty in seeking information, thereby more accurately evaluating the performance of the model. The model performance index values ​​in the Korean conversation interaction platform obtained by analysis help to improve the accuracy of the model's performance evaluation and optimize model efficiency and information quality.

[0085] Specifically, the user feedback data in the Korean dialogue interaction platform during the monitoring period is statistically tracked, and the specific process is as follows: the user feedback data in the Korean dialogue interaction platform during the monitoring period is tracked, including the frequency of occurrence of negative feedback words in the feedback information of each user.

[0086] The time interval from the start time to the end time of each user's Korean conversation during the monitoring cycle is statistically tracked to obtain the duration of each user's Korean conversation.

[0087] It should be noted that the frequency of occurrence of negative feedback words in each user's feedback information refers to the number of times the negative words appear in all the user's feedback information. Text mining technology (such as word frequency statistics) is usually used to calculate the frequency of occurrence of negative feedback words. Negative feedback words refer to words that contain negative emotions, such as "bad", "unsatisfied", and "disappointed".

[0088] It should be noted that the duration of each Korean conversation of each user refers to the time interval from the start to the end of the conversation, and the duration can be calculated using a data processing tool or an analysis script.

[0089] The frequency of negative feedback words in the feedback information of each user in the Korean dialogue interaction platform is extracted. Combined with the duration of each Korean conversation of each user, the user interaction quality index value in the Korean dialogue interaction platform is analyzed.

[0090] It should be noted that the user interaction quality index value in the Korean conversation interaction platform represents a numerical quantitative result obtained by analyzing user feedback data in the Korean conversation interaction platform. It is used to comprehensively quantify the impact of the frequency of negative feedback words on the user interaction quality index value in the Korean conversation interaction platform. It can be obtained through the following analysis method. The specific analysis conditions are as follows:

[0091] ;

[0092] Where, Indicates the user interaction quality index value in the Korean dialogue interaction platform. represents the frequency of negative feedback words in the feedback information of each user of the i-th user, Indicates the set reference frequency, Indicates the impact factor corresponding to the set frequency, represents the duration of the jth Korean conversation of the i-th user, Indicates the set reference duration. Indicates the impact factor corresponding to the set duration, Indicates the number of each user, , represents the total number of users, j represents the number of each Korean conversation, , Indicates the total number of Korean conversations.

[0093] It should be noted that, in this embodiment, the preset impact factors corresponding to the frequency and the impact factors corresponding to the duration are obtained from the Korean dialogue tracking database, and the impact factors corresponding to the frequency and the impact factors corresponding to the duration are both numbers between 0 and 1.

[0094] It should be noted that the influence factor corresponding to the preset frequency obtained in the Korean conversation tracking database represents the numerical value of the influence of the frequency of occurrence of negative feedback words in the user's feedback information on the user interaction quality index value in the Korean conversation interaction platform, and the corresponding relationship is a pre-set mapping relationship. For example, the frequency of occurrence of negative feedback words in the user's feedback information and the influence factors corresponding to the preset frequencies in the Korean conversation tracking database form a mapping set, and the frequency of occurrence of negative feedback words in the real-time feedback information of a certain user is input into the mapping set to obtain the influence factor corresponding to the frequency of occurrence of negative feedback words in the feedback information of the user.

[0095] It should be noted that the influence factor corresponding to the preset duration obtained in the Korean conversation tracking database represents the numerical value of the influence of the duration of the Korean conversation on the user interaction quality index value in the Korean conversation interaction platform, and the corresponding relationship is a pre-set mapping relationship. For example, the duration of the Korean conversation and the influence factor corresponding to the preset duration in the Korean conversation tracking database form a mapping set, and the real-time duration of a Korean conversation of a certain user is input into the mapping set to obtain the influence factor corresponding to the duration of the Korean conversation of the user.

[0096] It should be explained that the analysis method of this embodiment, by combining the frequency of occurrence of negative feedback words in the feedback information of each user and the duration of each Korean conversation of each user for analysis, obtains the user interaction quality index value in the Korean conversation interaction platform, which helps to improve the accuracy of the user interaction quality index value analysis.

[0097] It should be added that the frequency of negative feedback words in each user's feedback information in this formula is correlated with the duration of each user's Korean conversation, and does not exist independently. For example, a longer conversation duration is often associated with a higher frequency of negative feedback words, because the user may have experienced a long waiting time or repeated inquiries in the conversation and failed to get a satisfactory answer, so he expressed more negative emotions and gave negative feedback. Comprehensive analysis can more accurately evaluate the user interaction quality index value.

[0098] It should be noted that a high frequency of negative feedback words usually indicates a poor user experience and may affect the overall interaction quality index value. A longer conversation duration may indicate that the user needs more support or the conversation quality is higher, but it may also indicate a poor user experience, resulting in a long interaction time. Analyzing the frequency of negative feedback words in each user's feedback information and the duration of Korean conversations can improve the accuracy of the user interaction quality index value analysis.

[0099] In this implementation plan, analyzing the frequency of negative feedback words in each user's feedback information helps to identify delay issues in the user experience, thereby optimizing system response time and improving user satisfaction. Analyzing the duration of each user's Korean conversation reveals the depth of the Korean conversation, helps improve conversation guidance strategies, and increases user engagement. The user interaction quality index value obtained by analysis in the Korean conversation interaction platform can comprehensively evaluate the quality of user interaction.

[0100] Specifically, the model comprehensive performance representation value of the Korean dialogue interaction platform is obtained through processing. The specific analysis process is: based on the model performance index value in the Korean dialogue interaction platform and the user interaction quality index value in the Korean dialogue interaction platform, a comprehensive analysis is performed to obtain the model comprehensive performance representation value of the Korean dialogue interaction platform.

[0101] Specifically, the specified number of model training set updates and injections of the Korean dialogue interaction platform is obtained through centralized processing. The specific process is: the integrated data processor receives the model optimization correction ratio coefficient and the model comprehensive performance representation value of the Korean dialogue interaction platform, analyzes and obtains the model optimization index value of the Korean dialogue interaction platform, and then matches and obtains the specified number of model training set updates and injections of the Korean dialogue interaction platform.

[0102] It should be noted that the model optimization index value of the Korean conversation interaction platform is analyzed and then matched to obtain the specified number of model training set updates and injections of the Korean conversation interaction platform. The specific process is to construct a mapping set between the model optimization index value interval of the Korean conversation interaction platform and its corresponding specified number of model training set updates and injections of the Korean conversation interaction platform, input the real-time model optimization index value of the Korean conversation interaction platform, match the interval in which the model optimization index value of the Korean conversation interaction platform is located, and obtain the specified number of model training set updates and injections of the Korean conversation interaction platform corresponding to the model optimization index value of the Korean conversation interaction platform through the mapping set, wherein the mapping relationship is a one-to-one relationship.

[0103] Specifically, the model comprehensive performance representation value of the Korean dialogue interaction platform is analyzed under the following conditions:

[0104] ;

[0105] Where, Represents the comprehensive performance representation value of the model of the Korean dialogue interaction platform, Represents the model performance index value in the Korean dialogue interaction platform, Indicates the weight factor corresponding to the set model performance index value, Indicates the user interaction quality index value in the Korean dialogue interaction platform. It represents the weight factor corresponding to the set user interaction quality index value, and e represents a natural constant.

[0106] In a specific embodiment, Table 1 shows different model performance index values ​​and user interaction quality index values ​​and corresponding model comprehensive performance representation values. In this embodiment, the weight factor corresponding to the model performance index value is 0.7, and the weight factor corresponding to the user interaction quality index value is 0.3.

[0107] Table 1 Performance index values ​​of different models and user interaction quality index values ​​and corresponding model comprehensive performance representation values

[0108]

[0109] It should be explained that in Table 1, as the model performance index values ​​and user interaction quality index values ​​of the Korean conversation interaction platform increase, the model comprehensive performance representation value decreases accordingly. Table 1 lists the model performance index values ​​and user interaction quality index values ​​of three different Korean conversation interaction platforms, and the corresponding model comprehensive performance representation values ​​are obtained through the formula. These model comprehensive performance representation values ​​provide the degree of influence of different model performance on the model comprehensive performance representation value. The model comprehensive performance representation value comprehensively considers the model performance data and interaction quality data of the Korean conversation interaction platform, provides data support for the model update of the Korean conversation interaction platform, and improves the accuracy of the model comprehensive performance evaluation of the Korean conversation interaction platform.

[0110] What needs to be explained is that Figure 2 This is an example diagram of the comprehensive performance representation value of the model for the Korean dialogue interaction platform, such as Figure 2 As shown, the x-axis represents the model performance index value in the Korean conversation interaction platform, and the y-axis represents the model comprehensive performance representation value of the Korean conversation interaction platform. In this embodiment, three different groups of example parameters are defined in the figure, corresponding to different situations of the three curves, respectively represented by solid lines, dashed lines and dotted lines, and the corresponding curve labels are a, b, and c.

[0111] It should be explained that when the user interaction quality index value in the Korean dialogue interaction platform is 1.0, the functional relationship between the model performance index value in the Korean dialogue interaction platform and the model comprehensive performance representation value is shown in curve a. When the user interaction quality index value in the Korean dialogue interaction platform is 1.4, the functional relationship between the model performance index value in the Korean dialogue interaction platform and the model comprehensive performance representation value is shown in curve b. When the user interaction quality index value in the Korean dialogue interaction platform is 1.8, the functional relationship between the model performance index value in the Korean dialogue interaction platform and the model comprehensive performance representation value is shown in curve b. As shown in curve c, the comprehensive performance representation value of the model in the Korean dialogue interaction platform decreases as the model performance index value in the Korean dialogue interaction platform increases, and when the user interaction quality index value in the Korean dialogue interaction platform increases, the comprehensive performance representation value of the model in the Korean dialogue interaction platform decreases accordingly. The curve can be used to quickly and accurately obtain the comprehensive performance representation value of the model of the Korean dialogue interaction platform, which solves the problem in the existing technology that the comprehensive performance representation value of the model of the Korean dialogue interaction platform cannot be accurately analyzed due to the insufficient analysis process, thereby realizing accurate analysis of the comprehensive performance representation value of the model of the Korean dialogue interaction platform.

[0112] It should be noted that in this embodiment, the weight factors corresponding to the preset model performance index values ​​and the weight factors corresponding to the user interaction quality index values ​​are obtained from the Korean dialogue tracking database, and the weight factors corresponding to the model performance index values ​​and the weight factors corresponding to the user interaction quality index values ​​are both numbers between 0 and 1.

[0113] It should be noted that the weight factor corresponding to the preset model performance indicator value obtained in the Korean conversation tracking database represents the numerical value of the influence of the model performance indicator value in the Korean conversation interaction platform on the model comprehensive performance representation value of the Korean conversation interaction platform, and the corresponding relationship is a pre-set mapping relationship. For example, the model performance indicator value in the Korean conversation interaction platform and the weight factor corresponding to the preset model performance indicator value in the Korean conversation tracking database form a mapping set, and the model performance indicator value in the real-time Korean conversation interaction platform is input into the mapping set to obtain the weight factor corresponding to the model performance indicator value in the Korean conversation interaction platform.

[0114] It should be noted that the weight factor corresponding to the preset user interaction quality index value obtained in the Korean conversation tracking database represents the numerical value of the influence of the user interaction quality index value in the Korean conversation interaction platform on the model comprehensive performance representation value of the Korean conversation interaction platform, and the corresponding relationship is a pre-set mapping relationship. For example, the user interaction quality index value in the Korean conversation interaction platform and the weight factor corresponding to the preset user interaction quality index value in the Korean conversation tracking database form a mapping set, and the real-time user interaction quality index value in the Korean conversation interaction platform is input into the mapping set to obtain the weight factor corresponding to the user interaction quality index value in the Korean conversation interaction platform.

[0115] It should be explained that the analysis method of this embodiment combines the model performance index value in the Korean dialogue interaction platform and the user interaction quality index value in the Korean dialogue interaction platform for analysis to obtain the model comprehensive performance representation value of the Korean dialogue interaction platform, which helps to accurately adjust the number of training sets, thereby enhancing the overall performance and reliability of the model.

[0116] Specifically, the model optimization index values ​​of the Korean dialogue interaction platform are analyzed under the following conditions:

[0117] ;

[0118] Where, represents the model optimization index value of the Korean dialogue interaction platform, Represents the comprehensive performance representation value of the model of the Korean dialogue interaction platform, Represents the model optimization correction scale coefficient of the Korean dialogue interaction platform.

[0119] It should be explained that the analysis method of this embodiment, by combining the model comprehensive performance representation value of the Korean dialogue interaction platform and the model optimization correction proportional coefficient of the Korean dialogue interaction platform for analysis, obtains the model optimization index value of the Korean dialogue interaction platform, which helps to improve the accuracy of the training set update, thereby improving the accuracy and responsiveness of the model.

[0120] It should be added that a training method for a Korean dialogue state tracking model also includes a Korean dialogue tracking database, which is used to store the influencing factors corresponding to Korean dialogue turns, the influencing factors corresponding to speech rate, the influencing factors corresponding to input length, the reference response time, the threshold corresponding to speech rate, the threshold corresponding to input length, the influencing factors corresponding to response time, the reference frequency, the reference duration, the reference repeated inquiry rate, the influencing factors corresponding to frequency, the influencing factors corresponding to duration, the influencing factors corresponding to repeated inquiry rate, the weight factors corresponding to model performance index values, and the weight factors corresponding to user interaction quality index values ​​obtained by analyzing historical data.

[0121] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0122] The preferred embodiments of the present invention disclosed above are intended only to help illustrate the present invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the present invention to the specific embodiments described. Obviously, numerous modifications and variations are possible based on the contents of this specification. These embodiments are selected and described in detail in this specification to better explain the principles and practical applications of the present invention, thereby enabling those skilled in the art to better understand and utilize the present invention.

Claims

1. A training method for a Korean dialogue state tracking model, characterized in that: include: The application platform of the Korean dialogue state tracking model is marked as a Korean dialogue interaction platform, and user dialogues in the Korean dialogue interaction platform are sensed within a preset tracking supervision cycle to obtain user dialogue interaction feature data in the Korean dialogue interaction platform within the tracking supervision cycle; According to the user dialogue interaction feature data in the Korean dialogue interaction platform during the tracking supervision period, the model optimization correction proportional coefficient of the Korean dialogue interaction platform is obtained through processing and transmitted to the integrated data processor; Statistically tracking the model performance data and user feedback data in the Korean conversational interaction platform during the supervision period, obtaining the model comprehensive performance representation value of the Korean conversational interaction platform through processing, and transmitting it to the integrated data processor; The integrated data processor receives the model optimization correction ratio coefficient and the model comprehensive performance representation value of the Korean dialogue interaction platform, and obtains the model training set of the Korean dialogue interaction platform through centralized processing and injects the specified number of updates; The Korean dialogue state tracking model is updated and injected with a specified number of training sets using the model training set to perform training.

2. The method for training a Korean dialogue state tracking model according to claim 1, characterized in that: The user dialogue interaction feature data in the Korean dialogue interaction platform during the tracking and supervision period includes the speech speed of each Korean dialogue of each user, the input length of each Korean dialogue of each user, and the Korean dialogue turns of each user.

3. The method for training a Korean dialogue state tracking model according to claim 1, characterized in that: The model optimization correction ratio coefficient of the Korean dialogue interaction platform is obtained, and the specific analysis process is as follows: In the tracking and monitoring cycle, the speech speed of each Korean conversation of each user and the input length of each Korean conversation of each user are counted; The Korean conversation turns of each user in the tracking supervision cycle are extracted, and the model optimization correction proportional coefficient of the Korean conversation interaction platform is obtained by comprehensive analysis based on the speech speed of each user's Korean conversation and the input length of each user's Korean conversation.

4. The method for training a Korean dialogue state tracking model according to claim 1, characterized in that: The statistical tracking of model performance data and user feedback data in the Korean dialogue interaction platform during the supervision period includes the statistical tracking of model performance data in the Korean dialogue interaction platform during the supervision period and the statistical tracking of user feedback data in the Korean dialogue interaction platform during the supervision period.

5. The method for training a Korean dialogue state tracking model according to claim 4, characterized in that: The statistical tracking of model performance data in the Korean dialogue interaction platform during the supervision period is as follows: The model performance data in the Korean dialogue interaction platform during the tracking supervision period includes the model generation response time in the Korean dialogue interaction platform, the user's Korean dialogue start time, the number of repeated inquiries of the same question by each user, and the total number of inquiries by each user; The difference between the model generation response time and the user's Korean conversation start time in the Korean conversation interaction platform during the tracking supervision cycle is recorded as the response time, and the average value is taken to obtain the average response time of the model in the Korean conversation interaction platform; The average of the ratio of the number of times each user repeatedly asks the same question to the total number of times each user asks the question is taken to obtain the average repeated inquiry rate of each Korean conversation of each user; According to the average response time of the model in the Korean conversation interaction platform, combined with the average repeated inquiry rate of each Korean conversation of each user, the model performance index value in the Korean conversation interaction platform is analyzed and obtained.

6. The method for training a Korean dialogue state tracking model according to claim 4, characterized in that: The statistical tracking of user feedback data in the Korean dialogue interaction platform during the supervision period is as follows: The user feedback data in the Korean dialogue interaction platform during the tracking supervision period includes the frequency of occurrence of negative feedback words in the feedback information of each user; The time interval from the start time of each user's Korean conversation to the end time of each user's Korean conversation during the monitoring cycle is statistically tracked to obtain the duration of each Korean conversation of each user; The frequency of negative feedback words in the feedback information of each user in the Korean dialogue interaction platform is extracted, and combined with the duration of each Korean dialogue of each user, the user interaction quality index value in the Korean dialogue interaction platform is analyzed.

7. The method for training a Korean dialogue state tracking model according to claim 1, characterized in that: The processing obtains the model comprehensive performance representation value of the Korean dialogue interaction platform, and the specific analysis process is: According to the model performance index values ​​in the Korean conversation interaction platform and the user interaction quality index values ​​in the Korean conversation interaction platform, a comprehensive analysis is performed to obtain the comprehensive performance representation value of the model of the Korean conversation interaction platform.

8. The method for training a Korean dialogue state tracking model according to claim 1, characterized in that: The model training set of the Korean dialogue interaction platform obtained through centralized processing is updated and injected into a specified number, and the specific process is: The integrated data processor receives the model optimization correction ratio coefficient and the model comprehensive performance representation value of the Korean dialogue interaction platform, analyzes and obtains the model optimization index value of the Korean dialogue interaction platform, and then matches and obtains the model training set update injection specified number of the Korean dialogue interaction platform.

9. The method for training a Korean dialogue state tracking model according to claim 7, characterized in that: The model comprehensive performance representation value of the Korean dialogue interaction platform, the specific analysis conditions are: ； In the formula, represents the comprehensive performance representation value of the model in the Korean dialogue interaction platform, represents the model performance index value in the Korean dialogue interaction platform, Indicates the weight factor corresponding to the set model performance index value, represents the user interaction quality index value in the Korean dialogue interaction platform, It represents the weight factor corresponding to the set user interaction quality index value, and e represents a natural constant.

10. The method for training a Korean dialogue state tracking model according to claim 1, characterized in that: The model optimization index value of the Korean dialogue interaction platform is analyzed under the following specific conditions: ； In the formula, represents the model optimization index value of the Korean dialogue interaction platform, represents the comprehensive performance representation value of the model in the Korean dialogue interaction platform, Represents the model optimization correction scale coefficient of the Korean dialogue interaction platform.

Images