A dialogue processing method, device, equipment and storage medium

Abstract

The application discloses a dialogue processing method and device, equipment and a storage medium, comprising: obtaining current question data and target role setting information of a role identifier; inputting the current question data and the target role setting information into a target language model of a current iteration cycle to obtain target reply data; the target language model is obtained by performing role dialogue training on a language model of a previous iteration cycle of the current iteration cycle based on a role dialogue sample set corresponding to each target abnormal type information in multiple target abnormal type information of the previous iteration cycle and a model updating mode corresponding to each target abnormal type information; the role dialogue sample set is obtained based on sub-abnormal distribution information of each target abnormal type information of the previous iteration cycle; and the sub-abnormal distribution information represents a historical dialogue data distribution of at least one sub-abnormal type information mapped by the target abnormal type information in the previous iteration cycle. The application can improve the role dialogue training effect of the language model.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, specifically to a dialogue processing method, apparatus, device, and storage medium. Background Technology

[0002] A dialogue system is a computer system that simulates humans and aims to form coherent and fluent conversations with them. With the development of artificial intelligence technology, dialogue systems have begun to use role-based dialogue models based on large language models. Role-based dialogue models can simulate corresponding characters to have conversations with users.

[0003] In related technologies, there is a lack of effective analysis of the business data generated by role-based dialogue models, making it difficult to effectively identify problems in the model's responses during actual online dialogues. This results in poor iterative training effects for role-based dialogue models, long iteration cycles, and difficulty in effectively improving the online role-based dialogue experience, thus reducing the user's role-based dialogue experience. Summary of the Invention

[0004] This application provides a dialogue processing method, apparatus, device, storage medium, and computer program product. By analyzing the distribution of historical dialogue data of different anomaly types in the previous iteration cycle, the targeting of the sample distribution in the role dialogue sample set of different anomaly types can be effectively improved. Based on the role dialogue sample set corresponding to each target anomaly type and the model update method corresponding to each target anomaly type, the language model of the previous iteration cycle is trained for role dialogue to obtain the language model of the current iteration cycle. At the same time, based on the sample accumulation speed of multiple target anomaly types in each iteration cycle, the cycle duration of each iteration cycle is adaptively adjusted. This can effectively improve the role dialogue training effect of the language model while improving the iteration speed of the language model, thereby improving the role dialogue experience of the target object.

[0005] Firstly, this application provides a dialogue processing method, the method comprising:

[0006] During the dialogue between the target object and the virtual interaction object, the current question data of the target object and the role identifier corresponding to the virtual interaction object are obtained;

[0007] Based on the current question data and the role identifier, the target role setting information is determined;

[0008] The current question data and the target role setting information are input into the target language model to predict the response and obtain the target response data;

[0009] Wherein, the target language model is the language model of the current iteration cycle associated with the role identifier; the language model of the current iteration cycle is obtained by training the language model of the previous iteration cycle with role dialogue sample sets corresponding to each of the multiple target anomaly types and the model update method corresponding to each of the multiple target anomaly types in the previous iteration cycle; the role dialogue sample set corresponding to each target anomaly type is obtained based on the sub-anomaly distribution information corresponding to each of the multiple target anomaly types in the previous iteration cycle, and the sub-anomaly distribution information is used to characterize the distribution of historical dialogue data of at least one sub-anomaly type corresponding to the target anomaly type in the previous iteration cycle; the cycle duration of the previous iteration cycle is adaptively adjusted based on the sample accumulation rate of the multiple target anomaly types.

[0010] Secondly, this application provides a dialogue processing apparatus, the apparatus comprising:

[0011] The current question data acquisition module is used to acquire the current question data of the target object and the role identifier corresponding to the virtual interaction object during the dialogue between the target object and the virtual interaction object.

[0012] The role setting information determination module is used to determine the target role setting information based on the current question data and the role identifier;

[0013] The response prediction module is used to input the current question data and the target role setting information into the target language model to predict the response and obtain the target response data.

[0014] Wherein, the target language model is the language model of the current iteration cycle associated with the role identifier; the language model of the current iteration cycle is obtained by training the language model of the previous iteration cycle with role dialogue sample sets corresponding to each of the multiple target anomaly types and the model update method corresponding to each of the multiple target anomaly types in the previous iteration cycle; the role dialogue sample set corresponding to each target anomaly type is obtained based on the sub-anomaly distribution information corresponding to each of the multiple target anomaly types in the previous iteration cycle, and the sub-anomaly distribution information is used to characterize the distribution of historical dialogue data of at least one sub-anomaly type corresponding to the target anomaly type in the previous iteration cycle; the cycle duration of the previous iteration cycle is adaptively adjusted based on the sample accumulation rate of the multiple target anomaly types.

[0015] Thirdly, this application provides a computer device including a processor and a memory, wherein the memory stores at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement a dialogue processing method as described in the first aspect.

[0016] Fourthly, this application provides a computer-readable storage medium storing at least one instruction or at least one program, which is loaded and executed by a processor to implement a dialogue processing method as described in the first aspect.

[0017] Fifthly, this application provides a computer program product comprising computer instructions that, when executed by a processor, implement a dialogue processing method as described in the first aspect.

[0018] The dialogue processing method, apparatus, device, storage medium, and computer program product provided in this application have the following technical effects:

[0019] The dialogue processing solution provided in this application acquires a preset number of historical dialogue data generated during online role-playing dialogues using the language model in the previous iteration cycle. The duration of the previous iteration cycle is adaptively adjusted based on the sample accumulation rate of various target anomaly types in the previous iteration cycle. By analyzing the distribution of historical dialogue data for different sub-anomaly types under each target anomaly type in the previous iteration cycle, the sub-anomaly distribution information corresponding to each target anomaly type is obtained. Based on the sub-anomaly distribution information corresponding to each target anomaly type, a role-playing dialogue sample set is generated, which can effectively improve the relevance of the sample distribution in the role-playing dialogue sample set. Then, targeted models are set for role-playing dialogue sample sets of different anomaly types. The system employs a model update method, and based on the role dialogue sample set corresponding to each type of target anomaly information and the model update method corresponding to each type of target anomaly information, it trains the language model of the previous iteration cycle for role dialogue to obtain the language model of the current iteration cycle. This can effectively improve the role dialogue training effect of the language model while increasing the iteration speed. The system associates the language model of the current iteration cycle with the role identifier. After obtaining the question data of the target object for that role identifier, it determines the target role setting information based on the question data and the role identifier. The language model outputs response data based on the question data and the target role setting information, which can improve the consistency and quality of the role setting of the model response, thereby improving the role dialogue experience of the target object.

[0020] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0021] To more clearly illustrate the technical solutions and advantages in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of the implementation environment of a dialogue processing method provided in an embodiment of this application;

[0023] Figure 2 This is a flowchart illustrating a dialogue processing method provided in an embodiment of this application;

[0024] Figure 3 This is a flowchart illustrating another dialogue processing method provided in an embodiment of this application;

[0025] Figure 4 This is a flowchart illustrating the language model training process in the current iteration cycle, as provided in an embodiment of this application.

[0026] Figure 5 This is a schematic diagram of a model iteration cycle provided in an embodiment of this application;

[0027] Figure 6 This is a flowchart illustrating a preference learning method provided in an embodiment of this application;

[0028] Figure 7 This is a flowchart illustrating the training process of an object preference detection model provided in an embodiment of this application;

[0029] Figure 8 This is a flowchart illustrating a knowledge base update method provided in an embodiment of this application;

[0030] Figure 9 This is a schematic diagram of a dialogue processing device provided in an embodiment of this application;

[0031] Figure 10 This is a schematic diagram of the hardware structure of a device for implementing a dialogue processing method provided in an embodiment of this application. Detailed Implementation

[0032] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout.

[0033] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0034] It is understood that in the specific implementation of this application, data such as user avatars are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0035] Before introducing the method embodiments provided in this application, a brief introduction will be given on the application scenarios, related terms or nouns that may be involved in the method embodiments of this application, so as to facilitate the understanding of those skilled in the art.

[0036] Machine Learning (ML) is an interdisciplinary field that involves probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance.

[0037] Large Language Model (LLM) is a computer model capable of processing and generating natural language. LLMs can predict the next word or sentence by learning the statistical patterns and semantic information of language data. As the input dataset and parameter space expand, the capabilities of LLMs also increase. It is used in various application areas such as machine learning, machine translation, speech recognition, and image processing, hence the name "multimodal large language model."

[0038] Dialogue system: also known as conversational agent, is a computer system that simulates human conversation, i.e., intelligent agent. It is designed to form coherent and fluent dialogues with humans. The main communication methods are voice / text / image, and of course, other methods such as gestures / touch can also be used.

[0039] Please see Figure 1 This is a schematic diagram illustrating the implementation environment of a dialogue processing method provided in an embodiment of this application, such as... Figure 1 As shown, the implementation environment may include at least client 01 and server 02.

[0040] Specifically, client 01 can include devices such as smartphones, desktop computers, tablets, laptops, in-vehicle terminals, digital assistants, smart wearable devices, and voice interaction devices. It can also include software running on the device, such as web pages provided to users by service providers, or applications provided by those service providers. Specifically, client 01 can be used to obtain the current question data input by the target object and the role identifier corresponding to the virtual interaction object during the dialogue between the target object and the virtual interaction object. Client 01 sends the current question data and role identifier to server 02 for dialogue processing on server 02.

[0041] Specifically, server 02 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. Server 02 may include network communication units, processors, and memory, etc. The terminal and the server can be connected directly or indirectly through wired or wireless communication, which is not limited herein. Specifically, server 02 can determine the target role setting information based on the current question data and role identifier, and input the current question data and target role setting information into the target language model running on server 02. This allows the target language model to predict responses based on the current question data and target role setting information, obtaining target response data. The target language model is the language model associated with the role identifier in the current iteration cycle. The language model in the current iteration cycle is obtained by training the language model of the previous iteration cycle with role dialogue sample sets corresponding to each type of target anomaly information and the model update method corresponding to each type of target anomaly information in the previous iteration cycle. The role dialogue sample sets corresponding to each type of target anomaly information are obtained based on the sub-anomaly distribution information corresponding to each type of target anomaly information in the previous iteration cycle. The sub-anomaly distribution information is used to characterize the distribution of historical dialogue data of at least one sub-anomaly type information mapped to the target anomaly type information in the previous iteration cycle. The duration of the previous iteration cycle is adaptively adjusted based on the sample accumulation rate of multiple target anomaly types.

[0042] In practical applications, besides executing the dialogue processing method provided in this application embodiment through the interaction between the client 01 and the server 02, the dialogue processing scheme provided in this application embodiment can be executed independently by the client 01 or independently by the server 02. Figure 1 This is just one example.

[0043] The following describes a dialogue processing method provided in this application. Figure 2This is a flowchart illustrating a dialogue processing method provided in an embodiment of this application. This application provides the operational steps of the method described in the embodiments or flowchart, but based on conventional or non-inventive methods, more or fewer operational steps may be included. The order of steps listed in the embodiments is merely one possible execution order among many and does not represent the only possible execution order. In actual system or server product execution, the method can be executed sequentially according to the embodiments or drawings, or in parallel (e.g., in a parallel processor or multi-threaded processing environment). Please refer to... Figure 2 The dialogue processing method provided in this application embodiment may include the following steps:

[0044] S201, During the dialogue between the target object and the virtual interaction object, obtain the target object's current question data and the role identifier corresponding to the virtual interaction object.

[0045] In the embodiments of this specification, the target object can refer to a real interactive object engaging in intelligent dialogue, such as a real user. This target object can input questions in the dialogue interface provided by the intelligent dialogue to obtain corresponding responses. A virtual interactive object can refer to a virtual person set up by the intelligent dialogue to converse with the target object, such as a chatbot.

[0046] In one specific embodiment, a virtual interactive object can correspond to a role identifier, which can be used to identify the role configured for the virtual interactive object, that is, the role currently played by the virtual interactive object. Indicatively, the role identifier may include, but is not limited to, role name, role number, etc.

[0047] S202, Based on the current question data and role identifier, determine the target role setting information.

[0048] In one specific embodiment, the character setting information can be descriptive information representing the character setting of a virtual interactive object. Specifically, the character setting here may include, but is not limited to: age, personality, language style, interests and preferences, personal achievements, and relationships with surrounding people, etc., which are not limited in this application. For example, the character setting information may include: My name is XXX, my favorite sport is XXX, I am good at..., my best friend is...

[0049] In one specific embodiment, the target role setting information can be the role setting information that is related to the current question data among multiple role setting information corresponding to the role identifier.

[0050] S203, Input the current question data and target role setting information into the target language model to predict the response and obtain the target response data;

[0051] The target language model is the language model associated with the role identifier in the current iteration cycle. The language model in the current iteration cycle is obtained by training the language model of the previous iteration cycle with role dialogues based on the role dialogue sample set corresponding to each of the multiple target anomaly types in the previous iteration cycle and the model update method corresponding to each of the multiple target anomaly types. The role dialogue sample set corresponding to each target anomaly type is obtained based on the sub-anomaly distribution information corresponding to each target anomaly type in the previous iteration cycle. The sub-anomaly distribution information is used to characterize the distribution of historical dialogue data of at least one sub-anomaly type that corresponds to the target anomaly type in the previous iteration cycle. The cycle duration of the previous iteration cycle is adaptively adjusted based on the sample accumulation rate of multiple target anomaly types.

[0052] In existing technologies, there is a lack of effective analysis of business data generated by role-based dialogue models, making it difficult to effectively identify problems in actual online dialogue processes. This results in poor iterative training effects of role-based dialogue models, long iteration cycles, and difficulty in effectively improving online role-based dialogue performance, thus reducing the user's role-based dialogue experience. Therefore, this application embodiment can adopt a data-driven approach to quickly improve and drive the upgrade of role-based dialogue models (i.e., language models with multi-turn dialogue capabilities that conform to the character's persona). Anomaly detection is performed on multiple historical dialogue data generated by the language model in the previous iteration cycle in online dialogue to obtain sub-anomaly type information corresponding to each historical dialogue data. Then, based on the preset mapping relationship information between different sub-anomaly type information and different target anomaly type information, the sub-anomaly distribution information corresponding to each target anomaly type information in the previous iteration cycle is determined. Based on the sub-anomaly distribution information corresponding to each target anomaly type information, a role-based dialogue sample set corresponding to each target anomaly type information is obtained. A targeted model update method is used to upgrade the model for the role-based dialogue sample set corresponding to each target anomaly type information to obtain the language model of the current iteration cycle. The language model of the current iteration cycle is then published online for anomaly detection and anomaly type distribution analysis of the next round of dialogue data.

[0053] In the embodiments of this specification, the target language model can be a dialogue model of a role that interacts with the target object. It should be noted that the language model in the embodiments of this specification can refer to a large language model using a generated Transformer architecture; this application does not limit this to that.

[0054] Optionally, a preset period duration can be used as the duration of each iteration cycle. This allows for iterative updates of the language model used for role-playing dialogue based on the preset period duration. For example, the preset period duration can be set to three weeks. Alternatively, the duration of each iteration cycle can be adaptively adjusted according to the sample accumulation rate of various target anomaly types within each iteration cycle. The language model can then be iteratively updated based on this duration. Generally, the faster the sample accumulation rate, the shorter the period duration. Illustratively, the sample accumulation rate of each target anomaly type can be measured by the accumulation time when the number of role-playing dialogue samples corresponding to each target anomaly type reaches a preset number. Specifically, this preset number can be pre-set based on the model iteration efficiency requirements in actual applications.

[0055] As can be seen from the above embodiments, a preset number of historical dialogue data points generated during online role-playing dialogues using the language model from the previous iteration cycle are obtained. The duration of the previous iteration cycle is adaptively adjusted based on the sample accumulation rate of various target anomaly types in the previous iteration cycle. By analyzing the distribution of historical dialogue data for different sub-anomaly types under each target anomaly type in the previous iteration cycle, the sub-anomaly distribution information corresponding to each target anomaly type is obtained. Based on the sub-anomaly distribution information corresponding to each target anomaly type, a role-playing dialogue sample set is generated. This effectively improves the specificity of the sample distribution in the role-playing dialogue sample set. Then, targeted models are set for role-playing dialogue sample sets with different anomaly types. The new approach trains the language model of the previous iteration cycle on role-based dialogue based on the role dialogue sample set corresponding to each type of target anomaly information and the model update method corresponding to each type of target anomaly information, thus obtaining the language model of the current iteration cycle. This can effectively improve the role-based dialogue training effect of the language model while improving the iteration speed of the language model. By associating the language model of the current iteration cycle with the role identifier, after obtaining the question data of the target object for the role identifier, the target role setting information is determined based on the question data and the role identifier. The language model outputs response data based on the question data and the target role setting information, which can improve the consistency and quality of the role setting of the model response, thereby improving the role-based dialogue experience of the target object.

[0056] In one specific embodiment, the target language model can predict responses to the current question data based on the target role setting information to obtain target response data, thereby ensuring the consistency between the target response data and the persona corresponding to the role identifier.

[0057] In a specific embodiment, such as Figure 3 As shown, the target role setting information determined based on the current question data and role identifier may include:

[0058] S2021: Extract multiple character setting information corresponding to the character identifier from the preset character setting knowledge base.

[0059] In one specific embodiment, a preset role setting knowledge base can be used to store multiple role setting information entries corresponding to each role identifier. Specifically, this can be achieved by collecting role setting source data, cleaning and integrating the source data, and obtaining multiple role setting information entries corresponding to each role identifier. Optionally, the role setting source data can be collected from external data sources. For example, external data sources may include, but are not limited to, website encyclopedia data.

[0060] S2022, determine the target role setting information associated with the current question data from multiple role setting information.

[0061] In a specific embodiment, feature extraction can be performed on the current question data to obtain the question feature information corresponding to the current question data. This question feature information is then matched with the setting feature information corresponding to each role setting information of the role identifier to obtain the feature matching degree for each role setting information. The target number of role setting information entries with the highest corresponding feature matching degree among the multiple role setting information entries are selected as the target role setting information. Specifically, this target number can be preset based on the feature matching accuracy requirements of the actual application.

[0062] In a specific embodiment, the question feature information corresponding to the current question data can be used to characterize the semantic features of the current question data, and the setting feature information corresponding to the role setting information can be used to characterize the semantic features of the role setting information. Specifically, the question feature information can be represented as a question feature vector, and the setting feature information can be represented as a setting feature vector. Correspondingly, the feature matching degree can include the feature similarity between the question feature vector and the setting feature vector. For example, cosine similarity can be used to characterize the feature similarity.

[0063] In one specific embodiment, multiple character setting information corresponding to each character setting information can be pre-generated, and then each character setting information can be pre-loaded into the vectorized retrieval component in the preset character setting knowledge base, and a corresponding index can be established. After extracting the question feature information corresponding to the current question data, the index established in the vectorized retrieval component can be used to perform a similarity search with the question feature information as the query vector, output the target number of character setting information with the highest feature similarity, and use the character setting information corresponding to the target number of character setting information as the target character setting information.

[0064] Accordingly, the above-mentioned input of the current question data and target role setting information into the target language model for response prediction can yield target response data that may include:

[0065] S2031, The target character setting information and the preset character setting instruction information are concatenated to obtain the target character setting instruction information.

[0066] Specifically, the preset role setting instruction information can be a pre-set instruction template information used to instruct on the role's character setting. For example, the preset role setting instruction information can be: "The description information of the digital human character who talks to the user is as follows {}. Please reply to the user's question according to the user's question and context?", where {} is a concatenation identifier, indicating that the target role setting information is concatenated here.

[0067] S2032, input the current question data and target role setting instruction information into the target language model to predict the response and obtain the target response data.

[0068] As can be seen from the above embodiments, by retrieving the target role setting information from the preset role setting knowledge base based on the current question data and role identifier, concatenating the current question data and the target role setting information to obtain the target role setting instruction information, and inputting it into the target language model for response prediction, the target response data can be obtained, which can effectively improve the consistency and accuracy of the persona in the response data.

[0069] In a specific embodiment, such as Figure 4 As shown, the language model for the current iteration cycle was trained in the following way:

[0070] S401: Obtain multiple historical dialogue data generated during the online role-playing dialogue of the language model in the previous iteration cycle.

[0071] Specifically, the aforementioned historical dialogue data can be multiple role dialogue data generated by the language model in the previous iteration cycle with different business objects after it went online. Specifically, the business objects here can refer to users who use role dialogue services or user accounts. In a specific embodiment, each historical dialogue data can include at least one round of role dialogue data.

[0072] In one specific embodiment, the aforementioned historical dialogue data is obtained in the following manner:

[0073] S4011: Obtain multiple raw dialogue data generated during online dialogue by the language model in the previous iteration cycle.

[0074] Specifically, each original dialogue data can be business data that has been anonymized and collected after the language model went live in the previous iteration cycle.

[0075] In practical applications, after acquiring multiple raw dialogue data sets, corpus distillation can be performed on these sets, followed by further filtering based on information content metrics. Specifically, corpus distillation can include steps such as noise removal and data cleaning. Noise removal refers to removing irrelevant information such as special characters and punctuation marks from the raw dialogue data, while data cleaning refers to removing sensitive content from the raw dialogue data to ensure the security and compliance of the dialogue data.

[0076] S4012 performs information content analysis on multiple raw dialogue data separately to obtain the information content index of each raw dialogue data.

[0077] In one specific embodiment, the information content index of each original dialogue data is used to characterize the amount of information contained in the corresponding original dialogue data. Illustratively, the information content index here may include, but is not limited to, information entropy, information gain, etc. Specifically, any information content analysis method in the prior art can be used to perform information content analysis on multiple original dialogue data, and this application does not limit this approach.

[0078] S4013: Select original dialogue data from multiple original dialogue data whose information content index is greater than the preset information content threshold, and use them as multiple historical dialogue data.

[0079] Specifically, the preset information threshold here can be pre-set based on the sample quality requirements and sample screening quantity requirements in actual applications.

[0080] In an optional embodiment, each original dialogue data and preset scoring rule indication information can be input into the dialogue text scoring model to score the text quality of each original dialogue data, obtaining a text quality score for each original dialogue data. Then, original dialogue data with text quality scores greater than a preset score threshold are selected from multiple original dialogue data as multiple historical dialogue data. Specifically, the preset scoring rule indication information is used to instruct scoring each original dialogue data from dimensions such as text misspellings, text fluency, and grammatical correctness. The dialogue text scoring model can be a pre-trained first large-scale language model, which is not limited in this application.

[0081] In an optional embodiment, after filtering multiple original dialogue data, the filtered original dialogue data can be deduplicated to ensure the diversity of sample data.

[0082] As can be seen from the above embodiments, by optimizing and simplifying multiple original dialogue data, and then performing dialogue anomaly analysis and model training based on the simplified original dialogue data, the training efficiency and performance of the role dialogue model can be improved.

[0083] S402, Perform dialogue anomaly detection on each historical dialogue data separately to obtain sub-anomaly type information corresponding to each historical dialogue data.

[0084] Specifically, the sub-anomaly type information corresponding to each historical dialogue data is used to represent the defect type of the model's response in the corresponding historical dialogue data. In a specific embodiment, the above-mentioned dialogue anomaly detection for each historical dialogue data to obtain the sub-anomaly type information corresponding to each historical dialogue data may include: inputting each historical dialogue data into a dialogue anomaly detection model for dialogue anomaly detection to obtain the sub-anomaly type information corresponding to each historical dialogue data. For example, a certain historical dialogue data is shown below: "User question: What is your relationship with character xx in game A? Model response: What? Jealous? Don't worry, I'm completely sincere towards you." Performing dialogue anomaly detection on this historical dialogue data yields the sub-anomaly type information "Does not conform to character settings."

[0085] In an optional embodiment, the aforementioned dialogue anomaly detection model can be a second large-scale language model trained on dialogue anomaly detection. Specifically, the second large-scale language model can be trained on dialogue anomaly detection using instruction fine-tuning. In a specific embodiment, anomaly detection indication information can be pre-set according to the actual dialogue anomaly detection task. This anomaly detection indication information is used to instruct dialogue anomaly detection on the input historical dialogue data. Specifically, the anomaly detection indication information can be presented as instruction description text. In an optional embodiment, the anomaly detection indication information can be used to indicate whether the input historical dialogue data contains any one of several dialogue anomaly situations. Illustratively, these multiple dialogue anomaly situations may include, but are not limited to: not conforming to role settings, context invalidation, not following user instructions, mechanical and rigid response content, confusion between subject and object, repetitive response content, revealing oneself as an AI or a large model, etc.

[0086] In another optional embodiment, the dialogue anomaly detection model can be a text classification model trained with supervised dialogue anomaly detection. Specifically, sample dialogue data can be manually labeled with dialogue defects, and one sub-anomaly type can be determined from multiple sub-anomaly type information as the labeled anomaly type information. Based on the sample dialogue data and the labeled anomaly type information, the text classification model to be trained is then trained for dialogue anomaly detection to obtain the dialogue anomaly detection model. For illustrative purposes, the text classification model to be trained can include, but is not limited to, artificial intelligence models such as FastText, TextCNN, and TextRNN; this application does not limit this. Optionally, before inputting historical dialogue data into the text classification model, a text semantic extraction model can be used to extract semantic features from the historical dialogue data to obtain dialogue semantic feature information. This dialogue semantic feature information is then input into the text classification model for dialogue anomaly classification to obtain the sub-anomaly type information corresponding to the historical dialogue data. For illustrative purposes, the text semantic extraction model here can be the BERT model.

[0087] S403, based on the preset mapping relationship information and the sub-anomaly type information corresponding to each historical dialogue data, determine the target anomaly type information to which each historical dialogue data belongs from multiple target anomaly type information. The preset mapping relationship information is used to characterize the mapping relationship between different sub-anomaly type information and different target anomaly type information. Each target anomaly type information has a mapping relationship with at least one sub-anomaly type information. The sub-anomaly type information mapped by different target anomaly type information is different.

[0088] In a specific embodiment, the target anomaly type information can be higher-level dialogue anomaly type information, and the sub-anomaly type information can be lower-level dialogue anomaly type information. Each target anomaly type information can have a mapping relationship with at least one sub-anomaly type information, and the sub-anomaly type information mapped to different target anomaly type information is different. Each target anomaly type information can play a role in summarizing and refining the at least one sub-anomaly type information mapped to itself. It can be understood that the at least one sub-anomaly type information mapped to each target anomaly type information can be at least one dialogue anomaly type information further subdivided under each target anomaly type information.

[0089] In a specific embodiment, preset mapping relationship information is used to characterize the mapping relationship between different sub-anomaly types and different target anomaly types, where the number of sub-anomaly types exceeds the number of target anomaly types. Specifically, the mapping relationship between different sub-anomaly types and different target anomaly types can be preset based on the actual needs of dialogue anomaly classification in practical applications. Illustratively, multiple sub-anomaly types may include: not conforming to role settings, context invalidation, not following user instructions, mechanical and rigid response content, confusion between subject and guest, repetitive response content, and revealing oneself as an AI or large model. Multiple target anomaly types may include: forgetting role settings, unnatural multi-sentence responses, and poor topic adherence. Correspondingly, a mapping relationship can be set between the sub-anomaly type "not conforming to role settings" and the target anomaly type "forgotten role settings," between the sub-anomaly types "mechanical and rigid response content," "confusion between subject and guest," "repetitive response content," and "revealing oneself as an AI or large model" and the target anomaly type "unnatural multi-sentence responses," and between the sub-anomaly types "context invalidation" and "not following user instructions" and the target anomaly type "poor topic adherence."

[0090] S404, Based on the historical dialogue data under each target anomaly type information, perform distribution analysis on at least one sub-anomaly type information mapped to each target anomaly type information, and determine the sub-anomaly distribution information corresponding to each target anomaly type information in the previous iteration cycle.

[0091] Specifically, the sub-anomaly distribution information corresponding to each type of target anomaly information in the previous iteration cycle can be used to characterize the distribution of historical dialogue data of at least one sub-anomaly type information mapped to the target anomaly type information in the previous iteration cycle.

[0092] In a specific embodiment, the above-mentioned distribution analysis of at least one sub-anomaly type information mapped to each target anomaly type information based on historical dialogue data under each target anomaly type information, to determine the sub-anomaly distribution information corresponding to each target anomaly type information in the previous iteration cycle, may include:

[0093] S4041, determine the number of dialogues in historical dialogue data under each sub-anomaly type information in at least one sub-anomaly type information mapped to each target anomaly type information;

[0094] S4042, based on the number of dialogues of each of the at least one sub-anomaly type information, perform a proportion analysis on each sub-anomaly type information to determine the dialogue proportion information corresponding to each sub-anomaly type information. The dialogue proportion information is used to characterize the ratio of the number of dialogues in the historical dialogue data under the corresponding sub-anomaly type information to the number of dialogues in the historical dialogue data under the target anomaly type information.

[0095] S4043, based on the dialogue proportion information of at least one sub-anomaly type information mapped by each target anomaly type information, determine the sub-anomaly distribution information corresponding to each target anomaly type information in the previous iteration cycle.

[0096] S405, based on the sub-anomaly distribution information, perform anomaly correction on the historical dialogue data corresponding to each target anomaly type information to obtain the role dialogue sample set corresponding to each target anomaly type information.

[0097] In a specific embodiment, the sub-anomaly distribution information corresponding to each target anomaly type information may include: the dialogue proportion information of at least one sub-anomaly type information mapped to each target anomaly type information. Accordingly, the total number of samples collected for the role dialogue sample set corresponding to the target anomaly type information can be predetermined, and the number of samples collected for each sub-anomaly type information can be determined based on the total number of samples collected and the dialogue proportion information of each sub-anomaly type information. The number of historical dialogue data collected for each sub-anomaly type information is selected from the historical dialogue data corresponding to each sub-anomaly type information, and the number of historical dialogue data collected for each sample is corrected to obtain the number of role dialogue samples collected for each sub-anomaly type information. Then, the role dialogue samples of each sub-anomaly type information are combined to obtain the role dialogue sample set corresponding to each target anomaly type information.

[0098] S406, based on the role dialogue sample set corresponding to each type of target anomaly information and the model update method corresponding to each type of target anomaly information, the language model of the previous iteration cycle is trained to obtain the language model of the current iteration cycle.

[0099] In a specific embodiment, the model update method corresponding to each target anomaly type can be pre-configured from multiple model update methods. Specifically, the configuration relationship between the target anomaly type and the model update method can be pre-set according to the role dialogue training requirements in actual applications. Illustratively, the multiple model update methods may include: knowledge base update method, model fine-tuning method, and preference learning method. The multiple target anomaly types may include: forgotten role settings, unnatural multi-sentence responses, and poor topic adherence. Correspondingly, the model update method corresponding to the target anomaly type "forgotten role settings" can be set to the knowledge base update method, the model update method corresponding to the target anomaly type "unnatural multi-sentence responses" can be set to the model fine-tuning method, and the model update method corresponding to the target anomaly type "poor topic adherence" can be set to the preference learning method.

[0100] In a specific embodiment, such as Figure 5 As shown, each iteration cycle can be divided into three stages: dialogue acquisition, sample training, and model validation. Specifically, the dialogue acquisition stage of the current iteration cycle is used to collect multiple anonymized historical dialogue data after the language model of the previous iteration cycle is launched, and to perform anomaly detection on the multiple historical dialogue data to obtain sub-anomaly type information for each historical dialogue data. The sample training stage of the current iteration cycle is used to perform clustering statistics on multiple historical dialogue data for various target anomaly types based on the sub-anomaly type information of each historical dialogue data to obtain the sub-anomaly distribution information corresponding to each target anomaly type information, thereby obtaining the role dialogue sample set corresponding to each target anomaly type information. Based on the role dialogue sample set corresponding to each target anomaly type information and the model update method corresponding to each target anomaly type information, the language model of the previous iteration cycle is trained to obtain the language model of the current iteration cycle. The model validation stage of the current iteration cycle is used to repair and validate the language model of the current iteration cycle, and to conduct comparative experiments on different model versions of the current iteration cycle online to specifically improve the dialogue effect of different role identifiers.

[0101] As can be seen from the above embodiments, constructing corresponding role training samples based on the distribution of historical dialogue data of each anomaly type in the previous iteration cycle in practical applications can improve the targeting of sample construction. Then, setting targeted model update methods for role dialogue sample sets of different anomaly types, and training the language model of the previous iteration cycle on role dialogue based on the role dialogue sample set corresponding to each target anomaly type information and the model update method corresponding to each target anomaly type information, to obtain the language model of the current iteration cycle, effectively improving the role dialogue training effect of the language model.

[0102] In a specific embodiment, "poor topic adherence" is set as the first anomaly type information, and the model update method corresponding to the first anomaly type information is set as the preference learning method. The role dialogue sample set corresponding to the first anomaly type information includes: first sample question data and multiple sample target response data corresponding to the first sample question data. Specifically, the multiple sample target response data corresponding to the first sample question data can be obtained in the following way:

[0103] S1: Obtain the first sample question data and multiple first sample response data corresponding to the first sample question data from the historical dialogue data corresponding to the first anomaly type information;

[0104] S2: Input the first sample question data, each first sample response data and the preset analysis prompt information into the dialogue correction model, perform question analysis on each first sample response data to obtain the question analysis data corresponding to each first sample response data, and use the preset analysis prompt information to indicate whether the analysis of each first sample response data conforms to the dialogue topic and role setting information of the dialogue.

[0105] S3: Input the first sample question data, each first sample response data and question analysis data into the dialogue correction model, correct the response for each first sample response data, and obtain the corrected response data corresponding to each first sample response data;

[0106] S4: Based on the corrected response data corresponding to multiple first sample response data, obtain multiple sample target response data.

[0107] Specifically, the dialogue correction model can be a third large-scale language model with the ability to reflect on sample dialogues. Through the dialogue correction model, problem analysis and response correction can be performed on sample response data.

[0108] In a specific embodiment, when the target anomaly type information is the first anomaly type information, such as Figure 6 As shown, the language model for the previous iteration cycle, trained based on the role dialogue sample set corresponding to each type of target anomaly information and the model update method corresponding to each type of target anomaly information, can be used to obtain the language model for the current iteration cycle, which may include:

[0109] S601, determine the target preference response data and at least two non-preference response data in the target response data of multiple samples, and determine the difference data corresponding to each non-preference response data. The difference data is used to characterize the difference in the degree of object preference between the corresponding non-preference response data and the target preference response data.

[0110] In one specific embodiment, the above-mentioned determination of target preference response data and at least two non-preference response data in multiple sample target response data, and determination of the difference data corresponding to each non-preference response data, may include:

[0111] S6011, Input the first sample question data and multiple sample target response data into the object tendency detection model to determine the object tendency detection data of each of the multiple sample target response data.

[0112] Specifically, the object preference detection data of the sample target response data can be used to predict the degree of preference of the model dialogue object (e.g., the model dialogue user) for the sample target response data, that is, the probability that the sample target response data is detected as preferred response data. In a specific embodiment, the object preference detection data can be represented as an object preference detection probability. The larger the corresponding value of the object preference detection data of the sample target response data, the higher the probability of it being detected as preferred response data.

[0113] S6012, based on object preference detection data, determine target preference response data and at least two non-preference response data.

[0114] In a specific embodiment, sample target response data whose corresponding object preference detection data meets a preset preference condition can be used as sample target response data, and sample target response data whose corresponding object preference detection data meets a preset non-preference condition can be used as non-preference response data. Specifically, when the object preference detection data is represented by object preference detection probability, the preset preference condition can be that the object preference detection probability is greater than a preset preference probability threshold, and the preset non-preference condition can be that the object preference detection probability is less than a preset non-preference probability threshold, wherein the preset preference probability threshold is greater than the preset non-preference probability threshold.

[0115] S6013, perform a difference analysis on the object tendency detection data of each non-preference response data and the object tendency detection data of the target preference response data to obtain the difference data corresponding to each non-preference response data.

[0116] Specifically, the difference data corresponding to each non-preference response data is used to characterize the difference in the degree of object preference between the corresponding non-preference response data and the target preference response data. In a specific embodiment, when the object preference detection data is expressed as the object preference detection probability, the difference between the object preference detection probability of each non-preference response data and the object preference detection probability of the target preference response data is used as the difference data corresponding to each non-preference response data.

[0117] As can be seen from the above embodiments, by inputting the first sample question data and multiple sample target response data into the object tendency detection model, determining the object tendency detection data of each of the multiple sample target response data, and determining the target preferred response data and at least two non-preferred response data based on the object tendency detection data, the accuracy of response classification can be improved.

[0118] In a specific embodiment, such as Figure 7 As shown, the above object preference detection model is trained in the following way:

[0119] S701, Obtain bias detection training data. The bias detection training data includes: fourth sample question data, multiple fourth sample response data corresponding to the fourth sample question data, and interactive feedback difference data corresponding to each fourth sample response data. The interactive feedback difference data is used to characterize the difference between the positive feedback statistical data and the negative feedback statistical data obtained from the corresponding fourth sample response data.

[0120] In a specific embodiment, the interactive feedback difference data corresponding to each fourth sample response data is used to characterize the difference between the positive feedback statistics and the negative feedback statistics obtained from the corresponding fourth sample response data. Specifically, the interactive feedback difference data corresponding to each fourth sample response data may include, but is not limited to: the difference between the positive feedback statistics and the negative feedback statistics, and the ratio between the positive feedback statistics and the negative feedback statistics.

[0121] In one specific embodiment, the positive feedback statistics obtained from each fourth sample response data can be statistical data on positive interaction information from different sampling objects (e.g., sampling users) to each fourth sample response data. Schematic, the positive feedback statistics may include, but are not limited to: the number of likes and the like rate. The negative feedback statistics obtained from each fourth sample response data can be statistical data on negative interaction information from different sampling objects to each fourth sample response data. Schematic, the negative feedback statistics may include, but are not limited to: the number of dislikes and the dislike rate.

[0122] In one specific embodiment, fourth sample question data, multiple fourth sample response data corresponding to the fourth sample question data, and interactive feedback difference data corresponding to each fourth sample response data can be collected from the historical dialogue data of the role dialogue system. Optionally, multiple fourth sample response data corresponding to different positions of the interactive feedback difference data in the difference space can be collected from the multiple sample response data corresponding to the fourth sample question data. Illustratively, interactive feedback difference data of a large number of sample dialogue data can be collected to determine the difference space, and sample response data corresponding to the interactive feedback difference data located in the top N% (e.g., top 10%) of the difference space, sample response data located in the middle of the difference space, and sample response data located in the tail M% (e.g., tail 20%) of the difference space can be extracted from the multiple sample response data corresponding to the fourth sample question data.

[0123] S702, based on the interaction feedback difference data of multiple fourth sample response data, determine the tendency labeling data of each of the multiple fourth sample response data.

[0124] In a specific embodiment, the range of values ​​for the output data of the tendency detection model to be trained can be predetermined, and the interaction feedback difference data of each fourth sample response data can be linearly mapped to the range of values ​​to obtain the tendency labeling data of each fourth sample response data. The interaction feedback difference data of each fourth sample response data is positively correlated with the tendency labeling data of each fourth sample response data.

[0125] In one specific embodiment, the preference labeling data of each fourth sample response data is used to represent the degree of preference of the model dialogue object (e.g., the model dialogue user) for each fourth sample response data, that is, the labeling probability of each fourth sample response data being a preferred response data.

[0126] S703, input the fourth sample question data and each fourth sample response data into the propensity detection model to be trained to predict the object's propensity, and obtain the propensity prediction data for each fourth sample response data.

[0127] In one specific embodiment, the tendency prediction data of each fourth sample response data is used to predict the degree of preference of the model dialogue object for each fourth sample response data, that is, the probability that each fourth sample response data is predicted as a preferred response data.

[0128] S704, Based on the tendency prediction data and the tendency labeling data of each fourth sample response data, determine the tendency classification loss information.

[0129] In one specific embodiment, the tendency classification loss information is used to characterize the difference between the tendency prediction data and the tendency labeling data of each fourth sample response data. Specifically, the smaller the difference between the tendency prediction data and the tendency labeling data of each fourth sample response data, the smaller the value of the tendency classification loss information; conversely, the larger the difference between the tendency prediction data and the tendency labeling data of each fourth sample response data, the larger the value of the tendency classification loss information. Specifically, the tendency classification loss information between the tendency prediction data and the tendency labeling data of each fourth sample response data can be calculated by combining a first preset loss function. Illustratively, the first preset loss function may include, but is not limited to, the cross-entropy loss function.

[0130] In one specific embodiment, a bias threshold can be preset, and the fourth sample response data whose corresponding bias prediction data is greater than the bias threshold can be regarded as positive samples (i.e., preferred response data), while the fourth sample response data whose corresponding bias prediction data is less than or equal to the bias threshold can be regarded as negative samples (i.e., non-biased response data), so that the bias detection model to be trained can learn to distinguish between preferred response data and non-biased response data.

[0131] S705, based on the index ranking information corresponding to the tendency labeling data of multiple fourth sample response data, performs ranking analysis on the tendency prediction data of multiple fourth sample response data to determine the preference ranking loss information.

[0132] In one specific embodiment, preference ranking loss information is used to characterize the ranking difference between the predicted tendency data and the labeled tendency data of multiple fourth sample response data. Specifically, the smaller the difference between the indicator ranking relationship of the predicted tendency data and the indicator ranking relationship of the labeled tendency data of multiple fourth sample response data, the smaller the value of the preference ranking loss information; conversely, the greater the difference between the indicator ranking relationship of the predicted tendency data and the indicator ranking relationship of the labeled tendency data of multiple fourth sample response data, the larger the value of the preference ranking loss information. Specifically, the preference ranking loss information between the indicator ranking relationship of the predicted tendency data and the indicator ranking relationship of the labeled tendency data of multiple fourth sample response data can be calculated by combining a second preset loss function, which can be set according to actual applications.

[0133] In a specific embodiment, when the number of multiple fourth sample response data is greater than 2, the multiple fourth sample response data can be combined in pairs to obtain multiple sample response groups. According to the size of the preset tendency labeling data, the two fourth sample response data in each sample response group are sorted to obtain the first indicator ranking information of each sample response group. And according to the size of the tendency prediction data, the two fourth sample response data in each sample response group are sorted to obtain the second indicator ranking information of each sample response group. Based on the second preset loss function, the ranking loss information between the first indicator ranking information and the second indicator ranking information of each sample response group is determined. Then, the ranking loss information of multiple sample response groups is fused to obtain the preference ranking loss information.

[0134] In an optional embodiment, the ranking loss information between the first indicator ranking information and the second indicator ranking information of each sample response group, based on the second preset loss function, can be expressed as the following formula:

[0135] L Rank =max(0,1-(P) a -P b ))

[0136] Among them, P a P represents the predicted propensity of the fourth sample response data with better propensity labeling among the two fourth sample response data in each sample response group. b This represents the predicted propensity data of the fourth sample response data that corresponds to the one with the worse propensity labeling data among the two fourth sample response data.

[0137] S706, based on the bias classification loss information and the preference ranking loss information, train the bias detection model to be trained to obtain the object bias detection model.

[0138] In a specific embodiment, training the propensity detection model to be trained based on propensity classification loss information and preference ranking loss information to obtain the object propensity detection model may include: updating the model parameters of the propensity detection model to be trained based on propensity classification loss information and preference ranking loss information; performing the next round of iterative training based on the updated propensity detection model to be trained until a preset convergence condition is met, and using the propensity detection model to be trained when the preset convergence condition is met as the object propensity detection model.

[0139] In one specific embodiment, the tendency detection model to be trained can be any existing artificial intelligence model with tendency classification capabilities, and this application does not impose any particular limitation on it.

[0140] In a specific embodiment, the preset convergence conditions can be set according to the actual application, such as the number of iterations of training reaching a preset number, the bias classification loss information and / or bias ranking loss information being less than a specified threshold, etc., which can be set according to the training speed and model accuracy requirements.

[0141] As can be seen from the above embodiments, the bias detection model is trained based on bias classification loss information and bias ranking loss information. This allows the bias detection model to learn to distinguish between biased and non-biased response data while ensuring that the bias prediction data output by the model has strict ranking. This ensures that the bias detection data output by the model can be used for comparison during the inference stage, thereby improving the effect of subsequent language model learning of response preferences.

[0142] S602, input the first sample question data, the target preference response data, and each non-preference response data into the language model of the previous iteration cycle to learn response preferences and obtain the preference loss information corresponding to each non-preference response data.

[0143] It should be noted that response preference learning is based on the direct preference feedback of the model dialogue object (e.g., the model dialogue user) to different responses to train and adjust the model parameters of the language model. It aims to improve the consistency between the language model and the user's response preferences, so that the responses generated by the language model are more in line with the user's expected topics and avoid topic drift.

[0144] Specifically, a first sample question data, the target preference response data corresponding to the first sample question data, and a non-preference response data corresponding to the first sample question data can be used as a sample data set. The language model can directly optimize its understanding of user response preferences by comparing the target preference response data and the non-preference response data, thereby improving the quality of the model's response. In a specific embodiment, maximum likelihood estimation can be performed on the object tendency detection data corresponding to the target preference response data and the object tendency detection data corresponding to the non-preference response data in a sample data set to obtain reference preference distribution information between the target preference response data and the non-preference response data.

[0145] In a specific embodiment, the above-mentioned inputting the first sample question data, the target preference response data, and each non-preference response data into the language model of the previous iteration cycle for response preference learning, to obtain the preference loss information corresponding to each non-preference response data, may include:

[0146] S6021, Input the first sample question data and target preference response data into the language model of the previous iteration cycle to obtain the predicted preference probability corresponding to the target preference response data;

[0147] S6022, Input the first sample question data and each non-preference response data into the language model of the previous iteration cycle to obtain the predicted preference probability corresponding to each non-preference response data;

[0148] S6023, Based on the predicted preference probability corresponding to the target preference response data and the predicted preference probability corresponding to each non-preference response data, determine the predicted preference distribution information between the target preference response data and each non-preference response data;

[0149] S6024, determine the preference loss information between the reference preference distribution information and the predicted preference distribution information.

[0150] In a specific embodiment, the preference loss information corresponding to each non-preference response data can be used to characterize the difference between the reference preference distribution between the target preference response data and each non-preference response data and the predicted preference distribution between the target preference response data and each non-preference response data.

[0151] In one specific embodiment, the DPO loss function can be used to determine the preference loss information between the reference preference distribution and the predicted preference distribution.

[0152] S603, loss adjustment is performed based on the difference data of at least two non-preference response data, and the loss adjustment weights corresponding to each of the at least two non-preference response data are determined.

[0153] In a specific embodiment, the loss adjustment weight corresponding to each non-preference response data is used to adjust the contribution of the preference loss information of the corresponding non-preference response data in model training. Specifically, a preset normalization function can be used to normalize the difference data of at least two non-preference response data. The preset normalization function can be set according to the actual application.

[0154] In an optional embodiment, the preset normalization function can be a normalization exponential function. Optionally, during the normalization process of the difference data of at least two non-preference response data based on the normalization exponential function, a preset temperature coefficient can be used to control the sensitivity of weight adjustment. The larger the preset temperature coefficient, the lower the sensitivity of weight adjustment. Specifically, the preset temperature coefficient can be set according to the weight adjustment requirements in the actual application.

[0155] S604, based on the loss adjustment weight, the preference loss information corresponding to at least two non-preference response data is weighted to obtain the target preference loss information.

[0156] In one specific embodiment, the target preference loss information can reflect the learning effect of the language model on the dialogue response preference. The target preference loss information is the loss information obtained by weighted fusion of the preference loss information corresponding to at least two non-preference response data based on the loss adjustment weight.

[0157] S605, based on the target preference loss information, update the language model of the previous iteration cycle to obtain the language model of the current iteration cycle.

[0158] As can be seen from the above embodiments, loss adjustment is performed based on the difference data of at least two non-preference response data, and loss adjustment weights are determined for each of the at least two non-preference response data. Based on the loss adjustment weights, the preference loss information corresponding to each of the at least two non-preference response data is weighted to obtain the target preference loss information. This can effectively adjust the contribution of preference loss information with different degrees of non-preference to model training. Thus, based on the target preference loss information, the language model of the previous iteration cycle is updated to obtain the language model of the current iteration cycle. This can effectively improve the language model's performance in learning response preferences, making the dialogue responses generated by the language model more in line with the user's expected topics and avoiding topic drift.

[0159] In one specific embodiment, "role setting forgetting" is set as the second anomaly type information, and the model update method corresponding to the second anomaly type information is set as the knowledge base update method, such as... Figure 8 As shown, the above method may further include:

[0160] S801, extract role keywords from the question data in the historical dialogue data corresponding to the second abnormality type information, and determine the role identifier to be updated and the setting identifier to be updated.

[0161] For illustrative purposes, the settings to be updated here may include, but are not limited to: age, personality, hobbies, etc.

[0162] S802 collects character setting source data based on the character identifier to be updated and the setting identifier to be updated.

[0163] Specifically, you can collect character setting source data related to the character identifier to be updated and character setting source data related to the character identifier to be updated from external data sources. For example, external data sources may include, but are not limited to, website encyclopedia data.

[0164] S803 updates the preset character setting knowledge base based on the character setting source data.

[0165] Specifically, by cleaning and integrating the source data of character settings, multiple character setting information corresponding to the character identifier to be updated can be obtained, setting feature information corresponding to each of the multiple character setting information can be generated, and then each setting feature information can be updated to the vectorized retrieval component in the preset character setting knowledge base, and the corresponding index can be established.

[0166] As can be seen from the above embodiments, by extracting role keywords from the question data in the historical dialogue data corresponding to the second abnormal type information, determining the role identifier to be updated and the setting identifier to be updated, and updating the preset role setting knowledge base based on the role setting source data corresponding to the role identifier to be updated and the setting identifier to be updated, the targeting of the knowledge base update can be improved, thereby effectively alleviating the problem of "role setting forgetting" in the language model.

[0167] In a specific embodiment, "unnatural multi-sentence responses" is set as the third anomaly type information, and the model update method corresponding to the third anomaly type information is set as the model fine-tuning method. The role dialogue sample set corresponding to the third anomaly type information may include: third sample question data and labeled response data corresponding to the third sample question data. Specifically, the labeled response data corresponding to the third sample question data can be obtained as follows: obtain the third sample question data and the third sample response data corresponding to the third sample question data from the historical dialogue data corresponding to the third anomaly type information, and perform correction processing on the third sample response data to obtain the labeled response data corresponding to the third sample question data. Correspondingly, the above-mentioned model training of the language model of the previous iteration cycle based on the role dialogue sample set corresponding to each target anomaly type information and the model update method corresponding to each target anomaly type information to obtain the language model of the current iteration cycle may include:

[0168] Based on the third sample question data and the corresponding labeled response data, the language model of the previous iteration cycle is fine-tuned to obtain the language model of the current iteration cycle.

[0169] Specifically, the model fine-tuning method here can be QLoRA fine-tuning. QLoRA is an efficient fine-tuning method that greatly reduces the memory required for fine-tuning while maintaining performance comparable to standard fine-tuning.

[0170] As can be seen from the above embodiments, by fine-tuning the language model using the third sample question data and the labeled response data corresponding to the third sample question data, training efficiency can be improved while ensuring the model's learning effect on the corrected natural responses, thereby improving the quality of the model's responses.

[0171] In one specific embodiment, the language model for the current cycle may include: multiple sub-language models, and the above method may further include:

[0172] S1: Conduct online grouped comparative tests on multiple sub-language models for role identification to obtain test feedback data.

[0173] Specifically, the multiple sub-language models here can be different versions of language models, or language models using different base models. The test feedback data here can be used to characterize the retention of different sub-language models with respect to the model's dialogue object. Illustratively, the test feedback data may include, but is not limited to: the number of dialogue turns with the model's dialogue object, the average dialogue time, etc.

[0174] S2: Based on test feedback data, determine the target language model corresponding to the role identifier from multiple sub-language models.

[0175] Specifically, the sub-language model whose corresponding test feedback data meets the preset model selection criteria is selected as the target language model corresponding to the role identifier. These preset model selection criteria can be pre-set based on the model selection needs in actual applications. For example, the preset model selection criteria could be set to the highest number of dialogue turns or the longest average dialogue time.

[0176] As can be seen from the above embodiments, by utilizing the technical solution provided in this application, a preset number of historical dialogue data generated during online role-playing dialogue using the language model in the previous iteration cycle are obtained. The duration of the previous iteration cycle is adaptively adjusted based on the sample accumulation rate of various target anomaly types in the previous iteration cycle. By analyzing the distribution of historical dialogue data for different sub-anomaly types under each target anomaly type in the previous iteration cycle, sub-anomaly distribution information corresponding to each target anomaly type is obtained. Based on the sub-anomaly distribution information corresponding to each target anomaly type, a role-playing dialogue sample set is generated. This effectively improves the targeting of the sample distribution in the role-playing dialogue sample set. Then, targeted methods are set for role-playing dialogue sample sets of different anomaly types. The model update method is tailored to specific needs. Based on the role dialogue sample set corresponding to each type of target anomaly information and the corresponding model update method, the language model of the previous iteration cycle is trained for role dialogue to obtain the language model of the current iteration cycle. This can effectively improve the role dialogue training effect of the language model while increasing the iteration speed. The language model of the current iteration cycle is associated with the role identifier. After obtaining the question data of the target object for the role identifier, the target role setting information is determined based on the question data and the role identifier. The language model outputs response data based on the question data and the target role setting information, which can improve the consistency and quality of the role setting of the model response, thereby improving the role dialogue experience of the target object.

[0177] This application also provides a dialogue processing device, such as... Figure 9 As shown, the device may include:

[0178] The current question data acquisition module 910 is used to acquire the current question data of the target object and the role identifier corresponding to the virtual interaction object during the dialogue between the target object and the virtual interaction object.

[0179] The role setting information determination module 920 is used to determine the target role setting information based on the current question data and role identifier;

[0180] The response prediction module 930 is used to input the current question data and target role setting information into the target language model to predict the response and obtain the target response data.

[0181] The target language model is the language model associated with the role identifier in the current iteration cycle. The language model in the current iteration cycle is obtained by training the language model of the previous iteration cycle with role dialogues based on the role dialogue sample set corresponding to each of the multiple target anomaly types in the previous iteration cycle and the model update method corresponding to each of the multiple target anomaly types. The role dialogue sample set corresponding to each target anomaly type is obtained based on the sub-anomaly distribution information corresponding to each target anomaly type in the previous iteration cycle. The sub-anomaly distribution information is used to characterize the distribution of historical dialogue data of at least one sub-anomaly type that corresponds to the target anomaly type in the previous iteration cycle. The cycle duration of the previous iteration cycle is adaptively adjusted based on the sample accumulation rate of multiple target anomaly types.

[0182] In one specific embodiment, the role setting information determination module 920 may include:

[0183] The information extraction unit is used to extract multiple pieces of character setting information corresponding to the character identifier from a preset character setting knowledge base;

[0184] The associated information determination unit is used to determine the target role setting information associated with the current question data from multiple role setting information;

[0185] The above-mentioned response prediction module 930 may include:

[0186] The instruction information splicing unit is used to splice the target role setting information and the preset role setting instruction information to obtain the target role setting instruction information;

[0187] The response prediction unit is used to input the current question data and the target role setting instruction information into the target language model to predict the response and obtain the target response data.

[0188] In a specific embodiment, the language model for the current iteration cycle described above is trained using the following method:

[0189] The historical dialogue data acquisition module is used to acquire multiple historical dialogue data generated when the language model conducted online role-playing dialogues in the previous iteration cycle.

[0190] The dialogue anomaly detection module is used to perform dialogue anomaly detection on each historical dialogue data separately and obtain the sub-anomaly type information corresponding to each historical dialogue data.

[0191] The type mapping module is used to determine the target anomaly type information to which each historical dialogue data belongs from multiple target anomaly type information based on preset mapping relationship information and sub-anomaly type information corresponding to each historical dialogue data. The preset mapping relationship information is used to characterize the mapping relationship between different sub-anomaly type information and different target anomaly type information. Each target anomaly type information has a mapping relationship with at least one sub-anomaly type information. The sub-anomaly type information mapped to different target anomaly type information is different.

[0192] The anomaly distribution analysis module is used to perform distribution analysis on at least one sub-anomaly type information mapped to each target anomaly type information based on historical dialogue data under each target anomaly type information, and to determine the sub-anomaly distribution information corresponding to each target anomaly type information in the previous iteration cycle.

[0193] The anomaly correction module is used to correct the historical dialogue data corresponding to each type of target anomaly information based on the sub-anomaly distribution information, so as to obtain the role dialogue sample set corresponding to each type of target anomaly information.

[0194] The model training module is used to train the language model of the previous iteration cycle based on the role dialogue sample set corresponding to each type of target anomaly information and the model update method corresponding to each type of target anomaly information, so as to obtain the language model of the current iteration cycle.

[0195] In one specific embodiment, the above-mentioned historical dialogue data acquisition module may include:

[0196] The raw dialogue data acquisition unit is used to acquire multiple raw dialogue data generated by the language model during online dialogue in the previous iteration cycle.

[0197] The information content analysis unit is used to perform information content analysis on multiple raw dialogue data separately to obtain the information content index of each raw dialogue data.

[0198] The data filtering unit is used to filter out original dialogue data from multiple original dialogue data that have an information content index greater than a preset information content threshold, and use them as multiple historical dialogue data.

[0199] In a specific embodiment, the model update method corresponding to the first anomaly type information among multiple target anomaly type information is a preference learning method. The role dialogue sample set corresponding to the first anomaly type information includes: first sample question data and multiple sample target response data corresponding to the first sample question data. The above-mentioned model training module may include:

[0200] The response preference analysis unit is used to determine the target preference response data and at least two non-preference response data in multiple sample target response data, and to determine the difference data corresponding to each non-preference response data. The difference data is used to characterize the difference in the degree of object preference between the corresponding non-preference response data and the target preference response data.

[0201] The response preference learning unit is used to input the first sample question data, the target preference response data, and each non-preference response data into the language model of the previous iteration to learn response preferences and obtain the preference loss information corresponding to each non-preference response data.

[0202] The loss adjustment weight determination unit is used to perform loss adjustment based on the difference data of at least two non-preference response data and to determine the loss adjustment weight corresponding to each of the at least two non-preference response data.

[0203] The loss weighting unit is used to weight the preference loss information corresponding to at least two non-preference response data based on the loss adjustment weight to obtain the target preference loss information.

[0204] The first model update unit is used to update the language model of the previous iteration based on the target preference loss information to obtain the language model of the current iteration.

[0205] In one specific embodiment, the above-mentioned response preference analysis unit may include:

[0206] The object bias detection unit is used to input the first sample question data and multiple sample target response data into the object bias detection model to determine the object bias detection data of each of the multiple sample target response data.

[0207] The target preference response data determination unit is used to determine target preference response data and at least two non-preference response data based on object preference detection data.

[0208] The difference analysis unit is used to perform difference analysis on the object tendency detection data of each non-preference response data and the object tendency detection data of the target preference response data, and obtain the difference data corresponding to each non-preference response data.

[0209] In one specific embodiment, the above-mentioned object preference detection model is trained using the following method:

[0210] The bias detection training data acquisition module is used to acquire bias detection training data, which includes: fourth sample question data, multiple fourth sample response data corresponding to the fourth sample question data, and interactive feedback difference data corresponding to each fourth sample response data. The interactive feedback difference data is used to characterize the difference between the positive feedback statistical data and the negative feedback statistical data obtained from the corresponding fourth sample response data.

[0211] The data annotation module is used to determine the tendency annotation data of each of the multiple fourth sample response data based on the interaction feedback difference data of each of the multiple fourth sample response data.

[0212] The object tendency prediction module is used to input the fourth sample question data and each fourth sample response data into the tendency detection model to be trained to predict the object tendency, and obtain the tendency prediction data of each fourth sample response data.

[0213] The propensity classification loss determination module is used to determine propensity classification loss information based on the propensity prediction data and the propensity labeling data of each fourth sample response data.

[0214] The preference ranking loss determination module is used to perform ranking analysis on the preference prediction data of multiple fourth sample response data based on the index ranking information corresponding to the preference labeling data of multiple fourth sample response data, and determine the preference ranking loss information.

[0215] The tendency detection training module is used to train the tendency detection model to be trained based on tendency classification loss information and preference ranking loss information, so as to obtain the object tendency detection model.

[0216] In one specific embodiment, the model update method corresponding to the second anomaly type information among multiple target anomaly type information is a knowledge base update method, and the above-mentioned device may further include:

[0217] The role keyword extraction module is used to extract role keywords from the question data in the historical dialogue data corresponding to the second abnormality type information, and to determine the role identifier to be updated and the setting identifier to be updated.

[0218] The character setting source data collection module is used to collect character setting source data based on the character identifier to be updated and the setting identifier to be updated;

[0219] The knowledge base update module is used to update the preset character setting knowledge base based on the character setting source data.

[0220] In a specific embodiment, the model update method corresponding to the third anomaly type information among multiple target anomaly types is a model fine-tuning method. The role dialogue sample set corresponding to the third anomaly type information includes: third sample question data and labeled response data corresponding to the third sample question data. The above-mentioned model training module may include:

[0221] The second model update unit is used to fine-tune the language model of the previous iteration based on the third sample question data and the labeled response data corresponding to the third sample question data, so as to obtain the language model of the current iteration.

[0222] It should be noted that the apparatus provided in the above embodiments is only illustrated by the division of the above functional modules when implementing its functions. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0223] This application provides a computer device including a processor and a memory. The memory stores at least one instruction or at least one program, which is loaded and executed by the processor to implement a dialogue processing method as provided in the above method embodiments.

[0224] Figure 10 A schematic diagram of the hardware structure of a device for implementing a dialogue processing method provided in an embodiment of this application is shown. This device may participate in or include the apparatus or system provided in the embodiment of this application. Figure 10 As shown, device 100 may include one or more processors 1002 (shown as 1002a, 1002b, ..., 1002n in the figure) (processor 1002 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 1004 for storing data, and a transmission device 1006 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 10 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, device 100 may also include a... Figure 10 The more or fewer components shown, or having the same Figure 10 The different configurations shown.

[0225] It should be noted that the aforementioned one or more processors 1002 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be wholly or partially integrated into any other element within device 100 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0226] The memory 1004 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the method described in the embodiments of this application. The processor 1002 executes various functional applications and data processing by running the software programs and modules stored in the memory 1004, thereby realizing the above-mentioned dialogue processing method. The memory 1004 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 1004 may further include memory remotely located relative to the processor 1002, and these remote memories can be connected to the device 100 via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0227] The transmission device 1006 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of device 100. In one example, the transmission device 1006 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 1006 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0228] The display may be, for example, a touchscreen liquid crystal display (LCD) that allows a user to interact with the user interface of device 100 (or a mobile device).

[0229] This application also provides a computer-readable storage medium, which can be disposed in a server to store at least one instruction or at least one program related to implementing a dialogue processing method in the method embodiment. The at least one instruction or at least one program is loaded and executed by the processor to implement the dialogue processing method provided in the above method embodiment.

[0230] Optionally, in this embodiment, the storage medium may be located at at least one of the multiple network servers in a computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0231] This invention also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform a dialogue processing method provided in the various optional embodiments described above.

[0232] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, the above description focuses on specific embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired results. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing are also possible or may be advantageous.

[0233] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device, equipment, and storage medium embodiments are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0234] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0235] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A dialogue processing method, characterized in that, The method includes: During the dialogue between the target object and the virtual interaction object, the current question data of the target object and the role identifier corresponding to the virtual interaction object are obtained; Based on the current question data and the role identifier, the target role setting information is determined; The current question data and the target role setting information are input into the target language model to predict the response and obtain the target response data; Wherein, the target language model is the language model of the current iteration cycle associated with the role identifier; the language model of the current iteration cycle is obtained by training the language model of the previous iteration cycle with role dialogue sample sets corresponding to each of the multiple target anomaly types and the model update method corresponding to each of the multiple target anomaly types in the previous iteration cycle; the role dialogue sample set corresponding to each target anomaly type is obtained based on the sub-anomaly distribution information corresponding to each of the multiple target anomaly types in the previous iteration cycle, and the sub-anomaly distribution information is used to characterize the distribution of historical dialogue data of at least one sub-anomaly type corresponding to the target anomaly type in the previous iteration cycle; the cycle duration of the previous iteration cycle is adaptively adjusted based on the sample accumulation rate of the multiple target anomaly types.

2. The method according to claim 1, characterized in that, The method further includes: Acquire multiple historical dialogue data generated during online role-playing dialogue using the language model from the previous iteration cycle; Perform dialogue anomaly detection on each historical dialogue data to obtain sub-anomaly type information corresponding to each historical dialogue data; Based on the preset mapping relationship information and the sub-anomaly type information corresponding to each historical dialogue data, the target anomaly type information to which each historical dialogue data belongs is determined from multiple target anomaly type information. The preset mapping relationship information is used to characterize the mapping relationship between different sub-anomaly type information and different target anomaly type information. Each target anomaly type information has a mapping relationship with at least one sub-anomaly type information. Different target anomaly type information maps to different sub-anomaly type information. Based on the historical dialogue data under each target anomaly type information, a distribution analysis is performed on at least one sub-anomaly type information mapped by each target anomaly type information to determine the sub-anomaly distribution information corresponding to each target anomaly type information in the previous iteration cycle. Based on the sub-anomaly distribution information, anomaly correction is performed on the historical dialogue data corresponding to each target anomaly type information to obtain the role dialogue sample set corresponding to each target anomaly type information. Based on the role dialogue sample set corresponding to each type of target anomaly information and the model update method corresponding to each type of target anomaly information, the language model of the previous iteration cycle is trained to obtain the language model of the current iteration cycle.

3. The method according to claim 2, characterized in that, The model update method corresponding to the first anomaly type information among the multiple target anomaly types is a preference learning method. The role dialogue sample set corresponding to the first anomaly type information includes: first sample question data and multiple sample target response data corresponding to the first sample question data. The step of training the language model of the previous iteration cycle based on the role dialogue sample set corresponding to each target anomaly type information and the model update method corresponding to each target anomaly type information to obtain the language model of the current iteration cycle includes: The target preference response data and at least two non-preference response data are determined from the plurality of sample target response data, and the difference data corresponding to each non-preference response data is determined. The difference data is used to characterize the difference in the degree of object preference between the corresponding non-preference response data and the target preference response data. The first sample question data, the target preference response data, and each non-preference response data are input into the language model of the previous iteration cycle to learn response preferences, thereby obtaining preference loss information corresponding to each non-preference response data. Loss adjustment is performed based on the difference data of each of the at least two non-preference response data, and the loss adjustment weight corresponding to each of the at least two non-preference response data is determined. Based on the loss adjustment weight, the preference loss information corresponding to each of the at least two non-preference response data is weighted to obtain the target preference loss information; Based on the target preference loss information, the language model of the previous iteration cycle is updated to obtain the language model of the current iteration cycle.

4. The method according to claim 3, characterized in that, The step of determining the target preferred response data and at least two non-preferred response data in the plurality of sample target response data, and determining the difference data corresponding to each non-preferred response data, includes: Input the first sample question data and the multiple sample target response data into the object tendency detection model to determine the object tendency detection data of each of the multiple sample target response data; Based on the object preference detection data, the target preference response data and the at least two non-preference response data are determined; A difference analysis is performed on the object tendency detection data of each non-preference response data and the object tendency detection data of the target preference response data to obtain the difference data corresponding to each non-preference response data.

5. The method according to claim 4, characterized in that, The object preference detection model was trained in the following manner: Acquire propensity detection training data, which includes: fourth sample question data, multiple fourth sample response data corresponding to the fourth sample question data, and interactive feedback difference data corresponding to each fourth sample response data. The interactive feedback difference data is used to characterize the difference between the positive feedback statistical data and the negative feedback statistical data obtained from the corresponding fourth sample response data. Based on the interaction feedback difference data of the multiple fourth sample response data, the tendency labeling data of the multiple fourth sample response data is determined; The fourth sample question data and the fourth sample response data are input into the propensity detection model to be trained to predict the object's propensity, thereby obtaining the propensity prediction data of each fourth sample response data. Based on the tendency prediction data and the tendency labeling data of each fourth sample response data, the tendency classification loss information is determined. Based on the index ranking information corresponding to the tendency labeling data of the multiple fourth sample response data, the tendency prediction data of the multiple fourth sample response data is ranked and analyzed to determine the preference ranking loss information. Based on the tendency classification loss information and the preference ranking loss information, the tendency detection model to be trained is trained to obtain the object tendency detection model.

6. The method according to claim 2, characterized in that, The model update method corresponding to the second anomaly type information among the multiple target anomaly types is a knowledge base update method, and the method further includes: Extract role keywords from the question data in the historical dialogue data corresponding to the second abnormal type information to determine the role identifier to be updated and the setting identifier to be updated. Based on the character identifier to be updated and the setting identifier to be updated, collect character setting source data; Based on the source data of the character settings, the preset character setting knowledge base is updated.

7. The method according to claim 2, characterized in that, The model update method corresponding to the third anomaly type information among the various target anomaly types is a model fine-tuning method. The role dialogue sample set corresponding to the third anomaly type information includes: third sample question data and labeled response data corresponding to the third sample question data. The step of training the language model of the previous iteration cycle based on the role dialogue sample set corresponding to each target anomaly type information and the model update method corresponding to each target anomaly type information to obtain the language model of the current iteration cycle includes: Based on the third sample question data and the corresponding labeled response data, the language model of the previous iteration cycle is fine-tuned to obtain the language model of the current iteration cycle.

8. The method according to any one of claims 1 to 7, characterized in that, The process of determining the target role setting information based on the current question data and the role identifier includes: Extract multiple character setting information entries corresponding to the character identifier from the preset character setting knowledge base; Determine the target character setting information associated with the current question data from the multiple character setting information; The step of inputting the current question data and the target role setting information into the target language model for response prediction to obtain target response data includes: The target role setting information and the preset role setting instruction information are concatenated to obtain the target role setting instruction information; The current question data and the target role setting instruction information are input into the target language model to predict the response, thereby obtaining the target response data.

9. The method according to claim 2, characterized in that, The multiple historical dialogue data were obtained in the following ways: Obtain multiple raw dialogue data generated during online dialogue using the language model from the previous iteration cycle; Information content analysis is performed on the multiple original dialogue data to obtain the information content index of each of the multiple original dialogue data; From the plurality of original dialogue data, original dialogue data with information content indicators greater than a preset information content threshold are selected as the plurality of historical dialogue data.

10. A dialogue processing apparatus, characterized in that, The device includes: The current question data acquisition module is used to acquire the current question data of the target object and the role identifier corresponding to the virtual interaction object during the dialogue between the target object and the virtual interaction object. The role setting information determination module is used to determine the target role setting information based on the current question data and the role identifier; The response prediction module is used to input the current question data and the target role setting information into the target language model to predict the response and obtain the target response data. Wherein, the target language model is the language model of the current iteration cycle associated with the role identifier; the language model of the current iteration cycle is obtained by training the language model of the previous iteration cycle with role dialogue sample sets corresponding to each of the multiple target anomaly types and the model update method corresponding to each of the multiple target anomaly types in the previous iteration cycle; the role dialogue sample set corresponding to each target anomaly type is obtained based on the sub-anomaly distribution information corresponding to each of the multiple target anomaly types in the previous iteration cycle, and the sub-anomaly distribution information is used to characterize the distribution of historical dialogue data of at least one sub-anomaly type corresponding to the target anomaly type in the previous iteration cycle; the cycle duration of the previous iteration cycle is adaptively adjusted based on the sample accumulation rate of the multiple target anomaly types.

11. A dialogue processing device, characterized in that, The device includes a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the dialogue processing method as described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction or at least one program segment, which is loaded and executed by a processor to implement the dialogue processing method as described in any one of claims 1 to 9.

13. A computer program product, characterized in that, The computer program product includes at least one instruction or at least one program segment, which is loaded and executed by a processor to implement the dialogue processing method as described in any one of claims 1 to 9.

Images