Optimization method of intelligent customer service for pre-defining tasks and identities

Abstract

An optimization method of an intelligent customer service for pre-defining tasks and identities comprises the following steps: acquiring original data from different data sources, performing data preprocessing and alignment processing, enabling a model to accurately identify personality and identities of characters in a dialogue through self-supervised training, and performing task and identity pre-defining; and carrying out reinforcement learning training on the intelligent customer service model by taking the model as a reward model so as to enhance the expressive force of the intelligent customer service model in the task. Through a large-scale data mining technology, a corpus representing specific personalities and identities is extracted from mass dialogue records. Then, the corpora are utilized to carry out fine tuning on a pre-trained language model, so that the language model can show more stable and consistent personality and identity in the dialogue process. And meanwhile, the image stability is improved by detecting whether the chat model output accords with the set image or not in the long-range dialogue.

Description

Technical Field

[0001] The present invention relates to a technology in the field of neural networks, and in particular to an optimization method for intelligent customer service with predefined tasks and identities. Background Art

[0002] The personalization and task compliance consistency of intelligent customer service can significantly improve user compliance and motivation, provide more accurate support by adapting to user personality and needs, and improve social and cognitive functions. However, the current intelligent customer service chat model has problems such as inconsistent personality and inconsistency in long-term conversations, which affect user experience. Summary of the invention

[0003] In view of the above-mentioned deficiencies in the prior art, the present invention proposes an optimization method for intelligent customer service with predefined tasks and identities. Through large-scale data mining technology, a corpus representing a specific personality and identity is extracted from a large number of conversation records. Then, the pre-trained language model is fine-tuned using these corpora so that it can show a more stable and consistent personality and identity during the conversation. At the same time, in long-distance conversations, the chat model output is tested to see if it meets the set image to improve the image stability.

[0004] The present invention is achieved through the following technical solutions:

[0005] The present invention relates to an optimization method for intelligent customer service with predefined tasks and identities. The method collects raw data from different data sources, performs data preprocessing and alignment, and then uses self-supervised training to enable a model to accurately identify the personality and identity of a person in a conversation. The model is then used as a reward model to perform reinforcement learning training on an intelligent customer service model to enhance the expressiveness of the intelligent customer model in tasks.

[0006] The generated labels refer to: recording the identification number, name and consistency level of the character in the Big Five model through the confirmed identity and personality type of the character in each conversation.

[0007] The data preprocessing is to clean the data and remove duplicate and unparseable content.

[0008] The alignment process refers to extracting dialogue content from the script through a neural network architecture combined with a regularization method, and aligning the dialogue text with the personality label.

[0009] The supervised training preferably performs multi-template encoding on input data of different formats and inputs them into a large language model in a unified form for training.

[0010] The reinforcement learning training preferably performs multi-template encoding on input data in different formats, inputs them into a large language model in a unified form, and uses the supervised trained recognition model as a reward model to optimize intelligent customer service.

[0011] The optimization method specifically includes:

[0012] Step 1: Acquisition and label extraction of personality type data: From multiple data sources (such as PersonalityDatabase, PDB) to crawl personality data. Record the character's identification number, name, identity, and Big Five personality

[0013] Step 2: Acquisition and cleaning of script data: Use crawlers to obtain script texts of movies and TV series from multiple resource libraries (such as Internet MovieScript Database, IMSDB). Clean the crawled data to remove duplicate and unparseable content. Extract dialogue content from the script through a neural network architecture combined with a regularization method, and align the dialogue text with personality tags and character identities based on the character information identified in the script.

[0014] Step 3: Align the dialogue data with personality and identity: The dialogue text extracted from the script is accurately aligned with the personality and identity to ensure that each dialogue corresponds to the personality and identity of its character. This step ensures the high quality and consistency of the dataset.

[0015] Step 4: Fine-tune the large language model: Use the constructed dataset to fine-tune the large language model, and enhance the model's expressiveness in personality and identity recognition tasks through supervised learning. The multi-template encoding format is used in the fine-tuning process to enable the model to better adapt to input data in different formats.

[0016] Step 5: Intelligent customer service optimization: Use reinforcement learning to enhance the expressiveness of the large language model for personality and identity recognition in predefined tasks and identity tasks.

[0017] The present invention relates to a system for implementing the above method, including: a data crawler unit, a data cleaning unit, a data alignment unit, a self-supervised training unit and a reinforcement learning unit, wherein: the data crawler unit crawls to obtain model training data and obtains dirty model training data; the data cleaning unit performs data cleaning processing based on the crawled data information to obtain clean data results; the data alignment unit performs data alignment processing based on personality identity data and conversation data to obtain final data results; the self-supervised training unit performs fine-tuning processing on a large language model based on the data to obtain a result that can accurately identify the identity and personality of the characters in the conversation. The reinforcement learning unit uses the self-supervised model as a reward model to optimize the expressiveness of the intelligent customer service in the task, and obtains a result that can maintain consistent personality and identity in a long-term conversation. Technical Effects

[0018] The present invention uses a detection model to calibrate the problem of intelligent customer service not following the established identity response in long-distance conversations, and calibrates the problem of forgetting in long-distance conversations. Compared with the prior art, the present invention can maintain the established identity response in the task and follow consistent responses in long-distance conversations. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a flow chart of the present invention;

[0020] Figure 2 Schematic diagram of the system of the present invention. DETAILED DESCRIPTION Example 1

[0021] like Figure 1 As shown, this embodiment involves an optimization method for intelligent customer service with predefined tasks and identities, which aims to improve user experience and satisfaction by enhancing the personalized service capabilities and task execution efficiency of the intelligent customer service system, specifically including:

[0022] Step 1: Crawler tool development and data acquisition

[0023] Step 101: Develop a crawler program using the Scrapy library of the Python programming language or an automation tool such as Selenium to crawl personality memory identity data about movie and TV series characters from the PDB website.

[0024] The data includes but is not limited to the following information: the name of the character, the film and television works to which it belongs, the character's personality, and the specific identity of each character.

[0025] Step 102, the data recorded for each character will contain detailed identification information, including the character's identification number (ID), name, identity, and consistency level on the Big Five personality model dimensions. Example 2

[0026] This embodiment develops a special crawler tool to automatically obtain script texts from multiple public movie and TV drama script databases, and cleans and structures these texts to provide a rich dialogue data source for building a high-quality personality prediction dataset. By effectively extracting and cleaning the script texts, it can ensure that the training data of the subsequent personality prediction model has high-quality dialogue information.

[0027] This embodiment specifically includes:

[0028] Step 1: Crawler tool development and script acquisition

[0029] Step 101: In order to achieve automated data acquisition, this embodiment designs and develops a crawler tool specifically for crawling movie and TV drama script text resources from multiple online script resource libraries. The main data sources include but are not limited to the following websites: Internet Movie Script Database (IMSDB), SimplyScripts, The Daily Script, TelevisionWriting, etc. The crawler automatically identifies and captures script text data by parsing the structure of the web page, including key information such as scene descriptions and character dialogues. Each script file usually contains a variety of structural information, such as the script title, author name, scene descriptions, character dialogues, comments, etc. The crawler tool extracts this information based on the HTML tag features of the web page to ensure that each script text can be completely and accurately captured. In order to improve the efficiency of the crawler program, the program design also includes the function of filtering advertisements, irrelevant content, and duplicate scripts, so as to ensure that the captured script text data has a high quality.

[0030] Step 2: Script cleaning and formatting. After the script text is captured, the original data needs to be cleaned and formatted to remove invalid information and data with abnormal formats, and to ensure the integrity and consistency of the script text structure. This step is mainly divided into two sub-steps:

[0031] Step 201, data cleaning - remove invalid data and data with abnormal formats. The original script data usually contains a lot of redundant information, such as scene descriptions, author notes, unfinished paragraphs, web page advertisements, and copyright notices. These invalid data not only take up storage space, but also interfere with the subsequent model's understanding of the text. Therefore, the data must be cleaned. The specific process is as follows:

[0032] Regular Expression Processing: Use regular expressions (Regular Expressions, Regex) as an efficient text matching tool to automatically identify and filter out irrelevant information in the script. First, process the scene descriptions and author notes. These contents are usually marked with brackets or special symbols, such as "(Interior / Night)", "(Actor standing by the window)", etc. Through the preset regular expression pattern, these irrelevant descriptions are automatically identified and deleted. In addition, it also includes deleting information such as "[Note: This paragraph is deleted]".

[0033] Advertisements or extra characters in script files: Some script texts may contain advertisements or copyright notices from websites, such as "This article comes from a certain script website", etc. These irrelevant contents are also automatically identified and deleted through regular expression patterns to ensure that the dataset is clean and noise-free.

[0034] Blank paragraphs and incomplete conversations: During the formatting process, some conversation paragraphs may be incomplete due to formatting issues, or contain only partial information (such as line break errors or irregular formats). To ensure data quality, the system first uses string matching and paragraph integrity check mechanisms to filter out incomplete or blank paragraphs. Each conversation should contain a complete sentence structure, such as checking whether each line of text ends with a period or other end symbol to determine whether it is a complete sentence. If an incomplete or missing paragraph is found, it will be removed from the dataset.

[0035] Step 202: Use a pre-trained natural language processing model combined with a rule-based segmentation method. In order to ensure that the dialogue and scene description data extracted from the script text are highly accurate, this step combines a pre-trained natural language processing model (such as BERT) and a segmentation method based on indentation rules. This method can achieve efficient text classification and extraction, and is particularly suitable for scenes such as script texts with complex structures and intertwined dialogue and description information.

[0036] Step 202-A, rule-based segmentation method: In order to assist the semantic analysis of the BERT model, this method adopts a segmentation strategy based on indentation rules. In a script, character dialogues usually have larger indentations, while scene descriptions and stage instructions have smaller indentations. Therefore, the dialogue part and the non-dialogue part can be better distinguished according to the indentation level:

[0037] Step 1) Dialogue: Dialogue paragraphs usually have large indents and often begin with the character name. Each character name is followed by the character's lines. For example, in the script "JACK: Must leave this city!", "JACK" is the character name, and the part immediately following it is the character's dialogue content.

[0038] Step 2) Scene description: Unlike the dialogue part, the scene description usually has a smaller indentation or no indentation. For example, the paragraph "(Interior / Night) Two people look at each other in an empty room" is usually used to describe actions, emotions or environments, and does not belong to dialogue data.

[0039] Step 202-B: Segmentation combining pre-trained model and rules:

[0040] Step 1) Scene switching and action description recognition: Scenes in scripts usually use specific markers (such as "INT. (indoor)" and "EXT. (outdoor)") to indicate scene switching. By combining indentation rules and semantic analysis of the BERT model, we can accurately identify scene switching points and effectively separate dialogues and scene descriptions in different scenes.

[0041] Step 2) Noise processing: Since script formats may differ between different authors, a single rule approach sometimes cannot cover all situations. By introducing pre-trained natural language processing models such as BERT and combining their understanding of text context, script texts with inconsistent formats can be processed more intelligently, thereby improving the accuracy of text processing.

[0042] The process of using BERT to recognize scripts is as follows:

[0043] BERT (Bidirectional Encoder Representations from Transformers) is a bidirectional language model based on the Transformer architecture that can perform in-depth analysis of text by understanding contextual semantic relationships. During the cleaning and segmentation process, BERT can identify the semantic structure of the text, especially when distinguishing complex conversations and scene descriptions, BERT shows extremely high accuracy.

[0044] In order to ensure that the BERT model can effectively extract dialogues and scene descriptions from the script text, the following key steps are required:

[0045] Step 1) Data preparation

[0046] Text cleaning: Use simple rule-based methods, such as regular expressions, to remove non-script content such as web page advertisements, copyright notices, blank lines, and other invalid symbols.

[0047] Data annotation: Manually or semi-automatically annotate a portion of the script text to clearly distinguish between dialogue paragraphs and scene descriptions. The annotated data is used to fine-tune the BERT model so that it can better understand different elements in the script (such as character dialogue and scene descriptions).

[0048] Step 2) Loading the BERT pre-trained model

[0049] The BERT model is obtained through pre-training on a large-scale corpus and subsequent task fine-tuning. The pre-trained BERT model can be loaded through open source libraries (such as Hugging Face's Transformers library).

[0050] Pre-training tasks:

[0051] Masked Language Model (MLM): By masking certain words in a sentence, the model is allowed to predict the masked words, thereby enhancing the model's contextual understanding ability.

[0052] NextSentence Prediction (NSP): Improves the model’s ability to understand the relationship between sentences by predicting whether two sentences are consecutive sentences.

[0053] Step 3) Formatting the script text

[0054] Tokenization: BERT uses the WordPiece tokenizer to break each sentence into more fine-grained word fragments and convert them into token IDs that the model can recognize. For each dialogue and scene description, tokenization is required to generate the corresponding token sequence.

[0055] Segment IDs and Attention Masks: The input of the BERT model needs to be labeled separately for different parts (such as dialogue and scene description). By adding special [CLS] and [SEP] tags before and after each paragraph of text, the model can be helped to identify the start and end of each sentence. Attention Masks are used to indicate which word segments the model should focus on.

[0056] Step 4) Fine-tune the BERT model

[0057] Since the pre-trained BERT model is a general language model, it needs to be fine-tuned to better adapt it to the specific tasks of the script. The fine-tuning process includes the following steps:

[0058] Define the task: Convert the script text processing task into a text classification task, requiring the model to determine whether the input text belongs to "dialogue" or "scene description".

[0059] Input data preparation: Annotate the paragraphs in the script as "dialogue" or "scene description" and input these annotated data into the BERT model for training.

[0060] Loss function and optimizer: The cross-entropy loss function is used to train the model so that it can more accurately distinguish between dialogue and scene descriptions. The Adam optimizer is also used for parameter optimization.

[0061] Step 5) Prediction and application of the model

[0062] After training, the BERT model can predict new script texts and automatically identify dialogues and scene descriptions in the scripts. When new script text is input, the model will predict the category (dialogue or scene description) of each paragraph of text sentence by sentence or paragraph by paragraph.

[0063] Step 6) Further processing of the results

[0064] The results output by the BERT model can be used to build a structured script text dataset, especially in the following scenarios:

[0065] Dialogue Extraction: Extract each dialogue and associate it with the character name.

[0066] Scene description annotation: The scene description content is stored separately for use in subsequent tasks such as sentiment analysis and action recognition.

[0067] This step ensures that the dialogue extraction in the script text is more accurate and consistent through the combination of regularized segmentation and model. The extracted dialogue data not only includes the direct speech content, but also can supplement some unlabeled dialogue content through contextual understanding. Example 3

[0068] Use the dataset to fine-tune large language models (LLMs). The fine-tuning process aims to introduce a variety of templates to encode the data to enhance the model's adaptability to different input formats, so that the model can accurately identify the personality and identity of the characters in the conversation. The specific steps are as follows:

[0069] Step 1: Data encoding: During the training data preparation phase, the input instructions are encoded by adding specific prefixes and suffixes. For example, the input instructions are prefixed with "task:", and identifiers such as "input:", "output:" are appended to the input instructions to clearly indicate the structure of the input data. This encoding method helps the model distinguish between the task description and the actual input data, thereby better understanding the task requirements.

[0070] Step 2: Template diversification: Considering the diversity of input data formats in actual application scenarios, the present invention uses a variety of different templates to encode input data. For example, in some cases, the prefix of the input instruction can be "task:", while in other cases no prefix is ​​required; similarly, the "input:" identifier can appear anywhere in the prompt, while "output:" can be placed at the end of the prompt, and different numbers of separators can be included in the middle. This diversified template design ensures that the model can handle input data of various formats and enhances the generalization ability of the model.

[0071] Step 3: Supervised learning training: Use standard supervised learning methods to train the model. Specifically, pair the conversation text with the corresponding personality label as input and output pairs, and adjust the model parameters by optimizing the loss function to improve the model's performance on the personality prediction task. During the training process, the model gradually improves its ability to understand the conversation content and the accuracy of personality prediction by repeatedly learning the relationship between input and output pairs.

[0072] Step 4: Model optimization: Through the above fine-tuning process, the model not only shows higher accuracy in processing personality and identity recognition tasks, but also significantly improves its ability to understand conversational contexts. The goal of model optimization is to ensure that the model has good generalization capabilities while maintaining high recognition accuracy, that is, accurate recognition can be made on unseen data.

[0073] Step 5: Result verification: In order to verify the effect of fine-tuning, a series of standardized test sets are used to evaluate the model. By comparing the performance of the model before and after fine-tuning, the performance improvement brought by fine-tuning can be intuitively observed.

[0074] After specific practical experiments, the LLAMA-2-7b-chat model was used as the basis. After fine-tuning, the model was able to achieve an accuracy of 85%, compared with the most advanced CHAT-GPT model with an accuracy of 82%. Example 4

[0075] The reward model is used to optimize the performance of predefined tasks in the intelligent customer service system. The training process aims to encode the data by introducing diverse templates to enhance the model's adaptability to different input formats, so that the model can more accurately imitate specific personality traits. The specific steps are as follows:

[0076] This method mainly relies on reinforcement learning technology and uses the reward model feedback mechanism to fine-tune the intelligent customer service model to better reflect the set personality and identity to meet task expectations.

[0077] To achieve this goal, the intelligent customer service module introduced background information, personality, and contextual dialogue fragments, while deleting the speech of the object to be simulated. In addition, a prompt system was designed to guide the model to generate language output that meets expectations.

[0078] The whole process is like Figure 2 As shown, the trained recognition model first evaluates whether the dialogue generated by the intelligent customer service module conforms to the preset context, and then adjusts the model parameters according to the evaluation results, and finally obtains a chatbot that can complete the predetermined task more accurately.

[0079] In actual operation, the Proximal Policy Optimization (PPO) algorithm is used to optimize the model parameters.

[0080] To further verify the effectiveness of this method, ChatGPT was used as an external evaluation tool to test the unadjusted basic version and the trained improved version of the intelligent customer service module. The results showed that among the 30 randomly selected task samples, the basic version model could only correctly complete 7, while the improved version model successfully completed 18.

[0081] Compared with the existing technology, the performance indicators of this method are improved in that: by automatically verifying the replies generated by the intelligent customer service, its task expressiveness is improved.

[0082] In the final intelligent customer service system, the following Figure 2 In the manner shown, the responses generated by the intelligent customer service are verified so that it can better serve users.

[0083] The above-mentioned specific implementation can be partially adjusted in different ways by those skilled in the art without departing from the principle and purpose of the present invention. The protection scope of the present invention shall be based on the claims and shall not be limited by the above-mentioned specific implementation. Each implementation scheme within its scope shall be subject to the constraints of the present invention.

Claims

1. A method for optimizing intelligent customer service with predefined tasks and identities, characterized in that: By collecting raw data from different data sources and after data preprocessing and alignment, the model is first trained through self-supervision to enable it to accurately identify the personality and identity of the characters in the conversation, and the model is used as a reward model to perform reinforcement learning training on the intelligent customer service model to enhance the expressiveness of the intelligent customer model in the task.

2. The optimization method of intelligent customer service with predefined tasks and identities according to claim 1 is characterized in that: The generated labels refer to: recording the character's identification number, name, and consistency level in the BigFive model through the confirmed identity and personality type of the character in each conversation.

3. The optimization method of intelligent customer service with predefined tasks and identities according to claim 1, characterized in that: The data preprocessing is to clean the data and remove duplicate and unparseable content.

4. The optimization method of intelligent customer service with predefined tasks and identities according to claim 1, characterized in that: The alignment process refers to extracting dialogue content from the script through a neural network architecture combined with a regularization method, and aligning the dialogue text with the personality label.

5. The optimization method of intelligent customer service with predefined tasks and identities according to claim 1, characterized in that: The supervised training described above performs multi-template encoding on input data of different formats and inputs the data into a large language model in a unified form for training.

6. The optimization method of intelligent customer service with predefined tasks and identities according to claim 1, characterized in that: The reinforcement learning training described herein performs multi-template encoding on input data of different formats, inputs the data into a large language model in a unified form, and uses the supervised trained recognition model as a reward model to optimize intelligent customer service.

7. The optimization method of intelligent customer service with predefined tasks and identities according to any one of claims 1 to 6, characterized in that: include: Step 1: Acquisition and label extraction of personality type data: crawl personality data from multiple data sources, record the character's identification number, name, identity, and Big Five personality; Step 2: Acquisition and cleaning of script data: Use crawlers to obtain script texts of movies and TV series from multiple resource libraries, clean the crawled data, remove duplicate and unparseable content, extract dialogue content from the script through a neural network architecture combined with a regularization method, and align the dialogue text with personality tags and character identities based on the character information identified in the script; Step 3: Align dialogue data with personality and identity: The dialogue text extracted from the script is accurately aligned with the personality and identity to ensure that each dialogue corresponds to the personality and identity of its character. This step ensures the high quality and consistency of the dataset. Step 4: Fine-tune the large language model: Use the constructed dataset to fine-tune the large language model, and enhance the model's expressiveness in personality and identity recognition tasks through supervised learning. Use a multi-template encoding format during fine-tuning to enable the model to better adapt to input data in different formats. Step 5: Intelligent customer service optimization: Use reinforcement learning to enhance the expressiveness of the large language model for personality and identity recognition in predefined tasks and identity tasks.

8. An optimization system for intelligent customer service with predefined tasks and identities implementing any of the methods described in claims 1-7, characterized in that: include: The data crawler unit, the data cleaning unit, the data alignment unit, the self-supervised training unit and the reinforcement learning unit include: the data crawler unit crawls to obtain model training data and obtains dirty data for model training; the data cleaning unit performs data cleaning processing based on the crawled data information to obtain clean data results; the data alignment unit performs data alignment processing based on the personality identity data and the conversation data to obtain the final data results; the self-supervised training unit fine-tunes the large language model based on the data to obtain the result that can accurately identify the identity and personality of the characters in the conversation; the reinforcement learning unit uses the self-supervised model as a reward model to optimize the expressiveness of the intelligent customer service in the task, and obtains the result of maintaining consistent personality and identity in long-term conversations.

Images