A method for constructing prompts based on multi-stage dialogues with emotional factors
Through multi-stage dialogue between emotional factors, the Prompt method is constructed, combined with the cosmetics sales knowledge base, the problem of insufficient user emotions and knowledge base in cosmetics sales is solved, and more accurate understanding of user needs and personalized services are achieved, and sales efficiency is improved. and user experience.
Patent Information
- Application Number
- CN202411011845.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-26
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2044-07-26
AI Technical Summary
In the cosmetics sales scenario, the existing large language models lack understanding of user emotions and mentality, and the sales knowledge base is insufficient, resulting in the inability to effectively reach user needs and provide personalized services.
Using multi-stage dialogue based on emotional factors to build Prompt method, identify user emotions through emotion recognition model and random forest method, and combine cosmetic sales knowledge base to build Prompt using CoT-SC method, introduce emotional factors and expertise, optimize dialogue process and Information retrieval.
It improves the big model's understanding of user needs and personalized service capabilities, can more accurately reach user needs, provide in-depth product information and emotional care, and enhances sales efficiency and user experience.
Smart Images

Figure CN118536608B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer large models, in particular to the field of large model prompt engineering, and specifically to a method for constructing a prompt based on a multi-stage dialogue of emotional factors. Background Art
[0002] Prompt Engineering is an emerging field that focuses on the formulation and optimization of prompt words, aiming to help people more effectively apply large language models (LLM) to various application scenarios and research projects. Researchers may use prompt engineering to improve the ability of large language models to handle complex tasks, including the implementation of question-answering systems and complex logical inference tasks. Initially, prompts were mainly generated through manual design, for example, designing some fixed templates or prefixes for a given task. Although this method is simple, it sometimes does not work well, especially when dealing with some complex tasks.
[0003] With the development of technology, people have begun to explore more complex prompt generation and optimization methods such as chain prompts (CoT) (J. Wei, X. Wang, D.Schuurmans, M. Bosma, F. Xia, E. Chi, QV Le, D. Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances inNeural Information Processing Systems, 35:24824–24837, 2022.), tree prompts (ToT) (S. Yao, D. Yu, J. Zhao, I. Shafran, TL Griffiths,Y. Cao and K.Narasimhan. 2023. Tree of Thoughts: Deliberate Problem Solving with LargeLanguage Models.ArXiv:2305.10601 [cs].), dynamic prompts, and context-aware prompts. These methods can better utilize the depth and width of large models, enabling them to play a role in a wider range of tasks and applications.
[0004] There are already many specific applications of large language models for sales, such as Langchain’s open source SalesGPT project.
[0005] Langchain's SalesGPT project is committed to using big model technology to create a robot that can communicate with users and sell products to them. The core goal is to improve the intelligence and automation of sales in order to improve sales efficiency and user experience.
[0006] This project breaks down the sales process into multiple steps, such as identifying needs, providing solutions, and promoting sales, and then trains SalesGPT to handle these tasks separately. By leveraging the large model's ability to understand and generate natural language, the robot can understand user needs, provide matching products, drive the sales process, and ultimately achieve sales.
[0007] As AI empowers all walks of life, sales is a good application scenario. Although there are many specific cases of large sales models, there are still many deficiencies and problems in the cosmetics sales scenario, such as: lack of understanding of user mentality and emotions, and insufficient knowledge base of cosmetics sales. Summary of the invention
[0008] In view of the deficiencies of the prior art, the purpose of the present invention is to provide a method for constructing Prompt based on multi-stage dialogues of emotional factors.
[0009] The objective of the present invention is achieved through the following technical solution: A method for constructing a prompt based on a multi-stage dialogue of emotional factors, comprising the following steps:
[0010] 1. Identify facial expression information and voice emotion information through the emotion recognition model and use the weighted average method to construct user emotion information with expression weights;
[0011] 2. Use the random forest method to establish a connection between user emotions and the conversation stage through the trained random forest model to help divide the conversation stage;
[0012] 3. Build a knowledge base for cosmetics sales;
[0013] 4. Combine user emotion information, dialogue stage, and cosmetics sales knowledge base to construct Prompt through the Cot-SC method.
[0014] Furthermore, the step 1 includes the following sub-steps:
[0015] 1.1. Use the VGG19 model to identify facial expression information, and use the wav2vec-2.0 model to identify emotional information in speech. Use a labeled data set in the recognition process, where different emotion categories are selected as labels in the data set. The emotion categories include happiness, sadness, anger, and surprise;
[0016] 1.2. Determine the ratio of different emotion weights, and fuse the emotion data obtained from facial expression information and voice information according to the pre-set expression and tone weight ratio.
[0017] Furthermore, in step 1.2, the weight ratio of expression and tone is set to 0.7 for expression and 0.3 for tone.
[0018] Furthermore, the step 2 includes the following sub-steps:
[0019] 2.1. Manually prepare a data set, in which the feature values in the data set include the weights of user emotions, and the classification labels are the appropriate dialogue stages corresponding to different emotions. For example, when the user emotion is angry, the dialogue stage is suitable for guiding to the customer objection stage;
[0020] 2.2. Divide the data set prepared in step 2.1, use 80% of the data as the training set and the remaining 20% as the test set, to ensure that the model has enough data for learning during the training process, and perform performance evaluation on the isolated test set; because the random forest algorithm and most machine learning algorithms can only process numerical data, ensure that all feature values are numerical;
[0021] 2.3. Build a random forest model, including importing the random forest classifier and setting model parameters; use the training set obtained in step 2.2 to train the random forest model;
[0022] 2.4. After training, use the test set obtained in step 2.2 to evaluate the accuracy of the random forest model, as well as other indicators including precision, recall, and F1 score. The indicators provide a comprehensive view of the model's prediction results, help understand the strengths and weaknesses of the model in specific situations, and thus provide a basis for further adjustment and optimization;
[0023] 2.5. Parameter tuning. Based on the evaluation indicators obtained in step 2.4, use the grid search method to automatically adjust the size of the trees in the random forest model to find the optimal result.
[0024] Furthermore, the random forest model described in step 3.3 is constructed using the scikit-learn machine learning library.
[0025] Furthermore, the step 3 includes the following sub-steps:
[0026] 3.1. Data acquisition: First, build a knowledge base in the common cosmetics sales field, including a cosmetics ingredient library, a cosmetics price library, and a library that updates coupons in real time. Then, build a user profile library that contains basic user information, such as skin type and purchase preferences.
[0027] 3.2. File preprocessing: Delete common invalid tags, such as HTML tags, and remove stop words and words that are common in the text but have little significance for analysis. After SpaCy preprocesses the file library and adjusts the consistency of the nouns in the text, the preprocessing is completed;
[0028] 3.3. File segmentation and vectorization: The text content is segmented according to paragraphs. After the text is segmented, the text blocks are vectorized to convert the original text into a numerical vector, so that texts with similar themes or contents are closer in the vector space, while irrelevant contents are farther away, which is helpful for subsequent information retrieval and analysis;
[0029] 3.4. Build file index: Build an inverted index to speed up the retrieval of relevant information.
[0030] Furthermore, in step 3.3, the vectorization process adopts the M3E model.
[0031] Furthermore, the step 4 includes the following sub-steps:
[0032] 4.1. Dialogue stage template construction: The dialogue stages are refined and divided into self-introduction stage, customer willingness judgment stage, product value introduction stage, customer demand analysis stage, solution generation stage, customer objection handling stage, negotiation invitation stage, and abnormal dialogue termination stage. GPT4.0 generates multiple sets of templates for each stage through prompts, which are manually optimized. The templates are designed to guide the dialogue to smoothly transition to the next stage until a solution is finally generated, customer objections are handled, and invitations are attempted;
[0033] 4.2. CoT-SC constructs prompts: Use the CoT-SC method to build a reasoning chain for the prompt templates of each stage after the division in step 4.2, add the user emotion information prepared in step 1 and the knowledge base professional information and user profile information prepared in step 3.1, and select the content with high frequency of occurrence and the highest consistency with the purpose of the conversation as the prompt.
[0034] The present invention also provides an electronic device, comprising a memory and a processor, wherein the memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the method of constructing a prompt based on a multi-stage dialogue of emotional factors.
[0035] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method of constructing a prompt based on a multi-stage dialogue of emotional factors is implemented.
[0036] The beneficial effect of the present invention is that emotional factors are introduced as an important factor in judging customer willingness and expectations. The introduction of emotional factors can directly affect the output effect of the large model, and can more accurately reach user needs and use industry knowledge to help users make purchasing decisions. In addition, the present invention constructs a comprehensive and in-depth cosmetics knowledge base, which can provide a large amount of information and knowledge about specific fields, so that the large model can obtain enough information to provide accurate and in-depth services. Therefore, by understanding and adapting to this subtle but important way of communication, the large model can not only provide product information, but also provide appropriate emotional care to alleviate user concerns. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] Figure 1 This is the overall framework diagram of this method;
[0038] Figure 2 It is a flowchart that introduces emotional factors to construct Prompt;
[0039] Figure 3 It is a flowchart for constructing a knowledge base for cosmetics sales. DETAILED DESCRIPTION
[0040] The present invention will be further described below in conjunction with the accompanying drawings.
[0041] like Figure 1 As shown, a method for constructing Prompt based on multi-stage dialogue of emotional factors, the specific implementation scheme of the present invention can be divided into several steps:
[0042] 1. Identify facial expression information and voice emotion information through the emotion recognition model and use the weighted average method to construct user emotion information with expression weights.
[0043] 1.1. First, the training of face and voice emotion recognition models is carried out. For facial emotion recognition, the convolutional neural network model (CNN) in deep learning is used for training. Because CNN has excellent performance in the field of image recognition, the present invention uses the VGG19 pre-trained model for facial emotion recognition, and the wav2vec 2.0 model launched by Facebook AI in 2020 is used for voice emotion recognition. This model can obtain good voice emotion recognition results with its powerful self-supervised learning ability. Both models require a large number of face and voice data sets marked with emotion category labels for training, and the labels include common emotions such as happiness, sadness, anger, surprise, etc.
[0044] 1.2. In the process of emotion data fusion, the weights of facial emotion recognition and voice emotion recognition are set. This requires determining which type of data can more accurately reflect the user's true emotions based on previous research or experimental results. Since facial emotion data is more stable and can better reflect the user's inner emotions, a higher weight is given to facial emotion recognition results, such as a facial emotion weight of 0.6 and a voice emotion weight of 0.4. After multiple manual weight parameter adjustments, the random forest model can achieve the best results when the weight parameter settings are close to 0.7 (facial emotion) and 0.3 (voice emotion). Then, the two emotion recognition results are weighted averaged according to the set weights to construct a comprehensive, weighted emotion factor score. This method takes into account the advantages of each and improves the accuracy and reliability of overall emotion recognition through effective fusion.
[0045] 2. Use the random forest method to establish a connection between user emotions and the conversation stage through the trained random forest model to help divide the conversation stage, such as Figure 2 shown.
[0046] 2.1. The preparation and organization of the data set is the first step in this process. The data that needs to be collected and organized includes emotional feature vectors from user facial expressions and voice information, as well as user emotional weights calculated based on a ratio of 0.7 (facial emotions) to 0.3 (voice emotions). In addition, the stage of each conversation is marked as a classification label for use in model training. Ensuring that this data is accurate is the key to building an effective model.
[0047] 2.2. Data partitioning will be performed according to the ratio of 8:2, with 80% of the data used as the training set and the remaining 20% as the test set. This not only ensures that the model has sufficient data to learn during training, but also retains a portion of the data for subsequent performance verification. In addition, all feature values must be converted to numerical format to meet the processing requirements of the random forest algorithm.
[0048] 2.3. Next, we use the scikit-learn machine learning library to build a random forest model. This includes importing the random forest classifier and setting appropriate model parameters, such as deciding to use 27 trees (n_estimators=27). Then, we train the model using the training set segmented in the previous step to learn how to identify the conversation stage based on the weight of the user's emotions and other features.
[0049] 2.4. After the model is trained, its performance is evaluated through the test set. The accuracy, precision, recall and F1 score are used to comprehensively consider the prediction ability of the model. The evaluation results play an important role in adjusting and optimizing the model, thereby continuously improving its efficiency and effectiveness in practical applications.
[0050] 2.5. Parameter tuning. Based on the evaluation indicators obtained in step 2.4, use the grid search method to automatically adjust the size of the trees in the random forest model to find the optimal result.
[0051] 3. Build a knowledge base for cosmetics sales, such as Figure 3 shown.
[0052] 3.1. Data acquisition: Data collection is the first step in building a knowledge base. At this stage, we focus on collecting multi-dimensional data related to the cosmetics industry. Specific operations include: creating a cosmetic ingredient database containing multiple information such as ingredients, efficacy, and applicable skin types; establishing a cosmetic price database that fully reflects market price fluctuations; and building a discount information database containing the latest discount activities and coupon information. In addition, by collecting users' personal information, including skin type, purchase history, and preferences, we establish a personal user archive, laying a solid foundation for personalized services.
[0053] 3.2. File preprocessing: The goal of the data preprocessing stage is to improve data quality and ensure that the information is accurate. First, by using efficient text processing tools such as SpaCy, invalid information in the data text, such as HTML tags, and unnecessary text information such as stop words, is removed. Next, the expression of cosmetics-related nouns is unified to ensure data consistency and accuracy.
[0054] 3.3. File segmentation and vectorization: Perform text segmentation and vectorization to better organize and understand data content. Use paragraphs as units to organize text, and then use advanced model technologies such as M3E for vectorization, converting text information into numerical vectors to facilitate mining and analysis of similarities and differences between texts.
[0055] 3.4. Constructing file index: Constructing an effective file index mechanism is the key to the knowledge base's rapid response to queries. In particular, the inverted index technology is used to map keywords in the document with document identifiers. The establishment of this mapping relationship greatly improves the retrieval efficiency, allowing users to quickly locate the required cosmetics-related information in the massive data. Through the close cooperation of this series of steps, users can be provided with a comprehensive and responsive cosmetics sales knowledge base, which greatly enhances the quality and efficiency of the customer service experience.
[0056] 4. Combine user emotion information, dialogue stage, and cosmetics sales knowledge base content to construct prompts through the Cot-SC method, such as Figure 2 shown.
[0057] 4.1. Conversation stage strategy: Refine the conversation stages and manage and control each stage through specific strategies. From the initial self-introduction to clarifying customer needs, and then to the detailed introduction of products or services, each step has a targeted prompt template designed to ensure that the conversation can transition smoothly and gradually deepen. At the same time, corresponding response prompts are also prepared for scenarios such as unexpected interruptions in the conversation to ensure the continuity and effectiveness of the service.
[0058] 4.2. Use the CoT-SC model to generate and screen a series of logical reasoning chains to construct prompts, so as to ensure the accuracy and relevance of each conversation. In the specific implementation, select appropriate reasoning templates based on the conversation stage, the customer's basic information and emotional state, and the knowledge base content that needs to be conveyed. Then, according to the reasoning chain generated by the model, select the content with high frequency and most consistent with the purpose of the conversation as prompts, so as to ensure the pertinence and efficiency of the conversation strategy.
[0059] An embodiment of the present invention also provides an electronic device, including a memory and a processor, wherein the memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement a method of constructing a prompt based on a multi-stage dialogue of emotional factors in the above embodiment.
[0060] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, and when the program is executed by a processor, a method for constructing a prompt based on a multi-stage dialogue of emotional factors according to the above embodiment is implemented.
[0061] The computer-readable storage medium may be an internal storage unit of any device with data processing capability described in any of the aforementioned embodiments, such as a hard disk or a memory. The computer-readable storage medium may also be any device with data processing capability, such as a plug-in hard disk, a smart media card (SMC), an SD card, a flash card, etc. equipped on the device. Furthermore, the computer-readable storage medium may also include both an internal storage unit of any device with data processing capability and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capability, and may also be used to temporarily store data that has been output or is to be output.
[0062] The above embodiments are only used to illustrate the design ideas and features of the present invention, and their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, any equivalent changes or modifications made based on the principles and design ideas disclosed by the present invention are within the protection scope of the present invention.
Claims
1. A method for constructing prompts based on multi-stage dialogues with emotional factors, characterized in that: The following steps are involved: (1) Identify facial expression information and voice emotion information through the emotion recognition model and use the weighted average method to construct user emotion information with the weight of each expression; (2) Use the random forest method to establish a connection between user emotions and the stage of the conversation through the trained random forest model; The step (2) includes the following sub-steps: (2.1) manually preparing a data set, wherein the feature values in the data set include the weights of user emotions, and the classification labels are the dialogue stages corresponding to different emotions; (2.2) Divide the data set prepared in step (2.1), use 80% of the data as the training set and the remaining 20% as the test set, and ensure that all feature values are numerical; (2.3) Construct a random forest model and train the random forest model using the training set obtained in step (2.2); (2.4) After training, use the test set obtained in step (2.2) to evaluate the accuracy of the random forest model, as well as other indicators including precision, recall, and F1 score; (2.5) Parameter tuning: Based on the evaluation indicators obtained in step (2.4), use the grid search method to automatically adjust the size of the trees in the random forest model to find the optimal result; (3) Build a knowledge base for cosmetics sales; (4) The user emotion information, dialogue stage, and cosmetics sales knowledge base are combined to construct Prompt through the Cot-SC method; The step (4) includes the following sub-steps: (4.1) Dialogue stage template construction: The dialogue stages are refined and divided into self-introduction stage, customer intention judgment stage, product value introduction stage, customer demand analysis stage, solution generation stage, customer objection handling stage, negotiation invitation stage, and abnormal dialogue termination stage. GPT4.0 generates multiple sets of templates for each stage through prompts, which are manually optimized. (4.2) CoT-SC Prompt Construction: Use the CoT-SC method to construct a reasoning chain for the prompt template of each stage after the division in step (4.1), add the user emotion information prepared in step (1) and the knowledge base professional information and user profile information in the cosmetics sales knowledge base constructed in step (3), and select the content with high frequency of occurrence and the highest consistency with the purpose of the conversation as the prompt of this stage.
2. According to the method of claim 1, the method is characterized in that: The step (1) includes the following sub-steps: (1.1) Recognize facial expression information through the VGG19 model and recognize emotional information in speech through the wav2vec-2.0 model. In the recognition process, a labeled data set is used, where different emotion categories are selected as labels in the data set. The emotion categories include happiness, sadness, anger, and surprise. (1.2) Determine the ratio of different emotion weights, and fuse the emotion data obtained from facial expression information and voice information according to the pre-set expression and tone weight ratio.
3. According to claim 2, a method for constructing prompts based on multi-stage dialogues of emotional factors is characterized in that: In the step (1.2), the weight ratio of expression and tone is set to 0.7 for expression and 0.3 for tone.
4. According to the method of claim 1, the method is characterized in that: In the step (2.3), the random forest model is constructed using the scikit-learn machine learning library.
5. According to the method of claim 1, the method is characterized in that: The step (3) includes the following sub-steps: (3.1) Data acquisition: First, build a knowledge base in the common cosmetics sales field, including a cosmetics ingredient library, a cosmetics price library, and a real-time updated coupon library. Then, build a user profile library containing basic user information. (3.2) File preprocessing: Delete common invalid tags, remove stop words and meaningless words, and complete the preprocessing after SpaCy preprocesses the file library and adjusts the consistency of nouns in the text; (3.3) File segmentation and vectorization processing: The text content is segmented according to paragraphs. After the text is segmented, the text blocks are vectorized; (3.4) Build file index: Build an inverted index to speed up the retrieval of relevant information.
6. The method for constructing prompts based on multi-stage dialogues based on emotional factors according to claim 5, characterized in that: In the step (3.3), the vectorization process adopts the M3E model.
7. An electronic device comprising a memory and a processor, characterized in that: The memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement a method for constructing a prompt based on a multi-stage dialogue of emotional factors as described in any one of claims 1-6 above.
8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, a method for constructing a prompt based on a multi-stage dialogue of emotional factors as described in any one of claims 1-6 is implemented.
