Customer service method and system based on virtual digital human, electronic equipment and storage medium
Through the analysis of facial emotions between virtual digital customer service and users, we can determine whether real customer service needs to be involved, which solves the problem of poor communication between virtual digital customer service and language organization capabilities and improves interaction efficiency and service quality.
Patent Information
- Application Number
- CN202411513351.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-28
- Publication Date
- 2024-11-26
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
When faced with users with slightly poor language organization capabilities, existing virtual digital customer service poor communication, resulting in a decline in customer service quality, affecting customer evaluation, and the time for real customer service to intervene is not timely, affecting interactive feedback.
By obtaining the set of image frames of the user's face, performing emotional recognition processing, determining whether the set intervention conditions are met, a real customer service intervention request is generated, and a real customer service intervention request is connected to the real customer service. The preset emotional coefficient formula is used to calculate the emotional coefficient of the partition period, and determining whether a real person is needed. Customer service intervened.
It has achieved timely and proactive intervention in real customer service during the interaction process, reducing the impact of poor communication, improving interaction efficiency, and improving customer service quality.
Smart Images

Figure CN119027129A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of intelligent interaction technology, and more specifically, to a customer service method, system, electronic device and storage medium based on virtual digital human. Background Art
[0002] Virtual digital human customer service is a customer service solution that combines artificial intelligence technology with multimedia interaction. It simulates human appearance and communication behavior to implement traditional face-to-face customer service. It can significantly reduce the manpower and venue expenses related to corporate customer service, and is therefore favored by many companies.
[0003] In the prior art, virtual digital human customer service still has the problem of limited intelligence, especially when facing users with slightly poor language organization skills, there will be obvious communication problems in the entire interaction process, which leads to a decline in customer service quality and affects customer evaluation. At this stage, the above problems are mainly solved by coordinating the interaction of virtual digital human customer service with the interaction of real customer service, that is, most customer service questions are answered through the interaction of virtual digital human customer service. When the virtual digital human customer service cannot meet the needs, the real customer service intervenes. This can not only reduce labor costs, but also ensure the normal progress of customer service work and the overall customer service quality. However, the timing of the intervention of real customer service is generally determined based on the content of the user's voice feedback, which makes the intervention of real customer service untimely, resulting in poor interactive feedback evaluation of virtual digital human customer service.
[0004] In summary, at this stage, there is an urgent need to develop a customer service method, system, electronic device and storage medium based on virtual digital humans to solve the above-mentioned technical problems. Summary of the invention
[0005] An object of the present invention is to provide a new technical solution of a customer service method, system, electronic device and storage medium based on virtual digital human.
[0006] According to a first aspect of the present invention, a customer service method based on a virtual digital human is provided. The methods include: Acquire a set of facial image frames of the user during the interaction between the virtual digital human customer service and the user; Performing recognition processing on each facial image frame in the facial image frame set to obtain a corresponding emotion type set; Based on the emotion type set, it is determined whether a set intervention condition is met, and if the set intervention condition is met, a live customer service intervention request is generated to access live customer service.
[0007] Optionally, the step of obtaining a set of facial image frames of the user during the interaction between the virtual digital human customer service and the user is specifically as follows: During the interaction period between the virtual digital human customer service and the user, an image acquisition module is used to acquire multiple facial image frames of the user at a set acquisition time interval to obtain the facial image frame set.
[0008] Optionally, the performing recognition processing on each facial image frame in the facial image frame set to obtain a corresponding emotion type set specifically includes: Dividing the interaction time period based on the response start time to obtain a plurality of partition time periods; Dividing the facial image frame set into a corresponding plurality of facial image frame subsets according to the divided plurality of partition time periods; Recognition processing is performed on each facial image frame in each of the facial image frame subsets to obtain corresponding multiple emotion type subsets to form the emotion type set.
[0009] Optionally, the step of determining whether a set intervention condition is met based on the emotion type set, and When the set intervention conditions are met, a live customer service intervention request is generated to access live customer service, specifically including: Counting the number of various emotion types in each subset of the emotion types; Substitute the number of various emotion types into the preset emotion coefficient formula to calculate the emotion coefficient of the corresponding partition period; It is determined whether the emotion coefficient corresponding to each partition period meets the set intervention condition, and if the set intervention condition is met, a real customer service intervention request is generated to access the real customer service.
[0010] Optionally, the preset emotion coefficient formula is expressed as: G=(a1*r1+a2*r2+…+a n *r n ) / (a1+a2+…+a n ) Among them, G is the sentiment coefficient of the partition period, a i is the number of emotion types, r i for a i The corresponding preset influence coefficient, n is the number of emotion types, 1≤i≤n.
[0011] Optionally, the setting intervention condition is: There is a partition period corresponding to an emotion coefficient greater than the first emotion coefficient threshold; Alternatively, the emotion coefficients corresponding to k consecutive partition time periods are greater than a second emotion coefficient threshold, wherein the first emotion coefficient threshold is greater than the second emotion coefficient threshold.
[0012] Optionally, when the duration of a partition period is less than a preset duration threshold, the emotion coefficient corresponding to the partition period is not calculated.
[0013] According to a second aspect of the present invention, a customer service system based on a virtual digital human is provided, the system comprising: An image acquisition module is configured to obtain a set of facial image frames of the user during the interaction between the virtual digital human customer service and the user; A facial emotion recognition module is configured to perform recognition processing on each facial image frame in the facial image frame set to obtain a corresponding emotion type set; The controller is configured to determine whether a set intervention condition is met based on the emotion type set, and generate a live customer service intervention request to access live customer service if the set intervention condition is met.
[0014] According to a third aspect of the present invention, there is provided an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of a customer service method based on a virtual digital human as described in the first aspect of the present invention are implemented.
[0015] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of a customer service method based on a virtual digital human as described in the first aspect of the present invention are implemented.
[0016] According to an embodiment disclosed by the present invention, a customer service method and system based on a virtual digital human of the present invention has the following beneficial effects: A customer service method based on virtual digital humans of the present invention analyzes the facial emotions of users interacting with virtual digital customer service to obtain the user's emotional response during the interaction process, and analyzes it as the user's response to the intelligent interaction, thereby obtaining the user's feedback on the intelligent interaction, and can proactively intervene in real customer service in a timely manner when abnormal situations occur, thereby reducing the impact of poor communication and improving interaction efficiency.
[0017] Further features and advantages of the present invention will become apparent from the following detailed description of exemplary embodiments of the present invention with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.
[0019] Figure 1A flow chart of a customer service method based on a virtual digital human provided according to an embodiment Figure 1 ; Figure 2 A flow chart of a customer service method based on a virtual digital human provided according to an embodiment Figure 2 ; Figure 3 A structural block diagram of a customer service system based on a virtual digital human provided according to an embodiment; Figure 4 A schematic diagram of an electronic device. DETAILED DESCRIPTION
[0020] Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present invention unless otherwise specifically stated.
[0021] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.
[0022] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.
[0023] In all examples shown and discussed herein, any specific values should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.
[0024] Embodiment 1: The embodiment of the present invention provides a customer service method based on a virtual digital human, see Figure 1 As shown, the method includes: Step S1: obtaining a set of facial image frames of a user during the interaction between the virtual digital human customer service and the user; Optionally, in step S1 of this embodiment, during the interaction period between the virtual digital human customer service and the user, an image acquisition module is used to acquire multiple facial image frames of the user at a set acquisition time interval to obtain the facial image frame set.
[0025] It should be noted that in this embodiment, the facial image frames of the user during the interaction can be collected through the image acquisition module. Of course, facial image frames of the virtual digital human customer service and the real customer service who intervenes later can also be collected.
[0026] In this embodiment, the image acquisition module may be a camera, and the number of cameras may be one or more, for example: it may include a camera for acquiring user images, a camera for acquiring virtual digital human customer service images, and a camera for acquiring real person customer service images, which is not specifically limited here.
[0027] In this embodiment, the collection time interval may be set to a fixed value, such as 5 seconds, 6 seconds or 7 seconds, etc. It may also be a variable value, and the value of the collection time interval may be changed according to the interaction scenario, and no specific limitation is made here.
[0028] Step S2: performing recognition processing on each facial image frame in the facial image frame set to obtain a corresponding emotion type set; In this embodiment, the facial emotion recognition algorithm in the prior art can be used to perform recognition processing on each facial image frame to determine the corresponding emotion type.
[0029] In this embodiment, the recognized emotion types include one or more of anger, doubt, sadness, calmness, surprise, fear, disgust and joy. It should be noted that each facial image frame can determine an emotion type, and multiple facial image frames can have the same emotion type or different emotion types.
[0030] Optionally, in step S2 of this embodiment, each facial image frame in the facial image frame set is subjected to recognition processing to obtain a corresponding emotion type set, specifically including: Step S21: Divide the interaction time period based on the response start time to obtain multiple partition time periods; In this embodiment, the partition time is set as the start time of each customer service response, that is, the start time of each customer service voice response; of course, the partition time can also be set as the end time of each customer service response, that is, the end time of each customer service voice response. In this embodiment, the time period between two adjacent partition times is marked as a partition period.
[0031] Step S22: dividing the facial image frame set into a corresponding plurality of facial image frame subsets according to the divided plurality of partition time periods; Step S23: performing recognition processing on each facial image frame in each facial image frame subset to obtain corresponding multiple emotion type subsets to form an emotion type set. Here, each emotion type subset is a set of emotion types of multiple facial image frames corresponding to each partition period.
[0032] Step S3: determining whether the set intervention conditions are met based on the emotion type set, and generating a live customer service intervention request to access live customer service if the set intervention conditions are met.
[0033] Optionally, in step S3 of this embodiment, it is determined whether the set Intervention conditions, and when the set intervention conditions are met, generate a live customer service intervention request to access live customer service, specifically including: Step S31: counting the number of various emotion types in each emotion type subset; It should be noted that, in this embodiment, the types of emotion types included in different emotion type subsets may be the same or different, and the number of a certain emotion type included may be the same or different. For example, the first emotion type subset includes 3 emotion types, namely joy, anger, and calmness, and the number of each emotion type recognized is 3, 4, and 5 respectively; the second emotion type subset includes 2 emotion types, namely disgust and doubt, and the number of each emotion type recognized is 7 and 5 respectively.
[0034] Step S32: Substituting the number of various emotion types into the preset emotion coefficient formula to calculate the emotion coefficient of the corresponding partition period; In this embodiment, the preset emotion coefficient formula is expressed as: G=(a1*r1+a2*r2+…+a n *r n ) / (a1+a2+…+a n ) Among them, G is the sentiment coefficient of the partition period, a i is the number of emotion types, r i for a i The corresponding preset influence coefficient, n is the number of emotion types, 1≤i≤n.
[0035] It should be noted that a corresponding preset influence coefficient is set in advance for each emotion type. The setting principle is: the more an emotion type can express the user's dissatisfaction with the customer service response, the greater the corresponding influence coefficient; for example: the preset influence coefficient corresponding to anger is 0.6, the preset influence coefficient corresponding to disgust is 0.4, the preset influence coefficient corresponding to doubt is 0.2, the preset influence coefficient corresponding to calmness is 0, the preset influence coefficient corresponding to joy is -0.2, and so on.
[0036] For example, the first emotion type subset (i.e., the first partition period) contains three emotion types, namely joy, anger, and calmness. The number of each emotion type identified is 3, 4, and 5 respectively. The emotion coefficient of this partition period is: G=(3*(-0.2)+4*0.6+5*0) / (3+4+5)=0.15; the second emotion type subset contains two emotion types, namely disgust and doubt. The number of each emotion type identified is 7 and 5 respectively. The emotion coefficient of this partition period is: G=(7*0.4+5*0.2) / (7+5)=0.316.
[0037] Step S33: determine whether the emotion coefficient corresponding to each partition period meets the set intervention conditions, and if the set intervention conditions are met, generate a real customer service intervention request to access the real customer service.
[0038] Optionally, the intervention condition is set in this embodiment as: there is a partition period corresponding to an emotion coefficient greater than a first emotion coefficient threshold; or, the emotion coefficients corresponding to k consecutive partition periods are greater than a second emotion coefficient threshold, wherein the first emotion coefficient threshold is greater than the second emotion coefficient threshold.
[0039] It should be noted that the first emotion coefficient threshold, the second emotion coefficient threshold and the value of k in this embodiment are determined according to actual conditions and are not specifically limited here. For example, when the preset influence coefficient corresponding to anger is 0.6, the preset influence coefficient corresponding to disgust is 0.4, the preset influence coefficient corresponding to doubt is 0.2, the preset influence coefficient corresponding to calm is 0, and the preset influence coefficient corresponding to joy is -0.2, the first emotion coefficient can be set to 1.2, the second emotion coefficient can be set to 0.8, and the value of k can be 3.
[0040] In this embodiment, when it is determined that the emotion coefficient corresponding to a partition period is greater than the first emotion coefficient threshold; or the emotion coefficients corresponding to k consecutive partition periods are greater than the second emotion coefficient threshold, it means that the virtual digital human customer service response cannot meet the user's needs at this time, and it is necessary to connect to the real customer service for response processing.
[0041] Optionally, in this embodiment, when the duration of a partition period is less than a preset duration threshold, the emotion coefficient corresponding to the partition period is not calculated. This can reduce the impact of the poor representativeness of emotion changes and proportions of various emotions in very short conversations on the judgment results.
[0042] It should be noted that, in this embodiment, the preset duration threshold is determined according to actual conditions and is not specifically limited here. For example, when the collection time interval is 5 seconds, the preset duration threshold may be 8 seconds.
[0043] Optionally, this embodiment further includes, before step S1: generating a virtual digital human customer service, and displaying the virtual digital human customer service on an interactive display device.
[0044] In this embodiment, if the user chooses to activate the virtual digital human customer service during the interactive activity, the virtual digital human customer service is generated by the rendering module and displayed on the interactive display device. In addition, in the process of displaying the virtual digital human customer service, the angle of the virtual digital human customer service can be adjusted by obtaining the facial position of the customer to ensure that the face and / or eyes of the virtual digital human customer service can move corresponding to the face of the customer, and the naturalness of the entire customer service process is ensured. Specifically, the image information of the customer is obtained by the camera installed on the user side, and the position of the user in the activity area, the height of the customer, and the position of the user's face relative to the virtual digital human customer service displayed on the interactive display device are obtained based on the obtained image information of the user, and then the relative position between the virtual digital human customer service and the user's face is obtained, and the facial orientation and / or eye position of the virtual digital human are adjusted according to the relative position.
[0045] Optionally, after step S3, this embodiment further includes: determining a real customer service, and obtaining facial image information of the real customer service through a camera on the customer service side, and then rendering the facial expression of the virtual digital human customer service according to the facial image information of the real customer service.
[0046] See also Figure 2 , the following is a specific example to explain the method of this implementation in detail: In the first step, if the user chooses to activate the virtual digital human customer service during the activity, the virtual digital human customer service is generated by rendering and displayed on the interactive display device; The second step is to obtain the user's facial image frame at a preset time interval t1, perform facial emotion recognition on each facial image frame in time sequence, and obtain the emotion type corresponding to each facial image frame; The third step is to set the partition time. The partition time is the starting time of each customer service response, that is, the starting time of each customer service voice reply. It should be noted that the customer service here can be a virtual digital customer service or a real person customer service.
[0047] Mark the time period between two adjacent partition moments as a partition period, and obtain the emotion type corresponding to each facial image frame in each partition period; The fourth step is to obtain the number of facial image frames expressing various types of emotions in a partition period. For the convenience of expression, the number of facial image frames expressing various types of emotions is expressed as a1, a2, ..., a n, the emotion coefficient G of the corresponding partition period is calculated according to the above preset influence coefficient formula; and the emotion coefficient G corresponding to each partition period is calculated in turn; Step 5: When the emotion coefficient G corresponding to a partition period is greater than the preset value Gy1 (the first emotion coefficient threshold), it is considered that the virtual digital human customer service response cannot meet the user's needs within the partition period; When the emotion coefficient G corresponding to k consecutive partition periods is greater than the preset coefficient Gy2 (the second emotion coefficient threshold), it is also considered that the virtual digital human customer service response cannot meet the user's needs within the corresponding partition period; Step 6: When it is determined that the response of the virtual digital human customer service in the corresponding partition time period cannot meet the user's needs, the real customer service will be actively proposed to intervene. If the user chooses the real customer service to intervene, the next step will be entered. If the user chooses not to intervene, the virtual digital human customer service will continue to provide service; The seventh step is to identify a real customer service representative, obtain the facial image information of the real customer service representative through the camera on the real customer service side, and then render the facial expression of the virtual digital human customer service representative based on the facial image information of the real customer service representative.
[0048] In summary, the customer service method based on virtual digital human in the embodiment of the present invention first obtains the user's facial image frame set during the interaction between the virtual digital human customer service and the user; and recognizes and processes each facial image frame in the facial image frame set to obtain the corresponding emotion type set; finally, based on the emotion type set, determines whether the set intervention condition is met, and generates a real customer service intervention request to access the real customer service if the set intervention condition is met. The method of the embodiment of the present invention analyzes the facial emotions of the user interacting with the virtual digital customer service to obtain the user's emotional response during the interaction process, and analyzes it as the user's response to the intelligent interaction, obtains the user's feedback on the intelligent interaction, and can actively intervene in the real customer service in a timely manner when an abnormal situation occurs, thereby reducing the impact caused by poor communication and improving the interaction efficiency.
[0049] Embodiment 2: See also Figure 3 As shown, an embodiment of the present invention provides a customer service system based on a virtual digital human, the system comprising: An image acquisition module is configured to obtain a set of facial image frames of the user during the interaction between the virtual digital human customer service and the user; A facial emotion recognition module is configured to perform recognition processing on each facial image frame in the facial image frame set to obtain a corresponding emotion type set; The controller is configured to determine whether a set intervention condition is met based on the emotion type set, and generate a live customer service intervention request to access live customer service if the set intervention condition is met.
[0050] Optionally, in the customer service system based on virtual digital humans of this embodiment, the image acquisition module is specifically configured to: during the interaction period between the virtual digital human customer service and the user, obtain multiple facial image frames of the user at a set acquisition time interval to obtain a facial image frame set.
[0051] Optionally, in the customer service system based on virtual digital human of this embodiment, the facial emotion recognition module is specifically configured as follows: The interaction period is divided based on the response start time to obtain multiple partition periods; Dividing the facial image frame set into a corresponding plurality of facial image frame subsets according to the divided plurality of partition time periods; Recognition processing is performed on each facial image frame in each facial image frame subset to obtain corresponding multiple emotion type subsets to form an emotion type set.
[0052] Optionally, in the customer service system based on virtual digital human of this embodiment, the controller is specifically configured Set to: Count the number of various emotion types in each emotion type subset; Substitute the number of various emotion types into the preset emotion coefficient formula to calculate the emotion coefficient of the corresponding partition period; It is determined whether the emotion coefficient corresponding to each partition period meets the set intervention condition, and if the set intervention condition is met, a real customer service intervention request is generated to access the real customer service.
[0053] Optionally, in the customer service system based on virtual digital human in this embodiment, the preset emotion coefficient formula is expressed as: G=(a1*r1+a2*r2+…+a n *r n ) / (a1+a2+…+a n ) Among them, G is the sentiment coefficient of the partition period, a i is the number of emotion types, r i for a i The corresponding preset influence coefficient, n is the number of emotion types, 1≤i≤n.
[0054] Optionally, the intervention conditions in the customer service system based on virtual digital humans in this embodiment are set as follows: There is a partition period corresponding to an emotion coefficient greater than the first emotion coefficient threshold; Alternatively, the emotion coefficients corresponding to k consecutive partition time periods are greater than a second emotion coefficient threshold, wherein the first emotion coefficient threshold is greater than the second emotion coefficient threshold.
[0055] Optionally, in the customer service system based on virtual digital humans in this embodiment, when the duration of a partition period is less than a preset duration threshold, the emotion coefficient corresponding to the partition period is not calculated.
[0056] The customer service system based on virtual digital humans in this embodiment also includes an intelligent interaction module, which is used to collect and obtain the user's voice, convert the user's voice into text information, perform semantic analysis, and provide corresponding answers to questions; a rendering module, which is used to render the dynamic image of the virtual digital human customer service; and an interactive display device, usually a display screen, which is used to display the image of the virtual digital human customer service.
[0057] Among them, the image acquisition module includes a camera installed on the customer side and a camera installed on the live customer service side, which are used to collect image information of users in interactive activities and facial image information of live customer service respectively. The image information collected by the camera on the customer side is transmitted to the facial emotion recognition module, and the image information collected by the camera on the live customer service side is transmitted to the rendering module.
[0058] In summary, the customer service system based on virtual digital humans in the embodiment of the present invention analyzes the facial emotions of users interacting with virtual digital humans to obtain the user's emotional response during the interaction process, and analyzes it as the user's response to the intelligent interaction module, thereby obtaining the user's feedback on the intelligent interaction module. When an abnormal situation occurs (such as the intelligent interaction module cannot give the answer the user needs or the intelligent interaction module does not understand the user's needs), real customer service is promptly and proactively intervened, thereby reducing the impact of poor communication and improving interaction efficiency.
[0059] Embodiment three: The present invention discloses an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of a customer service method based on a virtual digital human in any one of the first embodiments of the present invention are implemented.
[0060] Figure 4 is a structural diagram of an electronic device according to an embodiment of the present invention, such as Figure 4As shown, the electronic device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the electronic device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, near field communication (NFC) or other technologies. The display screen of the electronic device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the housing of the electronic device, or an external keyboard, touchpad or mouse, etc.
[0061] Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a structural diagram of the part related to the technical solution of the present disclosure, and does not constitute a limitation on the electronic device to which the technical solution of the present application is applied. The specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0062] Embodiment 4: The embodiment of the present invention discloses a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any one of the customer service methods based on a virtual digital human in the first embodiment of the present invention are implemented.
[0063] Please note that the technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification. The above embodiments only express several implementation methods of the present application, and their descriptions are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that for ordinary technicians in this field, without departing from the concept of the present application, several variations and improvements can be made, which all belong to the scope of protection of the present application. Therefore, the scope of protection of the patent in this application shall be based on the attached claims.
[0064] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.
[0065] Although some specific embodiments of the present invention have been described in detail by way of example, it will be appreciated by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the present invention. It will be appreciated by those skilled in the art that the above embodiments may be modified without departing from the scope and spirit of the present invention. The scope of the present invention is defined by the appended claims.
Claims
1. A customer service method based on virtual digital human, It is characterized in that The method comprises: Acquire a set of facial image frames of the user during the interaction between the virtual digital human customer service and the user; Performing recognition processing on each facial image frame in the facial image frame set to obtain a corresponding emotion type set; Based on the emotion type set, it is determined whether a set intervention condition is met, and if the set intervention condition is met, a live customer service intervention request is generated to access live customer service.
2. The customer service method based on virtual digital human according to claim 1, It is characterized in that The step of obtaining the facial image frame set of the user during the interaction between the virtual digital human customer service and the user is specifically as follows: During the interaction period between the virtual digital human customer service and the user, an image acquisition module is used to acquire multiple facial image frames of the user at a set acquisition time interval to obtain the facial image frame set.
3. The customer service method based on virtual digital human according to claim 2, It is characterized in that The recognition processing is performed on each facial image frame in the facial image frame set to obtain a corresponding emotion type set, specifically including: Dividing the interaction time period based on the response start time to obtain a plurality of partition time periods; Dividing the facial image frame set into a corresponding plurality of facial image frame subsets according to the divided plurality of partition time periods; Recognition processing is performed on each facial image frame in each of the facial image frame subsets to obtain corresponding multiple emotion type subsets to form the emotion type set.
4. The customer service method based on virtual digital human according to claim 3, It is characterized in that The determining whether a set intervention condition is met based on the emotion type set, and generating a live customer service intervention request to access live customer service if the set intervention condition is met, specifically includes: Counting the number of various emotion types in each of the emotion type subsets; Substitute the number of various emotion types into the preset emotion coefficient formula to calculate the emotion coefficient of the corresponding partition period; It is determined whether the emotion coefficient corresponding to each partition period meets the set intervention condition, and if the set intervention condition is met, a real customer service intervention request is generated to access the real customer service.
5. The customer service method based on virtual digital human according to claim 4, It is characterized in that The preset emotion coefficient formula is expressed as: G=(a 1 *r 1 +a 2 *r 2 +…+a n *r n ) / (a 1 +a 2 +…+a n ) Among them, G is the sentiment coefficient of the partition period, a i is the number of emotion types, r i for a i The corresponding preset influence coefficient, n is the number of emotion types, 1≤i≤n.
6. The customer service method based on virtual digital human according to claim 4, It is characterized in that The intervention conditions are set as follows: There is a partition period corresponding to an emotion coefficient greater than the first emotion coefficient threshold; Alternatively, the emotion coefficients corresponding to k consecutive partition time periods are greater than a second emotion coefficient threshold, wherein the first emotion coefficient threshold is greater than the second emotion coefficient threshold.
7. The customer service method based on virtual digital human according to any one of claims 4 to 6, It is characterized in that When the duration of a partition period is less than a preset duration threshold, the emotion coefficient corresponding to the partition period is not calculated.
8. A customer service system based on virtual digital people, It is characterized in that The system comprises: An image acquisition module is configured to obtain a set of facial image frames of the user during the interaction between the virtual digital human customer service and the user; A facial emotion recognition module is configured to perform recognition processing on each facial image frame in the facial image frame set to obtain a corresponding emotion type set; The controller is configured to determine whether a set intervention condition is met based on the emotion type set, and generate a live customer service intervention request to access live customer service if the set intervention condition is met.
9. An electronic device, It is characterized in that The electronic device includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, the steps in the customer service method based on a virtual digital human described in any one of claims 1 to 7 are implemented.
10. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the steps of the customer service method based on a virtual digital human described in any one of claims 1 to 7 are implemented.
Citation Information
Patent Citations
Property information management method, system and equipment suitable for future community and medium
CN115375508A
E-commerce platform customer service information processing method, server and storage medium
CN117332074A
Customer service assistance service judgment method and device based on user emotional state recognition
CN117746480A
