Adaptive interaction control method and device applied to digital human

Abstract

The application relates to the technical field of computers and discloses a self-adaptive interaction control method and device applied to a digital person, which comprises the following steps: when an initial interaction instruction is detected, determining an interaction personnel corresponding to the initial interaction instruction, collecting multi-dimensional interaction information of the interaction personnel in the process of executing an initial interaction operation according to the initial interaction instruction, determining an interaction demand and a personnel portrait of the interaction personnel according to the initial interaction instruction and the multi-dimensional interaction information of the interaction personnel, generating interaction feedback information according to the interaction demand and the personnel portrait, and driving the digital person to interact with the interaction personnel according to the interaction feedback information. It can be seen that the application can improve the intelligent degree and the individualization level of the digital person interaction, improve the feedback accuracy of the digital person interaction to the user demand, and significantly improve the interaction experience and the satisfaction of the user.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a self-adaptive interaction control method and device applied to digital person. BACKGROUND

[0002] With the rapid development of computer graphics technology, artificial intelligence and human-computer interaction field, digital person technology has gradually moved from theoretical research to practical application, and has become an important part of virtual reality (VR), augmented reality (AR), mixed reality (MR) and other frontier technology fields. As a highly simulated virtual image, digital person not only can provide visual realistic experience, but also can realize real-time interaction with users through complex algorithms and models, so as to show wide application prospect in entertainment, education, medical treatment, business and other fields.

[0003] In the application of digital person interaction technology, the traditional digital person interaction control method often depends on the preset interaction script or fixed interaction process, which limits the interaction flexibility and intelligent level of digital person. When the behavior or demand of the user changes, the traditional interaction method often has difficulty in making timely and appropriate response, thereby reducing the user's interaction experience and satisfaction. Therefore, it is particularly important to propose a technical solution which can improve the intelligent and personalized level of digital person interaction, and also can improve the user's interaction experience and satisfaction. SUMMARY

[0004] The present application provides a self-adaptive interaction control method and device applied to digital person, which can improve the intelligent and personalized level of digital person interaction, and also can improve the user's interaction experience and satisfaction.

[0005] In order to solve the above technical problems, the present application discloses a self-adaptive interaction control method applied to digital person, which comprises:

[0006] When the initial interaction instruction is detected, the interaction personnel corresponding to the initial interaction instruction is determined, and the multi-dimensional interaction information of the interaction personnel is collected in the process of executing the initial interaction operation according to the initial interaction instruction;

[0007] According to the initial interaction instruction and the multi-dimensional interaction information of the interaction personnel, the interaction demand and the personnel portrait of the interaction personnel are determined;

[0008] According to the interaction demand and the personnel portrait, the interaction feedback information is generated, and the digital person is driven to interact with the interaction personnel according to the interaction feedback information.

[0009] As an optional implementation, in the first aspect of the present application, the multi-dimensional interaction information comprises voice information and action information of the interaction personnel, and the personnel portrait comprises a personnel emotion portrait.

[0010] The determining of the interaction demand and the personnel portrait of the interaction personnel according to the initial interaction instruction and the multi-dimensional interaction information of the interaction personnel comprises:

[0011] The voice information of the interaction personnel is subjected to voice recognition to obtain voice text information, and the voice text information is subjected to text analysis to obtain a text keyword set;

[0012] The context information of each keyword in the text keyword set is obtained, and the interaction demand of the interaction personnel is determined according to the context information of each keyword and the initial interaction instruction;

[0013] The voice emotion of the interaction personnel is determined according to the voice information, and the action emotion of the interaction personnel is determined according to the action information;

[0014] The personnel emotion portrait of the interaction personnel is determined according to the voice emotion and the action emotion of the interaction personnel.

[0015] As an optional implementation, in the first aspect of the present application, the action information comprises limb action information, gaze movement information and facial expression information;

[0016] The determining of the voice emotion of the interaction personnel according to the voice information and the determining of the action emotion of the interaction personnel according to the action information comprise:

[0017] The audio feature information of the interaction personnel is determined according to the voice information, and the audio emotion of the interaction personnel is determined according to the audio feature information, the audio feature information comprising audio intensity features and pronunciation intonation features;

[0018] The keyword emotion label corresponding to each keyword in the text keyword set is analyzed, the text emotion of the interaction personnel is determined according to the keyword emotion label corresponding to each keyword, and the voice emotion of the interaction personnel is determined according to the audio emotion and the text emotion;

[0019] The action information is subjected to emotion feature recognition to obtain a first emotion feature corresponding to the limb action information, a second emotion feature corresponding to the gaze movement information and a third emotion feature corresponding to the facial expression information;

[0020] obtaining preset emotion feature weight information corresponding to the action information, and determining the action emotion of the interaction personnel according to the first emotion feature, the second emotion feature, the third emotion feature, and the emotion feature weight information.

[0021] As an optional implementation, in the first aspect of the present application, the determining of the interaction demand of the interaction personnel according to the context information of each keyword and the initial interaction instruction comprises:

[0022] determining the speech integrity of the speech information of the interaction personnel according to the context information of each keyword, and determining the initial interaction demand of the interaction personnel according to the initial interaction instruction;

[0023] judging whether the speech integrity is greater than a preset speech integrity threshold, and when the speech integrity is greater than the speech integrity threshold, determining the interaction demand of the interaction personnel according to the initial interaction demand and the context information of each keyword;

[0024] when the speech integrity is less than the speech integrity threshold, obtaining historical interaction information of the interaction personnel in a preset time period before the current time, the historical interaction information comprising historical speech information and historical interaction demand;

[0025] predicting the remaining speech information of the interaction personnel according to the historical interaction information and the context information of each keyword, and determining the interaction demand of the interaction personnel according to the context information of each keyword, the remaining speech information, and the initial interaction demand.

[0026] As an optional implementation, in the first aspect of the present application, the interaction demand comprises a question answering demand and / or an operation execution demand;

[0027] the generating of the interaction feedback information according to the interaction demand and the personnel portrait comprises:

[0028] when the interaction demand comprises the question answering demand, determining at least one question answer text for the question answering demand;

[0029] generating a question answering voice corresponding to the personnel emotion portrait and a first interaction animation for the digital person according to each question answer text and the personnel emotion portrait, the interaction feedback information comprising the question answering voice and the first interaction animation;

[0030] when the interaction demand comprises the operation execution demand, determining the personnel permission of the interaction personnel and the operation permission of the operation execution demand;

[0031] determine whether the personnel permission matches the operation permission, and generate a permission mismatch prompt voice and a second interaction animation when the personnel permission does not match the operation permission, the interaction feedback information comprising the permission mismatch prompt voice and the second interaction animation;

[0032] determine an operation execution step corresponding to the operation execution requirement when the personnel permission matches the operation permission, and determine a third interaction animation corresponding to each operation execution step, the interaction feedback information comprising the third interaction animation corresponding to each operation execution step.

[0033] As an optional implementation, in the first aspect of the present application, the method further comprises:

[0034] determine an additional interaction requirement of each additional interaction personnel other than the interaction personnel when it is determined that the initial interaction instruction corresponds to at least one additional interaction personnel other than the interaction personnel;

[0035] determine whether the additional interaction requirement of each additional interaction personnel conflicts with the interaction requirement of the interaction personnel, and determine a requirement conflict reason between the additional interaction requirement and the interaction requirement when the additional interaction requirement conflicts with the interaction requirement;

[0036] generate requirement change feedback information for each additional interaction personnel according to the requirement conflict reason between the additional interaction requirement and the interaction requirement, the requirement change feedback information comprising requirement conflict prompt information and requirement change suggestion information;

[0037] determine a requirement feedback time of each additional interaction personnel according to the requirement feedback time of the interaction personnel corresponding to the interaction feedback information and the requirement feedback sequence of each additional interaction personnel;

[0038] determine a requirement feedback time of each additional interaction personnel according to the requirement feedback time and the requirement feedback sequence of the additional interaction personnel, and drive the digital person to prompt the additional interaction personnel according to the requirement feedback time.

[0039] As an optional implementation, in the first aspect of the present application, the method further comprises:

[0040] determine a current interaction scene corresponding to the interaction personnel, and determine a scene privacy of the current interaction scene;

[0041] determine an information privacy degree of the interactive person according to the interaction demand of the interactive person, and determine a current interaction privacy degree of the interactive person according to the scene privacy degree and the information privacy degree;

[0042] For each of the additional interactive persons, obtain the personnel interaction permission of the additional interactive person, and determine whether the personnel interaction permission of the additional interactive person matches the current privacy degree of the interactive person;

[0043] For each of the additional interactive persons, when the personnel interaction permission of the additional interactive person does not match the current privacy degree of the interactive person, generate privacy security feedback information for the interactive person.

[0044] The second aspect of the present application discloses an adaptive interaction control device applied to a digital person, the device comprising:

[0045] A determination module is configured to determine an interactive person corresponding to an initial interaction instruction when the initial interaction instruction is detected, and collect multi-dimensional interaction information of the interactive person in a process of performing an initial interaction operation according to the initial interaction instruction;

[0046] The determination module is further configured to determine an interaction demand and a personnel portrait of the interactive person according to the initial interaction instruction and the multi-dimensional interaction information of the interactive person.

[0047] A generation module is configured to generate interaction feedback information according to the interaction demand and the personnel portrait, and drive the digital person to interact with the interactive person according to the interaction feedback information.

[0048] As an optional implementation, in the second aspect of the present application, the multi-dimensional interaction information comprises voice information and action information of the interactive person, and the personnel portrait comprises a personnel emotion portrait.

[0049] The determination module determines the interaction demand and the personnel portrait of the interactive person according to the initial interaction instruction and the multi-dimensional interaction information of the interactive person in the following manner:

[0050] Voice information of the interactive person is subjected to voice recognition to obtain voice text information, and the voice text information is subjected to text analysis to obtain a text keyword set;

[0051] Context information of each keyword in the text keyword set is obtained, and the interaction demand of the interactive person is determined according to the context information of each keyword and the initial interaction instruction.

[0052] determine a voice emotion of the interaction personnel according to the voice information, and determine a motion emotion of the interaction personnel according to the motion information;

[0053] determine a personnel emotion portrait of the interaction personnel according to the voice emotion and the motion emotion of the interaction personnel.

[0054] As an optional implementation form, in the second aspect, the motion information comprises limb motion information, eye movement information and facial expression information;

[0055] The determining module determines a voice emotion of the interaction personnel according to the voice information, and determines a motion emotion of the interaction personnel according to the motion information.

[0056] determine audio feature information of the interaction personnel according to the voice information, and determine an audio emotion of the interaction personnel according to the audio feature information, wherein the audio feature information comprises audio intensity feature and pronunciation intonation feature;

[0057] analyze a keyword emotion label corresponding to each keyword in the keyword set, determine a text emotion of the interaction personnel according to the keyword emotion label corresponding to each keyword, and determine a voice emotion of the interaction personnel according to the audio emotion and the text emotion;

[0058] perform emotion feature recognition on the motion information to obtain a first emotion feature corresponding to the limb motion information, a second emotion feature corresponding to the eye movement information and a third emotion feature corresponding to the facial expression information;

[0059] obtain preset emotion feature weight information corresponding to the motion information, and determine a motion emotion of the interaction personnel according to the first emotion feature, the second emotion feature, the third emotion feature and the emotion feature weight information.

[0060] As an optional implementation form, in the second aspect, the determining module determines the interaction demand of the interaction personnel according to the context information of each keyword and the initial interaction instruction.

[0061] determine voice integrity of the voice information of the interaction personnel according to the context information of each keyword, and determine an initial interaction demand of the interaction personnel according to the initial interaction instruction;

[0062] determine the interaction demand of the interaction personnel according to the initial interaction demand and the context information of each keyword when the voice integrity is greater than the preset voice integrity threshold.

[0063] obtaining historical interaction information of the interaction personnel in a preset time period before the current time when the voice integrity is less than the voice integrity threshold, the historical interaction information including historical voice information and historical interaction demand;

[0064] predicting remaining voice information of the interaction personnel according to the historical interaction information and context information of each keyword, and determining an interaction demand of the interaction personnel according to the context information of each keyword, the remaining voice information and the initial interaction demand.

[0065] As an optional implementation form, in the second aspect, the interaction demand includes a question answering demand and / or an operation execution demand.

[0066] The generating module generates the interaction feedback information according to the interaction demand and the personnel portrait, and the manner specifically includes:

[0067] when the interaction demand includes the question answering demand, determining at least one question answer text for the question answering demand;

[0068] generating a question answering voice corresponding to the personnel emotional portrait and a first interaction animation for the digital person according to each question answer text and the personnel emotional portrait, the interaction feedback information including the question answering voice and the first interaction animation;

[0069] when the interaction demand includes the operation execution demand, determining a personnel permission of the interaction personnel and an operation permission of the operation execution demand;

[0070] judging whether the personnel permission matches the operation permission, and when the personnel permission does not match the operation permission, generating a permission mismatch prompt voice and a second interaction animation, the interaction feedback information including the permission mismatch prompt voice and the second interaction animation;

[0071] when the personnel permission matches the operation permission, determining an operation execution step corresponding to the operation execution demand, and determining a third interaction animation corresponding to each operation execution step, the interaction feedback information including the third interaction animation corresponding to each operation execution step.

[0072] As an optional implementation form, in the second aspect, the determining module is further configured to, when it is determined that at least one additional interaction personnel other than the interaction personnel corresponds to the initial interaction instruction, determine an additional interaction demand of each additional interaction personnel.

[0073] The apparatus further includes:

[0074] a judgment module, configured to judge, for each of the additional interactive personnel, whether the additional interactive demand of the additional interactive personnel conflicts with the interactive demand of the interactive personnel, and determine a demand conflict cause between the additional interactive demand and the interactive demand when the additional interactive demand conflicts with the interactive demand;

[0075] The generation module is further configured to generate, for each of the additional interactive personnel, demand change feedback information for the additional interactive personnel according to the demand conflict cause between the additional interactive demand and the interactive demand, the demand change feedback information including demand conflict prompt information and demand change suggestion information.

[0076] The determination module is further configured to determine a demand feedback time consumption of the interactive personnel according to the interactive feedback information corresponding to the interactive personnel, and determine a demand feedback sequence of each of the additional interactive personnel according to the additional interactive demand of each of the additional interactive personnel.

[0077] The determination module is further configured to determine, for each of the additional interactive personnel, a demand feedback time of the additional interactive personnel according to the demand feedback time consumption and the demand feedback sequence of the additional interactive personnel, and drive the digital person to prompt the additional interactive personnel according to the demand feedback time.

[0078] As an optional implementation, in the second aspect of the present application, the determination module is further configured to determine a current interactive scenario corresponding to the interactive personnel, and determine a scene privacy degree of the current interactive scenario.

[0079] The determination module is further configured to determine an information privacy degree of the interactive personnel according to the interactive demand of the interactive personnel, and determine a current interactive privacy degree of the interactive personnel according to the scene privacy degree and the information privacy degree.

[0080] The device further comprises:

[0081] The acquisition module is configured to acquire, for each of the additional interactive personnel, a personnel interactive permission of the additional interactive personnel, and judge whether the personnel interactive permission of the additional interactive personnel matches the current privacy degree of the interactive personnel.

[0082] The generation module is further configured to generate, for each of the additional interactive personnel, privacy security feedback information for the interactive personnel when the personnel interactive permission of the additional interactive personnel does not match the current privacy degree of the interactive personnel.

[0083] The third aspect of the present application discloses another adaptive interactive control device applied to a digital person, the device comprising:

[0084] a memory storing executable program code;

[0085] a processor coupled with the memory;

[0086] The processor invokes the executable program code stored in the memory to execute the adaptive interactive control method for digital people disclosed in the first aspect of the application.

[0087] The fourth aspect of the application discloses a computer storage medium storing computer instructions, which are invoked to execute the adaptive interactive control method for digital people disclosed in the first aspect of the application.

[0088] Compared with the prior art, the embodiments of the application have the following beneficial effects:

[0089] In the embodiments of the application, when the initial interactive instruction is detected, the interactive personnel corresponding to the initial interactive instruction is determined, and in the process of executing the initial interactive operation according to the initial interactive instruction, the multi-dimensional interactive information of the interactive personnel is collected, the interactive demand and the personnel portrait of the interactive personnel are determined according to the initial interactive instruction and the multi-dimensional interactive information of the interactive personnel, the interactive feedback information is generated according to the interactive demand and the personnel portrait, and the digital person and the interactive personnel are driven to interact according to the interactive feedback information. It can be seen that the implementation of the application can improve the intelligent degree and the individualization level of the digital person interaction, improve the feedback accuracy of the digital person interaction to the user demand, and also can significantly improve the user's interactive experience and satisfaction. BRIEF DESCRIPTION OF DRAWINGS

[0090] In order to more clearly illustrate the technical solutions in the embodiments of the application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0091] Figure 1 is a flowchart of the adaptive interactive control method for digital people disclosed by the embodiments of the application;

[0092] Figure 2 is a flowchart of another adaptive interactive control method for digital people disclosed by the embodiments of the application;

[0093] Figure 3 is a structural schematic diagram of the adaptive interactive control device for digital people disclosed by the embodiments of the application;

[0094] Figure 4is another structural schematic view of the adaptive interactive control device applied to a digital human disclosed by the embodiment of the present application.

[0095] Figure 5 is another structural schematic view of the adaptive interactive control device applied to a digital human disclosed by the embodiment of the present application. DETAILED DESCRIPTION

[0096] In order to enable persons skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by persons skilled in the art without creative labor fall within the protection scope of the present application.

[0097] The terms "first", "second", and the like in the specification and claims of the present application and the above-mentioned drawings are used to distinguish different objects, and are not used to describe a specific order. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, device, product, or end including a series of steps or units is not limited to the listed steps or units, but can optionally include steps or units not listed, or can optionally include other steps or units inherent to the process, method, product, or end.

[0098] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily all refer to the same embodiment, nor is it necessarily mutually exclusive of other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0099] The present application discloses an adaptive interactive control method and device applied to a digital human, which can improve the intelligent degree and individualization level of digital human interaction, improve the feedback accuracy of digital human interaction to user demand, and significantly improve the user's interaction experience and satisfaction. The following will be described in detail.

[0100] Embodiment one

[0101] Please refer to Figure 1 , Figure 1 is a flowchart of an adaptive interactive control method applied to a digital human disclosed by the embodiment of the present application. Among them, Figure 1The adaptive interaction control method applied to a digital person described can be applied to an adaptive interaction control device applied to a digital person, which can include a smart device for interacting with a user, the smart device can include any one of a user device, a smart server or a smart platform, the user device can internally embed a digital person program for interacting with the user, specifically, the user device includes at least one of a smart phone (Android phone, iOS phone, etc.) with a digital person interaction function, a smart phone number watch, and a smart home device, etc., the smart server includes a local server or a cloud server, and the embodiments of the present application are not limited. As shown in Figure 1 The adaptive interaction control method applied to a digital person can include the following operations:

[0102] 101. When the initial interaction instruction is detected, determine the interaction personnel corresponding to the initial interaction instruction, and in the process of executing the initial interaction operation according to the initial interaction instruction, collect multi-dimensional interaction information of the interaction personnel.

[0103] In the embodiments of the present application, optionally, the initial interaction instruction can include a wake-up instruction for the digital person, or a quick wake-up instruction for the digital person set by the interaction personnel in advance, such as one or more combinations of a quick voice wake-up instruction, a quick action wake-up instruction, a quick expression wake-up instruction, and a quick key wake-up instruction, etc., and the present application is not limited.

[0104] In the embodiments of the present application, optionally, the interaction personnel corresponding to the initial interaction instruction can include a single interaction personnel, or a plurality of interaction personnel, when a plurality of interaction personnel are included, a master interaction personnel and a plurality of slave interaction personnel can be determined in each interaction personnel according to the position information, the permission information, the instruction level information, etc. of each interaction personnel, the initial interaction operation can include an operation of waking up the digital person according to the initial interaction instruction, and the multi-dimensional interaction information of the interaction personnel can include voice information and action information of the interaction personnel, wherein the action information can be obtained by image video collection of the interaction personnel, or can be obtained by action scanning detection of the interaction personnel, the action information can include one or more combinations of limb action information, gaze movement information, and facial expression information, etc., and the present application is not limited.

[0105] 102. Determine the interaction demand and the personnel portrait of the interaction personnel according to the initial interaction instruction and the multi-dimensional interaction information of the interaction personnel.

[0106] In the embodiment of the present application, optionally, the interaction demand of the interaction personnel can include a question answering demand and / or an operation execution demand, wherein the question answering demand can include a demand of the interaction personnel asking a question to the digital person and asking the digital person to answer the question, for example, the interaction personnel can ask the digital person about the current weather or time after waking up the digital person, and the digital person answers the question according to the question; for another example, the interaction personnel can ask the digital person about the action of a certain dance or a certain exercise after waking up the digital person, and the digital person performs the action according to the question; the operation execution demand can include a demand of the interaction personnel asking the digital person about the execution steps of a certain operation, for example, the interaction personnel can ask the digital person about how to use the function of a certain software or ask the digital person about the cooking steps of a certain dish after waking up the digital person, and the digital person determines the specific steps according to the question and guides the interaction personnel to operate, which is not limited by the present application.

[0107] In the embodiment of the present application, optionally, the personnel portrait of the interaction personnel can include a personnel emotion portrait of the interaction personnel, wherein the personnel emotion portrait can include a current emotion portrait of the interaction personnel in the process of performing the initial interaction operation according to the initial interaction instruction, and can also include a label emotion portrait corresponding to the interaction personnel determined according to a plurality of historical emotion portraits of the interaction personnel, which is not limited by the present application.

[0108] 103. According to the interaction demand and the personnel portrait, interaction feedback information is generated, and the digital person is driven to interact with the interaction personnel according to the interaction feedback information.

[0109] In the embodiment of the present application, optionally, the interaction feedback information includes interaction feedback voice and interaction feedback animation for the interaction demand of the interaction personnel, wherein the interaction feedback voice is used to play feedback to the interaction personnel through the digital person, and the digital person performs corresponding feedback action according to the interaction feedback animation, the content of the interaction feedback voice and the interaction feedback animation corresponds to the personnel portrait of the interaction personnel, which is not limited by the present application.

[0110] It can be seen that the implementation Figure 1 The described adaptive interaction control method applied to the digital person can determine the interaction personnel corresponding to the initial interaction instruction when detecting the initial interaction instruction, collect multi-dimensional interaction information of the interaction personnel in the process of performing the initial interaction operation according to the initial interaction instruction, determine the interaction demand and the personnel portrait of the interaction personnel according to the initial interaction instruction and the multi-dimensional interaction information of the interaction personnel, generate interaction feedback information according to the interaction demand and the personnel portrait, and drive the digital person to interact with the interaction personnel according to the interaction feedback information, which can improve the intelligent degree and the individualization level of the digital person interaction, improve the feedback accuracy of the digital person interaction to the user demand, and also can significantly improve the interaction experience and satisfaction of the user.

[0111] In an optional embodiment, the adaptive interaction control method applied to the digital human can further include the following operations:

[0112] When it is determined that at least one additional interaction personnel other than the interaction personnel corresponds to the initial interaction instruction, the additional interaction demand of each additional interaction personnel is determined.

[0113] For each additional interaction personnel, it is determined whether the additional interaction demand of the additional interaction personnel conflicts with the interaction demand of the interaction personnel, and when the additional interaction demand conflicts with the interaction demand, the demand conflict reason between the additional interaction demand and the interaction demand is determined.

[0114] For each additional interaction personnel, demand change feedback information for the additional interaction personnel is generated according to the demand conflict reason between the additional interaction demand and the interaction demand, and the demand change feedback information includes demand conflict prompt information and demand change suggestion information.

[0115] According to the interaction feedback information corresponding to the interaction personnel, the demand feedback time of the interaction personnel is determined, and according to the additional interaction demand of each additional interaction personnel, the demand feedback sequence of each additional interaction personnel is determined.

[0116] For each additional interaction personnel, the demand feedback time of the additional interaction personnel is determined according to the demand feedback time and the demand feedback sequence of the additional interaction personnel, and the digital human is driven to prompt the additional interaction personnel according to the demand feedback time.

[0117] In the optional embodiment, when it is determined that at least one additional interaction personnel other than the interaction personnel corresponds to the initial interaction instruction, the additional interaction demand of each additional interaction personnel is determined, the demand response priority of each additional interaction personnel is less than the demand response priority of the interaction personnel, for each additional interaction personnel, it is determined whether the additional interaction demand of the additional interaction personnel conflicts with the interaction demand of the interaction personnel, when the additional interaction demand conflicts with the interaction demand, the demand conflict reason between the additional interaction demand and the interaction demand is determined, the demand conflict reason can include the demand implementation logic conflict reason between the additional interaction demand and the interaction demand, for example, the interaction demand of the interaction personnel is to increase the volume, and the additional interaction demand of the additional interaction personnel is to decrease the volume or turn off the sound, according to the demand conflict reason between the additional interaction demand and the interaction demand, the demand change feedback information for the additional interaction personnel is generated, and the demand change feedback information can include demand conflict prompt information and demand change suggestion information, the demand conflict prompt information can be used to prompt that the additional interaction demand of the additional interaction personnel conflicts with the interaction demand, and the demand change suggestion information can be used to provide the suggestion of demand change for the additional interaction personnel, which is not limited in the embodiment.

[0118] In the optional embodiment, optionally, the demand feedback time corresponding to the interactive personnel is determined according to the interactive feedback information corresponding to the interactive personnel. Specifically, the voice time for the digital human to complete the interactive feedback voice and the animation time for the digital human to complete the interactive feedback animation can be determined, and the longer one of the voice time and the animation time is determined as the demand feedback time corresponding to the interactive personnel. The demand feedback order of each additional interactive personnel can be determined according to the additional interactive demand of each additional interactive personnel. Specifically, the demand feedback order of each additional interactive personnel can be determined according to the content and implementation complexity of the additional interactive demand of each additional interactive personnel, or the demand feedback order of each additional interactive personnel can be determined according to the additional interactive demand of each additional interactive personnel and the demand response priority of each additional interactive personnel. The demand feedback time and the demand feedback order of the additional interactive personnel are determined to determine the demand feedback time of the additional interactive personnel, and the digital human is driven to prompt the additional interactive personnel according to the demand feedback time, so that the additional interactive personnel knows the specific feedback time of the additional interactive demand. This embodiment is not limited.

[0119] It can be seen that the optional embodiment can prioritize the interactive demand of the interactive personnel when there are multiple additional interactive personnel other than the interactive personnel, improve the interactive experience and satisfaction of the interactive personnel, and while feeding back the interactive demand of the interactive personnel, the demand feedback time of each additional interactive personnel can be prompted according to the demand feedback order of the additional interactive personnel and the demand feedback time corresponding to the interactive personnel, so that each additional interactive personnel knows the demand feedback progress information, and the interactive experience and satisfaction of the additional interactive personnel are improved.

[0120] In another optional embodiment, the adaptive interactive control method applied to the digital human can further include the following operations:

[0121] The current interactive scene corresponding to the interactive personnel is determined, and the scene privacy of the current interactive scene is determined.

[0122] The information privacy of the interactive personnel is determined according to the interactive demand of the interactive personnel, and the current interactive privacy of the interactive personnel is determined according to the scene privacy and the information privacy.

[0123] For each additional interactive personnel, the personnel interaction permission of the additional interactive personnel is obtained, and it is judged whether the personnel interaction permission of the additional interactive personnel matches the current privacy of the interactive personnel.

[0124] For each additional interactive personnel, when the personnel interaction permission of the additional interactive personnel does not match the current privacy of the interactive personnel, the privacy security feedback information for the interactive personnel is generated.

[0125] In the optional embodiment, optionally, the current interaction scene corresponding to the interaction personnel can include a family scene, an office scene, an outdoor scene, a meeting scene, a car-riding scene, etc., different interaction scenes have different scene privacy degrees, and the scene privacy degrees of different interaction scenes can be set by the user, the information privacy degree of the interaction personnel can represent the privacy degree of the information content corresponding to the interaction demand of the interaction personnel, and the current interaction privacy degree of the interaction personnel is determined according to the scene privacy degree and the information privacy degree, which is not limited in the embodiment.

[0126] In the optional embodiment, optionally, when the personnel interaction permission of the additional interaction personnel does not match the current privacy degree of the interaction personnel, it can be indicated that the additional interaction personnel has no permission to know the content of the interaction demand of the interaction personnel, or the interaction personnel can not want the additional interaction personnel to know the content of the interaction demand of the interaction personnel, at this time, the privacy security feedback information for the interaction personnel is generated, and after receiving the privacy security feedback information, the interaction personnel can ask the additional interaction personnel to avoid, or authorize the additional interaction personnel to know the content of the interaction demand of the interaction personnel, which is not limited in the embodiment.

[0127] It can be seen that by implementing the optional embodiment, the current interaction scene corresponding to the interaction personnel can be determined, the scene privacy degree of the current interaction scene is determined, the information privacy degree of the interaction personnel is determined according to the interaction demand of the interaction personnel, and the current interaction privacy degree of the interaction personnel is determined according to the scene privacy degree and the information privacy degree, for each additional interaction personnel, the personnel interaction permission of the additional interaction personnel is obtained, and it is judged whether the personnel interaction permission of the additional interaction personnel matches the current privacy degree of the interaction personnel, for each additional interaction personnel, when the personnel interaction permission of the additional interaction personnel does not match the current privacy degree of the interaction personnel, the privacy security feedback information for the interaction personnel is generated, which can remind the interaction personnel of the privacy security in the digital human interaction process, protect the privacy security of the interaction personnel, improve the safety and reliability of the digital human interaction process, and further improve the interaction experience and satisfaction of the user.

[0128] Embodiment two

[0129] Please refer to Figure 2 , Figure 2 is a flowchart of an adaptive interaction control method applied to a digital human according to an embodiment of the application. In the flowchart, Figure 2The adaptive interaction control method applied to a digital person described can be applied to an adaptive interaction control device applied to a digital person, which can include a smart device for interacting with a user, the smart device can include any one of a user device, a smart server or a smart platform, the user device can be built-in with a digital person program for interacting with the user, specifically, the user device includes but is not limited to at least one of a smart phone (Android phone, iOS phone, etc.) with a digital person interaction function, a smart phone number watch and a smart home device, etc., the smart server includes a local server or a cloud server, and the embodiments of the present application are not limited. As shown in Figure 2 The adaptive interaction control method applied to a digital person can include the following operations:

[0130] 201. When the initial interaction instruction is detected, the interaction personnel corresponding to the initial interaction instruction is determined, and the multi-dimensional interaction information of the interaction personnel is collected in the process of executing the initial interaction operation according to the initial interaction instruction.

[0131] 202. The speech information of the interaction personnel is subjected to speech recognition to obtain speech text information, and the speech text information is subjected to text analysis to obtain a text keyword set.

[0132] In the embodiments of the present application, optionally, the multi-dimensional interaction information of the interaction personnel can include speech information and action information of the interaction personnel, wherein the action information can be obtained by image video collection on the interaction personnel, or can be obtained by action scanning detection on the interaction personnel, the action information can include one or more combinations of limb action information, eye movement information and facial expression information, the speech information of the interaction personnel can be subjected to speech recognition to obtain speech text information, and the speech text information can be subjected to text analysis to obtain a text keyword set, the text keyword set can include a plurality of keywords, further, a sensitive word set pre-set by the interaction personnel can be obtained, and the sensitive word set is marked on the text keyword set according to the sensitive word set, and the present application is not limited.

[0133] 203. The context information of each keyword in the text keyword set is obtained, and the interaction demand of the interaction personnel is determined according to the context information of each keyword and the initial interaction instruction.

[0134] In the embodiment of the present application, optionally, the context information of each keyword in the text keyword set can include context semantic information and / or context context information of the keyword, and the interaction demand of the interaction personnel is determined according to the context information of each keyword and the initial interaction instruction. The interaction demand of the interaction personnel can include a question answering demand and / or an operation execution demand. The question answering demand can include a demand of the interaction personnel to ask a question to the digital person and ask the digital person to answer the question. The operation execution demand can include a demand of the interaction personnel to ask the digital person for the execution steps of an operation. The present application is not limited thereto.

[0135] 204. Determine the voice emotion of the interaction personnel according to the voice information, and determine the action emotion of the interaction personnel according to the action information.

[0136] In the embodiment of the present application, optionally, the voice emotion of the interaction personnel can represent the sound emotion characteristics of the interaction personnel when issuing the voice instruction, and the action emotion can represent the action emotion characteristics of the interaction personnel in the process of performing the initial interaction operation according to the initial interaction instruction. The present application is not limited thereto.

[0137] 205. Determine the personnel emotion portrait of the interaction personnel according to the voice emotion and the action emotion of the interaction personnel.

[0138] In the embodiment of the present application, optionally, the personnel emotion portrait can include the current emotion portrait of the interaction personnel in the process of performing the initial interaction operation according to the initial interaction instruction. Further, the personnel emotion portrait can also include a label emotion portrait corresponding to the interaction personnel determined according to a plurality of historical emotion portraits of the interaction personnel. The present application is not limited thereto.

[0139] 206. Generate interaction feedback information according to the interaction demand and the personnel portrait, and drive the digital person to interact with the interaction personnel according to the interaction feedback information.

[0140] In the embodiment of the present application, other descriptions of steps 201 and 206 are described in detail in the embodiment one with reference to steps 101 and 103. The embodiment of the present application will not be repeated here.

[0141] It can be seen that the embodiment of the present application can realize the following effects. Figure 2The adaptive interaction control method applied to the digital person can determine the interaction personnel corresponding to the initial interaction instruction when detecting the initial interaction instruction, and in the process of executing the initial interaction operation according to the initial interaction instruction, collect multi-dimensional interaction information of the interaction personnel, perform speech recognition on the speech information of the interaction personnel to obtain speech text information, perform text analysis on the speech text information to obtain a text keyword set, obtain context information of each keyword in the text keyword set, and determine the interaction demand of the interaction personnel according to the context information of each keyword and the initial interaction instruction. The interaction demand of the interaction personnel can be determined according to the speech information of the interaction personnel, the accuracy and reliability of determining the interaction demand are improved, the speech emotion of the interaction personnel is determined according to the speech information, and the action emotion of the interaction personnel is determined according to the action information. The personnel emotion portrait of the interaction personnel is determined according to the speech emotion and the action emotion of the interaction personnel, the accuracy and reliability of determining the personnel emotion portrait are improved, and the feedback accuracy of the digital person to the demand of the interaction personnel is improved. The interaction demand and the personnel portrait of the interaction personnel are determined according to the initial interaction instruction and the multi-dimensional interaction information of the interaction personnel, the interaction feedback information is generated according to the interaction demand and the personnel portrait, and the digital person is driven to interact with the interaction personnel according to the interaction feedback information. The intelligent degree and the individualization level of the digital person interaction are improved, the feedback accuracy of the digital person interaction to the demand of the user is improved, and the interaction experience and the satisfaction of the user are also significantly improved.

[0142] In an optional embodiment, the action information includes limb action information, gaze movement information, and facial expression information.

[0143] Determining the speech emotion of the interaction personnel according to the speech information and determining the action emotion of the interaction personnel according to the action information can include the following operations:

[0144] Determining the audio feature information of the interaction personnel according to the speech information and determining the audio emotion of the interaction personnel according to the audio feature information, the audio feature information including audio intensity features and pronunciation intonation features;

[0145] Analyzing the keyword emotion label corresponding to each keyword in the text keyword set, determining the text emotion of the interaction personnel according to the keyword emotion label corresponding to each keyword, and determining the speech emotion of the interaction personnel according to the audio emotion and the text emotion;

[0146] Performing emotion feature recognition on the action information to obtain first emotion features corresponding to the limb action information, second emotion features corresponding to the gaze movement information, and third emotion features corresponding to the facial expression information;

[0147] The preset emotion feature weight information corresponding to the action information is acquired, and the action emotion of the interactive person is determined according to the first emotion feature, the second emotion feature, the third emotion feature and the emotion feature weight information.

[0148] In the optional embodiment, the action information can include one or more combinations of limb action information, line-of-sight movement information and facial expression information, and the action information can be obtained by image video acquisition on the interactive person and analysis on the acquired image video, or can be obtained by action scanning detection on the interactive person, such as infrared scanning, and the audio feature information of the interactive person can include audio intensity feature and pronunciation tone feature, the audio intensity feature can represent the voice size feature of the interactive person, and the pronunciation tone feature can represent the timbre and tone of the interactive person, which is not limited in the embodiment.

[0149] In the optional embodiment, each keyword in the text keyword set can correspond to at least one keyword emotion label, and the voice emotion of the interactive person is determined according to the audio emotion and the text emotion, wherein, optionally, the first emotion intensity corresponding to the audio emotion and the second emotion intensity corresponding to the text emotion of the interactive person can be determined, the first emotion coefficient for the audio emotion is determined according to the first emotion intensity, the second emotion coefficient for the text emotion is determined according to the second emotion intensity, and the voice emotion of the interactive person is determined according to the audio emotion, the text emotion, the first emotion coefficient and the second emotion coefficient, which is not limited in the embodiment.

[0150] In the optional embodiment, the emotion feature recognition on the action information can determine the emotion feature corresponding to the action information according to the action amplitude and the action duration corresponding to the action information, including the first emotion feature corresponding to the limb action information, the second emotion feature corresponding to the line-of-sight movement information and the third emotion feature corresponding to the facial expression information, and the emotion feature weight information corresponding to the action information can include the first emotion feature weight corresponding to the limb action information, the second emotion feature weight corresponding to the line-of-sight movement information and the third emotion feature weight corresponding to the facial expression information, and the action emotion of the interactive person is determined according to the first emotion feature, the second emotion feature, the third emotion feature, the first emotion feature weight, the second emotion feature weight and the third emotion feature weight, which is not limited in the embodiment.

[0151] It can be seen that the optional embodiment can determine the audio emotion of the interactive personnel according to the audio feature information of the interactive personnel, determine the text emotion of the interactive personnel according to the audio text, and further determine the speech emotion of the interactive personnel, thereby improving the accuracy of the determined speech emotion and the closeness of the speech emotion to the real emotion of the interactive personnel, determining the action emotion of the interactive personnel according to the action emotion features of different parts of the interactive personnel, thereby improving the accuracy and reliability of the determined action emotion, and further improving the accuracy of the subsequent determination of the personnel emotion portrait of the interactive personnel.

[0152] In another optional embodiment, determining the text emotion of the interactive personnel according to the keyword emotion label corresponding to each keyword can include the following operations:

[0153] According to the at least one keyword emotion label corresponding to each keyword, determining the label appearance ratio of each keyword emotion label;

[0154] For each keyword emotion label, when the label appearance ratio of the keyword emotion label is greater than a preset ratio threshold, determining the keyword emotion label as a target keyword emotion label;

[0155] Determining an emotion label vector of each target keyword emotion label, and determining the similarity between the emotion label vector of each target keyword emotion label and a preset standard emotion label vector;

[0156] According to the similarity between the emotion label vector of each target keyword emotion label and the preset standard emotion label vector, determining an emotion label weight for each target keyword emotion label;

[0157] According to the emotion label weight for each target keyword emotion label and each target keyword emotion label, determining the text emotion of the interactive personnel.

[0158] In the optional embodiment, optionally, the label appearance ratio of each keyword emotion label can be calculated according to the number of occurrences of each keyword emotion label and the total number of keyword emotion labels, and when the label appearance ratio of the keyword emotion label is greater than a preset ratio threshold, it indicates that the emotion corresponding to the keyword emotion label is strong in the speech text of the interactive personnel. This embodiment is not limited.

[0159] In the optional embodiment, optionally, the similarity between the emotion label vector of each target keyword emotion label and the preset standard emotion label vector can represent the emotional closeness between the target keyword emotion label and the preset standard emotion label, and according to the similarity, the emotion label weight for each target keyword emotion label can be determined, which can improve the accuracy of determining the text emotion. This embodiment is not limited.

[0160] It can be seen that, by implementing the optional embodiment, the target keyword emotion label can be determined according to the label occurrence ratio of at least one keyword emotion label corresponding to each keyword in the speech text of the interactive person, the emotion label weight for each target keyword emotion label can be determined according to the similarity between the emotion label vector of each target keyword emotion label and the preset standard emotion label vector, and the text emotion of the interactive person can be determined according to the emotion label weight for each target keyword emotion label and each target keyword emotion label, so that the accuracy and reliability of determining the text emotion of the interactive person can be improved, and the text emotion of the interactive person can be obtained by analyzing the keyword label data, thereby improving the efficiency of determining the text emotion of the interactive person.

[0161] In yet another optional embodiment, determining the interaction demand of the interactive person according to the context information of each keyword and the initial interaction instruction can include the following operations:

[0162] determining the speech integrity of the speech information of the interactive person according to the context information of each keyword, and determining the initial interaction demand of the interactive person according to the initial interaction instruction;

[0163] determining whether the speech integrity is greater than a preset speech integrity threshold, and when the speech integrity is greater than the speech integrity threshold, determining the interaction demand of the interactive person according to the initial interaction demand and the context information of each keyword;

[0164] when the speech integrity is less than the speech integrity threshold, obtaining historical interaction information of the interactive person within a preset time period before the current time, the historical interaction information including historical speech information and historical interaction demand;

[0165] predicting the remaining speech information of the interactive person according to the historical interaction information and the context information of each keyword, and determining the interaction demand of the interactive person according to the context information of each keyword, the remaining speech information and the initial interaction demand.

[0166] In the optional embodiment, optionally, the context information of each keyword in the text keyword set can include context semantic information and / or context context information of the keyword, the semantic integrity of the speech information of the interactive person can be determined according to the context semantic information of each keyword, and then the speech integrity of the speech information of the interactive person can be determined, the initial interaction demand of the interactive person can include the demand of awakening the digital person and / or the demand of specific target instruction, for example, the demand of querying target information, when the speech integrity is greater than the speech integrity threshold, it indicates that the interactive person has described the demand information that needs the digital person to feedback, at this time, the interaction demand of the interactive person can be determined according to the initial interaction demand and the context information of each keyword, which is not limited in the embodiment.

[0167] In the optional embodiment, optionally, when the voice integrity is less than the voice integrity threshold, it indicates that the interactive personnel does not describe the complete demand information requiring the digital human to feedback, or there is a part of unclear situation in the voice of the interactive personnel, at this time, the remaining voice information of the interactive personnel can be predicted according to the historical interaction information of the interactive personnel in the preset time period before the current time and the context information of each keyword, for example, the historical interaction information of the interactive personnel in the 30 minutes before the current time, the historical interaction information can include historical voice information and historical interaction demand, and the interaction demand of the interactive personnel is determined according to the context information of each keyword, the remaining voice information and the initial interaction demand, which is not limited in the embodiment.

[0168] It can be seen that by implementing the optional embodiment, the voice integrity of the voice information of the interactive personnel can be determined according to the context information of each keyword, the initial interaction demand of the interactive personnel can be determined according to the initial interaction instruction, when the voice integrity is greater than the voice integrity threshold, the interaction demand of the interactive personnel can be determined according to the initial interaction demand and the context information of each keyword, when the voice integrity is less than the voice integrity threshold, the historical interaction information of the interactive personnel in the preset time period before the current time is obtained, the remaining voice information of the interactive personnel is predicted according to the historical interaction information and the context information of each keyword, and the interaction demand of the interactive personnel is determined according to the context information of each keyword, the remaining voice information and the initial interaction demand, the voice content can be automatically predicted and filled in the case that the voice of the interactive personnel is not complete or clear, and then the interaction demand of the interactive personnel is determined, the accuracy and flexibility of determining the interaction demand are improved, and then the interaction experience and satisfaction of the user are improved.

[0169] In yet another optional embodiment, the interaction demand includes a question answering demand and / or an operation execution demand;

[0170] Generating the interaction feedback information according to the interaction demand and the personnel portrait can include the following operations:

[0171] When the interaction demand includes the question answering demand, determining at least one question answer text for the question answering demand;

[0172] According to each question answer text and the personnel emotion portrait, a question answering voice corresponding to the personnel emotion portrait and a first interaction animation for the digital human are generated, and the interaction feedback information includes the question answering voice and the first interaction animation;

[0173] When the interaction demand includes the operation execution demand, determining a personnel permission of the interactive personnel and an operation permission of the operation execution demand;

[0174] determine whether the personnel authority matches the operation authority, and when the personnel authority does not match the operation authority, generate a permission mismatch prompt voice and a second interaction animation, and the interaction feedback information includes the permission mismatch prompt voice and the second interaction animation;

[0175] When the personnel authority matches the operation authority, determine an operation execution step corresponding to the operation execution demand, and determine a third interaction animation corresponding to each operation execution step, and the interaction feedback information includes the third interaction animation corresponding to each operation execution step.

[0176] In this optional embodiment, optionally, the interaction demand of the interactive personnel can include a question answering demand and / or an operation execution demand. When the interaction demand includes the question answering demand, at least one question answer text for the question answering demand is determined, and a question answering voice corresponding to the personnel emotional portrait and a first interaction animation for the digital person are generated according to each question answer text and the personnel emotional portrait. The voice emotion and voice tone of the question answering voice are consistent with the personnel emotional portrait. The first interaction animation represents the digital person animation displayed to the interactive personnel, that is, the digital person displays actions according to the first interaction animation while playing the question answering voice. Optionally, the digital person can be displayed according to different devices of the interactive personnel, including but not limited to device screen display, holographic projection display, smart glasses wearing display, etc. This embodiment is not limited.

[0177] In this optional embodiment, optionally, when the interaction demand includes the operation execution demand, the personnel authority of the interactive personnel and the operation authority of the operation execution demand are determined. When the personnel authority does not match the operation authority, a permission mismatch prompt voice and a second interaction animation are generated. The permission mismatch prompt voice is used to prompt the interactive personnel that the interactive personnel does not have the operation authority corresponding to the operation execution demand. When the personnel authority matches the operation authority, an operation execution step corresponding to the operation execution demand is determined, and a third interaction animation corresponding to each operation execution step is determined. The digital person demonstrates according to the third interaction animation to guide the interactive personnel to complete the operation execution step corresponding to the operation execution demand. This embodiment is not limited.

[0178] It can be seen that the optional embodiment can determine at least one question answer text for the question answering demand when the interaction demand includes the question answering demand, generate the question answering voice corresponding to the personnel emotion portrait and the first interaction animation of the digital person according to each question answer text and the personnel emotion portrait, determine the personnel authority of the interaction personnel and the operation authority of the operation execution demand when the interaction demand includes the operation execution demand, judge whether the personnel authority matches the operation authority, generate the authority mismatch prompt voice and the second interaction animation when the personnel authority does not match the operation authority, determine the operation execution step corresponding to the operation execution demand and determine the third interaction animation corresponding to each operation execution step when the personnel authority matches the operation authority, can provide different feedback content and different feedback animations of the digital person according to different interaction demands of the interaction personnel, can improve the intelligent degree and the individualization level of the digital person interaction, improve the feedback accuracy of the digital person interaction to the user demand, and can also significantly improve the interaction experience and the satisfaction of the user.

[0179] Embodiment three

[0180] Please refer to Figure 3 , Figure 3 is a structural schematic diagram of an adaptive interaction control device for a digital person disclosed by the embodiment of the application. Wherein, Figure 3 The adaptive interaction control device for a digital person described can include a smart device for interacting with a user, the smart device can include any one of a user device, a smart server or a smart platform, the user device can be built-in with a digital person program for interacting with the user, specifically, the user device includes but is not limited to at least one of a smart phone (Android phone, iOS phone, etc.) with a digital person interaction function, a smart phone number watch and a smart home device, etc., the smart server includes a local server or a cloud server, and the embodiment of the application does not limit it. As shown in Figure 3 The adaptive interaction control device for a digital person can include:

[0181] The determining module 301 is configured to determine an interaction personnel corresponding to an initial interaction instruction when the initial interaction instruction is detected, and collect multi-dimensional interaction information of the interaction personnel in a process of executing an initial interaction operation according to the initial interaction instruction.

[0182] The determining module 301 is further configured to determine an interaction demand and a personnel portrait of the interaction personnel according to the initial interaction instruction and the multi-dimensional interaction information of the interaction personnel.

[0183] The generating module 302 is configured to generate interaction feedback information according to the interaction demand and the personnel portrait, and drive the digital person to interact with the interaction personnel according to the interaction feedback information.

[0184] It can be seen that the implementation Figure 3 The adaptive interaction control device applied to the digital person described can determine the interaction personnel corresponding to the initial interaction instruction when detecting the initial interaction instruction, collect multi-dimensional interaction information of the interaction personnel in the process of performing the initial interaction operation according to the initial interaction instruction, determine the interaction demand and the personnel portrait of the interaction personnel according to the initial interaction instruction and the multi-dimensional interaction information of the interaction personnel, generate interaction feedback information according to the interaction demand and the personnel portrait, and drive the digital person to interact with the interaction personnel according to the interaction feedback information, which can improve the intelligent degree and the personalized level of the digital person interaction, improve the feedback accuracy of the digital person interaction to the user demand, and also can significantly improve the interaction experience and the satisfaction of the user.

[0185] In an optional embodiment, as shown in Figure 4 The multi-dimensional interaction information includes voice information and action information of the interaction personnel, and the personnel portrait includes a personnel emotion portrait.

[0186] The specific manner in which the determination module 301 determines the interaction demand and the personnel portrait of the interaction personnel according to the initial interaction instruction and the multi-dimensional interaction information of the interaction personnel includes:

[0187] The voice information of the interaction personnel is subjected to voice recognition to obtain voice text information, and the voice text information is subjected to text analysis to obtain a text keyword set;

[0188] Context information of each keyword in the text keyword set is obtained, and the interaction demand of the interaction personnel is determined according to the context information of each keyword and the initial interaction instruction;

[0189] The voice emotion of the interaction personnel is determined according to the voice information, and the action emotion of the interaction personnel is determined according to the action information;

[0190] The personnel emotion portrait of the interaction personnel is determined according to the voice emotion and the action emotion of the interaction personnel.

[0191] It can be seen that the implementation Figure 4The adaptive interaction control device applied to a digital person described herein can determine the interaction personnel corresponding to the initial interaction instruction when detecting the initial interaction instruction, and in the process of performing the initial interaction operation according to the initial interaction instruction, collect multi-dimensional interaction information of the interaction personnel, perform speech recognition on the speech information of the interaction personnel to obtain speech text information, perform text analysis on the speech text information to obtain a text keyword set, obtain context information of each keyword in the text keyword set, and determine the interaction demand of the interaction personnel according to the context information of each keyword and the initial interaction instruction. The interaction demand of the interaction personnel can be determined according to the speech information of the interaction personnel, improving the accuracy and reliability of determining the interaction demand. The speech emotion of the interaction personnel is determined according to the speech information, and the action emotion of the interaction personnel is determined according to the action information. The personnel emotion portrait of the interaction personnel is determined according to the speech emotion and the action emotion of the interaction personnel, improving the accuracy and reliability of determining the personnel emotion portrait, and further improving the feedback accuracy of the digital person to the demand of the interaction personnel. The interaction demand and the personnel portrait of the interaction personnel are determined according to the initial interaction instruction and the multi-dimensional interaction information of the interaction personnel. The interaction feedback information is generated according to the interaction demand and the personnel portrait, and the digital person interacts with the interaction personnel according to the interaction feedback information. The intelligent degree and the individualization level of the digital person interaction are improved, the feedback accuracy of the digital person interaction to the demand of the user is improved, and the interaction experience and the satisfaction of the user are also significantly improved.

[0192] In another optional embodiment, as shown in Figure 4 The action information includes limb action information, gaze movement information, and facial expression information.

[0193] The specific manner in which the determining module 301 determines the speech emotion of the interaction personnel according to the speech information and determines the action emotion of the interaction personnel according to the action information includes:

[0194] The audio feature information of the interaction personnel is determined according to the speech information, and the audio emotion of the interaction personnel is determined according to the audio feature information. The audio feature information includes audio intensity features and pronunciation intonation features.

[0195] The keyword emotion label corresponding to each keyword in the text keyword set is analyzed, the text emotion of the interaction personnel is determined according to the keyword emotion label corresponding to each keyword, and the speech emotion of the interaction personnel is determined according to the audio emotion and the text emotion.

[0196] The emotion feature recognition is performed on the action information to obtain the first emotion feature corresponding to the limb action information, the second emotion feature corresponding to the gaze movement information, and the third emotion feature corresponding to the facial expression information.

[0197] The preset emotion feature weight information corresponding to the action information is acquired, and the action emotion of the interaction personnel is determined according to the first emotion feature, the second emotion feature, the third emotion feature and the emotion feature weight information.

[0198] It can be seen that, in the embodiments Figure 4 The adaptive interaction control device for digital people described herein can determine the audio emotion of the interaction personnel according to the audio feature information of the interaction personnel, determine the text emotion of the interaction personnel according to the audio text, and further determine the speech emotion of the interaction personnel, thereby improving the accuracy of the determined speech emotion and the closeness of the speech emotion to the real emotion of the interaction personnel, determining the action emotion of the interaction personnel according to the action emotion features of different parts of the interaction personnel, thereby improving the accuracy and reliability of the determined action emotion, and further improving the accuracy of the subsequent determination of the personnel emotion portrait of the interaction personnel.

[0199] In yet another optional embodiment, as Figure 4 The determination module 301 determines the specific manner of the interaction demand of the interaction personnel according to the context information of each keyword and the initial interaction instruction, which includes:

[0200] According to the context information of each keyword, the speech integrity of the speech information of the interaction personnel is determined, and the initial interaction demand of the interaction personnel is determined according to the initial interaction instruction;

[0201] It is judged whether the speech integrity is greater than a preset speech integrity threshold value, and when the speech integrity is greater than the speech integrity threshold value, the interaction demand of the interaction personnel is determined according to the initial interaction demand and the context information of each keyword;

[0202] When the speech integrity is less than the speech integrity threshold value, the historical interaction information of the interaction personnel within a preset time period before the current time is acquired, and the historical interaction information includes historical speech information and historical interaction demand;

[0203] According to the historical interaction information and the context information of each keyword, the remaining speech information of the interaction personnel is predicted, and the interaction demand of the interaction personnel is determined according to the context information of each keyword, the remaining speech information and the initial interaction demand.

[0204] It can be seen that, in the embodiments Figure 4The adaptive interaction control device applied to the digital person described herein can determine the speech integrity of the speech information of the interaction personnel according to the context information of each keyword, determine the initial interaction demand of the interaction personnel according to the initial interaction instruction, when the speech integrity is greater than the speech integrity threshold, determine the interaction demand of the interaction personnel according to the initial interaction demand and the context information of each keyword, when the speech integrity is less than the speech integrity threshold, obtain the historical interaction information of the interaction personnel within a preset time period before the current time, predict the remaining speech information of the interaction personnel according to the historical interaction information and the context information of each keyword, and determine the interaction demand of the interaction personnel according to the context information of each keyword, the remaining speech information and the initial interaction demand. The speech content can be automatically predicted and filled in the case that the speech of the interaction personnel is not complete or clear, and then the interaction demand of the interaction personnel is determined, the accuracy and flexibility of determining the interaction demand are improved, and the interaction experience and satisfaction of the user are improved.

[0205] In yet another optional embodiment, as shown in Figure 4 The interaction demand includes a question answering demand and / or an operation execution demand;

[0206] The specific manner in which the generation module 302 generates the interaction feedback information according to the interaction demand and the personnel portrait includes:

[0207] When the interaction demand includes the question answering demand, at least one question answer text for the question answering demand is determined;

[0208] According to each question answer text and the personnel emotion portrait, a question answering voice corresponding to the personnel emotion portrait and a first interaction animation for the digital person are generated, and the interaction feedback information includes the question answering voice and the first interaction animation;

[0209] When the interaction demand includes the operation execution demand, the personnel permission of the interaction personnel and the operation permission of the operation execution demand are determined;

[0210] It is judged whether the personnel permission matches the operation permission, when the personnel permission does not match the operation permission, a permission mismatch prompt voice and a second interaction animation are generated, and the interaction feedback information includes the permission mismatch prompt voice and the second interaction animation;

[0211] When the personnel permission matches the operation permission, the operation execution steps corresponding to the operation execution demand are determined, and the third interaction animation corresponding to each operation execution step is determined, and the interaction feedback information includes the third interaction animation corresponding to each operation execution step.

[0212] It can be seen that the implementation Figure 4The adaptive interaction control device for digital people described herein can determine at least one question answer text for the question answering demand when the interaction demand includes the question answering demand, generate a question answering voice corresponding to the personnel emotion portrait and a first interaction animation for the digital people according to each question answer text and the personnel emotion portrait, determine the personnel authority of the interaction personnel and the operation authority of the operation execution demand when the interaction demand includes the operation execution demand, judge whether the personnel authority matches the operation authority, generate an authority mismatch prompt voice and a second interaction animation when the personnel authority does not match the operation authority, determine the operation execution steps corresponding to the operation execution demand and determine the third interaction animation corresponding to each operation execution step when the personnel authority matches the operation authority, can provide different feedback content and different feedback animations of the digital people according to different interaction demands of the interaction personnel, can improve the intelligent degree and the individualization level of the digital people interaction, improve the feedback accuracy of the digital people interaction to the user demand, and can also significantly improve the interaction experience and satisfaction of the user.

[0213] In yet another optional embodiment, as shown in Figure 4 The determining module 301 is further configured to determine the additional interaction demand of each additional interaction personnel when it is determined that the initial interaction instruction corresponds to at least one additional interaction personnel other than the interaction personnel.

[0214] The adaptive interaction control device for digital people can further include:

[0215] The judging module 303 is configured to judge whether the additional interaction demand of each additional interaction personnel conflicts with the interaction demand of the interaction personnel, and determine the demand conflict reason between the additional interaction demand and the interaction demand when the additional interaction demand conflicts with the interaction demand.

[0216] The generating module 302 is further configured to generate demand change feedback information for each additional interaction personnel according to the demand conflict reason between the additional interaction demand and the interaction demand, and the demand change feedback information includes demand conflict prompt information and demand change suggestion information.

[0217] The determining module 301 is further configured to determine the demand feedback time of the interaction personnel according to the interaction feedback information corresponding to the interaction personnel, and determine the demand feedback sequence of each additional interaction personnel according to the additional interaction demand of each additional interaction personnel.

[0218] The determining module 301 is further configured to determine the demand feedback time of each additional interaction personnel according to the demand feedback time and the demand feedback sequence of the additional interaction personnel, and drive the digital people to prompt the additional interaction personnel according to the demand feedback time.

[0219] It can be seen that the embodiments Figure 4 The adaptive interaction control device for digital people described can prioritize the interaction needs of the interaction person when there are multiple additional interaction persons other than the interaction person, improve the interaction experience and satisfaction of the interaction person, and while feeding back the interaction needs of the interaction person, can prompt the demand feedback time of each additional interaction person according to the demand feedback order of the additional interaction person and the demand feedback time corresponding to the interaction person, so that each additional interaction person knows the demand feedback progress information of himself, improves the interaction experience and satisfaction of the additional interaction person.

[0220] In yet another optional embodiment, as Figure 4 The determination module 301 is also used to determine the current interaction scene corresponding to the interaction person, and determine the scene privacy of the current interaction scene.

[0221] The determination module 301 is also used to determine the information privacy of the interaction person according to the interaction needs of the interaction person, and determine the current interaction privacy of the interaction person according to the scene privacy and the information privacy.

[0222] The adaptive interaction control device for digital people described can also include:

[0223] The acquisition module 304 is used to acquire the personnel interaction permission of each additional interaction person, and judge whether the personnel interaction permission of the additional interaction person matches the current privacy of the interaction person.

[0224] The generation module 302 is also used to generate privacy security feedback information for the interaction person when the personnel interaction permission of each additional interaction person does not match the current privacy of the interaction person.

[0225] It can be seen that the embodiments Figure 5 The adaptive interaction control device for digital people described can determine the current interaction scene corresponding to the interaction person, and determine the scene privacy of the current interaction scene, determine the information privacy of the interaction person according to the interaction needs of the interaction person, and determine the current interaction privacy of the interaction person according to the scene privacy and the information privacy, acquire the personnel interaction permission of each additional interaction person, and judge whether the personnel interaction permission of the additional interaction person matches the current privacy of the interaction person, generate privacy security feedback information for the interaction person when the personnel interaction permission of each additional interaction person does not match the current privacy of the interaction person, which can remind the interaction person of privacy security during the interaction process of the digital person, protect the privacy security of the interaction person, improve the security and reliability of the digital person interaction process, and further improve the interaction experience and satisfaction of the user.

[0226] Embodiment four

[0227] Please refer to Figure 5 , Figure 5 is another structure diagram of the adaptive interactive control device for digital human disclosed by the embodiment of the present application. As shown in the figure, the adaptive interactive control device for digital human can include: ​

[0228] a memory 401 storing executable program codes;

[0229] a processor 402 coupled with the memory 401;

[0230] The processor 402 invokes the executable program codes stored in the memory 401 to execute the steps in the adaptive interactive control method for digital human described in the embodiment one or the embodiment two of the present application.

[0231] Embodiment five

[0232] The embodiment of the present application discloses a computer storage medium, which stores computer instructions. When the computer instructions are invoked, the steps in the adaptive interactive control method for digital human described in the embodiment one or the embodiment two of the present application are executed.

[0233] Embodiment six

[0234] The embodiment of the present application discloses a computer program product, which includes a non-transitory computer readable storage medium storing a computer program, and the computer program is operable to make a computer execute the steps in the adaptive interactive control method for digital human described in the embodiment one or the embodiment two.

[0235] The above described device embodiments are only schematic, wherein the modules illustrated as separate components can or can not be physically separate, and the components illustrated as modules can or can not be physical modules, i.e., can be located in one place, or can be distributed on multiple network modules. Part or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment scheme. Those skilled in the art can understand and implement without creative labor.

[0236] ​Those skilled in the art can clearly understand the implementation of the various embodiments by means of software and the necessary general hardware platform through the above specific description of the embodiments, and of course, the embodiments can also be implemented by hardware. Based on such understanding, the above technical solutions, essentially or in the sense of contribution to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a computer readable storage medium, including a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an one-time programmable read-only memory (OTPROM), an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM), or other optical disk storage, a magnetic disk storage, a magnetic tape storage, or any other medium that can be used to carry or store data in a computer readable manner.

[0237] Finally, it should be noted that: the self-adaptive interaction control method and device applied to digital human disclosed by the embodiments of the present application are only the preferred embodiments of the present application, and are used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: the technical solutions recorded in the foregoing embodiments can still be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An adaptive interaction control method applied to a digital human, characterized by, The method comprises: When the initial interaction instruction is detected, determine the interaction personnel corresponding to the initial interaction instruction, and collect multi-dimensional interaction information of the interaction personnel in the process of performing an initial interaction operation according to the initial interaction instruction; According to the initial interaction instruction and the multi-dimensional interaction information of the interaction personnel, determine the interaction demand and the personnel portrait of the interaction personnel; According to the interaction demand and the personnel portrait, generate interaction feedback information, and drive the digital person to interact with the interaction personnel according to the interaction feedback information; And the multi-dimensional interaction information includes voice information and action information of the interaction personnel, and the personnel portrait includes a personnel emotional portrait; According to the initial interaction instruction and the multi-dimensional interaction information of the interaction personnel, the interaction demand and the personnel portrait of the interaction personnel are determined, comprising: Carrying out speech recognition on the voice information of the interaction personnel to obtain voice text information, and carrying out text analysis on the voice text information to obtain a set of text keywords; Obtain the context information of each keyword in the set of text keywords, and determine the interaction demand of the interaction personnel according to the context information of each keyword and the initial interaction instruction; Determine the voice emotion of the interaction personnel according to the voice information, and determine the action emotion of the interaction personnel according to the action information; According to the voice emotion and the action emotion of the interaction personnel, determine the personnel emotional portrait of the interaction personnel; And the method further comprises: When at least one additional interaction personnel corresponding to the initial interaction instruction except the interaction personnel is determined, determine the additional interaction demand of each additional interaction personnel; Determine the current interaction scene corresponding to the interaction personnel, and determine the scene privacy of the current interaction scene; According to the interaction demand of the interaction personnel, determine the information privacy of the interaction personnel, and determine the current interaction privacy of the interaction personnel according to the scene privacy and the information privacy; For each additional interaction personnel, obtain the personnel interaction permission of the additional interaction personnel, and judge whether the personnel interaction permission of the additional interaction personnel matches the current privacy of the interaction personnel; For each additional interaction personnel, when the personnel interaction permission of the additional interaction personnel does not match the current privacy of the interaction personnel, generate privacy security feedback information for the interaction personnel; Wherein, the interaction personnel receives the privacy security feedback information and asks the additional interaction personnel to avoid, or authorizes the additional interaction personnel to know the content of the interaction demand of the interaction personnel.

2. The adaptive interaction control method applied to a digital human according to claim 1, wherein, The action information includes body action information, eye movement information and facial expression information; According to the voice information, the audio feature information of the interaction personnel is determined, and the audio emotion of the interaction personnel is determined according to the audio feature information, and the audio feature information includes audio intensity feature and pronunciation tone feature; ​ analyze a keyword emotion label corresponding to each keyword in the keyword set, determine a text emotion of the interaction personnel according to the keyword emotion label corresponding to each keyword, and determine a speech emotion of the interaction personnel according to the audio emotion and the text emotion; perform emotion feature recognition on the action information to obtain a first emotion feature corresponding to the body movement information, a second emotion feature corresponding to the gaze movement information, and a third emotion feature corresponding to the facial expression information; obtain preset emotion feature weight information corresponding to the action information, and determine an action emotion of the interaction personnel according to the first emotion feature, the second emotion feature, the third emotion feature, and the emotion feature weight information.

3. The adaptive interaction control method for a digital human according to claim 1 or 2, wherein The determining of the interaction demand of the interaction personnel according to the context information of each keyword and the initial interaction instruction includes: determining a speech integrity of the speech information of the interaction personnel according to the context information of each keyword, and determining an initial interaction demand of the interaction personnel according to the initial interaction instruction; determining whether the speech integrity is greater than a preset speech integrity threshold, and when the speech integrity is greater than the speech integrity threshold, determining the interaction demand of the interaction personnel according to the initial interaction demand and the context information of each keyword; when the speech integrity is less than the speech integrity threshold, obtaining historical interaction information of the interaction personnel in a preset time period before the current time, the historical interaction information including historical speech information and a historical interaction demand; predicting remaining speech information of the interaction personnel according to the historical interaction information and the context information of each keyword, and determining the interaction demand of the interaction personnel according to the context information of each keyword, the remaining speech information, and the initial interaction demand.

4. The adaptive interaction control method for a digital human according to claim 1 or 2, wherein The interaction demand includes a question answering demand and / or an operation execution demand; The generating of the interaction feedback information according to the interaction demand and the personnel portrait includes: when the interaction demand includes the question answering demand, determining at least one question answer text for the question answering demand; generating a question answering speech corresponding to the personnel emotion portrait and a first interaction animation for a digital person according to each question answer text and the personnel emotion portrait, the interaction feedback information including the question answering speech and the first interaction animation; when the interaction demand includes the operation execution demand, determining a personnel permission of the interaction personnel and an operation permission of the operation execution demand; determining whether the personnel permission matches the operation permission, and when the personnel permission does not match the operation permission, generating a permission mismatch prompt speech and a second interaction animation, the interaction feedback information including the permission mismatch prompt speech and the second interaction animation; When the personnel authority matches the operation authority, an operation execution step corresponding to the operation execution requirement is determined, and a third interaction animation corresponding to each operation execution step is determined, and the interaction feedback information includes the third interaction animation corresponding to each operation execution step.

5. The adaptive interaction control method for a digital human according to claim 1 or 2, wherein The method further includes: For each additional interaction personnel, it is judged whether the additional interaction requirement of the additional interaction personnel conflicts with the interaction requirement of the interaction personnel, and when the additional interaction requirement conflicts with the interaction requirement, a requirement conflict reason between the additional interaction requirement and the interaction requirement is determined; For each additional interaction personnel, demand change feedback information for the additional interaction personnel is generated according to the requirement conflict reason between the additional interaction requirement and the interaction requirement, and the demand change feedback information includes requirement conflict prompt information and demand change suggestion information; According to the interaction feedback information corresponding to the interaction personnel, a demand feedback time consumption corresponding to the interaction personnel is determined, and a demand feedback sequence of each additional interaction personnel is determined according to the additional interaction requirement of each additional interaction personnel; For each additional interaction personnel, a demand feedback time of the additional interaction personnel is determined according to the demand feedback time consumption and the demand feedback sequence of the additional interaction personnel, and the digital person is driven to prompt the additional interaction personnel according to the demand feedback time.

6. An adaptive interactive control device for digital humans, characterized in that, The device is used to execute the adaptive interaction control method for digital people according to any one of claims 1-5, and the device includes: A determination module is configured to determine an interaction personnel corresponding to an initial interaction instruction when the initial interaction instruction is detected, and to collect multi-dimensional interaction information of the interaction personnel in a process of executing an initial interaction operation according to the initial interaction instruction; The determination module is further configured to determine an interaction requirement and a personnel portrait of the interaction personnel according to the initial interaction instruction and the multi-dimensional interaction information of the interaction personnel; A generation module is configured to generate interaction feedback information according to the interaction requirement and the personnel portrait, and to drive a digital person to interact with the interaction personnel according to the interaction feedback information.

7. An adaptive interactive control device for digital humans, characterized in that, The device includes: a memory storing executable program codes; a processor coupled with the memory; The processor invokes the executable program codes stored in the memory to execute the adaptive interaction control method for digital people according to any one of claims 1-5.

8. A computer storage medium, characterized in that The computer storage medium stores computer instructions, and the computer instructions are used to execute the adaptive interaction control method for digital people according to any one of claims 1-5 when invoked.

Images