Virtual robot interaction method and device, storage medium and electronic equipment

Abstract

The invention relates to an interaction method and device of a virtual robot, a storage medium and electronic equipment. The method comprises the following steps: acquiring interaction information which is input by a user and interacts with the virtual robot; the interaction information is input into a control model of the virtual robot, and the control model is obtained through training by taking the interaction information input by a user of a video live broadcast platform and behavior response information of an anchor for the interaction information as model training samples; and according to behavior control information output by the control model based on the interaction information, performing behavior control on the virtual robot. According to the embodiment of the invention, interaction between the virtual robot and the user is realized, the real-time performance, the flexibility and the applicability of the virtual robot are improved, and the requirements of emotion and action communication between the user and the virtual robot are met.

Description

[0001] Divisional application

[0002] This application is a divisional application. The application number of the original application is: 201811217722.7, the application date is October 18, 2018, and the name of the invention is "Interactive method, device, storage medium and electronic device for virtual robots". Technical Field

[0003] The present disclosure relates to the field of human-computer interaction, and in particular, to a virtual robot interaction method, device, storage medium and electronic device. Background Art

[0004] At present, virtual idols have become a new highlight in the entertainment field and are gradually loved and sought after by people. However, traditional virtual idols are mainly pre-realized based on pre-set characters, plot development, interaction methods and other elements of the system, and cannot interact with the audience in real time, and have low flexibility and applicability.

[0005] With the development of the live streaming industry, users can watch live broadcasts on live streaming platforms, interact with live broadcasts through text, and give virtual prizes to anchors. The existing virtual idol technology cannot be applied to live streaming platforms to achieve live broadcasts, and the traditional auxiliary robots in the live broadcast room are also relatively simple, mainly based on voice, and cannot meet people's emotional and action communication experience. Summary of the invention

[0006] The main purpose of the present disclosure is to provide a virtual robot interaction method, device, storage medium and electronic device to solve the problems existing in the above-mentioned related technologies.

[0007] In order to achieve the above-mentioned purpose, the first aspect of the embodiment of the present disclosure provides an interactive method of a virtual robot, comprising:

[0008] Acquire interactive information input by a user for interacting with the virtual robot;

[0009] Inputting the interactive information into the control model of the virtual robot, wherein the control model is trained using the interactive information input by users of the live video platform and the host's behavioral response information to the interactive information as model training samples;

[0010] The virtual robot is behavior controlled according to the behavior control information output by the control model based on the interaction information.

[0011] Optionally, a method for training the control model is also included, comprising:

[0012] Obtaining interactive information input by users and information about the host's behavioral response to the interactive information from the live video platform;

[0013] The control model is trained by obtaining interactive information input by users from the live video broadcast platform and the host's behavioral response information to the interactive information as model training samples.

[0014] Optionally, obtaining the anchor's behavior response information to the interactive information input by the user from the live video platform includes:

[0015] Extracting the anchor's body movement information from the anchor's video according to the human posture analysis module; and / or

[0016] Extracting the host's facial expression information from the host's video according to the facial expression analysis module; and / or

[0017] The host's voice information is extracted from the host's audio according to the voice analysis module.

[0018] Optionally, the control model includes a deep learning network, which is divided into three branches of body movement output, facial expression output, and voice output through a convolutional network and a fully connected layer; the interactive information input by the user on the live video platform includes text information input by the user into the live chat room and picture information of a virtual gift given by the user to the anchor, and the behavioral response information includes the anchor's body movement information, facial expression information, and voice information;

[0019] The method of obtaining interactive information input by the user from the live video platform and the host's behavioral response information to the interactive information as model training samples to train the control model includes:

[0020] The text information and the picture information of the virtual gift are used as training inputs to train the body movements, facial expressions and voice of the virtual robot.

[0021] Optionally, before obtaining the interactive information input by the user for interacting with the virtual robot, the method further includes:

[0022] Get the preference information entered by the user;

[0023] Determining a target control model that matches the preference information from among multiple types of control models of the virtual robot;

[0024] The step of inputting the interactive information into the control model of the virtual robot comprises:

[0025] inputting the interaction information into the target control model;

[0026] The step of controlling the behavior of the virtual robot according to the behavior control information output by the control model based on the interaction information includes:

[0027] The virtual robot is behavior controlled according to the behavior control information output by the target control model based on the interaction information.

[0028] A second aspect of the disclosed embodiment provides an interactive device for a virtual robot, comprising:

[0029] A first acquisition module, used to acquire interaction information input by a user to interact with the virtual robot;

[0030] A model input module, used to input the interactive information into a control model of the virtual robot, wherein the control model is trained using the interactive information input by users of a live video broadcasting platform and the host's behavioral response information to the interactive information as model training samples;

[0031] A control module is used to perform behavior control on the virtual robot according to the behavior control information output by the control model based on the interaction information.

[0032] Optionally, it also includes:

[0033] The second acquisition module is used to obtain the interactive information input by the user and the host's behavioral response information to the interactive information from the video live broadcast platform;

[0034] The model training module is used to obtain the interactive information input by the user from the video live broadcast platform and the host's behavioral response information to the interactive information as model training samples to train the control model.

[0035] Optionally, the second acquisition module includes:

[0036] A first acquisition submodule is used to extract the anchor's body movement information from the anchor's video according to the human posture analysis module; and / or

[0037] A second acquisition submodule is used to extract the host's facial expression information from the host's video according to the facial expression analysis module; and / or

[0038] The third acquisition submodule is used to extract the host's voice information from the host's audio according to the voice analysis module.

[0039] Optionally, the control model includes a deep learning network, which is divided into three branches of body movement output, facial expression output, and voice output through a convolutional network and a fully connected layer; the interactive information input by the user on the live video platform includes text information input by the user into the live chat room and picture information of a virtual gift given by the user to the anchor, and the behavioral response information includes the anchor's body movement information, facial expression information, and voice information;

[0040] The model training module is used to:

[0041] The text information and the picture information of the virtual gift are used as training inputs to train the body movements, facial expressions and voice of the virtual robot.

[0042] Optionally, the device further comprises:

[0043] A third acquisition module is used to acquire preference information input by the user;

[0044] A determination module, configured to determine a target control model matching the preference information from among a plurality of types of control models of the virtual robot;

[0045] The model input module is used to input the interactive information into the target control model;

[0046] The control module is used to perform behavior control on the virtual robot according to the behavior control information output by the target control model based on the interaction information.

[0047] A third aspect of an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0048] A fourth aspect of the present disclosure provides an electronic device, including:

[0049] a memory having a computer program stored thereon;

[0050] A processor is used to execute the computer program in the memory to implement the steps of the method described in the first aspect.

[0051] By adopting the above technical solution, at least the following technical effects can be achieved: the historical data of the live video platform, including the interactive information input by the user and the behavioral response information of the anchor to the interactive information, is used as a model training sample to train a control model, and the output of the control model is the control information for controlling the behavior of the virtual robot. In this way, based on the control model, by collecting the interactive information input by the user to interact with the virtual robot in real time, the real-time interactive response control of the virtual robot with the user can be realized, which improves the real-time performance, flexibility and applicability of the virtual robot and meets the needs of the user for emotional and action communication with the virtual robot.

[0052] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present disclosure but do not constitute a limitation of the present disclosure. In the accompanying drawings:

[0054] Figure 1 is a flowchart of a virtual robot interaction method provided by an embodiment of the present disclosure;

[0055] Figure 2 is a flowchart of a method for training a virtual robot control model provided by an embodiment of the present disclosure;

[0056] Figure 3 is a schematic diagram of a control model training process provided by an embodiment of the present disclosure;

[0057] Figure 4 is a schematic diagram of another control model training process provided by an embodiment of the present disclosure;

[0058] Figure 5 is a structural schematic diagram of an interactive device of a virtual robot provided by an embodiment of the present disclosure;

[0059] Figure 6 is a structural schematic diagram of an interactive device of a virtual robot provided by an embodiment of the present disclosure;

[0060] Figure 7 is a structural schematic diagram of another virtual robot training device provided by an embodiment of the present disclosure;

[0061] Figure 8 It is a structural schematic diagram of another electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0062] The specific implementation of the present disclosure is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the present disclosure, and is not used to limit the present disclosure.

[0063] The present disclosure provides a method for interacting with a virtual robot. Figure 1 As shown, including:

[0064] S11. Obtaining interactive information input by a user for interacting with the virtual robot.

[0065] In one possible implementation, the disclosed embodiment may combine animation technology with online live broadcast technology to display the animated image of a virtual character in an online live broadcast room. The interactive information input by the user may be text information input by the user in the live broadcast room of the virtual robot and / or picture information of gifts given by the user.

[0066] The above is only an example of possible application scenarios of the embodiments of the present disclosure. In another possible implementation, the virtual robot may not be used for live broadcasting, but may be built into a separate terminal product and produced and sold as a chat robot or an emotional interaction robot. The present disclosure does not limit this.

[0067] S12. Input the interactive information into the control model of the virtual robot, wherein the control model is trained using the interactive information input by users of the live video platform and the host's behavioral response information to the interactive information as model training samples.

[0068] Specifically, based on the historical playback information of the video live broadcast platform, a large number of samples can be obtained. The text information entered by the audience in the chat room of each anchor's live broadcast room and the picture information of the virtual gifts given can be used as the above-mentioned interactive information. In addition, the anchor's behavioral response information can be extracted from the anchor's video and audio, thereby obtaining a large number of model training samples, so that the trained control model's control of the virtual robot is closer to the anchor's real reaction.

[0069] S13. Performing behavior control on the virtual robot according to the behavior control information output by the control model based on the interaction information.

[0070] Specifically, the behavior control of the virtual robot may include controlling the body movements, facial expressions, and voice output of the virtual robot displayed as an animated image.

[0071] The above method uses the historical playback data of the video live broadcast platform, including: the interactive information input by the user and the behavioral response information of the anchor to the interactive information, as a model training sample to train a control model, and the output of the control model is the control information for controlling the behavior of the virtual robot. In this way, based on the control model, by collecting the interactive information input by the user to interact with the virtual robot in real time, the real-time interactive response control of the virtual robot with the user can be realized, which improves the real-time performance, flexibility and applicability of the virtual robot and meets the needs of the user for emotional and action communication with the virtual robot.

[0072] In order to enable those skilled in the art to better understand the technical solution provided by the embodiments of the present disclosure, the interactive method of the virtual robot provided by the embodiments of the present disclosure is described in detail below.

[0073] First, with respect to the control model described in step S12, the disclosed embodiment also includes a training method for the control model. It is worth noting that the training of the control model is performed in advance based on samples collected from the live video platform. Subsequently, during the interaction between the virtual robot and the user, there is no need to train the control model every time, or the control model can be periodically updated based on new samples collected from the live video platform.

[0074] Specifically, the training method of the control model of the virtual robot is as follows: Figure 2 As shown, including:

[0075] S21. Obtaining interactive information input by the user and the host's behavioral response information to the interactive information from the live video platform.

[0076] For example, the interactive information input by the user on the live video platform includes text information input by the user into the live chat room and / or picture information of a virtual gift given by the user to the anchor.

[0077] S22, obtaining the interactive information input by the user from the live video platform and the host's behavioral response information to the interactive information as model training samples to train the control model.

[0078] The following is an explanation of how to obtain the anchor's behavior response information:

[0079] Method 1: Extract the anchor's body movement information from the anchor's video based on the human posture analysis module.

[0080] The limb movement information is mainly the position information of the limb joints. The input of the human posture analysis module is continuous image frames. The probability map of the posture is obtained through convolutional neural network learning, and then combined with optical flow information to generate an intermediate mixed probability distribution map, and finally the joint position information can be obtained.

[0081] Method 2: Extract the host's facial expression information from the host's video based on the facial expression analysis module.

[0082] Specifically, the face area can be first extracted from the host video through the face detection module, and then the expression classification result can be generated through deep neural network learning.

[0083] Method 3: Extract the host's voice information from the host's audio based on the voice analysis module.

[0084] First, a sentence of speech is converted into an image as input, that is, each frame of speech is first Fourier transformed, and then time and frequency are used as the two dimensions of the image. Then, the entire sentence of speech is modeled through a convolutional network, and the output unit directly corresponds to the final recognition result, such as a syllable or Chinese character.

[0085] It is worth noting that the above three implementation methods can be selectively implemented according to actual needs (such as product function design). That is to say, in step S21, obtaining the host's behavioral response information to the interactive information input by the user from the video live broadcast platform includes: extracting the host's body movement information from the host's video according to the human posture analysis module; and / or extracting the host's facial expression information from the host's video according to the facial expression analysis module; and / or extracting the host's voice information from the host's audio according to the voice analysis module.

[0086] The following takes the interactive information input by the user in the live video platform, including the text information input by the user into the live chat room and the picture information of the virtual gift given by the user to the anchor, and the behavioral response information including the anchor's body movement information, facial expression information and voice information as an example to illustrate the training of the control model.

[0087] Specifically, the control model may include a deep learning network, which is divided into three branches: body movement output, facial expression output, and voice output through a convolutional network and a fully connected layer. The interactive information input by the user and the host's behavioral response information to the interactive information obtained from the live video platform are used as model training samples to train the control model, including: using the text information and the picture information of the virtual gift as training inputs to train the body movements, facial expressions and voice of the virtual robot.

[0088] For example, Figure 3 and Figure 4 Schematic diagrams of control model training are shown respectively. Figure 3 The source of the training data is shown in Figure 4 The process of training the control model based on the deep learning network is shown in FIG. Figure 3As shown in the figure, text information and gift pictures are used as input samples of the deep learning network, body movement information and facial expression information extracted from the anchor video by the human posture analysis module and the facial expression analysis module, and voice information extracted from the anchor audio by the voice analysis module are used as output samples marked by the deep learning network. Figure 4 As shown in the figure, the deep neural network is divided into three branches: body movement output, facial expression output, and voice output through the convolutional network and the fully connected layer, so as to realize the training of the body movement, facial expression and voice of the virtual robot respectively.

[0089] It is worth mentioning that human posture analysis, facial expression analysis and speech analysis can all be achieved through deep learning of neural networks.

[0090] In a possible implementation of the embodiment of the present disclosure, before the user interacts with the virtual robot, the user may be allowed to select the virtual robot according to his or her own preferences. For example, before step S11, the preference information input by the user may be obtained, and a target control model matching the preference information may be determined from the multiple types of control models of the virtual robot, wherein the multiple types of control modules may be control models obtained by collecting data and training according to anchors of different personality types; accordingly, step S12 includes: inputting the interaction information into the target control model; step S13 is: controlling the behavior of the virtual robot according to the behavior control information output by the target control model based on the interaction information.

[0091] The preference information may be target tag information selected by the user from tag information for the user to select, and the tag information may be, for example, a host personality tag, a host performance style tag, etc.

[0092] For example, the disclosed embodiment can classify anchors according to the personality labels, performance type labels, etc. presented to each anchor on the live video platform, and pre-train the control model according to the historical playback information of each type of anchor, so that the user can input preference information for selection. In this way, it is possible to control the virtual robot to interact with the user based on the user's preferences, which is equivalent to realizing the user's customization of the virtual robot's personality and improving the user experience. In specific implementation, the appearance of the virtual robot can also be customized according to the user's preferences, and the disclosure does not limit this.

[0093] Based on the same inventive concept, the present disclosure also provides an interactive device for a virtual robot, which is used to implement the interactive method for a virtual robot provided in the above method embodiment, such as Figure 5 As shown, the device comprises:

[0094] A first acquisition module 51 is used to acquire interaction information input by a user to interact with the virtual robot;

[0095] A model input module 52, used to input the interactive information into the control model of the virtual robot, wherein the control model is trained using the interactive information input by the user of the live video platform and the host's behavioral response information to the interactive information as model training samples;

[0096] The control module 53 is used to perform behavior control on the virtual robot according to the behavior control information output by the control model based on the interaction information.

[0097] By using the above device, the device can use the historical playback data of the video live broadcast platform, including: the interactive information input by the user and the behavioral response information of the anchor to the interactive information, as a model training sample to train a control model, and the output of the control model is the control information for controlling the behavior of the virtual robot. In this way, based on the control model, by collecting the interactive information input by the user to interact with the virtual robot in real time, the real-time interactive response control of the virtual robot with the user can be realized, which improves the real-time performance, flexibility and applicability of the virtual robot and meets the needs of the user for emotional and action communication with the virtual robot.

[0098] Alternatively, if Figure 6 As shown, the device also includes:

[0099] A third acquisition module 54 is used to acquire preference information input by a user;

[0100] A determination module 55, configured to determine a target control model that matches the preference information from among multiple types of control models of the virtual robot;

[0101] The model input module 52 is used to input the interactive information into the target control model;

[0102] The control module 53 is used to perform behavior control on the virtual robot according to the behavior control information output by the target control model based on the interaction information.

[0103] The present disclosure also provides a virtual robot training device for implementing Figure 2 The training method of the virtual robot provided, such as Figure 7 As shown, the device comprises:

[0104] The second acquisition module 56 is used to acquire the interactive information input by the user and the host's behavioral response information to the interactive information from the live video platform; the model training module 57 is used to train the control model using the interactive information input by the user and the host's behavioral response information to the interactive information obtained from the live video platform as model training samples. For example, the interactive information input by the user on the live video platform includes text information input by the user into the live chat room and / or picture information of a virtual gift given by the user to the host.

[0105] Optionally, the second acquisition module 56 may include:

[0106] A first acquisition submodule is used to extract the anchor's body movement information from the anchor's video according to the human posture analysis module; and / or

[0107] A second acquisition submodule is used to extract the host's facial expression information from the host's video according to the facial expression analysis module; and / or

[0108] The third acquisition submodule is used to extract the host's voice information from the host's audio according to the voice analysis module.

[0109] Optionally, the control model includes a deep learning network, which is divided into three branches of body movement output, facial expression output, and voice output through a convolutional network and a fully connected layer; the interactive information input by the user on the live video platform includes text information input by the user into the live chat room and picture information of a virtual gift given by the user to the anchor, and the behavioral response information includes the anchor's body movement information, facial expression information, and voice information;

[0110] The model training module 57 is used to:

[0111] The text information and the picture information of the virtual gift are used as training inputs to train the body movements, facial expressions and voice of the virtual robot.

[0112] It is worth noting that the virtual robot interaction device and training device provided above can be set up separately or integrated into the same server. For example, the interaction device and the training device implement part or all of the server in the form of software, hardware or a combination of both, and the present disclosure does not limit this.

[0113] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0114] The embodiment of the present disclosure also provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the above-mentioned virtual robot interaction method when executed by a processor.

[0115] The present disclosure also provides an electronic device, including:

[0116] a memory having a computer program stored thereon;

[0117] The processor is used to execute the computer program in the memory to implement the steps of the above-mentioned virtual robot interaction method.

[0118] It is worth noting that the electronic device can be used as a control device for the virtual robot, or the virtual robot can also run on the electronic device, which is not limited in the present disclosure.

[0119] Figure 8 is a block diagram of the electronic device provided by the embodiment of the present disclosure. Figure 8 As shown, the electronic device 800 may include: a processor 801 and a memory 802. The electronic device 800 may also include one or more of a multimedia component 803, an input / output (I / O) interface 804, and a communication component 805.

[0120] The processor 801 is used to control the overall operation of the electronic device 800 to complete all or part of the steps in the above-mentioned virtual robot interaction method. The memory 802 is used to store various types of data to support the operation of the electronic device 800, and these data may include, for example, instructions for any application or method used to operate on the electronic device 800, and application-related data, such as contact data, sent and received messages, pictures, audio, video, etc. The memory 802 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, referred to as EPROM), programmable read-only memory (Programmable Read-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic memory, flash memory, magnetic disk or optical disk. The multimedia component 803 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone, which is used to receive external audio signals. The received audio signal may be further stored in the memory 802 or sent through the communication component 805. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 804 provides an interface between the processor 801 and other interface modules, and the above-mentioned other interface modules may be keyboards, mice, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 805 is used for wired or wireless communication between the electronic device 800 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G or 4G, or a combination of one or more of them, so the corresponding communication component 805 may include: Wi-Fi module, Bluetooth module, NFC module.

[0121] In an exemplary embodiment, the electronic device 800 can be implemented by one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), controllers, microcontrollers, microprocessors or other electronic components to execute the above-mentioned virtual robot interaction method.

[0122] Among them, the above-mentioned computer-readable storage medium provided in the embodiment of the present disclosure can be the above-mentioned memory 802 including program instructions, and the above-mentioned program instructions can be executed by the processor 801 of the electronic device 800 to complete the above-mentioned virtual robot interaction method.

[0123] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings; however, the present disclosure is not limited to the specific details in the above embodiments. Within the technical concept of the present disclosure, a variety of simple modifications can be made to the technical solution of the present disclosure, and these simple modifications all fall within the protection scope of the present disclosure.

[0124] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present disclosure will not further describe various possible combinations.

[0125] In addition, various embodiments of the present disclosure may be arbitrarily combined, and as long as they do not violate the concept of the present disclosure, they should also be regarded as the contents disclosed by the present disclosure.

Claims

1. A virtual robot interaction method, characterized in that: include: Obtaining preference information input by a user, and determining a target control model matching the preference information from a plurality of types of control models of the virtual robot; Acquiring interaction information between the user input and the virtual robot; Obtain the host's behavioral response information to the interactive information; The interactive information input by the user and the host's behavioral response information to the interactive information are used as model training samples to train the control model; The control model includes a deep learning network, which is divided into three branches: body movement output, facial expression output, and speech output through a convolutional network and a fully connected layer; Using text information and picture information as training inputs of the target control model, training the body movements, facial expressions and voices of the virtual robot, and obtaining behavior control information output based on the interactive information; The virtual robot is subjected to behavior control according to the behavior control information output by the target control model.

2. The method according to claim 1, characterized in that The obtaining of the host's behavioral response information to the interactive information includes: The anchor's body movement information is extracted from the anchor's video according to the human posture analysis module; specifically including: The continuous image frames are used as the input of the human posture analysis module. The probability map of the posture is obtained through convolutional neural network learning. Then, the intermediate mixed probability distribution map is generated by combining the optical flow information, and finally the joint position information can be obtained.

3. The method according to claim 1, characterized in that The obtaining of the host's behavioral response information to the interactive information includes: Extract the anchor's facial expression information from the anchor's video according to the facial expression analysis module; specifically including: First, the face area is extracted from the host video through the face detection module, and then the expression classification results are generated through deep neural network learning.

4. The method according to claim 1, characterized in that: The obtaining of the host's behavioral response information to the interactive information includes: Extract the host's voice information from the host's audio according to the voice analysis module; specifically including: First, a sentence of speech is converted into an image as input, that is, each frame of speech is first Fourier transformed, and then time and frequency are used as the two dimensions of the image. Then, the entire sentence of speech is modeled through a convolutional network, and the output unit directly corresponds to the final recognition result, such as a syllable or Chinese character.

5. The method according to claim 1, characterized in that The interactive information input by the user on the live video platform includes the text information input by the user into the live chat room and the picture information of the virtual gift given by the user to the anchor, and the behavioral response information includes the anchor's body movement information, facial expression information and voice information.

6. The method according to claim 1, characterized in that The obtaining of the preference information input by the user and determining a target control model matching the preference information from the multiple types of control models of the virtual robot includes: The anchors are classified according to the personality labels, performance type labels, etc. presented for each anchor on the live video platform, and the control models are trained in advance based on the historical playback information of each type of anchor, so that users can input their preference information and choose, so as to control the virtual robot to interact with the user based on the user's preferences.

7. The method according to any one of claims 2 to 4, characterized in that The text information and gift pictures are used as input samples of the deep learning network. The body movement information and facial expression information extracted from the host video by the human posture analysis module and the facial expression analysis module, and the voice information extracted from the host audio by the voice analysis module are used as output samples marked by the deep learning network.

8. The method according to claim 1, characterized in that The step of controlling the behavior of the virtual robot according to the behavior control information output by the target control model includes: The virtual robot is behavior controlled according to the behavior control information output by the target control model based on the interaction information.

9. An interactive device for a virtual robot, characterized in that: include: A third acquisition module is used to acquire preference information input by the user; A determination module, configured to determine a target control model matching the preference information from among a plurality of types of control models of the virtual robot; A first acquisition module, used to acquire interaction information between the user input and the virtual robot; The second acquisition module is used to obtain the host's behavioral response information to the interactive information; A model training module is used to train a control model using the interactive information obtained from the user input and the host's behavioral response information to the interactive information as model training samples; the control model includes a deep learning network, which is divided into three branches: body movement output, facial expression output, and voice output through a convolutional network and a fully connected layer; Using text information and picture information as training inputs of the target control model, training the body movements, facial expressions and voices of the virtual robot, and obtaining behavior control information output based on the interactive information; A control module is used to perform behavior control on the virtual robot according to the behavior control information output by the target control model.

10. The device according to claim 9, characterized in that The second acquisition module includes: A first acquisition submodule is used to extract the anchor's body movement information from the anchor's video according to the human posture analysis module; and / or A second acquisition submodule is used to extract the host's facial expression information from the host's video according to the facial expression analysis module; and / or The third acquisition submodule is used to extract the host's voice information from the host's audio according to the voice analysis module.

11. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 8.

Images