Avatar control method, apparatus, and related device

Abstract

The present application provides an avatar control method, comprising: acquiring live streaming interaction data of a live stream, the live streaming interaction data comprising live streaming bullet screen data and live streamer speech data, and the live stream containing an avatar and a live streamer; according to the live streaming bullet screen data, determining a target question to be answered, the live streaming bullet screen data being sent by viewers of the live stream; according to said target question, determining target interaction content, the target interaction content comprising answered content of said target question; according to the target interaction content, determining target interaction data, the target interaction data being used for displaying the target interaction content to the viewers of the live stream by means of the avatar, and the target interaction data comprising target audio data and target video data; and adding the target interaction data to live streaming data of the live stream. Therefore, interaction with the viewers is implemented by means of the avatar so as to promptly respond to the interaction between the viewers and the live streamer, thus improving the viewing experience of the live stream. The present application further provides a corresponding apparatus and a related device.

Description

A virtual image control method, device and related equipment

[0001] This application claims priority to a Chinese patent application filed with the State Intellectual Property Office on August 3, 2023, with application number 202310973189.1 and titled “A virtual image control method, computing cluster and related equipment”. This application also claims priority to a Chinese patent application filed with the State Intellectual Property Office on December 1, 2023, with application number 202311638844.4 and titled “A virtual image control method, device and related equipment”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] The present application relates to the field of media technology, and in particular to a virtual image control method, device and related equipment. Background Art

[0003] With the continuous evolution of computer technology, live streaming is becoming increasingly popular. In live streaming scenarios, the host can upload a live data stream through a terminal device. The live data stream can be forwarded by a live streaming server (or other device) to the corresponding client of the live broadcast audience. The live data stream can include video data stream and / or audio data stream.

[0004] In live streaming scenarios, hosts need to present their content to users. Furthermore, in some scenarios, hosts are also responsible for managing the live streaming room, interacting with other hosts, and interacting with viewers. When live streaming workloads are hectic, hosts may find it difficult to balance all aspects.

[0005] Summary of the Invention

[0006] In view of this, the embodiments of the present application provide a method for controlling a virtual image, aiming to provide a systematic solution to the aforementioned problem of the difficulty for anchors to balance various aspects of a live broadcast scenario. The present application also provides a corresponding apparatus, a computing device cluster, a computer-readable storage medium, and a computer program product.

[0007] In a first aspect, the present application provides a method for controlling a virtual avatar. The method can be implemented by a virtual avatar control device in a live broadcast system, and can be used to assist broadcasts through a virtual avatar. The virtual avatar control device can be deployed in at least one data center. Specifically, during a live broadcast in a live broadcast room, the virtual avatar control device can obtain live broadcast interaction data. This live broadcast interaction data includes live commentary data (i.e., commentary data sent by viewers to the host) and host voice data (i.e., voice data emitted by the host during the live broadcast). The virtual avatar control device can then determine a target unanswered question based on the live commentary data. The target interaction content can then be determined based on the unanswered question and the live broadcast interaction data. The target interaction content includes the response to the target unanswered question. The virtual avatar control method can then determine target interaction data based on the target interaction content and add the target interaction data to the live broadcast data in the live broadcast room. The target interaction data is used to display the target interaction content to viewers in the live broadcast room via the virtual avatar. When viewing the live broadcast data with the added target interaction data, viewers can see both the live broadcast data corresponding to the host and the process of the virtual avatar displaying the target interaction content. During the live broadcast process, the virtual image control device can actively obtain the live broadcast interaction data of the live broadcast room, and determine the corresponding target interactive content based on the questions raised in the barrage, and then display the target interactive content to the audience in the live broadcast room through the virtual image. In this way, the live broadcast interaction data that needs to be interacted can be automatically determined based on the live broadcast interaction data, and the target interactive content can also be displayed to the audience through the virtual image. Automatic interaction with the audience is achieved through the virtual image, so that the interaction between the audience and the anchor can receive a timely response, thereby improving the viewing experience of the live broadcast. In addition, using the virtual image to assist in the broadcast also means that the anchor does not need to focus on the interaction with the audience, but can focus more on the live broadcast content itself, thereby improving the efficiency of the anchor's live broadcast.

[0008] In some possible implementations, the target unanswered question can be determined based on the host's voice data. Specifically, when determining the target unanswered question, at least one unanswered question can be first determined based on the live broadcast barrage data. Then, based on the host's voice data, the unanswered question from the at least one unanswered question can be selected as the target unanswered question. In other words, the host's voice data does not include audio data for answering the target unanswered question. The target unanswered question is a question raised by the audience that has not been answered by the host. In this way, the host's unanswered questions can be supplemented by a virtual avatar, improving the viewing experience of the live broadcast room.

[0009] In some possible implementations, the avatar can respond to the target pending question when the host stops answering. Specifically, the host's voice data can be detected for pauses, and the target interaction data can be added to the live broadcast data after the pause occurs. This prevents the host and avatar from speaking simultaneously, which could affect the live broadcast experience.

[0010] In some possible implementations, the target interactive content can be determined based on a pre-set virtual image knowledge base. Specifically, when determining the target question to be answered, the live broadcast barrage data can be clustered first to determine the intent information set. Then, the target question to be answered can be determined based on the barrage intent information set. In this way, clustering the barrage data can filter out meaningless information in the barrage, so as to provide targeted responses to the questions raised by the audience. When determining the target interactive content based on the target question to be answered, the answer content of the target question to be answered can be determined from the virtual image knowledge base based on the target question to be answered, thereby determining the target interactive content based on the answer content of the target question to be answered. In this way, questions raised by the audience in the live broadcast room can be answered.

[0011] In some possible implementations, the avatar knowledge base can also be updated. Specifically, if a host proactively answers a target question, the avatar knowledge base can be updated based on the target question and the audio data the target host sent to answer the question. In other words, if a host proactively answers a question posed by a viewer, the avatar knowledge base can also be updated based on the host's answer and the viewer's question. This allows the avatar knowledge base to be updated, allowing for more accurate responses to subsequent viewer questions that are similar in style and content to the host's answer.

[0012] In some possible implementations, when generating target interactive data, audio data may be generated first and then video data. Specifically, corresponding target audio data may be generated first based on the target interactive content. Then, target video data may be generated based on the relevant data of the virtual image and the target audio data. The relevant data of the virtual image may include, for example, model data of the virtual image and other data. In this way, the target video data obtained may be the video data of the virtual image when it emits the audio corresponding to the target interactive content. From the perspective of the audience watching the live broadcast, it is equivalent to the virtual image emitting sound while synchronously performing corresponding actions. In this way, synchronizing the audio and video of the virtual image improves the fluency of the virtual image and enhances the live broadcast effect.

[0013] In some possible implementations, the target audio data can be matched to the state of the target livestreamer. Specifically, the livestreamer's voice data can be analyzed to determine the livestreamer's emotional category. The livestreamer's emotional category represents the emotional state of the livestreamer. Then, based on the target interactive content, target audio data matching the livestreamer's emotional category can be generated. This allows the avatar's virtual emotions to change with the livestreamer's emotions, improving the synchronization between the avatar and the livestreamer, and enhancing the livestream's effectiveness.

[0014] In some possible implementations, the host can also instruct the virtual image to complete the corresponding action through instructions. Specifically, the host's voice data can be parsed to determine the host's instruction information. Specifically, the instruction interaction data can be determined based on the host's instruction information, and the instruction interaction data can be added to the live broadcast data of the live broadcast room. The instruction interaction data is used to show the audience in the live broadcast room through the virtual image that the virtual image has completed the action indicated by the host's instruction information. In other words, when the host instructs the virtual image to complete a certain action, multimedia data of the virtual image completing the action can be generated, and the multimedia data can be added to the live broadcast data of the live broadcast room. In this way, the audience watching the live broadcast can hear and / or see the virtual image completing the action indicated by the host. In this way, the host can instruct the virtual image to complete an action through instructions, which increases the flexibility of the virtual image and helps to improve the live broadcast effect.

[0015] In some possible implementations, there may be communication barriers between the host and the audience, for example, if the host's language differs from the audience's. To address this, subtitles can be added to the live broadcast room. Specifically, after acquiring the host's voice data, the voice data can be translated to obtain target subtitle data, which is then added to the live broadcast room data. This allows viewers to see the subtitles within the live broadcast room video and understand what the host is saying. This achieves barrier-free communication between the host and the audience.

[0016] In some possible implementations, the live broadcast room can also be adjusted based on the host's instructions. Specifically, the target host can issue live broadcast room operation instructions through voice, and the live broadcast room operation instructions are used to adjust the live broadcast parameters of the live broadcast room. After obtaining the host's voice data, the host's voice data can be parsed to determine the live broadcast room operation instructions issued by the target host. Then, the virtual image control device can adjust the live broadcast data of the target live broadcast room based on the live broadcast room operation instructions. In this way, the target host can adjust the live broadcast data of the target live broadcast room through voice, without the need for the target host to make manual adjustments, thereby improving the efficiency of the live broadcast.

[0017] In a second aspect, the present application provides a virtual image control device, which is located in at least one data center. The device includes: an acquisition module for acquiring live interaction data of a live broadcast room, which includes live barrage data and anchor voice data, and the live broadcast room includes a virtual image and an anchor; a question determination module for determining target questions to be answered based on the live barrage data, which is sent by the audience in the live broadcast room; an interactive content determination module for determining target interactive content based on the target questions to be answered and the live interaction data, which includes the answer content of the target questions to be answered; an interactive data determination module for determining target interactive data based on the target interactive content, which is used to display the target interactive content to the audience in the live broadcast room through a virtual image, and the target interactive data includes target audio data and target video data; a live data adjustment module for adding target interactive data to the live data in the live broadcast room.

[0018] In some possible implementations, the question determination module is specifically used to determine at least one question to be answered based on the live broadcast barrage data; and to determine the target question to be answered from at least one question to be answered based on the host's voice data, and the host's voice data does not include audio data corresponding to the answer content of the target question to be answered.

[0019] In some possible implementations, the problem determination module is further used to determine whether a pause occurs in the host's voice data.

[0020] In some possible implementations, the virtual image control device includes a virtual image knowledge base and a question determination module, which is used to cluster live broadcast barrage data and determine a barrage intention information set; determine the target question to be answered based on the barrage intention information set; and an interactive content determination module, which is used to determine the answer content of the target question to be answered from the virtual image knowledge base based on the target question to be answered; and determine the target interactive content based on the answer content.

[0021] In some possible implementations, the question determination module is further configured to update the virtual image knowledge base based on the target question to be answered and the audio data used to answer the target question to be answered.

[0022] In some possible implementations, the interactive data determination module is specifically used to generate corresponding target audio data based on the target interactive content; generate target video data based on the virtual image and the target audio data, and the target video data is the video data when the virtual image emits the audio corresponding to the target interactive content.

[0023] In some possible implementations, the interactive data determination module is specifically used to analyze the host's voice data to determine the host's emotional category; and generate target audio data that matches the host's emotional category based on the target interactive content.

[0024] In some possible implementations, the interactive data determination module is also used to determine the anchor's command information based on the anchor's voice data; determine the command interaction data based on the anchor's command information, and the command interaction data is used to show the audience in the live broadcast room through the virtual image that the virtual image completes the action indicated by the anchor's command information; the live broadcast data adjustment module is also used to add the command interaction data to the live broadcast data in the live broadcast room.

[0025] In some possible implementations, the live broadcast data adjustment module is further used to determine target subtitle data based on the host's voice data; and add the target subtitle data to the live broadcast data in the live broadcast room.

[0026] In some possible implementations, the live broadcast data adjustment module is also used to parse the host's voice data and determine the live broadcast room operation instructions; and adjust the live broadcast data of the live broadcast room according to the live broadcast room operation instructions.

[0027] In a third aspect, the present application provides a computing device cluster, wherein the computing device includes at least one computing device, and the at least one computing device includes at least one processor and at least one memory; the at least one memory is used to store instructions, and the at least one processor executes the instructions stored in the at least one memory, so that the computing device cluster executes the multi-role multi-terminal real-time synchronous code review method in the above-mentioned second aspect or any possible implementation of the second aspect. It should be noted that the memory can be integrated into the processor or can be independent of the processor. The at least one computing device may also include a bus. The processor is connected to the memory via a bus. The memory may include a readable memory and a random access memory.

[0028] In a fourth aspect, the present application provides a computer-readable storage medium, which stores instructions. When the computer-readable storage medium is run on at least one computing device, the at least one computing device executes the method described in the first aspect or any implementation of the first aspect.

[0029] In a fifth aspect, the present application provides a computer program product comprising instructions, which, when executed on at least one computing device, enables the at least one computing device to execute the method described in the first aspect or any one of the implementations of the first aspect.

[0030] Based on the implementation methods provided in the above aspects, this application can also be further combined to provide more implementation methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0032] FIG1 is a schematic diagram of an application scenario provided by an embodiment of the present application;

[0033] FIG2 is a flow chart of a virtual image control method provided in an embodiment of the present application;

[0034] FIG3 is a schematic diagram of a structure of a virtual image control provided by an embodiment of the present application;

[0035] FIG4 is a schematic diagram of a structure of a computing device provided in an embodiment of the present application;

[0036] FIG5 is a schematic diagram of a structure of a computing device cluster provided in an embodiment of the present application;

[0037] FIG6 is a schematic diagram of an implementation method of a computing device cluster provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] The following will describe the solutions in the embodiments provided in this application in conjunction with the drawings in this application.

[0039] The terms "first," "second," and the like in the specification and claims of this application and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate and are merely used to describe the manner in which objects with the same attributes are described in the embodiments of this application.

[0040] Live streaming involves interaction between the host and the audience. Viewers can use barrages to express their opinions or questions, and the host can view and provide feedback, which may include both video and audio. For example, viewers can send barrages to the host, who can view and respond to them. Another example is a host expressing their opinions during a live broadcast, and viewers can use barrages to comment on their opinions.

[0041] However, in some scenarios, the host has a lot of work to do, making it difficult to balance the live broadcast content and interaction with the audience. For example, in a gaming live broadcast, the host may not be able to read and respond to the barrage comments in time while playing the game. Another example is a live broadcast of selling goods, where the host may not be aware of certain information about the products being sold in the live broadcast room. If the host is unable to interact with the audience in a timely manner, it may affect the audience's viewing experience.

[0042] For this reason, in some live broadcast rooms, there may be a role of a live broadcast assistant, which is used to assist the anchor to interact with the audience. In the embodiment of the present application, assisting the anchor to interact with the audience can be referred to as assisting the broadcast. In the embodiment provided in the present application, the assisting anchor has a virtual image.

[0043] Optionally, some streamers may employ staff members as livestream assistants, allowing them to handle some or all of the interaction with viewers. Specifically, these assistants can focus on the barrage of comments in the livestream and respond promptly to them, thereby enhancing audience interaction. For example, the assistants might be responsible for answering questions from the barrage, thanking guests for gifts, and adjusting relevant parameters of the livestream. Furthermore, the assistants can adjust settings such as brightness, filters, and bitrate.

[0044] However, this method requires hiring additional staff, which is costly. Furthermore, the staff acting as live broadcast assistants may make mistakes and miss interactive content.

[0045] Alternatively, some broadcasters can use a bullet screen reader to assist in interacting with their audience. The bullet screen reader can be software installed on the terminal device used by the broadcaster. The bullet screen reader can obtain the bullet screen data in the live broadcast room, and thereby obtain the bullet screen data sent by the audience, convert some or all of the bullet screen into voice, and play it back to the broadcaster, so that the broadcaster can receive the bullet screen sent by the user in the form of sound. The broadcaster can then respond based on the voice. This eliminates the need for the broadcaster to personally check the bullet screen, improving the efficiency of the interaction between the broadcaster and the audience. However, this method actually still relies on the broadcaster to complete the interaction with the bullet screen. Therefore, when the broadcaster is busy, he or she may not be able to interact with the audience in a timely manner.

[0046] Based on this, embodiments of the present application provide a method for controlling a virtual avatar. The method can be implemented by a virtual avatar control device in a live broadcast system, and can be used to assist the broadcast through a virtual avatar. The virtual avatar control device can be deployed in at least one data center. Specifically, during a live broadcast in a target live broadcast room, the virtual avatar control device can obtain live broadcast interaction data. This live broadcast interaction data includes live commentary data (i.e., commentary data sent by viewers to the host) and host voice data (i.e., voice data emitted by the host in the target live broadcast room during the live broadcast). The virtual avatar control device can then determine a target unanswered question based on the live commentary data. The target interaction content can then be determined based on the unanswered question and the live broadcast interaction data. The target interaction content includes the response to the target unanswered question. The virtual avatar control method can then determine target interaction data based on the target interaction content and add the target interaction data to the live broadcast data in the target live broadcast room. The target interaction data is used to display the target interaction content to viewers in the target live broadcast room via the virtual avatar. When viewing the live broadcast data with the added target interaction data, viewers can see both the live broadcast data corresponding to the host and the process of the virtual avatar displaying the target interaction content. During the live broadcast process, the virtual image control device can actively obtain the live broadcast interaction data of the live broadcast room, and determine the corresponding target interactive content based on the questions raised in the barrage, and then display the target interactive content to the audience in the target live broadcast room through the virtual image. In this way, the target interactive content can be automatically determined based on the live broadcast interaction data, and the target interactive content can also be displayed to the audience through the virtual image. Automatic interaction with the audience is achieved through the virtual image, so that the interaction between the audience and the anchor can receive a timely response, thereby improving the viewing experience of the live broadcast. In addition, using the virtual image to assist the broadcast also means that the anchor does not need to focus on the interaction with the audience, but can focus more on the live broadcast content itself.

[0047] The live broadcast system includes a host terminal, a live broadcast server, and a viewer client. The host terminal provides services to assistant broadcasters, while the viewer client provides services to viewers. For example, the host terminal may include a host client and a host server. The viewer client may include a viewer client. The host client may be live broadcast software installed on the host's terminal device, used to collect relevant data during the host's live broadcast. The host server may be software installed on a server, used to process relevant data during the live broadcast. Optionally, the host server may be installed in at least one data center. The viewer client refers to software installed on the user's terminal device.

[0048] The host client can obtain the live data stream and send it to the host server. The host server can process the live data stream and forward it to the live server. The live server can send the live data stream to the viewer client. The viewer client can receive the live data stream and play it to the user in the form of video (or audio). As an example, the virtual image control device can be deployed on the host side of the live broadcast system. As another example, the virtual image control device can be deployed on the live broadcast server of the live broadcast system. The following description uses the virtual image control device deployed on the host side as an example.

[0049] For example, in the application scenario shown in Figure 1, anchor A can use the anchor terminal 10 for live broadcast, audience B can use the audience client 20 to watch the live broadcast of anchor A, and audience C can use the audience client 30 to watch the live broadcast of anchor A. The live broadcast server 40 is used to realize data processing and forwarding during the live broadcast.

[0050] The host terminal 10 includes a live broadcast interactive device 110 and an avatar control device 120. The live broadcast interactive device 110 is used to collect data generated by host A during the live broadcast to obtain a live broadcast data stream, and transmit the live broadcast data stream to the live broadcast server 40. The data collected by the live broadcast interactive device 110 may include, for example, at least one of video data collected by a camera, audio data collected by a microphone, and video and / or audio data collected by screen recording. The live broadcast interactive device 110 can also receive barrage data sent by the live broadcast server 40 and display the barrage data to host A.

[0051] The avatar control device 120 can collect live interaction data from host A during the live broadcast, determine target interactive content based on the live interaction data, and then, based on the avatar, determine target interactive data during the avatar's display of the target interactive content, adding the target interactive data to the live data stream. In this way, the live data stream received by the live broadcast server 40 includes both the data collected by the live interaction device 110 and the target interactive data inserted by the avatar control device 120. The video data viewed by the viewer clients 20 and 30 then includes both the video data broadcast by host A and the video data of the target interactive content displayed by the avatar. In this way, the avatar can assist the host in interacting with the audience.

[0052] As shown in Figure 1, the virtual image control device 120 may include an acquisition module 121, a question determination module 122, an interactive content determination module 123, an interactive data determination module 124, and a live broadcast data adjustment module 125. The acquisition module 121 may be used to acquire live broadcast interaction data from a target live broadcast room. The question determination module 122 may be used to determine target questions to be answered based on live broadcast barrage data. The interactive content determination module 123 may be used to determine target interactive content based on the target questions to be answered. The interactive data determination module 124 may be used to determine target interactive data based on the target interactive content and the virtual image. The live broadcast data adjustment module 125 may be used to add the target interactive data to the live broadcast data of the target live broadcast room.

[0053] In the implementation shown in Figure 1, the avatar control device 120 runs on the host terminal 10. Optionally, the host terminal 10 may include a host client and a host server. The host client may be installed on the terminal device used by host A, while the host server may be installed in a data center. Alternatively, the avatar control device 120 may be a module within the host client or within the host server. In other words, the avatar control device 120 may be located on the terminal device or in the data center.

[0054] Next, various non-limiting specific implementations of the virtual image control process are described in detail.

[0055] Refer to Figure 2, which is a flow chart of the virtual image control method in an embodiment of the present application. This method can be applied to the application scenario shown in Figure 1 above, or it can also be applied to other applicable application scenarios. The following description will be given using the application scenario shown in Figure 1 as an example. It should be noted that in the application scenario shown in Figure 1, the virtual image control device 120 can not only include an acquisition module 121, an interactive content determination module 123, an interactive data determination module 124 and a live broadcast data adjustment module 125, but can also include a virtual image knowledge base 126 and an emotion perception module 127. In addition, for the functions of each module, please refer to the relevant description of the following embodiment.

[0056] The method shown in FIG2 may specifically include:

[0057] S201: The acquisition module 121 acquires the live broadcast interaction data of the live broadcast room.

[0058] In order to assist the host in interacting with the audience in the target live broadcast room, the avatar control device 120 can obtain the live broadcast interaction data of the target live broadcast room, thereby determining the content to be displayed during the interaction process based on the live broadcast interaction data. Specifically, the avatar control device 120 can obtain the live broadcast interaction data of the target live broadcast room through the acquisition module 121.

[0059] In the embodiments of the present application, the live broadcast room refers to the live broadcast room corresponding to the virtual image control device 120, which can also be referred to as the target live broadcast room. For example, in the implementation shown in Figure 1, the live broadcast room corresponding to anchor A can be regarded as the target live broadcast room. For the sake of convenience, the anchor corresponding to the target live broadcast room can be referred to as the target anchor in this application.

[0060] Live interaction data refers to the interactive data generated by the target live broadcast room during the live broadcast, used to convey information to the other party. Interaction data refers to the data sent by one object to convey information to other objects. Live interaction data can include live barrage data and host voice data. Live barrage data is the data sent by users while watching the live broadcast in the target live broadcast room, used to convey information to the host. Host voice data is the data corresponding to the voice uttered by the target host during the live broadcast, used to interact with other objects.

[0061] Optionally, the host voice data may include audio data emitted during the target host's communication with viewers, or may include audio data emitted during the target host's communication with other parties. For example, if the target host is connected to another host via a live broadcast, the host voice data may also include audio data emitted by the target host during the live broadcast.

[0062] In an embodiment of the present application, live broadcast barrage data can be obtained from the live broadcast server 40. Specifically, the audience of the target live broadcast room can send barrages to the live broadcast server 40 through the audience client. The live broadcast server 40 can receive barrages sent by multiple viewers who are watching the target live broadcast room, and send these barrages to the anchor terminal 10 so that the anchor client can display the barrages sent by the audience to the anchor. Accordingly, the acquisition module 121 can obtain the live broadcast barrage data sent by the live broadcast server 40. Optionally, the live broadcast interaction device 110 can receive the barrage data sent by the live broadcast server. The acquisition module 121 can obtain the live broadcast barrage data sent by the live broadcast server 40 from the live broadcast interaction device 110 through a preset interface.

[0063] In the implementation shown in Figure 1, the avatar control device 120 is deployed on the host's terminal 10. Accordingly, the acquisition module 121 can obtain the host's voice data through relevant modules of the host's client. For example, it can obtain the host's voice data through the microphone of the host's terminal device. As previously described, the voice of the target live stream can be transmitted from the host's terminal 10 to the live streaming server 40, which then transmits it to the viewer client of the audience watching the target live stream. In this way, the acquisition module 121 can obtain the host's voice data from the host's terminal 10.

[0064] It's understood that if the avatar control device is deployed elsewhere, the host's voice data can be obtained by parsing the live data stream of the target live broadcast room. For example, assuming the avatar control device is deployed on the live broadcast server, the avatar control device can parse the live data stream sent by the host to obtain the host's voice data.

[0065] In live broadcast scenarios, data is mostly transmitted in the form of data streams. Accordingly, the live broadcast interaction data acquired by acquisition module 121 can be data in data stream format. In other words, acquisition module 121 can acquire the live broadcast barrage data stream and the host's voice data stream. Accordingly, subsequent processing based on the live broadcast interaction data can also be based on the data stream.

[0066] After acquiring the live interaction data of the target live broadcast room, the acquisition module 121 may send the live interaction data to the interactive content determination module 123 so that the interactive content determination module 123 may determine the target interactive content according to the target question to be answered.

[0067] S202: The question determination module 122 determines the target question to be answered based on the live broadcast barrage data.

[0068] After acquiring live interaction data, the question determination module 122 can determine target questions to be answered based on the live broadcast barrage data, thereby controlling the avatar to respond to the questions. The target questions to be answered are questions posed by viewers watching the target live broadcast room via barrage. In this embodiment of the present application, the avatar control device 120 can control the avatar to answer some or all of the questions posed by viewers, thereby enhancing the viewer's live broadcast experience and fulfilling the function of a live broadcast assistant.

[0069] In order to control the virtual image to answer questions raised by the audience. First, the questions that need to be answered can be determined. In the embodiment of the present application, the questions to be answered, which are raised by the audience through the barrage and determined by the question determination module 122, can be referred to as target questions to be answered. Optionally, the target questions to be answered can be one question or multiple questions. Specifically, the question determination module 122 can parse the live barrage data, determine the questions asked by the audience in the target live broadcast room, and then determine the questions to be answered from the questions asked by the audience in the live broadcast room.

[0070] In actual scenarios, the target live broadcast room may have a large number of viewers, and each viewer watching the target live broadcast room can send barrages. The barrages sent by viewers can be used to ask questions to the target anchor, and can also have other purposes. Each viewer may ask questions to the target anchor, and the questions asked by different viewers may be the same or different. Moreover, the same viewer may ask multiple different questions to the target anchor through multiple barrages, or may ask multiple identical or similar questions to the target anchor through multiple barrages. This results in the possibility that there may be multiple barrages for asking questions in the live broadcast barrage data, and the questions raised in different barrages may be the same or different. In addition, in addition to asking questions to the target anchor, viewers in the target live broadcast room can also achieve other purposes by sending barrages. This results in the presence of both barrages for asking questions and barrages that are not used for asking questions in the live broadcast barrage data.

[0071] To this end, the question determination module 122 can first pre-process the barrage in the live barrage data, filter out non-repetitive barrages used to ask questions to the target anchor from the live barrage data, and then determine the questions to be answered based on these barrages. Optionally, the live barrage data can be clustered first to determine a barrage intention information set, and then the target questions to be answered are determined based on the barrage intention information set. The barrage intention information set includes multiple barrage intention information, each barrage intention information corresponds to the intention information of one (or more) barrages, indicating the intention of the viewer who sent this (or more) barrages.

[0072] First, the problem determination module 122 can first perform repeated filtering on the live broadcast barrage data. Among them, repeated filtering refers to deleting barrages with consistent content. In this way, barrages with consistent content can be deleted from the live broadcast barrage data, thereby reducing the number of barrages that need to be processed in subsequent steps.

[0073] Next, the question determination module 122 can cluster the filtered live broadcast barrage data to obtain a barrage intention information set. For example, clustering can be performed based on the similarity of the barrages, or clustering can be performed based on the semantics of the barrages. The embodiment of the present application does not limit the clustering method. After determining the barrage intention information set, the question determination module 122 can determine multiple questions to be answered from the barrage intention information set. Each question to be answered corresponds to a barrage intention information, which represents a question raised by one (or more) viewers through the barrage.

[0074] After clustering barrages, comments of the same type are grouped together. That is, if there are multiple comments with similar expressions in the live broadcast comment data, these comments will be clustered together. Comments of different types express different meanings, while comments of the same type express the same meaning. This allows subsequent processing to use a single comment from a category to replace that category, ignoring the remaining comments in that category. For example, the comment at the center of the cluster can be used as the representative of that category for subsequent processing. This method of replacing a category with a single comment further reduces the number of comments to be processed, improving processing efficiency. Furthermore, by clustering barrages, comments that are not used to ask questions to the target streamer can be filtered out. For example, if the live broadcast comment data includes comments that ask questions to the target streamer as well as comments that are used to communicate with other viewers, after clustering the live broadcast comment data, comments with different purposes will be clustered into multiple categories. Comments that ask questions to the target streamer will not be clustered with other comments. Then, when determining the questions to be answered, we can only focus on those used to ask questions to the target anchor and ignore the bullet comments for other purposes, thereby reducing the number of bullet comments to be processed. The embodiment of the present application does not limit the specific implementation method of clustering.

[0075] After determining multiple unanswered questions, the question determination module 122 determines a target unanswered question from the multiple unanswered questions. Optionally, the question determination module 122 may evaluate the question based on its importance. For example, the question determination module 122 may evaluate the importance of each unanswered question to obtain an importance evaluation result for the unanswered question. The importance evaluation result for the unanswered question indicates the importance of the unanswered question.

[0076] In some implementations, the question determination module 122 can perform an importance assessment on the question to be answered in combination with at least one assessment influencing factor, thereby obtaining an importance assessment result. It should be noted that the at least one assessment influencing factor used to perform the importance assessment on the target question to be answered can be an assessment influencing factor configured by the administrator; it can also be an assessment influencing factor adjusted from the initially set assessment influencing factor based on actual business needs; it can also be an assessment influencing factor newly configured by the administrator before receiving the target question to be answered. The embodiment of the present application does not limit the number and content of the configured assessment influencing factors.

[0077] In some embodiments, the configuration process of an evaluation rule includes receiving an evaluation rule configuration request, the evaluation rule configuration request being used to request configuration of one or more of a type, a priority, and a weight of at least one evaluation influencing factor. Based on this, at least one evaluation influencing factor, along with the priority and weight of each evaluation influencing factor, is stored based on the information contained in the evaluation rule configuration request.

[0078] Optionally, when there are multiple configured evaluation influencing factors, the question determination module 122 may also select a preset number of evaluation influencing factors with higher priorities to perform importance evaluation on the questions to be answered in response to the administrator's configuration. The embodiment of the present application does not limit the number of evaluation influencing factors used when performing importance evaluation on the questions to be answered.

[0079] After determining the importance assessment results for each pending question, target pending questions can be determined based on the importance assessment results. Alternatively, the N most important pending questions can be determined as target pending questions. N is a positive integer. Alternatively, N can be a pre-configured number of questions that the avatar can answer.

[0080] In some implementations, the questions raised by the audience through the barrage may be relatively abstract, making it difficult to determine the content of the answer. Accordingly, after selecting the questions to be answered, before determining the target questions to be answered, the questions to be answered can be clustered to determine the root questions of the questions to be answered, and the root questions can be used as the target questions to be answered. Optionally, the question determination module 122 can determine the root questions corresponding to the questions to be answered from the corpus through semantic clustering. The corpus pre-stores multiple root question corpora, and each root question corpus corresponds to one root question. Optionally, the corpus can be a virtual image knowledge base 126.

[0081] In one possible implementation, the multiple fundamental question corpora pre-stored in the corpus can be divided into multiple levels, each level corresponds to at least one fundamental question corpus, and the granularity of the fundamental question and answer corpora corresponding to different levels can be different. Optionally, the corpus includes multiple corpus scenarios, such as live broadcast, conference chat, webinar, online class, etc.; each corpus scenario corresponds to a category tree, which includes multiple levels, each level has different granularity, each level includes at least one category node, and each category node corresponds to at least one fundamental question. Granularity indicates the degree of detail of the description of the fundamental question and answer corpus in this level. The granularity of the high level is large, and the problem description of the fundamental problem corresponding to its category node is relatively macro; the granularity of the low level is small, and the problem description of the fundamental problem corresponding to its category node is more detailed.

[0082] When categorizing unanswered questions, the classification can be performed based on a target hierarchy. Specifically, multiple fundamental questions included in the target hierarchy can be first determined, and then the fundamental questions corresponding to the unanswered questions can be selected from the multiple fundamental questions as target unanswered questions. The target hierarchy used for classification can be a pre-set hierarchy. This allows multiple fundamental questions to be obtained from the fundamental question and answer corpus corresponding to the target hierarchy, making the selection of target unanswered questions more accurate. This also reduces the computational complexity of corpus-based semantic clustering, significantly improving the efficiency of semantic clustering.

[0083] In actual scenarios, viewers in the target live broadcast room may ask the target anchor multiple questions, and the target anchor may answer some of these questions but not the rest. Therefore, when controlling the avatar to interact with the audience, it is possible to answer questions raised by the audience that have not been answered by the target anchor, while remaining silent on questions that have already been answered by the target anchor. In other words, when determining the target unanswered questions, the question determination module 122 may select unanswered questions as the target unanswered questions.

[0084] Specifically, the question determination module 122 can determine at least one pending question based on the live broadcast barrage data, and then determine the target pending question from the pending questions based on the host's voice data. Specifically, the question determination module 122 can analyze the host's voice data to determine which pending questions have been answered by the target host based on the host's voice data. Next, the question determination module 122 can exclude the pending questions that have been answered by the target host from the barrage question set. The remaining pending questions in the barrage question set are then the pending questions that have not been answered by the target host. The question determination module 122 can determine the target pending question from these.

[0085] In some possible implementations, the question determination module 122 can convert the speech of the target host into text, thereby identifying the pending questions that the target host has answered through the text. Specifically, the question determination module 122 can invoke the speech-to-text function of the host terminal 10 to convert the host's speech data into text data. The question determination module 122 can then analyze the converted text data to determine the information expressed by the target host during the live broadcast. The text data can then be matched with the barrage question information set to determine the portion of the text data that answers the pending questions raised by the audience.

[0086] Optionally, the avatar control device 120 can assist the target anchor in interacting when the target anchor pauses. A pause in the target anchor's speech can include a pause in the target anchor's speech. Specifically, if the avatar control device 120 controls the avatar to interact with the audience while the target anchor is interacting with the audience, this may result in two entities interacting with the audience simultaneously during the live broadcast, affecting the audience's viewing experience. To this end, in some possible implementations, the avatar can be controlled to interact with the audience when the anchor pauses.

[0087] That is, the question determination module 122 can determine whether the target host has paused based on the host's voice data. Specifically, the question determination module 122 can determine whether there are pauses in the host's voice data stream. For example, the host's voice data stream can be monitored to determine whether there is a portion containing the host's voice. If a pause occurs, for example, if the host's voice does not appear within a preset duration, it means that the target host has paused during the live broadcast. In this way, the question determination module 122 can analyze the host's voice data to determine the target question to be answered. This can prevent the target host and the virtual image from interacting with the audience at the same time, thereby improving the audience's experience of watching the live broadcast.

[0088] Furthermore, when the host pauses, there's a high probability that questions posed by viewers won't be answered. For example, the host may leave the terminal device used for the live broadcast for some reason, preventing them from seeing the questions posed by viewers and, therefore, from answering them. Another example is that the host may turn off their microphone for some reason, preventing the host's voice from being captured, and the audience from receiving their responses. In these scenarios, the host's pauses also lead to questions posed by viewers not being answered in a timely manner. Therefore, after determining that the target host has paused, interacting with the audience through the virtual avatar control device 120 can promptly overcome the problem of viewers' questions not being answered in a timely manner.

[0089] Optionally, the live broadcast interaction data may be determined after determining that the target live broadcast is in the live broadcast process. For example, the barrage data within a preset time before a pause may be used as the live barrage data in the live broadcast interaction data, and / or the audio data emitted by the host within a preset time before a pause may be used as the host voice data in the live broadcast interaction data.

[0090] S203: The interactive content determination module 123 determines target interactive content according to the target question to be answered.

[0091] After determining the target unanswered question, the interactive content determination module 123 can determine the target interactive content based on the target answer question. The target interactive content can include the answer to the target unanswered question. In other words, to assist the target anchor in interacting with the audience in the target live broadcast room, the avatar control device 120 can mimic the target anchor and determine the target interactive content for responding to the barrage. In other words, the target interactive content can be used to respond to the barrage on behalf of the target anchor.

[0092] In this embodiment of the present application, the target host's response to the barrage includes the target host's answer to a question posed by the audience. Accordingly, the avatar control device 120 can control the avatar to answer the audience's question on behalf of the target host. The target interactive content can include the answer to the audience's question. That is, the interactive content determination module 123 can determine the content required to answer the audience in the target live broadcast room based on the live broadcast interaction data.

[0093] After determining the target question to be answered, the interactive content determination module 123 may determine the answer to the target question to be answered. The answer to the target question to be answered is the aforementioned target interactive content. Optionally, if the interactive content determination module 123 determines multiple target questions to be answered, the target interactive content may include the answer to each target question to be answered. Furthermore, the order of the answers to the target questions to be answered in the target interactive content may match the order in which the target questions to be answered were posed.

[0094] Optionally, the answer content for the target question to be answered can be determined based on the virtual image knowledge base 126 of the host terminal 10, and then the target interactive content can be generated based on the answer content. The answer content for the target question to be answered can be the answer to the target question to be answered. The virtual image knowledge base 126 is a pre-set database used to answer questions raised by the audience. The virtual image knowledge base 126 can include multiple preset questions and corresponding answers to each preset question. After determining the target question to be answered, a preset question corresponding to the target question to be answered can be selected from the multiple preset questions based on the target question to be answered, and the answer to the preset question can be determined as the target interactive content.

[0095] Optionally, the avatar control device 120 can also update the avatar knowledge base 126. For example, if the target host proactively answers a question posed by an audience member, the avatar knowledge base 126 can be updated based on the target host's answer and the audience member's question. Specifically, if the target host proactively answers a target unanswered question, the avatar knowledge base can be updated based on the target unanswered question and the audio data emitted by the target host to answer the unanswered question.

[0096] In the implementation described above, the target interactive content determined by the interactive content determination module 123 can be used to replace the host in answering the audience's questions. In some other possible implementations, the interactive content determination module 123 can also determine the target interactive content in other interactive scenarios.

[0097] In some implementations, the target interactive content can also be used to control the avatar to perform certain actions. The interactive content determination module 123 can determine the action the avatar needs to perform based on the host's voice data and / or live comment data. For example, the target host can instruct the avatar to perform certain actions through voice, or viewers can instruct the avatar to perform certain actions through commentary. For ease of explanation, the action the avatar is instructed to perform will be referred to as the target action.

[0098] For example, consider a target host instructing a virtual avatar to complete a target action through voice. The interactive content determination module 123 can first determine the instruction information based on the host's voice data, and then determine the target interactive content based on the instruction information. The target interactive content corresponds to the target action and can, for example, include model data of the virtual avatar completing the target action. By rendering this model data, video data of the virtual object completing the target action can be obtained.

[0099] Optionally, multiple preset instruction information can be pre-set in the avatar control device 120, and a correspondence between the preset instruction information and interactive content can be established. In this way, the target host and / or the audience in the target live broadcast room can instruct the avatar to complete the corresponding interactive content through the preset instruction information. Specifically, after parsing the host's voice data or live broadcast barrage data to obtain instruction information, if the instruction information matches a certain preset instruction information, the interactive content corresponding to the preset instruction information can be determined as the target interactive content.

[0100] After the interactive content determination module 123 determines the target interactive content, the interactive content determination module 123 may send the target interactive content to the interactive data determination module 124 so that the interactive data determination module 124 may determine the target interactive data.

[0101] S204: The interaction data determination module 124 determines target interaction data according to the target interaction content.

[0102] After determining the target interactive content based on the live broadcast interaction data, the interactive data determination module 124 can determine the target interactive data based on the target interactive content. The target interactive data user displays the target interactive content, including target video data and target audio data, to the audience in the target live broadcast room via an avatar. The avatar can be a pre-set animated or physical image, used to assist the target host in interacting with the bullet comment.

[0103] The target interaction data includes media data for displaying the target interaction content to the audience of the target live broadcast room through the virtual image. By inserting the target interaction data into the live broadcast data of the target live broadcast room, the audience of the target live broadcast room can see and / or hear the target interaction content displayed by the virtual image. Specifically, the target interaction data may include video data, which is data for the virtual image to display the target interaction content, for example, it may include animation data of the virtual image completing the target action. The target interaction data may also include audio data, which corresponds to the target interaction content. In order to distinguish it from the audio data and video data in the live broadcast data stream, the audio data in the target interaction data will be referred to as target audio data, and the video data in the target interaction data will be referred to as target video data.

[0104] As previously mentioned, the target interactive content can include answers to questions raised by the audience, or it can include model data of the avatar completing the target action. The following describes these two implementation methods.

[0105] In a first implementation, the target interactive content includes answers to questions posed by the audience. The target interactive data may include target audio data and target video data. When determining the target interactive data, the interactive data determination module 124 may first determine the target audio data based on the target interactive content, and then determine the target video data based on the target audio data and the avatar.

[0106] Specifically, if the target interactive content is in text format, the interactive data determination module 124 may first convert the target interactive content into target audio data. For example, the target audio data corresponding to the target interactive content may be generated based on a preset audio library. The target audio data is used to simulate the avatar reading the text corresponding to the target interactive content. In this way, the target audio data is inserted into the live broadcast data, allowing users viewing the target live broadcast room to listen to the target audio data and thus understand the answer to the question posed.

[0107] Optionally, the interactive data determination module 124 can also generate target audio data with reference to the situation of the target anchor, so that the generated target audio data is combined with the situation of the target anchor, so that the virtual image matches the actual situation of the target anchor, thereby improving the auxiliary broadcast effect.

[0108] Specifically, the host's voice data can be analyzed by the emotion perception module 127 to determine the target host's emotional information during the live broadcast. Then, the interactive data determination module 124 can generate target audio data corresponding to the emotional information based on the target interactive content. Optionally, multiple audio databases can be pre-set in the virtual image control device 120, and different audio databases correspond to different emotional information. For example, the audio of audio database A in the multiple audio databases can be recorded under an angry emotion, and the audio of audio database B in the multiple audio databases can be recorded under a happy emotion. After determining the emotional information, the audio database that matches the emotional information can be selected based on the emotional information, and the target audio data can be generated using the selected audio database. In this way, the emotions of the generated target audio data match the actual emotions of the target host, and the emotions of the target host felt by the audience in the target live broadcast room match the emotions of the virtual image, achieving the effect of the virtual image simulating the emotions of the target host.

[0109] After generating the target audio data based on the target interactive content, corresponding target video data can be generated based on the target audio data and the virtual image. The target video data can be a video of the virtual image emitting the target audio data. That is, if the virtual image includes a face, the target video data can show the facial movements of the virtual image during the emission of the target audio data. For example, if the virtual image is an animated image corresponding to the target anchor, the video data can include an animation of the animated image emitting the sound corresponding to the target audio data. The target video data can be synchronized with the target audio data. That is, the virtual image displayed by the target video data at a certain point in time is consistent with the virtual image that emitted the target audio data at that point in time.

[0110] In this way, after the target video data and target audio data are inserted into the live broadcast data of the target live broadcast room, the audience watching the target live broadcast room can hear the audio corresponding to the target interactive content and see the virtual avatar emitting the audio corresponding to the target interactive content. In this way, the audience watching the target live broadcast room is equivalent to watching the virtual avatar read out the audio corresponding to the target interactive content, achieving the purpose of having the virtual avatar answer questions raised by the audience in the target live broadcast room, and improving the viewing experience of the audience in the target live broadcast room.

[0111] In a second implementation, the target interactive content includes model data of the virtual image in the process of completing the target action.

[0112] The target interaction data may include target video data. When determining the target interaction data, the interaction data determination module 124 may render the model data as visualized two-dimensional image data to obtain target video data. Alternatively, if the avatar is an animated avatar, the rendered target video data may be animation data of the avatar performing the target action.

[0113] Optionally, the target interaction data may also include target audio data. The target audio data may correspond to the target video data. For example, the target audio data may be determined based on the target action. Specifically, assuming that multiple actions that the avatar can perform are preconfigured, audio data may be set for one or more of the multiple actions. When determining the target interaction data, the interaction data determination module 124 may select the audio data corresponding to the target action as the target audio data.

[0114] After acquiring the live broadcast interaction data, the interaction data determination module 124 may send the target interaction data to the live broadcast data adjustment module 125 so as to insert the target interaction data into the live broadcast data of the target live broadcast room.

[0115] S205: The live broadcast data adjustment module 125 adds the target interactive data to the live broadcast data of the target live broadcast room.

[0116] After determining the target interaction data corresponding to the live broadcast interaction data, the live broadcast data adjustment module 125 may add the target interaction data to the live broadcast data of the target live broadcast room.

[0117] In actual scenarios, the live broadcast data of the target live broadcast room can be a data stream. The live broadcast interactive device 110 can collect the video data stream and the audio data stream and upload them as the live broadcast data stream of the target live broadcast room to the live broadcast server 40. In the embodiment of the present application, the live broadcast data adjustment module 125 can add the target interactive data in the form of a video stream to the live broadcast data stream of the target live broadcast room.

[0118] Specifically, if the target interactive data includes target video data and target audio data, the live data adjustment module 125 can first determine the target video data stream and the target audio data stream, and synchronize the target video data stream and the target audio data stream with the live data stream of the target live room.

[0119] The live data adjustment module 125 can add the target video data stream to the live data stream to obtain a video data stream with the target interactive data added. For example, the live data adjustment module 125 can add the frame corresponding to the target video data stream to a specific location in the live broadcast screen of the target live broadcast room. In this way, when watching the target live broadcast, viewers in the target live broadcast room can see the virtual avatar displaying the target interactive content at the specific location, while seeing the target host's live broadcast content at locations outside the specific location.

[0120] The live broadcast data adjustment module 125 can add the target audio data stream to the audio data stream of the live broadcast data stream to obtain an audio data stream with the target interactive data added. For example, the live broadcast data adjustment module 125 can superimpose the target audio data stream with the audio data stream of the live broadcast data stream. In this way, when watching the target live broadcast, viewers in the target live broadcast room can hear the sound produced by the virtual avatar during the display of the target interactive content, as well as the voice of the target anchor during the live broadcast.

[0121] In this way, viewers watching the target livestream can not only see the target host during the livestream, but also see the target interactive content displayed through the avatar. This is equivalent to using the avatar to assist the host in interacting with the audience in the livestream, allowing viewers to receive timely responses to their interactions with the host, thus improving the viewing experience of the livestream.

[0122] In the above implementations, the avatar control device can control the avatar to assist the host in interacting with the audience in the live broadcast room. In some possible implementations, the avatar control device can also assist the host in live broadcasting. The following describes two implementations of the avatar control device assisting the host in live broadcasting. It is understood that the avatar control device can also assist the host in live broadcasting in other ways. These details will not be elaborated here.

[0123] In a first implementation, the virtual image control device may have language translation and subtitle display functions to assist the target anchor and the audience watching the target live broadcast room to understand each other's language.

[0124] In real-world scenarios, the host and the audience may not speak the same language. For example, the target host and the audience in the target livestream may be from different countries or regions. Consequently, the audience in the target livestream may not understand what the host is saying during the livestream, nor may the host understand the meaning of the comments sent by the audience. Therefore, the avatar control device can include a language translation function to facilitate communication between the host and the audience in the target livestream.

[0125] On one hand, after acquiring the live broadcast barrage data, the avatar control device can translate the live broadcast barrage data from the second target language into text in the first target language. The avatar control device can then display the translated text in the first target language on the host's end. The target host can understand the content of the barrage sent by viewers in the target live broadcast room by viewing the content displayed by the avatar control device. The first target language is the language used by the target host, and the second target language is the language used by viewers in the target live broadcast room. Optionally, the first target language and second target language can be set on the host's end.

[0126] On the other hand, after acquiring the host's voice data, the avatar control device can first convert the host's voice data into text data, and then convert the text in the first language into text in a second target language to obtain target subtitle data. The avatar control device can then add the target subtitle data to the live broadcast data of the target live broadcast room. For example, the avatar control device can generate a video data stream based on the target subtitle data and add the video data stream generated based on the target subtitle data to the live broadcast data stream of the target live broadcast room. In this way, viewers watching the target live broadcast room can see the target subtitle data in the live broadcast screen of the target live broadcast room, thereby understanding the words spoken by the target host during the live broadcast.

[0127] In the second implementation method, the virtual image control device can assist the target anchor to control the target live broadcast room.

[0128] In real-world scenarios, the host can control various live broadcast parameters in the live broadcast room, such as adjusting the brightness of the live broadcast room, the live broadcast room filter, the camera angle, etc. Currently, controlling the live broadcast parameters of the live broadcast room requires the host to do it manually, which is inefficient. In the embodiments of the present application, the virtual image control device can help the target host adjust the live broadcast parameters of the target live broadcast room, thereby improving the target host's live broadcast efficiency.

[0129] Specifically, the target host can issue live broadcast room operation commands via voice, which are used to adjust the live broadcast parameters of the live broadcast room. After acquiring the host's voice data, the virtual image control device can parse the host's voice data to determine the live broadcast room operation commands issued by the target host. The virtual image control device can then adjust the live broadcast parameters of the target live broadcast room based on the live broadcast room operation commands. In this way, the target host can adjust the live broadcast parameters of the target live broadcast room via voice, eliminating the need for manual adjustment by the target host, thereby improving live broadcast efficiency.

[0130] The present application also provides a virtual image control device, wherein the virtual image control device can be applied to the host terminal 10 in the implementation shown in FIG1 , for example, it can be run on a host client or a host server to implement the functions of the virtual image control device in the implementation shown in FIG2 . Specifically, as shown in FIG3 , the virtual image control device 300 includes:

[0131] An acquisition module 310 is configured to acquire live interaction data of a live broadcast room, wherein the live interaction data includes live commentary data and the host's voice data, and the live broadcast room includes the virtual image and the host;

[0132] A question determination module 320 is configured to determine a target question to be answered based on the live broadcast barrage data, wherein the live broadcast barrage data is sent by viewers in the live broadcast room;

[0133] An interactive content determination module 330 is configured to determine target interactive content based on the target question to be answered and the live interactive data, wherein the target interactive content includes an answer to the target question to be answered;

[0134] An interaction data determination module 340 is configured to determine target interaction data based on the target interaction content, wherein the target interaction data is used to present the target interaction content to viewers in the live broadcast room through the virtual avatar, and the target interaction data includes target audio data and target video data;

[0135] The live broadcast data adjustment module 350 is used to add the target interactive data to the live broadcast data of the live broadcast room.

[0136] The acquisition module 310, problem determination module 320, interactive content determination module 330, interactive data determination module 340, and live data adjustment module 350 can all be implemented via software or hardware. For example, the implementation of the problem determination module 320 will be described below using the problem determination module 320 as an example. Similarly, the implementation of the acquisition module 310, interactive content determination module 330, interactive data determination module 340, and live data adjustment module 350 can be similar to the implementation of the problem determination module 320.

[0137] As an example of a software functional unit, the problem determination module 320 may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Furthermore, the computing instance may be one or more. For example, the problem determination module 320 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one data center or multiple geographically close data centers. Typically, a region may include multiple AZs.

[0138] Similarly, multiple hosts / virtual machines / containers running the code can be distributed within the same virtual private cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Cross-region communication between two VPCs within the same region, or between VPCs in different regions, requires a communication gateway within each VPC to interconnect the VPCs.

[0139] As an example of a hardware functional unit, problem determination module 320 may include at least one computing device, such as a server. Alternatively, problem determination module 320 may be implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0140] The multiple computing devices included in problem determination module 320 can be distributed in the same region or in different regions. The multiple computing devices included in problem determination module 320 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in problem determination module 320 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of servers, ASICs, PLDs, CPLDs, FPGAs, GALs, and other computing devices.

[0141] It should be noted that, in other embodiments, the acquisition module 310 can be used to execute any step in the virtual image control method, the problem determination module 320 can be used to execute any step in the virtual image control method, the interactive content determination module 330 can be used to execute any step in the virtual image control method, the interactive data determination module 340 can be used to execute any step in the virtual image control method, and the live broadcast data adjustment module 350 can be used to execute any step in the virtual image control method. The steps that the acquisition module 310, the problem determination module 320, the interactive content determination module 330, the interactive data determination module 340 and the live broadcast data adjustment module 350 are responsible for implementing can be specified as needed. By implementing different steps in the virtual image control method, the acquisition module 310, the problem determination module 320, the interactive content determination module 330, the interactive data determination module 340 and the live broadcast data adjustment module 350 respectively implement all functions of the virtual image control device.

[0142] This application also provides a computing device 100. As shown in FIG4 , computing device 100 includes a bus 102, a processor 104, a memory 106, and a communication interface 108. Processor 104, memory 106, and communication interface 108 communicate with each other via bus 102. Computing device 100 may be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in computing device 100.

[0143] Bus 102 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, among others. Buses may be classified as address buses, data buses, control buses, and the like. For ease of illustration, FIG4 illustrates a single bus line, but this does not imply a single bus or type of bus. Bus 102 may include a path for transmitting information between various components of computing device 100 (e.g., memory 106, processor 104, and communication interface 108).

[0144] The processor 104 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0145] The memory 106 may include volatile memory, such as random access memory (RAM). The processor 104 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0146] Memory 106 stores executable program code. Processor 104 executes this executable program code to implement the functions of the aforementioned acquisition module 310, question determination module 320, interactive content determination module 330, interactive data determination module 340, and live data adjustment module 350, thereby implementing the avatar control method. In other words, memory 106 stores instructions for executing this stored method.

[0147] Alternatively, the memory 106 stores executable code, and the processor 104 executes the executable code to implement the functions of the aforementioned avatar control device, thereby implementing the avatar control method. In other words, the memory 106 stores instructions for executing the avatar control method.

[0148] The communication interface 108 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 100 and other devices or a communication network.

[0149] Embodiments of the present application also provide a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0150] As shown in Figure 5, the computing device cluster includes at least one computing device 100. The memory 106 of one or more computing devices 100 in the computing device cluster may store the same instructions for executing the virtual character control method.

[0151] In some possible implementations, the memory 106 of one or more computing devices 100 in the computing device cluster may also store partial instructions for executing the avatar control method. In other words, the combination of one or more computing devices 100 can jointly execute the instructions for executing the avatar control method.

[0152] It should be noted that the memory 106 in different computing devices 100 in the computing device cluster can store different instructions, each for executing a portion of the functions of the avatar control device. In other words, the instructions stored in the memory 106 in different computing devices 100 can implement the functions of one or more of the acquisition module 310, the question determination module 320, the interactive content determination module 330, the interactive data determination module 340, and the live broadcast data adjustment module 350.

[0153] In some possible implementations, one or more computing devices in a computing device cluster may be connected via a network. The network may be a wide area network or a local area network, etc. FIG6 shows a possible implementation. As shown in FIG6 , two computing devices 100A and 100B are connected via a network. Specifically, the network is connected via a communication interface in each computing device. In this type of possible implementation, the memory 106 in the computing device 100A stores instructions for executing the functions of the acquisition module 310 and the live broadcast data adjustment module 350. At the same time, the memory 106 in the computing device 100B stores instructions for executing the functions of the problem determination module 320, the interactive content determination module 330, and the interactive data determination module 340.

[0154] The connection method between the computing device clusters shown in Figure 6 can be based on the consideration that the virtual image control method provided in this application can be divided into two steps: determining interactive data and adjusting live broadcast data. Therefore, it is considered that the function of determining the target interactive data is executed by the computing device 100B, and the function of adjusting the live broadcast data is executed by the computing device 100B.

[0155] It should be understood that the functions of the computing device 100A shown in FIG6 may also be completed by multiple computing devices 100. Similarly, the functions of the computing device 100B may also be completed by multiple computing devices 100.

[0156] The present application also provides another computing device cluster. The connection relationship between the computing devices in this computing device cluster can be similar to the connection relationship between computing device clusters described in Figures 5 and 6. However, the memory 106 of one or more computing devices 100A in this computing device cluster can store the same instructions for executing the avatar control method.

[0157] In some possible implementations, the memory of one or more computing devices 100B in the computing device cluster may also store partial instructions for executing the avatar control method. In other words, the combination of one or more computing devices can jointly execute the instructions for executing the avatar control method.

[0158] It should be noted that the memory 106 in different computing devices 100A in the computing device cluster can store different instructions for performing partial functions of task processing. In other words, the instructions stored in the memory 106 in different computing devices 100A can implement the functions of one or more devices in the client, server, and model.

[0159] Embodiments of the present application also provide a computer program product containing instructions. This computer program product can be software or a program product containing instructions that can be run on a computing device or stored on any available medium. When this computer program product is run on at least one computing device, it causes the at least one computing device to execute the avatar control method.

[0160] Embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device, or a data storage device such as a data center that contains one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive). The computer-readable storage medium includes instructions that instruct a computing device to execute the virtual image control method, or instruct a computing device to execute the virtual image control method.

[0161] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the protection scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A virtual image control method, characterized in that: The method is applied to a virtual image control device, the virtual image control device is located in at least one data center, and the method includes: Acquire live broadcast interaction data of a live broadcast room, wherein the live broadcast interaction data includes live broadcast barrage data and the host voice data, and the live broadcast room includes the virtual image and the host; Determine a target question to be answered based on the live broadcast barrage data, wherein the live broadcast barrage data is sent by a viewer of the live broadcast room; Determine target interactive content according to the target question to be answered, the target interactive content including the answer content of the target question to be answered; Determine target interaction data according to the target interaction content, the target interaction data is used to display the target interaction content to the audience of the live broadcast room through the virtual image, and the target interaction data includes target audio data and target video data; The target interaction data is added to the live broadcast data of the live broadcast room.

2. The method according to claim 1, characterized in that Determining the target question to be answered according to the live broadcast barrage data includes: Determine at least one question to be answered according to the live broadcast barrage data; The target question to be answered is determined from the at least one question to be answered according to the host voice data, and the host voice data does not include audio data corresponding to the answer content of the target question to be answered.

3. The method according to claim 1 or 2, characterized in that: Before adding the target interactive data to the live broadcast data of the target live broadcast, the method further includes: It is determined that a pause occurs in the host voice in the host voice data.

4. The method according to any one of claims 1 to 3, characterized in that: The virtual image control device includes a virtual image knowledge base, and the step of determining a target question to be answered based on the live broadcast barrage data includes: Clustering the live broadcast barrage data to determine a barrage intention information set; Determining the target question to be answered according to the barrage intention information set; Determining the target interactive content according to the target question to be answered includes: According to the target question to be answered, determining the answer content of the target question to be answered from the virtual image knowledge base; The target interactive content is determined according to the reply content.

5. The method according to claim 4, characterized in that The method further comprises: The virtual image knowledge base is updated according to the target question to be answered and the audio data used to answer the target question to be answered.

6. The method according to any one of claims 1 to 5, characterized in that: Determining target interaction data according to the target interaction content includes: Generate the corresponding target audio data according to the target interactive content; The target video data is generated according to the virtual image and the target audio data, and the target video data is the video data when the virtual image emits the audio corresponding to the target interactive content.

7. The method according to claim 6, characterized in that Generating the corresponding target audio data according to the target interactive content includes: Analyze the host's voice data to determine the host's emotion category; The target audio data matching the host's emotion category is generated according to the target interactive content.

8. The method according to any one of claims 1 to 7, characterized in that: The method further comprises: Determine anchor instruction information according to the anchor voice data; Determining instruction interaction data according to the anchor instruction information, wherein the instruction interaction data is used to show the audience of the live broadcast room through the virtual image that the virtual image completes the action indicated by the anchor instruction information; The instruction interaction data is added to the live broadcast data of the live broadcast room.

9. The method according to any one of claims 1 to 8, characterized in that: The method further comprises: Determine target subtitle data according to the host voice data; The target subtitle data is added to the live broadcast data of the live broadcast room.

10. The method according to any one of claims 1 to 9, characterized in that: The method further comprises: Analyze the anchor's voice data to determine the live broadcast room operation instructions; Adjust the live broadcast data of the live broadcast room according to the live broadcast room operation instruction.

11. A virtual image control device, characterized in that: The virtual image control device is located in at least one data center, and the device includes: An acquisition module, used to acquire live broadcast interaction data of a live broadcast room, wherein the live broadcast interaction data includes live broadcast barrage data and the host voice data, and the live broadcast room includes the virtual image and the host; A question determination module, used for determining a target question to be answered based on the live broadcast barrage data, wherein the live broadcast barrage data is sent by the audience of the live broadcast room; An interactive content determination module, configured to determine target interactive content according to the target question to be answered and the live interactive data, wherein the target interactive content includes an answer to the target question to be answered; An interactive data determination module, used to determine target interactive data according to the target interactive content, wherein the target interactive data is used to display the target interactive content to the audience of the live broadcast room through the virtual image, and the target interactive data includes target audio data and target video data; The live broadcast data adjustment module is used to add the target interactive data to the live broadcast data of the live broadcast room.

12. The device according to claim 11, characterized in that The question determination module is specifically used to determine at least one question to be answered based on the live broadcast barrage data; and to determine the target question to be answered from the at least one question to be answered based on the host voice data, wherein the host voice data does not include audio data corresponding to the answer content of the target question to be answered.

13. The device according to claim 11 or 12, characterized in that The problem determination module is also used to determine whether a pause occurs in the host's voice data.

14. The device according to any one of claims 11 to 13, characterized in that The virtual image control device includes a virtual image knowledge base, The question determination module is used to cluster the live broadcast barrage data to determine a barrage intention information set; and determine the target question to be answered based on the barrage intention information set; The interactive content determination module is used to determine the answer content of the target question to be answered from the virtual image knowledge base according to the target question to be answered; and determine the target interactive content according to the answer content.

15. A computing device cluster, characterized in that: The computing device cluster includes at least one computing device, each computing device including a processor and a memory: The memory is used to store instructions; The processor is configured to cause the computing device cluster to execute the method according to any one of claims 1 to 10 according to the instructions.

16. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, which, when executed on a computing device, enable the computing device to perform the method according to any one of claims 1 to 10.

17. A computer program product comprising instructions which, when executed on a computing device, cause the computing device to perform the method according to any one of claims 1 to 10.

Images